This blog post aims to resume all the cool new features and improvements we’ve made to OctoPerf since the beginning of the year.
Improvements and features in this section are related to the online virtual user design exclusive to OctoPerf.
Enable / Disable actions
This is a must have to quickly rework any existing virtual user. Any action can now be disabled to prevent it from being executed when running the virtual user. Actions no longer need to be removed from the virtual user to be taken out of the execution. This feature is especially useful to debug virtual users by running repeated validations.
We know that handling dynamic parameters can sometimes be especially challenging. The best performance testers deserve the best tools to work with. We have vastly improved the ways you can search for values inside various aspects of the virtual user.
It’s now possible to search for a token in very specific locations like query parameter names, post parameter values or even recorded responses. Be sure to find where this nasty dynamic parameter comes from!
Virtual User Validation Zone
Previously, all the virtual user validations were executed in our main datacenter in Ireland by default. Now, it’s possible to select the provider and region where to run the virtual user validation. This is especially useful when your virtual user only runs on an on-premise location.
Please note that launching a virtual user validation in some regions make take some time as we spin up the required machine.
Validation JMeter Logs
Now, when running a virtual user validation to check if your virtual user is running properly, the JMeter logs can be viewed and downloaded at the end of the execution. This is useful to debug JSR223 scripts which may log information. It’s also convenient to see if JMeter ran without any issue during the validation before scaling massively on our cloud platform.
This feature is also available on all our test results. All the JMeter logs are stored on our servers at the end of a test for each JMeter engine being run during a load test.
With advanced scripting, you can easily tackle even the most complex load testing issues at ease.
JMeter JMX Import
We’ve vastly improved our JMeter importing capabilities. Now, user defined variables and test wide constants are also imported. Various fixes have been made to support the most exotic JMeter JMX configurations. Beanshell scripts are automatically converted to JSR223 scripts for enhanced performances.
Charles Proxy Support
At OctoPerf, we believe that you should never be locked with any paid software. We’d rather have you stay by providing an amazingly useful software rather than locking you in with proprietary technologies. We’ve added the possibility to export your virtual user in JMX format, anytime, for free. This way, you can keep a JMeter compliant copy of your virtual user design. Simply click on the Download JMX button when editing your virtual user.
Project File Explorer
With an increasing number of csv files and post file parameters to manage, it became difficult to see which files where actually stored on our servers. The Project file explorer makes it easy to list, add or delete existing files to your test.
Those files are automatically sent to all JMeter load generators when launching a test, making it insanely easy to scale test with file resources.
VU Duplication and copy
Sometimes, you need virtual users with just slightly different behavior and don’t want to record it again. Copy / paste feature is for you! You can now copy and paste virtual user actions.
You can also copy paste entire virtual users (for example to backup an original record, or to test out scenario variations).
As a bonus, when copying and pasting actions, recorded requests and responses are copied too.
To prevent issues before running a load test, we’ve incorporated a feature which checks the sanity of your project. It performs various checks like verifying if a CSV variable has effectively an existing file linked to it.
We have added a few advanced settings to our scenarios to satisfy our advanced users.
We have drastically improved our load generators spin-up time in the last few months. Tests will be launched way faster.
UserLoad: Apply all
The apply all button located in the top right corner on the userload allows to apply the same userload settings to all other userloads within the scenario. This avoids to have to redo the same settings several times in the case you have very similar userloads.
UserLoad: Global Thinktime
Setting each thinktime inside the virtual user manually can be a time consuming task. Sometimes, you just want to override all thinktimes with a given value and be set. This feature has been exactly designed for this purpose.
By clicking on Duration tab inside the virtual user profile, there is a new section called Think time. By enabling the override, all thinktimes within the associated virtual user will be replaced by the thinktime defined here:
Constant Thinktime: replaces all thinktimes with a constant one,
Random thinktime: replaces all thinktimes with a variable thinktime whose value is within a given range,
Throughput thinktime: replaces all thinktimes by trying to stick to a given throughput in hits per unit of time.
This allows to simulate users with fine-tuned thinktime settings and unmatched realism.
UserLoad: DNS Settings
Websites today often involve CDN (Content Delivery Network) today to improve their loading speed. Load testing websites with CDNs can be tedious due to nasty dns issues. When load testing a CDN without proper DNS configuration, most of the virtual users will just load test the same server, which is unrealistic. We’ll cover the whole thing in a dedicated blog post.
It’s now possible to clear DNS cache on each virtual user iteration to ensure new domain IP resolution on each round. You can even specify your own DNS servers if you would like to. With these advanced settings, it becomes easy to realistically load test websites powered by CDNs. Once again, we’ve made our best to make things easy to understand and to configure.
Previously, the maximum number of HTTP redirections was limited to 5. We’ve increased this number to 20 to support a broader number of websites.
Caution, this is a highly advanced setting. The memory allocated per virtual user on each load generator can be tuned here (16Mb max). This setting allows to override our optimized memory settings in the case your scenario may need some tuning due to heavy pages. We recommend to leave this settings as is. We have already spent time tuning JMeter’s memory settings to give you the best possible performances.
JMeter logs can now be downloaded at the end of a test. All JMeter engines logs are kept in a central place which makes it easy to download them all. Those logs are ingested by our Jenkins CI plugin to generate performance test trends.
Behind the scene
Those are the improvements we’ve made you may haven’t noticed.
We’re constantly upgrading many of the open-source apis and apps to ensure you get the latest benefits of those updates. We recently upgraded our Elasticsearch cluster to the latest version which should result in a small performance boost. We also keep track of the latest available Rancher version and upgrade each time a new stable release is available. This is one of the greatest benefits for our customers: we take care of the upgrades.
As for the backend, we keep an eye on what’s new in this field and constantly upgrade the APIs we use. It’s a very time consuming task on the frontend side as the technologies on this side are moving especially fast. Good news is that we take care of that too :)