Overview of statistical techniques for results gennerated using JMeter
Introduction What does statistical analysis have to do with performance testing you may ask, more than you would think is the answer.
Due to the large volumes of result data that is generated and analysed you are performing statistical analysis of your data when presenting your results.
90th Line, 95th Line, 99th Line, Average, Median. Are all examples of values in the JMeter Aggregate Report.
Average, Std.Dev. Are examples of values in the JMeter Summary Report as well as the Graph Results.
Using Test Fragments, Module Controllers and Include Controllers in JMeter
Introduction The key to all good automation is modularisation which is effectively isolating small chunks of your application under test into separate tests and then re-using these smaller chunks in your larger, more complicated tests.
Example of good candidates for modularisation are logon and logoff as well as any other high-volume journeys through your application such as search functionality.
The rationale for this approach is that if these high volume common functional activities change you only need to make the update in a single test rather than in multiple tests.
Introduction When we think of performance testing we normally think of thousands of requests with thousands of users generating huge volumes on our application under test or increasing the load until the application under test fails or runs out of resources.
This is not a good approach when determining load for your performance tests for several reasons and can in some cases render your performance testing meaningless.
This post will outline some of the pitfalls that are commonly made when it comes to generating load and will look at ways to suggest improvements to your approach to load profiles.
Introduction JMeter has many plug-ins and downloadable JAR Files you can use to support your testing, it is also possible to write your own custom Jar files and use these as well.
Why would you want to do that, you may say, well whilst it is unlikely that you would need to create anything too complicated.
You may feel the need to create utilities to support your own in-house performance testing or just something to make your testing process easier.
Introduction The principles behind performance testing API’s does not differ from the principles behind the performance testing of any application.
Many API’s however are Asynchronous and a valid response from the API does not necessarily mean the transaction is complete which can cause a problem when measuring the performance of API’s.
There are however ways around this and we will explore these in this post.
Volumes Before we get into the details of testing it is important to understand that in a microservices architecture API’s may be called by both external consumers and internal consumers.
Why documentation is overrated in Agile Performance Testing
Introduction Once upon a time documentation was one of the most important aspects of Quality Assurance and this was not limited to the functional test efforts but the non-functional testing as well.
We spent days, weeks, months even creating Performance Test Strategies, Approaches, Plans, Test Case, Completion Reports etc.
Most of these documents were required before any automation could be written and before a sensible performance testing framework could be considered.
Exploring the hidden benefits of repurposing performance testing scripts.
Introduction If you are a performance testing specialist or a QA Manager or Programme Manager or anyone involved in the production of quality software then you understand why performance testing is required and its benefits in ensuring your products meet your Quality Criteria for release into production.
The costs of delivering performance testing are easily worth the investment as software that performs not only ensures you and your company have a reputation for delivering well performing software but business users will, in my opinion, overlook and embrace small functional workarounds if the software performs well.
Simple way for creating complex business scenarios in JMeter using Groovy scripts and the JMeter Switch Controller.
Introduction Creating complex performance testing scenarios in JMeter can be a complicated but necessary problem you will encounter as you build tests to mirror real user behaviour in your testing.
There are many add-ins that can support you in the creation of these scenarios. Which is good if they do what you want them to do. But if you want the flexibility to build tests without the limitations of 3rd party add-ins then there are several techniques you can use which come with the standard JMeter install.
Introduction The Beanshell Interpreter in JMeter can act as a server which as stated in the JMeter documentation is accessible by telnet or http.
So what, I hear you say, well this can be useful and we will explore some of these uses in this Blog Post.
Before we move on the definition of a Beanshell can be found on the official Web Site.
But at a high level:
You can use BeanShell interactively for Java experimentation and debugging as well as to extend your applications in new ways.
Making sure your performance testing execution strategy is suitable.
Introduction When we consider performance testing most of the focus of the approach is placed on the script creation activity and making sure that requirements are covered, these are very important parts of your performance testing approach but so is the Test Execution.
The frequency at which you run your tests is important and can save you time and effort in your script maintenance.
A good performance testing execution strategy gives you the maximum amount of benefit it can by finding performance issues early.