Introduction Regression Testing, as all Quality Assurance professionals know, is ensuring that previously developed and tested software continues to operate after a change.
Performance Regression being a subset of regression testing as a discipline is therefore ensuring that previously developed and tested continues to meet its performance criteria after a change.
There are subtle differences in the way that performance regression testing is approached when compared with functional regression testing and we will look to explore these in this Blog Post.
Overview of performance testing Kafka with JMeter. How to publish messages onto a Kafka topic?
Introduction This post is about Kafka and the process I have been through recently writing a performance test for an application that subscribes to messages from this technology.
The test I ended up with was in the end very straightforward but there were several hurdles that took a while to resolve. I hope that reading this post will hopefully help you avoid them.
The performance testing concepts we will discuss are focused on how to publish messages onto a Kafka topic and will not discuss how to write a test to consume messages from Kafka.
Is JMeter old technology or is it still suitable for modern day challenges?
Introduction “JMeter is old technology”, I hear this a lot.
“Let’s use this tool or that tool instead of JMeter as it’s the latest”, I hear this a lot.
“We need a lightweight tool without the GUI interface to write our tests as that will make us more agile”, I hear this a lot.
It’s all nonsense it really is, there seems to always be a call for using the most modern technology for all forms of testing whether its Performance Testing or Automated Functional Testing and the result of this is that the people writing the tests spend to long learning the new tools and not enough time building something that will ensure the software being developed is the best it can be.
Overview of JMeter plugins and how they can be used
Introduction JMeter in its vanilla form is a very powerful tool for performance testing, the ability to create complicated test scenarios using the out-of-the box samplers, timers and logic controller can sometimes be extremely difficult and require you to write your own complimentary code.
There are however Plugin’s that exist that can help in building some of the more challenging scenarios.
In this post we are going to look at some of these and discuss how they can be used.
Overview of statistical techniques for results gennerated using JMeter
Introduction What does statistical analysis have to do with performance testing you may ask, more than you would think is the answer.
Due to the large volumes of result data that is generated and analysed you are performing statistical analysis of your data when presenting your results.
90th Line, 95th Line, 99th Line, Average, Median. Are all examples of values in the JMeter Aggregate Report.
Average, Std.Dev. Are examples of values in the JMeter Summary Report as well as the Graph Results.
Using Test Fragments, Module Controllers and Include Controllers in JMeter
Introduction The key to all good automation is modularisation which is effectively isolating small chunks of your application under test into separate tests and then re-using these smaller chunks in your larger, more complicated tests.
Example of good candidates for modularisation are logon and logoff as well as any other high-volume journeys through your application such as search functionality.
The rationale for this approach is that if these high volume common functional activities change you only need to make the update in a single test rather than in multiple tests.
Introduction When we think of performance testing we normally think of thousands of requests with thousands of users generating huge volumes on our application under test or increasing the load until the application under test fails or runs out of resources.
This is not a good approach when determining load for your performance tests for several reasons and can in some cases render your performance testing meaningless.
This post will outline some of the pitfalls that are commonly made when it comes to generating load and will look at ways to suggest improvements to your approach to load profiles.
Introduction JMeter has many plug-ins and downloadable JAR Files you can use to support your testing, it is also possible to write your own custom Jar files and use these as well.
Why would you want to do that, you may say, well whilst it is unlikely that you would need to create anything too complicated.
You may feel the need to create utilities to support your own in-house performance testing or just something to make your testing process easier.
Introduction The principles behind performance testing API’s does not differ from the principles behind the performance testing of any application.
Many API’s however are Asynchronous and a valid response from the API does not necessarily mean the transaction is complete which can cause a problem when measuring the performance of API’s.
There are however ways around this and we will explore these in this post.
Volumes Before we get into the details of testing it is important to understand that in a microservices architecture API’s may be called by both external consumers and internal consumers.
Why documentation is overrated in Agile Performance Testing
Introduction Once upon a time documentation was one of the most important aspects of Quality Assurance and this was not limited to the functional test efforts but the non-functional testing as well.
We spent days, weeks, months even creating Performance Test Strategies, Approaches, Plans, Test Case, Completion Reports etc.
Most of these documents were required before any automation could be written and before a sensible performance testing framework could be considered.