Why a test plan
At this point you should have all the prerequisites. The test plan is a good way to present them. It will act as a contract between you and whoever approves the performance. In it you will define all the steps taken to validate the performance. This way you also protect from unrealistic expectations. For instance an hidden objective that has not come up yet.
Another use is in case someone else runs the next test campaign. The test plan will provide all the required information. Both in terms of running similar tests and comparison with the previous version. So it is definitely a must for testing teams.
Note that in some situations it might not be worth taking the time: - On very short test campaigns (a few days or less). Because the perimeter will usually be quite small in these cases, - When performing a lot of automated tests on a regular basis. But you can write a test plan for the first tests and update it from time to time.
As a general rule, try to build at least a short plan and send it by mail. Otherwise you may have difficulties justifying your choices later.
Test plan content
Let’s take a look at the different sections of a test plan.
Context and objectives
A few words about the context of the tests can’t hurt. Especially since the next test campaign might have a very different context. Which means the objectives of this campaign might not be reusable later.
For instance if the campaign is about testing a new hardware component. Well maybe next time you will just be interested in non regression testing. And because of this you might run different tests.
Describe the test scenarios to be used during the campaign. A detailed description is great, but a simple one is enough at this point. The idea is to:
- Be able to reproduce the same scenarios on a new campaign later,
- Make it clear to whoever will write the tests scripts.
Infrastructure under test
It is a good place to show the infrastructure diagram. First because it might change later. And if you are working on a new test campaign based on this one that’s some valuable input.
Then also because you can explain why you chose this environment. It could also help design better tests later.
Load policy and tests
A load policy can be complex to define. In particular when analyzing statistics. Writing down your math helps a lot. And of course it will help others understand what you’ve done. Which is important since you should have feedbacks on this document.
Once you defined the load policy, try to present all the tests you will run. These tests might not be run in the end. Because you will have performance issues to fix earlier. Or simply because they will become irrelevant at a later stage. But once again your are presenting your methodology here. It is important to show what you are trying to achieve.
Especially important when you do the tests for another company or another team. You need to give visibility on the external help you will need. For instance if a database expert should be involved. Mention it even if you were not able to get one. That will act as a reminder for next time.
Obviously you want the planning in there. Because other teams will need the environment your are using. Or because you want to test between two events on the application.
When building it, also give enough time for the remaining prerequisites to be done. Some of them might need a lot of delay.
And speaking of prerequisites, list whatever actions are still required here. I would recommend to associate a name and delivery date to each one of them. It must be clear that you will not be able to run the tests as planned otherwise.
For instance, if you do not get the list of login the day before the tests, you will not be able to validate them and the first tests may fail.
Last piece of advice
The most important is to get feedback on this document. Maybe you want to schedule another meeting to walk people through it. Maybe you need to ask for explicit validation before starting the tests. But in any case, if you do not get any form of approval, you will pay it later. It’s all about getting people on board with the proposed methodology at this point. If they are not, you will hear bad comments later.
Oh and when sending the test plan copy/paste item 7 in the body of the mail. This way nobody can pretend they did not know you are waiting for their help.
Objectives give your tests their direction. You should have at least one. Always think objectives in terms of risks covered. The most important at this stage is to identify any hidden agenda. If there is an hidden objective your test results might be questioned later.
Skills and resources
It is very unlikely that you will ever be able to master all the skills required for a load testing campaign. In particular if you consider all the possible infrastructures, frameworks and various technologies. Which is why you must reserve timeslots with experts to assist during your tests.
The test environment will have a large impact on what kind of tests you can launch. Depending on your campaign objectives you will prefer an environment over another. It is a question of balance between realism and flexibility of each environment.
Select the most relevant use cases. Remember that you want to test only the performance-heavy features. A small amount of scenarios also ensure a quick and efficient test campaign.
The test plan will serve as documentation for later use. It is a contract stating what you will and will not test. You should also use it to state the various prerequisites to the tests. If you choose not to make one be careful of the consequences.
This wrap up concludes our second chapter, next time we will discuss the design of test scripts.