Maximizing Testing Efficiency: Parameterizing Playwright Scripts

Gérald Pereira Real Browser Nov 3, 2023

OctoPerf is JMeter on steroids!

Parameterization in testing is a powerful technique that allows you to run the same test script with different inputs, configurations, or scenarios. In the world of browser automation and testing, Playwright provides various methods to parameterize your scripts, making it easier to validate different use cases and ensure the robustness of your web applications.

In this blog post, we’ll explore several approaches to parameterizing Playwright scripts.

Why Parameterize Playwright Scripts?

Parameterizing your Playwright scripts offers several benefits, including:

Increased Test Coverage: By running the same script with various parameters, you can verify different test scenarios, ensuring comprehensive coverage of your application.
Reduced Code Duplication: Parameterization helps you avoid writing redundant code for similar test cases, making your test suite more maintainable.
Efficient Testing: Parameterization allows you to test multiple data sets, configurations, and inputs without having to write separate test scripts for each case.
Improved Maintainability: Changes in test data or configurations are easier to manage when the script is parameterized.

Prerequisites

Before diving into Playwright tests parameterizing, ensure you have the following prerequisites in place:

Playwright: You should have Playwright installed. Playwright Trace Viewer is included as part of the Playwright package, so you don’t need to install it separately. Please refer to our previous blog post on Getting Started with Playwright to install the Playwright testing environment.
A Playwright Automation Script: You’ll need a Playwright script that you want to parameterize. If you don’t have one, you can create the file tests/petstore.spec.ts and use the following script for its content.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


import {test, expect} from '@playwright/test';

test('test', async ({page}) => {
    await page.goto('https://petstore.octoperf.com/');
    await page.getByRole('link', {name: 'Enter the Store'}).click();
    await page.getByRole('link', {name: 'Sign In'}).click();
    await page.locator('input[name="username"]').click();
    await page.locator('input[name="username"]').fill('user1');
    await page.locator('input[name="password"]').click();
    await page.locator('input[name="password"]').fill('pass');
    await page.getByRole('button', {name: 'Login'}).click();
    await page.locator('#SidebarContent').getByRole('link').first().click();
    await page.getByRole('link', {name: 'FI-SW-01'}).click();
    await page.getByRole('link', {name: 'EST-1'}).click();
    await page.getByRole('link', {name: 'Add to Cart'}).click();
});

In this script we simulate a user that browses a fictitious e-commerce web application JPetstore.

We will start by parameterizing login and password used to connect to the website. Then we will se how to loop over random items and add them to the buyer’s cart.

Methods to Parameterize Playwright Scripts

Parameterizing Playwright tests is a valuable practice that allows you to run the same test scenario with different input data or configurations. This helps ensure that your application behaves consistently across various test cases.

Now, let’s explore different methods to parameterize your Playwright scripts:

Using and array of values to execute a test with varying inputs.
Using environment variables to execute a test using external inputs (set in a .env file).
Using a CSV file to read variable inputs from a Comma-separated values file.
Using OctoPerf load-testing platform to run Playwright scripts in the Cloud with sharded inputs.

Using Array of Values

The first approach to parameterizing Playwright specs is straightforward:

Declare an array of credentials,
Loop over each login/password pair,
Execute the test for each value.

Create a file named petstore-array.spec.ts with the following content:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24


import {test} from '@playwright/test';

const credentials = [
    {login: 'user1', password: 'pass'},
    {login: 'user2', password: 'pass'},
    {login: 'j2ee', password: 'j2ee'},
]

for (const loginPassword of credentials) {
    test(`test with ${loginPassword.login}`, async ({page}) => {
        await page.goto('https://petstore.octoperf.com/');
        await page.getByRole('link', {name: 'Enter the Store'}).click();
        await page.getByRole('link', {name: 'Sign In'}).click();
        await page.locator('input[name="username"]').click();
        await page.locator('input[name="username"]').fill(loginPassword.login);
        await page.locator('input[name="password"]').click();
        await page.locator('input[name="password"]').fill(loginPassword.password);
        await page.getByRole('button', {name: 'Login'}).click();
        await page.locator('#SidebarContent').getByRole('link').first().click();
        await page.getByRole('link', {name: 'FI-SW-01'}).click();
        await page.getByRole('link', {name: 'EST-1'}).click();
        await page.getByRole('link', {name: 'Add to Cart'}).click();
    });
}

Running Playwright for this script will show you that 3 tests are run: one for each value of the credentials array:

1
2
3
4
5
6
7
8


ubuntu@pop-os:~/Dev/workspaces/playwright-petstore$ npx playwright test tests/petstore-array.spec.ts

Running 3 tests using 3 workers
  3 passed (1.9s)

To open last HTML report run:

  npx playwright show-report

This is confirmed when opening the generated HTML report, you can see the 3 executions:

Playwright Report Array

While this solution allows to execute the same test with several values, these are internal and cannot be easily updated by an external tool like an automation framework. Using environment variables is the go-to solution for such use case.

Using Environment Variables

Let’s use environment variables to parameterize your script. You can pass variables at runtime, allowing you to change configurations without modifying the script.

Create a new test file named petstore-env.spec.ts with the updated following content:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


import {test} from '@playwright/test';

test(`test with ${process.env.LOGIN}`, async ({page}) => {
    console.log(process.env.LOGIN, process.env.PASSWORD);
    await page.goto('https://petstore.octoperf.com/');
    await page.getByRole('link', {name: 'Enter the Store'}).click();
    await page.getByRole('link', {name: 'Sign In'}).click();
    await page.locator('input[name="username"]').click();
    await page.locator('input[name="username"]').fill(process.env.LOGIN);
    await page.locator('input[name="password"]').click();
    await page.locator('input[name="password"]').fill(process.env.PASSWORD);
    await page.getByRole('button', {name: 'Login'}).click();
    await page.locator('#SidebarContent').getByRole('link').first().click();
    await page.getByRole('link', {name: 'FI-SW-01'}).click();
    await page.getByRole('link', {name: 'EST-1'}).click();
    await page.getByRole('link', {name: 'Add to Cart'}).click();
});

In NodeJs, environment variables are read using the syntax process.env.VARIABLE_NAME. Here we read the user login with process.env.LOGIN and the userr password with process.env.PASSWORD.

The easiest way to pass environment variables to Playwright is to set them directly in the command line: LOGIN=j2ee PASSWORD=j2ee npx playwright test tests/petstore-env.spec.ts

Another solution is to create a file name .env that declares the variable values:

1
2


LOGIN=j2ee
PASSWORD=j2ee

The dotenv library can then be used to read them before the Playwright execution. Simply uncomment the following line in the configuration file playwright.config.ts:

1
2
3
4
5


/**
 * Read environment variables from file.
 * https://github.com/motdotla/dotenv
 */
require('dotenv').config();

And install the dotenv package with the command npm install dotenv.

When running your Playwright script, the values are read from the file, no need to set them in the command line. The line console.log(process.env.LOGIN, process.env.PASSWORD); in the script displays the values in the console: j2ee j2ee :

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


ubuntu@pop-os:~/Dev/workspaces/playwright-petstore$ npx playwright test tests/petstore-env.spec.ts

Running 1 test using 1 worker
[chromium] › petstore-env.spec.ts:3:5 › test with j2ee
j2ee j2ee
  1 passed (2.0s)

To open last HTML report run:

  npx playwright show-report

Using CSV Variables

To use a CSV variable in a Playwright script, you can follow these steps:

Read the CSV File: You need to read the CSV file and parse its content in your Playwright script. You can use libraries like csv-parser for Node.js to read and parse CSV files. First, install the libraries:

1

npm install path fs csv-parse

Load and Parse the CSV Data: In your Playwright script, use the csv-parser library to read and parse the data from the CSV file. Here’s an example of how to do this:

1
2
3
4
5
6
7
8
9


import {test} from '@playwright/test';
import {parse} from 'csv-parse/sync';
import * as fs from "fs";
import * as path from "path";

const credentials = parse(fs.readFileSync(path.join(__dirname, 'credentials.csv')), {
    columns: true,
    skip_empty_lines: true
});

Use CSV Data in Playwright Automation: You can now use the parsed CSV data in your Playwright automation scripts. For example, you can iterate through the credentials and use them in your Playwright test:

1
2
3
4
5
6
7
8


for (const loginPassword of credentials) {
    test(`test with ${loginPassword.login}`, async ({page}) => {
        [...]
        await page.locator('input[name="username"]').fill(loginPassword.login);
        await page.locator('input[name="password"]').fill(loginPassword.password);
        [...]
    });
}

In this example, we assume that your CSV file credentials.csv has a column named ‘login’, another column named ' password', and that each row contains a couple of credentials used to connect to the Petstore Web-application.

1
2
3
4


"login","password"
"user1","pass"
"user2","pass"
"j2ee","j2ee"

By following these steps, you can read and use data from a CSV file in your Playwright automation scripts. This approach is helpful when you need to automate repetitive tasks on a large number of web pages or when you want to parameterize your scripts with data from an external source.

1
2
3
4
5
6
7
8


ubuntu@pop-os:~/Dev/workspaces/playwright-petstore$ npx playwright test tests/petstore-csv.spec.ts

Running 3 tests using 3 workers
  3 passed (2.1s)

To open last HTML report run:

  npx playwright show-report

You can check the generated HTML report for executions: as defined in the CSV file, the test is run for users user1, user2 and j2ee.

Playwright Report CSV

Here is the complete sample file petstore-csv.spec.ts content:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26


import {test} from '@playwright/test';
import {parse} from 'csv-parse/sync';
import * as fs from "fs";
import * as path from "path";

const credentials = parse(fs.readFileSync(path.join(__dirname, 'credentials.csv')), {
    columns: true,
    skip_empty_lines: true
});

for (const loginPassword of credentials) {
    test(`test with ${loginPassword.login}`, async ({page}) => {
        await page.goto('https://petstore.octoperf.com/');
        await page.getByRole('link', {name: 'Enter the Store'}).click();
        await page.getByRole('link', {name: 'Sign In'}).click();
        await page.locator('input[name="username"]').click();
        await page.locator('input[name="username"]').fill(loginPassword.login);
        await page.locator('input[name="password"]').click();
        await page.locator('input[name="password"]').fill(loginPassword.password);
        await page.getByRole('button', {name: 'Login'}).click();
        await page.locator('#SidebarContent').getByRole('link').first().click();
        await page.getByRole('link', {name: 'FI-SW-01'}).click();
        await page.getByRole('link', {name: 'EST-1'}).click();
        await page.getByRole('link', {name: 'Add to Cart'}).click();
    });
}

Using OctoPerf Load Testing Platform

OctoPerf is a performance testing and load testing platform designed for evaluating the performance, scalability, and reliability of web applications and systems. It provides a comprehensive set of tools and features that enable developers, testers, and organizations to simulate real-world user activity, measure system performance, and identify performance bottlenecks.

The SaaS version is available at https://api.octoperf.com/ui/.

To use a CSV variable in a Playwright script executed in the OctoPerf load testing Cloud, you can follow these steps:

Create a CSV Variable: Assuming you have created an account and a Project, you need to create a blank CSV Variable.

OctoPerf Create CSV Variable

Upload a credentials CSV file: Download the sample credentials.csv file and upload it in your CSV Variable.

OctoPerf Upload CSV File

Check the CSV Variable columns: The login and password columns should appear at the bottom of your CSV Variable editor.

OctoPerf CSV Columns

Create a Playwright Virtual User using the following specs file.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


import {test} from '@playwright/test';

test(`test with ${process.env.login}`, async ({page}) => {
    console.log(process.env.login, process.env.password);
    await page.goto('https://petstore.octoperf.com/');
    await page.getByRole('link', {name: 'Enter the Store'}).click();
    await page.getByRole('link', {name: 'Sign In'}).click();
    await page.locator('input[name="username"]').click();
    await page.locator('input[name="username"]').fill(process.env.login);
    await page.locator('input[name="password"]').click();
    await page.locator('input[name="password"]').fill(process.env.password);
    await page.getByRole('button', {name: 'Login'}).click();
    await page.locator('#SidebarContent').getByRole('link').first().click();
    await page.getByRole('link', {name: 'FI-SW-01'}).click();
    await page.getByRole('link', {name: 'EST-1'}).click();
    await page.getByRole('link', {name: 'Add to Cart'}).click();
});

Validate you Virtual User: Running 3 iterations of the created virtual user in Debug Mode should result in 3 validation results, each one using a different line of the uploaded CSV file.

OctoPerf Debug stdout

One of the advantages of running your Playwright scripts through OctoPerf is that CSV files are automatically split across injectors when running your tests using different user profiles.

Parameterizing Playwright Projects

Another approach to test variabiliszation in Playwright to parameterize at the Project level.

Indeed, Playwright supports running multiple test projects at the same time. They are defined in the configuration playwright.config.ts file. Concretely we are going to define a type for the user credentials with default values, then override it in the project definition.

Let’s start by creating a file petstore-test.ts in the tests folder of your Playwright Node.js project:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


import {test as base} from '@playwright/test';

export type PetstoreTestOptions = {
    login: string;
    password: string;
};

export const petstoreTest = base.extend<PetstoreTestOptions>({
    // Define an option and provide a default value.
    // We can later override it in the config.
    login: ['user1', {option: true}],
    password: ['pass', {option: true}],
});

The type PetstoreTestOptions defines a credentials type with a login and a password. The constant petstoreTest extends the default Playwright test by adding options for the login and password.

Create the petstore.spec.ts spec file with the following content:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


import {petstoreTest} from "./petstore-test";

petstoreTest('test', async ({page, login, password}) => {
    console.log(login, password);
    await page.goto('https://petstore.octoperf.com/');
    await page.getByRole('link', {name: 'Enter the Store'}).click();
    await page.getByRole('link', {name: 'Sign In'}).click();
    await page.locator('input[name="username"]').click();
    await page.locator('input[name="username"]').fill(login);
    await page.locator('input[name="password"]').click();
    await page.locator('input[name="password"]').fill(password);
    await page.getByRole('button', {name: 'Login'}).click();
    await page.locator('#SidebarContent').getByRole('link').first().click();
    await page.getByRole('link', {name: 'FI-SW-01'}).click();
    await page.getByRole('link', {name: 'EST-1'}).click();
    await page.getByRole('link', {name: 'Add to Cart'}).click();
});

The login and password current values are given to the script as test parameters async ({page, login, password}).

Running the test without specifying values in the project will use the defaults (specified in petstore-test.ts):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


ubuntu@pop-os:~/Dev/workspaces/playwright-petstore$ npx playwright test tests/petstore.spec.ts

Running 1 test using 1 worker
[chromium] › petstore.spec.ts:3:13 › test
user1 pass
  1 passed (2.0s)

To open last HTML report run:

  npx playwright show-report

Here the console logs user1 pass.

Let’s update the Playwright configuration file playwright.config.ts to set a value for the login:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


import {defineConfig, devices} from '@playwright/test';
import {PetstoreTestOptions} from "./tests/petstore-test";

export default defineConfig<PetstoreTestOptions>({
    /* Configure projects for major browsers */
    projects: [
        {
            name: 'chromium',
            use: {
                login: 'user2',
                ...devices['Desktop Chrome']
            },
        },
    ],
});

Three things must be updated here:

PetstoreTestOptions must be imported import {PetstoreTestOptions} from "./tests/petstore-test";.
It must be used as a type parameter for the configuration export default defineConfig<PetstoreTestOptions>.
The login must be defined in the project section projects: [{name: 'chromium', use: { login: 'user2' }}]

Running the test will now show user2 pass in the console logs:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


ubuntu@pop-os:~/Dev/workspaces/playwright-petstore$ npx playwright test tests/petstore.spec.ts

Running 1 test using 1 worker
[chromium] › petstore.spec.ts:3:13 › test
user2 pass
  1 passed (2.0s)

To open last HTML report run:

  npx playwright show-report

Using Loops and Random Selectors

Until now, we only parameterized the credentials used by our virtual user to connect to the PetStore website. Let’s use loops and random values to make it add random items to the e-commerce cart.

The following file petstore-loop.spec.ts defines a method addRandomItemToCart that does exactly that. The main test ‘test’ goes to the PetStore home and enters it. A for loop then adds 3 random item to the cart.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30


import {test, expect, Page} from '@playwright/test';

test('test', async ({page}) => {
    await page.goto('https://petstore.octoperf.com/');
    await page.getByRole('link', {name: 'Enter the Store'}).click();
    for (let i = 0; i < 3; i++) {
        await addRandomItemToCart(page);
    }
});


const addRandomItemToCart = async (page: Page) => {
    await page.locator('#LogoContent').getByRole('link').click();
    const categories = page.locator('#SidebarContent').getByRole('link');
    const categoriesCount = await categories.count();
    expect(categoriesCount).toBe(5);
    await categories.nth(Math.floor(Math.random() * categoriesCount)).click();

    const products = page.getByRole('table').getByRole('link');
    const productsCount = await products.count();
    expect(productsCount).toBeGreaterThan(0);
    await products.nth(Math.floor(Math.random() * productsCount)).click();

    const items = page.getByRole('table').getByRole('link', {name: 'Add to Cart'});
    const itemsCount = await items.count();
    expect(itemsCount).toBeGreaterThan(0);
    await items.nth(Math.floor(Math.random() * itemsCount)).click();

    await expect(page.getByRole('heading', {name: 'Shopping Cart'})).toBeVisible();
}

Links are extracted from each page using locators: const locator = page.locator('container').getByRole('link');.
The number of link is then computed using the count() method: const count = await locator.count();.
Finally, a random link is clicked: await locator.nth(Math.floor(Math.random() * count)).click();.

Let’s run this script with traces ON:

1
2
3
4
5
6
7
8


ubuntu@pop-os:~/Dev/workspaces/playwright-petstore$ npx playwright test tests/petstore-loop.spec.ts --trace on

Running 1 test using 1 worker
  1 passed (6.1s)

To open last HTML report run:

  npx playwright show-report

And open the generated trace with the command line npx playwright show-trace test-results/petstore-loop-test-chromium/trace.zip:

Petstore Trace

We can see in the Actions on the left and in the timeline at the top that 3 items were added to the cart:

Petstore Cart

Conclusion

Parameterizing Playwright scripts is a fundamental practice that enhances the efficiency, coverage, and maintainability of your browser automation tests. By choosing the method that best suits your testing needs, you can ensure that your web applications are thoroughly tested across various scenarios, configurations, and inputs. Whether you opt for external data sources, environment variables, or command-line arguments, parameterization empowers you to create robust, scalable, and adaptable Playwright test scripts.