Skip to content

End-to-end Guidelines

This file explains how we plan to do end-to-end testing using Playwright. Our first goal is to test that each page in the app loads and works as expected.

This guide should be updated as the time passes with new use-cases we encounter.

Why End-to-end Tests

End-to-end tests are one part of our testing setup. They are used to check that everything works together.

  • We already test our design-kit components with unit tests (although not all of them)
  • End-to-end tests will focus on checking that the full page is shown correctly to the user
  • End-to-end will assert that when a button is pressed an API call is performed

What We Will Test

  • We will write tests for full pages
  • These tests will check that the page loads, important parts are visible, and basic content appears correctly

What We Will Not Test (for now)

  • We will be writing tests for feature flows as a first step
  • We will see how much testing pages are viable and maybe include them

Where Tests Will Run

  • We'll use the anonymous dump to set up the environment. This keeps data the same in all places and helps PMs find problems faster
  • Test data will also come from the anonymous dump, not from fixtures or data created on-the-fly. We can add Playwright-specific data later if needed

Login in Tests

  • Endpoint TBD will be used to login a user into the app. This endpoint will only be available on environments where end-to-end is enabled

Running Tests in CI

  • End-to-end tests can be run by adding the label playwright to your pull requests

Folder Structure

Since we will start with testing flows we will have a folder named flow:

Test flow file: laravel/resources/e2e/flows/ssf.spec.ts

Then, when we introduce page tests the file should follow the same folder path as the Vue page it tests

Page component: laravel/resources/inertia/views/pages/dealers/dashboard/Page.vue Test file: laravel/resources/e2e/pages/dealers/dashboard/Page.spec.ts

Targeting Elements

  • We will use data-testid attributes to find elements in the test
  • Do not use CSS classes or text content to find elements

Naming Rules

  • Test files must follow the folder structure of the pages

Who Writes Tests

  • Developers must write tests for new pages
  • Product managers can also help write or change tests
  • Include tests when product managers request them

Test example

import { setInputSelectValue } from "./utils/forms/input-select";
import { expect, test } from "@playwright/test";
import * as path from "node:path";
import { fileURLToPath } from "node:url";

test("it can submit advert through ssf (italian)", async ({ page }) => {
  await page.goto("/it");

  await page.getByTestId("nav-sell-your-classic").click();
  await page.waitForURL("/it/vendere-il-mio-veicolo");
});

Playwright Installation Notes

When running the tests on your local machine, you may be prompted to install missing browsers, such as chromium.

By default, these will be installed globally on your system. However, it can also be installed in the local /carandclassic/laravel/node_modules directory by triggering a Hermetic Install.

To do this prepend the following env var to the cli command: PLAYWRIGHT_BROWSERS_PATH=0

After installation has finished you can find the browser binaries installed in /carandclassic/laravel/node_modules/playwright-core/.local-browsers

Configuration

No additional configuration is required to run the tests on local machine. Remember to always run car build:all before running Playwright so that it can access the latest Vue rendered pages.

Note

The command car pw is an alias for car playwright to run Playwright commands. The rest of this document will refer as car pw for brevity.

Projects

Devices & browsers (aka projects) are configured in the playwright.config.ts file. Note that mobile devices have a name starting with Mobile. This ensures that mobile tests are targeted correctly.

GitHub Workflows

For tests to run on PREnv or Staging, make your PR contains the labels 'playwright' and 'staging' or 'prenv'. This will trigger the Playwright tests on the respective environment.

You can run the tests on your local machine to ensure they pass before pushing your changes to the remote repository. This is a good practice to avoid failing tests on the CI/CD pipeline.

Workers

Parallel workers can be configured in playwright.config.ts but is not ignored by Git.

Running tests can be resource intensive, you may decrease the number of parallel workers on CLI by using the option -j or --workers <workers> to set the number of workers for the test. Example:

car pw test -j 2

Running the tests on UI Mode

Running Playwright in UI mode is a great developer's experience. It allows you to:

  • Navigate the test timeline
  • Interact with the test lines in the browser
  • Pick projects
  • Find existing and new locators
  • Debug issues as they arise.
  • and more..

playwright-ui-mode.png

Demo video: https://playwright.dev/docs/test-ui-mode

Since the application runs on your local machine (and not the container), Yarn needs to be installed on your local machine.

Info

The UI mode is not available in the Car & Classic container, so you need to run it on your local machine through Yarn.

Below explains how to start the Playwright server and client to run the tests in UI mode/application.

Start the Playwright server

First, run the server from the container with a port for the application to listen to.

car pw run-server --host 0.0.0.0 --port=3000

Launch the Playwright UI client on your machine

Using Yarn: 

cd laravel
PW_TEST_CONNECT_WS_ENDPOINT=ws://127.0.0.1:3000/ yarn playwright test --ui

Using npm: 

cd laravel
PW_TEST_CONNECT_WS_ENDPOINT=ws://127.0.0.1:3000/ npx playwright test --ui

Running the tests on CLI

You may run the tests locally without additional setup.

Warning

Always run car build:all before running Playwright tests to ensure the latest Vue rendered pages are available. Failing to do so may result in tests failing due to missing elements or outdated content.

Info

Remember that GitHub Environments will run the tests over CLI, so it is advised to have a final run on CLI before pushing your changes to the remote repository.

Running all tests

To run all tests:

car pw test

Running test for a specific file

To test a specific file:

car pw test filename

The filename can be a wildcard, for example auction.spec.ts will run both example1.auction.spec.ts and example2.auction.spec.ts. Wildcard * can also be used.

Running test for a specific device

To run tests for a specific device, you can use the --project flag with the device name as defined in playwright.config.ts.

car pw test --project "Mobile Safari"

Wildcard * can be used in project names, for example Mobile* will run projects Mobile Safari and Mobile Chrome.

Running test for a specific test or tag

You can run a single test by using the description of the test or a tag. For example, to run tests with the tag @health:

car pw test --grep "@health"

Will run all tests containing the tag @health.