Debug School

rakesh kumar
rakesh kumar

Posted on

What Can Playwright Test? A Practical Testing Order from Smoke Tests to Security

Build confidence in your application by starting with a small smoke suite, expanding functional coverage, and preserving those checks as regression tests. Once the core journeys are reliable, add accessibility, visual comparisons, performance measurements, and deeper security testing.

This guide uses TypeScript and Playwright Test for examples. The same testing concepts apply to JavaScript and Python projects. Existing Python suites can grow in place without a language migration.

What Playwright can test

Playwright automates browser interactions and assertions across Chromium, Firefox, and WebKit. It can also make HTTP requests directly through its API request context. Together, these capabilities support browser journeys, API checks, and tests that combine both.

Capability What you can verify
Navigation Pages, redirects, menus, links, route parameters, and browser history
Forms Required fields, validation, valid submissions, and error messages
Authentication Login, logout, protected routes, and password-reset journeys
Business workflows Registration, organizer onboarding, trip publishing, bookings, and checkout
Data interfaces Search, filters, sorting, pagination, empty states, and CRUD
Browser interactions Dialogs, tabs, popups, frames, uploads, and downloads
Network behavior Requests, responses, API failures, and controlled mocked responses
Compatibility Browser engines, viewport sizes, mobile emulation, and locale settings
Accessibility Keyboard behavior, accessible names, and automated axe checks
Visual changes Screenshot comparisons against reviewed baselines
Diagnostics Traces, screenshots, videos, reports, and browser console evidence

Device emulation helps test responsive behavior. Include real-device checks when hardware, mobile-browser behavior, or device integrations matter. External email delivery and SMS/OTP delivery also require an appropriate test mailbox or provider sandbox.

The recommended implementation order

This is a practical order for building a new automation program. The categories overlap. A login test can be functional, smoke, and regression at the same time.

Order Test type Purpose Playwright's role
1 Smoke Confirm the application is available and critical journeys work Primary browser/API tool
2 Functional Verify each feature against requirements Primary browser tool
3 Regression Preserve existing behavior after changes Reuse and expand functional tests
4 API and integration Check service responses and connected data flows Direct API testing and UI/API combinations
5 Accessibility Check automated rules and interaction accessibility Playwright plus axe and manual assessment
6 Visual regression Detect unexpected layout and appearance changes Screenshot assertions
7 Performance Measure speed, responsiveness, capacity, and stability Browser signals with complementary tools
8 Security Verify security controls and identify vulnerabilities Targeted control checks with dedicated tools

Use API-backed setup earlier when it makes functional tests reliable. Keep basic authorization expectations inside feature tests from the beginning. This roadmap postpones building dedicated performance and security automation until the core journeys stabilize. Secret handling, dependency hygiene, and secure implementation still belong in everyday development.

1. Smoke tests

Smoke tests answer one question. Is this deployment working well enough for further testing?

Start with five to ten fast checks. For a travel application such as HolidayLandmark, candidates include the home page, trip listings, a trip detail page, login, organizer dashboard, and the primary trip-publishing journey.

Scenario What to assert
Home page Expected heading and primary navigation appear
Listings Results or a valid empty state appear
Detail page Correct title and essential details appear
Login A designated test user reaches the expected destination
Dashboard Authorized user sees the correct dashboard
Main action A test trip can be published in a test environment
Logout A subsequent protected request is rejected or requires login

A page returning HTTP 200 is only one signal. Also assert the content or behavior that matters. A server can return a successful status while displaying an application error.

Use a smaller read-only production smoke suite if your monitoring scope requires it. Keep registration, trip creation, bookings, payments, and deletion in an isolated environment with test accounts and sandbox providers.

import { test, expect } from '@playwright/test';

test('home page opens', { tag: ['@smoke', '@regression'] }, async ({ page }) => {
  await page.goto('/');
  // Replace this accessible name with your application's actual heading.
  await expect(page.getByRole('heading', { name: 'Explore trips' })).toBeVisible();
});
Enter fullscreen mode Exit fullscreen mode

2. Functional tests

Functional testing checks whether a feature behaves according to its requirements. Cover successful inputs and the boundaries around them.

For every major module, review the following coverage.

  • Happy paths with valid input.
  • Required fields, invalid formats, minimum/maximum lengths, and boundary values.
  • Success messages, error messages, loading states, and empty states.
  • Search, filters, sorting, and pagination.
  • Create, read, update, and delete operations where the feature supports them.
  • Guest, user, organizer, and admin permissions.
  • Upload/download behavior and supported file rules.
  • Navigation, redirects, and deep links.
  • Password reset and email/OTP journeys using test delivery infrastructure.
  • Recoverable failures, duplicate submissions, and retry behavior where relevant.

For trip publishing, verify required fields, date rules, permitted prices, image restrictions, successful publishing, and how the published record appears. Derive the expected results from the actual product requirements.

Prefer accessible locators and retrying assertions. Avoid fixed sleeps for normal synchronization.

test('search shows the matching trip', { tag: '@functional' }, async ({ page }) => {
  await page.goto('/trips');
  await page.getByRole('searchbox', { name: 'Search trips' }).fill('City walk');
  await page.getByRole('button', { name: 'Search', exact: true }).click();
  await expect(page.getByRole('link', { name: 'City walk', exact: true })).toBeVisible();
});
Enter fullscreen mode Exit fullscreen mode

This example assumes a seeded trip named “City walk” and matching accessible names. Adapt routes, labels, and data to your application. See the official Playwright best practices.

3. Regression tests

Regression testing checks whether previously working behavior still works after a change. It is a purpose for selecting and rerunning tests, so you can reuse functional, API, accessibility, and visual checks.

Build the regression suite around important journeys and known defects. When a bug is fixed, add a test that would have caught it. For example, if editing a trip accidentally clears its itinerary, create a trip, edit an unrelated field, and assert the itinerary remains intact.

Run a targeted subset during development and a broader suite before release. Independent tests should create or acquire their own data, work in any order, and avoid sharing mutable accounts across parallel workers.

npx playwright test --grep @smoke
npx playwright test --grep @functional
npx playwright test --grep @regression
Enter fullscreen mode Exit fullscreen mode

A test with multiple tags participates in each matching suite. See Playwright tags and annotations.

4. API and integration tests

API tests verify HTTP behavior directly. Integration tests verify that connected components exchange and persist data correctly.

Area Useful coverage
Responses Expected status, headers, and response structure
Validation Missing fields, invalid values, and boundary cases
Access control Unauthenticated and unauthorized requests are denied
Data Created or updated values can be retrieved correctly
Collections Filters, ordering, pagination, and empty responses
UI/API integration API-created test data appears correctly in the UI
Failures UI handles an unavailable or rejected API response
test('trip listing API returns a collection', { tag: '@api' }, async ({ request }) => {
  const response = await request.get('/api/trips');
  expect(response.status()).toBe(200);
  const body = await response.json();
  // This assertion assumes the API returns an object with a data array.
  expect(Array.isArray(body.data)).toBe(true);
});
Enter fullscreen mode Exit fullscreen mode

Expand assertions to match the documented contract. The example above does not validate every field. Mocked responses help isolate UI states, while tests against real test services verify the actual integration. See Playwright API testing.

5. Accessibility tests

Accessibility testing checks whether people with different abilities can use the application.

Automate checks for detectable issues such as missing accessible names, certain contrast failures, invalid ARIA usage, and form-label problems. Add focused interaction tests for keyboard navigation, visible focus, dialog focus behavior, and understandable validation feedback.

npm install --save-dev @axe-core/playwright
Enter fullscreen mode Exit fullscreen mode
import AxeBuilder from '@axe-core/playwright';

test('trip page has no detected axe violations', { tag: '@a11y' }, async ({ page }) => {
  await page.goto('/trips');
  await expect(page.getByRole('heading', { name: 'Trips', exact: true })).toBeVisible();
  const results = await new AxeBuilder({ page }).analyze();
  expect(results.violations).toEqual([]);
});
Enter fullscreen mode Exit fullscreen mode

Run checks in meaningful states, including an open modal and a form with errors. Automated scans cover part of accessibility. Manual keyboard and screen-reader assessment remains necessary, and a clean scan does not establish WCAG compliance. See Playwright accessibility testing.

6. Visual regression tests

Visual regression tests compare the current rendering with an approved screenshot. They can catch clipped text, missing buttons, broken spacing, unexpected colors, and responsive layout changes.

test('trip listing layout', { tag: '@visual' }, async ({ page }) => {
  await page.goto('/trips');
  await expect(page.getByRole('heading', { name: 'Trips', exact: true })).toBeVisible();
  await expect(page).toHaveScreenshot('trip-listing.png', {
    animations: 'disabled',
    fullPage: true,
  });
});
Enter fullscreen mode Exit fullscreen mode

Use fixed test data, viewport, fonts, locale, browser version, and operating environment. Wait for application-specific loading to complete. Stabilize changing dates and content or mask only the regions that genuinely vary. Review each baseline before accepting it, including the first generated baseline.

npx playwright test --grep @visual
npx playwright test --grep @visual --update-snapshots
Enter fullscreen mode Exit fullscreen mode

Updating snapshots approves a new comparison target. Review the image changes before committing them. See Playwright visual comparisons.

7. Performance tests

Performance testing measures how quickly the system responds and how it behaves under traffic. Playwright can capture browser timing and network evidence during real journeys. Use Lighthouse CI for repeatable web audits and k6, JMeter, or Locust for workload testing.

Measurement What it tells you
First Contentful Paint When the first content becomes visible
Largest Contentful Paint When the largest eligible content element renders
Interaction to Next Paint Responsiveness to user interactions
Cumulative Layout Shift Unexpected movement of visible content
Request count and transferred bytes Network cost of loading a journey
API latency percentiles Response-time distribution under defined conditions
Error rate and throughput Reliability and capacity at a chosen workload

The Core Web Vitals are LCP, INP, and CLS. Time to Interactive is a retired Lighthouse metric, so use current metrics when designing a new baseline. Lab measurements describe controlled runs. Field measurements describe real users, devices, and interactions. See Web Vitals and Lighthouse performance scoring.

Distinguish performance scenarios before choosing a workload.

Scenario Purpose
Load Check expected traffic
Stress Find behavior beyond expected capacity
Spike Check sudden increases in traffic
Soak Check sustained traffic over a longer period
Scalability Check behavior as traffic and resources change

Define traffic mix, concurrency or arrival rate, duration, test data, and acceptable error/latency thresholds. Capture server CPU, memory, database, and queue metrics alongside results. API timing from a single Playwright run includes client/network effects and cannot establish backend capacity.

Run controlled performance checks weekly on staging, before major releases, and after substantial frontend, database, or infrastructure changes. Choose budgets from product targets and a repeatable baseline. See Grafana k6 documentation.

8. Security tests

Security testing verifies that controls enforce the intended rules and investigates vulnerabilities. Playwright can automate login, session, and access-control scenarios, including direct API assertions.

Control Example expectation
Authentication Invalid credentials cannot establish a session
Authorization A normal user cannot read or change admin resources
Object access User A cannot access user B's private records
Logout The old authenticated session cannot access protected resources
Password reset Expired or previously consumed reset links are rejected
Input handling Invalid input is rejected with controlled errors
Error disclosure Responses avoid stack traces and sensitive implementation details

Assert backend enforcement. Hiding an admin button alone cannot prove authorization. Check both protected page behavior and the corresponding API behavior. Authentication state files can grant access to accounts and should be excluded from Git. See Playwright authentication.

Complement these checks with the following activities.

Activity Coverage Example tools
SAST Potential vulnerabilities in source code CodeQL, Semgrep
SCA Known vulnerable dependencies npm audit, Dependabot, Snyk
Secret scanning Accidentally committed credentials Gitleaks or repository secret scanning
DAST Security behavior of a running application ZAP or another DAST scanner
Manual review Business logic, architecture, and complex abuse cases Skilled security review

Browser storage should contain only values allowed by the application's authentication design. Public client identifiers differ from confidential credentials. Review token storage, exposure, lifetime, and application requirements together.

Use isolated test accounts and an agreed test scope. Run destructive scans, brute-force attempts, injection testing, and penetration testing against production only with explicit written authorization and a defined operating plan. Automated findings require review, and clean scans cannot establish complete security. See the OWASP Web Security Testing Guide and ZAP documentation.

Set up a maintainable framework

Install Playwright and choose TypeScript when initializing a new Node.js test project.

npm init playwright@latest
npm install --save-dev dotenv
npx playwright install
Enter fullscreen mode Exit fullscreen mode

Use folders such as tests/smoke/, tests/functional/, tests/api/, tests/accessibility/, and tests/visual/. Keep reusable fixtures in fixtures/ and authentication setup in tests/auth.setup.ts. Tests can retain regression tags in their existing feature folders.

Store environment-specific URLs and test credentials in ignored environment files or CI secrets. Commit an .env.example with placeholders only. For example, use TEST_ENV=staging and load .env.staging locally. CI can inject the same variables directly.

// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';
import dotenv from 'dotenv';

const environment = process.env.TEST_ENV ?? 'local';
dotenv.config({ path: `.env.${environment}` });

const baseURL = process.env.BASE_URL;
if (!baseURL) {
  throw new Error('BASE_URL must be configured for the selected environment');
}

export default defineConfig({
  testDir: './tests',
  forbidOnly: Boolean(process.env.CI),
  retries: process.env.CI ? 1 : 0,
  reporter: [
    ['list'],
    ['html', { open: 'never' }],
    ['junit', { outputFile: 'test-results/junit.xml' }],
  ],
  use: {
    baseURL,
    trace: 'retain-on-failure',
    screenshot: 'only-on-failure',
    video: 'retain-on-failure',
  },
  projects: [
    { name: 'chromium', use: { ...devices['Desktop Chrome'] } },
    { name: 'firefox', use: { ...devices['Desktop Firefox'] } },
    { name: 'webkit', use: { ...devices['Desktop Safari'] } },
  ],
});
Enter fullscreen mode Exit fullscreen mode

Ignore .env and .env.* while explicitly allowing .env.example. Also ignore playwright/.auth/, node_modules/, playwright-report/, and test-results/. Retain reports through CI artifacts with suitable access and retention settings because traces and screenshots can contain sensitive test data.

This configuration does not implement authentication automatically. Add an authentication setup project and reusable storage state or login fixture when your tests require it. Allocate separate accounts or data for concurrently running tests that modify server-side state.

A practical seven-day rollout

Treat the following as an initial plan for a small scope. Move forward when the previous stage is reliable, even if it takes more than a day.

Day Focus Deliverable
1 Setup, environments, fixtures, and 5–10 smoke checks Repeatable smoke suite
2 Functional coverage for one or two important modules Happy paths and validation checks
3 More functional coverage and API-backed setup Isolated data and integration checks
4 Regression selection and known bug coverage Tagged regression suite
5 CI integration, reporting, and failure diagnosis Reports and retained failure evidence
6 Accessibility and stable visual baselines axe, keyboard, and screenshot checks
7 Performance baseline followed by basic security automation Initial metrics and reviewed findings

Before expanding, confirm tests pass repeatedly, work independently, and fail with useful evidence when the expected behavior breaks. Resolve flaky data, brittle locators, and synchronization problems before increasing suite size.

What to run and when

Trigger Suggested scope
Local feature development Related functional and API tests
Pull request Fast smoke plus targeted regression
Staging deployment Smoke followed by broader functional/API regression
Nightly Broader cross-browser regression and selected accessibility/visual checks
Weekly or major release Controlled performance runs and scoped security scans
Production deployment Authorized, small, read-only smoke checks

Investigate initial failures even when a retry passes. A retry can provide diagnostic evidence, while recurring flaky tests need a cause and an owner. Keep reports tied to the commit, environment, browser, run time, test name, expected result, actual result, and failure evidence.

The code examples in this article are templates. Their routes, accessible names, API shapes, credentials, and seeded records must be adapted to the application before execution. They have not been run against HolidayLandmark.

Top comments (0)