Issue 1:
System info

Playwright Version: [v1.32.3]

Operating System: [ubuntu-20.04]

Browser: [Chromium]

Other info:

Source code

I provided exact source code that allows reproducing the issue locally.

implementing based on https://www.npmjs.com/package/@web/test-runner-visual-regression

Issue 2:
System info

Playwright Version: v1.32.0-focal, v1.35.1

Operating System: Ubuntu 22.04.2 LTS

Browser: Chromium

Docker version 24.0.4, build 3713ee1

Docker Compose version v2.19.1

self-hosted azure devops agent (3.217.1, 3.220.2 и 3.220.5 (latest))

we use m6i.2xlarge AWS instance type

Other info:

Dockerfile

FROM mcr.microsoft.com/playwright:v1.28.1-focal as ci-stage

WORKDIR /app
ENV PATH /app/node_modules/.bin:$PATH

COPY package*.json .npmrc ./
RUN npm ci
RUN npx playwright install --with-deps chromium
COPY . .

FROM ci-stage as build-stage
ARG BUILD_TYPE
RUN npm run build:$BUILD_TYPE

docker-compose.yml

version: '3.4'

# Command to run docker-compose
#docker-compose -f docker-compose-playwright.yml up --build --abort-on-container-exit

services:
database:
image: "postgres"
restart: always
ports:
- "5447:5432"
environment:
- POSTGRES_NAME=
- POSTGRES_USER=
- POSTGRES_PASSWORD=

api:
ports:
- "7002:80"
build:
context: .
dockerfile: ./Docker-Compose.Dockerfile
depends_on:
- database
entrypoint: ["dotnet", "Entry.dll"]

playwright:
build:
context: ../Frontend
dockerfile: ../Frontend/Playwright.Dockerfile
args:
- BUILD_TYPE=docker-compose
entrypoint: ["/bin/sh", "-c" , "sleep 30 && npm run test:pw:headless"]
environment:
- NODE_ENV=docker-compose
- CI=true
ports:
- "8082:8080"
volumes:
- /opt/vsts/work/_temp/playwright-report:/app/playwright-report
depends_on:
- api

npm script test:pw:headless launch frontend application and execute playwright test command

Playwright config

{
testDir: './playwright',
/* Maximum time one test can run for. */
timeout,
expect: {
/**
* Maximum time expect() should wait for the condition to be met.
* For example in `await expect(locator).toHaveText();`
*/
timeout,
},
/* Run tests in files in parallel */
fullyParallel: true,
/* Fail the build on CI if you accidentally left test.only in the source code. */
forbidOnly: !!process.env.CI,
/* Retry on CI only */
retries: process.env.CI ? 2 : 0,
/* Opt out of parallel tests on CI. */
workers: 6,
/* Reporter to use. See https://playwright.dev/docs/test-reporters */
reporter: isDev ? [
['list'], ['html'],
] : [
['list'],
['html', { open: 'never' }],
process.env.CI ? ['junit'] : ['list'],
],
/* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
globalSetup: require.resolve('./playwright.global-setup'),
testMatch: 'playwright/specs/**/*.spec.ts',
use: {
/* Maximum time each action such as `click()` can take. Defaults to 0 (no limit). */
actionTimeout: 0,
/* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
trace: 'retain-on-failure',
/* Base URL to use in actions like `await page.goto('/')`. */
baseURL: 'http://localhost:8080',
screenshot: 'only-on-failure',
testIdAttribute: 'data-test',
},
}

We have a 1-hour timeout for our builds. In the screenshot, you can see that consequent builds were cancelled by a timeout or cancelled manually.

Solution

Downgrade to Playwright playwright:v1.28.1-focal solves the issue, and now everything works well.

Also, there is no useful information even with DEBUG=pw:browser*.

I think that issue is somehow related to the Docker image.

I hope this issue will help somebody.