Issue 1:
System info

Playwright Version: [v1.30]

Operating System: [ Jenkins CI, Docker, Ubuntu 20]

Browser: [Chromium]

Other info:

Source code

Link to the GitHub repository with the repro

https://github.com/kLjubomir/playwright_reproducible

Additional information on Jenkins and Dockerization:

dockerfile content

#
# Dockerfile Description
#
FROM ubuntu

MAINTAINER Paul Brown

RUN apt-get update -y && apt-get install -y \
curl
RUN curl -sL https://deb.nodesource.com/setup_18.x | bash -
RUN apt-get -y install nodejs
RUN npm install -g playwright

Playwright Setup - Jenkins

A Jenkinsfile is used, with this stage to run the tests inside a docker container and not the Jenkins host.

stage('Running Tests') {
agent {
docker {
image “url to docker image” (dockerfile to generate is attached)
registryUrl “docker registry url”
registryCredentialsId XXXXXXXXXXX
args "-u root --shm-size=4gb -v $WORKSPACE/$tests_Dir:/tmp/tests"
reuseNode true
}
}
steps {
script {
String testsDir = "/tmp/tests"
RunBrowserTests(friendlyAppName, Target_URL, Test_Specs_GLOB, NPM_Script, testsDir, "", Target_Environment, Secret_Data_Path, Technology_Stack, "", "", Extra_Switches, "", "$WORKSPACE", CheckBooleanParam(Use_Script), Script_Config)
}
}
}

RunBrowserTests does the following….

Runs the bash script here : https://bitbucket.org/eurotaxglass1/av-automatron/src/master/runtests.sh

E.g : bash /tmp/tests/runtests.sh staging Regression true true

Steps

Run the tests over Jenkins repeatedly.

Expected

Tests are initiated concurrently, one param "de" runs with a project and grep of "de" and the other param "uk" runs with project and grep "uk", each run produces its own report, one in the "DE" and one in the "UK" folder of the playwright-report folder.

Actual

If running these tests on repeat, there will eventually within 10-20 runs be an error on Jenkins CI containing the Pirates dependency description, and when that occurs one of the concurrent runs will not execute or produce results, while the other concurrent run will. This happens with production code a lot more (every 2-4 runs) than with the boilerplate obscured code I have provided in the issue.

Issue 2:
System info

Playwright Version: [v1.34.3]

Operating System: [Ubuntu 20]

Browser: [All]

Other info:

Source code

I provided exact source code that allows reproducing the issue locally.

Link to the GitHub repository with the repro

[https://github.com/your_profile/playwright_issue_title]

or

Config file

import dotenv from 'dotenv';

require('./sentry')

dotenv.config();

export default defineConfig({

globalSetup: require.resolve('./global-setup'),

testDir: './tests',
timeout: 70000,
forbidOnly: !!process.env.CI,
retries: process.env.CI ? 2 : 0,
workers: process.env.CI ? 2 : undefined,
reporter: [['html', { open: 'never' }], ['list']],

expect: {
timeout: 20000,
},

use: {
headless: !!process.env.NLAB_APP_TESTS_HEADLESS,
actionTimeout: process.env.NLAB_APP_TESTS_ACTIONTIMEOUT ? +process.env.NLAB_APP_TESTS_ACTIONTIMEOUT : 5000,
baseURL: process.env.NLAB_APP_TESTS_CONTOUR,
ignoreHTTPSErrors: !!process.env.NLAB_APP_TESTS_IGNOREHTTPSERRORS,
screenshot: process.env.NLAB_APP_TESTS_SCREENSHOT as 'only-on-failure' || 'only-on-failure',
video: process.env.NLAB_APP_TESTS_VIDEO as 'retain-on-failure' || 'retain-on-failure',
storageState: 'state.json',
trace: process.env.NLAB_APP_TESTS_TRACE as TraceMode || 'retain-on-failure',
},

projects: [

process.env.NLAB_APP_TESTS_BROWSER === 'webkit' ? {
name: 'webkit',
use: {
...devices['Desktop Safari'],

viewport: { width: 1800, height: 900 },
},
} :

process.env.NLAB_APP_TESTS_BROWSER === 'firefox' ? {
name: 'firefox',
use: {
...devices['Desktop Firefox'],
viewport: { width: 1800, height: 900 },
},
} :

{
name: 'chromium',
use: {
...devices['Desktop Chrome'],
contextOptions: {
permissions: ['clipboard-read', 'clipboard-write'],
},
viewport: { width: 1800, height: 900 },
},
},
],
});
[playwright-report.zip](https://github.com/microsoft/playwright/files/11601331/playwright-report.zip)

Test file (self-contained)

it('should check the box using setChecked', async ({ page }) => {
await page.setContent(`<input id='checkbox' type='checkbox'></input>`);
await page.getByRole('checkbox').check();
await expect(page.getByRole('checkbox')).toBeChecked();
});

Steps

[Run the test]

[...]

Expected

[Describe expected behavior]

Actual

[Describe actual behavior]