Issue 1:
Context:

Playwright Version: 1.29.2

Operating System: macOS Ventura 13.1

Node.js version: v18.13 (but tested 16.19 as well)

Browser: Chromium

Extra:

Does happen for both M1 and Intel chips

We use the mcr.microsoft.com/playwright:v1.29.2-focal docker image in CI, the difference is the same when comparing local vs in the image.

envinfo output

## System:
- OS: macOS 13.1
- Memory: 1.71 GB / 32.00 GB
## Binaries:
- Node: 18.13.0 - ~/.nvm/versions/node/v18.13.0/bin/node
- npm: 8.19.3 - ~/.nvm/versions/node/v18.13.0/bin/npm
## Languages:
- Bash: 3.2.57 - /bin/bash

Code Snippet

More importantly than the test: This snippet has to be run locally and then inside the docker image / some sort of CI for the bug to happen.

import { expect, test } from "@playwright/test";

test("Onboarding Step 1", async ({ page }) => {
await page.goto("/");
await page.getByText("Login").click();
await expect(page.getByTestId("onboarding-heading")).toContainText(
"Onboarding Step 1"
);
await expect(page).toHaveScreenshot(
"onboarding-step-1.png",
{ fullPage: true }
);
});

Describe the bug

Screenshot have flaky dimensions. Our locally generated, full-page, screenshots sometimes differ from those created inside docker by 1px, sometimes 2px height. If running the same test multiple times in the same environment (either docker or local), the screenshots match all the time. All tests are run with headless chromium. I've also experimented with any antialiasing and font rendering setup. This did not help either.

Result, for example:

Error: Screenshot comparison failed:

Expected an image 1280px by 1740px, received 1280px by 1739px.

Call log:
- expect.toHaveScreenshot with timeout 5000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- Expected an image 1280px by 1740px, received 1280px by 1739px.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- captured a stable screenshot
- Expected an image 1280px by 1740px, received 1280px by 1739px.

Issue 2:
Context:

Playwright Version: 1.27.1

Operating System: MacOS 13.0

Node.js version: 16.15.0

Browser: All

Extra: Also CI using MacOS 12 (Github Actions)

Describe the bug

Subsequent screenshots (on different test runs) of the same page are coming out with different heights. This is mainly due to a weird "compression" (like smashing) effect on one of the screenshots.

These two example come from the same code, same server (NextJS locally) running, just yarn test two times in a row:

I'm sorry for the long image, but that may be part of the problem (although short pages sometimes have this problem too).

If you compare the images using something like https://www.diffchecker.com/image-diff/ you can see the weird effect.

Is this a known bug? The differences are even greater once we run the tests on CI (more screenshots are different), but it's flaky, so sometimes it happens and sometimes everything is good.

EDIT: Just to clarify, by compression I mean that the pixels themselves are smashed, so everything is shorter, which in turn causes the heights of the screenshots to differ, like on of them was "smashed", which is wrong, e.g. causes the square logo to turn into a rectangle, etc

Thanks in advance!