pytest-playwright-visual-snapshot 0.5.1
Easy pytest visual regression testing using playwright
Requires Python: >=3.11
Keywords: pytest, playwright, visual, regression, testing

[![Release Notes](https://img.shields.io/github/release/iloveitaly/pytest-playwright-visual-snapshot)](https://github.com/iloveitaly/pytest-playwright-visual-snapshot/releases)
[![Downloads](https://static.pepy.tech/badge/pytest-playwright-visual-snapshot/month)](https://pepy.tech/project/pytest-playwright-visual-snapshot)
![GitHub CI Status](https://github.com/iloveitaly/pytest-playwright-visual-snapshot/actions/workflows/build_and_publish.yml/badge.svg)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

# Pytest Plugin for Visual Testing with Playwright

As of 2025-03-22 of all of the existing packages to do simple visual regression testing using playwright are long dead. I had a bunch of updates I wanted to make to existing systems so I rewrote the plugin with a bunch of updates:

- snapshots are always written on CI to easily download them as artifacts
- ability to mask out certain elements which cause comparison brittleness
- failing on `--update-snapshots` to make users manually review images
- snapshot name is optional, `test_name[browser][os].png` is auto-generated by default
- multiple snapshots in a single test, file names are auto incremented
- updated folder structure: `snapshots/file_name/test_name/test_name[browser][os].png`
- ability to configure directories, etc via ini + pytest config.

You can see this implemented in a [working project here](https://github.com/iloveitaly/python-starter-template/).

## Installation

```bash
uv add --dev pytest-playwright-visual-snapshot
```

## Usage

This plugin provides a `assert_snapshot` fixture which is used to create snapshots and compare it.

```python
def test_myapp(page, assert_snapshot: AssertSnapshot):
    page.goto("https://example.com")
    assert_snapshot(page)
```

Then, run pytest:

```bash
pytest
```

The first time you run pytest, snapshots will be created, and you will get the error:

```console
New snapshot(s) created. Please review images
```

The next run, the snapshots comparison will take place. To update snapshots, run:

```bash
pytest --update-snapshots
```

After updating, tests will fail and you will need to review images.

In case of a mismatch, `snapshot_tests_failures` folder will be created with `actual_..`, `expected_..` and `diff_..` images generated.

## Configuration

View all configuration options by running `pytest --help`. Here's a quick example:

```python
# NOTE this runs on any pytest invocation, even if no tests are run
def pytest_configure(config: Config):
  config.option.playwright_visual_snapshots_path = Path("...")
  config.option.playwright_visual_snapshot_failures_path = Path("...")
```

### Masking Elements

You can mask certain elements during screenshot capture to prevent them from causing comparison failures. This is useful for dynamic content like timestamps, user avatars, etc.

Configure global masks in your pytest.ini:

```ini
[pytest]
playwright_visual_snapshot_masks =
    [data-clerk-component="UserButton"]
    .timestamp
    #dynamic-content
```

Or directly via `pytest_configure`:

```python
def pytest_configure(config: Config):
    config.option.playwright_visual_snapshot_masks = [
        '[data-clerk-component="UserButton"]',
        '.timestamp',
        '#dynamic-content'
    ]
```

Or specify masks directly in your test:

```python
def test_with_custom_masks(page, assert_snapshot: AssertSnapshot):
    page.goto("https://example.com")
    assert_snapshot(page, mask_elements=[".user-avatar", "#timestamp"])
```

### Handling Size Differences

By default, if a snapshot's dimensions change (width or height), pytest will raise an exception without generating visual comparison images. This can happen when taking snapshots of individual elements whose size changes due to visual modifications - for example, a button with updated padding or a container that dynamically adjusts its height.

To see visual diffs even when dimensions change, use the `--ignore-size-diff` flag:

```bash
pytest --ignore-size-diff
```

Or configure it globally in your pytest.ini:

```ini
[pytest]
playwright_visual_ignore_size_diff = true
```

Or via `pytest_configure`:

```python
def pytest_configure(config: Config):
    config.option.playwright_visual_ignore_size_diff = True
```

When enabled, snapshots with different dimensions will still generate the `actual_`, `expected_`, and `diff_` images in the failures folder, allowing you to see exactly what changed visually.

**Example use-case:** You're testing a UI component and update the button padding. Without `--ignore-size-diff`, you'd just get an exception. With it enabled, you get visual diff images showing the padding changes, making it easy to review and decide if the changes are intentional.

### Disabling Visual Snapshots Locally

If CI screenshots are the source of truth, you can disable local visual assertions to keep developer runs fast and avoid creating/comparing snapshots; use `pytest --disable-visual-snapshots` (or set `playwright_visual_disable_snapshots = true` in `pytest.ini`). When disabled, `assert_snapshot` is a noop and logs a warning.

### GitHub Actions Script

The CI Chrome will be slightly different than your dev chrome. You'll want to pull down screenshots from your CI run and use those for comparison. Here's a script to do that:

```shell
failed_run_id=$(gh run list --status=failure --workflow=workflow_name.yml --json databaseId --jq '.[0].databaseId')
PLAYWRIGHT_RESULT_DIRECTORY=

rm -rf ${PLAYWRIGHT_RESULT_DIRECTORY}/${failed_run_id} && \
mkdir -p ${PLAYWRIGHT_RESULT_DIRECTORY}/${failed_run_id} && \
gh run --dir ${PLAYWRIGHT_RESULT_DIRECTORY}/${failed_run_id} download $failed_run_id && \
cp -R ${PLAYWRIGHT_RESULT_DIRECTORY}/${failed_run_id}/test-results/${PLAYWRIGHT_VISUAL_SNAPSHOT_DIRECTORY}/ ${PLAYWRIGHT_VISUAL_SNAPSHOT_DIRECTORY}/
```

## API

### Fixture Parameters

- `threshold` - sets the threshold for the comparison of the screenshots:`0` to `1`. Default is `0.1`
<!-- - `name` - `.png` extensions only. Default is `test_name[browser][os].png` (recommended) -->
- `fail_fast` - If `True`, will fail after first different pixel. `False` by default
- `mask_elements` - List of CSS selectors to mask during screenshot capture. These will be combined with any globally configured masks.

### Command Line Options

- `--update-snapshots` - Update existing snapshots with new screenshots
- `--ignore-size-diff` - Generate visual diffs even when snapshot dimensions differ (instead of raising an exception)
- `--disable-visual-snapshots` - Disable visual snapshot assertions

## Alternatives

- https://github.com/kumaraditya303/pytest-playwright-snapshot - long dead
- https://github.com/symon-storozhenko/pytest-playwright-visual - fork of the above repo, long dead
- https://github.com/Visual-Regression-Tracker/Visual-Regression-Tracker - requires a separate server to run

---

*This project was created from [iloveitaly/python-package-template](https://github.com/iloveitaly/python-package-template)*