Automated visual regression testing with TypeScript, Puppeteer, Jest and Jest Image Snapshot
Tags: jest, typescript, puppeteer, testing
Published: 2021-04-16 16:14:16

# Introduction

As developers, it is our job to ensure that our users get an experience with no regression.  
Like any good developer, when I add a feature or fix a bug, I also create unit, integration, and end-to-end tests. This assures that, when the existing code is modified, nothing is accidentally broken, and confirms that user flows are functional.

Like many people, I've gotten used to using a manual process to visually check that the design looks as intended.

Refactoring, adding a new component, or updating a package can sometimes change the appearance of the application. It can get laborious to click through every possible user journey, and we are not immune to forgetting a test or to miss a small visual change.

So how can we make sure that the visuals are always correct and less painful to test? I had heard of automatic testing for no visual regression before. I looked into using Selenium some time ago, but changed projects and no longer needed it.

Now that I have a need for visual checks in my current project, I’ve found there are different libraries - easy to learn - that can take screenshots of current web pages and compare generated screenshots with a screenshot baseline to find regressions in the user interface (UI).

In this article, I will explain how to use one of these libraries - Jest-Image-Snapshot (Jest matcher) - in a Typescript project.

# Project example

I will use the <a href="https://github.com/process-analytics/bpmn-visualization-js"><u>bpmn-visualization</u></a> TypeScript library (version 0.10.0) as an example. (This example has been simplified so it shows more clearly the configuration and features explained in this article.) The goal of this project is to load BPMN content, and render it. 
Automated visual tests will simplify our life with each refactoring, addition of a new component, update of the positioning algorithm of the different BPMN elements, or update of the MxGraph rendering library.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/obyz4kbyl2n1c1xxwrbf.png)


# Prerequisites

As first step, we need to install the required packages as devDependencies:

-   **Jest** + its type definition: A JavaScript Testing Framework

[<u>Jest</u>](https://jestjs.io/) is a fully featured testing framework, developed by Facebook. It needs very little configuration and works basically out of the box.

> *npm install -D jest @types/jest*

-   **Puppeteer** + its type definition: A Node library to control Chrome or Chromium, both in headless mode and with a user interface. It is possible to perform most of the actions that are done manually on a browser and take screenshots.

> *npm install -D puppeteer @types/puppeteer jest-puppeteer*

-   **Jest-Image-Snapshot** + its type definition: A Jest matcher to perform image comparisons

> *npm i -D jest-image-snapshot @types/jest-image-snapshot*

# Configuration

Let’s configure the previous libraries.

## Configure Jest

I won't go into detail here on all the different ways to configure Jest.
If you already use Jest for your unit/e2e tests, this is not new for you. If you would like more explanations about Jest, there are many great articles available.

In this example, we have 3 Jest configurations: unit tests, e2e tests, and performance tests. We added the visual tests in the e2e test suite.
Here I will just explain how we configure Jest for the e2e tests.

First, create the Jest configuration file (_jest.config.js_) at **./test/e2e** directory:
{% gist https://gist.github.com/csouchet/dda62f5f7fec77f60a32e90509459173 %}

This configuration sets the root directory to the root project directory, runs *.ts* files with ts-jest module and looks for *.spec.ts* and *test.ts* files under any subdirectory of **./test/e2e** directory.

## Configure Puppeteer

-   Specify the preset in the Jest configuration (_./test/e2e/jest.config.js_), as specified in the [official documentation of Jest](https://jestjs.io/docs/en/puppeteer):
{% gist https://gist.github.com/csouchet/b572f31f31a1c8fac81137e7880ac7ec %}

-   Create a new file (_./test/e2e/jest-puppeteer.config.js_) for the Puppeteer configuration to run the server & [launch](https://pptr.dev/#?product=Puppeteer&version=v5.5.0&show=api-puppeteerlaunchoptions) the browser once for all tests:
{% gist https://gist.github.com/csouchet/0e3834451dc541c052022e08f61fcc5f %}

With this configuration, we start a server on the port **10002** with a timeout of 30s, start a browser with a timeout of 2 minutes, and pipe the browser process stdout and stderr into **process.stdout** and **process.stderr**.

## Extend the Jest expect assertion mechanism to use Jest Image Snapshot

This is the part that might be new, but with a little configuration, we will be ready soon.

By default, **Jest** doesn’t know anything about **Jest-Image-Snapshot** and its assertion **toMatchImageSnapshot**. So we’ll need to extend Jest. For that, create a new file (_./test/e2e/jest.image.ts_), like the following:
{% gist https://gist.github.com/csouchet/f581b3bdc718f66759265b6a0be737fe %}

To avoid extending Jest in each test file or import the previous file globally in all test files, we need to configure Jest to run it immediately after the test framework has been installed in the environment with [**setupFilesAfterEnv**](https://jestjs.io/docs/en/configuration#setupfilesafterenv-array) (Jest property) in _./test/e2e/jest.config.js_.
{% gist https://gist.github.com/csouchet/bd3e625eace7402c69d813757f107c0b %}

## Add a new command

To simply the test execution, add the following script in the **package.json** file:
{% gist https://gist.github.com/csouchet/12e53d3a81e7de83f93ba666f0293074 %}

Now, you can run your e2e tests with the following command:
`npm run test:e2e`

**Note**: **cross-env** is useful if you run the tests on different OS.

# Test

You can find the different properties to customize **Jest-Image-Snapshot** in its [README](https://github.com/americanexpress/jest-image-snapshot#%EF%B8%8F-api) on Github.

## Create a basic test with Jest-Image-Snapshot

If everything is configured correctly, we are now ready to create our first visual regression test (_./test/e2e/bpmn.rendering.test.ts_) by combining **Puppeteer** and **Jest** and **Jest-Image-Snapshot**!
{% gist https://gist.github.com/csouchet/dc5af253ed3060774f4feae781fcc60d %}

After the test runs, a new directory will be created -
**\_\_image\_snapshots\_\_** - with an image for each *toMatchImageSnapshot* call. The names of the snapshots are computed by default with **testPath**, **currentTestName**, **counter** and **defaultIdentifier**.

**Example of generated snapshot:**

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/akwk1ojiu69wrkvhh7mn.png)
_bpmn-rendering-test-ts-no-bpmn-gateway-visual-regression-1-snap.png_

**Note**: Make sure that the snapshot files are committed in your source control so they are shared with other developers and CI environments.

## Test on different machines

One issue with one-to-one pixel matching is that there is a good chance that the test will be in error on a machine other than on which it was developed, because every environment has slightly different ways of rendering the same application.

For example, suppose that we want to run the tests on the CI environment every time we create a pull request to the master branch in GitHub.
Without any modifications to the code, the test is passed locally; but on the CI environment, it fails with a message like this:

> *Error: Expected image to match or be a close match to snapshot but was 0.0005804554357724534% different from snapshot (2.7861860917077763 differing pixels).*

And a new image file for the diff is stored in the **\_\_image\_snapshots\_\_/\_\_diff\_output\_\_** directory with the name **&lt;snapshot\_name&gt;-diff.png**.

If we use the previous test, we’ll have something like this:
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/p08p9qxwlxsd9ap6opju.png)
_bpmn-rendering-test-ts-no-bpmn-gateway-visual-regression-1-diff.png_

You can modify the previous **jest-image-snapshot** configuration (_./test/e2e/bpmn.rendering.test.ts_), and update the value of *failureThreshold (default value: 0)* & *failureThresholdType (default value: pixel)*. These properties are used to calculate the threshold of tolerated differences (before the test fails).
{% gist https://gist.github.com/csouchet/5952bffac22eb46a7fe7b46d20f7ca7a %}

**Warning**: If you increase the failure threshold too much, when there is too much difference between local and CI environments, it may be impossible to detect visual regressions.

## Order the snapshots

If you have 10 or more tests, it can become complicated to find which screenshot corresponds to which test/feature in the directory **\_\_image\_snapshots\_\_**.

Modify the **customSnapshotsDir** property to have a different value according to the tests.

-   _./test/e2e/helpers/visu-utils.ts_
{% gist https://gist.github.com/csouchet/fb45208eefb1cc838fe21387e2ddfe1f %}

-   _./test/e2e/bpmn.rendering.test.ts_
{% gist https://gist.github.com/csouchet/003ad914eca9c529fd300723d85cc67a %}

-   _./test/e2e/bpmn.navigation.test.ts_
{% gist https://gist.github.com/csouchet/e9b12947f592bf13b3763e0d87b9ced8 %}

## Reuse the snapshots

Sometimes the expected result/snapshot is the same even after different actions. To avoid having a lot of identical snapshots in the Github repository, it’s better to reuse a snapshot.

For that, it’s necessary to override the default **customSnapshotIdentifier** & **customDiffDir** properties.

-   **customSnapshotIdentifier**: the custom name to give this snapshot. This prevents the name of the snapshots from being computed with **testPath**, **currentTestName**, **counter** and **defaultIdentifier**.

-   **customDiffDir**: the custom absolute path of a directory to keep this diff in. As we use the same snapshot in different tests, to know which diff file corresponds to which test, we need to set a different value according to the tests.   


**Example**

-   _./test/e2e/helpers/visu-utils.ts_
{% gist https://gist.github.com/csouchet/830ed5ca947d13780cdbe81436666bd2 %}

-   _./test/e2e/diagram.rendering.test.ts_
{% gist https://gist.github.com/csouchet/0f070cb42258dfd756cbea5e10cdf735 %}

# Conclusion

With so many operating systems, web browsers, and screen resolutions, Visual Testing can be a powerful tool to assure that an application works well in all possible environments. It is definitely worth trying it as a complement to other sets of tests.

Now you have everything you need to start your first visual regression test in Typescript with Jest & Puppeteer.

Thank you for reading and I hope I helped or inspired you :)

# References

-   Github repository of bpmn-visualization: [<u>https://github.com/process-analytics/bpmn-visualization-js</u>](https://github.com/process-analytics/bpmn-visualization-js)

-   Github repository of Jest-image-snapshot: [<u>https://github.com/americanexpress/jest-image-snapshot</u>](https://github.com/americanexpress/jest-image-snapshot)

-   Puppeteer documentation: [<u>https://pptr.dev/</u>](https://pptr.dev/)

-   How to use Puppeteer with Jest: [<u>https://jestjs.io/docs/en/puppeteer</u>](https://jestjs.io/docs/en/puppeteer)

-   Main image made with: [<u>https://www.canva.com</u>](https://www.canva.com)