Holistic Visual Regression Testing
Tags: nextjs, storybook, webdev, tutorial
Published: 2023-03-09 13:48:08

## Introduction to Holistic Visual Testing 🖼️

As front-end development continues to evolve, it's more important than ever to have a robust testing strategy in place to ensure that your applications look and function as expected. One of the most effective ways to do this is through visual regression testing, which allows you to catch and fix visual bugs before they reach production.

**Holistic visual regression testing** is an approach that aims to test the entire application, including both layout and interactions, as a whole.

In this example, we will set up a complex visual testing suite for our Next.js application. Here is the list of tools we will use:

* **Storybook**
    
* **Playwright**
    
* **GitHub actions**
    
* **Lost Pixel Platform**
    

As the final result, we want to make sure that the following parts of our applications are looking the way they are intended to look:

* Individual components - **visual unit tests**
    
* Pages - **visual integration tests**
    
* Results of playwright/cypress tests - **visual e2e tests**
    

Lost Pixel makes it easy to compose all of this in a single place to make the management of your visual testing a breeze. Here is the final version of the code for the whole integration:

Here is the final version of the [code on GitHub](https://github.com/lost-pixel/lost-pixel-next-holistic-visual-tests)

## Setting up Next application 🔼

We will clone the [app-playground](https://github.com/vercel/app-playground.git) repo from Next.js, it has rudimentary parts of the actual application, and that's precisely what we need.

```bash
git clone https://github.com/vercel/app-playground.git

pnpm install

pnpm dev
```

![image](https://cdn.sanity.io/images/809vdkvp/production/066def66917e0662bae053bf76bd4fa759d167c5-1896x1194.png)

## Visual unit tests 🖼️

We will use Storybook in combination with Lost Pixel to make sure our app components are looking the way we expect them to look.

Let's install Storybook:

```bash
npx storybook@next init && pnpm install postcss-loader style-loader @storybook/addon-postcss
```

Inside `.storybook/main.js` we need to make Storybook & Tailwind.css play well with each other :D

```javascript
const path = require('path');
module.exports = {
  stories: ['../stories/**/*.mdx', '../stories/**/*.stories.@(js|jsx|ts|tsx)'],
  addons: [
    '@storybook/addon-links',
    '@storybook/addon-essentials',
    '@storybook/addon-interactions',
    {
      name: '@storybook/addon-postcss',
      options: {
        cssLoaderOptions: {
          // When you have splitted your css over multiple files
          // and use @import('./other-styles.css')
          importLoaders: 1,
        },
        postcssLoaderOptions: {
          // When using postCSS 8
          implementation: require('postcss'),
        },
      },
    },
  ],
  framework: {
    name: '@storybook/nextjs',
    options: {},
  },
  docs: {
    autodocs: 'tag',
  },
};
```

Inside `.storybook/preview.js` we need to import tailwind styles on top of the file:

```javascript
import '../styles/globals.css';

export const parameters = {
  actions: { argTypesRegex: '^on[A-Z].*' },
  controls: {
    matchers: {
      color: /(background|color)$/i,
      date: /Date$/,
    },
  },
};
```

Inside `stories/Button.stories.tsx` let's write our first story that we will use in tests:

```javascript
import type { Meta, StoryObj } from '@storybook/react';
import React from 'react';

import Button from 'ui/Button';

// More on how to set up stories at: https://storybook.js.org/docs/7.0/react/writing-stories/introduction
const meta: Meta<typeof Button> = {
  title: 'Example/Button',
  component: Button,
};

export default meta;
type Story = StoryObj<typeof Button>;

export const Default: Story = {
  render: () => <Button kind="default">Default</Button>,
};

export const Error: Story = {
  render: () => <Button kind="error">Error</Button>,
};
```

Now we run our storybook to see our components displayed in an isolated manner:

```bash
npm run storybook
```

![image](https://cdn.sanity.io/images/809vdkvp/production/9c7106ebbc5756aac595d56fde22d7777624567d-1024x450.png)

Our stories will serve as the basis for the unit visual tests. Let's integrate this first step with Lost Pixel Platform to see how it will work there.

## Lost Pixel Platform ⚙️

You can use the [quickstart documentation](https://docs.lost-pixel.com/user-docs/setup/lost-pixel-platform) from [lost-pixel.com](http://lost-pixel.com) to set up the platform to work with your repositories via installing the official Lost Pixel GitHub app. In this guide, we will focus on setting up the right configuration, assuming you have done the platform setup already!

You can use Lost Pixel Onboarding to easily set up the project by copying & pasting the respective files.

In our `lostpixel.config.ts` at the root of our project, let's define the tests for Storybook

```javascript
import { CustomProjectConfig } from 'lost-pixel';

export const config: CustomProjectConfig = {
  storybookShots: {
    storybookUrl: './storybook-static',
  },

  lostPixelProjectId: 'clde9r3rh00v3m50vlq8y0k78',
  apiKey: process.env.LOST_PIXEL_API_KEY,
};
```

In our `.github/workflows/vrt.yml` at the root of our project let's define our CI/CD job:

```yaml
on: [push]

jobs:
  build:
    runs-on: ubuntu-latest
    name: Lost Pixel

    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - name: Setup Node
        uses: actions/setup-node@v3
        with:
          node-version: 16.x

      - name: Install dependencies
        run: npm install --legacy-peer-deps

      - name: Build Storybook
        run: npm run build-storybook

      - name: Lost Pixel
        uses: lost-pixel/lost-pixel@v3.0.4
        env:
          LOST_PIXEL_API_KEY: ${{ secrets.LOST_PIXEL_API_KEY }}
```

With this simple setup, we are already covering the visual unit tests 🙌🏼

![image](https://cdn.sanity.io/images/809vdkvp/production/d0cb270199226fe61560ed0c9ae61ce57b24d9a6-1856x2050.png?)

Let's make it really holistic now!

## Visual integration tests 🖼️ ↔️ 🖼️

Sometimes we care not only about how our components will be used in isolation but also about how they will look on the page when they are combined with a plethora of other components & styles. Let's explore how we can easily expand our initial setup to monitor our pages continuously!

Let's extend our `lostpixel.config.ts` to include our page shots.

```typescript
import { CustomProjectConfig } from 'lost-pixel';

export const config: CustomProjectConfig = {
  storybookShots: {
    storybookUrl: './storybook-static',
  },
  pageShots: {
    pages: [
      { path: '/layouts/books/fiction', name: 'fiction-books' },
      { path: '/layouts/books/biography', name: 'biography-pages' },
    ],
    baseUrl: 'http://172.17.0.1:3000',
  },
  lostPixelProjectId: 'clde9r3rh00v3m50vlq8y0k78',
  apiKey: process.env.LOST_PIXEL_API_KEY,
};
```

We also need to extend `.github/workflows/vrt.yml` to build & run our Next.js app so we can access pages in Lost Pixel:

```yaml
on: [push]

jobs:
  build:
    runs-on: ubuntu-latest
    name: Lost Pixel

    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - name: Setup Node
        uses: actions/setup-node@v3
        with:
          node-version: 16.x

      - name: Install dependencies
        run: npm install --legacy-peer-deps

      - name: Build Storybook
        run: npm run build-storybook

      - name: Build Next app
        run: npm run build

      - name: Run Next app
        run: npm run start &

      - name: Lost Pixel
        uses: lost-pixel/lost-pixel@v3.0.4
        env:
          LOST_PIXEL_API_KEY: ${{ secrets.LOST_PIXEL_API_KEY }}
```

Now we are talking 🚀 Our visual tests are covering not only **unit** parts with individual components but also **the integration** part where we are making the screenshots for the whole pages.

![image](https://cdn.sanity.io/images/809vdkvp/production/14cd5947e14ffbc2af96c9b11168133fca0e5714-2300x2038.png)

Let's make it an ultimate setup with a magic flavour of the feature of Lost Pixel called custom shots 🪄✨✨✨

## Visual E2E tests 🖼️ ⚙️ 🖼️ ⚙️ 🖼️

What if I told you that on any given CI run, you can create the screenshots using your logic & they will be available immediately on Lost Pixel Platform for the visual regression testing? Enter custom shots 📦

We will be using Playwright to interact with the page & make sure that after some set of interactions, we are always looking at the same UI.

```bash
npm install --save-dev @playwright/test
```

Let's also extend our `package.json` with `test:e2e` command:

```bash
...
"scripts": {
  "dev": "next dev",
  "build": "next build",
  "start": "next start",
  "test:e2e": "playwright test",
}
```

Now we will create our E2E test that will not do much, but will give you a good understanding of how you could integrate Lost Pixel into any test like this!

In `e2e/example.spec.ts` :

```typescript
import { test } from '@playwright/test';

test('lost-pixel e2e', async ({ page }) => {
  // Perform some action on the page, like clicking the button.
  // 📒 http://172.17.0.1:3000 is already fit for GitHub action runtime
  await page.goto('http://172.17.0.1:3000/context');
  await page.click('data-test-id=context-click-counter');
  await page.click('data-test-id=context-click-counter');
  // Make a shot for Lost Pixel to test
  // 📒 Path is arbitrary, but I advise you to store all of the custom shots in a single folder, e.g. lost-pixel. 
  await page.screenshot({ path: 'lost-pixel/a.png', fullPage: true });
});
```

Our `ContextClickCounter` component gets a new `data-test-id`, so the code above works:

```javascript
      <button
        // Here is the important part
        data-test-id="context-click-counter"
        onClick={() => setCount(count + 1)}
        className="rounded-lg bg-gray-700 px-3 py-1 text-sm font-medium tabular-nums text-gray-100 hover:bg-gray-500 hover:text-white"
      >
        {count} Clicks
      </button>
```

Now we are ready to integrate our E2E with Lost Pixel! Let's extend our `lostpixel.config.ts` to include our custom shots.

```javascript
import { CustomProjectConfig } from 'lost-pixel';

export const config: CustomProjectConfig = {
  storybookShots: {
    storybookUrl: './storybook-static',
  },
  pageShots: {
    pages: [
      { path: '/layouts/books/fiction', name: 'fiction-books' },
      { path: '/layouts/books/biography', name: 'biography-pages' },
    ],
    baseUrl: 'http://172.17.0.1:3000',
  },
  customShots: {
    // 📒 currentShotsPath should be matching the one above from playwright test
    currentShotsPath: './lost-pixel',
  },
  lostPixelProjectId: 'clde9r3rh00v3m50vlq8y0k78',
  apiKey: process.env.LOST_PIXEL_API_KEY,
};
```

And finally, we extend our GitHub Action declaration file by running Playwright tests:

```yaml
on: [push]

jobs:
  build:
    runs-on: ubuntu-latest
    name: Lost Pixel

    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - name: Setup Node
        uses: actions/setup-node@v3
        with:
          node-version: 16.x

      - name: Install dependencies
        run: npm install --legacy-peer-deps

      - name: Build Storybook
        run: npm run build-storybook

      - name: Build Next app
        run: npm run build

      - name: Run Next app
        run: npm run start &

      - name: Playwright tests
        run: npx playwright install --with-deps && npm run test:e2e

      - name: Lost Pixel
        uses: lost-pixel/lost-pixel@v3.0.4
        env:
          LOST_PIXEL_API_KEY: ${{ secrets.LOST_PIXEL_API_KEY }}
```

After Lost Pixel Platform runs this time, you should see something similar to this:

![image](https://cdn.sanity.io/images/809vdkvp/production/f36f66c4976292959369bc045746e71539648066-1856x854.png)

After interacting with our page & clicking the button two times, we made a screenshot of a page to test visually. Sometimes it's very useful because apart from asserting the presence or absence of some elements on the page that E2E tests traditionally rely on, the layout could be broken after a particular interaction and here visual regression tests will catch this for us.

## Summary 🔼

We have built a holistic visual testing framework over an existing Next.js application using the tools suitable for every task: **Storybook** for visual unit tests, **Pages** for visual integration tests & **Playwright** for visual E2E tests. We have use **GitHub Actions** to make sure our setup runs continuously on every push. Finally, we have used the **Lost Pixel Platform** to manage our visual tests in one central place & notify us whenever visual regression is found.

![image](https://cdn.sanity.io/images/809vdkvp/production/61df865a0b5598939a29b186556504e42024be10-1696x1082.png)

If you liked the read and enjoyed setting up your visual tests, you might as well like our [Visual Regression Testing Discord](https://discord.gg/WqVjk49g9m). There you can find help with any issues, chat about your setup & discuss the good practices for visual tests.