Improving Design System Visual Testing Consistency with Docker
Tags: webdev, javascript, vue, testing
Published: 2023-05-06 08:56:20

Visual Testing is a great way to test the consistency of your UI component in a Design System. However, it can become an issue when working on a project with multiple developers and trying to run the test either locally or in CI, as the UI may appear different for each developer. Therefore, it is necessary to run the tests on the same browser and environment to ensure consistency.

# Docker

[Docker](https://www.docker.com/) is an open-source platform that enables developers to create, deploy, and run applications inside isolated containers. By using Docker, we can ensure consistent results across different environments and browser version.

# Implementation

This project was inspired by an awesome [design system](https://telekom.github.io/scale/) that implemented their own visual testing, which inspired me to explore how it works. You can see the full implementation for this project in this [GitHub repository](https://github.com/akbarnafisa/contra-ui-vue). Now, let's discuss how to implement it.

## Create UI Component

[This project](https://github.com/akbarnafisa/contra-ui-vue) uses Lerna to manage its packages

```jsx
- packages
	- core // Core component
	- design-tokens // Tokens
	- visual-test // UI Testing
```

- `core` package: this is the core component of the Design System. We use Vue 3, Vite, and TypeScript. We also use Storybook to display the component use cases and Jest for unit testing.
- `design-tokenks` package: We generate the tokens from this package.
- `visual-testing` package: We test the UI components from this package.

## Create Visual Test

This package utilizes several libraries

- `jest`: We use it to create our test cases.
- `jest-html-reporter`: We create a custom reporter to show which UI tests fail.
- `jest-image-snapshot`: We use this library to capture the UI screen and check changes between tests.
- `jest-puppeteer-docker`: We use this library to run the Puppeteer in the Docker.
- `puppeteer`: We use this library to interact with the browser to open the page to check for visual changes.

### **Prerequisites**

Before we run the test, we need to install Docker. The easiest way to do it is by installing [Docker Desktop](https://www.docker.com/products/docker-desktop). We also need to build the Storybook from the `core` package so that we can use it for visual testing.

### Setup Jest

We set the [Jest configuration](https://github.com/akbarnafisa/contra-ui-vue/blob/main/packages/visual-test/jest.config.js) to support the Visual test.

```jsx
module.exports = {
  preset: 'jest-puppeteer-docker',
  // specify a list of setup files to be executed after the test framework has been set up but before any test suites are run.
  setupFilesAfterEnv: ['./jest-setup/test-environment-setup.js'],
  // executed once before any test suites are run
  globalSetup: './jest-setup/setup.js',
  // The function will be triggered once after all test suites
  globalTeardown: './jest-setup/teardown.js',
  testMatch: ['**/?(*.)+(visual.spec).[tj]s?(x)'],
  modulePathIgnorePatterns: ['<rootDir>/dist/'],
	// add jest-html-reporter to be our costume reporters
  reporters: [
    'default',
    [
      'jest-html-reporter',
      {
        outputPath: './visual-test-result/index.html',
        pageTitle: 'Test Result',
        includeFailureMsg: true,
        // Path to a javascript file that should be injected into the test report,
        customScriptPath: './inject-fail-images.js',
      },
    ],
  ],
};
```

Then, we add the setup file to the initialization of the HTTP server from the build file of Storybook.

```jsx
const { setup: setupPuppeteer } = require('jest-puppeteer-docker');
const path = require('path');
const http = require('http');
const fs = require('fs');

const server = http.createServer((req, res) => {
  // parse URL
  const url = new URL(req.url, `http://${req.headers.host}`);

  // serve static files from "static" directory
  const filePath = path.join(__dirname, '../storybook-static', url.pathname);
  fs.readFile(filePath, (err, data) => {
    if (err) {
      // if file not found, return 404 error
      res.writeHead(404, { 'Content-Type': 'text/plain' });
      res.write('404 Not Found');
      res.end();
    } else {
      // if file found, return file contents
      res.writeHead(200, { 'Content-Type': getContentType(filePath) });
      res.write(data);
      res.end();
    }
  });
});

// helper function to get content type based on file extension
function getContentType(filePath) {
  const extname = path.extname(filePath);
  switch (extname) {
    case '.html':
      return 'text/html';
    case '.css':
      return 'text/css';
    case '.js':
      return 'text/javascript';
    case '.json':
      return 'application/json';
    case '.png':
      return 'image/png';
    case '.jpg':
    case '.jpeg':
      return 'image/jpeg';
    default:
      return 'application/octet-stream';
  }
}

module.exports = async (jestConfig) => {
  // start server on port 3000
  global.__SERVER__ = server.listen(3000, () => {
    console.log('Server started on port 3000');
  });

  await setupPuppeteer(jestConfig);
};
```

Then, we add a teardown configuration. In this configuration, we close the HTTP server and copy the test results to the reporter directory file. For more information about setup and teardown configuration, you can [read the documentation](https://github.com/gidztech/jest-puppeteer-docker#advanced-config) of `jest-puppeteer-docker`.

```jsx
const fs = require('fs');
const fse = require('fs-extra');
const path = require('path');

const { teardown: teardownPuppeteer } = require('jest-puppeteer-docker');

module.exports = async function globalTeardown(jestConfig) {
  global.__SERVER__.close();
  await teardownPuppeteer(jestConfig);

  const dirname = path.join(__dirname, '..');

  fs.copyFileSync(
    `${dirname}/jest-reporters/inject-fail-images.js`,
    `${dirname}/visual-test-result/inject-fail-images.js`
  );

  try {
    fse.copySync(
      `${dirname}/src/__image_snapshots__/__diff_output__`,
      `${dirname}/visual-test-result/__diff_output__`,
      {
        overwrite: true,
      }
    );
  } catch {}
};
```

Next, we add `jest-puppeteer-docker` config. See another config [here](https://github.com/gidztech/jest-puppeteer-docker#jest-puppeteer-config).

```jsx
const getConfig = require('jest-puppeteer-docker/lib/config');

const baseConfig = getConfig();
const customConfig = Object.assign(
  {
    connect: {
      defaultViewport: {
        width: 1040,
        height: 768,
      },
    },
    browserContext: 'incognito',
    chromiumFlags: '–ignore-certificate-errors',
  },
  baseConfig
);

module.exports = customConfig;
```

Finally, we add a global function to navigate to the Storybook page and another global function to do visual test.

```jsx
const { toMatchImageSnapshot } = require('jest-image-snapshot');

jest.setTimeout(10000);

expect.extend({ toMatchImageSnapshot });

global.goto = async (id) => {
  await global.page.goto(
    `http://host.docker.internal:3000/iframe.html?id=${id}&viewMode=story`
  );
  await page.waitForNavigation({
    waitUntil: 'networkidle0',
  });
};

global.testUI = async () => {
  await global.page.waitForSelector('#root');
  const previewHtml = await global.page.$('body');
  expect(await previewHtml.screenshot()).toMatchImageSnapshot();
};
```

### Setup Jest Reporter

After we finish testing, we usually want to check the test results. If there are any failed tests, we can display the comparison result here `packages/visual-test/visual-test-result/`.

```jsx
document.addEventListener('DOMContentLoaded', () => {
  [...document.querySelectorAll('.failureMsg')].forEach((fail, i) => {
    const imagePath = `__diff_output__/${
      (fail.textContent.split('__diff_output__/')[1] || '').split('png')[0]
    }png`;

    if (imagePath) {
      const div = document.createElement('div');
      div.style = 'margin-top: 16px';

      const a = document.createElement('a');
      a.href = `${imagePath}`;
      a.target = '_blank';

      const img = document.createElement('img');
      img.src = `${imagePath}`;
      img.style = 'width: 100%';

      a.appendChild(img);
      div.appendChild(a);
      fail.appendChild(div);
    }
  });
});
```

### **Adding tests**

To add new tests, usually it’s easier to take a screenshot of each Storybook page. Take a look at the example below, where we have a button component with 5 different pages.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ylfzfihj3mmsps2s59fn.png)

Then, we add the test case for each page.

```jsx
describe('Button', () => {
  test.each([['variants'], ['size'], ['disabled'], ['full-width']])(
    '%p',
    async (variant) => {
      await global.goto(`buttons-button--${variant}`);
      await global.page.evaluateHandle(`document.querySelector(".c-button")`);
      await global.testUI();
    }
  );
});
```

You can also use [Puppeteer API](https://pptr.dev/api/puppeteer.puppeteernode) if you want to take a screenshot after specific element action.

### Running Test

To run the tests you can follow these steps:

- Build Storybook by using `yarn workspace @contra-ui/vue build-storybook --quiet`
- Copy the Storybook build file from the `core` package to `visual-test` by using `yarn workspace @contra-ui/visual-test copy`
- Open your Docker desktop and run the test by using `yarn workspace @contra-ui/visual-test test`. If you run the test for the first time, it will take a while for Docker to download the image and build the container.
- Check the test result in `packages/visual-test/src/image_snapshots` to see the screenshot if it’s expected

### **Checking the failing tests**

To check the failed test you can open the  HTML report file in `packages/visual-test/visual-test-result/`.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7jeuc5pmbdw2anchjd4k.png)

Jest will place the differences in a folder, which you can inspect at `packages/visual-test/src/__image_snapshots__/__diff_output__/`. To update the test, you can add the `-u` flag: `yarn workspace @contra-ui/visual-test test`

### CI Setup

Next, let’s move the step that we have done locally to CI. There are three additional steps that are needed to automatically create PR to update the failed tests.

```yaml
name: build-pr
on:
  pull_request:
    branches:
      - main

jobs:
  install-dependency:
    name: Install depedency
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
      - name: Install Node.js
        uses: actions/setup-node@v3
        with:
          node-version: 16
          cache: 'yarn'
      - name: Install dependencies
        run: yarn --immutable

  visual-tests:
    name: Visual tests
    needs: [install-dependency]
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
      - name: Install Node.js
        uses: actions/setup-node@v3
        with:
          node-version: 16
          cache: 'yarn'

      - name: Install dependencies
        run: yarn --immutable

      - name: Run Lerna bootstrap
        run: yarn lerna:bootstrap

      - name: Build Storybook
        run: yarn workspace @contra-ui/vue build-storybook --quiet

      - name: Copy Storybook for visual tests
        run: yarn workspace @contra-ui/visual-test copy

      - name: Run visual tests
        run: yarn workspace @contra-ui/visual-test test -u

      - name: Check for any snapshots changes
        run: sh scripts/porcelain.sh

      - name: Set patch branch name
        if: failure()
        id: vars
        run: echo ::set-output name=branch-name::"visual-snapshots/${{ github.head_ref }}"

      - name: Create pull request with new snapshots
        if: failure()
        uses: peter-evans/create-pull-request@v4
        with:
          commit-message: 'test(visual): update snapshots'
          title: 'update visual snapshots: ${{ github.event.pull_request.title }}'
          body: This is an auto-generated PR with visual snapshot updates for \#${{ github.event.number }}.
          labels: automated pr
          branch: ${{ steps.vars.outputs.branch-name }}
          base: ${{ github.head_ref }}
```

- Step `name: Check for any snapshots changes`
In this step, we will check if there is a new commit. Because we set `-u` flag, it will automatically modify the existing file. When we find any modified file, the script exits with a non-zero exit code.
    
    ```bash
    echo "--------"
    echo "Checking for uncommitted build outputs..."
    if [ -z "$(git status --porcelain)" ];
    then
        echo "Working copy is clean"
    else
        echo "Another Pull Request has been created. Please check it to accept or reject the visual changes."
        git status
        exit 1
    fi
    ```
    
- Step `name: Set patch branch name`
In this step, we will save the branch name as an output variable if there are any previous steps have failed.
- Step `name: Create pull request with new snapshots`
In this step, we will create a pull request for the failed branch, this step will make it easier for us to see what UI changed and we can accept the changes if it’s expected, here is the pull request example:
    - main pull request: https://github.com/akbarnafisa/contra-ui-vue/pull/5
    - updated visual test pull request: https://github.com/akbarnafisa/contra-ui-vue/pull/6

You also need to update your workflow permissions on the Setting page to enable the GitHub Action to create Pull Request.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/dwyoclxw6i7sjcged6el.png)

# **Wrapping it up**

The usage of docker can be really used fully to make our visual tests more consistent, both locally and in CI. I hope this post has given you some ideas. You can see the full code in this  [GitHub repository](https://github.com/akbarnafisa/contra-ui-vue).