Setup Image Comparison Service of WDIO to make UI Testing more impactful
Tags: webdriverio, wdioimagecomparisonservice, visualregression, screenshottesting
Published: 2023-05-04 11:37:58

In the previous [article](https://blog.kiprosh.com/webdriverio-elements-chaining/), we talked about [WebdriverIO](https://webdriver.io/docs/gettingstarted/). A tool that has become popular very fast in the field of test automation due to its powerful advantages. In this article, we will talk about one of its advantages that has advanced our test automation suite. We have an application that helps our clients develop stunning websites. Whenever a new feature is developed that may impact the UI of the site, WebdriverIO can help us test the change in the UI of the sites. WebdriverIO provides a service called as *[wdio-image-comparison-service](https://webdriver.io/docs/wdio-image-comparison-service/)*, which performs **pixel-by-pixel** image comparison.

We already have a **wdio-based** automation project that does end-to-end functional testing. We expanded it to  screenshot test all the pages for thousands of sites. We explored various tools for image comparison. We then discovered that WebdriverIO furthermore offers an image comparison tool. Then we began looking into how we could incorporate this "wdio-image-comparison-service" into our already-running automation project.

### Set-up

Referring to the official [doc](https://webdriver.io/docs/wdio-image-comparison-service/) for `wdio-image-comparison-service`*,* we installed the module:

```js
npm install --save wdio-image-comparison-service
```

We added the below options to the Test runner services in `wdio.conf.js` file:

```js
services: [
  'chromedriver',
  [
    'image-comparison',
    // The options
    {
      // Some options, see the docs for more
      baselineFolder: join(process.cwd(), './tests/'),
      formatImageName: '{tag}-{logName}-{width}x{height}',
      screenshotPath: join(process.cwd(), '.tmp/'),
      savePerInstance: true,
      autoSaveBaseline: true,
      blockOutStatusBar: true,
      blockOutToolBar: true,
      disableCSSAnimation: false,
      ignoreColors: true,
      ignoreTransparentPixel: true,
    },
  ],
],
```

Above is the default setup. With this default setup, we thought that we had completed configuring the setup and the next step was to create a spec file to compare screenshots.

1.  `browser.saveScreen("homepage")` function saves the screenshot as a baseline image with the name `homepage.png`.
2.  `browser.checkScreen("homepage")` function saves the screenshot of the current page and compares it with the baseline image having the same name i.e. `homepage.png`.

However, we encountered an issue with the default setup. When we attempted to create a spec to compare screenshots of the two different doodles, it did not detect any mismatches, and the spec passed. We will explain this issue in more detail below:

We created a spec file called `image_compare.js` as follows:

```js
describe("Image Comparison", () => {
  it("saves the baseline image", async () => {
    await browser.url("https://www.google.com/doodles/teachers-day-2021-september-11");
    await browser.saveScreen("homepage");
  });

  it("compares with baseline image", async () => {
    await browser.url("https://www.google.com/doodles/teachers-day-2022-september-11");

    const result = await browser.checkScreen("homepage");

    expect(result).toEqual(0);
  });
});
```

In the above spec, we were comparing screenshots of the `Teacher's Day - 2021` homepage and `Teacher's Day - 2022` homepage. And we were expecting some mismatch percentage. But the spec got passed. When we investigated, we found that `wdio-image-comparison-service` saved the baseline image under `.tmp` folder. The `.tmp` folder has `actual` as a subfolder. Under the `actual` folder, it saved the baseline image.

In our case, when we ran the spec, it overrode Teacher's Day - 2021 `homepage.png` with Teacher's Day - 2022 `homepage.png` inside `.tmp/` folder. Inside `./tests/` folder also, it saved Teacher's Day - 2022 `homepage.png`. When we investigated more on this, then we got to know that internally this package uses `fs-extra` plugin and it has limited Windows support. But, running the same script on Mac or Linux did not result in any issues or failures. We also found out that this is an existing [issue](https://github.com/wswebcreation/wdio-image-comparison-service/issues/82). We have also added a [comment](https://github.com/wswebcreation/wdio-image-comparison-service/issues/82#issuecomment-1040224972) there about what could be a workaround.

### Workaround

As we previously said, the baseline picture is saved by 'wdio-image-comparison-service' by default under the '.tmp' folder. In 'wdio.conf.js', we removed the following code from the'services' section:

```js
baselineFolder: join(process.cwd(), './tests/'),
screenshotPath: join(process.cwd(), '.tmp/'),
```

Then, we specified the `testOptions` in the `it` block where screenshots are compared.

```js
const testOptions = {
  actualFolder: join(process.cwd(), './.tmp/checkActual'),
  baselineFolder: join(process.cwd(), './.tmp/actual'),
  diffFolder: join(process.cwd(), './.tmp/testDiff'),
  returnAllCompareData: true,
};
```

-   `checkActual`: The current image will be saved in this folder. It is called as `actualFolder`.
-   `actual`: The baseline image will be saved in this folder. It is called as `baselineFolder`.
-   `diffFolder`: This image gets created with all the mismatches highlighted after the two screenshots are compared.
-   `returnAllCompareData`: By default, `check` methods of this service returns only mismatch percentage. When this option is passed as `true`, then it returns the current image `filename`, `folders` that contain the path of `actual`, `baseline`, and `diff`, and `misMatchPercentage`.

After these modifications, the updated code is as follows:

`wdio.conf.js`

```js
services: [
  'chromedriver',
  [
    'image-comparison',
    // The options
    {
      // Some options, see the docs for more
      savePerInstance: true,
      autoSaveBaseline: true,
      blockOutStatusBar: true,
      blockOutToolBar: true,
      disableCSSAnimation: false,
      ignoreColors: true,
      ignoreTransparentPixel: true,
    },
  ],
],
```

`image_compare.js`

```js
import { join } from 'path';

describe("Image Comparison", () => {
  it("saves the baseline image", async () => {
    await browser.url("https://www.google.com/doodles/teachers-day-2021-september-11");
    await browser.saveScreen("homepage");
  });

  it("compares with baseline image", async () => {
    await browser.url("https://www.google.com/doodles/teachers-day-2022-september-11");
    
    const testOptions = {
      actualFolder: join(process.cwd(), './.tmp/checkActual'),
      baselineFolder: join(process.cwd(), './.tmp/actual'),
      diffFolder: join(process.cwd(), './.tmp/testDiff'),
      returnAllCompareData: true,
    };

    const result = await browser.checkScreen("homepage", testOptions);

    expect(result).toEqual(0);
  });
});
```

When we passed `testOptions` as a parameter to the `checkScreen` function, then `wdio` easily identified where our baseline images are saved and with which image to compare.

Now, if you run the above spec, then it will properly do the comparison, highlight the mismatch, and return the mismatch percentage.


<table>
  <tr>
      <td align="center"><b>Teacher's Day - 2021</b></td>
      <td align="center"><b>Teacher's Day - 2022</b></td>
  </tr>
  <tr>
    <td><img alt="image" src="https://blog.kiprosh.com/content/images/2023/04/homepage-chrome-1050x652-dpr-1.5-1.png"></td>
    <td><img alt="image" src="https://blog.kiprosh.com/content/images/2023/04/homepage-chrome-1050x652-dpr-1.5-2.png"></td>
  </tr>
  <tr>
      <td colspan="2" align="center"><b>Image Difference</b></td>
  </tr>
  <tr>
    <td colspan="2" align="center"><img alt="image" src="https://blog.kiprosh.com/content/images/2023/04/homepage-chrome-1050x652-dpr-1.5-3.png"></td>
  </tr>
</table>


The `wdio-image-comparison-service` is not just limited to capturing and comparing screens, it can also compare the full page screen, and elements, and also checks if the website supports the keyboard's `TAB` key. Please refer to `wdio-image-comparison-service` [options](https://github.com/wswebcreation/wdio-image-comparison-service/blob/HEAD/README.md).

### Mobile Screenshot Testing

If screenshot testing for mobile devices is needed, then just resizing your browser is never a good idea because this module compares visuals of what the end user is going to see. [Wdio Devtools](https://webdriver.io/docs/devtools-service/) can be helpful in such cases. `Wdio Devtools` is in itself a distinct, extensive subject that offers additional benefits.

We just need to pass `service` as `devtools` in `wdio.conf.js` file. We can emulate a specific device by `await browser.emulateDevice('iPhone X')`. Please refer to [Device emulation](https://webdriver.io/docs/devtools-service/#device-emulation).

We have created a small demo project [wdio-image-comparison-demo](https://github.com/mayankshukla94/wdio-image-comparison-demo) that you can clone for better understanding. You can explore other [options](https://github.com/wswebcreation/wdio-image-comparison-service/blob/HEAD/README.md) available for `wdio-image-comparison-service`.

**References:**

-   [Image Comparison (Visual Regression Testing) Service](https://webdriver.io/docs/wdio-image-comparison-service/)
-   [`wdio-image-comparison-service` options](https://github.com/wswebcreation/wdio-image-comparison-service/blob/HEAD/README.md)
-   [`wdio-image-comparison-service` issue](https://github.com/wswebcreation/wdio-image-comparison-service/issues/82)
-   [Wdio Devtools Device Emulation](https://webdriver.io/docs/devtools-service/#device-emulation)