SvelteKit Accessibility Testing: Automated CI A11y Tests
Tags: webdev, svelte, javascript, a11y
Published: 2022-03-28 16:22:41

---
title: "SvelteKit Accessibility Testing:  Automated CI A11y Tests"
published: "true"
description: "SvelteKit Accessibility Testing: see how you can use Playwright to run automated axe accessibility & htmlcs tests in your Svelte CI workflow."
tags: "webdev, svelte, javascript, a11y"
canonical_url: "https://rodneylab.com/sveltekit-accessibility-testing/"
cover_image: "https://dev-to-uploads.s3.amazonaws.com/uploads/articles/55dy0lzfwi4szxyr9f5v.png"
---


## ☑️ Why Include Accessibility Testing in your Workflow?

SvelteKit accessibility testing does not have to be complicated &mdash; we&rsquo; see that in this post.  The need for accessibility testing in your apps is clear, and there is no point rehearsing the motivation for accessibility testing itself here. However, a valid question is why  include accessibility testing in the continuous integration process? In other words, once you have built your site and improved the accessibility, why do you need to continue running tests? 

The simple answer is that it is very easy, as you update a site, to break its accessibility.  SvelteKit tooling goes a long way to help you make your site accessible, with helpful accessibility integration in the VSCode plugin. Not to mention handy hints from the svelte-check tool.  However, these don&rsquo;t currently check how the new tint of your brand colour impacts the colour contrast ratio. Nor if you accidentally reused a previously taken `id` on a new element which you just added.

In short, your initial accessibility tests help to set you on the right track, while automated integration tests run with each commit or push, help make sure your site stays accessible.  Of course these are automated tests and have limitations, so to cover all bases you may also want to get the site audited by an accessibility professional.  That said the automated tests are still a handy tool to keep in your tool belt.

## 🧱 What we're Building

In this post we'll see how you can use **Playwright** to run integration tests as you develop or each time you build your site.  Playwright is an automated testing tool like Cypress.  We just use it for accessibility testing here, but you can also use it for more complete end-to-end and integration testing. On top  we level up the testing using the **pa11y** tool, running **axe** and **htmlcs** tests.  Then we add polish with **Husky**, merging the tests into your continuous integration process.  If that sounds  like something you can get excited about, then why don&rsquo;t we get cracking?

## ⚙️ SvelteKit Accessibility Testing: Setup

We will run through using the <a aria-label="Open Svelte Kit M D Svex starer Git Hub repo" href="https://github.com/rodneylab/sveltekit-blog-mdx">SvelteKit MDsveX starter</a> but if you already have a SvelteKit site, it might make more since for you to follow along but working on a test branch of your own site.  If you are using the starter, let&rsquo;s get going by cloning it locally:

```shell
git clone https://github.com/rodneylab/sveltekit-blog-mdx.git sveltekit-accessibility-testing
cd sveltekit-accessibility-testing
pnpm install
```

Next we&rsquo;ll install the extra packages needed (do this whether you're working on the starter or your own site):

```shell
pnpm add -D @playwright/test pa11y pa11y-ci playwright-core
pnpx install playwright
```

Playwright can run tests in various browsers and the second step above installs all the browser packages Playwright needs.  This can take a few minutes, depending on your internet connection, so you might want to brew a coffee once you start that step running.  You will also need to rerun that step each time you update Playwright.


### `playwright.config.js`

Once you have all of that installed, let&apos;s move on to adding a Playwright config file.  Create `playwright.config.js` in your project&rsquo;s root folder and add the following content:


```javascript
import { devices } from '@playwright/test';

/** @type {import('@playwright/test').PlaywrightTestConfig} */
const config = {
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 2 : 0,
  use: {
    trace: 'on-first-retry',
  },
  projects: [
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
    },
    {
      name: 'firefox',
      use: { ...devices['Desktop Firefox'] },
    },
    {
      name: 'webkit',
      use: { ...devices['Desktop Safari'] },
    },
  ],
  // webServer: {
  //   command: 'pnpm run build && pnpm run preview',
  //   port: 3000,
  // },
};
export default config;
```

There&rsquo;s a couple of ways you can run the tests.  If you want to run tests with a dev server spinning in the background, then the config above, as it is, is what you need.  Alternatively, once you&rsquo;re  ready to commit, you can stop your dev server and have Playwright automatically build and test the site. In this case uncomment lines `24`&ndash;`27` (also change the port in line `26` to suit your needs).

The final piece of setup it so add an environment variable to make things a little more configurable.  If you are working on your own site, you will need to install the `dotenv` package.  Either way, update the `.env` file to include the following line:

```plaintext
SITE_TEST_URL="http://localhost:3000"
```

Once again change the port if needed.

## 🏡 Home Page Test

We will place all of our test files in a new `tests` folder in the project root.  Create a `pages` folder within `tests` then add an `index.spec.mjs` file (if you prefer TypeScript create `index.spec.ts`).  Add the following content:

```javascript
import { expect, test } from '@playwright/test';
import 'dotenv/config';
import pa11y from 'pa11y';

const path = '/';
const url = `${process.env.SITE_TEST_URL}${path}`;

test.describe('it has no accessbility issues', () => {
  test.beforeEach(async ({ page }) => {
    await page.goto(url);
  });
  test('it has no issues identified by htmlcs or axe', async () => {
    const results = await pa11y(url, {
      runners: ['axe', 'htmlcs'],
    });
    if (results.issues) {
      results.issues.forEach(({ code, message, runner, selector }) => {
        console.log({ code, message, runner, selector });
      });
    }
    expect(results.issues.length).toBe(0);
  });
});
```

This is a basic Playwright test for our home page.  It does not look too different to test files from other testing tools, so we won&apos;t spend too much time on it.  If there is anything that needs clarification, please drop a comment below so I can update the post.

In line `14` you see we run both axe and HTML Code Sniffer tests.  Together these two runners will <a aria-label="Read morea bout configuring P A 11 Y" href="https://accessibility-manual.dwp.gov.uk/best-practice/automated-testing-using-axe-core-and-pa11y#single-page-tests-using-axe-core-and-pa11y">find 35% of accessibility issues</a>.  I have noticed that the axe tests can fail on contrast ratio when you use background gradients (even when manual testing shows contrast ratios are good).  You will probably want to <a aria-label="Open p a 11 y configutation" href="https://github.com/pa11y/pa11y#configuration">tweak configuration</a> on such pages, if you run into this issue.

You can test the home page now, from the Terminal:

```shell
pnpx pa11y http://localhost:3000/
```

We will look at some scripts and integrated testing  in a moment.  Before that, try breaking accessibility to reassure yourself the tests work.  As an example, you could temporarily remove the title in the HTML head.  In the starter, this is injected in `src/lib/components/SEO/index.svelte`.  Just comment out the `  <title>{pageTitle}</title>` at line `27` to test.

You can create a similar test file, to the one above, for other pages.  In the starter, we have a contact page so copy the contents above to `tests/pages/contact.spec.mjs` and just update line `4` in the new file:

```javascript
import { expect, test } from '@playwright/test';
import pa11y from 'pa11y';

const path = '/contact';
const url = `${process.env.SITE_TEST_URL}${path}`;
```

Next let's see what we can do for blog posts.

## 🖋 SvelteKit Blog Post Accessibility Testing

We will create a single test file for all blog posts, so you can see how to parameterise Playwright tests.  This pattern will save you creating new test files as you write fresh blog posts.  Make a new folder: `tests/posts` and in there create `index.spec.mjs`:

```javascript
import { expect, test } from '@playwright/test';
import 'dotenv/config';
import { existsSync, lstatSync, readdirSync } from 'node:fs';
import { join, resolve } from 'node:path';
import pa11y from 'pa11y';

export const BLOG_PATH = 'src/content/blog';
const __dirname = resolve();
export function getSlugs(location) {
  const directories = readdirSync(location).filter((element) =>
    lstatSync(`${location}/${element}`).isDirectory(),
  );
  const articles = [];

  directories.forEach((element) => {
    const contentPath = `${location}/${element}/index.md`;
    if (existsSync(contentPath)) {
      articles.push(element);
    }
  });
  return articles;
}

const postLocation = join(__dirname, BLOG_PATH);
const slugs = getSlugs(postLocation);

test.describe('it has no accessbility issues', () => {
  for (const slug of slugs) {
    const url = `${process.env.SITE_TEST_URL}/${slug}`;
    test.beforeEach(async ({ page }) => {
      await page.goto(url);
    });
    test(`${slug}: it has no issues identified by htmlcs or axe`, async () => {
      const results = await pa11y(url, {
        runners: ['axe', 'htmlcs'],
      });
      if (results.issues) {
        results.issues.forEach(({ code, message, runner, selector }) => {
          console.log({ code, message, runner, selector, url });
        });
      }
      expect(results.issues.length).toBe(0);
    });
  }
});
```

Here at the top we have a function to generate a list of all the slugs for blog posts.  You might need to adapt this to work on your own site.  Then in line `28` you see the `for` loop which runs the tests for each slug.  If you update this code for your own site, be cognisant that the existing `for` loop contains `async` code so you will need to be careful if you opt for using the `forEach` array method.  In the test name at line `33` we include the slug parameter to give each test a unique name.

Try testing a blog post or two.  Next we will look at some integration scripts.  Add more test files if you have different types of pages in your own site.

## Husky Continuous Integration Scripts

We are almost done.  Tinker with the following steps to suit your own taste.  First add two new testing scripts to `package.json`:

```json
{
  "name": "sveltekit-blog-mdx",
  "version": "2.0.0",
  "scripts": {
    // ...TRUNCATED
    "test:a11y": "pa11y-ci --sitemap 'http://localhost:3000/sitemap.xml' --sitemap-find 'https://example.com' --sitemap-replace 'http://localhost:3000'",
    "test": "playwright test"
  },
```

Update the ports and URLs here depending on your site and local setup.  I would run the `test` script with each commit, to check my work as I go along (or even between commits, depending on what I am working on).  This will run all of the test files in the `tests` folder.  Then I would do a double-check before pushing, which involves running `test:a11y` on all pages in the sitemap.  I would build the site and run it in preview mode, making sure the sitemap is up-to-date first.  With the starter, you can update the sitemap by running `pnpm run generate:sitemap`.

With those scripts defined, let&rsquo;s add Husky scripts.  These will run when we commit and push the code.  If you are running on your own site, install `husky` to get this going.

When you are ready, update `.husky/pre-commit` and `.husky/pre-push`:

```shell
#!/bin/sh
. "$(dirname "$0")/_/husky.sh"

pnpm run prettier:check && pnpm run lint:scss && pnpm run test
```

```shell
#!/bin/sh
. "$(dirname "$0")/_/husky.sh"

pnpm run check && pnpm run test:a11y
```

Then finally to complete setup in the Terminal run:

```shell
pnpx husky install
```

![SvelteKit Accessibility Testing: Output from test:pa11y Script.  Shows 5 tests run each passing with no errors.](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pwsr7vyhibyse6e59n7t.png)

That&rsquo;s it. I hope it has all worked well for you.  Let me know if you had any issues or if anything needs more clarification.

## 🙌🏽 SvelteKit Accessibility Testing: Wrapping Up

In this post we have seen a nice way of adding automated a11y testing to your Svelte project.  In particular we have seen:

- how to configure Playwright with SvelteKit,
- a way to use pa11y with axe and htmlcs tests in Playwright,
- how to parameterise Playwright tests.

The full code for the app is available in the <a aria-label="Open the Svelte Kit Accessibility Testing code in the Rodney Lab Git Hub Repo" href="https://github.com/rodneylab/sveltekit-accessibility-testing">repo on Rodney Lab GitHub</a>.

I hope you found this article useful and am keen to hear how you plan to use the new tests in your Svelte projects.

## 🙏🏽 SvelteKit Accessibility Testing: Feedback

Have you found the post useful? Would you prefer to see posts on another topic instead? Get in touch with ideas for new posts. Also if you like my writing style, get in touch if I can write some posts for your company site on a consultancy basis. Read on to find ways to get in touch, further below. If you want to support posts similar to this one and can spare a few dollars, euros or pounds, please <a aria-label="Support Rodney Lab via Buy me a Coffee" href="/giving/">consider supporting me through Buy me a Coffee</a>.


Finally, feel free to share the post on your social media accounts for all your followers who will find it useful. As well as leaving a comment below, you can get in touch via <a aria-label="Reach out on Twitter" href="https://twitter.com/messages/compose?recipient_id=1323579817258831875">@askRodney</TwitterMessageLink> on Twitter and also <a aria-label="Contact Rodney Lab via Telegram" href="https://t.me/askRodney">askRodney on Telegram</a>. Also, see <a aria-label="Get in touch with Rodney Lab" href="/contact">further ways to get in touch with Rodney Lab</a>. I post regularly on <a aria-label="See posts on Astro" href="/tags/astro/">Astro</a> as well as <a aria-label="See posts on svelte kit" href="/tags/sveltekit/">SvelteKit</a>. Also <a aria-label="Subscribe to the Rodney Lab newsletter" href="/about/#newsletter">subscribe to the newsletter to keep up-to-date</a> with our latest projects.