Integrate tests with Buildkite Test Analytics
Tags: cypress, jest, buildkite, testing
Published: 2022-06-26 11:17:57

Buildkite introduced a new feature called [Test Analytics](https://buildkite.com/test-analytics) 📈 this month and in my attempt to rank high on SEO on this subject, you're in the right article on how I did it (considering you already have access to Buildkite).

Just a bit of context (which no one cares), I just joined a scale-up where we started tracking _stuff_ and understandably, it includes the quality of our delivery lifecycle (or else, what's the point of my existence 😂). One of the metrics I personally wanted to check is how predictable and trust-worthy our tests are. The tendency is to ignore tests (especially on the Slack alerts where it becomes normal) when there is pressure to deliver and the amount of failing tests exceed the succeeding ones. That is just unacceptable for me as a Quality engineer and something that should be closely monitored.
***
**_"You can't improve what you don't measure."_** — Peter Drucker
***
For our `jest` frontned unit tests, there is a readily-available integrator (which makes life so easy!) where you just simply need to install (and obviously, include in your source code repo) the `npm` package called **_[buildkite-test-collector](https://www.npmjs.com/package/buildkite-test-collector)_** and include the reporter in your config file (e.g. `jest.config.js`):
```json
reporters: [
  'default',
  'buildkite-test-collector/jest/reporter'
],
testLocationInResults: true
```
![jest setup page](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/g586vx7r8z644vglxe4s.png)

For our `cypress` end-to-end tests, it is a bit trickier as it involves a couple more steps. You need to use the default **_[junit reporter](https://docs.cypress.io/guides/tooling/reporters)_** and include the reporter in your config file (e.g. `cypress.json` or `cypress.config.ts` for Cypress 10 and above):
```json
  "reporter": "junit",
  "reporterOptions": {
    "mochaFile": "results/junit-[hash].xml",
    "toConsole": true
  }
```
Once done, you need to go to your test or build pipeline (or wherever you run your tests) > Build steps (or `pipeline.yml`), and [import the JUnit XML file](https://buildkite.com/docs/test-analytics/importing-junit-xml). As lazy as I am (some call it smart 😎), I used the [Test collector plugin](https://github.com/buildkite-plugins/test-collector-buildkite-plugin) in this instance.
```yaml
steps:
  - label: '🧪 Run cypress tests'
    commands: 'npx cypress run --record --parallel --ci-build-id $BUILDKITE_BUILD_NUMBER'
    plugins: # pass details to Test Analytics
      - test-collector#v1.0.0:
          files: 'results/junit-*.xml'
          format: 'junit'
    artifact_paths:
      - 'results/**/*'
```
 
Once the configuration is done, include the `BUILDKITE_ANALYTICS_TOKEN` value on your pipeline's **Build Steps** or `pipeline.yml`
![Build step page](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3mwnlu6thtzc73x9uo4s.png)
 
Once the tokens are set, run your pipelines and then the magic happens! 🪄
![Test analytics page](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k3m2ba3pqgo0c21335lt.png)
 
_Note: Unsure why the test summary is not included in the main test suite page for the unit tests (maybe the Buildkite Gods can help us and I will update this write-up)._

Click a specific test suite and the "devil (if heaps of failing tests) is in the details" 👿
![Test suite details page](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/v2gw7gmttjewm2akhueq.png)

This is good alternative to [Cypress dashboards](https://dashboard.cypress.io/) too but free of charge if you're already subscribed to Buildkite.