Generating Code Coverage from the Continuous Integration Process
Tags: coverage, code, ci, circleci
Published: 2020-03-09 19:43:53

---
title: Generating Code Coverage from the Continuous Integration Process
published: true
description:
cover_image: https://images.unsplash.com/photo-1508317469940-e3de49ba902e
tags: coverage,code,continuous_integration, CircleCI
series: [Continuous Integration]
---

In the [previous article](https://dev.to/ivandotv/beginners-guide-to-nodejs-continuous-integration-nge) we started working with the CircleCI platform, by setting up testing infrastructure for our code.

In this article, we are going to set up code coverage for the project.

## The Plan

- Instruct test runner (`jest`) to generate and save **code coverage** files for our tests.
- Upload those reports to external service for aggregation and display. For that, we are going to use [codecov.io](https://codecov.io/) for code coverage.

First, you will need to go to codecov.io to create an account and chose the repository you want to show code coverage for. Then you need to install [codecov.io application](https://github.com/marketplace/codecov) from the github marketplace.

As a reminder, this is the CircleCI configuration file from the previous article.

```yaml
version: 2.1
jobs:
  node-v10:
    docker:
      - image: circleci/node:10
    steps:
      - test
  node-v12:
    docker:
      - image: circleci/node:12
    steps:
      - test
  node-v13:
    docker:
      - image: circleci/node:13
    steps:
      - test
commands:
  test:
    steps:
      - checkout
      - restore_cache:
          key: dependency-cache-{{ checksum "package-lock.json" }}
      - run:
          name: install-dependancies
          command: npm ci
      - save_cache:
          key: dependency-cache-{{ checksum "package-lock.json" }}
          paths:
            - $HOME/.npm
      - run:
          name: unit test
          command: |
            npm run ci:test
workflows:
  version: 2
  build_and_test:
    jobs:
      - node-v10
      - node-v12
      - node-v13
```

We are going to continue to build upon this file by adding code coverage functionality.

## How is Code Coverage Generated

Since we are using [Jest](https://jestjs.io/) for testing the code, there is not much that we need to do to set up code coverage, because jest comes with code coverage out of the box.

Next, we need to modify `ci:test` npm script inside the `package.json`

Our previous `npm` script for running `jest` tests was:

```json
"ci:test": "jest --runInBand --ci"
```

We are going to change the script slightly so we can instruct jest to also generate code coverage files and save them in the `coverage` directory in the root of the project.

```json
"ci:test": "jest --runInBand --ci  --reporters=default  --coverage --coverageDirectory=coverage"
```

## Setting up Code Coverage on the CircleCI platform

Setting up code coverage is fairly easy. After `jest` tests the code and generates code coverage files we need to save them somewhere and then upload them to the codecov.io
In the [previous article](https://dev.to/ivandotv/beginners-guide-to-nodejs-continuous-integration-nge) we talked about how code is running inside Linux containers and how they are ephemeral and are discarded as soon as the `jobs` that are running inside them are done, so we are going to need to take the files that are generated by jest and save them somewhere outside of the containers before the containers are destroyed.

Then, after we save the files somewhere outside the containers, the only step left is to upload those files to the [codecov.io](https://codecov.io/) service.

To use the [codecov.io](https://codecov.io/) service via the circleCI platform, we are going to use the official codecov.io "**_orb_**".

## CircleCI Orbs Concept Explained

CircleCI platform has a concept of `orbs`,
from the [docs](https://circleci.com/docs/2.0/configuration-reference/#orbs-requires-version-21):

> Orbs are reusable packages of CircleCI configuration that you may share across projects, enabling you to create encapsulated, parameterized commands, jobs, and executors that can be used across multiple projects.

You can consider them as **classes** in object oriented programming. A blueprint for doing certain tasks. Orbs simplify tasks by hiding all necessary steps that need to be done and only exposing a few methods to get the tasks going.

You can take a [look](https://github.com/codecov/codecov-circleci-orb/blob/master/orb.yml) at the actual code for the `codecov orb` to see how much typing using the `orbs` saves us. Also when using officially supported orbs you can be sure that your CircleCI jobs will work as expected when integrating with the external services. Many online services that integrate with CircleCI have their own `orbs` that make it easy to integrate with the continuous integration process.

To use any _orb_ we need to declare it under the `orbs` key, so let's add the `orbs` key to the `config.yml` file.

```yml
version: 2.1
orbs:
  codecov: codecov/codecov@1.0.5
jobs:
```

Next, we are going to add additional steps to be executed after the code is successfully tested.

```yml
jobs:
  # ... node-v10 job
  node-v12:
  docker:
    - image: circleci/node:12
  steps:
    - test # test the code
    - store-coverage-data # store coverage files outside of the container command
    - upload-coverage # upload files to the codecov.io command
  # ... node-v13 job
```

You might notice that in the previous example I have omitted the code for the `node-v10` and `node-v13` jobs. The reason for this is that I have decided to only use code coverage from one job/container (`node-v12`). Coverage data from the `node-v10` and `node-v13` jobs will be ignored, Jest will create code coverage but it will not be used by the codecov.io orb (we could further optimize those jobs to not generate code coverage at all but that is beyond the scope of this article).

## Code Coverage Commands

Now let's take a look at the commands that are used in the steps above.

```yml
commands:
  store-coverage-data:
    steps:
      - store_artifacts:
          path: coverage
  upload-coverage:
    steps:
      - codecov/upload:
          file: coverage/coverage-final.json
```

- step: `store-coverage-data`

  - `store_artifacts` - This is a special built-in command in the CircleCI platform that can save anything inside the container to the CircleCI permanent storage (move it outside of the container). We are using it to save our coverage files. From the [docs:](https://circleci.com/docs/2.0/artifacts/#artifacts-overview)

    > Artifacts persist data after a job is completed and may be used for longer-term storage of the outputs of your build process. For example, when a Java build/test process finishes, the output of the process is saved as a .jar file. CircleCI can store this file as an artifact, keeping it available long after the process has finished.

  - `path` This is the path inside the container that we want to be saved externally. Please note that we are using a relative path to the `coverage` directory inside our working directory (root directory of our checked out code)

- step: `upload-coverage`
  - `codecov/upload`: This is a custom command (`upload`) defined via `codecov` orb. As you can see you can treat it as a class instance method `codecov->upload`
    This command uploads required files for the code coverage service.
    - `file` path to the file we want to upload.

## Using Jest-Junit for Test Results

There is one additional step that we can add when setting up code coverage. We can store test performance results for later review.
Inside the circleCI UI, there can be an additional tab that can show you which tests are failing and which tests took a long time to run.
To enable this functionality, we need to install additional node module [jest-junit](https://www.npmjs.com/package/jest-junit) and configure it via `package.json`

```json
"jest-junit": {
  "outputDirectory": "./reports/junit",
  "outputName": "test-results.xml"
}
```

once again, we need to modify `ci:test` npm script inside the `package.json`

```json
"ci:test": "jest --runInBand --ci  --reporters=default --reporters=jest-junit --coverage --coverageDirectory=coverage"
```

Next we need to modify circleCI configuration file and add another `step` to the `test` command: `store_test_results`
From the [docs](https://circleci.com/docs/2.0/configuration-reference/#store_test_results)

> `store_test_results` - Special step used to upload and store test results for a build. Test results are visible on the CircleCI web application, under each build’s “Test Summary” section. Storing test results is useful for timing analysis of your test suites.

modified configuration file:

```yml
commands:
  test:
    steps:
      - checkout
      - restore_cache:
          key: dependency-cache-{{ checksum "package-lock.json" }}
      - run:
          name: install-dependancies
          command: npm ci
      - save_cache:
          key: dependency-cache-{{ checksum "package-lock.json" }}
          paths:
            - $HOME/.npm
      - run:
          name: unit test
          command: |
            mkdir -p ./reports/junit
            npm run ci:test
      - store_test_results: #<--- store test results for CircleCI
          path: ./reports/junit/
```

And that's it, that is all it takes to set up code coverage via the CircleCI platform.

Complete config file:

```yml
version: 2.1
orbs:
  codecov: codecov/codecov@1.0.5
jobs:
  node-v10:
    docker:
      - image: circleci/node:10
    steps:
      - test
  node-v12:
    docker:
      - image: circleci/node:12
    steps:
      - test
      - store-coverage-data
      - upload-coverage
  node-v13:
    docker:
      - image: circleci/node:13
    steps:
      - test
commands:
  store-coverage-data:
    steps:
      - store_artifacts:
          path: coverage
  test:
    steps:
      - checkout
      - restore_cache:
          key: dependency-cache-{{ checksum "package-lock.json" }}
      - run:
          name: install-dependancies
          command: npm ci
      - save_cache:
          key: dependency-cache-{{ checksum "package-lock.json" }}
          paths:
            - $HOME/.npm
      - run:
          name: unit test
          command: |
            npm run ci:test
      - store_test_results:
          path: ./reports/junit/
  upload-coverage:
    steps:
      - codecov/upload:
          file: coverage/coverage-final.json
workflows:
  version: 2
  build_and_test:
    jobs:
      - node-v10
      - node-v12
      - node-v13
```

### Bonus: Configuring Jest to Fail

You can also set up Jest to fail the test if code coverage stats **are below** a certain threshold. You can do it on a global level or per directory/file.

This goes into your `package.json` file.

```json
"jest":{
  "coverageThreshold": {
    "global": {
      "branches": 50,
      "functions": 50,
      "lines": 50,
      "statements": 50
    },
    "./src/components/": {
      "branches": 40,
      "statements": 40
    },
  }
}
```

You can read more about jest.io coverage threshold [here](https://jestjs.io/docs/en/configuration#coveragethreshold-object)

To use code coverage with private repositories, you will need to set up a `$CODECOV_TOKEN` environment token via [CircleCI dashboard](https://circleci.com/docs/2.0/env-vars/#setting-an-environment-variable-in-a-project)
You can obtain the token when you connect your repository with codecov.io via their [website](https://codecov.io/).