Beginners Guide to Node.js Continuous Integration
Tags: jest, tests, circleci, javascript
Published: 2020-02-03 22:56:09

---
title: Beginners Guide to Node.js Continuous Integration
published: true
description: Set up continuous integration and deployment for nodejs applications on CircleCI platform
tags: jest,tests,CircleCI,javascript
series: [Continuous Integration]
cover_image: https://images.unsplash.com/photo-1488590528505-98d2b5aba04b?ixlib=rb-1.2.1&ixid=eyJhcHBfaWQiOjEyMDd9&auto=format&fit=crop&w=1000&q=80
---

In this series of articles, we will set up continuous integration and deployment for nodejs applications on [CircleCI platform](circleci.com).

## The Workflow

Every time we push code to the repository CircleCI will be notified of the new code, and it will automatically kick off a continuous integration process that will run our unit tests (via Jest) in node `v10`, `v12`, and `v13`.

First, you will need to create an account at [circleci.com](https://circleci.com/) and follow the instruction to choose the repository you want to use.

Second, you will need to install [CircleCI application](https://github.com/marketplace/circleci) from the github marketplace. With the application installed CircleCi will integrate with the chosen repository, and make it easier to see what is going on with the tests. Any time you commit to a branch or create a pull request circleCI will run the tests.

![example of circleCI UI integration with github repository](https://raw.githubusercontent.com/ivandotv/blog-posts/master/posts/circleci-integration/assets/all-checks-passed.png)

Next, we need to create a circleCI configuration file.

## CircleCI configuration file

For the circleCI platform to recognize the repository as ready for the integration, we need to have a special configuration file present in the repository. The default location for the file inside the repository is `.circleci/config.yml`

This file contains the instructions for the CircleCI platform on how to run tests on the repository.

```yaml
version: 2.1
jobs:
  node-v10:
    docker:
      - image: circleci/node:10
    steps:
      - test
  node-v12:
    docker:
      - image: circleci/node:12
    steps:
      - test
  node-v13:
    docker:
      - image: circleci/node:13
    steps:
      - test
commands:
  test:
    steps:
      - checkout
      - restore_cache:
          key: dependency-cache-{{ checksum "package-lock.json" }}
      - run:
          name: install-dependancies
          command: npm ci
      - save_cache:
          key: dependency-cache-{{ checksum "package-lock.json" }}
          paths:
            - $HOME/.npm
      - run:
          name: unit test
          command: |
            npm run ci:test
workflows:
  version: 2
  build_and_test:
    jobs:
      - node-v10
      - node-v12
      - node-v13
```

## Configuration File explained

The file is divided into three sections: `jobs`, `commands` and `workflows`. First, we will concentrate on the `jobs` key

### Jobs

**Jobs** are collections of `steps` that are executed inside the container.

```yaml
jobs:
  node-v10:
    docker:
      - image: circleci/node:10
    steps:
      - test
  node-v12:
    docker:
      - image: circleci/node:12
    steps:
      - test
  node-v13:
    docker:
      - image: circleci/node:13
    steps:
      - test
```

In the above code excerpt, we have defined three jobs and named them `node-v10` and and `node-v12` (names are arbitrary).

Next, we have a `docker` key which we can use to pass various options for the customization of the Linux container that is going to be created (we are using Linux containers for testing our code, but circleCI can also spin up complete virtual machines: Linux, Windows, and MacOS).

`image` option defines what container image we are going to use. In this case, we are using the default CircleCI images for different node runtime versions.
You are free to use other images but the default CircleCI images are sufficient in most cases and they come with some essential tools included (git, ssh, tar, ca-certificates, curl, wget)
You can see exactly what is included on the [circleCI docker hub page](https://hub.docker.com/r/circleci/node/).

### Steps

**Steps** ( when inside a **Job** ) are executable commands that are executed during a job.

In this case, we have only one step: `test` (the name is arbitrary)

This step is a `command` which contains more steps. The reason we have this step as a `command` is because of the code reuse. As you can see all jobs (node-v10, node-v12, node-v13) are using the same command `test`, if we were to have only one job we could just take all the steps and put them under the `job` key.

like this:

```yaml
jobs:
  node-v12:
    docker:
      - image: circleci/node:12
    steps:
      - checkout
      - restore_cache:
          key: dependency-cache-{{ checksum "package-lock.json" }}
      - run:
          name: install-dependancies
          command: npm ci
      - save_cache:
          key: dependency-cache-{{ checksum "package-lock.json" }}
          paths:
            - $HOME/.npm
      - run:
          name: unit test
          command: |
            npm run ci:test
```

### Test Command

**Command** is a sequence of instructions to be executed in a job. The main purpose of commands is to enable the reuse of a single command definition across multiple jobs ( e.q. `test` command in all jobs v10,v12,v13). Notice also that commands have their own steps.

The name of the command is completely arbitrary. The purpose of this command is to:

1. check out the code from the repository.
2. try to use cached `node modules` so it doesn't need to download the dependencies again ([more on that later](#saving-and-restoring-npm-cache)).
3. install the dependencies.
4. save the installed dependencies in a cache.
5. run the tests.

Steps are run in order, from top to bottom.

- _checkout_ (step 1) This is a special step built into the CircleCI platform which is used to check out the source code from the repository ( [link to docs](https://circleci.com/docs/2.0/configuration-reference/#checkout) ).

- _restore\_cache_ (step 2) another built in step that is used to "restore a previously saved cache" ([more on that later](#saving-and-restoring-npm-cache)) [official documentation](https://circleci.com/docs/2.0/configuration-reference/#restore_cache)

  - _key_ a string under which to look for the cache (we could have many more different caches under different keys)

- _run_ (step 3) used for invoking command-line programs. You can invoke any command that is available inside the container. ( [link to docs](https://circleci.com/docs/2.0/configuration-reference/#run) )

  - _name_ - used in the CircleCI UI to easily differentiate from other steps/commands
  - _command_ - the actual command that is going to run (`npm ci`)

- _save\_cache_ (step 4) Generates and stores a cache of a file or directory of files such as dependencies or source code in CircleCI object storage ( [link to docs](https://circleci.com/docs/2.0/configuration-reference/#save_cache) )

- _run_ (step 5) used for invoking command line programs ( [link to docs](https://circleci.com/docs/2.0/configuration-reference/#run) )

  - _name_ - used in the circleCI UI to easily differentiate from other steps/commands
  - _command_ - the actual command that is going to run (`npm ci:test`)[more on that later](#running-the-actual-tests)

### Workflows

**Workflows** are collections of jobs that are executed on every code push.

```yaml
workflows:
  version: 2
  build_and_test:
    jobs:
      - node-v10
      - node-v12
      - node-v13
```

Workflows key determines which `workflows` (which consist of jobs) are going to run and in what order.
Here we are declaring one workflow `build_and_test` with three jobs `node-v10`, `node-v12` and `node-v13`.
These jobs are going to run in parallel ( they can also run in sequence or conditionally )
Since we only have one workflow as soon as the new code push to the repository is detected, CircleCI will launch three Linux containers (docker images with different nodejs versions) and run the tests and report back.

![example of circleCI ui integration with github repository](https://raw.githubusercontent.com/ivandotv/blog-posts/master/posts/circleci-integration/assets/all-checks-passed.png)

### Saving and restoring NPM cache

Installing nodejs dependencies from scratch can be a time-consuming process so to speed up the installation process we are going to use one of the recommended caching strategies when working with node modules.

#### saving the cache

```yaml
- save_cache:
    key: dependency-cache-{{ checksum "package-lock.json" }}
    paths:
      - $HOME/.npm
- run:
    name: unit test
    command: |
      npm run ci:test
```

We are saving the contents of `.npm` directory for later use. This is the directory that is storing global npm cache ( not global modules). This directory is located in the users home directory.

To reliably validate and invalidate the cache we need to know when the cache becomes invalid. To do that we are saving the cache under a key that is going to be different every time the `package-lock.json` file is changed.
So this line of code `{{ checksum "package-lock.json" }}` generates a unique string based on the contents of the `package-lock.json`
So our key is going to look something like this: `dependency-cache-4260817695`

#### restoring the cache

```yaml
- restore_cache:
    key: dependency-cache-{{ checksum "package-lock.json" }}
```

In the above step, we are trying to restore the cached version of the `.npm` directory.
Same as in the **saving the cache** step we are going to calculate the cache key via the `package-lock.json` file, and if the file hasn't been changed we are going to get the same key (`dependency-cache-4260817695`). That means that dependencies are the same and we can leverage the cache.

### Running the actual tests

```yaml
- run:
    name: unit test
    command: |
      npm run ci:test
```

As you can see in the above example we are running the `ci:test` task from the `package.json` file. It is always a good option to have different tasks for running the tests locally and in continuous integration.

`ci:test` is an npm script that is created for the sole purpose of testing the code in a continuous integration environment.

```json
"ci:test": "jest --runInBand --ci"
```

As you can see, we are using [jest](https://jestjs.io/) testing framework.

- `--runInBand`

  [from the docs](https://jestjs.io/docs/en/cli#--runinband):

  > Run all tests serially in the current process, rather than creating a worker pool of child processes that run tests.

  By default, Jest detects how many cores your CPU has, and automatically spreads the tests over all cores. This can be a problem when running in docker or virtual machine because sometimes Jest will not get the correct number when querying for number of cores and you will get "out of memory error"

  Note that you can also use another option to limit the number of workers `--maxWorkers=<num>` This can yield faster tests, but you need to know exactly how many CPU cores you have in your testing environment.

In the case of CircleCI free accounts get their [`medium`](https://circleci.com/pricing/) tier with [2vCPUs and 4GB of RAM](https://circleci.com/docs/2.0/configuration-reference/#resource_class).

- `--ci`
  [from the docs](https://jestjs.io/docs/en/cli#--ci):

  > When this option is provided, Jest will assume it is running in a CI environment. This changes the behavior when a new snapshot is encountered. Instead of the regular behavior of storing a new snapshot automatically, it will fail the test and require Jest to be run with --updateSnapshot.

So if you don't use snapshots in your testing, this option shouldn't concern you.

Also note that by default jest is looking for `.js`, `.jsx`, `.ts` and `.tsx` files inside of `__tests__` folders, as well as any files with a suffix of `.test` or `.spec` (e.g. `Component.test.js` or `Component.spec.js`). You can read about it [here](https://jestjs.io/docs/en/configuration#testregex-string--arraystring).

### About the `npm ci` command

If you are wondering why we are `npm ci` instead of `npm install` here is the reason straight [from the npm docs.](https://docs.npmjs.com/cli/ci.html)

> This command is similar to npm-install, except it’s meant to be used in automated environments such as test platforms, continuous integration, and deployment – or any situation where you want to make sure you’re doing a clean install of your dependencies. It can be significantly faster than a regular npm install by skipping certain user-oriented features. It is also more strict than a regular install, which can help catch errors or inconsistencies caused by the incrementally-installed local environments of most npm users.

### Conclusion

That's it. Now every time you push code from your local computer to the `origin` CircleCi will get notified and it will run your tests.

This was a basic setup of continuous integration and testing.

In the next article, we will add code coverage reports to the process.

### Addendum

CircleCI has a `cli` program that you can [install locally](https://circleci.com/docs/2.0/local-cli/) on your computer to automate or even run some tasks locally. One of the best features is that you can use it [validate your `circleci.yml` configuration file](https://circleci.com/docs/2.0/local-cli/#validate-a-circleci-config). Validating the configuration file locally can save you a lot of time, especially when working with more complex workflows.