Migrating a non-trivial Travis CI pipeline to GitHub Actions
Tags: github
Published: 2021-01-08 23:46:07

> Image source: https://unsplash.com/photos/Grg6bwZuBMs

Avid readers of my blog know that I’m really appreciating a proper CI / CD setup. Until a few weeks ago, my go-to solution for such setups has been [Travis CI](https://travis-ci.com). Unfortunately, Travis slightly changed its business model which drastically limits its use for open-source projects. For me, this was a huge setback, since I’m relying on Travis CI to provide [nut.js](https://github.com/nut-tree/nut.js "nut.js repo"). And since I’m not making any money out of this project, it’s hard to reason paying at least 69$ per month for CI. 
As I had already started using [GitHub Actions](https://github.com/features/actions "GitHub Actions page") in other repos, I decided to migrate [nut.js](https://github.com/nut-tree/nut.js "nut.js repo") to GitHub Actions as well.

## The existing pipeline

So, what do we have to migrate?

A [nut.js build](https://travis-ci.com/github/nut-tree/nut.js/builds/206388055 "nut.js Travis build") consists of two to three stages:

- `sonar`
- `test`
- `deploy` (only triggered on tags and on commits to the `develop` branch)

The `sonar` stage run on Linux with a fixed node version and executes **all** tests.
That is:

- all platform independent tests
- all platform dependent tests
- all tests requiring a GUI
- all tests running in a Docker container

Generated coverage reports are merged and reported to [sonarcloud](https://sonarcloud.io/dashboard?id=nut-tree%3Anut.js "nut.js sonarcloud page").

The following `test` stage is a build matrix, running tests on both macOS and Linux with varying node versions.
On Linux, the `test` stage runs the exact same set of tests as the `sonar` stage while on macOS tests running in Docker containers are left out.

The optional `deploy` stage will either publish a new [@next snapshot release](https://www.npmjs.com/package/@nut-tree/nut-js/v/1.5.1-next.20210107115106 "nut.js snapshot release page") for build on the `develop` branch, or a new [@latest release](https://www.npmjs.com/package/@nut-tree/nut-js/v/1.5.0 "nut.js latest release") for tags.

In summary, the requirements for a successful migration to GitHub Actions are the following:

- Build stages
- Build matrices
- Docker support
- Running GUI tests
- Sonar reports 

## Let’s migrate!

The final setup is split into three distinct files, one for general branch and pull request build, one for snapshot releases and one for stable releases.
Let’s look at branch build first.

### Sonar stage

Or workflow should run on pushes to all branches except `develop` and any release branch under `release/**` as well as pull requests.

```yaml
name: Run CI
on:
  push:
    branches-ignore:
      - develop
      - release/**
  pull_request:
```

Travis stages are resembled by [jobs](https://docs.github.com/en/free-pro-team@latest/actions/reference/workflow-syntax-for-github-actions#jobs "GitHub Actions job docs") on GitHub Actions.

```yaml
jobs:
  sonar:
    runs-on: ubuntu-latest
    steps:
      - name: Set up Git repository
        uses: actions/checkout@v2
      - name: Set up node
        uses: actions/setup-node@v2
        with:
          node-version: 14
```

Our job runs on Linux (`ubuntu-latest` defaults Ubuntu 18.04, soon upgrading to Ubuntu 20.04), checks out our repo and installs node v14.x.x.
On Linux, Docker is available right out of the box, so we’re able to start a daemonized container:

```yaml
- name: Setup Docker
  run: |
    docker pull s1hofmann/nut-ci:latest
    docker run -it -d --name nut-ci --shm-size 4gb --user $(id -u):$(id -g) -v ${PWD}:${PWD}:rw s1hofmann/nut-ci:latest bash
```

What follows are some steps to initialise the project:

```yaml
- name: Install
  run: npm ci
- name: Compile
  run: npm run compile
- name: Init e2e test subpackage
  run: npm --prefix e2e/tests ci
- name: Clean coverage report
  run: npm run coverage:clean
```

Once set up, we now want to run tests which require a GUI. 
A common way of running headless GUI tests on Linux is using a [virtual framebuffer](https://www.x.org/releases/X11R7.6/doc/man/man1/Xvfb.1.xhtml "xvfb page").
Travis provides a simple to use [service](https://docs.travis-ci.com/user/gui-and-headless-browsers/#using-xvfb-to-run-tests-that-require-a-gui "Travis xvfb service") to enable a virtual framebuffer in a build, so let’s find a proper replacement for GitHub Actions.
[GabrielBB/xvfb-action](https://github.com/marketplace/actions/gabrielbb-xvfb-action "xvfb action page") provides a nice wrapper which will start a virtual framebuffer where possible. 
When running on Linux, a virtual framebuffer will be set up before running your action, while on other platforms it will be executed right away.
No additional configuration required when running in a cross-platform setup (like we do in our matrix build)!

Following the GUI and unit tests, let’s execute E2E tests running in a Docker container and merge all coverage reports:

```yaml
- name: Run Docker E2E tests
  run: docker exec nut-ci bash -c "bash $PWD/.build/build.sh ${PWD} 14"
- name: Merge coverage reports
  run: |
    npm run coverage:merge
    npm run coverage:merge-report
```

Travis provides an add-on for [sonarcloud](https://docs.travis-ci.com/user/sonarcloud/#inspecting-code-with-the-sonarqube-scanner "Travis sonarcloud add-on") to easily run sonarcloud analyses.
Luckily, there’s an official [SonarCloud](https://github.com/marketplace/actions/sonarcloud-scan "SonarCloud GitHub Action") Action to run scans on GitHub Actions.
Running `sonar-scanner` with an existing `sonar-project.properties` config becomes as easy as

```yaml
- name: Send results to SonarCloud
  uses: SonarSource/sonarcloud-github-action@v1.4
  env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
    SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }} 
```

Instead of storing the encrypted Sonar token in our workflow file like on Travis, we’re storing and referencing it as a [repo secret](https://docs.github.com/en/free-pro-team@latest/actions/reference/encrypted-secrets "GitHub repo secrets docs") and we’re good to go!

### Test stage

Following our `sonar` stage we want to run a build matrix with multiple platforms and node versions.
By default, jobs run in parallel, but since we only want to run our `test` job in case the previous `sonar` stage has been successful, we’re setting a requirement to the previous job:

```yaml
test:
  needs:
    - sonar
```

Now let’s configure the build matrix:

```yaml
strategy:
  matrix:
    os: [ ubuntu-latest, windows-latest, macos-latest ]
    node: [ 10, 12, 14 ]
    exclude:
      - os: ubuntu-latest
        node: 14
    runs-on: ${{matrix.os}}
    steps:
      - name: Set up Git repository
        uses: actions/checkout@v2
      - name: Set up node
        uses: actions/setup-node@v2
        with:
          node-version: ${{matrix.node}}
```

We want to run our tests on Linux, Windows and macOS using node version 10, 12 and 14.
But since our `sonar` stage already ran on Linux using node 14, we’re excluding it from our build matrix to save time and compute.
In total, our build matrix will expand into 8 separate builds, all running in parallel.

We also want to limit our Docker tests to Linux, so we can limit the whole Docker setup to build running on `ubuntu-latest`:

```yaml
- name: Setup Docker
  if: ${{matrix.os == 'ubuntu-latest'}}
  run: |
    docker pull s1hofmann/nut-ci:latest
    docker run -it -d --name nut-ci --shm-size 4gb --user $(id -u):$(id -g) -v ${PWD}:${PWD}:rw s1hofmann/nut-ci:latest bash

    ...

- name: Run Docker E2E tests
  if: ${{matrix.os == 'ubuntu-latest'}}
  run: docker exec nut-ci bash -c "bash $PWD/.build/build.sh ${PWD} ${{matrix.node}}"
```

### Deploy stage

Snapshot releases should only happen on commits to `develop`:

```yaml
name: Create snapshot release
on:
  push:
    branches:
      - develop
```

while latest releases are limited to semver tags:

```yaml
name: Create tagged release
on:
  push:
    tags:
      - v*.*.*
```

When releasing new packages we do not require a sonar scan, so we just skip this stage completely and only run our `test` stage.
Depending on the outcome of this stage, a package release is triggered:

```yaml
deploy:
  needs:
    - test
  runs-on: ubuntu-latest
  steps:
    - name: Set up Git repository
      uses: actions/checkout@v2
    - name: Set up node
      uses: actions/setup-node@v2
      with:
        node-version: 14
    - name: Publish tagged release
      uses: JS-DevTools/npm-publish@v1
      with:
        token: ${{ secrets.NPM_TOKEN }}
```

## To be resolved?

There’s quite a lot of duplication in our [workflows](https://github.com/nut-tree/nut.js/tree/develop/.github/workflows "nut.js workflows") which I’d like to reduce. Something similar to how [GitLab CI](https://docs.gitlab.com/ee/ci/yaml/includes.html "GitLab CI include docs") handles it would be nice, but it looks like it has yet to be supported by [GitHub Actions](https://github.com/actions/starter-workflows/issues/245).

Another thing which is not that great is missing support to skip builds via commit message.
It’s a common pattern to skip CI builds via commit message by prefixing the message with e.g. `[skip_travis]` or `[skip_ci]`.
Unfortunately, GitHub Actions do not yet support such kind of shortcut and will still run your workflows.

## Conclusion

Migrating the existing Travis CI pipeline to GitHub Actions turned out easier than initially expected.
Due to the wide range of community actions it was fairly easy to tick all the required boxes.

As a result we’re now able to run all our builds on a single CI system (Windows builds were carried out on [AppVeyor](https://www.appveyor.com "AppVeyor page") since [Windows build are in an early stage on Travis](https://docs.travis-ci.com/user/reference/windows) while the overall build runtime decreased!