Build and release go binaries for Mac and Linux  in GitHub Actions using 2 approaches
Tags: actionshackathon21, github, opensource
Published: 2021-12-07 14:30:22

[Instructions]: # (To submit to the GitHub Actions x DEV Hackathon 2021, please fill out all sections.)

Ok, let's get started. We choose one of our popular repositories, `awsu` _(Enhanced account switching for AWS, supports Yubikey as MFA source)_

{% github https://github.com/kreuzwerker/awsu %}

This tool is written in Golang and still used [travis-ci](https://travis-ci.org/) as CI. Furthermore, some parts of the release process were still manually, such as uploading the assets to a GitHub release and generating the release notes. We wanted to have this automated.

We started by defining our goals for the [hacktoberfest](https://hacktoberfest.digitalocean.com/) session:

- use the latest version of `Golang` for building, which is `1.17` when writing this post. This state implied
bumping the version of `Golang` used internally
and also removing deprecated package manager, such as `godep` and/or `govendor`.
- move from `travis` to `GitHub actions`
- provide a suitable config for [goreleaser](https://goreleaser.com/) and use it for release. Fortunately, there is already a [GitHub action](https://github.com/goreleaser/goreleaser-action) for this tool 🚀
- generate a [homebrew](http://brew.sh) formula pointing to a new release and
- update our [homebrew-taps](https://github.com/kreuzwerker/homebrew-taps) automatically 🍺 🤖

### My Workflow
The most tricky part was that libraries needed to be installed on the system `Linux` system. We realized that beforehand the binaries were built on a Mac (with an Intel chip) before, one native and the other in a docker environment with a `debian:stretch` docker image. As GitHub Actions runs on ubuntu runner, we decided to reproduce the build locally in this environment. This enabled us to go with our first iteration. 

#### Iteration 1 (using docker)
As GitHub Actions allows us to use `macos-latest` runners, install Docker automatically as it does not exist as for example, for the `ubuntu-latest` or `windows-latest` runners. Therefore we created PoC repository [docker-gh-action-test](https://github.com/mavogel/docker-gh-action-test) to verify this and provide a stable installation for `macos`. 

You can see all the changes in this pull request:
{% github https://github.com/kreuzwerker/awsu/pull/57 %}

We were happy as it worked as expected. However, our build time was with `11m9s` very long. So we explored other options are ended with trying the [artifacts](https://docs.github.com/en/actions/advanced-guides/storing-workflow-data-as-artifacts) feature, which we will talk about 
in the second iteration coming now.

#### Iteration 2 (using matrix builds and artifacts)
We started implementing the solution using 2 stages
1. a build stage with a matrix on `ubuntu` and `macos`, where we install the needed dependencies, pre-build the binary and then upload it to the artifacts store.
2. a release stage where we download the artifacts and release them with the prebuilt feature of [goreleaser-pro](https://goreleaser.com/pro/)

You can see all the changes in this pull request:
{% github https://github.com/kreuzwerker/awsu/pull/58 %} and a fix an upcoming pull request {% github https://github.com/kreuzwerker/awsu/pull/60 %}

This reduced the pipeline runtime from `11m9s` down to `1m15s` 🎉

![Release pipeline runtime](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fjse8henk6ub4an7t1ng.png)

### Submission Category: 

Maintainer Must-Haves


### Yaml File or Link to Code

As we described in the previous approach and the logic, here are the concrete implementations of the Actions

#### Iteration 1 (using docker)

```yaml
name: goreleaser

on:
  push:
    tags:
      - '*'

jobs:
  binaries:
    runs-on: macos-11
    steps:
      - name: Checkout
        uses: actions/checkout@v2
        with:
          fetch-depth: 0
      - name: Set up Go
        uses: actions/setup-go@v2
        with:
          go-version: 1.17
      - name: Updating and upgrading brew
        run: |
          ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
          brew --version
      - name: Install and start docker
        # https://github.com/docker/for-mac/issues/2359#issuecomment-943131345
        run: |
          brew install --cask docker
          sudo /Applications/Docker.app/Contents/MacOS/Docker --unattended --install-privileged-components
          open -a /Applications/Docker.app --args --unattended --accept-license
          echo "Waiting for docker to be up"
          while ! /Applications/Docker.app/Contents/Resources/bin/docker info &>/dev/null; do echo -n "."; sleep 1; done
      - name: Build linux binary
        run: |
          make build/awsu-linux-amd64
          # as it is the format goreleaser expects
          cp build/awsu-linux-amd64 build/awsu_linux_amd64
      - name: Run GoReleaser
        uses: goreleaser/goreleaser-action@v2
        with:
          distribution: goreleaser-pro
          version: latest
          args: release --rm-dist
        env:
          GORELEASER_KEY: ${{ secrets.GORELEASER_KEY }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          GORELEASER_TOKEN: ${{ secrets.GORELEASER_TOKEN }}
```

#### Iteration 2 (using matrix builds and artifacts)

```yaml
name: goreleaser

on:
  push:
    tags:
      - '*'

jobs:
  artifact-build:
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        os: [ubuntu-latest, macos-latest]
    steps:
      - name: Checkout
        uses: actions/checkout@v2
        with:
          fetch-depth: 0
      - name: Set up Go
        uses: actions/setup-go@v2
        with:
          go-version: 1.17
      - name: Setup dependencies
        if: matrix.os == 'ubuntu-latest'
        run: |
          sudo apt-get update -q
          sudo apt-get install -qqy build-essential software-properties-common pkg-config wget libpcsclite-dev
      - name: Build darwin
        if: matrix.os == 'macos-latest'
        run: |
          make build/awsu-darwin-amd64
          mv build/awsu-darwin-amd64 build/awsu-macos-latest-amd64
      - name: Build linux
        if: matrix.os == 'ubuntu-latest'
        run: |
          make build/awsu-linux-amd64
          mv build/awsu-linux-amd64 build/awsu-ubuntu-latest-amd64
      - uses: actions/upload-artifact@v2
        with:
          name: awsu-${{ matrix.os }}-amd64
          path: build/awsu-${{ matrix.os }}-amd64
      
  release-test:
    runs-on: ubuntu-latest
    needs: [artifact-build]
    steps:
      - name: Checkout
        uses: actions/checkout@v2
        with:
          fetch-depth: 0
      - name: Download macos
        uses: actions/download-artifact@v2
        with:
          name: awsu-macos-latest-amd64
          path: build
      - name: Download linux
        uses: actions/download-artifact@v2
        with:
          name: awsu-ubuntu-latest-amd64
          path: build
      - name: Correct goreleaser prebuilt path
        run: |
          # as it is the format goreleaser expects. See .goreleaser.yml -> prebuilt -> path
          mv build/awsu-ubuntu-latest-amd64 build/awsu_linux_amd64
          mv build/awsu-macos-latest-amd64 build/awsu_darwin_amd64
          ls -lash build
      - name: Run GoReleaser
        uses: goreleaser/goreleaser-action@v2
        with:
          distribution: goreleaser-pro
          version: latest
          args: release --rm-dist
        env:
          GORELEASER_KEY: ${{ secrets.GORELEASER_KEY }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          GORELEASER_TOKEN: ${{ secrets.GORELEASER_TOKEN }}
```

We hope this will help you folks our there 🎉 Happy to hear more improvements and suggestions or any questions to help clarify the internal logic.

### Additional Resources / Info

- The PoC repository for running `docker` in Github Actions: {% github https://github.com/mavogel/docker-gh-action-test %}
- Our GitHub organization [kreuzwerker](https://github.com/kreuzwerker)
- The actions used [upload-artifact](https://github.com/actions/upload-artifact), [download-artifact](https://github.com/actions/download-artifact),  [goreleaser-action](goreleaser/goreleaser-action@v2)
- This post is a successor of our internal [hacktoberfest](https://kreuzwerker.de/en/post/hacktoberfest-at-kreuzwerker) at [kreuzwerker](https://kreuzwerker.de)