Configuring Gitlab CI and Private Go Modules
Tags: go
Published: 2021-01-17 08:20:19

Last year I talked about [private repositories and Gitlab CI](https://mariocarrion.com/2019/01/04/go-docker-private-repos-gitlab-ci.html) as well as our story when we [migrated to modules](https://mariocarrion.com/2019/10/28/golang-migrating-to-go-modules.html), during that migration (and because of the way we organize our projects) we were _forced_ to start appending `.git` to our private modules and to use the `replace` directive in `go.mod` to explicitly indicate Go the exact location of that repository.

Something like:

```go
require (
	private.gitlab.instance/project/team/service-name v1.0.0
)

replace (
	private.gitlab.instance/project/team/service-name => private.gitlab.instance/project/team/service-name.git v1.0.0
)
```

That workflow was cumbersome, not idiomatic, and made upgrading our own internal dependencies harder than normal.

However after investigating even more, I realized there was a nicer alternative which involved using:

* [Gitlab's Personal Access Tokens](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html), and
* [`.netrc`](https://docs.gitlab.com/ee/user/project/new_ci_build_permissions_model.html#dependent-repositories)

This required 4 changes:

1. Using a Personal Access Token,
1. Updating `.gitlab-ci.yml` to properly use the new token,
1. Updating our `Dockerfile` to define the `.netrc` configuration, and
1. Remove `replace` directives in `go.mod`.

### Using a Personal Access Token

We specifically need a new token with the following scopes:

* `read_api`
* `repository_access`

For security concerns this token will be defined as a new [CI/CD environment variable](https://docs.gitlab.com/ee/ci/variables/), for that we will define two new variables:

* `GO_MODULES_USER` representing the Gitlab username and
* `GO_MODULES_PERSONAL_ACCESS_TOKEN` representing the token we just created.

The reason behind defining a new token instead of using `CI_JOB_TOKEN` is that this [token does not have enough permissions](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/29566) for accessing the Gitlab API used to determine the right repository when calling `go mod <xyz>` behind the scenes.

### Updating `.gitlab-ci.yml` to properly use the new token

Replacing the `docker build` instruction to pass in the new variables should do it, so from something like this:

```shell
docker build \
  --build-arg CI_JOB_TOKEN
```

We could change it to:

```shell
docker build \
  --build-arg GO_MODULES_USER
  --build-arg GO_MODULES_PERSONAL_ACCESS_TOKEN
```

### Updating our `Dockerfile` to define the `.netrc` configuration

Our Dockerfile will be changed to something like:

```dockerfile
FROM golang:1.15.0-alpine3.12

ARG GO_MODULES_USER
ARG GO_MODULES_PERSONAL_ACCESS_TOKEN

WORKDIR /project-name/

RUN go env -w GOPRIVATE="private.gitlab.instance" && \
    echo -e "machine private.gitlab.instance\nlogin ${GO_MODULES_USER}\npassword ${GO_MODULES_PERSONAL_ACCESS_TOKEN}" > ~/.netrc

COPY ["go.mod", "go.sum", "./"]
RUN go mod download

COPY . .

#-

FROM golang:1.15.0-alpine3.12

WORKDIR /project-name/

ENV PATH=/go/bin/:$PATH

COPY --from=0 /project-name/ /project-name/
COPY --from=0 /go/ /go/
```

Thanks to the multi-stage build we know for sure the credentials are not stored in the final Docker image being built 

### Remove `replace` directives in `go.mod`.

Finally the last step would be get rid of those `.git` replace directives:

```go
require (
	private.gitlab.instance/project/team/service-name v1.0.0
)
```

And with that we are back to the well known `go get/mod` workflow!

---

![The more you know](https://media.giphy.com/media/83QtfwKWdmSEo/giphy.gif "The more you know")