Go modules and private GIT repositories
Tags: go, github
Published: 2021-06-26 18:00:34

Some time ago i stumbled upon an issue that was giving me a headache. I had to download and import private Go module, and as it turns out, that can be little tricky. I'm using SSH keys for authenticating with my repositories, so this solution is applied only for that type of authentication.
Of course, i scouted internet looking for solutions, and there are some examples you can find, but none was working for me. Until i realized that there is small but crucial difference depending on which Git repository hosting service you are using. I will show you how to do it for GitHub and Bitbucket. Of course, all examples here are using URLs to my private repositories, and you will have to change them to match yours.

I prepared two simple private repositories for this little demo. Both are essentially the same. They have one package named `hello` with function `World()` returning string `Hello World!`. They are located on:

* GitHub - github.com/matijakrajnik/goprivatemodule
* Bitbucket - bitbucket.org/matijakrajnik/goprivatebucket

Now, if i tried simply running `go get` like below, that will not work.

```bash
go get github.com/matijakrajnik/goprivatemodule
go get bitbucket.org/matijakrajnik/goprivatebucket
```

First thing i had to do is set `GOPRIVATE` environment variable. This variable contains a list of module path prefixes that are considered to be private, so `go` will not use proxy nor trying to validate checksum from public database.

To set that variable, run:

```bash
go env -w GOPRIVATE=github.com/matijakrajnik/goprivatemodule
```

It's also possible to use glob pattern to match all of your repositories:

```bash
go env -w GOPRIVATE=github.com/matijakrajnik/*
```

And even make comma separated list to include multiple values:

```bash
go env -w GOPRIVATE=github.com/matijakrajnik/*,bitbucket.org/matijakrajnik/*
```

Setting `GOPRIVATE` variable is only first step. When you run `go get` command, Go uses git for downloading module source files. And by default, git uses HTTPS, which will not work because terminal prompts are disabled and git don't have any credentials for authentication. So you will end up will error like this:

```bash
$ go get -v github.com/matijakrajnik/goprivatemodule
go get: module github.com/matijakrajnik/goprivatemodule: git ls-remote -q origin in /home/matija/go/pkg/mod/cache/vcs/f53d9a18ffd3065378124236da32538eaf1adb1addf29c1ac217b2540997b252: exit status 128:
        fatal: could not read Username for 'https://github.com': terminal prompts disabled
Confirm the import path was entered correctly.
If this is a private repository, see https://golang.org/doc/faq#git_https for additional information.
```

Fortunately, if you are using SSH keys for authentication, solution is very simple. You just need to configure git to use `git` instead of `https`.

For GitHub repository use:

```bash
git config --global url."git@github.com:matijakrajnik/goprivatemodule.git".insteadOf "https://github.com/matijakrajnik/goprivatemodule"
```

For Bitbucket repository use:

```bash
git config --global url."git@bitbucket.org:matijakrajnik/goprivatebucket".insteadOf "https://bitbucket.org/matijakrajnik/goprivatebucket"
```

If you look carefully at these two commands above, you can see that subtle little difference i mentioned at the beginning. Notice that in GitHub URL you have to include `.git` suffix, while for Bitbucket you can't use it. This can also vary from one git service implementation to another, so if you have problem, try to add or remove that `.git` suffix.

You can check git configuration file located in `~/.gitconfig`, to check that commands above took effect. If everything is ok, you will see something like this:

```bash
[url "git@github.com:matijakrajnik/goprivatemodule.git"]
  insteadOf = https://github.com/matijakrajnik/goprivatemodule
[url "git@bitbucket.org:matijakrajnik/goprivatebucket"]
  insteadOf = https://bitbucket.org/matijakrajnik/goprivatebucket
```

With that setup i was able to download my private modules with:

```bash
go get github.com/matijakrajnik/goprivatemodule
go get bitbucket.org/matijakrajnik/goprivatebucket
```

And use them in my project:

```go
package main

import (
  "fmt"

  "github.com/matijakrajnik/goprivatemodule/hello"
)

func main() {
  fmt.Println(hello.World())
}
```

```go
package main

import (
  "fmt"

  "bitbucket.org/matijakrajnik/goprivatebucket/hello"
)

func main() {
  fmt.Println(hello.World())
}
```