Create Homebrew Taps for Private GitHub Repos
Published: 2022-10-04 00:00:30


I started a new job in July 2022. While the developer experience is great, one of the things that I missed at my new job was a CLI _to rule them all_ (like I had at my old job). I thought of building an easily extensible CLI tool which I did with _go_ and [cobra](https://github.com/spf13/cobra). To handle distribution to the engineers, we decided to use both `go install` and [_homebrew_](https://brew.sh/).  

Homebrew is a very popular package manager for macOS and Linux users. Users can easily install new packages with `brew install [PACKAGE]`. Homebrew handles the downloading and the installation of the packages to the appropriate directories and then creates symlinks to `/usr/local` (on Intel machines).

To create a Homebrew package, we create Ruby classes called _Formulas_. A Formula looks like this:

```ruby
class Wget < Formula
  homepage "https://www.gnu.org/software/wget/"
  url "https://ftp.gnu.org/gnu/wget/wget-1.15.tar.gz"
  sha256 "52126be8cf1bddd7536886e74c053ad7d0ed2aa89b4b630f76785bac21695fcd"

  def install
    system "./configure", "--prefix=#{prefix}"
    system "make", "install"
  end
end
```

While adding an open-source package is fairly straightforward, doing the same with a private package involves a bit of hacking. 

## First Steps
The first thing to do is create a Github release, if you use _go_, I highly recommend using [_goreleaser_](https://github.com/goreleaser/goreleaser). I wrote a tutorial on how to do that [here](https://justdamilare.github.io/blog/goreleaser). It truly makes releases in _go_ easy. If you're using another language, you should check out how to manage releases in GitHub [here](https://docs.github.com/en/repositories/releasing-projects-on-github/managing-releases-in-a-repository).

## Create a Homebrew Tap
The next thing to do is to create a [Homebrew tap](https://docs.brew.sh/Taps). A Homebrew tap is a GitHub repository that houses our Formulas. To add non-Homebrew core taps, we need to add the taps locally to homebrew with `brew tap username/package`. 
For us to use the single argument form of `brew tap`, we need to follow the [naming convention](https://docs.brew.sh/Taps#repository-naming-conventions-and-assumptions) and name our GitHub repo `homebrew-tap` since the GitHub repo must start with `homebrew-`.

## Create a Formula

If you have a release already in Github, we can go ahead with creating the Formula. In our `homebrew-tap` repo, we'll create a folder called `Formula`. In the `Formula` directory, we'll create a file called `mytool.rb`. 

Now, in our `Formula`, we cannot use the standard way of downloading the packages since our package is hosted in a private repo. To do this we simply add a new download strategy like such:

```ruby
require "formula"
require_relative "../custom_download_strategy.rb"

class Wget < Formula
  homepage "https://www.gnu.org/software/wget/"
  url "https://ftp.gnu.org/gnu/wget/wget-1.15.tar.gz", :using => GitHubPrivateRepositoryReleaseDownloadStrategy 
  sha256 "52126be8cf1bddd7536886e74c053ad7d0ed2aa89b4b630f76785bac21695fcd"

  def install
    system "./configure", "--prefix=#{prefix}"
    system "make", "install"
  end

  test do
      system "#{bin}/wget --help"
  end
end
```

Note the addition of `, :using => GitHubPrivateRepositoryReleaseDownloadStrategy` on the third line of the class.

In the root folder of `homebrew-tap`, we'll create a new file called `custom_download_strategy.rb`. Here, we'll add some code to support our download strategy.

```ruby
require "download_strategy"

class GitHubPrivateRepositoryDownloadStrategy < CurlDownloadStrategy
  require "utils/formatter"
  require "utils/github"

  def initialize(url, name, version, **meta)
    super
    parse_url_pattern
    set_github_token
  end

  def parse_url_pattern
    unless match = url.match(%r{https://github.com/([^/]+)/([^/]+)/(\S+)})
      raise CurlDownloadStrategyError, "Invalid url pattern for GitHub Repository."
    end

    _, @owner, @repo, @filepath = *match
  end

  def download_url
    "https://#{@github_token}@github.com/#{@owner}/#{@repo}/#{@filepath}"
  end

  private

  def _fetch(url:, resolved_url:, timeout:)
    curl_download download_url, to: temporary_path, timeout: timeout
  end

  def set_github_token
    @github_token = ENV["HOMEBREW_GITHUB_API_TOKEN"]
    unless @github_token
      raise CurlDownloadStrategyError, "Environmental variable HOMEBREW_GITHUB_API_TOKEN is required."
    end

    validate_github_repository_access!
  end

  def validate_github_repository_access!
    # Test access to the repository
    GitHub.repository(@owner, @repo)
  rescue GitHub::HTTPNotFoundError
    # We only handle HTTPNotFoundError here,
    # becase AuthenticationFailedError is handled within util/github.
    message = <<~EOS
      HOMEBREW_GITHUB_API_TOKEN can not access the repository: #{@owner}/#{@repo}
      This token may not have permission to access the repository or the url of formula may be incorrect.
    EOS
    raise CurlDownloadStrategyError, message
  end
end

# GitHubPrivateRepositoryReleaseDownloadStrategy downloads tarballs from GitHub
# Release assets. To use it, add
# `:using => GitHubPrivateRepositoryReleaseDownloadStrategy` to the URL section of
# your formula. This download strategy uses GitHub access tokens (in the
# environment variables HOMEBREW_GITHUB_API_TOKEN) to sign the request.
class GitHubPrivateRepositoryReleaseDownloadStrategy < GitHubPrivateRepositoryDownloadStrategy
  def initialize(url, name, version, **meta)
    super
  end

  def parse_url_pattern
    url_pattern = %r{https://github.com/([^/]+)/([^/]+)/releases/download/([^/]+)/(\S+)}
    unless @url =~ url_pattern
      raise CurlDownloadStrategyError, "Invalid url pattern for GitHub Release."
    end

    _, @owner, @repo, @tag, @filename = *@url.match(url_pattern)
  end

  def download_url
    "https://#{@github_token}@api.github.com/repos/#{@owner}/#{@repo}/releases/assets/#{asset_id}"
  end

  private

  def _fetch(url:, resolved_url:, timeout:)
    # HTTP request header `Accept: application/octet-stream` is required.
    # Without this, the GitHub API will respond with metadata, not binary.
    curl_download download_url, "--header", "Accept: application/octet-stream", to: temporary_path, timeout: timeout
  end

  def asset_id
    @asset_id ||= resolve_asset_id
  end

  def resolve_asset_id
    release_metadata = fetch_release_metadata
    assets = release_metadata["assets"].select { |a| a["name"] == @filename }
    raise CurlDownloadStrategyError, "Asset file not found." if assets.empty?

    assets.first["id"]
  end

  def fetch_release_metadata
    GitHub.get_release(@owner, @repo, @tag)
  end
end

```
We should have the following folder structure:

```
├── homebrew-tap
│   ├── Formula
│   │   ├── mytool.rb
└── custom_download_strategy.rb
```

## Usage

We're almost done. To install our tool via Homebrew, we need to export a Github token that provides access to our private repos. A new token can be created [here](https://github.com/settings/tokens). The token needs to have `repo` permissions. We need to export the token as `HOMEBREW_GITHUB_API_TOKEN`. To export the token, we run:

```
export HOMEBREW_GITHUB_API_TOKEN=<GITHUB_TOKEN>
# if you're using fish shell like me, run `set -x HOMEBREW_GITHUB_API_TOKEN <GITHUB_TOKEN>`
```
Then, we can run the command:

```
brew install username/homebrew-tap/mytool
```

This automatically adds the homebrew tap and install our tool.

## Summary

* Create a GitHub release with the binaries. This can be done very easily with _goreleaser_ in the case of _go_.
* Create a GitHub repo called `homebrew-tap`. An example is [here](https://github.com/justdamilare/homebrew-tap).
  * Create a `Formula` folder.
  * In the root of the repo, create a file called `custom_download_strategy.rb`. The contents are shown above
  * In the `Formula` folder, create a ruby file with the name of your tool like `mytool.rb`  
* Create a GitHub token [here](https://github.com/settings/tokens) and give `repo` permissions to the token. 
* Export the token with `export HOMEBREW_GITHUB_API_TOKEN=<GITHUB_TOKEN>`
* Install the tool with brew using `brew install username/homebrew-tap/mytool`.