Publishing and Installing Private GitHub Packages using Yarn and Lerna
Tags: javascript, lerna, yarn, npm
Published: 2020-05-19 14:44:42

I have a collection of snippets and utilities that I frequently reach for when building web stuff. Up until now this code has been managed in a very adhoc fashion – copied and pasted between codebases, un-versioned, and free from the burden of tests 😉

The temptation is to publish these utilities, collectively or individually, on a package registry such as [NPM](https://npmjs.com). But, as rewarding and exhilarating as it can be to open source code, it does [have its downsides](https://news.ycombinator.com/item?id=23208779). In particular, publicly publishing a package can signal to other developers that it's production-ready and bring with it the apparent obligation of supporting its use. Alternatively, sometimes the code is sensitive in nature or is not yet mature enough to see the light of day.

Publishing these packages privately is a good solution so long as it's economical and has an efficient enough workflow. To keep the organisational overhead low I'll keep them all in a single repository, following the monolithic repository pattern. (I can't help but feel "minilithic" would be a more appropriate name here.)

NPM doesn't allow users to publish private packages for free, but the GitHub Package Registry does (with strings attached). Given [GitHub's recent acquisition of NPM](https://github.blog/2020-03-16-npm-is-joining-github/) this might well change in the future 🤷‍♂️

## Setup the Mono-Repository

I'll use my `nuxt-modules` private GitHub repository, and the private packages within, as a working example.

Let's get started... In a terminal of your choice create a new project directory and initialise Git and Yarn:

```bash
> mkdir nuxt-modules
> cd nuxt-modules
> git init
> yarn init
```

Enable [Yarn Workspaces](https://classic.yarnpkg.com/en/docs/workspaces/) by configuring the `"workspaces"` property in `package.json`:

```json
{
  "name": "nuxt-modules",
  "private": true,
  "workspaces": ["packages/*"]
}
```

Initialise [Lerna](https://lerna.js.org/) with independent versioning enabled:

```bash
> lerna init --independent
```

Configure Lerna to play-nice with Yarn Workspaces and target the GitHub Package Registry in `lerna.json`:

```json
{
  "packages": ["packages/*"],
  "version": "independent",
  "npmClient": "yarn",
  "useWorkspaces": true,
  "command": {
    "publish": {
      "conventionalCommits": true,
      "message": "chore(release): publish",
      "registry": "https://npm.pkg.github.com",
      "allowBranch": "master"
    }
  }
}
```

Feel free to customise the other properties, these are just my preferences.

## Create the Packages

Populate the `packages/` directory with a sub-directory for each package. The directory names shouldn't be prefixed with the scope, but the `name` field in the `package.json` should, e.g. `packages/nuxt-html-validate` will contain a `package.json` with the `name` field set to `@saulhardman/nuxt-html-validate`.

You can create packages using Lerna's `lerna create` command or by hand. The bare-minimum for an NPM package is a JavaScript entry-point (e.g. `index.js`) and a `package.json`.

Development dependencies that are common to all of the packages should be installed in the mono-repository root. As an example, here's the command to install [ESLint](https://eslint.org/), passing the `-W` argument to the `add` command:

```shell
> yarn add --dev -W eslint
```

A critical step in this process is to run `yarn init` within each of the directories. It's then necessary to make a minor adjustment to the resulting `package.json` files to set the `repository.directory` and `publishConfig.registry` fields. Here is an example of the `@saulhardman/nuxt-html-validate` package which is located in the `packages/nuxt-html-validate/` sub-directory:

```json
{
  "repository": {
    "type": "git",
    "url": "ssh://git@github.com/saulhardman/nuxt-modules.git",
    "directory": "packages/nuxt-html-validate"
  },
  "publishConfig": {
    "registry": "https://npm.pkg.github.com/"
  }
}
```

The final result should look something like this:

```bash
.
├── .gitignore
├── LICENSE.md
├── lerna.json
├── package.json
├── packages
│   ├── nuxt-html-validate
│   │   ├── README.md
│   │   ├── index.js
│   │   └── package.json
│   ├── nuxt-release
│   │   ├── README.md
│   │   ├── index.js
│   │   └── package.json
│   ├── nuxt-robotize
│   │   ├── README.md
│   │   ├── index.js
│   │   └── package.json
│   └── nuxt-rss
│       ├── README.md
│       ├── index.js
│       └── package.json
└── yarn.lock
```

## Authenticate with the GitHub Package Registry

The next step is to authenticate with the Github Package Registry (replace `@saulhardman` with **your** GitHub username):

```bash
> npm login --registry=https://npm.pkg.github.com --scope=@saulhardman
```

To interact with the package repository API, GitHub requires you to create a [Personal Access Token](https://help.github.com/en/github/authenticating-to-github/creating-a-personal-access-token-for-the-command-line) (PAT) which you will use in-lieu of your password. Make sure that the 'repo', 'write:packages', 'read:packages', and 'delete:packages' options are selected:

![Screenshot of GitHub generate Personal Access Token page with relevant options selected](https://res.cloudinary.com/viewsource-io/image/upload/w_1400,h_564,f_auto,q_80/production/assets/images/github-personal-access-token-permissions.png)

With that in-hand the `.npmrc` is configured to point requests for `@saulhardman`-scoped packages to GitHub (rather than NPM) and provide the PAT as an `authToken` (replace `TOKEN` and `@saulhardman` with **your** respective values):

```shell
//npm.pkg.github.com/:_authToken=TOKEN
@saulhardman:registry=https://npm.pkg.github.com
```

Even though this Git repository will be private it's good practice not to commit keys and tokens. Accordingly, be sure to amend the `.gitignore` config to include the `.npmrc`.

## Publish the Packages

Create your private GitHub repository and push your initial commit containing your packages. It's my preference to set the `package.version` fields to `0.0.0` to begin with. At publish-time you can pass `minor` or `major` to have `0.1.0` or `1.0.0` be the initial release version:

```bash
> yarn lerna publish minor # initial release 0.1.0
> yarn lerna publish major # initial release 1.0.0
```

Once you've received a "Package published" response, you will be able to view your packages on the GitHub repository page:

![Screenshot of the GitHub repository page showing '4 packages' published](https://res.cloudinary.com/viewsource-io/image/upload/w_1400,h_109,f_auto,q_80/production/assets/images/4-packages-published.png)

## Installing Private GitHub Packages

The permissions workflow surrounding private packages is... not great. There is, as far as I'm aware, no way to scope PATs to organisations, repositories, or packages. The method outlined here will allow you to install all private packages that your GitHub account has access to.

To install a private package all that's required is an `.npmrc` to assign an access token and configure the scopes. The PAT could be the same one used above or a different PAT with read-only permissions (replace `TOKEN` with your PAT and `@saulhardman` with your GitHub username):

```shell
//npm.pkg.github.com/:_authToken=TOKEN
@saulhardman:registry=https://npm.pkg.github.com
```

Only packages in the scope `@saulhardman` will be installed from the GitHub Package Registry – all others will default to NPM. The `yarn add` command can be used as usual, e.g.:

```bash
> yarn add @saulhardman/nuxt-html-validate
```

## Installing Private GitHub Packages from GitHub Actions

Setting the `NODE_AUTH_TOKEN` environment variable on the `yarn install` step _should_ be enough, but in my experience it is not. There is a thread on the [GitHub Community Forum](https://github.community/t5/GitHub-Actions/Installing-npm-packages-from-the-GitHub-package-registry/td-p/30559) documenting a number of people's struggles.

An alternative – whether you're running `yarn install` directly or using a third-party action such as [`bahmutov/npm-install`](https://github.com/bahmutov/npm-install) – is to construct an `.npmrc` dynamically using a PAT stored as an [encrypted secret](https://help.github.com/en/actions/configuring-and-managing-workflows/creating-and-storing-encrypted-secrets):

```yaml
steps:
  - name: Configure NPM
    run: |
      echo "//npm.pkg.github.com/:_authToken=$NODE_AUTH_TOKEN" > .npmrc
      echo '@saulhardman:registry=https://npm.pkg.github.com' >> .npmrc
    env:
      NODE_AUTH_TOKEN: ${{ secrets.NODE_AUTH_TOKEN }}

  - name: Install Yarn Dependencies
    uses: bahmutov/npm-install@v1
```

## Closing Thoughts

I've created a number of private packages over the last few months – ranging from the Nuxt modules outlined above to Vue components and JavaScript utilities. I've thoroughly enjoyed it so far and I feel the initial overhead will be well worth the reward in the long term.

Discovering a bug in one usage context, fixing it, adding a test case if necessary, and then having that update trickle-down to other projects with very little friction is both satisfying and refreshing.

### Additional Resources

- [Configuring npm for use with GitHub Packages](https://help.github.com/en/packages/using-github-packages-with-your-projects-ecosystem/configuring-npm-for-use-with-github-packages#authenticating-with-a-personal-access-token)
- [Github Actions with private package](https://charliejwalter.net/github-actions-with-private-package-github-package-registry-or-npm/)
- [How to publish packages to the GitHub Package Registry](https://dev.to/jgierer12/how-to-publish-packages-to-the-github-package-repository-4bai)