Building a Monorepo with Yarn 2
Tags: javascript, node, tutorial, webdev
Published: 2020-12-22 21:19:13

In true JavaScript fashion, there was no shortage of releases in the JavaScript ecosystem this year. This includes the Yarn project’s release of Yarn 2 with a compressed cache of JavaScript dependencies, including a Yarn binary to reference,  that can be used for a zero-install deployment. 

Yarn is a package manager that also provides developers a project management toolset. Now, Yarn 2 is now officially supported by Heroku, and Heroku developers are able to take advantage of leveraging zero-installs during their Node.js builds. We’ll go over a popular use case for Yarn that is enhanced by Yarn 2: using workspaces to manage dependencies for your monorepo.

We will cover taking advantage of Yarn 2’s cache to manage monorepo dependencies. Prerequisites for this include a development environment with Node installed. To follow these guides, set up an existing Node project that makes use of a `package.json` too. If you don’t have one, use the [Heroku Getting Started with Node.js Project](https://github.com/heroku/node-js-getting-started).

## Workspaces

First off, what are workspaces? Workspaces is Yarn’s solution to a monorepo structure for a JavaScript app or Node.js project. A monorepo refers to a project, in this case, a JavaScript project, that has more than one section of the code base. For example, you may have the following set up:

```
/app
 - package.json
 - /server
   - package.json
 - /ui
   - package.json
```

Your JavaScript server has source code, but there’s an additional front end application that will be built and made available to users separately. This is a popular pattern for setting up a separation of concerns with a custom API client, a build or testing tool, or something else that may not have a place in the application logic. Each of the subdirectory’s `package.json` will have their own dependencies. How can we manage them? How do we optimize caching? This is where Yarn workspaces comes in.

In the root `package.json`, set up the subdirectories under the `workspaces` key. You should add this to your `package.json`:

```json
"workspaces": [
    "server",
    "ui"
]
```

For more on workspaces, visit here: [https://yarnpkg.com/features/workspaces](https://yarnpkg.com/features/workspaces)

Additionally, add the `workspaces-tools` plugin. This will be useful when running workspace scripts that you’ll use later. You can do this by running:

```sh
yarn plugin import workspace-tools
```

## Setting up Yarn

If you’re already using Yarn, you have a `yarn.lock` file already checked into your code base’s git repository. There’s other files and directories that you’ll need up to set up the cache. If you aren’t already using Yarn, install it globally.

```sh
npm install -g yarn
```

*Note: If you don’t have Yarn >=1.22.10 installed on your computer, update it with the same install command.*

Next, set up your Yarn version for this code base. One of the benefits of using Yarn 2 is that you’ll have a checked in Yarn binary that will be used by anyone that works on this code base and eliminates version conflicts between environments.

```sh
yarn set version berry
```

A `.yarn` directory and `.yarnrc.yml` file will both be created that need to be checked into git. These are the files that will set up your project’s local Yarn instance.

## Setting Up the Dependency Cache

Once Yarn is set up, you can set up your cache. Run yarn install:

```sh
yarn
```

Before anything else, make sure to add the following to the `.gitignore`:

```
# Yarn
.yarn/*
!.yarn/cache
!.yarn/releases
!.yarn/plugins
!.yarn/sdks
!.yarn/versions
```

The files that are ignored will be machine specific, and the remaining files you’ll want to check in. If you run `git status`, you’ll see the following:

```
Untracked files:
  (use "git add <file>..." to include in what will be committed)
    .gitignore
    .pnp.js
    .yarn/cache/
    yarn.lock
```


You’ve created new files that will speed up your install process:

* `.pnp.js` - This is the Plug’n’Play (PnP) file. The PnP file tells your Node app or build how to find the dependencies that are stored in `.yarn/cache`.
* `.yarn/cache` - This directory will have the dependencies that are needed to run and build your app.
* `yarn.lock` - The lock file still is used to lock the versions that are resolved from the `package.json`.

Check all of this in to git, and you’re set. For more information about Yarn 2’s zero-install philosophy, read here: [https://yarnpkg.com/features/zero-installs](https://yarnpkg.com/features/zero-installs)

## Adding Dependencies to Subdirectories

Now that Yarn and the cache are set up, we can start adding dependencies. As initially shown, we have a `server` directory and a `ui` directory. We can assume that each of these will be built and hosted differently. For example, my server is written in TypeScript, using Express.js for routing, and running on a Heroku web dyno. For the front end app, it is using Next.js. The build will be run during the app’s build process.

Add `express` to the server `dependencies`. 

```sh
yarn workspace server add express
```

Additionally, add `@types/express` and `typescript` to the `devDependencies`. You can use the `-D` flag to indicate that you’re adding `devDependencies`. 

```sh
yarn workspace server add @types/express typescript -D
```

We now have our dependencies in our `server` workspace. We just need to create our `ui` workspace. Next, build a Next.js app with the `yarn create` command.

```sh
yarn create next-app ui
```

Finally, run `yarn` again to update the cache and check these changes into git.

## Running Scripts with Workspaces

The last piece is to run scripts within the workspaces. If you look through your source code, you’ll see that there’s one global cache for all dependencies under your app’s root directory. Run the following to see all the compressed dependencies:

```sh
ls .yarn/cache
```

Now, lets run build scripts with workspaces. First, set up the workspace. For server, use `tsc` to build the TypeScript app. You’ll need to set up a TypeScript config and a `.ts` file first:

```bash
cd server
yarn dlx --package typescript tsc --init
touch index.ts
```

`yarn dlx` will run a command from a package so that it doesn’t need to be installed globally. It’s useful for one-off initializing commands, like initializing a TypeScript app.

Next, add the build step to the `server/package.json`.

```json
"scripts": {
    "build": "tsc",
    "start": "node index.js"
},
```

Change directories back to the application level, and run the build.

```bash
cd ..
yarn workspace server build
```

You’ll see that a `server/index.js` file is created. Add `server/*.js` to the `.gitignore`.

Since we already have `build` and `start` scripts in our Next.js app (created by the `yarn create` command), add a build script at the root level `package.json`.

```json
"scripts": {
    "build": "yarn workspaces foreach run build"
},
```

This is when the `workspaces-tool` plugin is used. Run `yarn build` from your app’s root, and both of your workspaces will build. Open a second terminal, and you’ll be able to run `yarn workspace server start` and `yarn workspace ui start` in each terminal and run the Express and Next servers in parallel.

## Deploy to Heroku

Finally, we can deploy our code to Heroku. Since Heroku will run the script is in the `package.json` under `start`, add a script to the `package.json`.

```json
"scripts": {
    "build": "yarn workspaces foreach run build",
    "start": "yarn workspaces server start"
},
```

[Heroku will use the `start` script](https://devcenter.heroku.com/articles/deploying-nodejs#specifying-a-start-script) from the `package.json` to start the `web` process on your app.

## Conclusion

There are [plenty more features](https://yarnpkg.com/features) that Yarn, and specifically Yarn 2, offers that are useful for Heroku developers. Check out the Yarn docs to see if there are additional workspace features that may work nicely with Heroku integration. As always, if you have any feedback or issues, please [open an Issue on GitHub](https://github.com/heroku/heroku-buildpack-nodejs/issues/new/choose).