JavaScript monorepos with Lerna
Tags: monorepo, javascript, beginners, cicd
Published: 2021-12-20 13:42:28

It's no secret that code sharing speeds up development. And there is no better way of teaming up and collaborating than with [a monorepo](https://semaphoreci.com/blog/what-is-monorepo) — provided you have the right tools to work with it.

## What is Lerna

Lerna is a monorepo manager for JavaScript projects. It helps you take a large codebase and split it into independently deployable packages. Lerna handles every step in the release process — from downloading dependencies, linking packages together, to testing and publishing updated packages to the NPM registry.

By running on top of traditional tools such as [npm](https://www.npmjs.com/) and [Yarn](https://yarnpkg.com/), Lerna can understand how the packages in the repository are interconnected. Needless to say, this makes it so easy to cross-reference and link packages in the same repository.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3i627jxr1fxzpat8aa6o.png)

## Who is using Lerna

You don't have to take my word on it. Lerna is an integral part of the development cycle of incredibly popular projects such as [Babel](https://github.com/babel/babel), Facebook's [Jest](https://github.com/facebook/jest), [Gatsby](https://github.com/gatsbyjs/gatsby), Google's [AngularJS](https://github.com/angular), [EmberJS](https://github.com/emberjs), and [MeteorJS](https://github.com/meteor).


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4vpdjkbjdjjpzr1cwg6v.png)

## Versioning modes in Lerna

Before using Lerna you need to decide on a versioning scheme for your repository. Lerna supports two modes: fixed and independent.

In **fixed mode**, Lerna maintains the same version for every package in the repository. Updated packages will always be bumped to the same version together. This is the default mode.

**Independent mode** means that each package is versioned separately, allowing maintainers to bump versions independently. On publication, you'll be prompted on what to do with each updated package.

## Creating a new monorepo

We have a little JavaScript monorepo demo to play with here:

{% github TomFern/semaphore-demo-monorepo-javascript %}

Feel free to fork it and clone it as you follow this tutorial.

We'll start by generating a Lerna config with `lerna init`.

``` bash
$ lerna init
Creating package.json
Creating lerna.json
Creating packages directory
Initialized Lerna files
```

Move all your applications, libraries, subprojects, and shared code into the `packages` folder. Each project should have a `package.json` and, ideally, a lockfile.

``` bash
$ lerna import api
$ lerna import web
```

Lerna should now be detecting the packages. Which, for the demo, are two: a GraphQL API service and a Next.js static website.

``` bash
$ lerna ls
api
web
found 2 packages
```

Use `lerna bootstrap` to download NPM dependencies and cross-link packages in the repository.

``` bash
$ lerna bootstrap
```

Now you should be able to run all tests found in every package with `lerna run`. Try them to make sure they work well as a group — the demo ships with unit and integration tests.

``` bash
$ lerna exec npm run lint
$ lerna exec npm dev &
$ lerna exec npm test
$ lerna exec npm run test integration
```

## Pre-release checks

We're going to publish the packages to [npmjs.com](https://www.npmjs.com/). To try out this part, you'll need at least a free account on the service. Once logged in, [generate an automation token](https://docs.npmjs.com/creating-and-viewing-access-tokens) and copy the value shown somewhere safe. We'll need it in a few minutes.

While you're at it, if you haven't already, authenticate your machine with [npm login](https://docs.npmjs.com/cli/v7/commands/npm-adduser).

NPM requires that all packages have unique identifiers, so we can't use the names that came with the demo repository. Hence, rename the packages by editing their respective `packages.json`.

Probably the easiest way of making the package name unique is by [scoping them](https://docs.npmjs.com/about-scopes). You can make a package scoped by prefixing the name with your NPM username. In my case, I would change the first few lines of `packages.json` like this:

``` bash
  "name": "@tomfern/api",
  "publishConfig": {
    "access": "public"
  }
```

Commit the changes to the Git repository is clean:

``` bash
$ git add lerna.json package.json packages
$ git commit -m "install lerna, ready to publish"
```

## Publishing your packages

Publishing a package is a two-step process. First, Lerna pushes all the changes to the remote repository and creates a Git tag. Then, it deploys the updates to NPM. Lerna uses Git tags to mark releases and track changes.

The first step is accomplished with `lerna version`.

``` bash
$ lerna version
? Select a new version (currently 0.0.0) (Use arrow keys)
  Patch (0.0.1)
  Minor (0.1.0)
  Major (1.0.0)
  Prepatch (0.0.1-alpha.0)
  Preminor (0.1.0-alpha.0)
  Premajor (1.0.0-alpha.0)
  Custom Prerelease
  Custom Version
```

Lerna wants to know what the following version number should be. Using [semantic versioning](https://semver.org/), we have to decide how to number this release:

-   **patch** (1.2.**X**): when it doesn't introduce behavior changes. For example, to fix a bug.
-   **minor** (1.**X**.3): when the version includes backward-compatible changes.
-   **major** (**X**.2.3): when version introduces breaking changes.

Before making the change, Lerna will ask for confirmation:

``` bash
Changes:
 - @tomfern/api: 1.0.0. => 1.2.3
 - @tomfern/web: 1.0.0 => 1.2.3

? Are you sure you want to create these versions?
```

After picking a version, Lerna creates a tag and pushes it:

``` bash
$ lerna publish from-git
Found 2 packages to publish:
 - @tomfern/api => 1.2.3
 - @tomfern/web => 1.2.3
? Are you sure you want to publish these packages?
```

You can also combine versioning and publishing in one command:

``` bash
$ lerna publish patch
```

## Change-detection

Lerna understands Git and JavaScript. Therefore, it can detect when a package has changed. To try it, change a file and run ~lerna changed~.

``` bash
$ lerna changed
Looking for changed packages since v1.2.3
@tomfern/api
found 1 package ready to publish
```

You can find per-package changes details with `lerna diff`.

Try publishing the updated version by re-running `lerna version` and `lerna publish`.

## Configuring the CI/CD pipeline

For this part you'll need a [Semaphore](https://semaphoreci.com) account. If you don't have one, you can create a trial account for free with GitHub.

Now the trick is to automate all these processes in a [CI/CD pipeline](https://semaphoreci.com/blog/cicd-pipeline). The plan is to:

1.  Install and cache all dependencies.
2.  Run all tests.
3.  If we are on a tagged release, publish the packages.

After login in into Semaphore, click on **create new** to add a new project.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/n11nqc3qtthsnqv7z1kz.png)

Choose the forked repository.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9wxw862qf1vck2hos9oi.png)Finally, select "single job" and click on **customize**.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0wl6slsjyt3m8ddvzfqy.png)

**Install job**

[The build stage](https://semaphoreci.com/blog/build-stage) bootstraps the repository and caches the downloaded dependencies. We use `lerna bootstrap` and then `npm exec cache` to store the contents of `node_modules` in the [Semaphore cache](https://docs.semaphoreci.com/reference/toolbox-reference/#cache).

``` bash
npm install --global lerna
checkout
lerna exec -- cache restore node-modules-\$LERNA_PACKAGE_NAME-$SEMAPHORE_GIT_BRANCH,node-modules-\$LERNA_PACKAGE_NAME
lerna bootstrap
lerna exec -- cache store node-modules-\$LERNA_PACKAGE_NAME-$SEMAPHORE_GIT_BRANCH,node-modules-\$LERNA_PACKAGE_NAME node_modules
```


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qnevjhiqw4d5bzy0rmdu.png)
**Test block**

No [continuous integration](https://semaphoreci.com/continuous-integration) should lack tests. Our demo includes three types of tests:

-   **Linter**: runs [eslint](https://eslint.org/) to run static code analysis tests.
-   **Unit tests**: executes unit tests in all packages.
-   **Integration test**: executes the integration test suite.

Click on **add block** and scroll down on the right pane to the **prologue**. The prologue is executed before any jobs in the block. Type the following commands to retrieve the cached dependencies.

``` bash
npm install --global lerna
checkout
lerna exec -- cache restore node-modules-\$LERNA_PACKAGE_NAME-$SEMAPHORE_GIT_BRANCH,node-modules-\$LERNA_PACKAGE_NAME
lerna bootstrap
```

The test jobs are all one-liners. This is the linter:

``` bash
lerna run lint
```

Create two more jobs in the block, one for the unit tests:

``` bash
lerna run test
```

And one for the integration tests:

``` bash
lerna run test-integration
```

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1y003dcrtcsk46eflqkf.png)

Click on "run the workflow" &gt; **start** to try out the pipeline.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zd8z3zgy3xgqesmz6fxz.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/u727q6xk6g4ngwrgpjbv.png)

## Continuous Deployment

The goal here is to publish packages to the NPM registry using [continuous delivery](https://semaphoreci.com/cicd).

We'll begin by creating a [secret](https://docs.semaphoreci.com/guided-tour/environment-variables-and-secrets/) on Semaphore. Click on settings on the main menu.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nm0m4p9jself2vm0nmbd.png)

Then go to secrets and press **create secret**. In value, type `NPM_TOKEN` and fill in the automation token generated earlier. Save the secret.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/f36toimmfo3kwiyaupqe.png)
Go back to the workflow in Semaphore and click on **edit workflow** to open the editor.

Click on **add promotion** to create a second pipeline. Enable the automatic promotion checkbox and type this line, which selects tagged releases:

``` bash
tag =~ '.*' AND result = 'passed'
```


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1yvsj26m2w5o8vg8n1dj.png)
Click on the first job on the delivery pipeline and use the following commands in the job.

``` bash
npm install --global lerna
checkout
echo "//registry.npmjs.org/:_authToken=$NPM_TOKEN" > .npmrc
lerna exec -- cache restore node-modules-\$LERNA_PACKAGE_NAME-$SEMAPHORE_GIT_BRANCH,node-modules-\$LERNA_PACKAGE_NAME node_modules
lerna bootstrap
lerna publish from-git --no-git-tag-version --no-push --yes
```

Scroll down and check the `NPM` secret created earlier.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8pm8kk2sfpifn35r8xva.png)
Save the pipeline. It will run one more time, but no releases will take place. Next, try updating one of the packages using `lerna version` from your own machine.

``` bash
$ git pull origin main
$ lerna version patch
```

The pipeline should start when Lerna pushes the tagged release.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2t5zpyuyia4btq7o66xy.png)
## Changed-based testing

Lerna detects by itself which packages have changed since the last release and publishes only the new versions. But this feature only works for publishing, not testing.

While Lerna doesn't support change-based testing, Semaphore does. And it's pretty [easy to configure](https://docs.semaphoreci.com/essentials/building-monorepo-projects/). The trick is in the `change_in` function, which calculates folder and file changes. Let's see how to use it.

To use [change\_in](https://docs.semaphoreci.com/reference/conditions-reference/#usage-examples-for-change_in), you'll need to create separate test paths for each package or group of packages. In other words, you have to edit the jobs in "Test" so they only operate on one of the packages using the `--scope` option. As an example, this makes the lint job run only on the `@tomfern/api` package.

``` bash
lerna run lint --scope @tomfern/api
```

Repeat the change in the rest of the test jobs.

``` bash
lerna run test --scope @tomfern/api

lerna run test-integration --scope @tomfern/api
```


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/c4m8gvfcllyfql2p77dx.png)
Now create a second testing block for the other package and make it dependent on the "Bootstrap" block. This time, use `--scope` to select the other package.

The magic trick comes now. Scroll down until you reach "Skip/Run conditions" and select **Run this block when conditions are met**. For example, the following condition is triggered when a file changes in the `/packages/api` folder.

``` bash
change_in('/packages/api/', { default_branch: 'main'})
```


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pfjdgok3p44setb6jsxy.png)
If your repository's default branch is `master`, you can omit the `{ default_branch: 'main' }` part.

Repeat the same procedure for the `web` package:

``` bash
change_in('/packages/web/', { default_branch: 'main'})
```

Click on **Run the workflow** to save the setup and try the pipeline. Well used, change detection can significantly speed up pipelines.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/e7rxyhoximmzljgwy9uz.png)

## Next steps

As always, there is still some room for improvement. For example, you might want to use Lerna's [package hoisting](https://github.com/lerna/lerna/blob/main/doc/hoist.md) to reduce the size of the `node_modules`.

Bear in mind that Lerna can team up with Yarn, if you so prefer. You can switch from `npm` to `yarn` by adding these lines into `lerna.json`:

``` bash
  "npmClient": "yarn",
  "useWorkspaces": true
```

One of the benefits of this is that we can use [Yarn workspaces](https://semaphoreci.com/blog/javascript-monorepos-yarn-workspaces) to avoid using `node_modules` altogether.

## That's it

Monorepos are gaining popularity. In great part, thanks to improved tooling support. If you have many JavaScript packages in one repository and want to publish them to NPM, Lerna is the right tool for the job.

Are you a JavaScript developer? We have a lot of exciting stuff for you:

-   [Reproducible Node Builds With npm ci](https://semaphoreci.com/continuous-integration)
-   [Getting Started with Node.js and Mocha](https://semaphoreci.com/community/tutorials/getting-started-with-node-js-and-mocha)
-   [Dockerizing a Node.js Web Application](https://semaphoreci.com/community/tutorials/dockerizing-a-node-js-web-application)
-   [How To Build and Deploy a Node.js Application To DigitalOcean Kubernetes Using CI/CD](https://semaphoreci.com/blog/nodejs-digitalocean-kubernetes)