15 Minute Nx on a bed of Clouds
Tags: nx, javascript, nxcloud, monorepo
Published: 2020-04-15 17:07:31

With all the focus these days on home cooking, I thought I'd write up a recipe of my own which you can follow without even having to go into the kitchen.  

For those of you craving the taste of a real monorepo, this recipe combines [Nx](https://nx.dev/) with a continuous integration service and then adds some fresh [Nx Cloud](https://nx.app/) for a piquant dash of the efficiency that comes with [computation memoization](https://en.wikipedia.org/wiki/Memoization).  

If it's not clear what computation memoization is or how it can help you, then this could be the recipe for you!  

Unlike a standard recipe the product of following this set of instructions should be _understanding_. The quality of what you create as a result of combining all the ingredients together doesn't matter. You could love or hate the result, but what I hope you'll come away with is a little more knowledge. That's what writing this achieved for me.  

I want to emphasise that I describe this as a recipe because there is no single right way to combine all these ingredients. What follows is something that worked for me. If you find any problems, errors or just have questions, feel free to tweet me [@jdpearce](https://twitter.com/jdpearce).  

## ⚠️ Requirements

There is only so much I can explain without making this article unwieldy, so I'll assume that anyone reading this has some understanding of JavaScript development and already has [Node](https://nodejs.org) and [npm](https://www.npmjs.com/) installed. You can use another package manager if you like, but I'm only going to provide instructions for the lowest common denominator.

I am going to assume you have _some_ idea what a [monorepo](https://en.wikipedia.org/wiki/Monorepo) actually is and why you might want to use one. Following this recipe may give you some insight into how they can be used, but I'm not going to go into detail about the pros and cons.

While I list having a [GitHub](https://github.com/) account as an ingredient, I'm not going to explain how to set up new repositories and I will assume a certain amount of knowledge of [git](https://git-scm.com/) itself. There are better articles out there which can go into detail about this. If you run into problems I will always recommend "[Oh Shit, Git!?!](https://ohshitgit.com/)" 😁  

## 🥬 Ingredients

- 1 x [Nx (Extensible Dev Tools for Monorepos)](https://nx.dev/) workspace (v9.2.2 as of writing)
- 1 x [Nx Cloud (Computation Memoization in the Cloud)](https://nx.app/) account
- 1 x [CircleCI (Continuous integration)](https://circleci.com/) account
- 1 x [GitHub](https://github.com/) account
- Back and/or front end frameworks to taste (I'm going to use [React](https://reactjs.org/) and [Express](https://expressjs.com/))

## 🥒 Preparation

Before we start, I recommend that you have accounts with [CircleCI](https://circleci.com/), [Nx Cloud](https://nx.app/) and [GitHub](https://github.com/) ready. You don't want to ruin the flow of a good home cooking session by having to run out to the shops and generate new passwords.  

Make sure your [CircleCI](https://circleci.com/) account is connected to your [GitHub](https://github.com/) account. You can achieve that via the [Account Integrations](https://circleci.com/account) dashboard.  

I'm going to assume that you also have [Nx](https://nx.dev/) installed globally. You don't need to do this, you can just as easily use `npx nx` or `yarn nx` to call it, but it does mean I have to write less. To install with `npm`:  

```bash
npm install -g @nrwl/cli
```

Now that we're all prepared, let's get cooking! 

## 1. Creating the Nx Workspace (Monorepo)
### ⏱ 3 minutes

Find a place on your file system where you want to create the new workspace and run the following command:  

```bash
npx create-nx-workspace@latest nx-cloud-recipe
```

You'll be asked a set of questions for the kind of workspace you want to create. I'm going to choose the `react-express` preset which will create two applications; a React front end and an Express API. You can call the front end application whatever you like, but I'll go with `todos` as that seems to be fairly standard ~~tutorial~~ recipe fare, and I'll just select `CSS` as the default stylesheet format for now.  

![](https://i.imgur.com/fdXWzhK.png)

Once the creation script has worked its magic, you should find a new workspace with an initialised git repository in the `nx-cloud-recipe` folder.  

```bash
cd nx-cloud-recipe
```

To check that everything is set up as expected, why not run a test:  

```bash
nx test todos
```

On my machine this runs in a couple of seconds and the output looks like this:  

![](https://i.imgur.com/oaJTT3g.png)


Or we could run all the tests in the monorepo:  

```bash
nx run-many --target=test --all
```

The output of that should look something like this:  

![](https://i.imgur.com/ZJLWEgm.png)

You should find that the output for this command is generated _almost instantaneously_. This is because Nx locally caches the output of certain computational tasks, such as in this case running tests. There only happens to be one test suite in the entire monorepo and we just ran it and therefore cached it.  

If we make changes to the `todos` application, the same command will be smart enough to recognise that something has changed and the test will be run.  

You can pause here if you wish to get a feel for this particular ingredient. Try running both applications and seeing the output at http://localhost:4200 -  

```bash
nx serve api & nx serve todos
```

> ⚠️ **NB** You'll need to run these in different instances of `cmd`/`powershell` if you're running on Windows. Alternatively, you could investigate the power of [`run-commands`](https://nx.dev/react/plugins_workspace_builders/run-commands), but that's beyond the scope of this recipe for the moment.  

## 2. Push Your Monorepo to GitHub 
### ⏱ 2 minutes

Create a new, empty repository on GitHub, I've called mine the same as the local monorepo, but you don't have to (you could call it `bob` or `alice` if you really wanted 🤷‍♀️). After you've done that, come back to the local command line to push the generated code:  

```bash
git remote add origin https://github.com/jdpearce/nx-cloud-recipe.git
git push -u origin master
```

> ⚠️ **NB** Don't try to push to my repo, I don't think you'll be allowed. Copy the link to your own repository from the screen that GitHub helpfully shows you with all the possible ways you can push code to it.  

## 3. Connect to CircleCI
### ⏱ 2 minutes

Go to your [CircleCI dashboard](https://circleci.com/dashboard) and select "Add Projects". This should bring you to a page something like this where you can search for the new GitHub repository that you've just created:  

![](https://i.imgur.com/Q6LneRc.png)

> ℹ️ **NB** If the repository doesn't appear in the list, you may have to refresh the page.  

Click on the "Set up Project" button and then select the "Node" configuration template:  

![](https://i.imgur.com/ouRxxeB.png)

Next we have to click the appallingly badly named "Start Building" button, which does nothing of the sort:  

![](https://i.imgur.com/pDnOuq1.png)

We're going to let CircleCI create a new branch called `circleci-project-setup` and commit a new file `.circle/config.yml`  

Click on the "Add Config" button and let it do it's thing.  

> ⚠️ **NB** I am not a big fan of [YAML](https://yaml.org/). I think it's difficult to write and more importantly difficult to _read_. Be warned that there is unfortunately quite a bit of it coming up. [YMMV](https://tvtropes.org/pmwiki/pmwiki.php/Main/YMMV), of course...  

## 4. Adding `build` and `test` jobs
### ⏱ 5 minutes

At your local command line:  

```bash
git fetch
git checkout circleci-project-setup
```

We're going to cheat a little here. Detailed configuration of CircleCI is far beyond the scope of this recipe so I'm going to provide you with a store-bought configuration that sets up both `build` and `test` jobs. There are many ways to do this, so don't believe for a moment that this is the best or only way to achieve this goal. If you have the time artisanal, hand-crafted YAML is the way to go, but store-bought is fine for now.  

> ℹ️ **NB** Deployment is also beyond the scope of this recipe, but my colleague [Rares Matei](https://twitter.com/volkeron) is publishing something covering deployment in the near future.  

Using your editor of choice, replace the contents of the `.circleci/config.yml` file with the following:  

```yaml
version: 2.1

orbs:
  node: circleci/node@2.0.1

# Reusable Commands
commands:
  npm_install:
    description: 'Install & Cache Dependencies'
    steps:
      - run: npm install
      - save_cache:
          key: nx-cloud-recipe-{{ checksum "package-lock.json" }}
          paths:
            - ~/.cache
            - node_modules

  restore_npm_cache:
    description: 'Restore Cached Dependencies'
    steps:
      - restore_cache:
          keys:
            - nx-cloud-recipe-{{ checksum "package-lock.json" }}
            - nx-cloud-recipe- # used if checksum fails

  setup:
    description: 'Setup Executor'
    steps:
      - checkout
      - attach_workspace:
          at: ~/project

# Available Jobs
jobs:
  initialise:
    executor:
      name: node/default
    steps:
      - checkout
      - restore_npm_cache
      - npm_install
      - persist_to_workspace:
          root: ~/project
          paths:
            - node_modules
            - dist

  build:
    executor:
      name: node/default
    steps:
      - setup
      - run:
          name: Build all affected projects
          command: npx nx affected:build --base=master --head=HEAD

  test:
    executor:
      name: node/default
    steps:
      - setup
      - run:
          name: Run all affected tests
          command: npx nx affected:test --base=master --head=HEAD

workflows:
  build-and-test:
    jobs:
      - initialise
      - build:
          requires:
            - initialise
      - test:
          requires:
            - initialise

```

😱

This is a really scary amount of YAML and personally I find it difficult to read. CircleCI's documentation is pretty unhelpful for the novice too, so I'll try to briefly explain what's going here.  

At the top level we have groupings of [`orbs`, `commands`, `jobs` and `workflows`](https://circleci.com/docs/2.0/jobs-steps/#section=getting-started). In the `orbs` group we indicate that we're using the [`circleci/node@2.0.1`](https://circleci.com/orbs/registry/orb/circleci/node) orb, which is a collection of bits and pieces for working with `node` projects. In particular it includes the default executor which is the environment used to run the jobs.  


The `commands` group declares and defines three commands that can be used within jobs:

- `npm_install` - runs a standard dependency installation and populates a local cache
- `restore_npm_cache` - restores from that local cache
- `setup` - checks out the code and restores a workspace

The `jobs` group declares and defines three jobs that we can sequence within workflows:

- `initialise` - check out the code, restore caches, run an npm install command and then persist all this to the workspace
-  `build` - builds all the affected projects
-  `test` - tests all the affected projects

> ℹ️ **NB** It's worth noting here that the build and test jobs use [`nx affected`](https://nx.dev/react/cli/affected) logic to decide which projects to build or test. Nx is smart enough to know which projects have been touched by a particular commit (or range of commits) and will run the appropriate target for those projects **_only_**. This should save you considerable amounts of time in your CI environment. In this case we're only running build and test for those projects which have changes when comparing between `HEAD` and the current `master` branch, so this should work well for pull requests.  

Lastly, the `workflows` group defines a single workflow called `build-and-test` which specifies that the `initialise` job must be run before either `build` or `test` can be run.

If you save, commit, and push this to GitHub, you should see something like this in CircleCI:

![](https://i.imgur.com/RvH841s.png)

🎉

If everything looks good as above, we can get that configuration into the `master` branch with a pull request.

> ℹ️ **NB** You _could_ make this workflow run only when a pull request is _created_, but that (along with many other configuration niceties) is beyond the scope of this recipe. Still, this is one of the many aspects of this recipe that you can experiment with later! 🧪

## 5. Connect to Nx Cloud
### ⏱ 3 minutes

The first step is to go to your [Nx Cloud dashboard](https://nx.app/) and create a new workspace. I've called mine the same as the repository, but again you don't have to do this if you're not really bothered about naming[^naming].

![](https://i.imgur.com/ileWE8p.png)

Once you've named the workspace, you'll be presented with this set of instructions for CI and local configuration. I've blocked out the tokens in the screenshot above so that nefarious web users don't store their caches in my workspace and use up my Nx Cloud Coupons (you should get 5 hours free when you first sign up).

We're now going to add that local _read-only_ token to our `nx-cloud-recipe` repository:

```bash
git checkout master
git pull
git checkout -b nx-cloud-configuration
npm install @nrwl/nx-cloud && nx g @nrwl/nx-cloud:init --token=<token>
```

(The last line here should be copied and pasted from the "Set up for local development" section shown above)

Next we need to add the _read-write_ token to our CircleCI setup:

![](https://i.imgur.com/6rhryg4.png)

(You can find this by selecting the "Workflows" section on the left and then clicking the little cog icon next to the `nx-cloud-recipe` workflow)

CircleCI is now ready for you to commit and push the `nx-cloud-configuration` branch:

```bash
git commit -am "feat: add nx cloud configuration"
git push --set-upstream origin nx-cloud-configuration
```

This should result in a beautiful green set of workflow steps which means you're ready to create a pull request and merge that back into `master`!

![](https://i.imgur.com/FNm8glj.png)

Time now for you to sit back, pour a glass of something enjoyable and serve the completed dish...  

---

[^naming]: Naming is, of course, one of the two hardest problems in computer science. Those being:  
    1. Naming things
    1. Cache invalidation
    1. Off-by-one errors


## 🍇 Taste the Fruits of Your Labour! 

[Nx](https://nx.dev/) and its [affected logic](https://nx.dev/react/cli/affected) has your back when it comes to saving time in your continuous integration environment. When it comes to saving _developer_ time locally, that's where [Nx Cloud](https://nx.app/) can really shine.

Check out the repo into a new folder, e.g.

```bash
cd ..
git clone https://github.com/jdpearce/nx-cloud-recipe.git nx-cloud-recipe2
cd nx-cloud-recipe2
npm install
```

> ⚠️ **NB** I'd rather you didn't clone my repository, but you can if you want. I make no guarantees about it having valid coupons on the account though, so you may not see any effect.

This repo shouldn't have any local cache since we haven't run build or test here before, however, if we try building everything:

```bash
nx run-many --target=build --all
```

You should see almost _instant_ output:

![](https://i.imgur.com/jUeNZ3B.png)


With the new [Nx Cloud](https://nx.app/) configuration in place, and because you haven't yet made any changes to the repository, the build process you run locally would be identical to the one that was run in CI, so Nx will read from the cache that was generated when the build ran in CI, saving us the amount of time it would normally have taken to run!  

If we check back on the [Nx Cloud](https://nx.app/) site, you should be able to see that we've had an impact on the graph:  

![](https://i.imgur.com/aH0ir0b.png)

A whole MINUTE saved! 🤯  

Ok, maybe that's not hugely impressive in this case, but then we did create this workspace mere minutes ago.  

ℹ️ **NB** One minute is the smallest unit that the chart labels will actually display, but the chart is drawn with finer precision, which is why the "minute" saved in `build` is bigger than the "minute" saved in `test` 🤓  

Imagine how much time you would save if this were a mature workspace with numerous apps and libraries[^noimagination]. We've seen cases where in only weeks, teams have saved hundreds of hours of developer time. That's _your_ time, and that means more time for you to do the things you care about.

[^noimagination]: You don't actually have to imagine this. The following chart shows that over two weeks one particular team of 20-30 saved enough time to make up a whole extra developer! ![](https://i.imgur.com/yr0lkaG.png)