Exploring the Monorepo #3: Build the source
Tags: node, typescript, productivity, monorepo
Published: 2021-07-01 20:32:26

### Table Of Contents
  * [Make it build](#1)
  * [Does it work?](#2)
  * [The Good](#3)
  * [The Bad](#4)
    * [What can we solve with scripting?](#4-1)

Today we're going to try something that will definitely work, it's just a question of how bad a developer-experience we'll end up with: We'll compile our Typescript libraries into Javascript.

This will work because it cuts Typescript entirely out of the dependency equation, but it'll also drive a huge wedge into our workflow because now changes to source-code has to be compiled before it can be seen by consumers. And I suspect we'll find other downsides too.

Hopefully we can fix or alleviate those problems with some tooling or scripting, but this article isn't meant to uncover  **the golden solution** that'll end this article-series… It's just that I'm so very tired of hitting errors that I want to end today with **something that works**. Sometimes we have to take a leap to understand the problem better, so let's dive into it!

## Make it build <a name="1"></a>
To figure out what it means to build Typescript to Javascript let's first try it out on the `libs/types` package. It's quite straightforward to set up compiling:
1) Ensure `tsconfig.json` has the `outDir` field specified, that's the folder Javascript gets output to:
```
$ cd libs/types
$ cat tsconfig.json
  "compilerOptions": {
    "outDir": "./dist"
```

2) Add a `build` script:
```
$ cat package.json
  "scripts": {
    "build": "tsc -b"
$ pnpm build
$ tree dist 
dist
├── index.d.ts
├── index.js
└── index.js.map
```

3) Ensure `package.json` entry-point fields are set to point to the files in the `dist` folder:
```
$ cat package.json 
  "main": "dist/index.js",
  "types": "dist/index.d.ts",
```

With that in place this library can now be used as a normal Javascript dependency, and consumers don't have to know it's written in Typescript. Next we just have to apply this to all the code!

## Does it work? <a name="2"></a>
So, the result is the usual good overview:
```
webby
├── apps
│  ├── api
│  │  ├── package.json
│  │  ├── prisma/
│  │  ├── src/
│  │  └── tsconfig.json
│  └── web
│     ├── package.json
│     ├── src/
│     ├── tsconfig.json
│     └── typings/
├── libs
│  ├── analytics
│  │  ├── jest.config.js
│  │  ├── package.json
│  │  ├── src/
│  │  └── tsconfig.json
│  ├── logging
│  │  ├── package.json
│  │  ├── src/
│  │  └── tsconfig.json
│  └── types
│     ├── package.json
│     ├── src/
│     └── tsconfig.json
├── package.json
└── tsconfig.json
```
> _ℹ️ This project is [prepared on GitHub1s](https://github1s.com/gaggle/exploring-the-monorepo/tree/attempt-build) if you'd like to familiarize yourself with the code._

I've [kept the pnpm workspace configuration](https://dev.to/jonlauridsen/attempt-2-workspaces-npm-pnpm-336a#2) because it was such an easy tool to work with, making installing dependencies and running the `build` script across all packages quite easy:
```
$ cd ../..
$ pnpm install
Scope: all 6 workspace projects
$ pnpm -r run build
Scope: all 6 workspace projects
```

So does `web` work now?
```
$ cd apps/web
$ pnpm start
[razzle] > Started on port 3000
```

Good, good. And `api`?
```
$ cd ../api
$ pnpm start
[api] > prisma generate && nodemon -w src/* -x 'ts-node src/api.ts'
[api] Error: Command failed with exit code 1: npm install -D prisma@2.26.0
[api]  ERROR  Command failed with exit code 1.
```

Oh no! But wait, why does it say `Command failed: npm install` when we're using pnpm??

Turns out this is a known issue the nice people at Prisma [are](https://github.com/prisma/prisma/issues/6603) [working](https://github.com/prisma/prisma/issues/5340) [on](https://github.com/prisma/prisma/issues/7934), the workaround for now is to install a specific version (as I write this they recommend using version `2.27.0-integration-fix-sdk-pnpm.2`). 

If we make that change to `package.json`'s dependencies, does that make `api` work?
```
$ pnpm install
- @prisma/client 2.26.0
+ @prisma/client 2.27.0-integration-fix-sdk-pnpm.2
- prisma 2.26.0
+ prisma 2.27.0-integration-fix-sdk-pnpm.2
$ pnpm start
[api] api started at http://localhost:3002
```

Oh my goodness, hooray! 🎉

Putting it all together we can now raise our product entirely from the root:
```
$ cd ../..
$ git clean -dxi .; # this prompts you for what to delete so it's safe to run
$ pnpm install && pnpm build
$ pnpm start
apps/web start: [razzle] > Started on port 3000
apps/api start: [api] api started at http://localhost:3002
```

We did it!

## The Good <a name="3"></a>

Taking a step back there are some things I very much like about this pattern:

* By building the code we are no longer bound to writing it in Typescript. **Any** language that compiles to Javascript will do. So encapsulation of each project has increased, which I'll count as wonderful.

* This allows us a lot of flexibility in what a library produces: For this article-series the libraries are just groupings of code, their built code is identical in function to the source. But what if we imagine we wanted to generate something different than the source-code? What if we had a library whose source-code downloads Swagger documentation from some remote API and generate a Javascript client? To do that we **must** have a build step, and with this article's approach building is now a "first-class concept" so we don't have to make weird one-off exceptions to support something like that.

* I really appreciate the simplicity of boiling everything down to Javascript, there's just that much less chance of anything going wrong.

Do you see other good things about this pattern? I'd love to hear your take on this.

But there are some big drawbacks too! 😓

## The Bad <a name="4"></a>
* We now have a workflow where changes to a library aren't reflected in consumers until the library gets rebuilt. So we have to remember to run `pnpm build` after every change 😬. That's not good because it's so easy to forget, and then whatever work we've just done will seem like it's missing in the consumer in possibly subtle and confusing ways. I don't know you so maybe you wouldn't have a problem with this, but I think for newcomers and juniors it'll be that little extra annoyance that we are so desperately trying to avoid.

* We end up with boilerplate code and configurations that are identical across all projects, e.g. `tsconfig.json` must specify `outDir` and `package.json` must have a `build` script + specify `main` & `types` fields… it's just an annoying amount of small details that have to be exactly right and it gets worse the more projects we add.

Are there other downsides you can think of? I'd love to hear them!

### What can we solve with scripting? <a name="4-1"></a>
We first and foremost need to not manually rebuild all the time. I see two paths ahead:
1. Dependency rebuilds are invoked whenever consumers run their scripts. So every time `apps/web` runs `start` it would first go out and rebuild its dependencies.
2. Rebuild dependencies via a watcher, so every time a package's code changes it rebuilds itself.

Can you think of other proposals?

We also would benefit from some solution to the boilerplate code and configurations, e.g. if a script could check all packages and fix or warn about misconfigurations then we'd probably have alleviated the issue well enough.

This isn't the article where we write the scripts or even decide exactly how to do it, but maybe that's a topic for the next article? At this point I'd very much like to hear from you, so please leave a comment with your thoughts or suggestions.