The chronicles of semantic-release and monorepos
Tags: monorepo, lerna, semrel, javascript
Published: 2020-05-30 16:27:04

Since [semantic-release](https://github.com/semantic-release/semantic-release) and [lerna](https://github.com/lerna/lerna) appeared, the community has made several attempts to combine these technologies together. One does not simply run release in parallel.
![Boromir jpg](https://dev-to-uploads.s3.amazonaws.com/i/pmj0lgphr2ahw9lz4082.jpg)
There are several significant hurdles. The first one is the `analyzeCommits` step. **semantic-release** tries to resolve a new version by making a cut of commits from the previous release tag in repo, but has no idea how these changes are related to packages.
The second issue is `publish`. Or `prepare`. Or rather [what happens between](https://github.com/semantic-release/semantic-release/blob/eed1d3c8cbab0ef05df39866c90ff74dff77dfa4/index.js#L190).
```javascript
await plugins.prepare(context);

  if (options.dryRun) {
    logger.warn(`Skip ${nextRelease.gitTag} tag creation in dry-run mode`);
  } else {
    // Create the tag before calling the publish plugins as some require the tag to exists
    await tag(nextRelease.gitTag, nextRelease.gitHead, {cwd, env});
    await addNote({channels: [nextRelease.channel]}, nextRelease.gitHead, {cwd, env});
    await push(options.repositoryUrl, {cwd, env});
    await pushNotes(options.repositoryUrl, {cwd, env});
    logger.success(`Created tag ${nextRelease.gitTag}`);
  }

  const releases = await plugins.publish(context);
```
When these `git add .`, ` git commit -m` and `git push --tags origin` queues from several async "threads" clash in runtime they produce unstashed changes and block each other.
The third problem is cross-dependencies. In most cases monorepository packages are interconnected, so it's necessary to update versions of these references in some way.

### 2015
[atlassian/lerna-semantic-release](https://github.com/atlassian/lerna-semantic-release)
If I'm not mistaken, it seems this was the first working solution. The key idea of **LSR** is to disassemble **semantic-release** for parts and use them to construct several step-like pipelines and run them with **Lerna**
```
# Pre
lerna-semantic-release pre # Set up the versions, tags and commits
# Perform
lerna-semantic-release perform # Publishes to npm
# Post
lerna-semantic-release post # Generates a changelog in each package
```
Under the hood we see [custom commits analyzer](https://github.com/atlassian/lerna-semantic-release/blob/c984ed08dea083cb4ae9394521c307a7c3fd81b3/packages/lerna-semantic-release-analyze-commits/index.js#L14), [custom changelog generator](https://github.com/atlassian/lerna-semantic-release/blob/caribou/packages/lerna-semantic-release-post/index.js#L52), [custom npm publisher](https://github.com/atlassian/lerna-semantic-release/blob/caribou/packages/lerna-semantic-release-perform/index.js#L36) and so on. Unfortunately, this tool design does not provide standard **semrel** extending features and any other plugins support. Moreover, once **semrel** changes its inners, this dramatically affects **LRS**. Still, it was worth a try.

### 2017
[pmowrer/semantic-release-monorepo](https://github.com/pmowrer/semantic-release-monorepo)
Another approach was proposed by [Patrick Mowrer](http://www.patrickmowrer.com/). He did not try to overcome the limitations of parallel running, and suggested that releases be performed sequentially.
```
lerna exec --concurrency 1 -- npx --no-install semantic-release -e semantic-release-monorepo
```
An important step forward was the use of **semrel** standard [extension techniques](https://github.com/semantic-release/semantic-release/blob/eed1d3c8cbab0ef05df39866c90ff74dff77dfa4/cli.js#L26). In essence, only one plugin required customization — `analyzeCommits` which was supplemented with [commit-by-package filtering](https://github.com/pmowrer/semantic-release-monorepo/blob/e6ef3afc172ad8841df3079696336190e73d63fd/src/only-package-commits.js#L34). Some implementation flaws are easily fixed with an [additional hook](https://github.com/qiwi/semantic-release-monorepo-hooks), but... cross-dependency problem was not considered at all.

### 2019
[dhoulb/multi-semantic-release](https://github.com/dhoulb/multi-semantic-release)
[Dave Houlbrooke](https://github.com/dhoulb) focused on implantation
of synchronization points inside the release flow. If we ~~cannot~~ should not change the **semrel** runner code, we can write a plug-in that will provide a coherence state of parallel threads. Each step of synthetic `Inline plugin` [injects wait conditions](https://github.com/dhoulb/multi-semantic-release/blob/561a8e66133d422d88008c32c479d1148876aba4/lib/createInlinePluginCreator.js#L74) in order to make sure all concurrent releases are in the same phase.

```javascript
async function generateNotes(pluginOptions, context) {
	// Set nextRelease for package.
	pkg._nextRelease = context.nextRelease;

	// Wait until all todo packages are ready to generate notes.
	await wait(() => todo().every(p => p.hasOwnProperty("_nextRelease")));
```
But it’s not enough just to keep parallel releases at one step. Some  actions, as we know, lie between **plugin[step]** calls. And with a moderately large number of packets the problem of unstashed changes appears again. However, it turned out this issue can be easily fixed by replacing internal asynchronous calls with [synchronous ones](https://github.com/dhoulb/multi-semantic-release/pull/2/files).
The killer feature of this solution is cross-dependencies update. **MSR** shared milticontext knows about which packages will be updated as part of all releases, so its possible to [update all the manifest files](https://github.com/dhoulb/multi-semantic-release/blob/561a8e66133d422d88008c32c479d1148876aba4/lib/createInlinePluginCreator.js#L158) too.
```javascript
// Loop through localDeps to update dependencies/devDependencies/peerDependencies in manifest.
	pkg._localDeps.forEach(d => {
	// Get version of dependency.
	const release = d._nextRelease || d._lastRelease;

	// Cannot establish version.
	if (!release || !release.version)
		throw Error(`Cannot release because dependency ${d.name} has not been released`);

	// Update version of dependency in manifest.
	if (manifest.dependencies.hasOwnProperty(d.name)) manifest.dependencies[d.name] = release.version;
	if (manifest.devDependencies.hasOwnProperty(d.name)) manifest.devDependencies[d.name] = release.version;
	if (manifest.peerDependencies.hasOwnProperty(d.name))
					manifest.peerDependencies[d.name] = release.version;
			});
```

### 2020
[qiwi/multi-semantic-release](https://github.com/qiwi/multi-semantic-release)
Dave's solution works great for a small number of packages (<20). But the implemented mechanism that interlocks releases phases is extremely slow for huge "enterprise" monorepos. This fork of **dhoulb/multi-semantic-release** replaces [`setImmediate` loops](https://github.com/dhoulb/multi-semantic-release/blob/561a8e66133d422d88008c32c479d1148876aba4/lib/wait.js#L13) and mentioned `execa.sync` hook with [event-driven flow](https://github.com/qiwi/multi-semantic-release/blob/51ed2042e1ee1877928929f6e349bf1eb984b48b/lib/createInlinePluginCreator.js#L26) and finally makes possible to run the most release operations in parallel.

```javascript
// Shared signal bus.
	const ee = new EventEmitter();

	// Announcement of readiness for release.
	todo().forEach((p) => (p._readyForRelease = ee.once(p.name)));

	// Status sync point.
	const waitFor = (prop, filter = identity) => {
		const promise = ee.once(prop);
		if (
			todo()
				.filter(filter)
				.every((p) => p.hasOwnProperty(prop))
		) {
			ee.emit(prop);
		}
		return promise;
	};
     ...
const publish = async (pluginOptions, context) => {
			pkg._prepared = true;
			const nextPkgToProcess = todo().find((p) => p._nextType && !p._prepared);

			if (nextPkgToProcess) {
				ee.emit(nextPkgToProcess.name);
			}

			// Wait for all packages to be `prepare`d and tagged by `semantic-release`
			await waitFor("_prepared", (p) => p._nextType);
```

### 202x
There're a lot of [monorepo related issues](https://github.com/semantic-release/semantic-release/search?q=monorepo&type=Issues) in **semrel** repo. NPM's intention to standardize [workspace notation](https://github.com/npm/rfcs/blob/latest/accepted/0026-workspaces.md) brings moderate optimism about the fate of monoreps. As soon as monoreps become more common practice, **semrel** will probably add built-in support for them.