Implicit Dependencies Management with Nx: A Practical Guide through Real-World Case Studies
Tags: node, tutorial, nx, javascript
Published: 2024-02-19 19:38:25

In this article, I will present two concrete cases of implicit dependencies and show you how to configure your Nx workspace and projects to deal with them.
You can find the repository that illustrates those cases on [GitHub](https://github.com/getlarge/nx-implicit-dependencies).

Before we dive into the cases, let’s take a moment to understand how Nx represents dependencies and how it can help us work with implicit dependencies.

In the context of a [workspace](https://nx.dev/reference/glossary#workspace), Nx maintains a graph of all projects it contains, including internal and external dependencies. It does its best to deduct relationships by continuously analyzing the workspace components, such as the source code, the projects' configuration, the installed dependencies, etc. All being explicit dependencies. For the rest - the implicit dependencies that only a human could be aware of - Nx offers a way to declare them so they can also be part of the graph.

We can use the following graph extract to understand its composition:

```json
{
  "graph": {
    "nodes": {
      "ts-interfaces": {
        "name": "ts-interfaces",
        "type": "lib",
        "data": {
          "root": "libs/shared/ts-interfaces",
          "name": "ts-interfaces",
          "targets": {
            // ...
          },
          "$schema": "../../../node_modules/nx/schemas/project-schema.json",
          "sourceRoot": "libs/shared/ts-interfaces/src",
          "projectType": "library",
          "implicitDependencies": ["schemas"],
          "tags": ["platform:js", "type:core"]
        }
      },
      "schemas": {
        "name": "schemas",
        "type": "lib",
        "data": {
          "root": "libs/shared/schemas",
          "name": "schemas",
          "targets": {
            // ...
          },
          "$schema": "../../../node_modules/nx/schemas/project-schema.json",
          "sourceRoot": "libs/shared/schemas/src",
          "projectType": "library",
          "tags": ["platform:shared", "type:core"],
          "implicitDependencies": []
        }
      },
      "coffee-dealer": {
        "name": "coffee-dealer",
        "type": "app",
        "data": {
          "root": "apps/coffee-dealer",
          "name": "coffee-dealer",
          "targets": {
            // ...
          },
          "$schema": "../../node_modules/nx/schemas/project-schema.json",
          "sourceRoot": "apps/coffee-dealer/src",
          "projectType": "application",
          "tags": ["type:app", "platform:node"],
          "implicitDependencies": []
        }
      }
    },
    "dependencies": {
      "ts-interfaces": [
        {
          "source": "ts-interfaces",
          "target": "schemas",
          "type": "implicit"
        }
      ],
      "schemas": [],
      "coffee-dealer": [
        {
          "source": "coffee-dealer",
          "target": "ts-interfaces",
          "type": "static"
        }
      ]
    }
  }
}
```

> **Note**
>
> - This graph is generated by running `nx graph --file=output.json && cat output.json ` in the repository root folder.
> - If you are unfamiliar with Nx, you might want to head to Nx’s documentation to understand the benefits of the **project graph** and the [mental model](https://nx.dev/concepts/mental-model#mental-model).

Under `graph.nodes`, each node represents a project in the workspace. The `graph.dependencies` include the edges defining the relationship between nodes/projects. The `type` property can be `static` or `implicit`, depending on whether the dependency is explicit or implicit.

## Case 1: The Libraries Hidden Bridge

This case occurred in an [integrated repository](https://nx.dev/getting-started/tutorials/integrated-repo-tutorial) composed of multiple apps and libraries. I reproduced a similar situation with the following setup:

- `coffee-dealer`, is a NestJS application importing TS interfaces from `ts-interfaces`
- `ts-interfaces`, is composed of auto-generated Typescript interfaces from `schemas`
- `schemas`, is composed of JSON schemas exported in a TS file

You can see the current relation between the projects in the project graph:

![Projects graph before](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/32x22qkn4vm2y9c0qkp9.png)

And the tasks graph for the three projects:

![Tasks graph before](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9ww3cdcesc1y30nzxqyo.png)

What is wrong with these graphs?

The `schemas` library is not linked to the `ts-interfaces` library, meaning that when building the application with `nx run coffee-dealer:build`:

1. The `schemas:bundle` task will not be included in the **task pipeline**, inducing the risk of using outdated schemas while generating the Typescript interfaces.
2. The `schemas:bundle` and `ts-interfaces:bundle` tasks runner is not caching inputs, so Nx cannot check whether the source definitions have changed at the next `bundle` run.

In the following sections, I will further present the two libraries and show you how to solve this issue.

### schemas

[Link to the library](https://github.com/getlarge/nx-implicit-dependencies/tree/main/libs/shared/schemas)

These JSON schemas represent the domain entities and value objects; in other words, they are a source of truth and can be derived into types for different programming languages (Go struct, Typescript interfaces … ).
The JSON schema definitions organization follows [this good advice](https://json-schema.org/understanding-json-schema/structuring#schema-identification) so that each definition represents a single domain entity or value object, making work easier during development. Before being shared, the definitions are bundled and exported into larger schemas representing business operations.

The logic to bundle the JSON schemas is located outside the `src` folder, in the `internals` folder, to avoid including those scripts and schemas in the build output. To still benefit from Typescript in our IDE, a separate config is created under `tsconfig.editor.json` and referenced in `tsconfig.json` to include the `internals` folder.

```json
// libs/schemas/tsconfig.editor.json
{
  "extends": "./tsconfig.json",
  "compilerOptions": {
    "outDir": "../../dist/out-tsc",
    "declaration": true,
    "resolveJsonModule": true,
    "esModuleInterop": true,
    "types": ["node"]
  },
  "include": ["src/**/*.ts", "internals/**/*.ts"],
  "exclude": ["jest.config.ts", "test/**/*.spec.ts", "test/**/*.ts"]
}
```

Now let’s zoom in on the `project.json`

- It contains an extra `namedInputs`, which references the content of `internals`, allowing Nx to use this as a cache input for a task
- In the `build` task, I added a dependency on the `bundle` task, meaning that before compiling the Typescript code, Nx will execute the `bundle` task
- The `bundle` task executes the code containing the logic to bundle schemas into the `src/lib` folder. The `cache` property is set to true to enable task runner caching. The `inputs` use the `internals` named input, and the `outputs` reference the glob `{projectRoot}/src/lib/*.json`, which matches all bundled schemas.

> **Note**
> If the cache is hit because the input has not changed, Nx will restore the existing outputs.

```json
// libs/schemas/project.json
{
  "name": "schemas",
  "$schema": "../../../node_modules/nx/schemas/project-schema.json",
  "sourceRoot": "libs/shared/schemas/src",
  "projectType": "library",
  "namedInputs": {
    "internals": ["{projectRoot}/internals/*.{json,ts}"]
  },
  "targets": {
    "build": {
      "executor": "@nx/js:tsc",
      "outputs": ["{options.outputPath}"],
      "options": {
        "outputPath": "dist/libs/shared/schemas",
        "main": "libs/shared/schemas/src/index.ts",
        "tsConfig": "libs/shared/schemas/tsconfig.lib.json",
        "assets": [
          "libs/shared/schemas/*.md",
          "libs/shared/schemas/src/lib/*.{json,js}"
        ]
      },
      "dependsOn": ["bundle"]
    },
    "bundle": {
      "executor": "nx:run-commands",
      "cache": true,
      "inputs": ["internals"],
      "outputs": ["{projectRoot}/src/lib/*.json"],
      "options": {
        "command": "npx ts-node libs/shared/schemas/internals/bundle.ts",
        "cwd": "."
      }
    }
  },
  "tags": ["platform:shared", "type:core"]
}
```

### ts-interfaces

[Link to the library](https://github.com/getlarge/nx-implicit-dependencies/tree/main/libs/shared/ts-interfaces)

The `ts-interfaces` library is composed of auto-generated Typescript interfaces. A custom script, located in `internals` for the same reason as the `schemas` library, loads the JSON schemas using the File system API instead of imports to avoid including `schemas` dependencies to generate the interfaces.

The drawback is that Nx can't detect the dependency between `schemas` and `ts-interfaces`; let’s see how to declare the relationship between those two libraries by examining the `project.json`. It contains a configuration similar to the `schemas` project with the following noticeable additions:

- The `implicitDependencies` declare a dependency on the `schemas` library, which creates the missing link in the project graph
- In the `bundle` target, `dependsOn` is now set to `^bundle`, meaning all `bundle` tasks from the dependencies will run first when calling `nx run ts-interfaces:bundle`
- The `bundle.inputs` now reference the `internals` named input from the current library
- The `bundle.outputs` now reference the path (glob) of the generated Typescript interfaces

```json
// libs/ts-interfaces/project.json
{
  "name": "ts-interfaces",
  "$schema": "../../../node_modules/nx/schemas/project-schema.json",
  "sourceRoot": "libs/shared/ts-interfaces/src",
  "projectType": "library",
  "namedInputs": {
    "internals": ["{projectRoot}/internals/*.ts"]
  },
  "implicitDependencies": ["schemas"],
  "targets": {
    "build": {
      "executor": "@nx/js:tsc",
      "outputs": ["{options.outputPath}"],
      "options": {
        "outputPath": "dist/libs/shared/ts-interfaces",
        "main": "libs/shared/ts-interfaces/src/index.ts",
        "tsConfig": "libs/shared/ts-interfaces/tsconfig.lib.json",
        "assets": ["libs/shared/ts-interfaces/*.md"]
      },
      "dependsOn": ["bundle"]
    },
    "bundle": {
      "executor": "nx:run-commands",
      "cache": true,
      "dependsOn": ["^bundle"],
      "inputs": ["^internals"],
      "outputs": ["{projectRoot}/src/lib/*.ts"],
      "options": {
        "command": "npx ts-node --project libs/shared/ts-interfaces/tsconfig.editor.json libs/shared/ts-interfaces/internals/bundle.ts",
        "cwd": "."
      }
    }
    // ...
  },
  "tags": ["platform:js", "type:core"]
}
```

After applying these changes, the project graph contains the expected relationship between the `schemas` and `ts-interfaces` libraries.

![Projects graph after](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/s3ug3qp4g2jtlivqcdoc.png)

And the tasks graph shows that the `schemas:bundle` task is now part of the `coffee-dealer:build` pipeline.

![Tasks graph after](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fprxrxgl2fxbhew3jldb.png)

### Enable cache for the bundle tasks

There are two ways to enable caching of tasks :

1. on a project basis by setting the `cache` property to `true` and the `inputs` to cache in the target, as shown previously for the `schemas` and `ts-interfaces` libraries
2. globally, by setting the `bundle` default values in `targetDefaults` and declaring `internals` in `namedInputs` in `nx.json`

```json
// nx.json
{
  "$schema": "./node_modules/nx/schemas/nx-schema.json",
  "targetDefaults": {
    // ...
    "bundle": {
      "cache": true,
      "dependsOn": ["^bundle"],
      "inputs": ["^internals"]
    }
    // ...
  },
  "namedInputs": {
    // ...
    "internals": [
      "{projectRoot}/internals/**/*.[jt]s",
      "{projectRoot}/internals/**/*.json"
    ]
    // ...
  }
}
```

> **Tips**
>
> The technique above allows you to declare a global rule for the cache and the named inputs. It is useful when you have many projects sharing the same configuration. This can also be applied to executors, such as `@nx/jest:jest` or `@nx/eslint:lint`.
> Have a look at the [Nx docs](https://nx.dev/core-features/cache-task-results), to learn how to further tweak the cache rules

Our case is now closed, and we can move on to the next one.

## Case 2: The software developer's optimism

This story is another case of implicit dependencies impacting developers' experience. I reproduced a similar situation with the following setup:

- `coffee-dealer`, is a NestJS application serving an HTTP API on port 3000
- `coffee-dealer-cli`, is a CLI tool to interact with `coffee-dealer` through its HTTP API
- `coffee-dealer-e2e`, is an end-to-end test suite sending HTTP requests to `coffee-dealer`

In this case, the developers would like to ensure that when they work locally on the `coffee-dealer` application with the watch mode enabled, the `coffee-dealer-cli` and `coffee-dealer-e2e` are always running against the latest build of the `coffee-dealer` application.

You can see the current relation between the projects in the project graph:

![Projects graph before](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/e11rbaltb7roa77dq4wb.png)

What can we deduct from this graph?

1. We can see an implicit dependency between `coffee-dealer` and `coffee-dealer-e2e`, created by default by the NestJS generator plugin, and no dependency between `coffee-dealer` and `coffee-dealer-cli`. There is no task dependency between the `coffee-dealer-e2e:e2e` task and the `coffee-dealer:serve` task, meaning that the end-to-end tests could run before the application is ready. It is the same for the `coffee-dealer-cli:run` task, which could run before the application's API is available. With Nx, it is currently impossible to declare a task dependency for a long-running task, such as `serve`, so we must find an alternative solution. See related issues: [nx issue #5570](https://github.com/nrwl/nx/issues/5570) and [nx issue #3748](https://github.com/nrwl/nx/issues/3748).
2. When running the application with `nx run coffee-dealer:serve`, it starts with the watch mode enabled. The watcher does not instantly detect changes; there is a delay (which depends on the compiler or task manager implementation and/or configuration) between the file(s) change detection and the build completion. It means there is a small window where the CLI or the end-to-end tests could run against a previous build version.

### coffee-dealer-e2e

[Link to the application](https://github.com/getlarge/nx-implicit-dependencies/tree/main/apps/coffee-dealer-e2e)

To dramatically decrease the risk of encountering the abovementioned issues, we can :

- Create a new task `check-port` and add it to `dependsOn` of the tasks that should check whether the `coffee-dealer` port is open before running the end-to-end tests
- Declare an extra `namedInput` to use the output of the `coffee-dealer:build` task as a cache input for the `coffee-dealer-e2e:e2e` task, using the [`dependentTasksOutputFiles` property](https://nx.dev/reference/inputs#outputs-of-dependent-tasks)
- Add a dependency on the `build` tasks to create a race condition between the `coffee-dealer:build` tasks that `coffee-dealer:serve` and `coffee-dealer-e2e:e2e` depend on.

```json
// apps/coffee-dealer-e2e/project.json
{
  "name": "coffee-dealer-e2e",
  "$schema": "../../node_modules/nx/schemas/project-schema.json",
  "implicitDependencies": ["coffee-dealer"],
  "namedInputs": {
    "coffee-dealer": [
      {
        "dependentTasksOutputFiles": "{workspaceRoot}/dist/apps/coffee-dealer/**/*.js",
        "transitive": false
      }
    ]
  },
  "projectType": "application",
  "targets": {
    "e2e": {
      "executor": "@nx/jest:jest",
      "dependsOn": ["build", "check-port"],
      "inputs": ["^production", "coffee-dealer"],
      "outputs": ["{workspaceRoot}/coverage/{e2eProjectRoot}"],
      "options": {
        "jestConfig": "apps/coffee-dealer-e2e/jest.config.ts",
        "passWithNoTests": true
      },
      "configurations": {
        "ci": {
          "ci": true
        },
        "development": {}
      }
    },
    "check-port": {
      "executor": "nx:run-commands",
      "cache": false,
      "options": {
        "command": "lsof -i:${PORT:-3000} -t"
      }
    },
    "lint": {
      "executor": "@nx/eslint:lint",
      "outputs": ["{options.outputFile}"]
    }
  }
}
```

Finally, ensure that `coffee-dealer-e2e:e2e` and `coffee-dealer:serve` will use the `development` configuration to use the same build output by setting the `defaultConfiguration` to `development` in the `serve` and `build` targets of the `coffee-dealer` project.

```json
// apps/coffee-dealer/project.json
{
  "name": "coffee-dealer",
  "$schema": "../../node_modules/nx/schemas/project-schema.json",
  "sourceRoot": "apps/coffee-dealer/src",
  "projectType": "application",
  "targets": {
    "build": {
      "executor": "@nx/webpack:webpack",
      "outputs": ["{options.outputPath}"],
      "defaultConfiguration": "development",
      "options": {
        "target": "node",
        "compiler": "tsc",
        "transformers": [
          "typia/lib/transform",
          {
            "name": "@nestia/core/lib/transform",
            "options": {
              "validate": "assert",
              "stringify": "assert"
            }
          }
        ],
        "outputPath": "dist/apps/coffee-dealer",
        "main": "apps/coffee-dealer/src/main.ts",
        "tsConfig": "apps/coffee-dealer/tsconfig.app.json",
        "assets": ["apps/coffee-dealer/src/assets"],
        "webpackConfig": "apps/coffee-dealer/webpack.config.js"
      },
      "configurations": {
        "development": {},
        "production": {}
      }
    },
    "serve": {
      "executor": "@nx/js:node",
      "defaultConfiguration": "development",
      "options": {
        "buildTarget": "coffee-dealer:build"
      },
      "configurations": {
        "development": {
          "buildTarget": "coffee-dealer:build:development"
        },
        "production": {
          "buildTarget": "coffee-dealer:build:production"
        }
      }
    }
    // ...
  },
  "tags": ["type:app", "platform:node"]
}
```

We can now check that the task pipeline is correct :

![Tasks graph after](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/87f7r698xsdrxztkvxvl.png)

> **Note**
>
> In this tasks graph, even though the `coffee-dealer:serve` task is not directly added to the `coffee-dealer-e2e:e2e` pipeline, they will both produce and consume the same build artifacts. Which is how we implictly tie the `coffee-dealer-e2e:e2e` task to the `coffee-dealer:serve` task.

---

We can test the solution with the script `test-race-watch.js`, which emulates a user's behavior. The goal is to demonstrate that it is possible to ensure that the e2e tests always run against the latest application version.

The script will change an API response from the `coffee-dealer` app and revert it after a few seconds, triggering a rebuild of the application. The e2e tests will run before and after each rebuild, and we should see the following output:

- e2e tests will fail when the API is changed, and the task won't use the cache
- e2e tests will pass when the API change is reverted, and the tasks will use the cache (when it is populated) for all tasks

```js
// test-race-watch.js
const { exec } = require('node:child_process');
const { assert } = require('node:console');
const { readFile, writeFile } = require('node:fs').promises;
const { join } = require('node:path');
const { PerformanceObserver, performance } = require('node:perf_hooks');
const { once } = require('node:stream');
const { format } = require('node:util');

const coffeeDealerAppServiceTsPath = join(
  __dirname,
  'apps',
  'coffee-dealer',
  'src',
  'app',
  'app.service.ts'
);
const port = 3010;
let counter = 0;
let isOriginalApi = true;

function runE2ETests(suite = '#1') {
  const shouldFail = !isOriginalApi;
  console.warn(
    '➡️ running E2E tests suite %s %s',
    suite,
    shouldFail ? 'should fail' : 'should pass'
  );
  const coffeeDealerE2EProcess = exec('npx nx run coffee-dealer-e2e:e2e', {
    env: { ...process.env, PORT: port },
  });

  coffeeDealerE2EProcess
    .once('spawn', () => {
      performance.mark(format('e2e-start:%s', suite));
      performance.mark(format('e2e-deps-start:%s', suite));
    })
    .once('exit', (code, signal) => {
      console.warn('🏁 test finished with code %s and signal %s', code, signal);
      performance.mark(format('e2e-end:%s', suite));
      performance.measure(
        format('e2e-run:%s', suite),
        format('e2e-start:%s', suite),
        format('e2e-end:%s', suite)
      );
      assert(
        shouldFail ? code !== 0 : code === 0,
        `e2e tests ${suite} should ${shouldFail ? 'fail' : 'pass'}`
      );
    });

  coffeeDealerE2EProcess.stdout.setEncoding('utf8').on('data', (data) => {
    // uncomment to see the output of the e2e tests task
    // console.log(data.trim());
    if (data.includes('Setting up')) {
      performance.mark(format('e2e-deps-end:%s', suite));
      performance.measure(
        format('e2e dependencies tasks:%s', suite),
        format('e2e-deps-start:%s', suite),
        format('e2e-deps-end:%s', suite)
      );
    }
  });
  coffeeDealerE2EProcess.stderr.pipe(process.stderr);
}

async function updateAppService() {
  const data = await readFile(coffeeDealerAppServiceTsPath, 'utf8');
  /**
   * Hello API is the value that should be returned by the API and what the e2e tests expect
   */
  const index = data.search('Hello API');
  let updatedData = '';
  if (index > 0) {
    // in that case e2e tests will fail
    updatedData = data.replaceAll('Hello API', 'Hello World!');
    isOriginalApi = false;
  } else {
    // in that case e2e tests will pass
    updatedData = data.replaceAll('Hello World!', 'Hello API');
    isOriginalApi = true;
  }

  console.warn(
    '🎬 updating API to %s version',
    isOriginalApi ? 'original' : 'new'
  );
  runE2ETests('#1');

  /**
   * should trigger build and reload of cofee-dealer app
   **/
  writeFile(coffeeDealerAppServiceTsPath, updatedData).then(() => {
    counter++;
    console.warn('➡️ updated API #%d', counter);
    runE2ETests('#3');
  });

  runE2ETests('#2');
}

function main() {
  const obs = new PerformanceObserver((items) => {
    console.log(
      'task %s duration : %d ms',
      items.getEntries()[0].name,
      items.getEntries()[0].duration
    );
    performance.clearMarks(items.getEntries()[0].name);
  });
  obs.observe({ type: 'measure' });

  const coffeeDealerServerProcess = exec(
    'npx nx run coffee-dealer:serve --inspect=false',
    {
      env: {
        ...process.env,
        PORT: port,
      },
    }
  );
  // uncomment to see the output of the server
  // coffeeDealerServerProcess.stdout.pipe(process.stdout);
  coffeeDealerServerProcess.stderr.pipe(process.stderr);
  once(coffeeDealerServerProcess, 'exit').then(([code, signal]) =>
    console.log('coffeeDealerServerProcess exit', code, signal)
  );

  /**
   * update API every 3-7 seconds
   */
  setInterval(() => {
    updateAppService();
  }, (Math.floor(Math.random() * 4) + 3) * 1000);
}

main();
```

### coffee-dealer-cli

[Link to the application](https://github.com/getlarge/nx-implicit-dependencies/tree/main/apps/coffee-dealer-cli)

We can apply the same techniques to the `coffee-dealer-cli` application by applying similar changes to the `project.json` file.

## Key Takeaways

To conclude our cases, Nx offers many ways to manage implicit internal and external dependencies. We can declare them in the `project.json` file, use named inputs to cache the output of a task, and use the `dependsOn` property to create a task pipeline. We can also use the `targetDefaults` property in the `nx.json` file to set global rules for the cache and the named inputs.

I hope this article has helped you understand the importance of managing implicit dependencies in your workspace or inspired you to improve it. If you have any questions or feedback, feel free to reach out to me on [Twitter](https://twitter.com/e_maleix) or [Linkedin](https://www.linkedin.com/in/edouard-maleix/).

In the next episode, I plan to explore the usage of [`nx watch`](https://nx.dev/recipes/running-tasks/workspace-watching) to observe the workspace changes and create a custom tasks pipeline triggered based on the changes.