Creating a Simple Generator for Nx Plugins
Tags: nx, plugins, typescript
Published: 2023-05-30 20:16:22

## Introduction

Welcome to the second blog post in our series on creating custom Nx plugins! This post targets developers who are familiar with the basics of the Nx ecosystem but have never built a plugin for Nx before. In this article, we'll dive deep into generators, learn how to set up a development environment, and create a simple generator for an Nx plugin. Let's get started!

### Table of Contents

1. Deep Dive into Generators
2. Setting Up the Development Environment
3. Creating a Simple Generator
4. Working with the Nx Devkit Tree
6. Summary

## Deep Dive into Generators

Nx Generators are an integral part of the Nx ecosystem, providing an interface for code generation and modification. They operate by reading and altering the Abstract Syntax Tree (AST) of your codebase, enabling precise and flexible modifications. This fine-grained control fosters creation of tailored development environments that reflect the unique needs of your project.

At its core, a generator is a function that accepts a `Tree` and an options object. The `Tree` represents your filesystem, and you interact with it to make file changes. The options object carries the values provided by the user when invoking the generator.

A fundamental concept in understanding Nx generators is their immutability. All changes you make to the `Tree` within a generator are staged, and not immediately applied. This staged approach allows changes to be grouped into an atomic commit, ensuring your filesystem stays in a consistent state even if the generator fails partway through execution.

To create a custom generator, you define a generator function in a file named `generator.ts`. The function receives a host `Tree` and an options object. For complex generators, Nx provides the `generateFiles` utility. This powerful utility employs schematics-like template syntax to interpolate variables into file contents and filenames.

For input control, Nx uses JSON Schema, allowing generators to accept complex inputs, validate them, and provide interactive prompts to the users. Defining a schema makes your generator self-documenting, as Nx can automatically generate descriptions, validations, and prompts based on it.

Generators can invoke other generators, allowing code reuse and composition. Nx provides helper functions like `runTasksInSerial` or `runTasksInParallel` to manage the lifecycle of these generator executions.

Testing generators is crucial, and Nx's virtual file system makes this easy. During tests, you can verify that the right changes are being applied without affecting your actual filesystem.

In conclusion, Nx Generators provide a powerful way to automate your development tasks, creating a robust, repeatable, and efficient development workflow. 

## Setting Up the Development Environment

Creating an Nx workspace and a custom plugin begins by using the `create-nx-workspace` utility. Let's create a new workspace named `nx-tools`.

```bash
npx create-nx-workspace@latest nx-tools
```

Select "empty" when prompted for a preset to create a bare workspace.

Now, incorporate the `@nx/nx-plugin` into your workspace. The `@nx/nx-plugin` furnishes essential tools for creating and managing Nx plugins.

```bash
nx add @nx/nx-plugin
```

With the Nx Plugin established, we'll now generate our custom plugin, named `nx-cdk`, in our workspace. This plugin will be designed to generate, build, and deploy AWS CDK projects, facilitating efficient development in our Nx monorepo.

```bash
nx g @nx/nx-plugin:plugin nx-cdk
```

This command creates a new plugin with the name `nx-cdk` and provides the preliminary code and configuration.

Finally, open your freshly created workspace in your preferred code editor. For Visual Studio Code users, this can be done with:

```bash
code nx-tools
```

With these steps completed, your development environment is primed for crafting your `nx-cdk` plugin.

## Creating a Simple Generator for our nx-cdk Plugin

Now that the development environment for `nx-tools` is set up, we can begin with creating a simple generator for our `nx-cdk` plugin. The generator will help scaffold AWS CDK applications.

Nx Devkit provides a built-in schematic to help generate the boilerplate for our generator. Run the following command to create the generator:

```bash
nx g @nrwl/nx-plugin:generator NxCdk --project=nx-cdk
```

This command does several things:

1. It adds a new generator named `nx-cdk` in the `nx-cdk` plugin.
2. It creates a new `nx-cdk` directory inside the generators directory of our nx-cdk plugin.
3. Inside the `generators/nx-cdk` directory, it creates a `generator.ts` file which is the main file for our generator, a `generator.spec.ts` file which contains the test cases for our generator, a `schema.json` file which will be used to define the options for our generator and a `schema.d.ts` file containing the interface definition for our schema.

Open the `schema.json` and modify the content as per your requirements:

```json
  "$schema": "http://json-schema.org/schema",
  "$id": "NxCdk",
  "title": "",
  "type": "object",
  "properties": {
    "name": {
      "type": "string",
      "description": "",
      "$default": {
        "$source": "argv",
        "index": 0
      },
      "x-prompt": "What name would you like to use?"
    },
    "tags": {
      "type": "string",
      "description": "Add tags to the project (used for linting)",
      "alias": "t"
    },
    "directory": {
      "type": "string",
      "description": "A directory where the project is placed",
      "alias": "d"
    }
  },
  "required": [
    "name"
  ]
}
```

Next, update the `generator.ts` file with the code to scaffold a new AWS CDK project in your workspace:

```typescript
import { readFileSync } from 'fs';
import * as path from 'path';
import { resolve } from 'path';

import {
    GeneratorCallback,
    Tree,
    addDependenciesToPackageJson,
    addProjectConfiguration,
    formatFiles,
    generateFiles,
    getWorkspaceLayout,
    names,
    offsetFromRoot
} from '@nx/devkit';
import { jestProjectGenerator } from '@nx/jest';
import { Linter, lintProjectGenerator } from '@nx/linter';
import { runTasksInSerial } from '@nx/workspace/src/utilities/run-tasks-in-serial';

import {
    awsCdkLibVersion,
    awsCdkVersion,
    constructsVersion,
    sourceMapSupportVersion,
    tsJestVersion
} from '../../utils/versions';
import { addJestPlugin } from './lib/add-jest-plugin';
import { addLinterPlugin } from './lib/add-linter-plugin';
import { NxCdkGeneratorSchema } from './schema';

interface NormalizedSchema extends NxCdkGeneratorSchema {
    projectName: string;
    projectRoot: string;
    projectDirectory: string;
    parsedTags: string[];
}

function normalizeOptions(tree: Tree, options: NxCdkGeneratorSchema): NormalizedSchema {
    const name = names(options.name).fileName;
    const projectDirectory = options.directory ? `${names(options.directory).fileName}/${name}` : name;
    const projectName = projectDirectory.replace(new RegExp('/', 'g'), '-');
    const projectRoot = `${getWorkspaceLayout(tree).appsDir}/${projectDirectory}`;
    const parsedTags = options.tags ? options.tags.split(',').map((s) => s.trim()) : [];

    return {
        ...options,
        projectName,
        projectRoot,
        projectDirectory,
        parsedTags
    };
}

function addFiles(tree: Tree, options: NormalizedSchema) {
    const templateOptions = {
        ...options,
        ...names(options.name),
        offsetFromRoot: offsetFromRoot(options.projectRoot),
        template: ''
    };
    generateFiles(tree, path.join(__dirname, 'files'), options.projectRoot, templateOptions);
}

export default async function (tree: Tree, options: NxCdkGeneratorSchema) {
    const tasks: GeneratorCallback[] = [];
    const normalizedOptions = normalizeOptions(tree, options);
    addProjectConfiguration(tree, normalizedOptions.projectName, {
        root: normalizedOptions.projectRoot,
        projectType: 'application',
        sourceRoot: `${normalizedOptions.projectRoot}/src`,
        targets: {
            bootstrap: {
                executor: '@myorg/nx-cdk:bootstrap'
            },
            deploy: {
                executor: '@myorg/nx-cdk:deploy'
            },
            destroy: {
                executor: '@myorg/nx-cdk:destroy'
            },
            diff: {
                executor: '@myorg/nx-cdk:diff'
            },
            ls: {
                executor: '@myorg/nx-cdk:ls'
            },
            synth: {
                executor: '@myorg/nx-cdk:synth'
            }
        },
        tags: normalizedOptions.parsedTags
    });
    addFiles(tree, normalizedOptions);
    tasks.push(addJestPlugin(tree));
    tasks.push(addLinterPlugin(tree));
    tasks.push(addDependencies(tree));
    await lintProjectGenerator(tree, { project: options.name, skipFormat: true, linter: Linter.EsLint });
    await jestProjectGenerator(tree, {
        project: options.name,
        setupFile: 'none',
        skipSerializers: true,
        testEnvironment: 'node'
    });
    await ignoreCdkOut(tree);
    await formatFiles(tree);
    return runTasksInSerial(...tasks);
}

function addDependencies(tree: Tree) {
    const dependencies: Record<string, string> = {};
    const devDependencies: Record<string, string> = {
        'aws-cdk': awsCdkVersion,
        'aws-cdk-lib': awsCdkLibVersion,
        constructs: constructsVersion,
        'source-map-support': sourceMapSupportVersion,
        'ts-jest': tsJestVersion
    };
    return addDependenciesToPackageJson(tree, dependencies, devDependencies);
}

async function ignoreCdkOut(tree: Tree) {
    const ignores = readFileSync(resolve(tree.root, '.gitignore'), { encoding: 'utf8' }).split('\n');
    if (!ignores.includes('cdk.out')) {
        ignores.push('# AWS CDK', 'cdk.out', '');
    }
    tree.write('./.gitignore', ignores.join('\n'));
}
```

Here's the explanation of what the `generator.ts` file does:

1. `NormalizedSchema` Interface: A TypeScript interface is defined that extends the NxCdkGeneratorSchema interface from `schema.d.ts`. It adds properties like `projectName`, `projectRoot`, `projectDirectory`, and `parsedTags`.
2. `normalizeOptions` Function: It accepts a Tree and generator options and returns the options after performing transformations like naming conventions, file path conventions, and tag parsing.
3. `addFiles` Function: This function generates files from the provided template and places them in the project directory.
4. The Main Generator Function: It's an async function that performs several tasks:
  - Normalize the options using the `normalizeOptions` function.
  - Add project configuration using the `addProjectConfiguration` function.
  - Generate files using the `addFiles` function.
  - Add Jest and Linter plugins to the Nx tree.
  - Add dependencies to the `package.json` file using the `addDependenciesToPackageJson` function.
  - Setup Jest and ESLint for the newly generated project.
  - Ignore the `cdk.out` directory in `.gitignore`.
  - Finally, format all the generated files.
5 `addDependencies` Function: Adds dependencies required for the AWS CDK project, like `aws-cdk`, `aws-cdk-lib`, `constructs`, `source-map-support`, and `ts-jest` to the `devDependencies` of `package.json`.
6. `ignoreCdkOut` Function: Reads the `.gitignore` file and checks if 'cdk.out' is included in the ignore list. If not, it adds 'cdk.out' to the list, which prevents the generated AWS CDK output files from being tracked by git.

The `schema.d.ts` file is where we define the TypeScript interface for our generator options. This interface helps TypeScript to understand the structure of the options and provide type checking. In our case, the interface for the `NxCdkGeneratorSchema` should look like this:

```typescript
export interface NxCdkGeneratorSchema {
    name: string;
    tags?: string; 
    directory?: string; 
}
```

This TypeScript interface aligns with the `schema.json` file and provides a typed way of accessing the generator options.

Testing is a critical part of any software development, and Nx generators are not an exception. Nx provides a built-in way to test generators using Jest. When we created our `NxCdk` generator, Nx also created a `generator.spec.ts` file in the `nx-cdk` directory. This is a test specification file for our generator.

```typescript
import { Tree } from '@nrwl/devkit';
import * as testingUtils from '../../../utils/testing';

import generator from './generator';
import { NxCdkGeneratorSchema } from './schema';

describe('myCdkApp generator', () => {
  let appTree: Tree;
  const options: NxCdkGeneratorSchema = { name: 'test' };

  beforeEach(() => {
    appTree = testingUtils.createEmptyWorkspace(Tree.empty());
  });

  it('should run successfully', async () => {
    await expect(
      generator(appTree, options)
    ).resolves.not.toThrowError();
  });
});
```

In this test case, we first create an empty Nx workspace. We then run our generator with a given set of options. The `expect` function is then used to assert that our generator should run without throwing any errors.

## Working with the Nx Devkit Tree

The Nx Devkit `Tree` object represents the workspace's file system. It allows you to read, write, and manipulate files and directories. In our simple generator, we use the `generateFiles` function to create files from templates located in the `./files` directory.


![File templates in files directory](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5qrh4rbwn5m34fkj03ty.png)



## Registering the Generator

Register the generator in `nx-cdk/generators.json`:

```json
{
  "$schema": "http://json-schema.org/schema",
  "name": "nx-cdk",
  "version": "0.0.1",
  "generators": {
    "nx-cdk": {
      "factory": "./src/generators/nx-cdk/generator",
      "schema": "./src/generators/nx-cdk/schema.json",
      "description": "nx-cdk generator"
    }
  }
}
```

## Summary

The blog post provides an in-depth guide on creating a simple generator for Nx plugins. It introduces the Nx ecosystem and the concept of generators, functions that manipulate your codebase via the Abstract Syntax Tree (AST). The key idea is the immutability of generators, where changes are staged and grouped into atomic commits for filesystem consistency.

The post presents a step-by-step tutorial on setting up an Nx workspace and crafting a custom Nx plugin. This involves creating a new workspace, incorporating the @nx/nx-plugin, generating a custom plugin, and opening the workspace in your code editor.

The main tutorial focuses on creating a generator for an nx-cdk plugin designed to generate, build, and deploy AWS CDK projects. This process includes running commands to create the generator, modifying the schema.json file to fit your requirements, and updating the generator.ts file with code to scaffold a new AWS CDK project.

Alright, you've made it this far and hopefully enjoyed the ride. The next stop on our journey will be implementing an executor. So grab some popcorn and stay tuned - I promise it will be just as fun, if not more! See you next time, folks!

----------

_Hey there, dear readers! Just a quick heads-up: we're code whisperers, not Shakespearean poets, so we've enlisted the help of a snazzy AI buddy to jazz up our written word a bit. Don't fret, the information is top-notch, but if any phrases seem to twinkle with literary brilliance, credit our bot. Remember, behind every great blog post is a sleep-deprived developer and their trusty AI sidekick._