How to Create Awesome Repeatable Project Setups for AWS CDK
Tags: aws, productivity, typescript, cdk
Published: 2021-09-13 08:01:42

Are you excited about using **AWS Cloud Development Kit (AWS CDK)** to define infrastructure-as-code?

Do you think that the AWS CDK project initialization is a bit clunky and does not give you what **you** need to hit the ground running with a new AWS CDK project?

Do you use Typescript? (by choice or by request)

If most of these have the answer yes, you do want to **continue reading this article!**

_(Cover photo by <a href="https://unsplash.com/@thisisengineering?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">ThisisEngineering RAEng</a> on <a href="https://unsplash.com/s/photos/productivity-coding-aws-infrastructure-as-code?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>)_

<!-- more -->

## What we want to accomplish

The goal is to make it easy and quick to set up a useful and productive project setup for use with AWS CDK.
A developer should **within a few minutes** have a setup that includes:

* Typescript compiler configuration
* Linting tool set-up installed and configured
* Unit-testing tooling in place and configured
* Sample code structure to start with
* Git project initiated
* AWS CDK base modules installed
* Workflows in place for validation and release (Github)
* Workflow for release to package registry (npm)
* Pull request templates in place (Github)
* Simplified project configuration

The list above is _only a subset of features_ that we will have in place, but we will not go into all of them.

### The magic sauce to make this happen is...

Most of the features mentioned above we will get from the tool [Projen](https://github.com/projen/projen/).
We will use that as a foundation to enable a project set-up that is **relevant to you and your organization**.
You can also read about Projen in [How to simplify project setup with Projen](https://tidycloudaws.com/articles/how-to-simplify-project-setup-projen/), and other articles at [Tidy Cloud AWS](https://tidycloudaws.com).

We will build our Projen-based project type, which you can use as a starting point for your project types.

You should have a _simple custom project type in place_ in **about 30-60 minutes ready to use by others** if you follow the steps outlined here.

I do recommend that you **first skim through the article here to get a gist of what it is all about**.
After that, you can go through the steps in more detail.

![AWS CDK icon](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hh0ro7pn34fke581ew18.png)

### Why use a tool like Projen?

Chance is that you, like I used to, _copied various files from an old project whenever a new project should be set up_.
If a colleague or friend needed a starting point, you just **pointed to a repository (possibly a template repository) to copy from**.

However, once you had copied the files from another repository, **you were on your own**.
You had to _understand all the details that needed to be changed_ and _you had to update these yourself_.

With the project set-up we do here, maintenance of the set-up is _much easier_.
You do **not need to understand and manage all nitty-gritty details**, if you do not want to.

In the same spirit as _infrastructure-as-code_ encourages scripted infrastructure set-up & maintenance, **Projen** encourages _scripted project set-up and maintenance_, i.e. **project-setup-as-code**.

Now that we have our goals defined, let us get started!

## The stuff that you will need to start with

There are three things that we will need to create a project set-up for developers to use.
These three things also happen to be what the developers that use the set-up will need as well.

* **Node.js** - You can download and install from [tthe Node web site](https://nodejs.org/en/). I recommend using the current long-term release (version 14.x).
* **yarn** - One of the Typescript/Javascript package managers. Install this globally after installation Node.js, for example running `npm install yarn --global`.
  You can use **npm** instead of **yarn**.
  However, the default is yarn in Projen, so we stick to that for simplicity, for now.
* **git client** - Download from here: <https://git-scm.com>

In addition, you should also have a [Github account](https://github.com/).
We will have workflows enabled for Github defined in our project set-up, so an account there is highly desirable.

Finally, you will also need an account for an NPM registry, such as <https://www.npmjs.com>.
This is for actually publishing your project type so it is available to others.
Other registries, such as **Github packages** are also fine to use.

We will cover the publication in more detail when we get to that point, so do not worry now if you do not have any NPM registry account yet.

![AWS CDK icon](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hh0ro7pn34fke581ew18.png)

## Create our project type baseline

I assume now that you at this stage have **node.js**, **yarn**, and **git** command-line client installed, and that you have a Github account you can log in to.

1. Open a command-line window and create a directory where you will store the project type definition.
   This will become a **git repository**.
   Let us name it _awesome-awscdk-project_:

```bash
mkdir awesome-awscdk-project
cd awesome-awscdk-project
```

2. Create a project using projen for creating project types
   We can use **Projen** itself to _create a project for defining project types for Projen_...
   In our project directory, use Projen to create a project with project type **jsii**.

```bash
npx projen new jsii
```

The command **npx** is part of the _Node.js_ installation and it will run an _NPM package_ without requiring that it has been installed first.
If we use the command **new** with Projen, we can create a project of a specified project type.
The project type **jsii** is a building block to build a translation layer between Typescript/Javascript data structures and some other languages.
Projen uses some metadata description features of JSII, hence we use that project type to create our Projen project type.

The result of executing this command may look a bit like this:

```bash
❯ ls -la
total 656
drwxr-xr-x   20 erikl  staff     640 Aug 30 10:50 .
drwxr-xr-x   59 erikl  staff    1888 Aug 30 10:49 ..
-r--------    1 erikl  staff    4118 Aug 30 10:50 .eslintrc.json
drwxr-xr-x   13 erikl  staff     416 Aug 30 10:50 .git
-r--------    1 erikl  staff    1084 Aug 30 10:50 .gitattributes
drwxr-xr-x    4 erikl  staff     128 Aug 30 10:50 .github
-r--------    1 erikl  staff     831 Aug 30 10:50 .gitignore
-r--------    1 erikl  staff     431 Aug 30 10:50 .mergify.yml
-rw-r--r--    1 erikl  staff     372 Aug 30 10:50 .npmignore
drwxr-xr-x    4 erikl  staff     128 Aug 30 10:50 .projen
-rw-r--r--    1 erikl  staff     892 Aug 30 10:50 .projenrc.js
-r--------    1 erikl  staff   11358 Aug 30 10:50 LICENSE
-rw-r--r--    1 erikl  staff      14 Aug 30 10:50 README.md
drwxr-xr-x  656 erikl  staff   20992 Aug 30 10:50 node_modules
-rw-r--r--    1 erikl  staff    2898 Aug 30 10:50 package.json
drwxr-xr-x    3 erikl  staff      96 Aug 30 10:50 src
drwxr-xr-x    3 erikl  staff      96 Aug 30 10:50 test
-r--------    1 erikl  staff     827 Aug 30 10:50 tsconfig.eslint.json
-r--------    1 erikl  staff     827 Aug 30 10:50 tsconfig.jest.json
-rw-r--r--    1 erikl  staff  278145 Aug 30 10:50 yarn.lock
```

Luckily, we do not have to bother with the details of most of the files here.
There are essentially four places we care about:

* The `.projenrc.js` file - our project config
* The `src` directory - Our custom project type code
* The `test` directory - Tests, of course
* The `README.md` file - to describe what we provide here

![AWS CDK icon](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hh0ro7pn34fke581ew18.png)

## Initial updates of the .projenrc.js file

In any project type created with **Projen**, the `.projenrc.js` file is the central point of configuration.
Open this file and you will see something like this:

```javascript
const { JsiiProject } = require('projen');
const project = new JsiiProject({
  author: 'Adam Author',
  authorAddress: 'adam.author@example.com',
  defaultReleaseBranch: 'main',
  name: 'awesome-awscdk-project',
  repositoryUrl: 'https://github.com/adamauthor/awesome-awscdk-project.git',

  // deps: [],                          /* Runtime dependencies of this module. */
  // description: undefined,            /* The description is just a string that helps people understand the purpose of the package. */
  // devDeps: [],                       /* Build dependencies for this module. */
  // packageName: undefined,            /* The "name" in package.json. */
  // projectType: ProjectType.UNKNOWN,  /* Which type of project this is (library/app). */
  // release: undefined,                /* Add release management to this project. */
});
project.synth();
```

If you need, update the **author** and **authorAddress** fields.
Also, update the **repositoryUrl** to the name you want to use in Github for the repository.
For example, if you want to store in a Github organization you belong to.

The general idea here is that in this file we create an object with the appropriate project type (JsiiProject in this case).
We do project configuration with the parameter fields we pass to this object.

![AWS CDK icon](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hh0ro7pn34fke581ew18.png)

### Create a Github repository

At this point, _login to Github_ and create an empty repository with the same name as specified in the **repositoryUrl** field.

Once this is done, execute the following commands to add the Github repository as the _remote_ for your local project repository - adjust the actual repository URL as needed:

```bash
git remote add origin git@github.com:adamauthor/awesome-awscdk-project.git
git branch -M main
git push -u origin main
```

This should get the initial project set-up committed

### Our initial project updates

Let us make some simple changes to get a feel for the workflow when working with Projen-based projects.

We can remove the commented out fields in our `.projenrc.js` file, and we can add a **description** field, describing our project.

I am a fan of the Jetbrains IDEs, so I tend to use these in many projects.
In this case, let us exclude the IDE config settings from our repository, by adding `.idea/` to our `.gitignore` file.
However, we do this through `.projenrc.js`.
The updated file should look something like this:

```javascript
const { JsiiProject } = require('projen');
const project = new JsiiProject({
  author: 'Adam Author',
  authorAddress: 'adam.author@example.com',
  defaultReleaseBranch: 'main',
  name: 'awesome-awscdk-project',
  repositoryUrl: 'https://github.com/adamauthor/awesome-awscdk-project.git',
  description: 'Collection of awesome project types for setting up AWS CDK-based apps with Typescript',
  gitignore: [
    '.idea/',
  ],
});
project.synth();
```

To get these updates reflected in the generated configuration files, we run the command

```bash
npx projen
```

This command will execute the `.projenrc.js` file and perform any updates that are described there.
If you check what files have changed in the project, you may see something like this:

![Changed file list](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gkym02mpmsi7246tbxck.png)

So essentially, if we do any changes in `.projenrc.js`, we then run `npx projen` to generate the updated files.
Typically, we do not touch any configuration files directly, besides `.projenrc.js`.


![AWS CDK icon](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hh0ro7pn34fke581ew18.png)

## Defining our project type

We are going to define a very simple custom project type, which we call **awscdk-closedsource-app-ts**.

When you create an AWS CDK App project with Projen, it sets several default entries, which for the most part are good.
However, if you are building a closed source solution, some defaults may not necessarily suit you.
Two examples of such defaults are:

* The project includes an Apache 2.0 license by default, which may not necessarily be what you want.
* A project includes a [Mergify](https://mergify.io) configuration. Mergify is a nice tool, but you may not have an account for it and it is not free for closed source projects.

So as an example of a new project type, we will define our new project type to be identical to the bundled **awscdk-app-ts**, except for the default settings of these two features above.

![AWS CDK icon](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hh0ro7pn34fke581ew18.png)

### Let's make a unit test!

In the `test` directory, we can remove the sample test file `hello.test.ts` and create our test file.
Let us name that `awscdk-app-closedsource-ts.test.ts`.
Our project set-up includes the installation of the [Jest](https://jestjs.io) test framework by default, so we can define a test using that.

Here we will assume that we will have a class **AwsCdkClosedSourceTypeScriptApp**, which represents our new project type.
We will import this from our `src` directory, create a project instance from it, and check settings in it, in this case for **Mergify**:

```typescript
import { AwsCdkClosedSourceTypeScriptApp } from '../src';

describe('Check settings specific for project type', () => {
  test('Mergify is not enabled for closedSource default', () => {
    const project = new AwsCdkClosedSourceTypeScriptApp({
      cdkVersion: '1.116.0',
      defaultReleaseBranch: 'main',
      name: 'myproject',
    });

    const hasNoMergifyConfig = project.github && !project.github?.mergify;
    expect(hasNoMergifyConfig).toBeTruthy();
  });
});
```

The documentation for the classes of the bundled project types is at <https://github.com/projen/projen/>.
In this documentation, you can see that the property **github** includes a **mergify** entry, which will define if the Mergify configuration is used.
The default if the **github** entry is not specified, is that it will be included.
So in our test, we can check that this configuration is not in place, after creating a project with the mandatory parameters.

---
**NOTE**

The test above is not a very good test to be honest.
Consider it just a simple example for illustration purposes. 
In the <https://github.com/cloudgnosis/aws-cdk-app-templates> repository there is an improved version of the unit tests, which does a better job of testing the project type.
A partial extract of the code looks like this:

```typescript
import * as fs from 'fs-extra';
import { AwsCdkClosedSourceTypeScriptApp } from '../src';
// @ts-ignore
import { mkdtemp, synthSnapshot } from './testutil';


describe('Check settings specific for project type', () => {
  let tmpTestDir: string = '';
  const originalConsoleError = console.error;

  beforeEach(() => {
    tmpTestDir = mkdtemp();
    fs.ensureDirSync(tmpTestDir);
    console.error = jest.fn();
  });

  afterEach(() => {
    fs.removeSync(tmpTestDir);
    console.error = originalConsoleError;
  });

  test('Mergify is not enabled and no license file for closedSource default', () => {

    const project = new AwsCdkClosedSourceTypeScriptApp({
      outdir: tmpTestDir,
      cdkVersion: '1.116.0',
      defaultReleaseBranch: 'main',
      name: 'myproject',
    });

    const snap = synthSnapshot(project);
    expect(snap['.mergify.yml']).toBeUndefined();
    expect(snap.LICENSE).toBeUndefined();
  });

```

The test code in this case generates the output files in a temporary directory, and the result read from disk.
Tests can then be performed on the generated output, and tests are not dependent on what the project object exposes.

---

![AWS CDK icon](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hh0ro7pn34fke581ew18.png)

### Let us make our project type!

Now, the next step is that create the source for our project type.
We do that with two changes:

1. In the sample `index.ts` file, we replace its content with:

```typescript
export * from './awscdk-app-closedsource-ts';
```

2. We create a source file `awscdk-app-closedsource-ts.ts` with the following content:

```typescript
import { AwsCdkTypeScriptApp, AwsCdkTypeScriptAppOptions } from 'projen';

export interface AwsCdkClosedSourceTypeScriptAppOptions extends AwsCdkTypeScriptAppOptions {
  /**
   * If set to true, some default values are modified compared to the settings for AwsCdkTypeScriptApp
   * Specifically, the following default values are changed:
   * - licensed is false by default
   * - githubOptions.mergify is false by default
   * @default The default is true.
   */
  readonly closedSource?: boolean;
}

/**
 * Closed source CDK App, Typescript
 *
 * @pjid awscdk-closedsource-app-ts
 */
export class AwsCdkClosedSourceTypeScriptApp extends AwsCdkTypeScriptApp {
  constructor(options: AwsCdkClosedSourceTypeScriptAppOptions) {
    super({
      licensed: options.closedSource === undefined ? false : !options.closedSource,
      githubOptions: {
        ...options.githubOptions,
        mergify: options.closedSource === undefined ? false : !options.closedSource,
      },
      ...options,
    });
  }
}
```

There are a few points to make about the code here:

* For our new project type, we inherit from the existing **AwsCdkTypeScriptApp**, so we get all its capabilities "for free".
* We add an optional convenience parameter called **closedSource**. Its default is what we expect it to be with this project type.
* We add comments describing our class and fields. Projen will automatically generate API documentation for it when we build and package the project type.
* We add the metadata entry **@pjid** in the comments to the project type class **AwsCdkClosedSourceTypeScriptApp**.
  This data entry is the exposed project type name on the command line.

This is all we have to do for our custom project type, in this example.
For your project types, you will likely do more. An example is to add other default files in `src` and `test`.
In this case, I recommend that you will have a look at the classes **SampleFile**, **SampleDir**, and **Component** in Projen, as a starting point.

With the current code in place, you will likely want to check that it works.
For this, you can run

```bash
npx projen test
```

If the code is ok, then the tests will pass.

## Packaging and publishing our project type

Before we can get ready to package and publish our custom project type, we need a bit of additional configuration, plus we need an NPM account.

![AWS CDK icon](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hh0ro7pn34fke581ew18.png)

### Additional configurations

There are a few items that you may want to add:

* **Minimum node.js version**. Projen will default to v10 for the workflows, which is a bit old. Github will complain in this case. I choose to set this to v14, specifically to a version that Github accepts here.
* **Dependency to projen**. I add projen as a peer dependency, the user of the project will need to have it anyway.
* **NPM access setting**. This is an optional setting, and it depends on whether you choose to publish a _scoped package_ or not. A scoped package will by default have access set to **PRIVATE**, which may not be what you want.
* **keywords**. You may want to add some keywords to facilitate a search for your project type after publishing it.
* **scoped package**. This is an optional setting and depends on whether you want to publish a scoped package or a global package. The example below includes a scoped package name.

The `.projenrc.js` file will look like this, with these additional configurations:

```javascript
const { JsiiProject, NpmAccess } = require('projen');
const project = new JsiiProject({
  author: 'Adam Author',
  authorAddress: 'adam.author@example.com',
  defaultReleaseBranch: 'main',
  name: '@example/awesome-awscdk-project',
  repositoryUrl: 'https://github.com/adamauthor/awesome-awscdk-project.git',
  description: 'Collection of awesome project types for setting up AWS CDK-based apps with Typescript',
  minNodeVersion: '14.15.0',
  npmAccess: NpmAccess.PUBLIC,
  peerDeps: [
    'projen',
  ],
  gitignore: [
    '.idea/',
  ],
  keywords: [
    'aws',
    'cdk',
    'projen',
    'typescript',
  ],
});
project.synth();
```

Run `npx projen` to perform the needed configuration file updates.

You can now also run

```bash
npx projen build
```

which will compile, test, generate docs and perform packaging steps locally.
If this all works fine, you should be good to go for the final steps before publishing your project-type package!

![AWS CDK icon](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hh0ro7pn34fke581ew18.png)

### NPM account settings

To publish your package to the public NPM registry, you need an account there.
It is free to sign up and register an account at [NPM](https://npmjs.com), so go ahead and do that, if you do not have an account already.

An important consideration here now is if you **want to be able to publish scoped packages or not**.

A scoped package looks like **@example/awesome-awscdk-project**, rather than **awesome-awscdk-project**.
The latter name is **global**, the former is within the scope **@example**.

To publish a scoped package, you must have an _organization_ in NPM.
It is free to create an organization, as long as you only publish public packages.
If you want to publish private scoped packages, you need to pay for this capability.
In your NPM account, you have an option to add an organization:

![Add an organization in NPM](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/v57mdyu8qet3izgxbp04.png)

If you want to publish a scoped package, also make sure you update `.projenrc.js`.
Either you change the **name** field to be the name of the scoped package, or you add the field **packageName** and set that to the name of the scoped package.

The final piece of the puzzle is an **NPM access token**.
This is your way to let the automated Github workflow that will release and publish your project type access NPM.

In your NPM account, choose the **Access tokens**.
Click on **Generate new token**.

For the Github workflow, you want to use the **Automation** token type.

![Select automation token type](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bh8bez5gqotfxf18y2cm.png)

Copy the generated token, since this is the only time you will be able to get it.

![AWS CDK icon](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hh0ro7pn34fke581ew18.png)

### Github access to NPM

1. In the Github web interface, go to your project type repository.
2. Select **Settings**, then **Secrets**.
3. Click on **New repository secret**.
4. Enter the name **NPM_TOKEN**, and paste the access token value from NPM here.
5. Click on **Add secret**

![Add NPM_TOKEN repository secret](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/l7ijtmik7vdgkjpew5qz.png)


## Release and publish

With all these settings done, you can commit all changes and perform a `git push`.
If you commit to the release branch defined in the project, `main`, the defined release workflow will execute automatically.
It will build and package a release of your project. It will also label it with a version number, starting from **0.0.1**.

Any update on the main branch, or a pull request merged into the main branch, will generate a new release.
If you are on major version 0, this will be a new patch release, e.g. 0.0.1 => 0.0.2 => 0.0.3 etc.

The release workflow follows [Conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) and [Semantic versioning](https://semver.org).
For example, if the major version is > 0 and a commit message on the release branch includes "feat: some message...", then it will automatically increase the minor version number.
Other messages will just increase the patch version number.
You set the major version explicitly in the `.projenrc.js` file, with the **majorVersion** field.

![AWS CDK icon](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hh0ro7pn34fke581ew18.png)

## Using the custom project type

To use a custom project type with Projen, you simply add a `--from` option when you create a new project:

```bash
npx projen new --from @example/awesome-awscdk-project awscdk-closedsource-app-ts
```

If your package only has a single project type defined, it is possible to skip the project type name, it will be used anyway.
If you have multiple project types in the package, you have to specify its name.
If you do not specify the name in that case, it will list the available project types.

![Tidy Cloud AWS](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pdx3bxj1n43bnaezsaj1.png)

## Final remarks

I hope this guide for creating your project types was useful to you!
The Projen documentation is not crystal clear on how to do this (yet).
It is still a version 0.x project, but quite useful anyway in my opinion. 

You can see a published example at  <https://www.npmjs.com/package/@cloudgnosis/aws-cdk-app-templates>, with the source code at <https://github.com/cloudgnosis/aws-cdk-app-templates>.

This package contains multiple types, including the type defined here in this article.

**NOTE**: While you can technically delete a published package in NPM, it is discouraged and the preferred route is to deprecate it.
As far as I know, you have to do this via the **npm command-line**.

To use the **npm deprecate** command, I had to set an access token in the ~/.npmrc file, as I was not able to use the **npm login** command for this.

```bash
//registry.npmjs.org/:_authToken=myaccesstokenhere
```

Happy coding!