AWS CDK Pipelines: Real-World Tips and Tricks — Part 1
By: Jels Boulangier
Tags: AWS, Data Engineering, Infrastructure As Code, Ci Cd Pipeline, Cloud Computing

The AWS Cloud Development Kit (AWS CDK) is an open source software development framework to define your cloud application resources using familiar programming languages. This latter is the biggest difference from other infrastructure-as-code tools, which use configuration files and special-purpose languages. AWS CDK is a framework which, under the hood, creates AWS CloudFormation templates to deploy AWS resources. The power of AWS CDK is that it allows you to construct an AWS CloudFormation template of more than 500 lines that deploys 50 AWS resources with a mere 20 lines of code! In addition to the high level of abstraction and default configuration, you can include programmatic logic when defining your infrastructure, use object-oriented techniques, and create and share libraries. Moreover, you can take advantage of code completion tools in your IDE since AWS CDK supports familiar programming language such as TypeScript, JavaScript, Python, Java, and C#/.Net.

… deploys 50 AWS resources with a mere 20 lines of code!

In July 2020, the AWS CDK development team made automatic deployment of infrastructure even better by introducing AWS CDK Pipelines. Before this, deploying CDK applications to multiple AWS accounts and regions was definitely doable, however, it needed some custom scripts to tie everything together and increase automation. Here is where CDK Pipelines comes into the picture. It increases the practicality of automated, multi-account, multi-region deployment by integrating a CDK application into a continuous delivery pipeline using AWS CodePipelines. The pipeline itself is also created with the CDK framework. In addition to the whole integrated system, which is purely defined in code, the pipeline created by CDK Pipelines is self-mutating! This means that you only need to deploy the pipeline once to get it started and, after that, the pipeline automatically updates itself when you add new parts to the CDK application’s source code. With CDK Pipelines, teams can easily create and share “pipelines-as-code” patterns for deploying applications and infrastructure across AWS accounts and regions.

Even though CDK Pipelines is still in developer preview and subjected to major changes, you can already use it in real-world applications. But be aware that because this libraries is still in its infancy, the community has yet to figure out what the best practices are for creating complex applications. Current documentation and demos of CDK Pipelines, although useful, are limited to simple minimum viable products. Because of this, I want to share my tips and tricks which I’ve put together by working on a project at my current job and through interaction with the CDK community.

The next sections in this article all answer a specific question which I had — and you most likely will too. All of them can be considered as stand-alone and follow the simple format: “How do I do this?” These sections are not chronological, nor do they explain the basics of CDK Pipelines.

How do I structure my CDK Pipelines application?

There is no one way you need to structure your CDK Pipelines application. However, through trial and error, and community interaction, a good one is to have both application code and infrastructure code in the same project/repository where each of these is located in a separate directory. Meaning that at root level, your project looks like this:

.

├── code

└── infrastructure

The code directory contains all files, information, configuration,… for the application code. Likewise, all CDK/infrastructure related files are in the infrastructure directory. Keep these two distinct directories even if the project is only infrastructure and no application code. It ensures standardisation across projects and application code might be introduced in future development. Additionally, the root directory contains hidden files/directories which are editor specific, a .gitignore file, a README.md file,…

How do I deploy cross-environment?

All resources which need to be deployed to the same environment — an AWS account and region combination — should be in the same Stage . Next, the resources in this stage can be deployed to multiple environments via configuration when creating the stage object. In order to keep an oversight, determining to which environment to deploy should happen at the PipelineStack level and not within a Stage itself. How exactly you create the configuration of multi-environments is up to you since this is often a personal preference (and I’m not yet satisfied with my approach 🤔. Edit: In the meantime, I’ve written a second article about CDK Pipelines in which I address a useful configuration setup.).

This is an example of deploying resources to four different AWS environments. Yet, only two distinct stages are created, Mystage and MyStageUsEast , that each will deploy different resources (the second one deploys to the us-east-1 region). Deployment of these two stages to the spike and development environments only differ by passing a different configuration object.