Running AWS CDK from a Lambda function
Tags: aws, iac, cloud, cdk
Published: 2020-11-12 09:57:18

This is a step by step guide to running AWS CDK inside an AWS Lambda Function using Lambda layers.

We will create an example CDK code, which will create an S3 bucket (can be any AWS resource, this is just an example) and then create another CDK code to deploy a Lambda function which will run the first one. Do you remember the Russian toy called "matryoshka"?

Why do we need this? There are two main reasons:
* For fun, to play with Lambda layers and AWS CDK.
* It can be sometimes useful and we actually use it in DevOpsBox.

All the source code is available here: https://github.com/devopsbox-io/example-cdk-from-lambda

# Prerequisites

You will need several tools:
- AWS CDK (tested with 1.71.0)
- Java JDK (tested with OpenJDK 11.0.9)
- NodeJS (required by AWS CDK, tested with v12.18.3)
- Docker (tested with 18.09.5)
- Maven (tested with 3.6.3)
- AWS CLI (only for testing, tested with 1.18.57)

AWS account with proper credentials is also required. The code will probably work with other versions too.

# The solution

First of all, we will create two CDK projects, one inside another (matryoshka):
``` bash
mkdir example-cdk-from-lambda
cd example-cdk-from-lambda
cdk init app --language=java

mkdir run-cdk-lambda
cd run-cdk-lambda
cdk init app --language=java
```
Yes, we are using Java here... You can achieve the same results in any other language supported by AWS CDK.

I will not change any package name just to allow you to easily do `git diff HEAD~1` and to see all my changes, however, I have removed all the tests and test dependencies - they are not important here.

## The smallest matryoshka

Here we just create an S3 bucket but it could be any CDK code.

We need the CDK S3 dependency [run-cdk-lambda/pom.xml](https://github.com/devopsbox-io/example-cdk-from-lambda/blob/master/run-cdk-lambda/pom.xml)
``` xml
<dependency>
    <groupId>software.amazon.awscdk</groupId>
    <artifactId>s3</artifactId>
    <version>${cdk.version}</version>
</dependency>
```
Next, we have to write some CDK code [run-cdk-lambda/src/main/java/com/myorg/RunCdkLambdaStack.java](https://github.com/devopsbox-io/example-cdk-from-lambda/blob/master/run-cdk-lambda/src/main/java/com/myorg/RunCdkLambdaStack.java):
``` java
new Bucket(this, "created-by-cdk-from-lambda", BucketProps.builder()
    .removalPolicy(RemovalPolicy.DESTROY)
    .build());
```

## The medium matryoshka

If you want to run the AWS CDK, you will have to execute the cdk binary because this is how it works. That is why we need to create a Lambda handler, which will be just a wrapper and will run the cdk.

We have to add the aws-lambda-java-core dependency to our [run-cdk-lambda/pom.xml](https://github.com/devopsbox-io/example-cdk-from-lambda/blob/master/run-cdk-lambda/pom.xml):
``` xml
<dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>aws-lambda-java-core</artifactId>
    <version>1.2.1</version>
</dependency>
```

Then, create the CdkWrapper class [run-cdk-lambda/src/main/java/com/myorg/CdkWrapper.java](https://github.com/devopsbox-io/example-cdk-from-lambda/blob/master/run-cdk-lambda/src/main/java/com/myorg/CdkWrapper.java):
``` java
public class CdkWrapper implements RequestHandler<Map<String, String>, String> {

    public static final String CDK_OUT_DIR = "/tmp/cdk.out";
    private static final String CDK_COMMAND = "java -cp . ";

    @Override
    public String handleRequest(Map<String, String> input, Context context) {

        runCdk(RunCdkLambdaApp.class);

        return "";
    }

    public static void runCdk(Class<?> cdkClass) {
        ProcessBuilder processBuilder = new ProcessBuilder(
                "cdk",
                "deploy",
                "--verbose",
                "--output", CDK_OUT_DIR,
                "--app", CDK_COMMAND + cdkClass.getName(),
                "--require-approval", "\"never\""
        );
        processBuilder.redirectOutput(ProcessBuilder.Redirect.INHERIT);
        processBuilder.redirectError(ProcessBuilder.Redirect.INHERIT);

        Process process;
        try {
            process = processBuilder.start();
        } catch (IOException e) {
            throw new RuntimeException("Cannot start cdk process!", e);
        }

        try {
            process.waitFor();
        } catch (InterruptedException e) {
            throw new RuntimeException("Cdk process interrupted!", e);
        }

        int exitValue = process.exitValue();

        if (exitValue != 0) {
            throw new RuntimeException("Exception while executing CDK!");
        }
    }
}
```
We are doing this in Java using the ProcessBuilder class because our CDK code is also written in Java and we will use the Java 11 runtime in Lambda.

We also have to pack our Java code into an "uber" jar (jar with all the dependencies). We use maven-shade-plugin to do that [run-cdk-lambda/pom.xml](https://github.com/devopsbox-io/example-cdk-from-lambda/blob/master/run-cdk-lambda/pom.xml):
``` xml
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-shade-plugin</artifactId>
    <version>3.2.4</version>
    <executions>
        <execution>
            <phase>package</phase>
            <goals>
                <goal>shade</goal>
            </goals>
        </execution>
    </executions>
</plugin>
```

## The largest matryoshka

The Java 11 Lambda runtime does not have NodeJS and AWS CDK binaries. We will use Lambda layers to put them inside. We will copy binaries from a docker image created from [docker/cdk/Dockerfile](https://github.com/devopsbox-io/example-cdk-from-lambda/blob/master/docker/cdk/Dockerfile):
``` Dockerfile
FROM node:lts

RUN apt-get update && \
    apt-get install -y zip && \
    rm -rf /var/lib/apt/lists/*

ENV AWS_CDK_VERSION=1.71.0
RUN mkdir -p /nodejs && \
    npm config set prefix /nodejs/bin && \
    npm install -g aws-cdk@${AWS_CDK_VERSION}

RUN cd /nodejs/bin && \
    zip -r --symlinks /opt/aws-cdk.zip *
RUN cd /usr/local && \
    zip -r /opt/node.zip bin/node
```

We need also the code to create the "uber" jar, the docker image with NodeJS and CDK binaries, and copy files from it (actually creating a temporary docker container). This will be a bash script [scripts/prepare-bin](https://github.com/devopsbox-io/example-cdk-from-lambda/blob/master/scripts/prepare-bin):
``` bash
#!/bin/bash
set -e

SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
PROJECT_DIR=$(realpath "${SCRIPT_DIR}/..")
DOCKER_IMAGE_TAG=aws-cdk-bin:latest
TMP_BIN_DIR="${PROJECT_DIR}/tmp"

mkdir -p ${TMP_BIN_DIR}

mvn package -f "${PROJECT_DIR}/run-cdk-lambda/pom.xml"
cp ${PROJECT_DIR}/run-cdk-lambda/target/run-cdk-lambda-*.jar ${TMP_BIN_DIR}/run-cdk-lambda.jar

docker build -t ${DOCKER_IMAGE_TAG} ${PROJECT_DIR}/docker/cdk

rm -f ${TMP_BIN_DIR}/docker-cid
docker create --cidfile ${TMP_BIN_DIR}/docker-cid ${DOCKER_IMAGE_TAG}
CID=$(cat ${TMP_BIN_DIR}/docker-cid)

trap "docker rm ${CID}" EXIT

docker cp ${CID}:/opt/node.zip ${TMP_BIN_DIR}/node.zip
docker cp ${CID}:/opt/aws-cdk.zip ${TMP_BIN_DIR}/aws-cdk.zip
```

Now we have to create the outer CDK code to create the lambda with layers. We need the CDK Lambda dependency [pom.xml](https://github.com/devopsbox-io/example-cdk-from-lambda/blob/master/pom.xml):
``` xml
<dependency>
    <groupId>software.amazon.awscdk</groupId>
    <artifactId>lambda</artifactId>
    <version>${cdk.version}</version>
</dependency>
```
and the CDK code [src/main/java/com/myorg/ExampleCdkFromLambdaStack.java](https://github.com/devopsbox-io/example-cdk-from-lambda/blob/master/src/main/java/com/myorg/ExampleCdkFromLambdaStack.java):
``` java
public class ExampleCdkFromLambdaStack extends Stack {
    public ExampleCdkFromLambdaStack(final Construct scope, final String id) {
        this(scope, id, null);
    }

    public ExampleCdkFromLambdaStack(final Construct scope, final String id, final StackProps props) {
        super(scope, id, props);

        createRunCdkLambda();
    }

    private void createRunCdkLambda() {
        String tmpBinDir = getTmpBinDir();

        PolicyStatement cloudformationPolicy = new PolicyStatement(PolicyStatementProps.builder()
                .resources(Arrays.asList(
                        "*"
                ))
                .actions(Arrays.asList(
                        "cloudformation:DescribeStacks",
                        "cloudformation:CreateChangeSet",
                        "cloudformation:DescribeChangeSet",
                        "cloudformation:GetTemplate",
                        "cloudformation:GetTemplateSummary",
                        "cloudformation:DescribeStackEvents",
                        "cloudformation:ExecuteChangeSet",
                        "cloudformation:DeleteChangeSet"
                ))
                .build());

        // we can restrict this policy to certain buckets only
        PolicyStatement s3Policy = new PolicyStatement(PolicyStatementProps.builder()
                .resources(Arrays.asList(
                        "*"
                ))
                .actions(Arrays.asList(
                        "s3:*"
                ))
                .build());

        LayerVersion nodeLayer = LayerVersion.Builder.create(this, "node-layer")
                .description("Layer containing node binary")
                .code(
                        Code.fromAsset(tmpBinDir + "/node.zip")
                )
                .build();

        LayerVersion cdkLayer = LayerVersion.Builder.create(this, "aws-cdk-layer")
                .description("Layer containing AWS CDK")
                .code(
                        Code.fromAsset(tmpBinDir + "/aws-cdk.zip")
                )
                .build();

        Function lambda = new Function(this, "RunCdk", FunctionProps.builder()
                .runtime(Runtime.JAVA_11)
                .handler("com.myorg.CdkWrapper")
                .code(Code.fromAsset(tmpBinDir + "/run-cdk-lambda.jar"))
                .layers(Arrays.asList(
                        nodeLayer,
                        cdkLayer
                ))
                .timeout(Duration.seconds(300))
                .memorySize(512)
                .initialPolicy(Arrays.asList(
                        cloudformationPolicy,
                        s3Policy
                ))
                .functionName("run-cdk")
                .build());
    }

    private String getTmpBinDir() {
        Path tmpPath = Paths.get("tmp");
        return tmpPath.toAbsolutePath().toString();
    }
}
```

This is our third, outermost "matryoshka" layer. Here we are creating the Lambda policies (CloudFormation execution and S3 full access), layers (NodeJS, AWS CDK binaries), and the actual Lambda resource with the com.myorg.CdkWrapper handler and the code from run-cdk-lambda.jar.

We will also change the cdk.json file to run the prepare-bin script before the CDK execution [cdk.json](https://github.com/devopsbox-io/example-cdk-from-lambda/blob/master/cdk.json):
``` json
{
  "app": "./scripts/prepare-bin && mvn -e -q compile exec:java",
  "context": {
    "@aws-cdk/core:enableStackNameDuplicates": "true",
    "aws-cdk:enableDiffNoFail": "true",
    "@aws-cdk/core:stackRelativeExports": "true"
  }
}
```

# Deploying

You have to set your AWS credentials first. You can do this in the `~/.aws/credentials` file or using environment variables. Then, you have to bootstrap the CDK:
``` bash
cdk bootstrap
```
Now you can create the Lambda function:
``` bash
cdk deploy --require-approval never
```

# Testing

Check your S3 buckets using:
``` bash
aws s3 ls
```
This should not return any bucket with the "runcdklambdastack-createdbycdkfromlambda" prefix.

Run the Lambda function:
``` bash
aws lambda invoke --function-name run-cdk tmp/lambda-out
```
It will run the AWS CDK and create the bucket. The first run can take about 1 minute to complete. Check again your S3 buckets then:
``` bash
aws s3 ls
```
It should return a bucket created by CDK run inside a Lambda function (with the "runcdklambdastack-createdbycdkfromlambda" prefix).

# Conclusion

Running AWS CDK inside a Lambda function is not an easy task. However, sometimes it is useful and I hope that this article will help you to do it, or maybe will only show you how to use lambda Layers to execute almost any executable binary inside an AWS Lamda. We do use it in DevOpsBox to create appropriate IAM roles and policies based on the roles read from Keycloak.

For more details about the DevOpsBox platform please visit https://www.devopsbox.io/