Implement Unit, Integration and Application Test for CDK Infrastructure and an EC2 Web Server Application
Tags: aws, cdk, testing, go
Published: 2021-06-21 10:46:21


With CDK you create Infrastructure as Code - IaC.  You can automate the test for the IaC code. 
The three test tastes -Unit, Integration and Application- should work closely together. Here I show you how. It is like the three steps of coffee tasting: 1 smell, 2 Taste, 3 Feel.

 You can start immediately with the GO implementation or achieve the same effect in another CDK-supported language such as TypeScript or Python by following the steps described. The cit GO integration and application tests are directly applicable to all CDK templates, no matter which programming language is used.


With a few lines of code, you got tests for the stack level, the physical resource level and the application. Let`s apply that to a CDK generated EC2 Load Balancer App.

## From Unit test to application test with a Load Balancer EC2 App

![Mapping](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kn4fv3pgng8vaty87nlc.png)

We will go through three test types in this post

1) The Unit test
2) The physical resource test called cit - CDK Infrastructure Testing
3) Application Test

### Overview

**Unit Test**:
The generated CloudFormation (Cfn) is tested. Usually, you can rely on the fact that e.g. the SNS Construct generates an SNS Resource, but...
When you use some programming logic inside the Construct, you can not be 100% sure that the right AWS resources are generated. This is what Unit testing is made for.

**Integration Test**:
Resource creation is tested. Sometimes on Monday, I ask myself the question "did I *really* deploy that last Friday?" - this happens when you run out of coffee.
So the next level is to test whether the resource in the Construct really is created. To get traceability I want to use the given Construct ID to access the physical resource. Traceability means that you know which Resource is created by which Construct.

**Application Test**:
The functionality of the application is tested. With an AWS application bundled with infrastructure, a new version of your app does not only consist of the software itself but also the changes in the infrastructure. So it makes sense to test the responses from the application to certain requests.

Let's walk through the steps. We use `go/alb_ec2` from the repository `https://github.com/tecracer/cdk-templates`. You will find the same CDK template in `typescript/alb_ec2`. Still, somebody has to code the python example...

![Load Balancer](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/667bm9olyklwyuajnk35.png)

CDK generated Web Server.

### **1 Smell: Unit Test - Template creation**

As discussed in [Part 1](aws-blog.de/2021/05/cit-build-cdk-infrastructure-testing-part-1-terratest-and-the-integrated-integration.html), the standard *Unit Tests* checks the CDK generated CloudFormation template. 

We create an Application Load Balancer with the **ConstructID** `LB`

This name should be meaningful to you. I like names short and sweet, so "LB". 
**This id will be used for all test types.**  You do not need to create Systems Manager Parameters or CloudFormation Exports like discussed in part 1, just use the Construct ID.

This is the Load Balancing Construct in GO CDK:



```go
lb := elasticloadbalancingv2.NewApplicationLoadBalancer(stack, aws.String("LB"),
  &elasticloadbalancingv2.ApplicationLoadBalancerProps{
    Vpc:              myVpc,
    InternetFacing:   aws.Bool(true),
    LoadBalancerName: aws.String("ALBGODEMO"),
  },

)
```

While it is a string `'LB'` in TypeScript, GO uses String pointers for efficiency. `aws.String("LB")` creates a string pointer.


This is the Load Balancing Construct in TypeScript CDK:

```javascript
const lb = new ApplicationLoadBalancer(this, 'LB', {
    vpc: albVpc,
    internetFacing: true
    
  });
```

This is the Load Balancing Construct in Python CDK:

```javascript
lb = elbv2.ApplicationLoadBalancer(self, "LB",
        vpc=vpc,
        internet_facing=True
    )
```        

The *second* parameter is the Construct ID. 

With the Construct defined, we can call `cdk synth`. This *synth*esizes the CloudFormation template in the directory `cdk.out`.

Tipp: Use `npx cdk@v2.0.0-rc.8` instead of `cdk` which will call the TypeScript transpiler , so you do no need `npm build` before.

In the CloudFormation template `cdk.out/AlbInstStack.template.json` this is the generated code for the Load Balancer:

```json
"LB8A12904C":{
  "Type": "AWS::ElasticLoadBalancingV2::LoadBalancer",
  "Properties": {
    "LoadBalancerAttributes": [
      {
        "Key": "deletion_protection.enabled",
        "Value": "false"
      }
    ],
    "Name": "ALBGODEMO",
```

Where `LB8A12904C` is the **Logical ID**.

We define the Unit Test for that in GO:

```go
func TestAlbInstStack(t *testing.T) {
	// GIVEN
	app := awscdk.NewApp(nil)

	// WHEN
	stack := alb_ec2.NewAlbEC2Stack(app, "MyStack", nil)

	// THEN
	bytes, err := json.Marshal(app.Synth(nil).GetStackArtifact(stack.ArtifactId()).Template())
	if err != nil {
			t.Error(err)
	}
  // Check
	template := gjson.ParseBytes(bytes)
	albName := template.Get("Resources.LB8A12904C.Properties.Name").String()
	assert.Equal(t, "ALBGODEMO", albName)
}
```

1) Given: An CDK app is created
2) When: The stack is created
3) Then: The data structure "StackArtifact" is translated to json. This is called "marshaling"
4) Check: the gson library is used to parse the json file

This is the part which you also can do in TS/Python. The `cdk init app` generates a test skeleton for you.

How do you know the name "LB8A12904C"? - Answer is you don't. You have to synthesize once and take the Logical ID out of the template file.

We run the test:

```bash
go test -run TestAlbInstStack -v
=== RUN   TestAlbInstStack
--- PASS: TestAlbInstStack (7.82s)
PASS
ok  	alb_ec2	8.430s
```

This test can not only be used for testing created parameters e.g. for the albName. It also proofs that the build process runs without errors. Try it out and change the name of the Load Balancer or change the Construct ID. The test will FAIL.

In TypeScript this test would look like:

```ts
test('Load Balancer exists', () => {
    const app = new cdk.App();
    // WHEN
    const stack = new AlbEc2.AlbEc2Stack(app, 'MyTestStack');
    // THEN
    const actual = app.synth().getStackArtifact(stack.artifactId).template;
    expect(actual.Resources.LB8A12904C.Type).toEqual("AWS::ElasticLoadBalancingV2::LoadBalancer")
});
```
To be exact this TypeScript test only checks the Type, not the property.

Currently, the integration and applications tests are failing:

```bash
go test  -v
=== RUN   TestALBRequest
    ssm.go:18:
        	Error Trace:	ssm.go:18
        	            				alb_ec2_test.go:35
        	Error:      	Received unexpected error:
        	            	ParameterNotFound:
        	            		status code: 400, request id: a9b1f50b-c462-4b9b-9bf4-72fbb4768114
        	Test:       	TestALBRequest
--- FAIL: TestALBRequest (0.37s)
=== RUN   TestAlbPhysicalResource
FATA[0000] Template AlbInstStack not found
exit status 1
FAIL	alb_ec2	1.145s
```

This is correct because we do not have a physical resource for the Load Balancer yet.

![Mapping](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/acyiiqrwfqufjtimhaj3.png)

We have tested the CloudFormation Template with a Unit test.

With a `cdk deploy` the template is sent to the CloudFormation service. CloudFormation generates the Stack, which includes the physical resource "load balancer". I think it's funny to talk about "physical" resources in the Cloud, but that is the AWS wording :). 

### **2 Taste: Integration test**

Now with `cdk deploy` the resources are generated. CDK adds all necessary auxiliary resources, which the Load Balancer needs to work.  
So we do not have to define everything in the Construct. All the physical resources together build the CloudFormation stack.

Different resources from the stack  can have different states during creation:

```bash
cdkstat AlbInstStack
Logical ID                       Pysical ID                       Type                             Status
----------                       ----------                       -----------                      -----------
ASG46ED3070                      autoscalingGroupCDKDEMO          AWS::AutoScaling::AutoScalingGr  CREATE_IN_PROGRESS
ASGInstanceProfile0A2834D7       AlbInstStack-ASGInstanceProfile  AWS::IAM::InstanceProfile        CREATE_COMPLETE
ASGInstanceSecurityGroup0525485  sg-0bb8c50c146e319d7             AWS::EC2::SecurityGroup          CREATE_COMPLETE
ASGInstanceSecurityGroupfromAlb  ASGInstanceSecurityGroupfromAlb  AWS::EC2::SecurityGroupIngress   CREATE_COMPLETE
ASGLaunchConfigC00AF12B          AlbInstStack-ASGLaunchConfigC00  AWS::AutoScaling::LaunchConfigu  CREATE_COMPLETE
CDKMetadata                      d650d230-cfa0-11eb-b478-06e6f2d  AWS::CDK::Metadata               CREATE_IN_PROGRESS
...
```

Each resource in the CloudFormation stack starts with the state `CREATE_IN_PROGRESS`and hopefully ends with `CREATE_COMPLETE`

When all resources have the state `CREATE_COMPLETE`, the stack is completed. The Load Balancer should be created. To be sure, we test that.

The test for the physical resource looks like:

```
func TestAlbPhysicalResource( t *testing.T){
	if testing.Short() {
        t.Skip("skipping integration test in short mode.")
    }
	alb, err := citalbv2.GetLoadBalancer(aws.String("AlbInstStack"), aws.String("LB"))
	assert.NilError(t,err,"The LoadBalancer should be retrievable without error")
	// Just read anything from alb
	applicationType := awselbv2types.LoadBalancerTypeEnumApplication
	assert.Equal(t, applicationType, alb.Type)
}
```

The `GetLoadBalancer` takes care of the translation from "Construct with ID **LB** from Stack **AlbInstStack**" to an [Load Balancer data structure ](https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/service/elasticloadbalancingv2/types#LoadBalancer).


With the AWS cli call:

```bash
aws cloudformation describe-stack-resource  --stack-name AlbInstStack --logical-resource-id LB8A12904C
````

you can see the CloudFormation data of the physical resource:

```json 
  "StackResourceDetail": {
        "StackName": "AlbInstStack",
        "StackId": "arn:aws:cloudformation:eu-central-1:555544446666:stack/AlbInstStack/d650d230-cfa0-11eb-b478-06e6f2d05224",
        "LogicalResourceId": "LB8A12904C",
        "PhysicalResourceId": "arn:aws:elasticloadbalancing:eu-central-1:555544446666:loadbalancer/app/ALBGODEMO/fa33a9bde8742fe6",
        "ResourceType": "AWS::ElasticLoadBalancingV2::LoadBalancer",
        "LastUpdatedTimestamp": "2021-06-17T19:22:01.310000+00:00",
        "ResourceStatus": "CREATE_COMPLETE",
        "Metadata": "{\"aws:cdk:path\":\"AlbInstStack/LB/Resource\"}",
        "DriftInformation": {
            "StackResourceDriftStatus": "NOT_CHECKED"
        }
    }
```        

The `citalbv2.GetLoadBalancer` uses this CloudFormation data to get the Load Balancer Data with the GO SDK.

Let us have a look at the physical test:

1) `if testing.Short() `

  If you call `go test -short` the short flag will be set and the test will be skipped. This is useful if you not have deployed the stack yet, so you know the test will fail.

2) The `assert.NilError` checks wether the resource is really there.

3) Check fields, `applicationType := awselbv2types.LoadBalancerTypeEnumApplication`
  As an example the type is checked whether it is really is an Application Load Balancer.


So you can check for the data fields of the resource itself - not the CloudFormation data.

Some of the data fields are: 

```go
AvailabilityZones []AvailabilityZone
CanonicalHostedZoneId *string
CreatedTime *time.Time
IpAddressType IpAddressType 
LoadBalancerArn *string
LoadBalancerName *string
SecurityGroups []string
State *LoadBalancerState
Type LoadBalancerTypeEnum
```

If you want to test some connected resources, you retrieve them with the SDK. 
If you want to check how many Security Groups are attached, you do that directly on `SecurityGroups []string` with `len(alb.SecurityGroups)`.

The shortest integration test - check for existence - is just:

```go
alb, err := citalbv2.GetLoadBalancer(aws.String("AlbInstStack"), aws.String("LB"))
assert.NilError(t,err,"The LoadBalancer should be retrievable without error")
```

To separate the test levels, you could also use tags in GO. You use different files for the 
tests and tag them like:

```go
// +build integration

package alb_ec2
```

to include.

With `go test --tags=integration` you would only run test files tagged with `integration`

This test written in GO can also be applied to CDK generated CloudFormation stacks from other programming languages!
At first sight, the idea to write a test in a different language seems strange. But this is the same as what you would do if using [Chef inspec](https://www.chef.io/products/chef-inspec) which uses Ruby. The difference is that you do not use a DSL (Domain Specific Language), but directly work on the AWS GO SDK. With a DSL you have limited possibilities, which the SDK you can test anything.

![cit](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ltsdn69wxveh02k4jvxa.png)

We have tested the physical Load Balancer resource with an integration test.

### **3 Feel: Application Test**

```go
func TestALBRequest(t *testing.T) {
	if testing.Short() {
        t.Skip("skipping integration test in short mode.")
    }
	storedUrl := terratest_aws.GetParameter(t,region,"/cdk-templates/go/alb_ec2")

	url := fmt.Sprintf("http://%s", storedUrl)

	sleepBetweenRetries, error := time.ParseDuration("10s")

	if error != nil {
		panic("Can't parse duration")
	}

	http_helper.HttpGetWithRetry(t, url, nil, 200 , "<h1>hello world</h1>", 20, sleepBetweenRetries)
	
}
```

This is almost the same as in [part1 blogpost](aws-blog.de/2021/05/cit-build-cdk-infrastructure-testing-part-1-terratest-and-the-integrated-integration.html). We are using terratest to send http requests. If we want the test for certain data in the response, you can add a helper function. 
See [https://github.com/gruntwork-io/terratest/tree/master/modules/http-helper](https://github.com/gruntwork-io/terratest/tree/master/modules/http-helper) for more details.

![Applicatopm](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9o0in7s3k76ja8vz0x8a.png)

We have tested the application.


## The CIT lib

This is a GO implementation of the concept to take the CDK Construct ID as the identifier for all test levels. If you want to code it in *your* language, this should be doable with these hints.

You can use the GO module from [https://github.com/megaproaktiv/cit](https://github.com/megaproaktiv/cit)

### How do you get the physical ID from the Construct id?

Let us have a look at the ALb CloudFormation:

```json
"LB8A12904C": {
    "Type": "AWS::ElasticLoadBalancingV2::LoadBalancer",
    "Properties": {
      "LoadBalancerAttributes": [
        {
          "Key": "deletion_protection.enabled",
          "Value": "false"
        }
      ],
...
    "Metadata": {
      "aws:cdk:path": "AlbStack/LB/Resource"
    }
  },
```
To know which Construct belongs to which resource, the CDK has to maintain the mapping state.
It is stored in the Metadata:

`"aws:cdk:path": "AlbStack/LB/Resource"`

The first part is the stack-name, the second is the Construct id, then "Resource".

### The algorithm:

1) Get the template from CloudFormation with the template name and the `GetTemplate` API call
2) Find the ConstructID name from the metadata. `AlbStack/LB/Resource`
3) Get the Logical ID from the Resource `LB8A12904C`
4) Call the `DescribeStackResource` API call with the Logical ID - this will give you the Physical ID

See the GO code: [PhysicalIDfromCID](https://github.com/megaproaktiv/cit/blob/main/getphysical.go)

1) Get template 

```go
	resGetTemplate, err := client.GetTemplate(context.TODO(), parameterStack)
```

2) / 3. Find Cid / Get the Logical ID

```go
for key, resource := range stack.Resources {
  if resource.Metadata != nil {
    if resource.Metadata["aws:cdk:path"] != "" {
      meta := resource.Metadata["aws:cdk:path"]
      log.Debug("Path: ",meta)
      templateConstructID := ExtractConstructID(&meta)
      if templateConstructID == *constructID {
          return &key, nil
      }
    }
  }
}
```

4) DescribeStackResource

```go
resourceDetail,err := client.DescribeStackResource(context.TODO(), parameterResource)
if err != nil {
  return nil, err
}
// find physicalid
physicalId := resourceDetail.StackResourceDetail.PhysicalResourceId
```	

### Low Level helper functions

If you want to check any AWS resource, you can use the `PhysicalIDfromCID` function, which implements the matching.
When you insist on not using go, just implement it for the language and testing framework of your choice.

### Higher Abstraction

During the last weeks, i implemented functions like

- GetLoadBalancer
- GetUser (iam)
- GetVpc, GetSecurityGroup 
and of course for Lambda
- GetFunctionConfiguration

Although it is easy to implement the function for several resources, a simpler call like just `GetLoadBalancer` is nice. Please contact me for adding other resources, because there are *many resources* and I will not add all of them in this lifetime.

In the next part we will apply this to Lambda functions.

## The End

The integration of all test types together has many advantages in my opinion. What is your opinion on this? Is it helpful for your project? I would be happy to hear your experiences!

I hope this concept or the cit framework implementation will help you with your projects also. 
For the last couple of projects, I started with an integrated or application test and found it quite useful to get the rights results and staying focused.

Some of the other integration test i used were:

- AWS Workspaces  and Workspaces User creation
- AWS Transfer sftp User and read/write test
- Application Load Balancer with Domain and installed software


For discussion please contact me on twitter @megaproaktiv
## Appendix

The repositories

Cit - CDK Integration Testing

- [https://github.com/megaproaktiv/cit](https://github.com/megaproaktiv/cit)

cdkstat - Show CloudFormation Stack status

- [https://github.com/megaproaktiv/cdkstats](https://github.com/megaproaktiv/cdkstats)

CDK  Templates using CIT and terratest for testing

- [https://github.com/tecracer/cdk-templates](https://github.com/tecracer/cdk-templates)

Terratest

[Quick Start](https://terratest.gruntwork.io/docs/getting-started/quick-start/)
- [https://github.com/gruntwork-io/terratest](https://github.com/gruntwork-io/terratest)

The tools

Awsume
- [awsume](https://awsu.me)
- [Taskfile](https://taskfile.dev/#/)
## Thanks


Photo by <a href="https://unsplash.com/@nate_dumlao?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Nathan Dumlao</a> on <a href="https://unsplash.com/s/photos/three?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>