Introducing cloudform – tame your AWS CloudFormation templates
Tags: learning, webdev, programming, coding
Published: 2018-02-26 07:20:18

---
title: Introducing cloudform – tame your AWS CloudFormation templates
published: true
description: Whenever we do anything at the backend at Bright Inventions, we define our infrastructure as code with the great help of AWS CloudFormation.
tags: learning, webdev, programming, coding 
---

Whatever we do here in [Bright Inventions](https://brightinventions.pl/), we deeply care about automation, traceability and repeatability. This is why we embraced the DevOps-related practices like [Continuous Delivery](https://brightinventions.pl/blog/teamcity-for-ios-project/) or [containerization](https://brightinventions.pl/blog/dockerizing-android-builds/), we are careful about properly set up [logging](https://brightinventions.pl/blog/http-request-logging-in-node/) and monitoring, we know our stuff when it comes to [reliability and resiliency](https://brightinventions.pl/blog/the-importance-of-timeouts/). This is also why, whenever we do anything at the backend, we define our [infrastructure as code](https://en.wikipedia.org/wiki/Infrastructure_as_Code) with the great help of [AWS CloudFormation](https://aws.amazon.com/cloudformation/).

## Why should I care?

In AWS CloudFormation all the cloud infrastructure required to run the application – from the hardware setup through the networking layer to the actual application environment – is defined within a JSON (or YAML) [template file](https://aws.amazon.com/cloudformation/aws-cloudformation-templates/). This is convenient as we can now keep the file in GIT and use it as an input to the deployment scripts. 

The problem is, though, that unless our project is really simple, our template file grows quickly. Our typical environment setup based on [Elastic Container Service](https://aws.amazon.com/ecs/) with VPC, failovers, firewalls, load balancing, auto-scaling, logging and monitoring requires ~2.5k lines in our JSON template file. And some of the constructs there – like passing RDS database connection string to the container instance that includes concatenation of multiple parameters, some of which are secret, some other are generated by other resources – are really verbose and complex. 

![Is this how your AWS CloudFormation templates looks like?](https://thepracticaldev.s3.amazonaws.com/i/m8y6ldtzgnupsz5wlidr.jpg)

## Enter cloudform

Our typical template file was too large. It was too verbose and repetitive. It was error-prone and hard to grasp. It was messy. But all was not lost – the template is just a plain old JSON, anyway. This is how [`cloudform`](https://www.npmjs.com/package/cloudform) was born. It is a TypeScript-based imperative way to define AWS CloudFormation templates. With `cloudform` you can use all the powers of TypeScript to define the elements of your infrastructure and let the library take care of generating that monstrous JSON file out of it.

So, for example, to define a [VPC](http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-vpc.html), instead of writing this lengthy and hairy piece of JSON:

```json
"VPC": {
  "Type": "AWS::EC2::VPC",
  "Properties": {
    "CidrBlock": {
      "Fn::FindInMap": [
        "NetworkingConfig",
        "VPC",
        "CIDR"
      ]
    },
    "EnableDnsHostnames": true,
    "Tags": [
      {
        "Key": "Application",
        "Value": {
          "Ref": "AWS::StackName"
        }
      },
      {
        "Key": "Network",
        "Value": "Public"
      },
      {
        "Key": "Name",
        "Value": {
          "Fn::Join": [
            "-",
            [
              {
                "Ref": "AWS::StackId"
              },
              "VPC"
            ]
          ]
        }
      }
    ]
  }
}
```

you can just have a `EC2.VPC` object definition that is totally equivalent:

```typescript
VPC: new EC2.VPC({
    CidrBlock: NetworkingConfig.VPC.CIDR,
    EnableDnsHostnames: true,
    Tags: [
        new ResourceTag('Application', Refs.StackName),
        new ResourceTag('Network', 'Public'),
        new ResourceTag('Name', Fn.Join('-', [Refs.StackId, 'VPC']))
    ]
})
```

Note what I have done here:
1. No more explicit `Type`, no `Properties` wrapper boilerplate. TypeScript's type is enough to figure it out. What's more, it gives us a compile-time check for the validity of the structure.
1. No more magic strings for `AWS::StackName` or `AWS::StackId` references. Compile-time constants, again.
1. Instead of complex `Fn::FindInMap` object traversing, I use TypeScript-level objects directly. <small>Yes, it results in a different entry in the template, but functionally it's an equivalent.</small>
1. I also simplified the construction of key-value pairs used in `Tags`. Strongly-typed one-liner instead of verbose 4 lines.

But this is not the end. With `cloudform` (or actually with TypeScript) we can go further:
1. Nothing stops us from using TypeScript-level variables, functions or conditions to build up our objects from reusable, configurable parts.
1. We can split the template definition into multiple files.
1. We can apply all the available refactoring techniques to keep our definitions tidy and easy to use.

With `cloudform`'s syntax our ~2.5k lines of JSON was reduced to ~1k lines of TypeScript object definitions.

## Getting started & usage

You're probably already eager to test it out, so let's jump to the crux of the matter. The package is obviously available for you on `npm`. Type the following:

`npm install --save-dev cloudform`

Now we need to define our template. Let's have a separate directory for it, so that it doesn't mix with our production or test code. It would also suggest we may break the file into smaller chunks and use standard [imports](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import) to link it together. Maybe `cloudformation/index.ts`?

This file needs to end with `cloudform` function call, so that the tool picks our template up correctly. It takes an object that consists of everything that CloudFormation template might consist of. 

```typescript
import cloudform, {Fn, Refs, EC2, StringParameter, ResourceTag} from "cloudform"

cloudform({
    Description: 'My template',
    Parameters: {
        DeployEnv: new StringParameter({
            Description: 'Deploy environment name',
            AllowedValues: ['dev', 'stage', 'production']
        })
    },
   
    Resources: {
        VPC: new EC2.VPC({
            CidrBlock: Fn.FindInMap('SubnetConfig', 'VPC', 'CIDR'),
            EnableDnsHostnames: true,
            Tags: [
                new ResourceTag('Application', Refs.StackName),
                new ResourceTag('Network', 'Public'),
                new ResourceTag('Name', Fn.Join('-', [Refs.StackId, 'VPC']))
            ]
        })
    }
})
```

The simple convention is used here – all the AWS types' namespaces are available directly as exports from the `cloudform` package. All the resources within this package are available inside. This way `EC2.VPC` object from our example translates into `AWS::EC2::VPC` type we can find [in CloudFormation documentation](http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-vpc.html). All the properties also match one-to-one, including casing. This API-level compliance is guaranteed to hold true because the `cloudform`'s types are generated from the [AWS-provided schema definition file](http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-resource-specification.html).

See also [the example included in the repository](https://github.com/bright/cloudform/blob/master/example/example.ts).

Now, run the binary that was installed for you specifying the path to your template sources root. The library will print the JSON to the stdout, so you might want to pipe it to the file somewhere:

`cloudform cloudformation/index.ts > template.out`

And if no compilation errors happen, `template.out` is now a JSON file ready to be used within CloudFormation.

## What cloudform is not

I can feel your excitement already 😆, but let's be clear what `cloudform` does and what is out of its scope. Basically it translates the TypeScript code into the JSON object as-is. Whatever you put in your TypeScript, as long as it compiles, will be included in the generated file. There is no validation or any domain-level checks implemented – if you reference a resource, parameter or condition that does not exists or use a nonsensical value for a parameter, `cloudform` will not warn you today (CloudFormation will do most probably). 

The goal of this package is not to create a full-blown DSL for CloudFormation. It is rather to get the most value out of TypeScript features, so if any more complex checks are to be implemented, it would most probably be about things that TypeScript can verify compile-time. If you need a full validation, there are some specialized tools out there already (see for example [`cfn-lint`](https://www.npmjs.com/package/cfn-lint) or [`cfn-check`](https://www.npmjs.com/package/cfn-check)), so it might have sense to chain it after `cloudform`.

Note also that `cloudform` does not attempt to interact with your AWS stack in any way, nor reading neither writing – no credentials are needed. It makes it 100% safe to experiment with. All the stack interactions (i.e. deployment) are outside of the scope of this tool.

## What's next?

The package source code is MIT-licensed, [available on GitHub](https://github.com/bright/cloudform). If you find out that something is missing or invalid in `cloudform`, feel free to open an issue on GitHub or – even better – contribute yourself!

In the follow-up post I'm sharing [a few more examples and recipes](https://brightinventions.pl/blog/aws-cloudformation-patterns-practices-cloudform/) how to build template generation into your Continuous Delivery workflow.

Originally published at [brightinventions.pl](https://brightinventions.pl/blog/) 

By Adam Bar, The Web Guy @ Bright Inventions
[Personal blog](https://whatwebcando.today/) [Twitter](https://twitter.com/NOtherDev) [Email](adam.bar@brightinventions.pl) [Github](https://github.com/NOtherDev)