AWS CloudFormation and Github Action
Tags: aws, github
Published: 2021-03-14 08:09:54


## Overview 

The focus of this article is to introduce the integration of AWS CloudFormation and Github Action.

## Introduction

**Github Action** is a CI/CD service provided by Github and it has free tier. With Github Action, you can choose to run CI/CD actions on your self-hosted server or Github docker. It's user friendly and there are multiple plugins available in Github Action Marketplace which reduce your time of development and avoid RTW (Reinvent the Wheel).

**AWS CloudFormation** is a service that gives developers an easy way to automate the infrastructure provisioning using Infrastructure as Code (IaC). What's so special about Infrastrucure as Code is that we are able to run tests because these infrastructure components are written in code.

In this demo, we will be integrating AWS CloudFormation with Github Action and do the following:
- Any branch other than main:
  1. **Linting**: Check the CloudFormation linting format using `cfn-lint`
  2. **Security check**: Check the security of each components in CloudFormation using `cfn-nag` to validate if they are adhered to security best practices from AWS.
- Main branch (protected):
  1. **Deploy**: Deploy the CloudFormation template to S3.

## Prerequisite

You may need the following to run on your projects:
1.  AWS credentials with programmatic access, so you need your AWS Access Key, AWS Secret Key
2. AWS S3 bucket with bucket policy such that a put action is allowed and ACL is allowed, you need the AWS S3 bucket name and make sure it is globally unique.
3. Github account


## Project Structure

```zsh
.
├── .github
│   └── workflows
│       ├── master.yml
│       └── test.yml
└── cloudformation
    ├── conformance-pack.yaml
    └── example.yaml
```

For this project demo, I write two CloudFormation templates:
1. `conformance-pack.yaml` to deploy a set of config rules and remediation actions.
2. `example.yaml` to deploy a simple S3 bucket with SNS trigger

Nevertheless, the main focus is on the Github Actions. Let's take a look on what does the Github Action do. 

### Github Actions

As you can see from the project structure, there are mainly two templates, `master.yaml` and `test.yaml` under the `.github/workflow` directory. GitHub Actions uses YAML syntax to define the events, jobs, and steps and these files are stored in a directory called .github/workflows. In order for you to run the Github Actions, you need to create this directory so that whenever you push a commit into your repository, Github Actions will run automatically. 

In this directory, create two files, `master.yaml` and `test.yaml`.


```yaml
# test.yaml

name: 'Testing Github Actions'


on: [push]

jobs:
  Cloudformation-checker:
    name: Check linting and security concerns
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: cfn-lint-action
        uses: ScottBrenner/cfn-lint-action@1.6.1
        with:
          args: "cloudformation/**/*.yaml"
      
      - name: cfn-nag-action
        uses: minchao/cfn-nag-action@v0.1
        with:
          args: '--input-path cloudformation/
```

This Github Action will be triggered if a commit is pushed into this repository for all the branches. It will validate the template syntax using `cfn-lint-action` and run security check using `cfn-nag-action` if the commit passed the `cfn-lint-action`. What's good abou this Github Action is that all of these steps are ready-made plugin from Github Action Marketplace and they have been created and used by the Github communities. For this project, I'm using the plugins from `ScottBrenner/cfn-lint-action@1.6.1` for `cfn-lint-action` and 
`minchao/cfn-nag-action@v0.1` for `cfn-nag-action`. There are several similar plugins in the Marketplace, please makes sure you read their `README.md` in order to enable the actions. 

```yaml
name: S3Deploy

on: 
  push:
    branches:
      - main
  pull_request:
    branches:
      - main

jobs:
  build:
    runs-on: ubuntu-latest
    
    steps:
      - name: "Checkout"
        uses: actions/checkout@v2
      - name: S3 Sync
        uses: jakejarvis/s3-sync-action@v0.5.1
        with:
          args: --acl bucket-owner-read --follow-symlinks
        env:
          AWS_ACCESS_KEY_ID: ${{secrets.AWS_ACCESS_KEY}}
          AWS_SECRET_ACCESS_KEY: ${{secrets.AWS_SECRET_KEY}}
          AWS_S3_BUCKET: ${{secrets.AWS_BUCKET}}
          SOURCE_DIR: './cloudformation'
```

Lastly, this Github Action will copy the files in the `cloudformation` directory to S3 bucket if a commit or a pull request is made into the main branch. As you can see from the `env` section, there are several secrets,  AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_S3_BUCKET. Please refer to [this documentation site](https://docs.github.com/en/actions/reference/encrypted-secrets) to learn how to input secrets into your Github Actions.

What's good about this is that you don't have to manually push the templates into S3 bucket by running multiple command line or browsing into the AWS management console. 

### Conclusion

Throughout this simple project, you can learn how to run a simple CI/CD pipeline to run simple template syntax check and security check on the CloudFormation templates and then push them into a dedicated S3 bucket. Of course, there are some improvements, for example, auto provision the infrastructures into different environments after a file is pushed to S3 bucket. Feel free to extend from here and hope to see more on CI/CD for Infrastructure as Code.

### Reference

- [Github Action documentaions](https://docs.github.com/en/actions/learn-github-actions)
- [Github Action secrets](https://docs.github.com/en/actions/reference/encrypted-secrets)
- [Github Action Marketplace](https://github.com/marketplace?type=actions)
- [CloudFormation](https://aws.amazon.com/cloudformation/)