Build AWS AMI with HashiCorp Packer using GitHub Actions
Tags: packer, aws, githubactions, github
Published: 2021-08-10 08:35:20

---
title: Build AWS AMI with HashiCorp Packer using GitHub Actions
published: true
date: 2021-08-10 07:02:53 UTC
tags: Packer,AWS,githubactions,GitHub
series: packer
canonical_url: https://blog.g33kzone.com/aws-ami-hashicorp-packer-github-actions
cover_image: https://cdn.hashnode.com/res/hashnode/image/upload/v1628346675112/-qD7vHnen.png
---

Security best practices recommend the latest base images (e.g. AWS AMI ) for spinning VM on the cloud. Up to date software patches reduce the risk of any security breach. This emphasizes a strong need to set up an Image Factory to automate the image creation process with the latest software versions.

Also, every organization has a standard list of software to be installed for a given image. Only the essential software must be part of the list to reduce the blast surface from a security standpoint. This list of software can either be installed once the VM is created or when the VM is in the process of creation. However, both these techniques are time-consuming and hence not recommended.

It would be preferred to bake this software list in the base image itself. [HashiCorp Packer](https://www.packer.io/) is an open-source tool that specializes in building automated machine images for multiple platforms from a single source configuration.

#### HashiCorp Packer Installation

Refer to HashiCorp [documentation](https://learn.hashicorp.com/tutorials/packer/get-started-install-cli?in=packer/aws-get-started) for Packer installation based on your hardware OS.

#### GitHub Actions

For this demo, we will use GitHub Actions to create CI/CD pipeline to automate this workflow and eventually push the baked image (AMI) in AWS. It is a platform to automate tasks within the software development lifecycle. It's an `event-driven` framework, which means we can carry series of commands for a given event or can be scheduled for one-off or repetitive tasks. (e.g. Execute a Test Suite on Pull Request creation, Adding labels to issues, Lint checks, etc.)

Actions are defined in YAML files, which allows pipeline workflow to be triggered using any GitHub events like on creation of Pull Requests, on code commits, and much more.

#### Prerequisites

- AWS User with Programmatic access
  - AWS Access Key ID
  - AWS Secret Access Key
- AWS IAM Privileges to create EC2 Instance (create, modify and delete EC2 instances). Refer [documentation](https://www.packer.io/docs/builders/amazon#iam-task-or-instance-role) for the full list of IAM permissions required to run the amazon-ebs builder.

In this post, we will bake Open JDK (Java 8) in our Ubuntu base Image and push it into AWS. Packer configurations can be written in HCL (_.pkr.hcl file extension) and JSON (_.pkr.json) formats. We will use the HCL language for this demo.

![Packer Flow.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1628578949157/M78aWQCCb.png)

Reference GitHub repository - [pkr-aws-ubuntu-java](https://github.com/g33kzone/pkr-aws-ubuntu-java)

#### Code Time

Let us start writing Packer configuration. (I am using a `Linux` machine for this demo)

#### Packer Configuration

Create Project folder `pkr-aws-ubuntu-java`

```bash
mkdir pkr-aws-ubuntu-java && cd $_

```

Create a file named `aws-demo.pkr.hcl`

```bash
touch aws-demo.pkr.hcl

```

Open your favorite IDE (e.g. VSCode). Copy the below code in `aws-demo.pkr.hcl` file.

```hcl
packer {
  required_plugins {
    amazon = {
      version = ">= 0.0.2"
      source = "github.com/hashicorp/amazon"
    }
  }
}
```

The `packer {}` block contains Packer settings, including a required Packer version. The `required_plugins` block in the Packer block, specifies the plugin required by the template to build your image. The plugin block contains a `version` and `source` attribute.

#### Source block

The source block configures a specific `builder` plugin, which is then invoked by the `build` block. Source blocks use builders and communicators to define virtualization type, image launch type, etc.

Copy the following code to `aws-demo.pkr.hcl` file.

```hcl
variable "ami_prefix" {
  type = string
  default = "packer-aws-ubuntu-java"
}

locals {
  timestamp = regex_replace(timestamp(), "[- TZ:]", "")
}

source "amazon-ebs" "ubuntu_java" {
  ami_name = "${var.ami_prefix}-${local.timestamp}"
  instance_type = "t2.micro"
  region = "us-east-1"
  source_ami_filter {
    filters = {
      name = "ubuntu/images/*ubuntu-xenial-16.04-amd64-server-*"
      root-device-type = "ebs"
      virtualization-type = "hvm"
    }
    most_recent = true
    owners = ["099720109477"]
  }
  ssh_username = "ubuntu"
}
```

Variable `ami_prefix` is used to define the AMI image. Local variable `timestamp` helps ensure uniqueness to AMI name.

The `amazon-ebs` builder launches the source AMI, runs provisioners within this instance, then repackages it into an EBS-backed AMI. This builder configuration launches a `t2.micro` AMI in the us-east-1 region using an `ubuntu:xenial` AMI as the base image.

It creates the AMI named `packer-aws-ubuntu-java+timestamp`. AMI names must be unique else it will throw an error.

It also uses the SSH communicator - by specifying the `ssh_username` attribute. Packer is then able to SSH into EC2 instance using a temporary keypair and security group to provision your instances.

#### Build Block

The `build` block defines what Packer should do with the EC2 instance after it launches.

```hcl
build {
  name = "packer-ubuntu"
  sources = [
    "source.amazon-ebs.ubuntu_java"
  ]

  provisioner "shell" {

    inline = [
      "echo Install Open JDK 8 - START",
      "sleep 10",
      "sudo apt-get update",
      "sudo apt-get install -y openjdk-8-jdk",
      "echo Install Open JDK 8 - SUCCESS",
    ]
  }
}
```

`provisioner` block helps automate modifications to your base image. It leverages shell scripts, file uploads, and integrations with modern configuration management tools such as Ansible, Chef, etc.

The above provisioner defines a shell provisioner and installs Open JDK 8 in the base image.

The final file `aws-demo.pkr.hcl` should look as below.

```hcl
packer {
  required_plugins {
    amazon = {
      version = ">= 0.0.2"
      source = "github.com/hashicorp/amazon"
    }
  }
}

variable "ami_prefix" {
  type = string
  default = "packer-aws-ubuntu-java"
}

locals {
  timestamp = regex_replace(timestamp(), "[- TZ:]", "")
}

source "amazon-ebs" "ubuntu_java" {
  ami_name = "${var.ami_prefix}-${local.timestamp}"
  instance_type = "t2.micro"
  region = "us-east-1"
  source_ami_filter {
    filters = {
      name = "ubuntu/images/*ubuntu-xenial-16.04-amd64-server-*"
      root-device-type = "ebs"
      virtualization-type = "hvm"
    }
    most_recent = true
    owners = ["099720109477"]
  }
  ssh_username = "ubuntu"
}

build {
  name = "packer-ubuntu"
  sources = [
    "source.amazon-ebs.ubuntu_java"
  ]

  provisioner "shell" {

    inline = [
      "echo Install Open JDK 8 - START",
      "sleep 10",
      "sudo apt-get update",
      "sudo apt-get install -y openjdk-8-jdk",
      "echo Install Open JDK 8 - SUCCESS",
    ]
  }
}
```

#### GitHub Actions

Create a new file in the .github/workflows directory named `github-actions-packer.yml`

![Folder Structure.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1628519387479/29r1YArQr.png)

We will schedule this workflow to run in the wee hours - let's say 04:00 Hrs in the morning.

`name` - The name of your workflow. GitHub displays the names of your workflows on your repository's actions page - "AWS AMI using Packer Config"

```yaml
name: AWS AMI using Packer Config

```

on - (Required) The name of the GitHub event that triggers the workflow. We have configured to trigger the workflow on schedule.

```yaml
on:
  schedule:
    # * is a special character in YAML so you have to quote this string
    - cron: '0 4 * * *'
```

`jobs` - A workflow run is made up of one or more jobs. These jobs can run in parallel or sequentially. Each job executes in a runner environment specified by runs-on.

`job name` - The name of the job displayed on GitHub.

`runs-on` - (Required) Determines the type of machine to run the job on. The machine can be either a GitHub-hosted runner or a self-hosted runner. Available GitHub-hosted runner types are: windows-latest / windows-2019 / windows-2016 / ubuntu-latest / ubuntu-20.04 etc.

```yaml
jobs:
  packer:
    runs-on: ubuntu-latest
    name: packer
```

`steps` - Sequence of tasks called steps within a Job. They can execute commands, set up tasks, or run actions in your repository, a public repository, or action published in a Docker registry.

The first step is to check out the source code in the runner environment.

`Checkout V2`- This action checks out your repository under $GITHUB\_WORKSPACE, so your workflow can access it.

```yaml
    steps:
      - name: Checkout Repository
        uses: actions/checkout@v2
```

To ensure access to the AWS Cloud environment we need to configure `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` in the runner environment. The values for these variables will be configured as GitHub Secrets in the below section.

`Configure AWS Credentials` - This action configures AWS credential and region environment variables for use in other GitHub Actions.

```yaml
      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          # aws-session-token: ${{ secrets.AWS_SESSION_TOKEN }} 
          # if you have/need it
          aws-region: us-east-1
```

`Init` initializes the Packer configuration used in the GitHub action workflow.

```yaml
      # Initialize Packer templates
      - name: Initialize Packer Template
        uses: hashicorp/packer-github-actions@master
        with:
          command: init
```

`Validate` checks whether the configuration has been properly written. It will throw an error otherwise.

```yaml
      # validate templates
      - name: Validate Template
        uses: hashicorp/packer-github-actions@master
        with:
          command: validate
          arguments: -syntax-only
          target: aws-demo.pkr.hcl
```

`Build` executes the Packer configuration.

```yaml
      # build artifact
      - name: Build Artifact
        uses: hashicorp/packer-github-actions@master
        with:
          command: build
          arguments: "-color=false -on-error=abort"
          target: aws-demo.pkr.hcl
        env:
          PACKER_LOG: 1
```

The complete file github-actions-packer.yml will look as below.

```yaml
name: AWS AMI using Packer Config

on:
  schedule:
    # * is a special character in YAML so you have to quote this string
    - cron: '0 4 * * *'

jobs:
  packer:
    runs-on: ubuntu-latest
    name: packer

    steps:
      - name: Checkout Repository
        uses: actions/checkout@v2

      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          # aws-session-token: ${{ secrets.AWS_SESSION_TOKEN }} 
          # if you have/need it
          aws-region: us-east-1

      # Initialize Packer templates
      - name: Initialize Packer Template
        uses: hashicorp/packer-github-actions@master
        with:
          command: init

      # validate templates
      - name: Validate Template
        uses: hashicorp/packer-github-actions@master
        with:
          command: validate
          arguments: -syntax-only
          target: aws-demo.pkr.hcl

      # build artifact
      - name: Build Artifact
        uses: hashicorp/packer-github-actions@master
        with:
          command: build
          arguments: "-color=false -on-error=abort"
          target: aws-demo.pkr.hcl
        env:
          PACKER_LOG: 1
```

The source code is ready and can be pushed to the GitHub repository. As configured, the workflow will be triggered at 04:00 hrs in the morning.

![Packer Actions 1.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1628576817615/Kfu1V7PYy.png)

![Packer Actions 2.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1628576910401/Fq8fWOlw7.png)

![Packer Actions 3.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1628576917881/UD6XM51kx.png)

![Packer Actions 4.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1628577773100/-nN3Wi9AI.png)