Build Jenkins EC2 AMI Using HachiCorp Packer
Tags: aws, packer, jenkins, ec2
Published: 2024-01-04 12:23:31

## Preparation

We need to prepare some tools, such as:

1.HashiCorp Packer. You can download it in [here](https://developer.hashicorp.com/packer/install?product_intent=packer). In this article, I use Packer v1.9.4

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xwf3ve2smf7txnwtv2ui.png)


2.AWS CLI. You can install it in [here](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html). In this article, I use aws-cli v2.8.3

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qa8l6a8pcpg452kra95o.png)


3.Your Favorite IDE (Integrated Development Environment).


4.HashiCorp Terraform (Optional, if you'd like to follow along on the provisioning Jenkins EC2 Using Terraform). You can download it in [here](https://developer.hashicorp.com/terraform/install). In this article, I use Terraform v1.3.2

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/o2ujvabduh75476rdxol.png)

---

## HashiCorp Packer Introduction
HashiCorp Packer is an open-source tool that automates the creation of any machine image for multiple platforms. Packer is no replacement for configuration management tools like Ansible, Puppet, or Chef. Packer uses these tools to install and configure software and dependencies while creating images (AMI for EC2). 

Packer uses a configuration file to create a machine image. Then, it uses builders to spin up an instance on the target platform and runs provisioners to configure applications or services. Once setup is done, it shuts down the temporary instance and saves the new images with any needed post-processing.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/p46bw0chliukp2s42esu.png)


Here are the steps in the process:
1. Boot a temporary instance using the base AMI defined in the HCL template file.
2. Provision the instance using configuration management tools like Ansible, Chef, or a simple automated script to configure the example into the desired state.
3. Create a new machine image from the temporary running instance and shut down the temporary instance after the AMI is registered.

---

## Create Jenkins Master EC2 AMI

**Step 1:** Verify Packer is available on our local machine. Just type `packer` in our terminal.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/032gedufrjzmg0hl10mh.png)

**Step 2:** Create an HCL template file named `jenkins-master.pkr.hcl` and fill it with the following code.
{% details jenkins-master.pkr.hcl %}
``` packer
packer {
  required_plugins {
    amazon = {
      source  = "github.com/hashicorp/amazon"
      version = "~> 1"
    }
  }
}

source "amazon-ebs" "jenkins-master" {
  ami_description = "Amazon Linux Image with Jenkins Server"
  ami_name        = "jenkins-master-{{timestamp}}"
  instance_type   = "${var.instance_type}"
  profile         = "${var.aws_profile}"
  region          = "${var.region}"
  source_ami_filter {
    filters = {
      name                = "amzn2-ami-hvm*"
      root-device-type    = "ebs"
      virtualization-type = "hvm"
    }
    most_recent = true
    owners      = ["amazon"]
  }
  ssh_username = "ec2-user"
  tags = {
    "Name"        = "Jenkins Master"
    "Environment" = "SandBox"
    "OS_Version"  = "Amazon Linux 2"
    "Release"     = "Latest"
    "Created-by"  = "Packer"
  }
}

build {
  name    = "jenkins-master"
  sources = ["source.amazon-ebs.jenkins-master"]

  provisioner "shell" {
    execute_command = "sudo -E -S sh '{{ .Path }}'"
    script          = "./setup-jenkins-master.sh"
  }
}
```
{% enddetails %}

This file consists of 3 main blocks:
1. `Packer` block that we can define our required plugins for AMI creation
2. `Source` block that we can define our source AMI (base AMI)
3. `Build` block that we can define several `provisioner` or what packer need to run to create a custom AMI

We can get the Source AMI using the `source_ami_filter` attribute. So that Packer will automatically populate the `source_ami` based on the filtering criteria that we are defined on the template. If multiple AMIs meet all of the filtering criteria provided in `source_ami_filter`, the `most_recent` attribute will select the newest Amazon Linux AMI.

The `build` block has a `provisioner` stage that is responsible for installing and configuring all needed dependencies. Packer fully supports multiple modern configuration management tools such as Ansible, Chef, or even Bash scripts are also supported. 

We can also see that several arguments need to pass a variable that can be overridden at the Packer runtime. So, let's create the variables file.

**Step 3:** Create the variables file and call it `variables.pkr.hcl` that defined our variable on our main packer template.
{% details variables.pkr.hcl %}
```packer
variable "aws_profile" {
  type    = string
  default = "default"
}

variable "instance_type" {
  type    = string
  default = "t3.micro"
}

variable "region" {
  type    = string
  default = "us-east-1"
}
```
{% enddetails %}

**Step 4:** We also need to create a bash script called `setup-jenkins-master.sh` for installing and configuring all dependencies in `provisioner` stage

{% details setup-jenkins-master.sh %}
``` bash
#!/bin/bash

echo "Installing Amazon Linux extras"
amazon-linux-extras install epel -y

echo "Install Jenkins stable release"
yum remove -y java
amazon-linux-extras install java-openjdk11 -y
wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat-stable/jenkins.repo
rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io-2023.key
yum install -y jenkins
chkconfig jenkins on
systemctl start jenkins
systemctl status jenkins
journalctl -u jenkins
```
{% enddetails %}

The script is straightforward, it will install Amazon Linux Extras, Java Development Kit (JDK), and also Jenkins. Once the Jenkins package is installed with the Yum package manager, the script configures Jenkins to start automatically if the machine has been restarted with the `chkconfig` command.

**Step 5:** We can also create an environment variable file that will override the default value on the `variables.pkr.hcl` file. This step is optional, but if you want, you can create a file with the pattern `*.auto.pkrvars.hcl`, for example `jenkins-master.auto.pkrvars.hcl`

{% details jenkins-master.auto.pkrvars.hcl %}
``` packer
aws_profile   = "packer" # change with your aws profile name
instance_type = "t3.micro"
region        = "ap-southeast-3"
```
{% enddetails %}



**Step 6:** We need to create a `packer` user with a custom IAM policy to create an AMI. For this, let's create a file called `packer-iam-policy.json`
> I assume that you are familiar with configuring AWS CLI. You can also refer to this [AWS documentation](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html).

{% details packer-iam-policy.json %}
```json
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action" : [
      "ec2:AttachVolume",
      "ec2:AuthorizeSecurityGroupIngress",
      "ec2:CopyImage",
      "ec2:CreateImage",
      "ec2:CreateKeypair",
      "ec2:CreateSecurityGroup",
      "ec2:CreateSnapshot",
      "ec2:CreateTags",
      "ec2:CreateVolume",
      "ec2:DeleteKeyPair",
      "ec2:DeleteSecurityGroup",
      "ec2:DeleteSnapshot",
      "ec2:DeleteVolume",
      "ec2:DeregisterImage",
      "ec2:DescribeImageAttribute",
      "ec2:DescribeImages",
      "ec2:DescribeInstances",
      "ec2:DescribeInstanceStatus",
      "ec2:DescribeRegions",
      "ec2:DescribeSecurityGroups",
      "ec2:DescribeSnapshots",
      "ec2:DescribeSubnets",
      "ec2:DescribeTags",
      "ec2:DescribeVolumes",
      "ec2:DetachVolume",
      "ec2:GetPasswordData",
      "ec2:ModifyImageAttribute",
      "ec2:ModifyInstanceAttribute",
      "ec2:ModifySnapshotAttribute",
      "ec2:RegisterImage",
      "ec2:RunInstances",
      "ec2:StopInstances",
      "ec2:TerminateInstances"
    ],
    "Resource" : "*"
  }]
}
```
{% enddetails %}

We can use this shell command to create a `packer` user using AWS CLI command and attach the `packer-iam-policy` to this user
``` shell
aws iam create-user --user-name packer

aws iam put-user-policy --user-name packer --policy-name packer-iam-policy --policy-document file://packer-iam-policy.json

aws iam create-access-key --user-name packer
```

Take note of the `AccessKeyId` and `SecretAccessKey` because the packer will need this key to access AWS resources.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hjvbnn4ekjqsv1n9j42k.png)


**Step 7:** With a properly configured IAM user, it is time to build our first image. We need to run `packer init .` for the first time. It will give the output like this.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fq7thkk3cbe2mszuldgt.png)


**Step 8:** After initializing, run the `packer build .` command to start the Packer Build.

Packer will create temporary Key Pair, Security Group, EC2 Instance, based on the configuration specified in the HCL template file and then execute the bash script on the deployed instance.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lon144qugq6msy41w3zb.png)


Packer Spin-Up Temporary EC2 Instance


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/r6dbda68utbvpzdrog7l.png)


At the end of running the `packer build .` command, Packer will automatically clean up the temporary resource (Key Pair, Security Group, EC2 Instance) and write outputs of the artifacts created as part of the build. Artifacts are the results of a build and are typically represented by the AMI ID.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/j7d4ztieguejitty7n0b.png)


Packer Automatically Terminate Our Temporary EC2 Instance


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yltl4c9xc0fldep6cucl.png)


**Step 9:** We can verify the newly created AMI on the EC2 Console. Go to `EC2` and then click on `AMIs`. Filter on the `Owned by me`.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2ydd05t9bwfr3sbxh0uy.png)

---

## Launch Our Jenkins EC2 Using the New AMI

Yeah!!! Finally, our Jenkins AMI has been created. Let’s test it out and see if Jenkins has been properly installed. For this, we can test Launch Instance via Console, but in this article I will use Terraform.

**Step 1:** Verify Terraform is available on our local machine. Just type `terraform` in our terminal.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lc228pikwj3fdki63mkw.png)



**Step 2:** Create a file named `main.tf`, and fill it with the following code.
{% details main.tf %}
``` terraform
provider "aws" {
  region  = var.region
  profile = var.aws_profile
}

data "aws_ami" "packer_image" {
  most_recent = true

  filter {
    name   = "tag:Created-by"
    values = ["Packer"]
  }

  filter {
    name   = "tag:Name"
    values = [var.app_name]
  }

  owners = ["self"]
}

resource "aws_security_group" "allow_jenkins" {
  name        = "allow_jenkins"
  description = "Allow inbound traffic to jenkins 8080"

  ingress {
    description = "Jenkins Inbound Traffic"
    from_port   = 8080
    to_port     = 8080
    protocol    = "tcp"
    cidr_blocks = var.allowed_ip
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name = "allow_jenkins"
  }
}

resource "aws_iam_role" "jenkins_role" {
  name = "jenkins-iam-role"

  assume_role_policy = jsonencode({
    "Version" = "2012-10-17",
    "Statement" = [
      {
        "Action" = "sts:AssumeRole",
        "Effect" = "Allow",
        "Sid"    = "",
        "Principal" = {
          "Service" = "ec2.amazonaws.com"
        }
      }
    ]
  })
}

resource "aws_iam_role_policy_attachment" "AmazonSSMManagedInstanceCore" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
  role       = aws_iam_role.jenkins_role.name
}

resource "aws_iam_instance_profile" "jenkins_instance_profile" {
  name = "jenkins-instance-profile"
  role = aws_iam_role.jenkins_role.name
}

resource "aws_instance" "jenkins_ami" {
  ami                    = data.aws_ami.packer_image.id
  instance_type          = var.instance_type
  iam_instance_profile   = aws_iam_instance_profile.jenkins_instance_profile.name
  vpc_security_group_ids = [aws_security_group.allow_jenkins.id]

  tags = {
    "Name" = var.app_name
  }
}
```
{% enddetails %}

This Terraform script will create a `Security Group` to open Inbound Access on port 8080 (default Jenkins Port) from the specific CIDR (we can change it on the Terraform Variable). It will also create an `IAM Role` with SSM Policy because we will access our EC2 Using `SSM Session Manager` (We don't open SSH port). Lastly, it will create an `EC2 Instance` in which the AMI will be queried based on the data block we filtered based on the `new AMI created by Packer`.

**Step 3:** We need to create a variable file to define our variable block that will be used on the main template.
So, let's create `variables.tf`, and fill it with the following code.

{% details variables.tf %}
``` terraform
variable "region" {
  type        = string
  description = "AWS Region"
  default     = "us-east-1"
}

variable "app_name" {
  type        = string
  description = "Application Name"
  default     = "Jenkins Master"
}

variable "instance_type" {
  type        = string
  description = "EC2 Instance Type"
  default     = "t3.small"
}

variable "aws_profile" {
  type        = string
  description = "Custom AWS Profile"
  default     = "default"
}

variable "allowed_ip" {
  type        = list(string)
  description = "List of Allow IP Address to Access EC2"
  default     = ["0.0.0.0/0"]
}
```
{% enddetails %}

**Step 4:** We can also create the outputs file, so the Terraform will print the output value once the provisioning process is done. This step is not mandatory, but it is still worth having when we deal with huge Terraform scripts with modules. :)
So, let's create `outputs.tf`, and fill it with the following code.

{% details outputs.tf %}
``` terraform
output "public_ip" {
  value = aws_instance.jenkins_ami.public_ip
}

output "public_dns" {
  value = aws_instance.jenkins_ami.public_dns
}

output "jenkins_url" {
  value = "http://${aws_instance.jenkins_ami.public_dns}:8080"
}
```
{% enddetails %}

**Step 5:** Last but not least, we can also create an environment variable file that will override the default value on the `variables.tf` file. We can name it `terraform.tfvars`, so Terraform will automatically detect the new value of variables.

{% details terraform.tfvars %}
``` terraform
region        = "ap-southeast-3"
instance_type = "t3.micro"
aws_profile   = "kobokan-aer" # change with your aws profile name
allowed_ip    = ["10.10.10.10/32"] # change with your IP

```
{% enddetails %}

**Step 6:** Finally, we can try to run our Terraform scripts. For the first, we need to run `terraform init` to initilizing the backend and install the required plugins.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/l9vmy2sp7yrd9jx9r1rp.png)

Terraform will create this file and directory if the initializing runs successfully.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/83fge8v8ydum8v9flnu3.png)

**Step 7:** After initializing, run the `terraform apply -auto-approve` to start resource provisioning.
> In the real project, always run the `terraform plan` first to see the list of resources Terraform will create.

We can see that Terraform will automatically pull the latest AMI created by Packer.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0c2j0e8n6k0j6k3cgugd.png)

At the end of the `terraform apply`, we can verify the Jenkins App will be automatically installed on our EC2. Simply copy the `jenkins_url` output on our favorite Browser.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/f8p9z7fsmgwn9ogswvsv.png)

Jenkins automatically installed and running

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/cx3gcy03hvay8hlvnynz.png)

**Step 8:** If you want, you can test to use the Jenkins. Just copy the Initial Admin Password on the EC2. Click on the Jenkins Master EC2 -> Connect -> Session Manager -> Connect

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jsivkhxokelj6qknkfi2.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/y48y88rkdscel69pyn2i.png)

**Step 9:** Copy this password on the Jenkins UI, and we only need to follow the steps that Jenkins provides will be straightforward.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ry4jyx7ktu640058swn3.png)

Yeayy!!! we successfully Set Up our Jenkins Master on the EC2 Using the AMI from Packer. Feel free to play around with that.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fgwrnlzrwictxelf24hu.png)

---

## Clean Up

To clean up our resources, simply run the following shell command on our local terminal.
``` shell
aws ec2 deregister-image --image-id <your-ami-id>

aws ec2 delete-snapshot --snapshot-id <your-snapshot-id>

# On the Terraform Working Directory
terraform destroy -auto-approve
```

---

## Conclusion

Now that we can use Jenkins AMI to create Jenkins instances. Terraform will pull the latest AMI created by Packer. We can still improve it by integrating it into the pipeline to automatically create AMI on the Packer workflow and provision the EC2 using Terraform. But it is totally AWSome for now :)

---

## Thank you for Reading

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gzazg2fu0e8q0qgarway.gif)