Building Immutable Infrastructure with Packer and GitLab CI
Tags: tutorial, gitlab
Published: 2021-12-04 19:24:17

## What is Packer?
[Packer](https://packer.io) is an Open Source tool that can be used to create identical images for different platforms, what it makes the process of creation and deployment of your infrastructure to be simple, as it uses a single configuration file.

### Builders
Builders are Packer components that are responsible of creating a machine and generate correspondent image. There are builders for Docker, Vagrant, VirtualBox, VMWare, etc.

Check the documentation [here](https://www.packer.io/docs/builders) for more information.

### Provisioners
Provisioners are tools that can be used to install software and configure it in generated images. Shell scripts and tools like [Ansible](https://www.ansible.com/), [Chef](https://www.chef.io/) and [Puppet](https://puppet.com/) can be used for provisioning.

Check the documentation [here](https://www.packer.io/docs/provisioners) for more information.

### Installation
Packer can be installed on Windows, Mac and Linux following instructions in the [documentation](https://learn.hashicorp.com/tutorials/packer/get-started-install-cli). If you're using Linux, Packer is available from the repositories of some distributions, in other case you can follow the installation instructions to add the official repository to the system configuration.

If you are building a Docker image, your must install Docker first on your system.

### Templates
For configuring Packer on what instructions to run and what plugins to use for building the image, you must use a [template](https://www.packer.io/docs/templates). There are two formats, [JSON](https://www.packer.io/docs/templates/legacy_json_templates), that is the template that has been used historically, and [HCL](https://www.packer.io/docs/templates/hcl_templates), that stands for HashiCorp Configuration Language, the recommended format since version 1.7.0. The template file must be named using `.pkr.json` or `.pkr.hcl` extensions.

## Using Packer for Vagrant
### What is Vagrant?
[Vagrant](ttps://vagrantup.com) is a tool for building and managing virtual environments. It works with VirtualBox, Docker, VMWare as providers and provisioning can be done using shell scripts and tools like Ansible, Chef and Puppet.

### Using JSON template
```json
{
 "provisioners": [
 {
 "type": "shell",
 "execute_command": "echo 'vagrant' | {{.Vars}} sudo -S -E bash '{{.Path}}'",
 "script": "scripts/install.sh"
 }
 ],
 "builders": [
 {
 "communicator": "ssh",
 "source_path": "ubuntu/focal64",
 "provider": "virtualbox",
 "type": "vagrant",
 "box_name": "python-dev"
 }
 ]
}
```
In the code snippet above, there are two arrays in the JSON object, `builders` of `vagrant` type as a Vagrant box is being built, and `provisioners` of `shell` type as a shell script is being used for provisioning.

In `builders` array:
- `communicator` is set to `ssh` as an SSH connection is required to access the virtual environment created with Vagrant
- `source_path` is equal to `ubuntu/focal64`. This is the based image that will be used for building the final image for the infrastructure.
- `virtualbox` is used as `provider` for Vagrant.
- `type` is set to `vagrant`.
- `python-dev` will be the name of the box created.

In `provisioners` array:
- `type` of provisioner is `shell` as a Bash script is being used for provisioning.
- `execute_command` is the set of instructions to run when the provisioning process is started.
- `script` is the path of the Bash script.

### Using HCL template
```hcl
source "vagrant" "autogenerated_1" {
 add_force = true
 box_name = "python-dev"
 communicator = "ssh"
 provider = "virtualbox"
 source_path = "ubuntu/focal64"
}

build {
 sources = ["source.vagrant.autogenerated_1"]

 provisioner "shell" {
 execute_command = "echo 'vagrant' | {{ .Vars }} sudo -S -E bash '{{ .Path }}'"
 script = "scripts/setup.sh"
 }

}
```
HCL is the recommended template since version 1.7.0 and it contains the same information as in the JSON template.

If you already have a JSON template but want to use HCL instead, you can run the following command to generate `.pkr.hcl` file from `.pkr.json` file:

```bash
$ packer hcl2_upgrade template.pkr.json
```

### Building image
For building the image, run any of the following commands:

```bash
$ packer build config.pkr.json
$ packer build config.pkr.hcl
```

Above command will create an `output_vagrant` directory where `package.box` will be stored and a `Vagrantfile` will be created.

### Running Virtual Env
Before starting the virtual environment, edit the `Vagrantfile` in the `output_vagrant` directory, removing the following lines:

```ruby
config.vm.define "source", autostart: false do |source|
	source.vm.box = "ubuntu/focal64"
	config.ssh.insert_key = false
 end
```

Start your virtual environment running:

```bash
$ vagrant up
$ vagrant ssh
```

## Using Packer for Docker
```hcl
packer {
 required_plugins {
 docker = {
 version = ">= 0.0.7"
 source = "github.com/hashicorp/docker"
 }
 }
}

variable "login_username" { 
 type = string
 default = "username"
}

variable "login_password" { 
 type = string
 default = "password"
}

source "docker" "ubuntu" {
 image = "ubuntu:xenial"
 commit = true
}

build {
 sources = [
 "source.docker.ubuntu"
 ]

 provisioner "shell" {
 script = "./scripts/install.sh"
 }

 post-processors {
 post-processor "docker-tag" {
 repository = "user/ubuntu-docker"
 tags = ["latest"]
 only = ["docker.ubuntu"]
 }

 post-processor "docker-push" {
 login = true
 login_username = var.login_username
 login_password = var.login_password
 }
 }
}
```

For building a Docker image using Packer, a [plugin](https://www.packer.io/docs/plugins) is required and can be obtained from [GitHub](https://github.com/hashicorp/docker). The plugin being used must be specified in the `required_plugins` block in the template.

### Post-processors
Post-processors are instructions run after the image is created and provisioned. In the code snippet above, the Docker image is published on Docker Hub after the image is built.

Two post-processors are defined, `docker-tag`, where `repository` and `tags` are set, and `docker-push`, where authentication details are specified, including username and password (or access token) at Docker Hub.

`login_username` and `login_password` in the `docker-push` post-processor are set after getting values from the command line when running `packer build`. These variables are set in the template:

```hcl
variable "login_username" { 
 type = string
 default = "username"
}

variable "login_password" { 
 type = string
 default = "password"
}
```

Where `type` is set to `string` and a `default` value is assigned. The default value will be replaced later.

### Building image
For building the Docker image run:

```bash
packer build -var login_username=”USER” -var login_password=”PASSWORD” ubuntu-docker.pkr.hcl
```

Image will be built and published on Docker Hub.

## GitLab CI
You can use GitLab CI for automating the building process.

At first you must:

- Create a GitLab repository.
- Upload your `.pkr.hcl` template.
- Upload your `scripts` directory. This contains your provisioning script, a shell script.

### Configure GitLab CI
- Go to Setting --> CI/CD in your repository.
- Click on `Expand` in the `Variables` section.
- Add the `CI_REGISTRY_USER` variable. The value will be your Docker Hub username.
- Add the `CI_REGISTRY_PASSWORD`. The value will be your Docker Hub password or access token.
- Create the `.gitlab-ci.yml` file in your repository. It will have two stages, `test` and `deploy`, and two jobs, `test-job` and `deploy-job`.

```yaml
image:
 name: docker:latest

before_script:
 - echo "Fetching packer"
 - wget https://releases.hashicorp.com/packer/1.7.8/packer_1.7.8_linux_amd64.zip
 - unzip packer_1.7.8_linux_amd64.zip
 - chmod +x packer
 - ./packer init .

stages:
 - test
 - deploy

test-job:
 stage: test
 script:
 - echo "Validating Packer template"
 - ./packer validate ./ubuntu-docker.pkr.hcl

deploy-job:
 stage: deploy
 services:
 - docker:dind
 script:
 - echo "Deploying Docker image..."
 - ./packer build -var login_username=$CI_REGISTRY_USER -var login_password=$CI_REGISTRY_PASSWORD ./ubuntu-docker.pkr.hcl
 - echo "Docker image successfully published."
```

Latest Docker image of Docker is required and before running any of the jobs, Packer must be installed as specified in `before_script`. To validate the Packer template, the following command is run in the `test-job`:

```bash
packer validate ubuntu.docker.pkr.hcl
```

In the `deploy-job`, the values of `login_username` and `login_password` are assigned from the variables in the configuration of the repository, `CI_REGISTRY_USER` and `CI_REGISTRY_PASSWORD`.

Once you commit that file, the pipeline is run and when it finishes your Docker image will be available from Docker Hub.