Implementing infrastructure-as-code with Ansible and GIT
Tags: ansible, linux, automation, devops
Published: 2021-10-01 22:17:04

---
title: "Implementing infrastructure-as-code with Ansible and GIT"
date: "2021-09-04"
lastmod: "2021-09-18"
summary: "Manage compute nodes using the infrastructure-as-code concept using Ansible and GIT"
tags:
  - ansible
  - linux
  - automation
  - devops
---
## What is "infrastructure-as-code"?

To keep the concept simple, think of your infrastructure as a picture (end-state) and the main characteristics (configuration) that you would use to describe it.

Using this information you should be able to reproduce the picture any time you need, and the result should always be the same as the original.

There are several tools and methods to implement this approach but the most important thing to consider is that you also need to adapt your management procedures to switch from the traditional imperative model to the infrastructure-as-code declarative model.

## Challenges

To successfully implement infrastructure-as-code special attention must be paid to the selection of supporting tools and the definition of the conceptual model that will represent the managed environment:

- Automation tool: takes control of the infrastructure configuration and performs the necessary actions to reach the desired end-state.
- Code Repository and Versioning: stores the infrastructure-model and automation scripts to manage the infrastructure and tracks changes.
- Infrastructure-model: conceptual data model that describes the desired end-state of the infrastructure.

## Implementing infrastructure-as-code

Let's take the following example scenario to demonstrate the implementation procedure:

- Environment: Home Office
- Ansible Control Node: small PC or VM installed with Centos 8.4
- Ansible Managed Nodes: notebooks with Ubuntu and workstations with Centos and Fedora

![Home Office Site](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0xmvc0pim74ce01aoenw.png)

Before starting the implementation procedure, make sure the following requirements are meet:

- Control Node:
  - OpenSSH client
  - Sudo
  - Python3
  - GIT
  - Regular user with SUDO configured for password less root privilege
- Managed Nodes:
  - Fresh OS install (standard setup)
  - OpenSSH server
  - Sudo
  - Python3
  - Regular user with SUDO configured for password less root privilege

### Define the Infrastructure Model

The infrastructure-model will have the following data structures:

- **Site**: Represents a group of **Nodes** that are managed by the same **Control Node**.
- **Node**: Compute node that is capable of hosting software components and that is fully managed by the Control Node.
- **Component**: Individual software product that is installed in a **Node**.
- **Service**: Group of **Components** configured in one or more **Nodes** to serve a particular function.

The data structures will be implemented using Ansible inventories, host_vars, and group_vars:

- **Inventory**: the `hosts.ini` file will be used to declare all the **Nodes** for the target **Site**. **Nodes** will be grouped based on the **Service** they provide or use.
- **GroupVars**: individual YAML files for **Components** and **Services** will be created for each **Node** group declared in the **Inventory**.
- **HostVars**: individual YAML files will be used for cases where the **Node** requires further customization.

### Create the Code Repository

Create a dedicated Linux account. This is to isolate content from regular users and facilitate activity auditing. Modify the shell variable `PROJECT_OWNER` to change the default name.

```shell
PROJECT_OWNER='sitectl'
sudo useradd -m "$PROJECT_OWNER"
sudo su - "$PROJECT_OWNER"

```

Create the project directory structure that will contain the infrastructure-model and automation scripts. Refer to the [Ansible Best Practices](https://docs.ansible.com/ansible/2.8/user_guide/playbooks_best_practices.html#directory-layout) document to further learn about the directory structure.
Modify the shell variable `PROJECT_PATH` to change the default project location.

```shell
export PROJECT_OWNER='sitectl'
export PROJECT_PATH="/home/${PROJECT_OWNER}/manager"
mkdir "$PROJECT_PATH"
cd "$PROJECT_PATH"
mkdir \
  'collections' \
  'files' \
  'inventories' \
  'inventories/group_vars' \
  'inventories/host_vars' \
  'playbooks' \
  'vars' \
  'etc' \
  'filter_plugins' \
  'library' \
  'module_utils' \
  'roles' \
  'var' \
  'templates'

```

Create a simple shell script to set environment variables for Ansible. Refer to the [Ansible Configuration](https://docs.ansible.com/ansible/latest/reference_appendices/config.html) documentation to understand what are the `ANSIBLE_*` shell variables doing.

```shell
cat > 'load_environment.sh' <<-EEOF
#!/bin/bash

declare -x PROJECT_OWNER='$PROJECT_OWNER'
declare -x PROJECT_PATH='$PROJECT_PATH'
declare -x PROJECT_END_STATE='${PROJECT_PATH}/inventories'

declare -x ANSIBLE_INVENTORY="\${PROJECT_END_STATE}/hosts.ini"
declare -x ANSIBLE_PRIVATE_KEY_FILE="/home/\${PROJECT_OWNER}/.ssh/id_rsa"
declare -x ANSIBLE_COLLECTIONS_PATHS="\${PROJECT_PATH}/collections"
declare -x ANSIBLE_ROLES_PATH="\${PROJECT_PATH}/roles"
declare -x ANSIBLE_GALAXY_CACHE_DIR="\${PROJECT_PATH}/var"
declare -x ANSIBLE_LOG_PATH="\${PROJECT_PATH}/var/ansible.log"
declare -x ANSIBLE_PYTHON_INTERPRETER='/usr/bin/python3.9'
declare -x ANSIBLE_PLAYBOOK_DIR="\${PROJECT_PATH}/playbooks"

PATH='/home/${PROJECT_OWNER}/.local/bin:/usr/bin:/usr/sbin'

EEOF

```

Create the Code Repository using GIT. Configure the `.gitignore` file to avoid tracking changes in the ansible-galaxy install location target (`collections/`) and in the repository for temporary and volatile files (`/var`)

```shell
source load_environment.sh
cat > '.gitignore' <<-EEOF
collections/
var/

EEOF
git config user.email "${PROJECT_OWNER}@localhost.localdomain"
git config user.name "$PROJECT_OWNER"
git init
git add .
git commit -m "Initial commit"

```

### Configure Ansible

Install the latest Ansible engine to the control node user. This method keeps the environment isolated, minimizes os-distribution dependencies, and facilitates module upgrades.

```shell
"$ANSIBLE_PYTHON_INTERPRETER" -m pip install ansible

```

Prepare remote access to Ansible Managed Nodes using OpenSSH keys. Modify the shell variable `PROJECT_MANAGED_NODES` to represent your environment and set the `PROJECT_MANAGED_USER` variable to the remote user name with password-less root privilege.

```shell
PROJECT_MANAGED_NODES='host1 host2 host3 host4'
PROJECT_MANAGED_USER='sysadmin'
ssh-keygen -t rsa -f "$ANSIBLE_PRIVATE_KEY_FILE" -N ""
for x in $PROJECT_MANAGED_NODES; do
  ssh-copy-id -i "$ANSIBLE_PRIVATE_KEY_FILE" ${PROJECT_MANAGED_USER}@${x}
done

```

Create the initial Ansible inventory registering the following **Node** groups:

- `[control_node]`: defines the Ansible Node.
- `[managed_nodes]`: defines Ansible Managed nodes for the target site.
- `[office_nodes]`: defines **Nodes** that will consume the **office-service**. The service provides users with common productivity applications. For this example, we'll use the image editor GIMP.

```shell
cat > "$ANSIBLE_INVENTORY" <<-EEOF
[control_node]
localhost

[managed_nodes]
$(for x in $PROJECT_MANAGED_NODES; do echo "$x"; done)

[office_nodes]
$(for x in $PROJECT_MANAGED_NODES; do echo "$x"; done)

EEOF

```

Create end-state configuration repositories. This is where the infrastructure-model will be implemented.

- `group_vars/`: one directory per host group
- `group_vars/host_group_x/`: one or more YAML files representing the components that will be available for all hosts in the group
- `host_vars/`: one directory per host
- `host_vars/hostx/`: one or more YAML files representing the components that will be available for the host

```shell
for x in control_node managed_nodes office_nodes; do
  mkdir "${PROJECT_END_STATE}/group_vars/${x}"
done
for x in $PROJECT_MANAGED_NODES; do
  mkdir "${PROJECT_END_STATE}/host_vars/${x}"
done

```

### Define component end-states

Now that the data repository for the infrastructure-model is created it can be populated with end-state targets. Notice that you can also add behaviour definitions to keep variable data separated from the code.

Define how is the Ansible engine going to connect to the managed nodes:

```shell
cat > "$PROJECT_END_STATE/group_vars/managed_nodes/ansible.yml" <<-EEOF
---
ansible_user: "$PROJECT_MANAGED_USER"
ansible_become_method: "sudo"
...

EEOF

```

Define the attributes for the `Linux Users` component. This definition will be applied to all hosts in the group `managed_nodes`:

```shell
cat > "$PROJECT_END_STATE/group_vars/managed_nodes/users.yml" <<-EEOF
---
managed_nodes_users:
  - name: "user1"
    description: "Regular User 1"
    uid: "10100"
  - name: "user2"
    description: "Regular User 2"
    uid: "10101"
...

EEOF

```

Define the attributes for the `Linux Package` component. This definition will be applied to all hosts in the group `office_nodes`:

```shell
cat > "$PROJECT_END_STATE/group_vars/office_nodes/packages.yml" <<-EEOF
---
office_nodes_packages:
  flatpak:
    - "flatpak"
  gimp:
    - "org.gimp.GIMP"
...

EEOF

```

### Bring the site to the target end-state

At this point end-state, and behaviour definitions are set. Now it's time to write the code that will apply it to the target hosts.

Create the Ansible Playbook that will configure the `managed_nodes` host group:

```shell
cat > "$ANSIBLE_PLAYBOOK_DIR/managed_nodes.yml" <<-EEOF
---
- name: "Manage Ansible Nodes"
  hosts: "managed_nodes"
  gather_facts: false

  tasks:
    - name: "Create Regular User Accounts"
      become: true
      ansible.builtin.user:
        create_home: true
        state: "present"
        name: "{{ item['name'] }}"
        comment: "{{ item['description'] | default( omit ) }}"
        uid: "{{ item['uid'] | default( omit ) }}"
      loop: "{{ managed_nodes_users }}"
...

EEOF

```

Create the Ansible Playbook that will configure the `office_nodes` host group:

```shell
cat > "$ANSIBLE_PLAYBOOK_DIR/office_nodes.yml" <<-EEOF
---
- name: "Manage Office Nodes"
  hosts: "office_nodes"
  gather_facts: false

  pre_tasks:

    - name: "Install FlatPak tools"
      become: true
      ansible.builtin.package:
        name: "{{ office_nodes_packages['flatpak'] }}"
        state: "present"

    - name: "Prepare FlatPak repository"
      become: true
      ansible.builtin.command:
        argv:
          - "/usr/bin/flatpak"
          - "--system"
          - "remote-add"
          - "flatpak"
          - "https://flathub.org/repo/flathub.flatpakrepo"
      register: result
      changed_when:
        - result['rc'] == 0

  tasks:
    - name: "Install GIMP from FlatHub"
      become: true
      community.general.flatpak:
        name: "{{ office_nodes_packages['gimp'] }}"
        state: "present"
...

EEOF

```

Execute the playbooks to apply the end-state:

```shell
ansible-playbook playbooks/managed_nodes.yml
ansible-playbook playbooks/office_nodes.yml

```

Save the changes to the repository:

```shell
git add inventories
git add playbooks
git commit -m "add office_node and managed_node plays"

```

## Next steps

Now the base structure is up and running more content can be added, either from Ansible Galaxy or developed in-house.

In addition to the automation engine and code repository you should evaluate incorporating:

- Code linters: [Ansible Lint](https://ansible-lint.readthedocs.io/en/latest/) and [YAMLlint](https://yamllint.readthedocs.io/en/stable/) can help to keep code consistent and standardized.
- Testing: [Ansible Molecule](https://molecule.readthedocs.io/en/latest/) can be used to build and run test environments for testing in-house roles
- Provisioning: [Terraform](https://www.terraform.io/docs/index.html) can be used to automate the creation of standardized VMs

Explore the [A:Platform64](https://readthedocs.org/projects/aplatform64/) project that facilitates the implementation of infrastructure-as-code by automating most of the tasks described in this tutorial.

## Copyright information

This article is licensed under a [Creative Commons Attribution 4.0 International License](http://creativecommons.org/licenses/by/4.0/). For copyright information on the product or products mentioned inhere refer to their respective owner.

## Disclaimer

Opinions presented in this article are personal and belong solely to me, and do not represent people or organizations associated with me in a professional or personal way. All the information on this site is provided "as is" with no guarantee of completeness, accuracy or the results obtained from the use of this information.