Kubernetes Configuration Management with Ansible (Part 2) 
Tags: ansible, devops, kubernetes, automation
Published: 2020-06-22 19:03:34

Follow-up of this [post](/baptistemm/kubernetes-configuration-management-with-ansible-part-1-4aem). We explained the directory organization and the manifest files naming convention, now let's explore the ansible part.

Ansible was chosen because I knew it a bit since 2015 when I started automating some repetitive tasks to create a Kafka cluster prototype. Ansible is a good candidate because it can talk to HTTP API and it comes with a list of [k8s](https://docs.ansible.com/ansible/latest/modules/list_of_clustering_modules.html) related modules that make easy to query or manage Kubernetes objects (they were rewritten a couple of time before being now really good).

## Overview of the Code

I don't plan to review all code, but have a look of key parts of the Ansible code.

* The cluster accept a parameter which is the target cluster, then we load the common variables file then the cluster variable file from `config` directory

```yaml
---
- name: "Deploy manifests on cluster {{ target_cluster }}"
  connection: local
  gather_facts: false
  vars:
    authorized_cluster: ["dev", "prod"]
  hosts: localhost
  
  pre_tasks:
    - name: Check if the target_cluster variable was properly set
      fail:
        msg: "You need to specify the variable target_cluster with one value ({{ authorized_cluster }})"
      when: target_cluster not in authorized_cluster

    - name: Load common variables
      include_vars:
        file: configs/common.yml

    - name: "Load cluster-scoped variables for {{ target_cluster }}"
      include_vars:
        file: "config/{{ target_cluster }}.yml"
```

* for the given cluster, we create a list of all yaml files from the `common` and `$cluster` directories. We will also define some variables than will be used later

```yaml
---
- name: "Retrieve manifests files from common and {{ cluster }} directories"
  find:
    paths: ["{{ yaml_dir }}/common", "{{ yaml_dir }}/{{ cluster }}"]
    patterns: ['*.yaml', '*.yml']
    file_type: file
    recurse: true
  register: manifests_list

- name: "Define variable from manifests filename"
  vars:
    filename: "{{ item.path | basename }}"
    kind: "{{ item.path | basename | regex_replace('.*_(.*)_.*_.*_.*\\.ya?ml', '\\1') }}"
    name: "{{ item.path | basename | regex_replace('.*_.*_(.*)_.*_.*\\.ya?ml', '\\1') }}"
    namespace: "{{ item.path | basename | regex_replace('.*_.*_.*_(.*)_.*\\.ya?ml', '\\1') }}"
    state: "{{ item.path | basename | regex_replace('.*_.*_.*_.*_(.*)\\.ya?ml', '\\1') }}"

  set_fact:
    manifests_information: "{{ manifests_information + [{ 'filepath': item.path, 'filename': filename, 'kind': kind, 'name': name, 'namespace': namespace, 'state': state }] }}"
  loop: "{{ manifests_list.files | sort(attribute='path') }}"
  loop_control:
    label: "{{ filename }}"
```

* Now we check each file honors the filename convention else we throw an error.
```yaml
- name: "Check files match the naming convention"
  fail:
    msg: >
      The file {{ item.filepath }} does not observe the manifest naming convention.
  when: item.filename == item.kind or
    item.filename == item.name or
    item.filename == item.namespace or
    item.state not in [ "present", "absent" ]
  with_items:
    - "{{ manifests_information }}"
  loop_control:
    label: "{{ item.filepath }}"
```

* for each file, call the `k8s` module, the path of of the file is passed to argument using the [lookup plugin template](https://docs.ansible.com/ansible/latest/plugins/lookup/template.html), so the manifest can have logic and variables inside.

```yaml
---
- name: "Deploy {{ item.kind }} {{ item.name }} on namespace {{ item.namespace }} to {{ item.state }}"
  k8s:
    host: "https://xxxxxx:6443"
    api_key: "{{ token }}"
    state: "{{ item.state }}"
    definition: "{{ lookup ('template', item.filepath) }}"
```

For the code we're done, for a stripped down version it does not require much than that to work.

## Coding your Manifests

One of the feature this small playbook permits is to have simple manifest yaml file like this
```yaml
---
apiVersion: project.openshift.io/v1
kind: Project
metadata:
  name: "{{ cluster-admin-ns }}"
...
```
but you can also using feature like Ansible variables and Jinja templating together.
For instance, I had to put a config file into a secret, which require to have the content base64 encoded. It's convenient when you can do that automatically.
* In this example, the content for the key `fluent.conf` come from a file `fluent.conf.j2` loaded by the `template` lookup (so the variable are replaced) then passed to the filter `b64encode`.
```yaml
---
apiVersion: v1
data:
  fluent.conf: "{{ lookup('template', 'fluent.conf.j2') | b64encode }}"
kind: secret
metadata:
  name: fluent-forwarder-secret
  namespace: logging
...
```
with the configuration file treated as a template
```
# fluent.conf.j2
# As this file is interpreted as a ansible template, you can add
# jinja code inside to put some logic.

<source>
  @type  forward
  port  24224
  tag logs.openshift
</source>

<match **>
  # Send all types to Splunk
  @type splunk_hec
  hec_host {{ splunk.host }}
  hec_port {{ splunk.port }}
  hec_token {{ splunk.token }}
  index {{ splunk.index }}
</match>
```
so it gives once loaded something like
```yaml
apiVersion: v1
data:
  fluent.conf: IyBmbHVlbnQuY29uZi5qMgojIEFzIHRoaXMgZmlsZSBpcyBpbnRlcnByZXRlZCBhcyBhIGFuc2libGUgdGVtcGxhdGUsIHlvdSBjYW4gYWRkCiMgamluamEgY29kZSBpbnNpZGUgdG8gcHV0IHNvbWUgbG9naWMuCgo8c291cmNlPgogIEB0eXBlICBmb3J3YXJkCiAgcG9ydCAgMjQyMjQKICB0YWcgbG9ncy5vcGVuc2hpZnQKPC9zb3VyY2U+Cgo8bWF0Y2ggKio+CiAgIyBTZW5kIGFsbCB0eXBlcyB0byBTcGx1bmsKICBAdHlwZSBzcGx1bmtfaGVjCiAgaGVjX2hvc3Qgc3BsdW5rLmRvbS50bGQKICBoZWNfcG9ydCA4MDg5CiAgaGVjX3Rva2VuIDM1NDM0My0zNDUzMzMtMTUyMzMKICBpbmRleCBmYWtlLWluZGV4CjwvbWF0Y2g+
kind: Secret
```
... Just what expect the secret API.
So I think we're good for now, I just wanted to give you an idea of what it is possible to do. This is a skeleton that can be enhanced with more check and feature.

Perhaps I'll come back with new idea.