Manage JSON with Puppet and Augeas
Tags: devops, puppet
Published: 2023-04-14 17:20:01

---
title: Manage JSON with Puppet and Augeas
published: true
date: 2023-03-30 05:00:00 UTC
tags: Devops,Puppet
canonical_url: https://h0tw1r3.github.io/posts/manage-json-puppet-augeas/
---
[JSON] (JavaScript Object Notation) is a popular data interchange format used in many applications. As they become more complex, managing [JSON] configuration files can become a daunting task. Fortunately, automation tools like [Puppet] and [Augeas] can simplify the process.

Lets explore how to use [Puppet] to automate changes to JSON configuration files.

## What is Puppet?

[Puppet] is an open-source configuration management tool that helps system administrators automate the deployment, configuration, and management of their infrastructure. With Puppet, you can define the desired state of your infrastructure using code, and Puppet will automatically enforce that state on your systems.

## What is Augeas?

[Augeas] is a configuration editing tool that provides a simple API for editing configuration files. Augeas is particularly useful for editing configuration files that have complex or non-standard syntax.

## Example Configuration

Let's say you have a JSON configuration file for your web application. In the following examples, we'll be using the following JSON in a file named `/tmp/myapp.conf`.

```json
{
    "database": {
        "host": "localhost",
        "port": 5432,
        "name": "mydb",
        "user": "myuser",
        "password": "mypassword"
    },
    "server": {
        "host": "localhost",
        "port": 8080
    }
}
```

## Simple Example

Suppose you want to change the port number for the "server" section from 8080 to 8000.

### Step 1: Create a Puppet manifest

Create `/tmp/myapp.pp`, that defines the desired state of the JSON configuration file:

```puppet
augeas { "change_server_port":
  incl    => '/tmp/myapp.conf',
  lens    => 'Json.lns',
  context => '/files/tmp/myapp.conf',
  changes => [
    'set dict/entry[.="server"]/dict/entry[.="port"]/number 8000',
  ],
}
```

### Step 2: Apply the Puppet manifest

```sh
$ cd /tmp && puppet apply myapp.pp
Notice: Compiled catalog for test.local in environment production in 0.02 seconds
Notice: /Stage[main]/Main/Augeas[change_server_port]/returns: executed successfully
Notice: Applied catalog in 0.02 seconds
```

Review the JSON configuration file. Notice Puppet changed the value!

```json
{
  "database": {
    "host": "localhost",
    "port": 5432,
    "name": "mydb",
    "user": "myuser",
    "password": "mypassword"
  },
  "server": {
    "host": "localhost",
    "port": 8000
  }
}
```

If you run `puppet apply` again, there should be no changes:

```
$ puppet apply myapp.pp
Notice: Compiled catalog for test.local in environment production in 0.03 seconds
Notice: Applied catalog in 0.02 seconds
```

That's the **idempotent** power of Puppet!

## Advanced Example

How about managing arbitrary dictionary values from [hiera]?

### Step 1: Modify the Puppet manifest file

Replace the content of `/tmp/myapp.pp` with:

```puppet
class myapp (
  String $conf_path = '/tmp/myapp.conf',
  Hash   $config    = {},
) {
  $changes = $config.map |$kv| {
    $keys = $kv[0].split('/')
   
    Integer[0,$keys.length].reduce([]) |$memo, $i| {
      if $i == $keys.length {
        Integer[0,($memo.length - 1)].map |$mi| {
          $change = sprintf('defnode entry %s "%s"', $memo[$mi][1,-1], $keys[$mi])
          if ($mi + 1) == $memo.length {
            $json_type = $kv[1] ? {
              Boolean => 'const',
              Numeric => 'number',
              default => 'string',
            }
            [ $change, sprintf('set $entry/%s "%s"', $json_type, $kv[1]) ]
          } else {
            [ $change ]
          }
        }
      } else {
        $memo << sprintf('%s/dict/entry[.="%s"]', $memo.join('/'), $keys[$i])
      }
    }
  }.flatten
   
  augeas { 'myapp':
    incl    => $conf_path,
    lens    => 'Json.lns',
    context => "/files${conf_path}",
    changes => $changes,
  }
}
   
include myapp
```

### Step 2: Create a hiera _configuration_ file

Create the file `/tmp/hiera.yaml` with the following content:

```yaml
---
version: 5
defaults:
  datadir: .
  data_hash: yaml_data
hierarchy:
 - name: Common
   data_hash: yaml_data
   path: 'myapp.yaml'
```

### Step 3: Create a hiera _data_ file

Create the file `/tmp/myapp.yaml` with the following content:

```yaml
---
myapp::config:
  database/ssl: true
  database/host: db.somewhere.local
  server/port: 9000
```
   
### Step 4: Apply the Puppet manifest

```sh
$ cd /tmp && puppet apply --hiera_config hiera.yaml myapp.pp
Notice: Compiled catalog for test.local in environment production in 0.04 seconds
Notice: /Stage[main]/Myapp/Augeas[myapp]/returns: executed successfully
Notice: Applied catalog in 0.04 seconds
```

Review the JSON configuration file. Notice Puppet changed the values and added a new key!

```json
{
    "database": {
        "host": "db.somewhere.local",
        "port": 5432,
        "name": "mydb",
        "user": "myuser",
        "password": "mypassword"
    ,"ssl":true},
    "server": {
        "host": "localhost",
        "port": 9000
    }
}
```

_Augeas does not keep the JSON "pretty" when adding new keys._

[JSON]: https://www.json.org/
[Puppet]: https://www.puppet.com/docs/puppet/
[Augeas]: https://forge.puppet.com/modules/puppetlabs/augeas_core/readme
[hiera]: https://www.puppet.com/docs/puppet/latest/hiera.html