Unit Testing Complex Jsonnet Objects and Arrays
Tags: jsonnet, kubernetes, unittest
Published: 2020-06-01 17:25:34

---
title: Unit Testing Complex Jsonnet Objects and Arrays
published: true
description: Unit testing kubernetes environment variables in Deployment manifest files
tags: jsonnet, kubernetes, unittest
---

## A Problem

As part of our deployment strategy we generate kubernetes manifest json files from templates. For this we use [Jsonnet](https://jsonnet.org/). We need confidence that the "rendered" json files are correct depending on configuration that has been given to it. Therefore, we unit test them. We use the really helpful [JsonnetUnit](https://github.com/yugui/jsonnetunit) to achieve this. It's a really handy library that behaves like a unit testing runner, but is written entirely in jsonnet.

One issue we had was that [JsonnetUnit](https://github.com/yugui/jsonnetunit) compares objects in their entirety. For example, to assert the state of pod environment variables in a kubernetes manifest, you need to compare the entire list of env vars.

```js
testEnvVars: {
    actual: deploy.spec.template.spec.containers[0].env,
    expect: [
            { name: 'ENVVAR1', value: 'VALUE1' },
            { name: 'FEATURE_A', value: 'true' },
        ]
  },
```

Now this is easy enough if you have a small list of env vars, but if the intention of your unit test is to check if some extra values are provided when a feature is turned on, you dont want to have to compare the entire list of environment variable for the sake of a checking a couple of new ones exist.

## A Solution
The following jsonnet code behaves like an "array contains" function. It returns a boolean response depending if the array (haystack) contains the required objects (needles). 

As you might expect, when working in Json as a testing language (🤨), it's not a simple as calling a lookup function on the array. Kubernetes doesn't make life easier either, as env vars are stored as an `array` of `objects` rather than the more traditional form of a map of `key`, `values`.

Because of this, the function needs to know the structure of the object it is looking for in advance. Which is fine until you want to do a similar test for a slightly different shaped object (`SecretEnvVars` et al.), which requires the code to be copy pasted and amended.

```js
{
  // containsEnvVars checks that all needles (k8s env variable "name" and "value" objs) exist in the haystack (pod env block) object
  // i.e. asserts array contains n objects
  containsEnvVars(haystack, needles):: (
    // create an array, 1 element per needle
    local results = [
      // check the needle is valid (it contains the two required fields)
      if (std.objectHas(needle, 'name') && std.objectHas(needle, 'value')) then
        // needle is valid

        // create a sub array to contain result of every haystack item for the current needle
        local h = [
          // if the haystack name and value both match the needle, add 'true' to the sub array
          if (needle.name == haystackItem.name) && needle.value == haystackItem.value then true
          for haystackItem in haystack
        ];
        // strip out the nulls, leaving only a true for the items where the needle and haystack match
        local prunedResults = std.prune(h);

        if std.length(prunedResults) == 0 then
           error "needle: " + needle + " was not found in the haystack"
        else
          true
      else
        // needle is not value, raise an error
        error 'needle ' + needle + ': doesnt contain the fields "name" and "value"'
      // body of the loop is the above logic
      for needle in needles
    ];

    // remove the empty sub arrays, these are where a needle was not matched to a haystackItem
    // therefore indicate a needle wasn't found
    local prunedResults = std.prune(results);

    //the number of results (bool true) remaining should equal the number of needles, indicating they were all found
    if std.count(prunedResults, true) < std.length(needles) then
      false
    else
      true
  )
}
```

An example usage. Here we simply want to test that an additional environment variable is present when a config flag is turned on.

```js
local test = import 'lib/jsonnetunit/jsonnetunit/test.libsonnet';
local utils = import 'lib/testhelpers.libsonnet';
local deploy = import 'templates/deployment-app.libsonnet';

local results = test.suite({
// Make sure the envVars contain ADDITIONAL_ENV: ADDITIONAL_VALUE without having to compare the entire map
testAdditionalEnvVars: {
  // In a real case, this vars block would be from the unit under test, in our case a rendered k8s manifest.
  local exampleVars: [
    { "name": "DB_HOST",        "value": "thedb.domain.com"          },
    { "name": "DB_PORT",        "value": 3306                        },
    { "name": "REDIS_HOST",     "value": "redis.anotherdomain.local" },
    { "name": "REDIS_PORT",     "value": 6379                        },
    { "name": "ADDITIONAL_ENV", "value": "ADDITIONAL_VALUE"          }
]

  actual: utils.containsEnvVars(exampleVars, [{ name: 'ADDITIONAL_ENV', value: 'ADDITIONAL_VALUE' }]),
  expect: true,
},
```

And of course, this test helper also needed testing itself.

```js
local test = import 'lib/jsonnetunit/jsonnetunit/test.libsonnet';
local utils = import 'lib/testhelpers.libsonnet';

local mockEnvs = [{ name: 'n1', value: 'v1' }, { name: 'n2', value: 'v2' }, { name: 'n3', value: 'v3' }];

test.suite({
  testContainsSingleEnvVar: {
    expect: utils.containsEnvVars(mockEnvs, [{ name: 'n1', value: 'v1' }]),
    actual: true,
  },

  testContainsTwoEnvVars: {
    actual: utils.containsEnvVars(mockEnvs, [{ name: 'n2', value: 'v2' }, { name: 'n3', value: 'v3' }]),
    expect: true,
  },

  testDoesntContainsTwoEnvVars: {
    actual: utils.containsEnvVars(mockEnvs, [{ name: 'n5', value: 'v5' }, { name: 'n6', value: 'v6' }]),
    expect: false,
  },

  ... etc
})

```

## A Bonus Feature

If we WERE only looking for `key`:`value` pairs in a map, the following would suffice

```js
// containsObject will assert that the k:v's provided exist in the haystack object
// i.e assert map contains n entries
containsObject(haystack, needles):: (

// create a local array containing the element 'true' if the key and value of each needle exists in haystack
local results = [
    if std.objectHas(haystack, need) && (needles[need] == haystack[need]) then
      true
    else
      false
    for need in std.objectFields(needles)
];

// results will contain 'true' or 'null' if a needle was matched. strip out the 'nulls'
local prunedResults = std.prune(results);

// if the number of 'true's left is the same length as the needles, then assume everything was found and return true
if std.count(prunedResults, true) < std.length(needles) then
    false
else
    true
),
```

and in use...
```js
testCantFindSingleNeedleInALargerObject: {
    local haystack = { k1: 'v1', k2: 'v2', k3: 'v3' },

    local needle = { k4: 'v4' },

    actual: utils.containsObjects(haystack, needle),
    expect: false,
  },
```