Using PyYaml to support YAML and JSON configuration files in your CLI tools
Tags: python, wsl, windows, automation
Published: 2021-02-24 09:49:22

---
title: Using PyYaml to support YAML and JSON configuration files in your CLI tools
published: true
description: 
tags: python, wsl, windows, automation
cover_image: https://cdn.pixabay.com/photo/2020/05/12/18/11/language-5164285_960_720.png
---

In 2019, I wrote the [wslrun](https://github.com/jmarhee/wslrun) tool as a proof of concept for a testing framework (i.e. for git hooks, local testing, etc.) CLI that used the Windows Subsystem for Linux (WSL). I designed it to be guided by a JSON file, like so:

```json
{
    "name": "PS Test",
    "description": "Tests building in multiple environments",
    "pullPolicy": "Never",
    "ciMode": "False",
    "stages": [
        {
            "image": "AlpineDub",
            "name": "Build on distro with a passing job",
            "steps": [
                "sudo apk --update add musl-dev",
                "env CGO_ENABLED=1 GOOS=linux GOARCH=amd64 go build -v -x -o package-amd64-alpine"
            ]
        },
        {
            "image": "UbuntuDub",
            "name": "Build on distro I know will fail",
            "steps": [
                "GOOS=linux GOARCH=amd64 go build -v -x -o package-amd64-ubuntu"
            ]
        },
        {
            "image": "AlpineLub",
            "name": "Build on image I know doesn't exist",
            "steps": [
                "env GOOS=linux GOARCH=amd64 go build -v -x -o package-amd64-alpine"
            ]
        }
    ]
}
```

Where `stages` is just a nested object containing the WSL distro (`image`) to execute the respective `steps` (the commands to be run in the WSL environment). 

However, I don't always want to write JSON by hand, and while it's easily generated by automation, which makes it a great format for handing this information over to another automated system (like a CI workflow), I wanted to support YAML as well, which was substantially easier for me to compose by hand. In that case, the above JSON became:

```yaml
---
name: PS Test
description: Tests building in multiple environments
pullPolicy: Never
ciMode: 'False'
stages:
- image: AlpineDub
  name: Build on distro with a passing job
  steps:
  - sudo apk --update add musl-dev
  - env CGO_ENABLED=1 GOOS=linux GOARCH=amd64 go build -v -x -o package-amd64-alpine
- image: UbuntuDub
  name: Build on distro I know will fail
  steps:
  - GOOS=linux GOARCH=amd64 go build -v -x -o package-amd64-ubuntu
- image: AlpineLub
  name: Build on image I know doesn't exist
  steps:
  - GOOS=linux GOARCH=amd64 go build -v -x -o package-amd64-alpine
```
Having my CLI tool handle this was exceptionally trivial to accomplish: The Python `pyyaml` package has a method, `safe_load` which can be used to, either, read JSON, or, for our purposes, render inputted YAML to be returned as JSON. 

In this case, the `wslrun` tool takes an argument, like:

```powershell
wslrun --pipeline config.json
```

or 

```powershell
wslrun --pipeline config.yaml
```
and in both cases, this file is read, and passed to a function like this:

```python
def manifestRead(manifestPath):
    try:
        with open(manifestPath, "rt") as manifest_json:
            manifest = yaml.safe_load(manifest_json)
        return manifest
    except OSError as e:
        return e
```

If the `manifestPath` can be read into a string (`manifest_json`) and, itself, read as valid YAML (or is already valid JSON), the `safe_load` function will return this object as `manifest`, which the rest of the CLI flow will operate on as described above.

The benefit to an implementation like this is: even if you are providing a JSON file, it acts as a check on the content's validity, but also a way to provide multiple formats for users to write configurations in, while remaining a comparatively low-effort yet effective way to accomplish this.