My first open source project - super simple cli testing tool
Tags: testing, go, opensource, shell
Published: 2019-04-01 08:51:00

---
title: My first open source project - super simple cli testing tool
published: true
description: Test your cli app with a simple yaml based test suite
tags: testing, go, opensource, shell
---

## Introduction

In this article I will cover my first open source project `commander`. It tests cli apps based on tests defined in a simple `yaml` file.

Check it out: https://github.com/SimonBaeumer/commander

## The problem?

For another project I needed a tool which had the following abilities:

 - Use the same tool for different operating systems (mainly osx, linux, windows)
 - Simple and easy writing of tests
 - Simple installation (no heavy lib or language installation)
 - Language independent
 - pls not another DSL

Especially the heavy installation and language independence were important for me because I wanted to test an app inside different docker containers.

Sadly there wasn't a single tool for it. `bats` just works for bash, go's `icmd` and python's `expect` require you to install a language and be familiar with it.

## Why using `commander`?


 - Easy installation
 - Generate tests automatically
 - Fast execution
 - Platform independent
 - Just `yaml`

## Installation

Download the binary and give it execution permissions. If you want to call it directly from your terminal add it your `path` environment variable.

### Linux & osx
```bash
# linux amd64
curl -L https://github.com/SimonBaeumer/commander/releases/download/v1.0.1/commander-linux-amd64 -o commander
chmod +x commander 

# osx amd64
curl -L https://github.com/SimonBaeumer/commander/releases/download/v1.0.1/commander-darwin-amd64 -o commander
chmod +x commander 
```

### windows amd64

```powershell
curl -L https://github.com/SimonBaeumer/commander/releases/download/v1.0.1/commander-windows-amd64.exe -o commander.exe
```

... or download it manually and add it to your `path`.

## Quick start!

```bash
# Let commander create your test!
$ .commander add echo hello dev.to
written to /current/working/dir/commander.yaml

# ... written to the default commander.yaml
$ cat commander.yaml
tests:
  echo hello dev.to:
    exit-code: 0
    stdout: hello dev.to

# Executes the generated commander config
$ commander test
Starting test file commander.yaml...

✓ echo hello dev.to

Duration: 0.003s
Count: 1, Failed: 0
```

## A full `commander.yaml` example

As you can see the test suite is easy to understand and write. 

```yaml
tests:
  echo hello world:
    stdout:
      lines:
        1: hello world # assert a specifc line
      contains:
        - hello world # this is the default
      exactly: hello # Match stdout exactly
    # The same can be done for stderr
    exit-code: 0
    config:
      timeout: 5ms # set timeout for test
      dir: /tmp # set working dir
      env:
        SOME_ENV: value

  it should have a title: # You can also define a title to be more clear...
    command: echo title # ... and define your command under test here
    exit-code: 0
```

## Define default test configs

Further you can define a default configuration for all tests. This can be overwritten by the local test configs.

```yaml
config: # Define the config globally
  timeout: 5ms
  dir: /tmp
  env:
    KEY: VALUE

tests:
  print env and dir:
    command: "echo $KEY $(pwd)"
    stdout: VALUE /tmp
    exit-code: 0
```

## Debbuging tests

Sometimes you need to debug your commands. This can be done with `--verbose` flag.

```bash
# Generate the test
$ commander add echo hello dev.to

# Execute it with logging output
$ commander test --verbose
Starting test file commander.yaml...

2019/03/28 16:56:48 title: 'echo hello dev.to'  ExitCode:  0
2019/03/28 16:56:48 title: 'echo hello dev.to'  Stdout:  hello dev.to
2019/03/28 16:56:48 title: 'echo hello dev.to'  Stderr:  
✓ echo hello dev.to

Duration: 0.002s
Count: 1, Failed: 0
```

## Executing

```bash
# Execute another file instead of the default
$ commander test another.yaml

# Filter tests
$ commander test commander.yaml "echo hello"
```

## Generating tests

```bash
# Add a test to the default commander.yaml
$ ./commander add echo hello
written to /tmp/commander.yaml

# Write to a given file
$ ./commander add --file=test.yaml echo hello
written to test.yaml

# Write to stdout and file
$ ./commander add --stdout echo hello
tests:
  echo hello:
    exit-code: 0
    stdout: hello

written to /tmp/commander.yaml

# Only to stdout
$ ./commander add --stdout --no-file echo hello
tests:
  echo hello:
    exit-code: 0
    stdout: hello
```

## Conclusion

With `commander` you can easy generate your test cases and execute them on different platforms without learning a new language or installing annoying libs and tools.

Contributions are always warmly welcomed!

I hope some of you can use it in their projects :)

Link: https://github.com/SimonBaeumer/commander

Notice: Please give me feedback for my first blog post!