Test command line outputs with Github Actions
Tags: actionshackathon21, githubactions, testing, cicd
Published: 2021-11-22 21:18:52

## Context

As an open source project maintainer on [Ritchie CLI](https://github.com/ZupIT/ritchie-cli), I recently had the necessity to tests if some command line outputs were working as expected.

### What happened

I had to test the CLI commands on various operating system (windows, linux and macos), and I couldn't find an action to help us achieving what we wanted.

The project was developed in Golang, so I could easily create a binary for each OS and execute it. But I had to use some command like `<command-line> &> output.txt` to get an output file, and then compare it with a fix `.txt` file on the repository to check if the command output was the expected one.

**Some of the issues I faced:**

☞ I couldn't test any error behaviour (because the workflow would failed) without adding a `|| true` at the end of the command above (to ignore the failure).

☞ Commands line could be different depending on the OS (DOS/Unix).

☞ It was difficult to check dynamic outputs (that would change for each execution, for example a _data_ or _version_ value that evolve all the time).

I ended up creating this action to resolve all those specific issues.

### What does this action?

This action allows the user to compare a command line output (success or error) with a file content located on the repository, or to check if it contains a specific expression.

Therefore, it can be used to check almost everything a command line can return on the screen or create:
☞ if a specific line appears in a file using a `cat` command.
☞ a whole file content,
☞ if a file is located inside a specific directory through a `ls` command after a file creation,
☞ ...

Moreover, **it's compatible with all os-supported runner**. I had an issue to test it with windows at first due to the different between DOS/Unix, but as I could use a **composite action** performing operations using **bash** (as well a [strip-ansi npm cli lib](https://www.npmjs.com/package/strip-ansi-cli)) to by-pass this problem.

#### How the action works

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/11d3bjdse19isuuoxbkv.png)

That way, I could check a command line output using 3 different ways:

☞ if the output is equal to an assert file
☞ if the output contains a specific line from an assert file
☞ if the output contains an expression

A successful display on the Actions tab on Github UI would look like this:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7nk1y7b77q0jp8oxzy8j.png) 

If the output is not the expected one, the action will also inform the difference (using a `diff` command) between both files / expressions.

## My Workflows

The action is called [Assert Command Line Output](https://github.com/GuillaumeFalourd/assert-command-line-output) and you will find below different examples using it (with success and error tests for each one).

The [action.yml file can be found here](https://github.com/GuillaumeFalourd/assert-command-line-output/blob/main/action.yml).

### `1️⃣ Assert file content`

#### Expecting command output to be EQUAL to `assert.txt` file content

```yaml
    steps:
      - uses: actions/checkout@v2.3.4
      - uses: GuillaumeFalourd/assert-command-line-output@v2
        with:
          command_line: ls -lha
          assert_file_path: path/to/assert.txt
          expected_result: PASSED
```

#### Expecting command output to be DIFFERENT than the `assert.txt` file content

```yaml
    steps:
      - uses: actions/checkout@v2.3.4
      - uses: GuillaumeFalourd/assert-command-line-output@v2
        with:
          command_line: ls -lha
          assert_file_path: path/to/assert.txt
          expected_result: FAILED
```

### `2️⃣ Assert specific file line`

#### Expecting command output line 3 to be EQUAL than the `assert.txt` file content in line 3

```yaml
    steps:
      - uses: actions/checkout@v2.3.4
      - uses: GuillaumeFalourd/assert-command-line-output@v2
        with:
          command_line: ls -lha
          assert_file_path: path/to/assert.txt
          expected_result: PASSED
          specific_line: 3
```

#### Expecting command output line 3 to be DIFFERENT than the `assert.txt` file content in line 3

```yaml
    steps:
      - uses: actions/checkout@v2.3.4
      - uses: GuillaumeFalourd/assert-command-line-output@v2
        with:
          command_line: ls -lha
          assert_file_path: path/to/assert.txt
          expected_result: FAILED
          specific_line: 3
```

### `3️⃣ Assert specific expression`

#### Expecting command output to contain specific expression

```yaml
    steps:
      - uses: actions/checkout@v2.3.4
      - uses: GuillaumeFalourd/assert-command-line-output@v2
        with:
          command_line: ls -lha
          contains: runner
          expected_result: PASSED
```

#### Expecting command output NOT to contain specific expression

```yaml
    steps:
      - uses: actions/checkout@v2.3.4
      - uses: GuillaumeFalourd/assert-command-line-output@v2
        with:
          command_line: ls -lha
          contains: error
          expected_result: FAILED
```

## Submission Category 

This action could be useful for **Maintainers** (Maintainer Must-Haves) as well as **Deployments** (DIY Deployments) to check some specific outputs from commands used inside a CI/CD pipelines on Github Actions, or any periodic checks.

### Yaml File or Link to Code

- [The action can be found on the Github Marketplace](https://github.com/marketplace/actions/assert-command-line-output)

- You can find a full implementation example for each OS on the action repository, using `cat` or `ls -lha` commands as demonstrations:

☞ [UBUNTU](https://github.com/GuillaumeFalourd/assert-command-line-output/blob/main/.github/workflows/ubuntu_test_command_output.yml)
☞ [MACOS](https://github.com/GuillaumeFalourd/assert-command-line-output/blob/main/.github/workflows/macos_test_command_output.yml)
☞ [WINDOWS](https://github.com/GuillaumeFalourd/assert-command-line-output/blob/main/.github/workflows/windows_test_command_output.yml)

### Additional Resources / Info

☞ You can contribute to the repository following the [Contributing.md file](https://github.com/GuillaumeFalourd/assert-command-line-output/blob/main/CONTRIBUTING.md)

☞ This action uses the [Apache License 2.0](https://github.com/GuillaumeFalourd/assert-command-line-output/blob/main/LICENSE)

## Conclusion

I believe everything is not perfect yet, this action evolved a lot since the first version, and I hope some of you will find it useful and will want to contribute to make it better. Thank you for reading :rocket: