Bonus Tip: How to Use GitHub Actions to Test a GitHub Action Whose Output Must be Visually Inspected
Tags: github, devops, tutorial, programming
Published: 2022-12-14 14:19:15

Last week, I posted a tutorial on [How to Test a GitHub Action with GitHub Actions](https://dev.to/cicirello/how-to-test-a-github-action-with-github-actions-2hag). The audience for that post, as well as this one, are maintainers of GitHub Actions. In that post, I explained my automated approach to testing GitHub Actions, including unit testing and integration testing, all within GitHub Actions itself. I assumed in that post that the Action you are testing is implemented in Python, although most of the post is applicable more generally, or can easily be adapted to other languages.

This week's post is a bonus tip related to testing a GitHub Action within GitHub Actions. In last week's post, I assumed that it was possible to fully automate the validation of the results of the Action we are testing. In particular, I had a step that used Python's `unittest` module to run a set of tests that verified correctness of any files produced or modified by the action. __But what if the action produces something that cannot be verified automatically? For example, what if it produces an image or something else that must be visually inspected for correctness?__

__Table of Contents:__ The rest of this post is organized as follows:
* [Workflow Artifacts to the Rescue](#workflow-artifacts-to-the-rescue)
* [Real Example](#real-example) from the [user-statistician](https://github.com/cicirello/user-statistician) GitHub Action
* [Complete Example Workflow](#complete-example-workflow): Last week's full workflow example, but with the new step added at the end
* [Where You Can Find Me](#where-you-can-find-me)


## Workflow Artifacts to the Rescue

I'm assuming that you have unit tests and already have of a workflow that runs your unit tests, such as something like [my example from last week](https://dev.to/cicirello/how-to-test-a-github-action-with-github-actions-2hag). Now let's consider a case where my previous approach to integration testing isn't feasible, such as if the output of the action itself is an image or something else that must be visually inspected. We can replace the integration testing steps of last week's workflow with the following step (or add this as an additional step if you can partially automate validating your integration test results). In this example step, I'm assuming your action produces an `svg` file that you want to inspect, but this can work with anything. 

```YAML
    - name: Upload files produced by action as a workflow artifact for inspection if necessary
      uses: actions/upload-artifact@v3
      with:
        name: name-for-the-artifact
        path: path/to/file/filename.svg
```

Or perhaps your action produces an entire directory of files, then you can use something like:

```YAML
    - name: Upload files produced by action as a workflow artifact for inspection if necessary
      uses: actions/upload-artifact@v3
      with:
        name: name-for-the-artifact
        path: path/to/directory
```

In both of the above cases, I've used the `actions/upload-artifact` action to upload either a file or an entire directory of files as a workflow artifact. In both cases, that action will zip up what the `path` input points to into a zip file with name based on the `name` input, such as `name-for-the-artifact.zip` in this example. It is then attached to the run of the workflow that produced it. 

In this way, you can download from the `Actions` tab of your repository, and inspect the results. Just go to the `Actions` tab, and find the relevant workflow run, such as from a PR or push.

## Real Example

I use this approach in the [user-statistician](https://github.com/cicirello/user-statistician), which generates an SVG with a detailed summary of your GitHub activity, such as contributions, language statistics, etc. I have unit tests implemented in Python with the `unittest` module. But, the SVG as a whole can only really be validated visually, such as to inspect that content fits appropriately, image dimensions are correct based on content, etc.

The last step of my workflow that runs the tests for that project is the following:

```YAML
    - name: Upload generated SVG as a workflow artifact for inspection if necessary
      uses: actions/upload-artifact@v3
      with:
        name: generated-image
        path: images/userstats.svg
```

After the workflow runs, such as while reviewing a PR, I can navigate to the `Actions` tab of the repository, find the workflow run, and I'll see the following (screenshot of an actual run).


![Example Workflow Artifacts](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ipdtpzvtgoxcxc8dvqoj.PNG)

Be aware that if you are using this approach in a private repository that workflow artifacts count against your storage limit. In public repositories, they do not count against your quota, but they are removed after some time period. The [default is 90 days](https://docs.github.com/en/actions/using-workflows/storing-workflow-data-as-artifacts). 

The complete workflow for this project is found at: [build.yml](https://github.com/cicirello/user-statistician/blob/main/.github/workflows/build.yml). The repository itself is:

{% github cicirello/user-statistician %}


## Complete Example Workflow

Here's last week's full example workflow for automated testing, but with the example artifact upload step added at the end.

```YAML
name: build

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:

  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v3

    - name: Setup Python
      uses: actions/setup-python@v4
      with:
        python-version: '3.10'

    - name: Run Python unit tests
      run: python3 -u -m unittest tests/tests.py

    - name: Verify that the Docker image for the action builds
      run: docker build . --file Dockerfile

    - name: Integration test 1
      uses: ./
      with:
        input-one: something
        input-two: true

    - name: Integration test 2
      uses: ./
      with:
        input-one: something else
        input-two: false

    - name: Verify integration test results
      run: python3 -u -m unittest tests/integration.py

    - name: Upload files produced by action as a workflow artifact for inspection if necessary
      uses: actions/upload-artifact@v3
      with:
        name: name-for-the-artifact
        path: path/to/file/filename.svg
```


## Where You Can Find Me

Follow me here on DEV:

{% user cicirello %}

Follow me on GitHub:

{% github cicirello/cicirello %}

Or visit my website:

{% embed https://www.cicirello.org/ %}