How to Test a GitHub Action with GitHub Actions
Tags: github, devops, tutorial, python
Published: 2022-12-07 14:15:18

[I maintain several GitHub Actions](https://actions.cicirello.org/), all of which are implemented in Python as [container actions](https://docs.github.com/en/actions/creating-actions/creating-a-docker-container-action). This post explains how to test a GitHub Action using a GitHub Actions workflow, including using the workflow as a required check on Pull Requests. Although some of this post is specific to testing an action that is implemented in Python, much of the post is more generally applicable to testing actions regardless of implementation language.

__Table of Contents:__ The rest of this post is organized as follows:
* [Steps to Test A GitHub Action](#steps-to-test-a-github-action)
  * [Preliminaries](#preliminaries)
  * [Checkout](#checkout)
  * [Run Unit Tests](#run-unit-tests): This step is Python specific, but can be easily modified for your chosen language.
  * [Build the Docker Container](#build-the-docker-container)
  * [Integration Test](#integration-test)
  * [Validate the Integration Test Results](#validate-the-integration-test-results)
* [Complete Example Workflow](#complete-example-workflow)
* [Real Examples](#real-examples)
  * [Testing the generate-sitemap Action](#testing-the-generatesitemap-action)
  * [Testing the jacoco-badge-generator Action](#testing-the-jacocobadgegenerator-action)
  * [Testing the javadoc-cleanup Action](#testing-the-javadoccleanup-action)
* [Where You Can Find Me](#where-you-can-find-me)
  


## Steps to Test A GitHub Action

In this section, I'll walk you through a GitHub Actions workflow for testing a GitHub Action.

### Preliminaries

First, within the `.github/workflows` directory of the repository, create a YAML file for the workflow. I usually name this workflow `build.yml`. Start by giving the workflow a name, and configuring the events that will trigger it to run. In this example, the workflow will run on both pushes and pull requests for the branch `main`. The snippet below also sets up a job that will run on an Ubuntu runner. 

```YAML
name: build

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:

  build:

    runs-on: ubuntu-latest
```

### Checkout

Next, checkout with the `actions/checkout` action.

```YAML
    steps:
    - uses: actions/checkout@v3
```

### Run Unit Tests

Our first set of tests are our unit tests. Although I implement actions as container actions, I conduct the unit testing outside of the Docker container. Since this is a Python example, we need two steps here. First, the `actions/setup-python` action is used to set up Python. In this example, Python 3.10 is used. The second step below uses the Python module `unittest` to execute our unit tests. This example assumes that there are unit tests in `tests/tests.py`. If any of the unit tests fail, then the failure will cause a non-zero exit code from Python, which will in turn fail the workflow. In this way, we can use this as a required PR check to ensure that all of our unit tests pass before merging a PR.

```YAML
    - name: Setup Python
      uses: actions/setup-python@v4
      with:
        python-version: '3.10'

    - name: Run Python unit tests
      run: python3 -u -m unittest tests/tests.py
```

### Build the Docker Container

Next, since this is a container action, we want to ensure that the Docker container successfully builds.

```YAML
    - name: Verify that the Docker image for the action builds
      run: docker build . --file Dockerfile
```

### Integration Test

Earlier, we ran unit tests external from any container. Now, we want to test the full integration of our action. For this, we want to use the action itself. Ordinarily, you run an action with `uses: username/repository@version`. However, we don't want to do that here. If we want to use this workflow as a PR check, we want to make sure that we run a version of the action that incorporates any changes from the PR we are checking. We can do this by specifying `uses: ./` which will direct the GitHub Actions framework to look for an action at the root of what we earlier checked out with the `actions/checkout` step.

In this example, I'm running the action we are testing twice with different inputs. You can have as many of these steps as needed. Keep in mind that each test here will be slower than each of your unit tests. After all, each of these is fully running the action, rather than simply testing one small unit; and additionally, the GitHub Actions framework must also build your Docker container.

```YAML
    - name: Integration test 1
      uses: ./
      with:
        input-one: something
        input-two: true

    - name: Integration test 2
      uses: ./
      with:
        input-one: something else
        input-two: false
```

### Validate the Integration Test Results

We now need a way to detect if the results of the above integration tests are correct. The various actions that I maintain produce files (e.g., [jacoco-badge-generator](https://github.com/cicirello/jacoco-badge-generator) produces coverage badges, and [generate-sitemap](https://github.com/cicirello/generate-sitemap) produces an XML sitemap) or edits existing files (e.g., [javadoc-cleanup](https://github.com/cicirello/javadoc-cleanup) inserts canonical links and a few other things into the head of javadoc pages). In cases like these, I use Python's `unittest` module to validate the results. In this case, I define unit test cases in `tests/integration.py` that verify that the files produced by the action are correct. If any of those tests fail, then Python will exit with a non-zero exit code which will cause the workflow to fail.

```YAML
    - name: Verify integration test results
      run: python3 -u -m unittest tests/integration.py
```

## Complete Example Workflow

Here's the complete example.

```YAML
name: build

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:

  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v3

    - name: Setup Python
      uses: actions/setup-python@v4
      with:
        python-version: '3.10'

    - name: Run Python unit tests
      run: python3 -u -m unittest tests/tests.py

    - name: Verify that the Docker image for the action builds
      run: docker build . --file Dockerfile

    - name: Integration test 1
      uses: ./
      with:
        input-one: something
        input-two: true

    - name: Integration test 2
      uses: ./
      with:
        input-one: something else
        input-two: false

    - name: Verify integration test results
      run: python3 -u -m unittest tests/integration.py
```

## Real Examples

I use this approach in several actions that I maintain. I maintain a website about these.

{% embed https://actions.cicirello.org/ %}

If you'd like to see a couple real examples of my approach to testing a GitHub Action within GitHub Actions, then take a look at the following three examples.

### Testing the generate-sitemap Action

This first real example is [generate-sitemap](https://github.com/cicirello/generate-sitemap). Files relevant to the example are as follows:

| What? | Direct Link to File |
| --- | --- |
| Workflow | [build.yml](https://github.com/cicirello/generate-sitemap/blob/master/.github/workflows/build.yml) |
| Directory of test data and code | [tests](https://github.com/cicirello/generate-sitemap/tree/master/tests) |
| Unit tests | [tests/tests.py](https://github.com/cicirello/generate-sitemap/blob/master/tests/tests.py) |
| Code to validate integration tests | [tests/integration.py](https://github.com/cicirello/generate-sitemap/blob/master/tests/integration.py) |

{% github cicirello/generate-sitemap %}

### Testing the jacoco-badge-generator Action

This next example is the [jacoco-badge-generator](https://github.com/cicirello/jacoco-badge-generator). Files relevant to the example are as follows:

| What? | Direct Link to File |
| --- | --- |
| Workflow | [build.yml](https://github.com/cicirello/jacoco-badge-generator/blob/main/.github/workflows/build.yml) |
| Directory of test data and code | [tests](https://github.com/cicirello/jacoco-badge-generator/tree/main/tests) |
| Unit tests | [tests/tests.py](https://github.com/cicirello/jacoco-badge-generator/blob/main/tests/tests.py) |
| Code to validate integration tests | [tests/integration.py](https://github.com/cicirello/jacoco-badge-generator/blob/main/tests/integration.py) |

{% github cicirello/jacoco-badge-generator %}

### Testing the javadoc-cleanup Action

This last example is [javadoc-cleanup](https://github.com/cicirello/javadoc-cleanup). Files relevant to the example are as follows:

| What? | Direct Link to File |
| --- | --- |
| Workflow | [build.yml](https://github.com/cicirello/javadoc-cleanup/blob/master/.github/workflows/build.yml) |
| Directory of test data and code | [tests](https://github.com/cicirello/javadoc-cleanup/tree/master/tests) |
| Unit tests | [tests/tests.py](https://github.com/cicirello/javadoc-cleanup/blob/master/tests/tests.py) |
| Code to validate integration tests | [tests/integration.py](https://github.com/cicirello/javadoc-cleanup/blob/master/tests/integration.py) |

{% github cicirello/javadoc-cleanup %}


## Where You Can Find Me

Follow me here on DEV:

{% user cicirello %}

Follow me on GitHub:

{% github cicirello/cicirello %}

Or visit my website:

{% embed https://www.cicirello.org/ %}