JaCoCo coverage badges, PR coverage checks, and PR coverage comments, from GitHub Actions
Tags: actionshackathon21, github, java, githubactions
Published: 2021-11-29 20:19:05



### My Workflow

This workflow builds a Java project, runs all JUnit tests, runs [JaCoCo](https://www.jacoco.org/) to generate a test coverage report, uses the [cicirello/jacoco-badge-generator](https://github.com/cicirello/jacoco-badge-generator) GitHub Action to generate instructions coverage and branches coverage badges from the JaCoCo coverage report, uploads the JaCoCo report as a workflow artifact, and if the workflow run is associated with a PR it then comments on the PR with the computed coverage percentages.

The coverage badges are generated with the [cicirello/jacoco-badge-generator](https://github.com/cicirello/jacoco-badge-generator), which I have been developing and maintaining for a while now. Initially, it was strictly a badge generator, but its functionality has expanded over time to include optional support for PR checks that can be used to fail workflow runs if coverage is below a target threshold or if coverage has decreased. The [cicirello/jacoco-badge-generator](https://github.com/cicirello/jacoco-badge-generator) includes the following functionality:
* Generates instructions coverage and/or branches coverage badges entirely within the action, without any external services.
* Badge color is based on coverage percentage, and a reasonable set of defaults is provided, but the colors and associated coverage intervals can be customized with action inputs. 
* Option to either generate the badges directly (which is the default behavior) or to generate [Shields JSON endpoints](https://shields.io/endpoint) for users who may want to customize badge appearance beyond what is directly supported by the action. I use the default of direct generation of badges in my own workflows.
* Option to generate a simple JSON file containing the coverage and branches coverage percentages, either in addition to the badges or instead of the badges.
* Option to fail the workflow run if coverage, or branches coverage, or both are below customizable thresholds (this option is disabled by default).
* Option to fail the workflow run if coverage, or branches coverage, or both have decreased since the most recent run (this option is disabled by default).

The default behavior of the [cicirello/jacoco-badge-generator](https://github.com/cicirello/jacoco-badge-generator) assumes that the JaCoCo coverage report is in the default location with the default filename when JaCoCo is run with Maven. But, the action has inputs that can be used if either the JaCoCo report is named differently or if it is in a different location, or both, so it is not limited to use with Maven projects. The GitHub repository provides examples with how to configure it for use with Maven as well as Gradle, but it is also applicable if you run JaCoCo with any other tool. 

The workflow that I am submitting to the Actions Hackathon is from my project [Chips-n-Salsa](https://github.com/cicirello/Chips-n-Salsa), a Java library of customizable, hybridizable, iterative, parallel, stochastic, and self-adaptive local search algorithms, specifically, the [build.yml](https://github.com/cicirello/Chips-n-Salsa/blob/master/.github/workflows/build.yml) workflow from that project. A summary of the steps of the workflow submitted to the Actions Hackathon is as follows (complete workflow later in this post):
1. Checkout (using the action [actions/checkout](https://github.com/actions/checkout)).
2. Checkout the `badges` branch nested within the first checkout in a `badges` directory. The purpose of this step is multifaceted. First, the default branch of this repository has required checks, so I cannot use the `GITHUB_TOKEN` to push the generated badges to the default branch. For security reasons, I don't want to use a `PAT` to override those checks. So I instead have a `badges` branch without required checks that is dedicated solely to storing the coverage badges that are generated by a later step of the workflow. Since the `badges` branch has no required checks, the `GITHUB_TOKEN` is sufficient for pushing to it. The second reason for this second checkout is that an action used later, which generates coverage badges, has additional functionality for checking if coverage has decreased during PRs that uses the coverage as stored in the prior badge to check for a decrease.
3. Set up the JDK with the [actions/setup-java](https://github.com/actions/setup-java) GitHub Action.
4. Build the library with Maven. This step just uses the shell to run mvn directly, with `mvn -B package -Pcoverage`. This will build the library and run all JUnit tests. The command line option `-Pcoverage` enables a profile `coverage` from the project's [pom.xml](https://github.com/cicirello/Chips-n-Salsa/blob/master/pom.xml), which configures JaCoCo to run during the test phase.
5. Generate JaCoCo coverage badges with the [cicirello/jacoco-badge-generator](https://github.com/cicirello/jacoco-badge-generator) GitHub Action, which I have been developing and maintaining for a while. I'm using the `badges-directory` input to change the directory for where to store the badges, and I'm enabling generating a branches coverage badge with the `generate-branches-badge` input. The action generates an instructions coverage badge by default, but the additional branches coverage badge must be enabled. I'm also enabling (with the `generate-summary` input) the generation of a JSON file containing the coverage percentages. This is actually a new feature of the [cicirello/jacoco-badge-generator](https://github.com/cicirello/jacoco-badge-generator) action that generates a simple JSON file with the computed coverage percentages, which may be useful for consumption by other workflow steps (such as later in this workflow).
6. Logs the coverage and branches coverage percentages to the workflow output, which may be useful when inspecting logs of workflow runs.
7. Uses the [actions/upload-artifact](https://github.com/actions/upload-artifact) GitHub Action to upload the complete coverage reports generated by   JaCoCo earlier in step 4 above as a workflow artifact. This may be useful if I need to inspect the coverage details provided by JaCoCo for a workflow run.
8. If this workflow run is not associated with a PR, then commit and push the coverage badges and summary file to the `badges` branch, using shell commands. Since the workflow runs on pushes, PRs, and `workflow_dispatch` events, then this step will specifically run on either pushes or `workflow_dispatch` events. The badges must be in sync with the state of the default branch, so it only makes sense to push them on events against the default branch, which is why PRs are excluded during this step. 
9. If this run is for a PR, then comment on the PR with the coverage percentages. This step is implemented simply with shell commands, and specifically uses the GitHub CLI to comment on the PR. This step is conditional and runs only on PRs. This is actually the newest addition to this workflow, added to it within the last week.

See the [Chips-n-Salsa](https://github.com/cicirello/Chips-n-Salsa) repository's README for live examples of the generated badges. And see [this PR comment](https://github.com/cicirello/Chips-n-Salsa/pull/324#issuecomment-981897835) for an example of the PR coverage comment created by the last step of the workflow.


### Submission Category: 

Maintainer Must-Haves


### Yaml File or Link to Code

The live version of this workflow is found in the [build.yml](https://github.com/cicirello/Chips-n-Salsa/blob/master/.github/workflows/build.yml) of the [Chips-n-Salsa](https://github.com/cicirello/Chips-n-Salsa) library, which at the time of submission is identical to the workflow found below:

```YAML
name: build

on:
  push:
    branches: [ master ]
    paths: [ '**.java', '.github/workflows/build.yml' ]
  pull_request:
    branches: [ master ]
  workflow_dispatch:

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
    - name: Checkout
      uses: actions/checkout@v2

    - name: Checkout badges branch to a badges directory nested inside first checkout
      uses: actions/checkout@v2
      with:
        ref: badges
        path: badges

    - name: Set up JDK 11
      uses: actions/setup-java@v2
      with:
        distribution: 'adopt'
        java-version: '11'

    - name: Build with Maven
      run: mvn -B package -Pcoverage

    - name: Generate JaCoCo badge
      id: jacoco
      uses: cicirello/jacoco-badge-generator@v2
      with:
        badges-directory: badges
        generate-branches-badge: true
        generate-summary: true

    - name: Log coverage percentages to workflow output
      run: |
        echo "coverage = ${{ steps.jacoco.outputs.coverage }}"
        echo "branches = ${{ steps.jacoco.outputs.branches }}"

    - name: Upload JaCoCo coverage report as a workflow artifact
      uses: actions/upload-artifact@v2
      with:
        name: jacoco-report
        path: target/site/jacoco/

    - name: Commit and push the coverage badges and summary file
      if: ${{ github.event_name != 'pull_request' }}
      run: |
        cd badges
        if [[ `git status --porcelain *.svg *.json` ]]; then
          git config --global user.name 'github-actions'
          git config --global user.email '41898282+github-actions[bot]@users.noreply.github.com'
          git add *.svg *.json
          git commit -m "Autogenerated JaCoCo coverage badges" *.svg *.json
          git push
        fi

    - name: Comment on PR with coverage percentages
      if: ${{ github.event_name == 'pull_request' }}
      run: |
        REPORT=$(<badges/coverage-summary.json)
        COVERAGE=$(jq -r '.coverage' <<< "$REPORT")%
        BRANCHES=$(jq -r '.branches' <<< "$REPORT")%
        NEWLINE=$'\n'
        BODY="## JaCoCo Test Coverage Summary Statistics${NEWLINE}* __Coverage:__ ${COVERAGE}${NEWLINE}* __Branches:__ ${BRANCHES}"
        gh pr comment ${{github.event.pull_request.number}} -b "${BODY}"
      env:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
```


### Additional Resources / Info

The repository for the `cicirello/jacoco-badge-generator` GitHub Action at the heart of the workflow submitted to the GitHub Actions Hackathon, which also has additional sample workflows using the action:

{% github cicirello/jacoco-badge-generator %}

The repository of [Chips-n-Salsa](https://github.com/cicirello/Chips-n-Salsa), a Java library of customizable, hybridizable, iterative, parallel, stochastic, and self-adaptive local search algorithms, whose [build.yml](https://github.com/cicirello/Chips-n-Salsa/blob/master/.github/workflows/build.yml) workflow is the subject of this submission to the GitHub Actions Hackathon:

{% github cicirello/Chips-n-Salsa %}

More information on the [cicirello/jacoco-badge-generator](https://github.com/cicirello/jacoco-badge-generator) GitHub Action, as well as other GitHub Actions that I maintain is found at: https://actions.cicirello.org/.