CMake Coverage Example: with GitHub Actions and codecov.io
Tags: cpp, gamedev, testing
Published: 2024-01-29 18:02:02

---
title: "CMake Coverage Example: with GitHub Actions and codecov.io"
published: "true"
description: "CMake Coverage Example ☑️ adding C++ code coverage tests using CMake with GitHub Workflow integration for pushing to codecov.io 📈."
tags: "cplusplus, gamedev, testing"
canonical_url: "https://rodneylab.com/cmake-coverage-example/"
cover_image: "https://dev-to-uploads.s3.amazonaws.com/uploads/articles/h4r79anwxq3r325fb6e6.png"

---
## 🧑🏽‍💻 CMake Coverage Example

In this post, we see a CMake coverage example using **GitHub Actions**.  We build the project and run unit tests on each git push, before uploading the latest code coverage results to **codecov.io**.  This is all working on a **CMake** C++ project.  Using gcov, genhtml and lcov locally, it is possible to generate an HTML coverage report, with graphs.  However, here we automate the process, running tests remotely.  This has added advantages. For example, you might configure your repo to **reject pull requests** that cause the test coverage to drop below a target percentage. 

## 🧱 What we are Building
![CMake Coverage Example: Screenshot shows codecov.io console. In the main view, a graph shows around 50% coverage over January.  To the right, a donut represents this information with red and green segments.](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xe5x2sl794k9v3sgsfei.png)

We will use an arkanoid-clone project that I set up for testing.  This is a clone of the classic arcade game built using CMake and SFML.  The repo itself is based on a YouTube tutorial, and you can find a link to the original tutorial in the <a href="https://github.com/rodneylab/arkanoid-clone">arkanoid-clone repo README</a>.  You might also want to reference the repo for any extra useful details missing here — do let me know, though, if there is anything, extra, I could add here to make the article clearer.

Let&rsquo;s start by adding Catch2 tests to the repo.  We assume you have a CMake repo ready to add tests, too.  If you don&rsquo;t, there is a fantastic <a href="https://www.jetbrains.com/help/clion/unit-testing-tutorial.html#setup-framework-catch">JetBrains tutorial on adding Catch2 unit tests in CMake</a>, which you can set up quickly as a toy project.

## ☑️ Setting up CMake Unit Tests

I will add Catch2 tests in a new `Catch_tests` directory, and need to include the new directory at the bottom of the main `CMakeLists.txt` file:

```cmake
# include test folder towards the bottom of CMakeLists.txt
option(RUN_UNIT_TESTS "Run Catch2 unit tests" ON)
if(RUN_UNIT_TESTS)
    enable_testing()
    add_subdirectory(Catch_tests)
endif()
```

Then, I create `Catch_tests/CMakeLists.txt`:

```cmake
include(FetchContent)
FetchContent_Declare(
    Catch2
    GIT_REPOSITORY https://github.com/catchorg/Catch2.git
    GIT_TAG v3.5.0)
FetchContent_MakeAvailable(Catch2)
 
add_executable(Catch_tests_run BallTests.cpp)
 
target_link_libraries(
    Catch_tests_run
    PRIVATE Ball_lib
            Brick_lib
            Paddle_lib
            arkanoid_compiler_flags)
target_link_libraries(Catch_tests_run PRIVATE Catch2::Catch2WithMain)
target_include_directories(Catch_tests_run PUBLIC "${PROJECT_SOURCE_DIR}/src")
 
include(Catch)
catch_discover_tests(Catch_tests_run)
```

This fetches the Catch2 dependency from GitHub, adds a test executable and finally, sets up Catch2 to discover the tests.

As an example test, you might have something like `src/Catch_tests/BallTests.cpp`:

```cpp
#include "Ball.h"
 
#include <SFML/Graphics.hpp>
#include <catch2/catch_test_macros.hpp>
 
TEST_CASE("BallCorrectlyInitialised", "[BallTests]")
{
    const Ball ball{256, 512};
    const sf::Color ball_colour{ball.shape.getFillColor()};

    CHECK(ball_colour == sf::Color::Red);
}
```

See the tutorial, mentioned above, for more on creating Catch2 tests and setting them up in CMake.

## ⛺️ Adding Coverage Tests

To start, download the <a aria-label="C Make l cov Coverage module" href="https://github.com/cmake-modules/lcov/raw/master/coverage.cmake">CMake lcov Coverage module</a> and add it to a new `cmake` folder in the root directory of your project:

```cmake
function(add_coverage_target exclude)

    find_program(GCOV gcov)
    if (NOT GCOV)
        message(WARNING "program gcov not found")
    endif()

    find_program(LCOV lcov)
    if (NOT LCOV)
        message(WARNING "program lcov not found")
    endif()

    find_program(GENHTML genhtml)
    if (NOT GENHTML)
        message(WARNING "program genhtml not found")
    endif()

    if (LCOV AND GCOV AND GENHTML)
        set(covname cov.info)
        add_compile_options(-fprofile-arcs -ftest-coverage)
        add_link_options(--coverage)
        add_custom_target(cov DEPENDS ${covname})
        add_custom_command(
            OUTPUT  ${covname}
            COMMAND ${LCOV} -c -o ${covname} -d . -b . --gcov-tool ${GCOV}
            COMMAND ${LCOV} -r ${covname} -o ${covname} ${exclude}
            COMMAND ${LCOV} -l ${covname}
            COMMAND ${GENHTML} ${covname} -output coverage
            COMMAND ${LCOV} -l ${covname} 2>/dev/null | grep Total | sed 's/|//g' | sed 's/Total://g' | awk '{print $1}' | sed s/%//g > coverage/total
        )
        set_directory_properties(PROPERTIES
            ADDITIONAL_CLEAN_FILES ${covname}
        )
    else()
        message(WARNING "unable to add target `cov`: missing coverage tools")
    endif()

endfunction()
```

Then, update the main `CMakeLists.txt` to find and use this, adding these lines towards the top of the file:

```cmake
set(CMAKE_MODULE_PATH "${CMAKE_MODULE_PATH};${PROJECT_SOURCE_DIR}/cmake")
include(coverage)
add_coverage_target("*/Catch_tests/*")
```

The argument of the `add_coverage_target` function is passed to lcov and gcov as an exclude path.

That is all you need to run the coverage tests.  We will convert the data to XML for codecov.io later, using gcovr.

## CMake Coverage Example: GitHub Test & Coverage Action

Next we need a GitHub action.  This runs on every pull request submission.  In our case, we use it to build the project, run tests and then generate a coverage report in XML format ready for codecov.io.  Create a GitHub workflow flow in `.github/workflows/ubuntu.yml` to do this:

```yaml
name: Ubuntu CI Test
on:
  push:
    branches: [main, master, dev]
  pull_request:
    branches: [main, master, dev]
permissions:
  contents: read
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@ee0669bd1cc54295c223e0bb666b733df41de1c5 # v2.7.0
      - name: install
        run: sudo apt-get update && sudo apt-get install lcov #libxrandr-dev libxcursor-dev libudev-dev libopenal-dev libflac-dev libvorbis-dev libgl1-mesa-dev libegl1-mesa-dev freeglut3-dev # OpenGL dependencies only needed for SFML
      - name: configure
        run: |
          cmake -H. -Bbuild -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Debug -DENABLE_COVERAGE=On
      - name: building
        run: |
          cmake --build build --config Debug --target Catch_tests_run
      - name: run unit tests
        working-directory: ./build/bin
        run: |
          ./Catch_tests_run
      - name: generate coverage
        working-directory: ./build
        run: |
          make cov
      - name: Install gcovr
        run: |
          pip install -r requirements.txt  --require-hashes
      - name: Generate JSON coverage report
        working-directory: ./build
        run: |
          gcovr -r .. . --branches --cobertura > coverage.xml
      - name: Upload coverage reports to Codecov
        uses: codecov/codecov-action@eaaf4bedf32dbdc6b720b63067d99c4d77d6047d # v3
        env:
          CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
```

I use git commit hashes instead of repo tags or version numbers for added security (in line `13`, for example).  In line `15` you can install gcovr and lcov as well as any extra dependencies needed to build your project. 

Lines `16` to `21` will configure, then build the project in the remote environment, after a pull request.  Then, the code in lines `22` to `25` will run the Catch2 unit tests.  If we broke something in the pull request, and the unit tests fail, the whole pull request is rejected: exactly what we are looking for.

In lines `26`-`29`, we generate the coverage report in a text format.  I had most success with codecov.io working in the XML format.  We can convert the report, running:

```shell
gcovr -r .. . --branches --cobertura > coverage.xml
```

Which is exactly what we do in lines `33`-`36`, because our working directory will be `build/` we adjust the path passed to gcovr.  You can find <a aria-label="more details on g cov r commands in the docs" href="https://gcovr.com/en/stable/index.html">more details on gcovr commands in the docs</a>.


The final lines are where we upload the XML coverage report to codecov.io.  You will need to obtain a codecov token and add this to your GitHub repo.  To get a token, go to `https://app.codecov.io/gh/YOUR-GITHUB-ACCOUNT`, log in with GitHub and select the repo you want to add coverage for.  Then, follow instructions on adding a repository secret.  That will cover step one, displayed, and we have tackled step 2 above!

![CMake Coverage Example: Screenshot shows codecov.io console. The title reads, Let's get your repo covered.  Below, there are two steps outlines, adding a repository secret and adding codecov to your Git Hub workflow.](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/plneqiz5ulcq3u0po7v6.png)

### Python `requirements.txt`

Finally, add a `requirements.txt` file in the project root directory, which will be used in line `32`, above, to install gcovr (and the lxml dependency):

```plaintext
gcovr==6.0 \\
  --hash=sha256:2e52019fdb76c6e327f48c2a2d8555fb5e362570b79cc74c5498804d1ce54a60
lxml==5.1.0 \\
  --hash=sha256:22b7ee4c35f374e2c20337a95502057964d7e35b996b1c667b5c65c567d2252a
```

## 💯 CMake Coverage Example: Testing your Work

Next time you merge a pull request, after a short delay, you will see a coverage report on the codecov.io console, for your project.

![CMake Coverage Example: Screenshot shows codecov.io console. In the main view the source code lines are highlighted green, if they were covered by the tests, and red and yellow are used for uncovered and partially covered lines.  At the top, a summary shows the 3-Month average of 49% coverage, for the file.](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ling174yzlvknyp9f88b.png)

![CMake Coverage Example: Screenshot shows a panel on the Git Hub pull request screen.The title reads Codecov Report.  A sort explanation is followed by a plain text table, summarizing the coverage results.](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mdb1z4xivfzveo4zpiup.png)

## 🙌🏽 CMake Coverage Example: Wrapping Up

In this CMake coverage example post, we saw a complete example of adding coverage tests to a CMake repo.  We used GitHub actions to run a test coverage report, and push it to codecov.io.  More specifically, we saw:

- how you can use **run Catch2 unit test in GitHub actions**;
- the **codecov.io setup process** for working in GitHub; and
- how to **run code coverage test in CI and push the result to codecov.io**.

I hope you found this useful.  If you need additional pointers, take a <a href="https://github.com/rodneylab/arkanoid-clone">look at the `arkanoid-clone` repo mentioned at the start of the post</a>, which fully implements the coverage approach discussed here.  Do let me know, though, if anything here is not clear, or would benefit for further clarification.

## 🙏🏽 CMake Coverage Example: Feedback

If you have found this post useful, see links below for further related content on this site.  Let me know if there are any ways I can improve on it. I hope you will use the code or starter in your own projects. Be sure to share your work on X, giving me a mention, so I can see what you did. Finally, be sure to let me know ideas for other short videos you would like to see. Read on to find ways to get in touch, further below. If you have found this post useful, even though you can only afford even a tiny contribution, please <a aria-label="Support Rodney Lab via Buy me a Coffee" href="https://rodneylab.com/giving/">consider supporting me through Buy me a Coffee</a>.

Finally, feel free to share the post on your social media accounts for all your followers who will find it useful. As well as leaving a comment below, you can get in touch via <a href="https://twitter.com/messages/compose?recipient_id=1323579817258831875">@askRodney</a> on X (previously Twitter) and also <a aria-label="Contact Rodney Lab via Telegram" href="https://t.me/askRodney">askRodney on Telegram</a>. Also, see <a aria-label="Get in touch with Rodney Lab" href="https://rodneylab.com/contact/">further ways to get in touch with Rodney Lab</a>. I post regularly on <a href="https://rodneylab.com/tags/deno/">Deno</a> as well as <a href="https://rodneylab.com/tags/seo/" aria-label="See posts on search engine optimisation">Search Engine Optimization</a> among other topics. Also, <a aria-label="Subscribe to the   Rodney Lab newsletter" href="https://newsletter.rodneylab.com/issue/latest-issue">subscribe to the newsletter to keep up-to-date</a> with our latest projects.