CMake on SMT32 | Episode 7: unit tests
Tags: embedded, cpp, microcontrollers, cmake
Published: 2024-02-15 12:00:00


It's been almost four years, and it's about time to revive this series on CMake and STM32 (or MCUs in general, STM32 just serve as a tangible example).

Funny enough, I'm not working with microcontrollers anymore.... This is a very recent change in my professional career, even think I will probably work again with MCUs sooner or later. However, I still use with CMake to build C++ projects! The real reason to resume this series is my desire to write an article Docker for builds. In 2020, I had never used Docker at all. Today, I use it very often, and I'm convinced it could have solved some build and environment issues back then.

But hey, you might be wondering why I'm talking about Docker. Isn't this article supposed to be about unit tests? Yep, it is. I originally planned to cover unit tests back in 2020, but things didn't pan out. It seems logical to write about this topic first. It will give us even more reasons to use Docker (subscribe so you don't miss the next episode)!

## Why unit tests?

I don't want to get into a lengthy debate about pros & cons about unit tests. Sure, they can be misused (so bad), but I believe they are a very powerful tool to improve software quality. There are tons of great articles on this topic out there if you want to if you want to delve into this topic. If you speak (or at least understand) French, [I was invited on a podcast to talk about unit tests](https://www.younup.fr/blog/podcast-tests-unitaires-vraiment-utiles), and I explained why I believe they are a crucial part of software development.

After 13+ years as a software developer, mostly working projects with MCUs (and especially STM32), I must say that unit tests aren't very common in the embedded world. Embedded projects sometimes feel more like electronics-with-some-code projects code than modern software projects. Hardware can be a real constraint too, so we tend to think that unit tests aren't for us.

In this article, I want to show that it doesn't have to be necessarily that way. With CMake, we can easily build unit tests for our embedded projects.

## Running tests on board

Since we're dealing with electronic devices, our first thought is naturally to run tests directly on the board.

Running unit tests on board is great, because the same compiler and processor are used for both the tests and the real application. The tests are very close to the real usage of the code.

To do this, we will need to create an additional firmware dedicated solely to running tests and generating a test report. This means we have to add another `add_executable()` in your `CMakeLists.txt`. Similar to the real application's target, we have call `target_compile_definitions()`, `target_include_directories()`, `target_link_options()`, and so on, to generate a suitable firmware for the board. This process is not different from what we covered in the previous episodes.

However, running tests on board presents several challenges:

- A test framework that compiles for the target must be found.
- Tests into may need to be split into several executables so that each one fits within the flash memory of the MCU.
- The standard output of your board must be captured to get the test report.
- Running tests takes time, simply because your have to program the board.
- Integration in CI pipeline may be problematic because it may not be feasible to connect a board to the server.

While these challenges are not insurmountable, the following section will explore the advantages of running tests on your computer, eliminating these constraints and offering additional benefits.

If you want to learn more about embedded tests, the great _MCU on Eclipse_ blog has recently published [an article on this topic](https://mcuoneclipse.com/2023/12/18/modern-on-target-embedded-system-testing-with-cmake-and-ctest/).

## Running tests on your computer

Instead of running the tests directly on board, an alternative approach is to compile them using a compiler for PC and execute them on our computer. This eliminates the constraints mentioned earlier.

You may argue that we are using different compiler and  means it won't accurately reflect of the real usage of the code. And in some ways, you're absolutely right. Indeed, even something as "basic" as the size of an integer could differ between `arm-none-eabi-gcc` and regular `gcc`.

But that's actually the silver lining! Compiling our project with another compiler may raise new warnings, highlighting non-portable constructs. Running our code on a different CPU may reveal undefined behaviors in our code. Finally, being able to compile a significant part of the project for another target demonstrates that the abstraction from the hardware layers are well-designed. Naturally, this also implies that we won't be able to test the parts that are tightly coupled to hardware features.

In the remainder of this article, I will demonstrate how to use CMake to build an embedded firmware and PC-based tests within the same `CMakeLists.txt`.

## Project structure

The structure of the project is as follows:

![tree of the project](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rywlj843xnch729ucggh.png)

Most elements are simply the result of the previous episodes. I have just added a directory (`tests`) and two files (`compute.hpp` and `test_compute.cpp`), because... we need something to test!

## A simple test case

The purpose here is to demonstrate some CMake features, don't expect amazing code.

First, we need something to test. For instance a simple [quadratic function](https://en.wikipedia.org/wiki/Quadratic_function) implementation. This is a good candidate for PC-based tests, because it doesn't have any dependencies to the BSP. The code goes into `compute.hpp`:

```cpp
namespace compute {

template <typename T>
T quadratic(T a, T b, T c, T x) {
    return a * x * x + b * x + c;
}

}
```

The tests are written in a separate file, `test_compute.cpp`. For this example, I have decided a use [Catch2](https://github.com/catchorg/Catch2), a great unit testing framework for C++ that I have been using for the last 5 years.

```cpp
#include <catch2/catch_test_macros.hpp>

#include "compute.hpp"

TEST_CASE("Compute quadratic function") {
    int a = 2;
    int b = 1;
    int c = 10;

    CHECK(compute::quadratic(a, b, c, 0) == 10);
    CHECK(compute::quadratic(a, b, c, 1) == 13);
    CHECK(compute::quadratic(a, b, c, 2) == 20);
    CHECK(compute::quadratic(a, b, c, -3) == 25);
}
```

True story: I found that my function had a bug when I wrote these tests. I had mistakenly written `a * a * x` instead of `a * x * x` in the implementation of `compute::quadratic()`.

## Create the executable for the tests

Similar to adding tests for on-board execution, we have to add another `add_executable()` in our `CMakeLists.txt`. However, things are sightly different: we need to handle two different compilers now. Fortunately, CMake makes it straightforward to determine whether we are [cross-compiling](https://cmake.org/cmake/help/book/mastering-cmake/chapter/Cross%20Compiling%20With%20CMake.html) (to create our STM32-based firmware) or not (to create our PC-based program), thanks to [the variable `CMAKE_CROSSCOMPILING`](https://cmake.org/cmake/help/latest/variable/CMAKE_CROSSCOMPILING.html). We will see in the next section that is variable is set when we provide CMake with a toolchain file.

The structure of our `CMakeLists.txt` will look like this:

```cmake
cmake_minimum_required(VERSION 3.15.3)

project(nucleo-f413zh)

enable_language(C CXX ASM)
set(CMAKE_C_STANDARD 99)
set(CMAKE_C_STANDARD_REQUIRED ON)
set(CMAKE_C_EXTENSIONS OFF)

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)

if(CMAKE_CROSSCOMPILING)
        message(STATUS "Cross compiling for board...")

        # Here goes the code from the previous episodes

else()
        message(STATUS "Compiling for PC...")

        # Here goes the code to build the unit tests
        # See below :)
endif()
```

Catch2 is available on [GitHub](https://github.com/catchorg/Catch2). You can get a copy of its source code and integrate it into your project source tree. You can also use [`FetchContent` module](https://cmake.org/cmake/help/latest/module/FetchContent.html) to get it from GitHub directly during the build process, as suggested in Catch2's [official documentation](https://github.com/catchorg/Catch2/blob/devel/docs/cmake-integration.md). We will use this technique here.

Let's complete the `else()` clause from the previous snippet to create our executable:

```cmake
message(STATUS "Compiling for PC...")

# Get Catch2
include(FetchContent)

FetchContent_Declare(
        Catch2
        GIT_REPOSITORY https://github.com/catchorg/Catch2.git
        GIT_TAG v3.5.2
)

FetchContent_MakeAvailable(Catch2)

# Create executable
add_executable(tests
        tests/test_compute.cpp)

target_include_directories(tests PRIVATE
        source)

target_link_libraries(tests PRIVATE
        Catch2::Catch2WithMain)
```

## Build and run the tests

As mentioned before, passing a toolchain file to CMake from the command-line automatically set `CMAKE_CROSSCOMPILING`. We will hence have to generate two "projects" from a single `CMakeLists.txt`.

On one side, we generate a project for the `arm-none-eabi-gcc` toolchain, just like in the first episode of this series. From the `nucleo` directory:

```bash
$ cmake -B build-embedded -DCMAKE_BUILD_TYPE=Debug -DCMAKE_TOOLCHAIN_FILE=arm-none-eabi-gcc.cmake
-- The C compiler identification is GNU 10.3.1
-- The CXX compiler identification is GNU 10.3.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/arm-none-eabi-gcc - skipped
...
-- Cross compiling for board...
...

$ cmake --build build-embedded/
Scanning dependencies of target nucleo-f413zh.out
[  3%] Building C object CMakeFiles/nucleo-f413zh.out.dir/BSP/Drivers/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_tim.c.obj
[  7%] Building C object CMakeFiles/nucleo-f413zh.out.dir/BSP/Drivers/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_tim_ex.c.obj
[ 11%] Building C object CMakeFiles/nucleo-f413zh.out.dir/BSP/Drivers/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_uart.c.obj
...
[100%] Linking CXX executable nucleo-f413zh.out
   text    data     bss     dec     hex filename
   7868      20    2892   10780    2a1c nucleo-f413zh.out
[100%] Built target nucleo-f413zh.out
```

On the other side, we generate another project to build with our default system compiler:

```bash
$ cmake -B build-tests -DCMAKE_BUILD_TYPE=Release                                             
-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
...
-- Compiling for PC...
...

$ cmake --build build-tests/
[  0%] Building CXX object _deps/catch2-build/src/CMakeFiles/Catch2.dir/catch2/benchmark/catch_chronometer.cpp.o
[  1%] Building CXX object _deps/catch2-build/src/CMakeFiles/Catch2.dir/catch2/benchmark/detail/catch_analyse.cpp.o
[  2%] Building CXX object _deps/catch2-build/src/CMakeFiles/Catch2.dir/catch2/benchmark/detail/catch_benchmark_function.cpp.o
[  3%] Building CXX object _deps/catch2-build/src/CMakeFiles/Catch2.dir/catch2/benchmark/detail/catch_run_for_at_least.cpp.o
...
 96%] Built target Catch2
[ 97%] Building CXX object _deps/catch2-build/src/CMakeFiles/Catch2WithMain.dir/catch2/internal/catch_main.cpp.o
[ 98%] Linking CXX static library libCatch2Main.a
[ 98%] Built target Catch2WithMain
[ 99%] Building CXX object CMakeFiles/tests.dir/tests/test_compute.cpp.o
[100%] Linking CXX executable tests
[100%] Built target tests
```

Because our executable depends on Catch2, CMake builds the library before.

Our tests are now ready to run:

```cpp
$ ls build-tests/
CMakeCache.txt  CMakeFiles  Makefile  _deps  cmake_install.cmake  tests

$ ./build-tests/tests 
Randomness seeded to: 179865564
===============================================================================
All tests passed (4 assertions in 1 test case)
```

Hurray! The tests passed!

## Conclusion

In this episode, we have discussed unit tests for embedded projects.

We have seen that running tests on board is great but there are some difficulties to overcome. Running tests on PC don't have these difficulties and may even offer additional benefits. If you have the chance to do both, do it!

We've seen how to manage both an STM32-based firmware and a PC-based program within the same `CMakeLists.txt`. We generate two separate build directories, one for each compiler and hence one for each executable. 

I hope that this episode convinced you to add unit tests to your STM32 (or any other MCU) projects. Happy tests!