How To Find Packages With CMake: The Basics
Tags: cmake, buildsystems, cpp, tutorial
Published: 2020-09-14 18:34:54


Whether we want to or not, many folks have to interact with CMake at least once in their life. If they are unlucky, they might have to find dependencies using `find_package`. If they are extremely unlucky, they might have to write a file to work with `find_package`. At the end of this, they might have a file that kind of works, but most likely is based off of a tutorial from 2008. A lot of things have changed since then. There's an easier way to do it these days!

In a previous post, I alluded to a project I've been working on since 2018. It's called IXM, and while it's not ready for general use, it is where I realized there is a common set of operations for finding packages, their components, and setting their correct properties. We'll be using some of what I've learned and figured out over the past two years to understand how `find_package` files work and how to make them useful for your projects and the folks who depend on them.

This tutorial is written for CMake 3.14 and later, though I personally recommend using CMake 3.16 as it is the version installed with the most recent Ubuntu LTS release, 20.04.

Additionally, this tutorial isn't meant to discuss how to write a `<package-name>-config.cmake` file. Those have a different set of options but also tend to be smaller in practice. Instead I'll be showing how to write what's known as a `find_package` `MODULE` file. That said, we will eventually tackle how to handle writing a usable `find_package` file that can be used in `cmake --find-package` mode, in the so-called [Script Mode](https://cmake.org/cmake/help/latest/manual/cmake.1.html#run-a-script) or with `cpack` for the [External Generator](https://cmake.org/cmake/help/latest/cpack_gen/external.html). For now, however, we'll focus on the most common workflow for CMake: Configure and Build.

# The Basics

Within CMake, there are several commands that are used when writing a `find_package` file. The most *important* ones are [find_program][1], [find_library][2], [find_path][3], and lastly [find_file][4]. Each of these has a purpose, but we will not always use them. Depending on *what* you are trying to find, you might also find yourself using [execute_process][5], [file(READ)][6], [file(STRINGS)][7], [string(REGEX MATCH)][8], and [mark_as_advanced][9]. Additionally, 99.9% of the time, you'll want to use the CMake provided module [FindPackageHandleStandardArgs][10]. 

Before we can use `find_package`, however, we need to make sure CMake can find it in the first place. The most common place to put your cmake scripts is inside the project's root directory under a `cmake/` directory. We can then add this path to our `CMAKE_MODULE_PATH` variable so CMake knows where to find us.
{% gist https://gist.github.com/bruxisma/5a84264e82e5864b12c875748cb563fe/37b960914146f282c94e3acb8e2a78879d230ee4 file=CMakeLists.txt %}

This is the bare minimum we need for CMake to find our following `find_package` files.

## Understanding the Commands

The commands most typically used have a *very large* amount of documentation and thus it can be a bit overwhelming. Worry not. *Most of the time* you don't need to worry about these additional parameters. We can cover them in detail in later posts.

### `find_program`
This is used to find the path to a file that the system considers *executable*. Whether it is a script or an actual executable is actually irrelevant. As long as you could execute it via something like `exec` (or `CreateProcess` on Windows), it can be used as an executable.

### `find_library`

This is used to find *both* shared and static libraries. The form for this is a bit odd, as languages with custom static library formats (e.g., Rust's `.rlib`) won't ever be found, however we *can* find `.so`, `.dylib`, `.dll`, `.a`, and `.lib` files by default. on macOS, `.framework`s are also searched for first. Users can configure this with the [CMAKE_FIND_FRAMEWORK](https://cmake.org/cmake/help/latest/variable/CMAKE_FIND_FRAMEWORK.html) variable.

## Finding an Executable

Finding executables in packages is one of the easiest things, compared to finding libraries. Executables can be made into imported targets, much like libraries, and they can then be used in the `add_custom_command` and `add_custom_target` commands with their imported name.

A popular tool that we can use as an example here is `sphinx-build`, the actual "compiler" for the [sphinx documentation framework](https://sphinx-doc.org).

Inside of a file named `FindSphinx.cmake`, we have

{% gist https://gist.github.com/bruxisma/5a84264e82e5864b12c875748cb563fe/37b960914146f282c94e3acb8e2a78879d230ee4 file=FindSphinx.cmake %}

You'll notice that, unlike what you might be used to, we use `Sphinx_EXECUTABLE` and not `SPHINX_EXECUTABLE`. This will make more even more sense in later posts, but it's consider "good behavior" to prefix your variables with the *package name*, and not an upper cased version of said package name. This also prevents a theoretical `find_package(SPHINX)` and a `find_package(Sphinx)` having a variable collision.

That's the basics of it! We can now let `find_package_handle_standard_args` take care of business for us, and then hide our cache variable if it's been found.

{% gist https://gist.github.com/bruxisma/5a84264e82e5864b12c875748cb563fe/350f46fa9b18b237eae084acc05bd78ed6307a36 file=FindSphinx.cmake %}

Seems simple enough, right? Well, we still need to *import* this executable so it's usable by the build system. To do that we're going to create what's known as an *imported executable* and set the `IMPORTED_LOCATION` property to the value stored in `Sphinx_EXECUTABLE`

{% gist https://gist.github.com/bruxism/5a84264e82e5864b12c875748cb563fe/e60c421241b06a27dbad76da3236175d1fc541d5 file=FindSphinx.cmake %}

Of course, it's *not enough* to just do this. We need to guard our sphinx target creation. What if we never found it in the first place? The file would error for users and that's *no bueno*!

{% gist https://gist.github.com/bruxisma/5a84264e82e5864b12c875748cb563fe/b0ee03d56c55a8e4deca99dddae7995e84d33d76 file=FindSphinx.cmake %}

Additionally, what if someone wants to name their target `Sphinx`? It's best to try to stay out of their way, or our simple documentation generator might interfere with someone's actual dependency or project! We can do this by *namespacing* the `Sphinx` target, which is only permitted for *imported* targets. 

{% gist https://gist.github.com/bruxisma/5a84264e82e5864b12c875748cb563fe/c113302c1a20d847bdec5fa7103e89272d619d84 file=FindSphinx.cmake %}

But wait! What if multiple projects in our build tree depend on `find_package(Sphinx)`? Then it will fail to work because `Sphinx::Sphinx` is already a target! We should fix that by only creating the target if we've found Sphinx *and* it hasn't been created yet.

{% gist https://gist.github.com/bruxisma/5a84264e82e5864b12c875748cb563fe/824604248639f6ae8a9d2ee2cf0f1dad86efb7bb file=FindSphinx.cmake %}

While this might seem simple, once we move into later posts in this tutorial, we will expand on this and get it to its *proper* state.

## Finding a Library

Finding libraries in CMake is deceptively simple. It should just be as easy as finding the path to a single file, setting a variable, and calling it a day... right?

If only that were the case. Remember, we have access to imported targets, and this can be used to provide more information to CMake when generating the build system, and prevents typos for variables. Even more important, we can create dependency lists of libraries and their components so that users can follow the "YOLO" principle (i.e., *You Only Link Once*).

Worse, sometimes these libraries provide `<Library>Config.cmake`/`<library>-config.cmake` files which might not create imported targets and simply export cache variables. One such library is [SDL2][11]. SDL2 is a wonderfully neat library, but it's CMake experience leaves a lot to be desired. Sounds like a perfect way to create our own `FindSDL2.cmake` file.

Using what we've learned above regarding `find_package` for executable based packages, we can start with a basic skeleton file that's eerily similar to what we did before.

{% gist https://gist.github.com/bruxisma/5a84264e82e5864b12c875748cb563fe/694f2fb775bf508c82b42f967cee7ff8ef497cb0 file=FindSDL2.cmake %}

Try to use this however, and you'll quickly find there is no way to `#include <SDL2/SDL.h>` 😱! That just won't do! We need to *find the path* where this file is found. This is where [`find_path`][3] comes into play to find the `SDL2_INCLUDE_DIR`. It's a requirement for us to use SDL2 at all, so it'll be added to the `REQUIRED_VARS` parameter to `find_package_handle_standard_args`. Because it's a cache variable, we'll also mark it as `ADVANCED` if we found SDL2, and finally add it to the `INTERFACE_INCLUDE_DIRECTORIES` property for `SDL2::SDL2`

{% gist https://gist.github.com/bruxisma/5a84264e82e5864b12c875748cb563fe/0a9bb82e6962ba667b12baac2762903d0c185a6a file=FindSDL2.cmake %}

Awesome! We can now just call `target_link_libraries(<my-target> PRIVATE SDL2::SDL2)` and we've got full access to its headers and automatically link against the library itself.

That... should be it right? Unfortunately, no. On Windows, we must sometimes link against `SDL2main` (other platforms are permitted to as well), because SDL2 sometimes overrides the `main` function with a redefinition. We can ask that `find_package` search for `SDL2main` by declaring a component called `SDL2::Main`. Ideally, we will always search for all components, and let `find_package` and `find_package_handle_standard_args` take care of the details. However we need to (of course) do some extra work.

Firstly, we need to find the `SDL2main` library itself. CMake's `find_package` cares less about how variables are named, but *does* care about how the `_FOUND` variables are named. Effectively, for each component in a package, `find_package_handle_standard_args` considers a component found if `<package>_<component>_FOUND` is true or false. Generally, you'll want to stick to a consistent naming convention, so we'll name our `_LIBRARY` variable `SDL2_Main_LIBRARY`, and if it's set *at all*, we'll just set `SDL2_Main_FOUND` to `YES` (one of the strings CMake considers to be a boolean)

We then pass `HANDLE_COMPONENTS` as a flag to `find_package_handle_standard_args`.

{% gist https://gist.github.com/bruxisma/5a84264e82e5864b12c875748cb563fe/7bf2ebba2cc1e8a524f5f873add91fd09ba9ca1c file=FindSDL2.cmake %}

Next we'll need to also mark the variable as advanced but *only* if its found and ONLY if `find_package_handle_standard_args` didn't return early. This might seem counter-intuitive, but that's CMake for ya! We might as well also import the library and make it depend on `SDL2::SDL2`.

{% gist https://gist.github.com/bruxisma/5a84264e82e5864b12c875748cb563fe/d745eab7e620b2bcf48121a723e16126702dcdd6 file=FindSDL2.cmake %}

That should be it, right? Again, things are not as simple as they may seem. SDL2 introduces bug fixes, new APIs, etc. in minor point releases. How do we know for sure we're building with SDL2 2.15? We need to check the version. How does SDL2 express this? Inside of the `SDL2/SDL_version.h` header. This, here, is the part where things will become very painful. The only solution we have is to... use a regex 😱😭.

SDL2 has the following C preprocessor defines declared:

 * `SDL_MAJOR_VERSION`
 * `SDL_MINOR_VERSION`
 * `SDL_PATCHLEVEL`

Luckily, these are fairly simple to find, so we can just get a list of matching lines via [`file(STRINGS)`][7]

{% gist https://gist.github.com/bruxisma/5a84264e82e5864b12c875748cb563fe/eebe25909be7af5612c23ce1ca3192e70c062132 file=FindSDL2.cmake %}

OK, but that doesn't actually get us the data right? So we need to extract it further, and this is where it *really* does get messy. We'll also pass it to `find_package_handle_standard_args` as the `VERSION_VAR` and then set the `VERSION` property of `SDL2::SDL2`

{% gist https://gist.github.com/bruxisma/5a84264e82e5864b12c875748cb563fe/924717d4f85a4d98bd842b63c9ea46a1d6ead048 file=FindSDL2.cmake %}

And that's all we need to do! We're done 🎉 (for now 😱)

Exhausting wasn't it? Luckily, once you write a file like these you rarely need to ever touch them again... At least, until the next entry in this series. 😈

[1]: https://cmake.org/cmake/help/latest/command/find_program.html
[2]: https://cmake.org/cmake/help/latest/command/find_library.html
[3]: https://cmake.org/cmake/help/latest/command/find_path.html
[4]: https://cmake.org/cmake/help/latest/command/find_file.html
[5]: https://cmake.org/cmake/help/latest/command/execute_process.html
[6]: https://cmake.org/cmake/help/latest/command/file.html#read
[7]: https://cmake.org/cmake/help/latest/command/file.html#strings
[8]: https://cmake.org/cmake/help/latest/command/string.html#regex-match
[9]: https://cmake.org/cmake/help/latest/command/mark_as_advanced.html
[10]: https://cmake.org/cmake/help/latest/module/FindPackageHandleStandardArgs.html
[11]: https://libsdl.org