Project: Apache Mesos (Orchestration & Management / Scheduling & Orchestration)
File: cmake-examples.md

title: Apache Mesos - CMake By Example layout: documentation When adding a new library or executable, prefer using the name directly as the target. E.g. `libprocess` is `add_library(process)`, and `mesos-agent` is `add_executable(mesos-agent)`. Note that, on platforms where it is conventional, `add_library` will prepend `lib` when writing the library to disk. Do not introduce a variable simply to hold the name of the target; if the name on disk needs to be a specific value, set the target property `OUTPUT_NAME`. When adding a third-party dependency, keep the principle of locality in mind. All necessary data for building with and linking to the library should be defined where the library is imported. A consumer of the dependency should only have to add `targetlinklibraries(consumer dependency)`, with every other build property coming from the graph (library location, include directories, compiler definitions, etc.). The steps to add a new third-party dependency are: Add the version and SHA256 hash to `Versions.cmake`. Add the URL/tarball file to the top of `3rdparty/CMakeLists.txt`. Find an appropriate location in `3rdparty/CMakeLists.txt` to declare the library. Add a nice header with the name, description, and home page. Use `add_library(IMPORTED)` to declare an imported target. A header-only library is imported with `add_library(INTERFACE)`. Use to obtain, configure, and build the library. Link the consumer to the dependency. Using header-only libraries in CMake is a breeze. The special `INTERFACE` library lets you declare a header-only library as a proper CMake target, and then use it like any other library. Let's look at Boost for an example. First, we add two lines to `Versions.cmake`: ``` set(BOOST_VERSION "1.53.0") set(BOOST_HASH "SHA256=CED7CE2ED8D7D34815AC9DB1D18D28FCD386FFBB3DE6DA45303E1CF193717038") ``` This lets us keep the versions (and the SHA256 hash of the tarball) of all our third-party dependencies in one location. Second, we add one line to the top of `3rdparty/CMakeLists.txt` to declare the location of the tarball: ``` set(BOOSTURL ${FETCHURL}/boost-${BOOST_VERSION}.tar.gz) ``` The `FETCH_URL` variable lets the `REBUNDLED` option switch between offline and online versions. The use of `BOOST_VERSION` shows why this variable is declared early; it's used a few times. Third, we find a location in `3rdparty/CMakeLists.txt` to declare the Boost library. ``` ... ``` We start with a proper header naming and describing the library, complete with its home page URL. This is for other developers to easily identify why this third-party dependency exists. ``` ... EXTERNAL(boost ${BOOSTVERSION} ${CMAKECURRENTBINARYDIR}) add_library(boost INTERFACE) adddependencies(boost ${BOOSTTARGET}) targetincludedirectories(boost INTERFACE ${BOOST_ROOT}) ... ``` Fourth, we declare the Boost target. To make things easier, we invoke our custom CMake function `EXTERNAL` to setup some variables for us: `BOOSTTARGET`, `BOOSTROOT`, and `BOOSTCMAKEROOT`. See for more explanation of `EXTERNAL`. Then we call `add_library(boost INTERFACE)`. This creates a header-only CMake target, usable like any other library. We use `add_dependencies(boost ${BOOSTTARGET})` to add a manual dependency on the `ExternalProjectAdd` step; this is necessary as CMake is lazy and won't execute code unless it must (say, because of a dependency). The final part of creating this header-only library in our build system is `targetincludedirectories(boost INTERFACE ${BOOSTROOT})`, which sets the `BOOSTROOT` folder (the destination of the extracted headers) as the include interface for the `boost` target. All dependencies on Boost will now automatically include this folder during compilation. Fifth, we setup the `ExternalProject_Add` step. This CMake module is incredibly flexible, but we're using it in the simplest case. ```

ExternalProject_Add( ${BOOST_TARGET} PREFIX ${BOOSTCMAKEROOT} CONFIGURECOMMAND ${CMAKENOOP} BUILDCOMMAND ${CMAKENOOP} INSTALLCOMMAND ${CMAKENOOP} URL ${BOOST_URL} URLHASH ${BOOSTHASH}) ``` The name of the custom target this creates is `BOOST_TARGET`, and the prefix directory for all the subsequent steps is `BOOSTCMAKEROOT`. Because this is a header-only library, and `ExternalProject_Add` defaults to invoking `cmake`, we use `CMAKE_NOOP` to disable the configure, build, and install commands. See [the docs](cmake.md#cmakenoop) for more explanation of `CMAKENOOP`. Thus this code will simply verify the tarball with `BOOST_HASH`, and then extract it from `BOOSTURL` to `BOOSTROOT` (a sub-folder of `BOOSTCMAKEROOT`). Sixth, and finally, we link `stout` to `boost`. This is the only change necessary to `3rdparty/stout/CMakeLists.txt`, as the include directory information is embedded in the CMake graph. ``` targetlinklibraries( stout INTERFACE ... boost ...) ``` This dependency need not be specified again, as `libprocess` and `libmesos` link to `stout`, and so `boost` is picked up transitively. Stout is a header-only library. Like Boost, it is a real CMake target, declared in `3rdparty/stout/CMakeLists.txt`, just without the external bits. ``` add_library(stout INTERFACE) targetincludedirectories(stout INTERFACE include) targetlinklibraries( stout INTERFACE apr boost curl elfio glog ...) ``` It is added as an `INTERFACE` library. Its include directory is specified as an `INTERFACE` (the `PUBLIC` property cannot be used as the library itself is just an interface). Its "link" dependencies (despite not being a real, linkable library) are specified as an `INTERFACE`. This notion of an interface in the CMake dependency graph is what makes the build system reasonable. The Mesos library and executables, and `libprocess`, do not have to repeat these lower level dependencies that come from `stout`. Third-party dependencies that we build are only more complicated because we have to encode their build steps too. We'll examine `glog`, and go over the differences from the interface library `boost`. Notably, when we declare the library, we use: ``` addlibrary(glog ${LIBRARYLINKAGE} IMPORTED GLOBAL) ``` Instead of `INTERFACE` we specify `IMPORTED` as it is an actual library. We add `GLOBAL` to enable our pre-compiled header module `cotire` to find the targets (as they would otherwise be scoped only to `3rdparty` and below). And most oddly, we use `${LIBRARY_LINKAGE}` to set it as `SHARED` or `STATIC` based on `BUILDSHAREDLIBS`, as we can build this dependency in both manners. See [the docs](cmake.md#library_linkage) for more information. We must patch our bundled version of `glog` so we call: ``` PATCHCMD(GLOGPATCHCMD glog-${GLOGVERSION}.patch) ``` This generates a patch command. See for more information. This library is an example of where we differ on Windows and other platforms. On Windows, we build `glog` with CMake, and have several properties we must set: ``` settargetproperties( glog PROPERTIES IMPORTEDLOCATIONDEBUG ${GLOGROOT}-build/Debug/glog${LIBRARYSUFFIX} IMPORTEDLOCATIONRELEASE ${GLOGROOT}-build/Release/glog${LIBRARYSUFFIX} IMPORTEDIMPLIBDEBUG ${GLOGROOT}-build/Debug/glog${CMAKEIMPORTLIBRARYSUFFIX} IMPORTEDIMPLIBRELEASE ${GLOGROOT}-build/Release/glog${CMAKEIMPORTLIBRARYSUFFIX} INTERFACEINCLUDEDIRECTORIES ${GLOG_ROOT}/src/windows IMPORTEDLINKINTERFACE_LIBRARIES DbgHelp INTERFACECOMPILEDEFINITIONS "${GLOGCOMPILEDEFINITIONS}") ``` The location of an imported library must be set for the build system to link to it. There is no notion of search through link directories for imported libraries. Windows requires both the `DEBUG` and `RELEASE` locations of the library specified, and since we have (experimental) support to build `glog` as a shared library on Windows, we also have to declare the `IMPLIB` location. Fortunately, these locations are programmatic based of `GLOG_ROOT`, set from our call to `EXTERNAL`. Note that we cannot use `targetincludedirectories` with an imported target. We have to set `INTERFACEINCLUDEDIRECTORIES` manually instead. This version of `glog` on Windows depends on `DbgHelp` but does not use a `#pragma` to include it, so we set it as an interface library that must also be linked, using the `IMPORTEDLINKINTERFACE_LIBRARIES` property. For Windows there are multiple compile definitions that must be set when building with the `glog` headers, these are specified with the `INTERFACECOMPILEDEFINITIONS`

For non-Windows platforms, we just set the Autotools commands to configure, make, and install `glog`. These commands depend on the project requirements. We also set the `IMPORTEDLOCATION` and `INTERFACEINCLUDE_DIRECTORIES`. ``` set(GLOGCONFIGCMD ${GLOGROOT}/src/../configure --with-pic GTESTCONFIG=no --prefix=${GLOG_ROOT}-build) set(GLOGBUILDCMD make) set(GLOGINSTALLCMD make install) settargetproperties( glog PROPERTIES IMPORTEDLOCATION ${GLOGROOT}-build/lib/libglog${LIBRARY_SUFFIX} INTERFACEINCLUDEDIRECTORIES ${GLOG_ROOT}-build/include) ``` To work around some issues, we have to call `MAKEINCLUDEDIR(glog)` to create the include directory immediately so as to satisfy CMake's requirement that it exists (it will be populated by `ExternalProject_Add` during the build, but must exist first). See for more information. Then call `GETBYPRODUCTS(glog)` to create the `GLOGBYPRODUCTS` variable, which is sent to `ExternalProject_Add` to make the Ninja build generator happy. See for more information. ``` MAKEINCLUDEDIR(glog) GET_BYPRODUCTS(glog) ``` Like with Boost, we call `ExternalProject_Add`: ``` ExternalProject_Add( ${GLOG_TARGET} PREFIX ${GLOGCMAKEROOT} BUILDBYPRODUCTS ${GLOGBYPRODUCTS} PATCHCOMMAND ${GLOGPATCH_CMD} CMAKEARGS ${CMAKEFORWARDARGS};-DBUILDTESTING=OFF CONFIGURECOMMAND ${GLOGCONFIG_CMD} BUILDCOMMAND ${GLOGBUILD_CMD} INSTALLCOMMAND ${GLOGINSTALL_CMD} URL ${GLOG_URL} URLHASH ${GLOGHASH}) ``` In contrast to an interface library, we need to send all the build information, which we set in variables prior. This includes the `BUILD_BYPRODUCTS`, and the `PATCH_COMMAND` as we have to patch `glog`. Since we build `glog` with CMake on Windows, we have to set `CMAKE_ARGS` with the `CMAKEFORWARDARGS`, and particular to `glog`, we disable its tests with `-DBUILD_TESTING=OFF`, though this is not a canonical CMake option. On Linux, we set the config, build, and install commands, and send them too. These are empty on Windows, so `ExternalProject_Add` will fallback to using CMake, as we needed. Finally, we add `glog` to as a link library to `stout`: ``` targetlinklibraries( stout INTERFACE ... glog ...) ``` No other code is necessary, we have completed adding, building, and linking to `glog`. The same patterns can be adapted for any other third-party dependency. The default configuration is always `Debug`, which means with debug symbols and without (many) optimizations. Of course, when deploying Mesos an optimized `Release` build is desired. This is one of the few inconsistencies in CMake, and it's due to the difference between so-called "single-configuration generators" (such as GNU Make) and "multi-configuration generators" (such as Visual Studio). In single-configuration generators, the configuration (debug or release) is chosen at configuration time (that is, when initially calling `cmake` to configure the build), and it is not changeable without re-configuring. So building a `Release` configuration on Linux (with GNU Make) is done via: ``` cmake -DCMAKEBUILDTYPE=Release .. cmake --build . ``` However, the Visual Studio generator on Windows allows the developer to change the release at build-time, making it a multi-configuration generator. CMake generates a configuration-agnostic solution (and so `CMAKEBUILDTYPE` is ignored), and the user switches the configuration when building. This can be done with the familiar configuration menu in the Visual Studio IDE, or with CMake via: ``` cmake .. cmake --build . --config Release ``` In the same build folder, a `Debug` build can also be built, with the binaries stored in `Debug` and `Release` folders respectively. Unfortunately, the current CMake build explicitly sets the final binary destination directories, and so the final libraries and executables will overwrite each other when building different configurations. Note that Visual Studio is not the only IDE that uses a multi-configuration generator, Xcode on Mac OS X does as well. See for more information. On Linux, the configuration option `-DBUILDSHAREDLIBS=FALSE` can be used to switch to static libraries where possible. Otherwise Linux builds shared libraries by default. On Windows, static libraries are the default. Building with shared libraries on Windows is not yet supported, as it requires code change to import symbols properly.