Thoughts about configure scripts and feature vectors
Published: 2022-05-06 14:17:10

For a long time, I’ve been wanting to rant about stuff like configure scripts. They indirectly contribute a lot to my worktime headaches these days. Given I’m restarting personal blogging here, let’s see if I can turn that rant into a post.

## What’s configure

Suppose you need to compile some software from a source tarball. The old unix tradition goes like `./configure && make && make install`. Nothing fancy about the `make` part, but why is `configure` needed in the first place?

Well, `configure` is just a shell script that probes the build environment and generates some C header file to be included in the source code. Each project has its unique `configure` script that probes for different things, headers, functions, any feature that the source code needs to check for build-time existence and provide a fallback code path when it’s absent.

Say that the source code needs to call the `foo` function, which exists on just some of the project’s supported platforms. If `configure` detects `foo`, it’ll write a `#define HAVE_FOO 1` line to the generated header. The source code can then include the auto-generated feature header, use CPP declarations like `#if defined(HAVE_FOO)` to decide whether function `foo` exists in the build environment.

`configure` is typically auto-generated from a template using autoconf. In some projects it can be a hand-written python script. There are also build systems like cmake that take over `configure`'s role completely, probes the build environment on their own and generate the feature header.

Anyway, my rants here are only related to the idea of build-time feature detection, and irrelevant to how `configure` is actually implemented (although that’s also annoying enough for it’s own blog post)

## What’s a feature vector

How many `HAVE_` macros do you have in your project?

```bash
~/ubuntu/ghc$ grep -rIF HAVE_ | wc -l
842
```

Wait a sec. Most of those should be mere duplications, for instance, `HAVE_FOO` is very likely to occur in multiple source locations. One should really check how many features (headers, functions, etc) are checked by `configure`.

```bash
~/ubuntu/ghc$ grep -rIF AC_CHECK_ | wc -l
171
```

The number above is a lower estimation, since in autoconf, a single `AC_CHECK_HEADERS` or `AC_CHECK_FUNCS` line can check multiple entities.

Now, we can introduce the concept of a “feature vector”: an `N`-dimentional boolean vector, where `N` corresponds to the number of things you’re checking at build-time. Each value of the feature vector is a point in the feature space, specifying a build-time configuration.

How large is the feature space?

- Definitely not as large as `2^N`. Most of the dimensions aren’t orthogonal, one can imagine clusters of things that either exist as a whole, or not exist at all.
- Still, way larger than the the space where people actually test on CI to avoid bit-rotting.

My rant is the second point above.

## Why the rant

- In GHC, one can pass various `configure` arguments to enable/disable features like unreg codegen, large address space, native IO manager, etc. The default configuration passes the test suite, but once you start messing with `configure` config, expect failed test cases. At least those cases should be explicitly marked fragile/broken in those configurations!
- In GHC, `unix`, probably other places I’ve hacked and forgotten: the API evolves, but people forgot to update the code in the `#else` -guarded parts, because it’s not tested on CI, maybe that particular CPP-checked thing is thought to exist on all platforms. Well, WASI is a rather restricted platform, so those bitrotten parts all come back to bite you when I target WASI.

There’s nothing to blame for the need to write portable code and do build-time feature detection. We all know untested code is bad, but much fewer people are aware: untested feature vectors are also bad!

Also, this isn’t just a matter of “code coverage”. It’s perfectly possible to achieve a high coverage rate by testing against just a few feature vectors, while leaving the potentially broken build-time configurations in the dark.

## How to solve it

Most software written with tons of `#ifdef` lack the feature vector mindset, and don’t have the testing logic to:

- Perform `QuickCheck`-style random testing in the feature space. Generate a feature vector, run the tests against it, and “shrinking” is just moving the point closer to the known-to-work base point, the default config you get when configuring on a typical platform without any custom arguments. This allows discovering failures that arise from complex & unintended interactions between different dimensions in the feature vector.
- Hide certain existing auto-detected features. This allows testing for restricted/exotic platforms, while still running the tests on a common platform. Not trivial to implement, especially for things in standard libraries, but it should be possible by using `poison` pragma, creating `cc` wrappers, or even isolated sysroots.