Thinking in Make
Published: 2017-12-24 21:51:37

---
title: "Thinking in Make"
published: true
tags: 
canonical_url: http://blog.153.io/2017/07/23/thinking-in-make/
---

Okay, but what _is_ a makefile. Actually what is it.

## Make is for running tasks

Here’s a simple makefile, as barebones as possible without it literally being an empty file:

```make
do-a-thing:
	echo “did the thing”
```

Save that as `makefile`, and in the directory, run `make do-a-thing`:

```
⟩ make do-a-thing
echo “did the thing”
did the thing
```

Tell Make what to run and it runs it. Each line like `do-a-thing:` specifies a _target_, and the indented lines (which need to be indented with actual tabs, sorry space-lovers) after the name are the shell commands to run for that target.

## Make is for generating files

When the commands in a target create a file, you should let Make know about it. If the name of a target is the same as the name of the file it creates, Make can track that:

```make
foo.txt:
	echo lorem ipsum > foo.txt
```
```
⟩ make foo.txt
echo lorem ipsum > foo.txt
⟩ make foo.txt
make: Nothing to be done for ‘foo.txt’
```

Wait what? Nothing to be done? When the file exists, Make notices, and realises it doesn’t need to do anything. `foo.txt` isn’t going to change if it runs the commands, so it doesn’t bother running them. Remove the file, and Make will create it again.

```bash
⟩ rm foo.txt
⟩ make foo.txt
echo lorem ipsum > foo.txt
```

## Make is for generating files _from other files_

Say you’re generating a file and it depends on the content of another file. Maybe you’re just copying it:

```make
foo.txt:
	cp bar.txt foo.txt
```

`make` doesn’t do anything when you update `bar.txt`:

```
⟩ make foo.txt
cp bar.txt foo.txt
⟩ echo hello > bar.txt
⟩ make foo.txt
make: Nothing to be done for ‘foo.txt’
```

This is the same as the last example: the file exists, so `make` won’t create it again. If you want `foo.txt` to update whenever `bar.txt` does, you need to tell `make` about it, by marking it as a _prerequisite_ of `foo.txt`:

```make
foo.txt: bar.txt
	cp bar.txt foo.txt
```

Everything after the colon in a target’s name is a prerequisite of that target. It can be a file name, the name of another target, or even the name of another target that’s a file to generate. Prerequisites can also be chained: a target that’s a prerequisite can also have prerequisites, and so on.

Running this example again does what you want: if you update `bar.txt` and run `make foo.txt` again, it’s updated. If `bar.txt` hasn’t been updated, `make` won’t do anything:

```
⟩ make foo.txt
cp bar.txt foo.txt
⟩ echo hello > bar.txt
⟩ make foo.txt
cp bar.txt foo.txt
```

## Make can work out filenames from patterns

What if you’ve got a whole folder’s worth of targets and prerequisites? You don’t want to have to write a target for each one. I mean, that just sounds boring, and you’d have to remember to update it whenever you added or deleted a file.

So if your file paths are predictable, e.g. if you want to express something like “here’s how to make any file in _this folder_ from the file with the same name in _that folder_”, you use _file patterns_. In the target and the prerequisite, write the bit that’s the hole to be filled in as `%`, e.g. `target-files/%.txt: source-files/%.txt`.

Then how do you refer to the actual filenames, if all we know is the pattern? `make` defines a handful of _automatic variables_ to use with pattern rules, which it sets based on the pattern and the filename it’s matching. The two you need to know are `$@`, which is the matched target filename, and `$<`, which is the prerequisite filename it infers from the target pattern, the target filename and the prerequisite pattern, by working out what in the filename replaced `%` in the pattern (the _stem_).

Given our example above, when you run `make target-files/foo.txt`, `make` matches it against `target-files/%.txt` and runs that target. It sets `$@` to `target-files/foo.txt`, and because the stem is `foo` (that’s what replaced `%` in the pattern), it sets `$<` to `source-files/foo.txt`.

All together:

```make
target-files/%.txt: source-files/%.txt
	cp $< $@
```
```
⟩ make target-files/foo.txt
cp source-files/foo.txt target-files/foo.txt
```

## Don’t tell Make what to do, let it find its own way

Ok, what if you want to `make` everything? All of the files? `make` is actually a fully-fledged (if a little arcane, but hey, it’s 40 years old) programming language, with a handful of builtins useful for generating lists of files you want it to build.

One of these is the `wildcard` function. It takes a shell-like wildcard expression and expands it to a list of files that match. Let’s say your `source-files` folder contains `foo.txt`, `bar.txt` and `baz.txt`, then `$(wildcard source-files/*.txt)` expands to `source-files/foo.txt source-files/bar.txt source-files/baz.txt`. Why does this use `*` when patterns use `%`? Because reasons.

Then there’s `patsubst`. Give it two patterns and a list of files, and it matches and **subst** itutes the **pat** terns. So `$(patsubst source-files/%.txt, target-files/%.txt, source-files/foo.txt source-files/bar.txt)` evaluates to `target-files/foo.txt target-files/bar.txt`.

I mentioned the automatic pattern variables above. You can define your own variables, and the syntax will look familiar: `source-files = $(wildcard source-files/*.txt)`. Refer to variables using `$(variable-name)`, e.g. `$(source-files)`, and if the name is a single character you can drop the `()`, like with `$<` and `$@` (which aren’t anything special; `make` has _very_ lenient syntax for variables, and pretty much the only characters you _can’t_ use are `=`, `#`, `:` or whitespace).

Variables can be used in prerequisites and commands. You’ll see them used in commands to pass long options to command-line programs. When used in prerequisites they can be used to make a bunch of files at once. A common use case is making every file in a folder:

```make
source-files = $(wildcard source-files/*.txt)
target-files = $(patsubst source-files/%.txt, target-files/%.txt, $(source-files))

all: $(target-files)

target-files/%.txt: source-files/%.txt
	cp $< $@
```
```
⟩ make all
cp source-files/foo.txt target-files/foo.txt
cp source-files/bar.txt target-files/bar.txt
cp source-files/baz.txt target-files/baz.txt
```

Breaking this down from the end, it’s the same target we saw before, which makes any `.txt` file in `target-files/`. The `all` target has no commands, and the variable `$(target-files)` as a prerequisite. When a target has no commands, it’s just there for the prerequisites. `make` makes those in order to make `all`, and then makes `all` by doing nothing. Finally `$(target-files)` is generated by using `wildcard` to get a list of files in `source-files` and `patsubst` to turn that list into a list of files in `target-files`. I’ll talk some more about this in a bit.

Alright now check this out. I’ve gone over how `make` only builds the things it needs to? This works _really well_ when building a bunch of files at once:

```
⟩ make all
make: Nothing to be done for ‘all’
⟩ echo hello > source-files/baz.txt
⟩ make all
cp source-files/baz.txt target-files/baz.txt
```

Update `baz.txt`, leave the other two alone, and that’s the only one that `make` decides to build. Internally, it’s looking at file modification times. When it discovers a target file depends on a prerequisite file, and both exist, it only makes the target if the prerequisite is newer.

Oh, and if you run just `make` on its own, it runs the first (non-pattern) target in the file. By convention, people call this `all`. So you could run the last two examples with `make` instead of `make all`.

## The part where I actually talk about the title of the post

What is the point of `make`? By now, I hope you’ll see it’s more than a simple task runner. If you have a bunch of input files, and you want to transform them into a bunch of output files, the tool you should be reaching for is `make`. If your input filenames look like your output filenames, see if you can write a pattern target.

These are the steps I go through when writing a makefile:

1. Write a simple rule using concrete filenames
2. Make the rule generic with patterns
3. Work out how to generate a list of output files from your input filenames
4. Make everything all at once
5. Receive admiration from your friends and colleagues

I’m still on step 4.

I’ll go through a concrete example: building Javascript sources using Babel. To start with, it’s a single file, `src/index.js` that I want to compile to `lib/index.js`. To transform the file we run `babel src/index.js -o lib/index.js`, so the whole rule would look like:

```make
lib/index.js: src/index.js
	babel src/index.js -o lib/index.js
```

Now I’ve got more `.js` files, I want to turn it into a pattern rule. The output and input files have a pretty clear pattern of `lib/%.js: src/%.js`, so using that with automatic pattern variables, we get:

```make
lib/%.js: src/%.js
	babel $< -o $@
```

To build all of the `.js` files at once, we need a list of target files that our source files would generate. So get the list of source files:

```make
source-files = $(wildcard src/*js)
```

And transform the output filenames:

```make
output-files = $(patsubst src/%.js, lib/%.js, $(source-files))
```

Then we can use the list as a prerequisite:

```make
all: $(output-files)
```

And so our final makefile looks like:

```make
source-files = $(wildcard src/*js)
output-files = $(patsubst src/%.js, lib/%.js, $(source-files))

all: $(output-files)

lib/%.js: src/%.js
	babel $< -o $@
```

## Extra credit: automatically rebuilding files

Because `make` doesn’t do anything it doesn’t have to, a _really really_ basic way of remaking things as your change the input files is to run it in a Bash loop:

```bash
while true; do
	make
	sleep 1
done
```

Of course, if you’re not updating anything you’ll spam your terminal with `make: Nothing to be done for ‘all’` forever. One alternative is a package called [`watch-make`](https://github.com/quarterto/watch-make) by (_coughs, looks bashful in a british-false-humility way_) me. It wraps `make`, gets the prerequisite files from it, and reruns `make` when they change:

```
⟩ wmake

  ⛭ running make
  │ babel src/index.js -o lib/index.js
  ✔︎ make
  ⚲ watching 2 files

  ✏︎ changed src/index.js
  ⛭ running make
  │ babel src/index.js -o lib/index.js
  ✔︎ make
  ⚲ watching 2 files
```

## Make can do a whole lot more

I’ve just scratched the surface here, but this covers way more than most makefiles use in practice. If you’re burning to know more, the [GNU Make documentation](https://www.gnu.org/software/make/manual/html_node/index.html) is exhaustive and very well written, but don’t blame me when you’re up to your elbows in static pattern rules and `SECONDEXPANSION`.

Before you know it you’ll be writing makefiles that make makefiles, and then no-one can save you.

Iä iä make’file fhtagn.

## What about autotools/automake?

No.