Introduction

Hello, and welcome back to CS631 Advanced Programming in the UNIX Environment. This is week 05, segment 7, and yes, we're _still_ talking about the different Unix tools that make up the integrated development environment we take for granted. This time, we'll take a look at the "make" utility, a tool used to selectively build a code project based on an understanding of the dependency relationship between different source- and object files. As we will see, 'make' is thus more than just a convenient way of saving ourselves some typing when we invoke the compiler. Here, let's start by illustrating why we might even want a tool like this to begin with: Ok, so here is our software project, consisting of several source files as well as a few header files. We might compile our program like this. Ok, that worked just fine. But now let's say we want to edit the file 'cmp.c'. We make a simple change here, enabling "sillysort". Now when we recompile the project, we are again running the same command. But note that in this case we _again_ compiled _all_ the source files, even though we only made a change in the file 'cmp.c'. Now let's consider a more efficient way of building our project. Instead of always compiling _all_ source files, we run "cc -c" against each individual file. Remember from our previous videos in this series that if we pass "-c" to the compiler, we will create object files only, and then use a separate stage to link the files together. So now we have these various .o files here, and we can link them into our executable like this. Ok, so far, so good. Now let's again edit the file 'cmp.c', comment out 'sillysort', but now, instead of compiling _all_ files, we'll just compile the file 'cmp.c', then link the object files. Ok, so this is a bit more efficient. The overall difference as shown here is relatively small, but imagine you have a complex software project with hundreds or thousands of source files -- not having to recompile _all_ of them is going to really save you some time here! Now let's consider what happens when we make a change to a different file. Let's say, we edit stat_flags.h and change the default mode down here. Now what files do we have to recompile? Well, let's see which files include this header file. There, main.c and stat_flags.c. Each file that includes this header file might now have changed, so we need to recompile _both_ main.c and stat_flags.c, and then again link the .o files. In other words, I need to keep track of which files have changed, and which files might be affected by that change.

Dependency Graph

Our software project can then be represented as a dependency graph like this. Here we see an illustration of these dependencies. We can tell that a change to cmp.c requires us to rebuild cmp.o and then 'ls'. And a change to stat_flags.h means that now the files main.c and stat_flags.c must be considered outdated, which means that we need to rebuild them and create the main.o and stat_flags.o object files and to then re-create the executable by linking the updated .o files on the right together with the .o files on the left -- which have not changed since our code change. Maintaining such a map of dependencies in your head is not going to be a scalable solution, so enter 'make'. 'make' is used to maintain these dependencies.

Our first Makefile

It reads a definition from a "makefile" and determines so-called "targets" as being derived or built from certain sources. The way 'make' then builds these targets is by executing the specified shell commands. So 'make' allows us to specify the commands we want to execute to build a given executable from the specified source files. So let's create one of these "Makefile"s. By convention, we use a target called "all" and then provide the command we want to execute: cc CFLAGS *.c -o ls Now I can type 'make', and there we go, it builds our executable. And if I were to make a change to, say, 'cmp.c', then all I need to do is type 'make' again, and it'll build "ls". But while this saved us some typing, this didn't really solve the dependency problem, now did it? We are still recompiling all code files. In fact, even if nothing has changed and I type 'make' again: it will again recompile _all_ files. So let's see if we can improve on this. For starters, we can change the "target" name. After all, we want to build the "ls" executable, so let's call our target "ls". This has the effect that now if we run 'make', it tells us that hey, this executable already exists, it must be up to date, no need to recompile all the code. So that's an improvement. But if we make a change to one of the source files -- and here we just fake this by 'touch'ing the file, thereby updating the last modified timestamp of the file, so it looks like it has been changed -- so if we update this file and run 'make' again... hmmm, no good. So we need to improve our Makefile. Here, lets do this: We grab the source files and now edit our Makefile. We tell 'make' that "ls" _depends_ on these.... ".o" files. If these .o files exist, then we link them in this first step. Next, we further tell 'make' how to create the 'cmp.o' file. 'cmp.o' depends on 'cmp.c', meaning whenever the mtime on 'cmp.c' is newer than the mtime on 'cmp.o', we run this command here. Now we repeat the same instruction for the other .o files... change the names... and now 'make' knows that to build 'ls' it needs to build 'cmp.o', which depends on 'cmp.c', and for which it runs 'cc CFLAGS -c cmp.c' and so on for the other object files, too. So let's see what that looks like. Ok, we run 'make', and it compiles each object file and then links them. If we now touch 'cmp.c'... make is smart enough to only recompile that one file. And if we run 'make' again, it knows it's got nothing to do. Neat.

Using a suffix rule

But our Makefile still feels rather clumsy. Here, look at all this repetition we have here. Let's be smarter. Let's create a better Makefile. To do that, we will use a so-called "suffix rule" as well as some variables to show you what make(1) can do. Like in a shell, we can assign values to variables, which, like in the shell, are written in upper case by convention. So here we define our program variable to be 'ls'. We can then define the list of object files we need as the variable OBJS, and we can also define our preferred CFLAGS here. Now with these variables in place, let's get rid of all this stuff here and create a "suffix rule". A "suffix rule" is a rule that tells make(1) how to build a file ending in one suffix from a file ending in a different suffix. So here, we create a rule that tells make(1) how to build a .o file from a .c file. We do that by running "cc CFLAGS -c". Because we now have a generic suffix rule, we can't just put the filename here, but make(1) knows about a few special variables. So here, "$<" means the initial suffix name (.c in our case), and "$@" means the target suffix name (.o in our case). So this block then covers all the individual definitions we had here before. Now we add our "all" target, which we simply set to PROG. When you run "make", the command will automatically look for the first target definition, so using "all" really is just convention. But we still want to use the program name, so we could also run "make ls", for example. Here we define the rules for PROG. PROG depends on the object files we had defined above, and which will then be created automatically via the suffix rule. Now as we read in the manual page, make(1) simply executes shell commands, so we can put anything in here, including diagnostic messages. By default, make(1) will print out the commands it executes, by prefixing this statement with an @, we suppress this. So this statement here simply lets make(1) tell us the dependencies that it considers. The actual building of the executable then is done as before via "cc". Another thing that we are used to is the "clean" target that allows us to easily remove any intermediate object files, so that we can then recreate the executable from scratch, so let's add that target here, too. Ok, now let's see what happens when we run "make": Here we go, we see that "make" builds each object file using the suffix rule we had specified. Note also that while our environment variable CFLAGS is set to "-Wall -Werror -Wextra", make(1) used the definition we had provided in the Makefile itself. If we run "make" again, nothing happens, because our target is up to date. If we change 'cmp.c', then "make" will selectively rebuild only that file, then recreate the executable. Likewise, if we update 'main.o', then make(1) will only run through the linking stage. "make clean" removes our object files and "make" rebuilds our executable. As we just saw, we can have make(1) use variables that we defined in the Makefile, which then override any environment variable we may have set. But what if we want to use a different value from the one set in the Makefile? We could edit the file, but there's another possibility: by specifying the variable declaration on the command-line, we can override that value, too: Now make(1) used "-Werror" in favor of both our initial environment variable as well as the hardcoded value. Next, note that up here we used the LDFLAGS variable, but that variable wasn't set at all. Likewise, we used the CC variable, but that, too, wasn't set! We don't have CC set in the environment, either, and still make(1) called the compiler correctly. That is, make(1) has a few _built-in_ variables that it defaults to. For the CC variable that is the "cc" command. LDFLAGS, however, is not defined at all, so remains empty unless we explicitly set it. Alright, so we've seen how make(1) sorts out the dependencies that we manually specified in the Makefile, which is really quite useful.

Using mkdep(1)

So when a .c file changes, it will selectively rebuild the corresponding .o file and rebuild the executable. But what happens when we change one of the header files? Hmm, nothing. Well, that's not surprising: we didn't tell make(1) about the dependency relationship between this header file and the .c files that include it. We know that if this header changes, then these two .c files need to be rebuilt. So we can add that to the Makefile. Here, main.o now depends on stat_flags.h, and stat_flags.o also depends on stat_flags.h. Cool, now make(1) again does the right thing. But that means that we have to tell make(1) again everything manually ourselves. For .c files to get to .o files, the suffix rule is obvious: just change the suffix. But for a .h file there is no one-to-one relationship. So wouldn't it be neat if we could automatically generate the dependencies? Enter "mkdep(1)", a tool to do just that. It goes through the given C files and generates dependency rules that make(1) understands. So let's grab the example given here... and add it to our Makefile. So now we have a new target here: "depend". When we run "make depend", make(1) invokes mkdep(1), which produces a file called ".depend". Here, this is what it looks like. Oh, wow, look at that: it determined exactly which header files each .c file needs, including those pulled in by other header files! So if any one of these files were to change, we'd have to rebuild this file. Now make(1) still knows all the rules from before: our suffix rule tells it how to build the .o from the .c file, but _now_ if we change the .h file... make will know that it has to rebuild these two files here. Neat, huh? Oh, and one last trick: see our .c.o suffix rule here? Let's delete it. And look at that: make(1) _still_ does the right thing! That's because just like make(1) has internal definitions for some variables such as CC, it _also_ knows about a few suffix rules. Building C projects is something that make(1) was developed for, so this rule -- to run "cc CFLAGS -c" for each .c file -- is something it has built in.

Summary

Okay, let's take a break and recap. All of these examples given here should be enough to give you an idea of what make(1) can do. And of course there are example Makefile included in the code samples on the course website, so make sure to play around with those on your own. As we've seen, make(1) is really just a "command generator". Well, not "just", but at its core, it really only runs the commands you tell it to run by writing a description into a "Makefile". but it is a bit more than that: it is able to sort out which commands to execute in which order based on the relationship of the files, which then allows you to avoid having to rebuild the entire project, to recompile every single source file when you only made a change to one or two files. That is, make(1) performs _selective_ rebuilds based on the internal dependency graph it determines based on your description as well as the last modified timestamp of the files in question. We've seen that we can simplify general logic using a number of rules and variables or "macros", and that make(1) has some of these built in. And while the examples we covered here are relatively simple, I hope that you can see that there is significant power in this tool. In fact, make(1) is really quite complex and has, not surprisingly, been implemented multiple times by the different Unix versions. And each version has extended the tool a bit and changed a few things here and there, so that we now have at least two primary flavors of make(1): BSD make and GNU make. These two differ quite a bit in how they assign and expand variables how they allow for the inclusion of other files; and how you can build flow control into the Makefile language itself. All of that should hopefully whet your appetite to not only use make(1) for your assignments in this class, but to learn a bit more and maybe seek a good online tutorial. Having a powerful build tool like make(1) at your fingertips is just one of the many great things about Unix as an integrated development environment. And yes, we still have a few more tools to cover in our next videos, so stay tuned and until the next time. Thanks for watching - cheers!