The ubiquitous make program's "killer" feature over a shell script is that it incrementally re-builds its outputs, by only building if an input file has been updated. Unfortunately, Make does not make it easy to do this correctly, leading to mysterious errors that go away when you do a clean build, or slow builds that rebuilding everything unnecessarily. This post walks through how to get it nearly completely right. However if possible, I suggest using the Ninja build system instead, since it was designed to support this in a simple and straightforward way, and is much faster to boot. Sadly in our case, we are already using Make, so I needed to understand how to use it correctly.
Here is a trivial Makefile that builds a C executable. I've posted the code on Github if you want to play along at home.
exe: exe.c [\t] gcc -o $@ $<
When you run make, it will build exe. If you edit exe.c then run make, it will get re-built correctly. However, if I update an header file hello.h, make incorrectly says `exe' is up to date. We have to tell make that exe depends on hello.h, in addition to exe.c.
We can update the Makefile to explicitly include the dependency, by listing it in the dependency list for the output exe (complete example on github):
exe: exe.c hello.h [\t] gcc -o $@ $<
Now, if either exe.c or hello.h are edited, exe gets rebuilt. However, this involves some duplication of information between the source code and the Makefile, and manual work to keep them in sync. What happens if hello.h is updated to include some additional header files? We need to manually update every Makefile rule that uses hello.h to list the additional files. Thankfully, we can get make to do this automatically.
C compilers can list all the header files that are included when they compile a file. For GCC and Clang, this gets output as a Makefile fragment so we can easily include it in our Makefile. We can instruct GCC to output this information by adding the -MMD flag, and use -include to include the output in our Makefile (complete example on github):
exe: exe.c gcc -MMD -MF exe.d -o $@ $< -include exe.dThe file
exe.d itself contains the following after compiling exe:
exe: exe.c hello.h other.h
This is a Makefile that specifies additional dependencies for the target exe. When Make encounters the same target multiple times, it merges the dependency lists. This means that whenever exe.c or any of the header files it includes are updated, exe will be rebuilt. As a side effect, it will also update the dependency list. This now only has one tiny flaw: if we remove or rename a header file that was included we get the following friendly error:
make: *** No rule to make target `other.h', needed by `exe'. Stop.
The generated dependency list tells make that exe requires other.h to be built. Since other.h does not exist, make can't build exe. We need to instead tell make that if the header files do not exist, try to rebuild the program anyway. This allows GCC to check if the missing file is an error (e.g. if I deleted other.h but forgot to remove the #include), or will update the dependency list if it no longer is needed. A make rule without any targets, commands, or dependencies does exactly what we want: Make considers this target to be "updated," causing anything that depends on it to be rebuilt. The really good news is that GCC has a -MP flag that outputs Makefile fragments in this format. The main Makefile becomes (complete example on github):
exe: exe.c gcc -MMD -MP -MF exe.d -o $@ $< -include exe.d
The generated exe.d Makefile fragment now looks like:
exe: exe.c hello.h other.h hello.h: other.h:
This solution is nearly perfect. There are only two minor flaws. The first is that if the tool itself is updated, the output files should be rebuilt. In the case of scripts or tools that are being actively developed, you should probably include them in the Makefile's dependency list, using the same techniques described here. In the case of compilers or other system tools that are pretty stable, I don't think its worth the effort. The second flaw is that if the command line flags are changed, the files will not be rebuilt. This seems pretty tricky to solve with make. If you want to solve this, I recommend switching to Ninja, since it does this correctly by default.
This technique can be used any program that takes input, transforms it, and writes output. For example, I recently added rules using lessc, the compiler for the Less CSS pre-processor to our Makefile. Less files can include other files, just like C programs, so we had problems with the output not getting rebuilt correctly. Thankfully, lessc include a --depends flag to output dependencies, so I wrote a Makefile that looks like this:
output.css: input.less lessc --depends $< > $@.d less $< > $@ -include input.css.d