From DesigningPatterns

Contents 1 Background 2 General Characteristics 3 Scenarios 4 Strengths 5 Flaws 6 Books 7 Links

Background

Make is a tool for constructing software, including automating compilation, release staging, and installation. There are many variants, but the most widely used variant probably is gmake. Unless otherwise specified, all information below refers to gmake. The content below assumes basic familiarity with make.

General Characteristics

• make dependencies can be represented by a directed acyclic graph. make issues often arise from not giving make enough information to build a complete graph.
• make has two kinds of variables.
  • "Simply Expanded" variables have values that are calculated at assignment time. These variables are created by assigning with :=.
  • "Recursively Expanded" variables have values that are calculated when referenced (and so may have different values at different references, even though there have been no intervening assignments). These variables are created by assigning with =.
• Variables used in targets and preqrequisites (as well as variable assignments) are evaluated as make interprets the makefile (top down). Variables used in commands, on the other hand, are evaluated after make has finished interpreting the makefile and tries to execute the commands. Therefore, variables in commands always evaluate to the last value of the variable when make was interpreting the makefile.
• make executes in two phases.
  • In the first phase, make parses the makefile and creates a database of targets (and associated rules). Variable assignments are executed (although the values of recursively expanded variables only are calculated when referenced) and prerequisites are evaluated during this phase.
  • In the second phase, make builds its dependency graph and then attempts to build its target, walking from the target back through the dependency graph. Rules are executed during this phase (using variables evaluated during this phase).
• It is possible to have make automagically generate the required dependencies from C++ header files; it can leverage some special command line gcc options.
• By default make does no checking of variable or function names; if you use the wrong name, the expression just will evaluate to nothing (and make issues no complaint).
  • If gmake is given the --warn-undefined-variables flag, it will print a warning when any undefined variables are referenced.
• It is common to require some code to execute or some targets to be evaluated every time make is run. This can be achieved by creating a "phony" target that does not exist in the file system. Since the target does not exist, make will try to rebuild it. If make knows that this target is phony, it can optimize this process somewhat (no need to match the target against implicit rules, for example). This can be accomplished in gmake by listing the target as a prerequisite of .PHONY. Making the target .PHONY also makes the build more robust, because someone later creating the bogus target as a file will not change make's behavior.
• $(eval) allows code to be passed to the gmake interpreter, which is a useful code generation trick (you could $(eval) a block of code for every directory to be processed in a non-recursive build system, for instance).
  • $(eval) processes its code argument twice.
    • During the first processing phase, all variables (including arguments are expanded).
    • During the second processing phase, the code actually is interpreted.
    • It very often is useful to defer the evaluation of a variable from the first to the second phase (say that the $(eval) block evaluates a variable that itself will be defined when the block is interpreted during the second phase). This can be accomplished by prefixing the variable by $$ rather than $ (that is, the normal $ is escaped by another $$).
• Using $(eval) inside of $(eval) is dangerous, because any $(eval) code inside $(eval) code will be expanded and executed before the rest of the code. This could be thought of as a feature (the O'Reilly Make book claims it as such), but I think that this is a bug. This also breaks with conditionals. $$(eval) causes make to die, so that does not defer the evaluation of the code.
  • I posted this to the bug-make group and got a response (that multi-line function macros just can be expanded without $(eval) within an $(eval) expansion) here.
• Make offers a facility called VPATH for helping to locate targets and prerequisites. Essentially, make will search the directories contained in the VPATH variable for targets and prerequisites.
  • There are issues with using this to search for targets; see VPATH this for an explanation. Basically, make does not really fully support this.
  • gmake also offers a vpath directory which can be applied to a specific target or pattern.
• The gmake manual mentions two types of prerequisites, "normal" and "order-only".
  • Establishing a relationship between a target and a normal prerequisite means that the prerequisite must be built before the target and that the target is generated from the prerequisite.
  • Establishing a relationship between a target and an order-only prerequisite means that the prerequisite must be built before the target but does not mean that the target is generated from the prerequisite.
• Make offers a facility called secondary expansion. This facility (enabled by putting .SECONDEXPANSION in the makefile) means that rules will be evaluated twice, once during the processing of the makefile and once after the processing of the makefile before make's dependency graph is built. Variables can be evaluated in the first pass ($) or in the second pass ($$), similar to their usage in $(eval)).
  • Secondary expansion's killer feature is that automatic variables can be used in the prerequisite expression, which is not the case during the initial expansion. Thus, the prerequisite can be derived from the target in a more complicated fashion that any allowed by the % placeholder in the initial expansion (because % is expanded after the other variables in the expression, meaning that functions like subst cannot be called on it correctly).
• One way in which order-only prerequisites seem useful is if the target lives in a directory that may not yet exist. Making the directory a normal prerequisite does work (the directory gets created), but pollutes the prerequisite list for the target. This can be an issue if multiple files are aggregated into the target (say that the target is a tarball), and the rule to build the target runs a command over all prerequisites, such as this

%.tar.gz:
        $(TAR) -cz -f $@ $^

If the directory is a normal prerequisite, it will be included in $^, which will cause incorrect arguments to be passed to tar; $^ does not include order-only prerequisites, however, so making the directory an order-only prerequisite will make this example work perfectly. There is a problem with this technique, however. Consider the following makefile:

DERIVED_OBJ_DIR = _linux

$(DERIVED_OBJ_DIR)/%.o: %.cc
	g++ -o $@ $<

.PHONY: all
all: $(DERIVED_OBJ_DIR)/test.o

$(DERIVED_OBJ_DIR):
	mkdir -p $(DERIVED_OBJ_DIR)

$(DERIVED_OBJ_DIR)/test.o: | $(DERIVED_OBJ_DIR)

This makefile builds _linux/test.o from test.cc. In order for make to build the object file in a different directory than the source file, I needed to add a special implicit rule to build object files in _linux from source files in the current directory. Note that _linux/test.o also has an order-only prerequisite that ensures that _linux exists before the rule to build _linux/test.o is executed. Here is the same makefile, with the implicit rule and the order-only prerequisite for _linux/test.o commented out.

DERIVED_OBJ_DIR = _linux

#$(DERIVED_OBJ_DIR)/%.o: %.cc
#	g++ -o $@ $<

.PHONY: all
all: $(DERIVED_OBJ_DIR)/test.o

$(DERIVED_OBJ_DIR):
	mkdir -p $(DERIVED_OBJ_DIR)

#$(DERIVED_OBJ_DIR)/test.o: | $(DERIVED_OBJ_DIR)

Make now exits with this error:

gmake: *** No rule to make target `_linux/test.o', needed by `all'.  Stop.

since it does not know how to build _linux/test.o, which is proper behavior. If I uncomment out the order-only prequisite for _linux/test.o, however:

DERIVED_OBJ_DIR = _linux

#$(DERIVED_OBJ_DIR)/%.o: %.cc
#	g++ -o $@ $<

.PHONY: all
all: $(DERIVED_OBJ_DIR)/test.o

$(DERIVED_OBJ_DIR):
	mkdir -p $(DERIVED_OBJ_DIR)

$(DERIVED_OBJ_DIR)/test.o: | $(DERIVED_OBJ_DIR)

make does not build _linux/test.o but also does. not exit with an error, which is wrong. That is, the order-only prerequisite expression counts as a rule for make, and it no longer will check if a real rule exists for _linux/test.o. I consider this to be a big negative of order-only prerequisites used in this context, as it nullifies some of make's error checking (of which there isn't that much to begin with). I posted this issue to the bug-make list here.

• Using an order-only prerequisite in order to ensure the existence of directories also can cause issues if those directories are used in the prerequisites of a rule. For example,

%.o: ../%.cc

seems equivalent to

DERIVED_OBJ_DIR = _linux
$(DERIVED_OBJ_DIR)/%.o: %.cc

But the first won't work if _linux does not exist. The reason is that when make encounters a target without a directory (%.o, for instance, as opposed to $(DERIVED_OBJ_DIR)/%.o) it matches just the file name against the target pattern and creates the prerequisite by appending the target's directory to the prerequisite. Thus, in the first case, the prerequisite for _linux/test.o actually is _linux/../test.cc to make, which is invalid if _linux does not yet exist (and thus make skips the rule).

• A cool trick for getting make to execute an action while processing a makefile (usually, of course, make only executes actions after it has fully processed the makefile and built the dependency graph) is to assign the results of the $(shell) function to a dummy variable. The only flaw with this method is that it will be executed by make in dry run mode.

Scenarios

1. Make is happiest when it is working with files (both targets and prerequisites) in its current directory. This really is the use case for which it was built.
2. If make must work with files that are not in the current directory (a violation of Scenario 1), it still can be made very happy by all of the prerequisites for a given target living in the same directory as that target. In fact, with gmake at least, this situation just requires a bit more care than that under Scenario 1 (users must specify paths for targets and prerequisites, for instance; VPATH may be able to help make this a bit better).
3. If make has to build targets with prerequisites in different directories (a violation of Scenarios 1 + 2), it can be appeased by adding the prerequisite directory to VPATH and there being identical paths to a target from its current working directory and to the target's prerequisites from the VPATH directory. With yet more care, this can be made as usable as Scenario 2.
4. If make has to build targets from prerequisites and VPATH cannot be manipulated in such a way so as to satisfy Scenario 3, then you are in serious trouble. Make still can work for you, but leveraging implicit rules will range from tricky to impossible. If there is a set relationship between target and prerequisites (say that the prerequisites for a target always are in the directory containing the target's directory), then this relationship can be expressed in the prerequisite portion of the implicit rules (%.o : ../%.c, for instance); secondary expansion increases the number of situations in which such a solution can be deployed. Be very careful defining implicit rules for each target/prerequisite directory pair if using make non-recursively ($(TARGET_DIR)/%.o : $(SOURCE_DIR)/%.c), as a rule for a directory higher in the directory hierarchy might get triggered for one lower in the hierarchy. On the other hand, if make is being run recursively, there is no problem defining a set of implicit rules for each make invocation since there is no possibility of conflict between invocations.

Strengths

• gmake is universally available in the UNIX world.
• gmake has a number of features that make it more usable than other make variants.
• gmake is supported and has an excellent user community.
• gmake is very portable.
• Make is very general.
• Make is the most widely known software construction tool.

Flaws

• Make is very low level; in fact, it is just about the lowest level build tool imaginable. More recent build systems have tended:
  • Be targeted at specific problem spaces, providing explicit support for those spaces (Rake for Ruby, Ant for Java).
  • Contain higher level workflow concepts, such as testing goals. In make, by contrast, all such concepts must be built from the ground up.
• Some languages have a syntax that is a pleasure to use (Java, Ruby). Others have a very functional, no-nonsense syntax that gets the job done (C++, Pascal). Make has a wretched syntax that is some mutant offspring of shell scripting.
  • There is no concept of a boolean; make conditionals are based on:
    • Whether (ifdef) or not (ifndef) a variable is defined
    • Whether (ifeq) or not (ifneq) two strings are equal
  • It is not easy to chain expressions in conditions (if((expr1) && (expr2)) for example).
  • The looping construct is an extremely primitive text replication function (foreach).
  • There are no proper functions; make "functions" essentially are preprocessor macros with no argument checking.
  • The function call syntax is extremely weird if arguments are involved (in this case, the function name and arguments all are passed as arguments to the $(call)) function.
    • $(call myfunc,.rb,.rb.validated) actually is quite different than $(call myfunc, .rb, .rb.validated). The whitespace between the commas and the arguments is significant in the second example; with this whitespace, the macro actually will expand with " .rb" and " .rb.validated"
  • Make does not check for undefined variables (and thus functions) by default (although I mention a flag to activate this for gmake above); this means that there is no protection against typos (which are extremely painful to debug).
  • Although $(eval) is wonderful functionality, it's implementation leaves a lot to be desired.
    • Being evaluated twice leads to some very tricky issues (only prefixing a variable expansion with a single $, rather than $$, might lead to its being evaluated too early).
    • Functions that normally need to be called with $(eval) cannot be called with $(eval) within a function that itself was being called with $(eval), at least without some unpleasant implications (see the discussion above).
• Make's underlying behavior is not extensible or customizable (dependency relationships, for instance, cannot be customized).
• No build auditing (and thus no "winkin" equivalent).
• A good build system under make might nullify one of make's key advantages, that most UNIX developers are familiar with it. In particular, a good build system might require the use of unorthodox syntax (including $(eval)), and almost becomes its own domain specific language.
• It's unclear how many improvements will be made to gmake going forward; new versions only seem to be released every couple of years.

Books

• This is an excellent book on gmake. It covers a number of useful topics, including recursive make, non-recursive make, separating binaries from sources, and $(eval). It is available on O'Reilly Safari.

Links

• This is a seminal discussion of how and why to avoid recursive build systems.
• The gmake manual, a very complete guide to gmake.
• A gmake page created by the gmake maintainer. This includes several excellent articles, including:
  • multiple architecture make guide
  • VPATH discussion
  • Rules for writing makefiles