ref: 248c637d25154f762a7d675662c405a169594ca0
dir: /doc/mk.ms/
.TL Maintaining Files on Plan 9 with Mk .AU Andrew G. Hume andrew@research.att.com Bob Flandrena bobf@plan9.bell-labs.com .AB .PP .CW Mk is a tool for describing and maintaining dependencies between files. It is similar to the UNIX program .CW make , but provides several new capabilities. .CW Mk\fR'\fPs flexible rule specifications, implied dependency derivation, and parallel execution of maintenance actions are well-suited to the Plan 9 environment. Almost all Plan 9 maintenance procedures are automated using .CW mk . .AE .NH 1 Introduction .PP .CW Mk performs the same task as .CW make [Feld79] but is different in many ways. This document describes .CW mk\fR'\fPs capabilities and discusses how they are used in Plan 9. There are occasional references to differences between .CW mk and .CW make , but an appendix summarizes the details. .PP .CW Mk works well in the Plan 9 environment. It supports program construction in a heterogeneous environment and exploits the power of multiprocessors by executing maintenance actions in parallel. It interacts seamlessly with the Plan 9 command interpreter .CW rc and accepts pattern-based dependency specifications that are not limited to describing rules for program construction. The result is a tool that is flexible enough to perform many maintenance tasks including database maintenance, hardware design, and document production. .PP This document begins by discussing the syntax of the input file, the pattern matching capabilities, and the special rules for maintaining archives. A brief description of .CW mk\fR'\fPs algorithm for deriving dependencies is followed by a discussion of the conventions used to resolve ambiguous specifications. The final sections describe parallel execution and special features. .PP An earlier paper [Hume87] provides a detailed discussion of .CW mk\fR'\fPs design. .NH 1 The \f(CWMkfile\fP .PP .CW Mk reads a file describing relationships among files and executes commands to bring the files up to date. The specification file, called a .CW mkfile , contains three types of statements: assignments, includes, and rules. Assignment and include statements are similar to those in C. Rules specify dependencies between a .I target and its .I prerequisites . When the target and prerequisites are files, their modification times determine if they are out of date. Rules often contain a .I recipe , an .I rc (1) script that produces the target from the prerequisites. .PP This simple .CW mkfile produces an executable from a C source file: .P1 CC=pcc f1: f1.c $CC -o f1 f1.c .P2 The first line assigns the name of the portable ANSI/POSIX compiler to the .CW mk variable .CW CC ; subsequent references of the form .CW $CC select this compiler. The only rule specifies a dependence between the target file .CW f1 and the prerequisite file .CW f1.c . If the target does not exist or if the prerequisite has been modified more recently than the target, .CW mk passes the recipe to .CW rc for execution. Here, .CW f1.c is compiled and loaded to produce .CW f1 . .PP The native Plan 9 environment requires executables for all architectures, not only the current one. The Plan 9 version of the same .CW mkfile looks like: .P1 </$objtype/mkfile CFLAGS=-w f1: f1.$O $LD $LDFLAGS -o f1 f1.$O f1.$O: f1.c $CC $CFLAGS f1.c .P2 The first line is an include statement that replaces itself with the contents of the file .CW /$objtype/mkfile . The variable .CW $objtype is inherited from the environment and contains the name of the target architecture. The prototype .CW mkfile for that architecture defines architecture-specific variables: .CW CC and .CW LD are the names of the compiler and loader, .CW O is the code character of the architecture. The rules compile the source file into an object file and invoke the loader to produce .CW f1 . Invoking .CW mk from the command line as follows .P1 % objtype=sparc mk kc -w f1.c kl $LDFLAGS -o f1 f1.k % .P2 produces the .CW sparc executable of program .CW f1 regardless of the current architecture type. .PP We can extend the .CW mkfile to build two programs: .P1 </$objtype/mkfile CFLAGS=-w ALL=f1 f2 all:V: $ALL f1: f1.$O $LD $LDFLAGS -o f1 f1.$O f1.$O: f1.c $CC $CFLAGS f1.c f2: f2.$O $LD $LDFLAGS -o f2 f2.$O f2.$O: f2.c $CC $CFLAGS f2.c .P2 The target .CW all , modified by the .I attribute .CW V , builds both programs. The attribute identifies .CW all as a dummy target that is not related to a file of the same name; its precise effect is explained later. This example describes cascading dependencies: the first target depends on another which depends on a third and so on. Here, individual rules build each program; later we'll see how to do this with a general rule. .NH 1 Variables and the environment .PP .CW Mk does not distinguish between its variables and .CW rc variables in the environment. When .CW mk starts, it imports each environment variable into a .CW mk variable of the same name. Before executing a recipe, .CW mk exports all variables, including those inherited from the environment, to the environment in which .CW rc executes the recipe. .PP There are several ways for a variable to take a value. It can be set with an assignment statement, inherited from the environment, or specified on the command line. .CW Mk also maintains several internal variables; these variables are described in .I mk (1). Assignments have the following decreasing order of precedence: .LP .in .7i 1) Command line assignment .br 2) Assignment statement .br 3) Imported from the environment .br 4) Implicitly set by \f(CWmk\fP .in 0 .LP For example, a command line assignment overrides a value imported from the environment. .PP All variable values are strings. They can be used for pattern matching and comparison but not for arithmetic. A .I list is a string containing several values separated by white space. Each member is handled individually during pattern matching, target selection, and prerequisite evaluation. .PP A .I namelist is a list produced by transforming the members of an existing list. The transform applies a pattern to each member, replacing each matched string with a new string, much as in the substitute command in .I sam (1) or .I ed (1). The syntax is .P1 ${\fIvar\fP:A%B=C%D} .P2 where .I var is a variable. The pattern .CW A%B matches a member beginning with the string .I A and ending with the string .I B with any string in between; it behaves like the regular expression .CW A.*B . When a member of the .I var list matches this pattern, the string .I C replaces .I A , .I D replaces .I B , and the matched string replaces itself. Any of .I A , .I B , .I C , or .I D may be the empty string. In effect, a namelist is generated by applying the .I ed (1) substitute command .P1 s/\fIA\fP(.*)\fIB\fP/\fIC\fP\e1\fID\fP/ .P2 to each member of a variable. .PP Namelists are useful for generating a list based on a predictable transformation. For example, .P1 SRC=a.c b.c c.c OBJ=${SRC:%.c=%.v} .P2 assigns the list \f(CW(a.v b.v c.v)\fP to .CW OBJ . A namelist may be used anywhere a variable is allowed except in a recipe. .PP Command output is assigned to a variable using the normal .CW rc syntax: .P1 var=`{rc command} .P2 The command executes in an environment populated with previously assigned variables, including those inherited from .CW mk\fR'\fPs execution environment. The command may be arbitrarily complex; for example, .P1 TARG=`{ls -d *.[cyl] | sed 's/..$//'} .P2 assigns a list of the C, yacc, and Alef source files in the current directory, stripped of their suffix, to the variable .CW TARG . .NH 1 The include statement .PP The include statement replaces itself with the contents of a file. It is functionally similar to the C .CW #include statement but uses a different syntax: .P1 <\fIfilename\fP .P2 The contents of the file are evaluated as they are read. An include statement may be used anywhere except in a recipe. .PP Unlike .CW make , .CW mk has no built-in rules. Instead, the include statement allows generic rules to be imported from a prototype .CW mkfile ; most Plan 9 .CW mkfiles use this approach [Flan95]. .NH 1 Rules .PP A rule has four elements: targets, prerequisites, attributes, and a recipe. It has the form: .P1 \fItargets\fP:\fIattributes\fP:\fIprerequisites\fP \fIrecipe\fP .P2 The first line, containing the targets, attributes, and prerequisites is the .I "rule header" ; it must begin at the left margin. The recipe contains zero or more lines, each of which begins with white space. One or more targets must be specified but the attributes, prerequisites, and recipe are optional. A rule specifies a dependency between the targets and the prerequisites; the recipe brings the targets up to date with the prerequisites. Attributes modify .CW mk\fR'\fPs evaluation of a rule. .PP Normally the target is a file that depends on one or more prerequisite files. .CW Mk compares the modification times of each target and each prerequisite; a target is considered out of date when it does not exist or when a prerequisite has been modified more recently. When a target is out of date, .CW mk executes the recipe to bring it up to date. When the recipe completes, the modification time of the target is checked and used in later dependency evaluations. If the recipe does not update the target, evaluation continues with the out of date target. .PP A prerequisite of one rule may be the target of another. When this happens, the rules cascade to define a multi-step procedure. For example, an executable target depends on prerequisite object files, each of which is a target in a rule with a C source file as the prerequisite. .CW Mk follows a chain of dependencies until it encounters a prerequisite that is not a target of another rule or it finds a target that is up to date. It then executes the recipes in reverse order to produce the desired target. .PP The rule header is evaluated when the rule is read. Variables are replaced by their values, namelists are generated, and commands are replaced by their output at this time. .PP Most attributes modify .CW mk\fR'\fPs evaluation of a rule. An attribute is usually a single letter but some are more complicated. This paper only discusses commonly used attributes; see .I mk (1) for a complete list. .PP The .CW V attribute identifies a .I virtual target; that is, a target that is not a file. For example, .P1 clean:V: rm *.$O $O.out .P2 removes executables and compiler intermediate files. The target is virtual because it does not refer to a file named .CW clean . Without the attribute, the recipe would not be executed if a file named .CW clean existed. The .CW Q silences the default printing of a recipe before it is executed. It is useful when the output of a recipe is similar to the recipe: .P1 default:QV: echo 'No default target; use mk all or mk install' .P2 .PP A recipe is an .CW rc script. The recipe is optional; when it is missing, the rule is handled specially, as described later. Unlike .CW make , .CW mk executes recipes without interpretation. After stripping the first white space character from each line it passes the entire recipe to .CW rc on standard input. Since .CW mk does not interpret a recipe, escape conventions are exactly those of .CW rc . Scripts for .CW awk and .CW sed commands can be embedded just as they would be specified from the command line. .CW Mk invokes .CW rc with the .CW -e flag, which causes .CW rc to stop if any command in the recipe exits with a non-zero status; the .CW E attribute overrides this behavior and allows .CW rc to continue executing in the face of errors. Before a recipe is executed, variables are exported to the environment where .CW rc can see them. Recipe commands may not read from standard input because .CW mk uses it internally. .PP References to a variable can yield different values depending on the location of the reference in the .CW mkfile . .CW Mk resolves variable references in assignment statements and rule headers when the statement is read. Variable references in recipes are evaluated by .CW rc when the recipe is executed; this happens after the entire .CW mkfile has been read. The value of a variable in a recipe is the last value assigned in the file. For example, .P1 STRING=all all:VQ: echo $STRING STRING=none .P2 produces the message .CW none . A variable assignment in a recipe does not affect the value of the variable in the .CW mkfile for two reasons. First, .CW mk does not import values from the environment when a recipe completes; one recipe cannot pass a value through the environment to another recipe. Second, no recipe is executed until .CW mk has completed its evaluation, so even if a variable were changed, it would not affect the dependency evaluation. .NH 1 Metarules .PP A .I metarule is a rule based on a pattern. The pattern selects a class of target and identifies related prerequisites. .CW Mk metarules may select targets and prerequisites based on any criterion that can be described by a pattern, not just the suffix transformations associated with program construction. .PP .CW Mk has two types of patterns: .I intrinsic patterns or regular expressions conforming to the syntax of .I regexp (6). The intrinsic patterns are shorthand for common regular expressions. The intrinsic pattern .CW % matches one or more of anything; it is equivalent to the regular expression .CW `.+' . The other intrinsic pattern, .CW & , matches one or more of any characters except \f(CW`/'\fP and \f(CW`.'\fP. It matches a portion of a path and is equivalent to the regular expression .CW `[^./]+' . An intrinsic pattern in a prerequisite references the string matched by the same intrinsic pattern in the target. For example, the rule .P1 %.v: %.c .P2 says that a file ending in .CW .v depends on a file of the same name with a .CW .c suffix: .CW foo.v depends on .CW foo.c , .CW bar.v depends on .CW bar.c , and so on. The string matched by an intrinsic pattern in the target is supplied to the recipe in the variable .CW $stem . Thus the rule .P1 %.$O: %.c $CC $CFLAGS $stem.c .P2 creates an object file for the target architecture from a similarly named C source file. If several object files are out of date, the rule is applied repeatedly and .CW $stem refers to each file in turn. Since there is only one .CW stem variable, there can only be one .CW % or .CW & pattern in a target; the pattern .CW %-%.c is illegal. .PP Metarules simplify the .CW mkfile for building programs .CW f1 and .CW f2 : .P1 </$objtype/mkfile CFLAGS=-w ALL=f1 f2 all:V: $ALL %: %.$O $LD -o $target $prereq %.$O: %.c $CC $CFLAGS $stem.c clean:V: rm -f $ALL *.[$OS] .P2 (The variable .CW $OS is a list of code characters for all architectures.) Here, metarules specify compile and load steps for all files. The loader rule relies on two internal variables set by .CW mk during evaluation of the rule: .CW $target is the name of the target and .CW $prereq the name of all prerequisites. Metarules allow this .CW mkfile to be easily extended; a new program is supported by adding its name to the third line. .PP A regular expression metarule must have an .CW R attribute. Prerequisites may reference matching substrings in the target using the form .CW \e\fIn\fP where .I n is a digit from 1 to 9 specifying the .I n th parenthesized sub-expression. In a recipe, .CW $stem\fIn\fP is the equivalent reference. For example, a compile rule could be specified using regular expressions: .P1 (.+)\e.$O:R: \e1.c $CC $CFLAGS $stem1.c .P2 Here, .CW \e1 and .CW $stem1 refer to the name of the target object file without the suffix. The variable .CW $stem associated with an intrinsic pattern is undefined in a regular expression metarule. .NH 1 Archives .PP .CW Mk provides a special mechanism for maintaining an archive. An archive member is referenced using the form .CW \fIlib\fP(\fIfile\fP) where .I lib is the name of the archive and .I file is the name of the member. Two rules define the dependency between an object file and its membership in an archive: .P1 $LIB(foo.v):N: foo.v $LIB: $LIB(foo.v) ar rv $LIB foo.v .P2 The first rule establishes a dependency between the archive member and the object file. Normally, .CW mk detects an error when a target does not exist and the rule contains no recipe; the .CW N attribute overrides this behavior because the subsequent rule updates the member. The second rule establishes the dependency between the member and the archive; its recipe inserts the member into the archive. This two-step specification allows the archive to represent the state of its members. Other rules can then specify the archive as a prerequisite instead of listing each member. .PP A metarule generalizes library maintenance: .P1 LIB=lib.a OBJS=etoa.$O atoe.$O ebcdic.$O $LIB(%):N: % $LIB: ${OBJS:%=$LIB(%)} ar rv $LIB $OBJS .P2 The namelist prerequisite of the .CW $LIB target generates archive member names for each object file name; for example, .CW etoa.$O becomes .CW lib.a(etoa.$O) . This formulation always updates all members. This is acceptable for a small archive, but may be slow for a big one. The rule .P1 $LIB: ${OBJS:%=$LIB(%)} ar rv $LIB `{membername $newprereq} .P2 only updates out of date object files. The internal variable .CW $newprereq contains the names of the out of date prerequisites. The .CW rc script .CW membername transforms an archive member specification into a file name: it translates .CW lib.a(etoa.$O) into .CW etoa.$O . .PP The .CW mkfile .P1 </$objtype/mkfile LIB=lib.a OBJS=etoa.$O atoe.$O ebcdic.$O prog: main.$O $LIB $LD -o $target $prereq $LIB(%):N: % $LIB: ${OBJS:%=$LIB(%)} ar rv $LIB $OBJS .P2 builds a program by loading it with a library. .NH 1 Evaluation algorithm .PP For each target of interest, .CW mk uses the rules in a .CW mkfile to build a data structure called a dependency graph. The nodes of the graph represent targets and prerequisites; a directed arc from one node to another indicates that the file associated with the first node depends on the file associated with the second. When the .CW mkfile has been completely read, the graph is analyzed. In the first step, implied dependencies are resolved by computing the .I "transitive closure" of the graph. This calculation extends the graph to include all targets that are potentially derivable from the rules in the .CW mkfile . Next the graph is checked for cycles; .CW make accepts cyclic dependencies, but .CW mk does not allow them. Subsequent steps prune subgraphs that are irrelevant for producing the desired target and verify that there is only one way to build it. The recipes associated with the nodes on the longest path between the target and an out of date prerequisite are then executed in reverse order. .PP The transitive closure calculation is sensitive to metarules; the patterns often select many potential targets and cause the graph to grow rapidly. Fortunately, dependencies associated with the desired target usually form a small part of the graph, so, after pruning, analysis is tractable. For example, the rules .P1 %: x.% recipe1 x.%: %.k recipe2 %.k: %.f recipe3 .P2 produce a graph with four nodes for each file in the current directory. If the desired target is .CW foo , .CW mk detects the dependency between it and the original file .CW foo.f through intermediate dependencies on .CW foo.k and .CW x.foo . Nodes associated with other files are deleted during pruning because they are irrelevant to the production of .CW foo . .PP .CW Mk avoids infinite cycles by evaluating each metarule once. Thus, the rule .P1 %: %.z cp $prereq $prereq.z .P2 copies the prerequisite file once. .NH 1 Conventions for evaluating rules .PP There must be only one way to build each target. However, during evaluation metarule patterns often select potential targets that conflict with the targets of other rules. .CW Mk uses several conventions to resolve ambiguities and to select the proper dependencies. .PP When a target selects more than one rule, .CW mk chooses a regular rule over a metarule. For example, the .CW mkfile .P1 </$objtype/mkfile CFLAGS=-w FILES=f1.$O f2.$O f3.$O prog: $FILES $LD -o $target $prereq %.$O: %.c $CC $CFLAGS $stem.c f2.$O: f2.c $CC f2.c .P2 contains two rules that could build .CW f2.$O . .CW Mk selects the last rule because its target, .CW f2.$O , is explicitly specified, while the .CW %.$O rule is a metarule. In effect, the explicit rule for .CW f2.$O overrides the general rule for building object files from C source files. .PP When a rule has a target and prerequisites but no recipe, those prerequisites are added to all other rules with recipes that have the same target. All prerequisites, regardless of where they were specified, are available in the recipe in variable .CW $prereq . For example, in .P1 </$objtype/mkfile CFLAGS=-w FILES=f1.$O f2.$O f3.$O prog: $FILES $LD -o $target $prereq %.$O: hdr.h %.$O: %.c $CC $CFLAGS $stem.c .P2 the second rule adds .CW hdr.h as a prerequisite of the compile metarule; an object file produced from a C source file depends on .CW hdr.h as well as the source file. Notice that the recipe of the compile rule uses .CW $stem.c instead of .CW $prereq because the latter specification would attempt to compile .CW hdr.h . .PP When a target is virtual and there is no other rule with the same target, .CW mk evaluates each prerequisite. For example, adding the rule .P1 all:V: prog .P2 to the preceding example builds the executable when either .CW prog or .CW all is the specified target. In effect, the .CW all target is an alias for .CW prog . .PP When two rules have identical rule headers and both have recipes, the later rule replaces the former one. For example, if a file named .CW mkrules contains .P1 $O.out: $OFILES $LD $LFLAGS $OFILES %.$O: %.c $CC $CFLAGS $stem.c .P2 the .CW mkfile .P1 OFILES=f1.$O f2.$O f3.$O <mkrules $O.out: $OFILES $LD $LFLAGS -l $OFILES -lbio -lc .P2 overrides the general loader rule with a special rule using a non-standard library search sequence. A rule is neutralized by overriding it with a rule with a null recipe: .P1 <mkrules $O.out:Q: $OFILES ; .P2 The .CW Q attribute suppresses the printing of the semicolon. .PP When a rule has no prerequisites, the recipe is executed only when the target does not exist. For example, .P1 marker: touch $target .P2 defines a rule to manage a marker file. If the file exists, it is considered up to date regardless of its modification time. When a virtual target has no prerequisites the recipe is always executed. The .CW clean rule is of this type: .P1 clean:V: rm -f [$OS].out *.[$OS] .P2 When a rule without prerequisites has multiple targets, the extra targets are aliases for the rule. For example, in .P1 clean tidy nuke:V: rm -f [$OS].out *.[$OS] .P2 the rule can be invoked by any of three names. The first rule in a .CW mkfile is handled specially: when .CW mk is invoked without a command line target all targets of the first non-metarule are built. When that rule has multiple targets, the recipe is executed once for each target; normally, the recipe of a rule with multiple targets is only executed once. .PP A rule applies to a target only when its prerequisites exist or can be derived. More than one rule may have the same target as long as only one rule with a recipe remains applicable after the dependency evaluation completes. For example, consider a program built from C and assembler source files. Two rules produce object files: .P1 %.$O: %.c $CC $CFLAGS $stem.c %.$O: %.s $AS $AFLAGS $stem.s .P2 As long as there are not two source files with names like .CW \fIfoo\fP.c and .CW \fIfoo\fP.s , .CW mk can unambiguously select the proper rule. If both files exist, there are ambiguous rules to produce .CW \fIfoo\fP.$O , and .CW mk exits with an error message. .PP In Plan 9, many programs consist of portable code stored in one directory and architecture-specific source stored in another. For example, the .CW mkfile .P1 </$objtype/mkfile CFLAGS=-w FILES=f1.$O f2.$O f3.$O f3.$O prog: $FILES $LD -o $target $prereq %.$O: %.$c $CC $CFLAGS $stem.c %.$O: ../port/%.c $CC $CFLAGS ../port/$stem.c .P2 builds the program named .CW prog using portable code in directory .CW ../port and architecture-specific code in the current directory. As long as the names of the C source files in .CW ../port do not conflict with the names of files in the current directory, .CW mk selects the appropriate rule. If like-named files exist in both directories, the specification is ambiguous and results in an error. An explicit target resolves the ambiguity. For example, adding the rule .P1 f2.$O: f2.c $CC $CFLAGS $f2.c .P2 to the previous .CW mkfile uses the architecture-specific version of .CW f2.c instead of the portable one. Here, the explicit rule documents which of the like-named source files is used to build the program. .PP .CW Mk\fR'\fP s heuristics can produce unintended results when rules are not carefully specified. For example, the rules that build object files from C or assembler source files .P1 %.$O: %.c $CC $CFLAGS $stem.c %.$O: %.s $AS $AFLAGS $stem.s .P2 illustrate a subtle pratfall. Adding a header file dependency to the compile rule .P1 %.$O: %.c hdr.h $CC $CFLAGS $stem.c .P2 produces the error message .P1 .CW "don't know how to make '\fIfile\fP.c'" .P2 when \fIfile\fP.s is an assembler source file. This occurs because .CW \fIfile\fP.s satisfies the assemble rule and .CW hdr.h satisfies the compile rule, so either rule can potentially produce the target. When a prerequisite exists or can be derived, all other prerequisites in that rule header must exist or be derivable; here, the existence of .CW hdr.h forces the evaluation of a C source file. Specifying the dependencies in different rules avoids this interpretation: .P1 %.$O: hdr.h %.$O: %.c $CC $CFLAGS $stem.c .P2 Although .CW hdr.h is an additional prerequisite of the compile rule, the two rules are evaluated independently and the existence of the C source file is not linked to the existence of the header file. However, this specification describes a different dependency. Originally, only object files derived from C files depended on .CW hdr.h ; now all object files, including those built from assembler source, depend on the header file. .PP Metarule patterns should be as restrictive as possible to prevent conflicts with other rules. Consider the .CW mkfile .P1 </$objtype/mkfile BIN=/$objtype/bin PROG=foo install:V: $BIN/$PROG %: %.c $CC $stem.c $LD -o $target $stem.$O $BIN/%: % mv $stem $target .P2 The first target builds an executable in the local directory; the second installs it in the directory of executables for the architecture. Invoking .CW mk with the .CW install target produces: .P1 0 mk: ambiguous recipes for /mips/bin/foo: /mips/bin/foo <-(mkfile:8)- /mips/bin/foo.c <-(mkfile:12)- foo.c /mips/bin/foo <-(mkfile:12)- foo <-(mkfile:8)- foo.c .P2 The prerequisite of the .CW install rule, .CW $BIN/$PROG , matches both metarules because the .CW % pattern matches everything. The .CW & pattern restricts the compile rule to files in the current directory and avoids the conflict: .P1 &: &.c $CC $stem.c $LD -o $target $stem.$O .P2 .NH 1 Missing intermediates .PP .CW Mk does not build a missing intermediate file if a target is up to date with the prerequisites of the intermediate. For example, when an executable is up to date with its source file, .CW mk does not compile the source to create a missing object file. The evaluation only applies when a target is considered up to date by pretending that the intermediate exists. Thus, it does not apply when the intermediate is a command line target or when it has no prerequisites. .PP This capability is useful for maintaining archives. We can modify the archive update recipe to remove object files after they are archived: .P1 $LIB(%):N: % $LIB: ${OBJS:%=$LIB(%)} names=`{membername $newprereq} ar rv $LIB $names rm -f $names .P2 A subsequent .CW mk does not remake the object files as long as the members of the archive remain up to date with the source files. The .CW -i command line option overrides this behavior and causes all intermediates to be built. .NH 1 Alternative out-of-date determination .PP Sometimes the modification time is not useful for deciding when a target and prerequisite are out of date. The .CW P attribute replaces the default mechanism with the result of a command. The command immediately follows the attribute and is repeatedly executed with each target and each prerequisite as its arguments; if its exit status is non-zero, they are considered out of date and the recipe is executed. Consider the .CW mkfile .P1 foo.ref:Pcmp -s: foo cp $prereq $target .P2 The command .P1 cmp -s foo.ref foo .P2 is executed and if .CW foo.ref differs from .CW foo , the latter file is copied to the former. .NH 1 Parallel processing .PP When possible, .CW mk executes recipes in parallel. The variable .CW $NPROC specifies the maximum number of simultaneously executing recipes. Normally it is imported from the environment, where the system has set it to the number of available processors. It can be decreased by assigning a new value and can be set to 1 to force single-threaded recipe execution. This is necessary when several targets access a common resource such as a status file or data base. When there is no dependency between targets, .CW mk assumes the recipes can be executed concurrently. Normally, this allows multiple prerequisites to be built simultaneously; for example, the object file prerequisites of a load rule can be produced by compiling the source files in parallel. .CW Mk does not define the order of execution of independent recipes. When the prerequisites of a rule are not independent, the dependencies between them should be specified in a rule or the .CW mkfile should be single-threaded. For example, the archive update rules .P1 $LIB(%):N: % $LIB: ${OBJS:%=$LIB(%)} ar rv $LIB `{membername $newprereq} .P2 compile source files in parallel but update all members of the archive at once. It is a mistake to merge the two rules .P1 $LIB(%): % ar rv $LIB $stem .P2 because an .CW ar command is executed for every member of the library. Not only is this inefficient, but the archive is updated in parallel, making interference likely. .PP The .CW $nproc environment variable contains a number associated with the processor executing a recipe. It can be used to create unique names when the recipe may be executing simultaneously on several processors. Other maintenance tools provide mechanisms to control recipe scheduling explicitly [Cmel86], but .CW mk\fR'\fPs general rules are sufficient for all but the most unusual cases. .NH 1 Deleting target files on errors .PP The .CW D attribute causes .CW mk to remove the target file when a recipe terminates prematurely. The error message describing the termination condition warns of the deletion. A partially built file is doubly dangerous: it is not only wrong, but is also considered to be up to date so a subsequent .CW mk will not rebuild it. For example, .P1 pic.out:D: mk.ms pic $prereq | tbl | troff -ms > $target .P2 produces the message .P1 .CW "mk: pic mk.ms | ... : exit status=rc 685: deleting 'pic.out'" .P2 if any program in the recipe exits with an error status. .NH 1 Unspecified dependencies .PP The .CW -w command line flag forces the files following the flag to be treated as if they were just modified. We can use this flag with a command that selects files to force a build based on the selection criterion. For example, if the declaration of a global variable named .I var is changed in a header file, all source files that reference it can be rebuilt with the command .P1 $ mk -w`{grep -l \fIvar\fP *.[cyl]} .P2 .NH 1 Conclusion .PP There are many programs related to .CW make , each choosing a different balance between specialization and generality. .CW Mk emphasizes generality but allows customization through its pattern specifications and include facilities. .PP Plan 9 presents a difficult problem, particularly because of its heterogeneous collection of architectures and languages. .CW Mk\fR'\fPs flexible specification language and simple interaction with .CW rc work well in this environment. .PP As a result, Plan 9 relies on .CW mk to automate almost all maintenance. Tasks as diverse as updating the network data base, producing the manual, or building a release are expressed as .CW mk procedures. .NH 1 References .LP [Cmel86] R. F. Cmelik, ``Concurrent Make: A Distributed Program in Concurrent C'', AT&T Bell Laboratories Technical Report, 1986. .LP [Feld79] S. I. Feldman, ``Make \(em a program for maintaining computer programs'', .I Software Practice & Experience , .R 1979 Vol 9 #4, pp. 255-266. .LP [Flan95] Bob Flandrena, ``Plan 9 Mkfiles'', this volume. .LP [Hume87] A. G. Hume, ``Mk: A Successor to Make'', .I USENIX Summer Conf. Proc., .R Phoenix, Az. .NH 1 Appendix: Differences between .CW make and .CW mk .PP The differences between .CW mk and .CW make are: .IP \(bu 3n .CW Make builds targets when it needs them, allowing systematic use of side effects. .CW Mk constructs the entire dependency graph before building any target. .IP \(bu .CW Make supports suffix rules and .CW % metarules. .CW Mk supports .CW % and regular expression metarules. (Older versions of .CW make support only suffix rules.) .IP \(bu .CW Mk performs transitive closure on metarules, .CW make does not. .IP \(bu .CW Make supports cyclic dependencies, .CW mk does not. .IP \(bu .CW Make evaluates recipes one line at a time, replacing variables by their values and executing some commands internally. .CW Mk passes the entire recipe to the shell without interpretation or internal execution. .IP \(bu .CW Make supports parallel execution of single-line recipes when building the prerequisites for specified targets. .CW Mk supports parallel execution of all recipes. (Older versions of .CW make did not support parallel execution.) .IP \(bu .CW Make uses special targets (beginning with a period) to indicate special processing. .CW Mk uses attributes to modify rule evaluation. .IP \(bu .CW Mk supports virtual targets that are independent of the file system. .IP \(bu .CW Mk allows non-standard out-of-date determination, .CW make does not. .PP It is usually easy to convert a .CW makefile to or from an equivalent .CW mkfile .