QEF home page
A Comparison of QEF and Make

Given that many people think of make as the standard software construction tool, we feel that it's useful to compare our product to modern makes.

Technical Documentation, Overviews, Tutorials, and Papers
Software Engineering Definitions
QEF Architecture
A Comparison of QEF and Make
A Cook's Tour of the QEF System
An Introductory Guide to QEF
The QEF Quick Reference Card
The QEF Specialist's Reference Card
Short descriptions of all the QEF tools
QEF Papers, Talks, and Presentations

Introduction

The QEF tool kit is comprised of many software tools, which support a robust and rich software process. Many people will initially expect that QEF is substantially similar to make. As such it is necessary for us to compare the QEF product to make, which we do in this article. The comparison will look at QEF and make with respect to:

The coverage is not complete; there are many issues beyond the scope of this presentation. The above represent the key issues that must be covered when evaluating processes used to convert your resources (your source and its maintainers) into your income (the product).

Build Configuration and Parameterization Controls

A construction system should provide mechanisms to specify, control, check, distribute and use parameters, options, and controls required in software construction.

Make provides just the following:

    VAR=VALUE # setting (via makefile or command line)
    ${VAR} # use (only within the makefile itself)

This has the following short comings:

  • no further standard manipulation provided
  • no standard or easily applied mechanism to extend
  • frequent use of ad-hoc compensations (e.g., cpp), poorly documented and inconsistently applied
  • ad-hoc extensions require user to specify inclusion or invoke make in non-standard way
  • no provision for use of parameters by other tools except as passed through make itself

QEF uses a program called qvrs which interprets a set of files that specify parameter and configuration controls on a per-directory basis. qvrs:

  • has been refined through long experience
  • provides a comprehensive mechanism to set, test, and manipulate configuration settings
  • provides scoping of settings throughout a project tree
  • provides host-wide settings
  • is consistent in usage across all projects, trees, subtrees, and directories
  • provides configuration information to any program that wants it
User Interface

Make:

  • does not promote or facilitate consistency between makefiles
  • requires users to determine what phases need to be run and when
  • does not provide convenient mechanism for multi-directory builds
  • does not provide mechanism for monitoring multiple makes
  • does not provide mechanism to gently halt make mid-flight
  • often requires multiple passes to do builds
  • does not provide facilities for using anything but make

QEF:

  • promotes consistency across directories and projects
  • provides a convenient and powerful multi-directory process
  • provides a gentle non-destructive halt
  • provides a multi-directory trace
  • provides trailers and headers to facilitate looking at multi-directory construction outputs
  • provides facilities for incorporating non-standard processes (you may even use make)

QEF encapsulates all information and processes in same interface -- ``All you ever say is "qef"!''.

Construction Expression

Make is a declarative assembler language.

  • Any variations from suffix rule provided recipes require complete recipe specification
  • No provisions for aggregate expression
  • No standard mechanisms for control of suffix rules via configuration
  • Highly error prone, particularly with respect to dependencies and dynamic macros

Within make, ``what you see is ALL you get!''

QEF provides a fully featured extensible construction process programming language qsg, with simple natural expressions such as:

    program hello.c parser.y
    commands shell.sh man1.1 lex.l
    finstall -d DestDir/data file.txt
    library -n dtree @src

This simple expression automatically implements standard qef operations like:

  • Install -- copy product files to operational locations
  • RemoveInst -- remove all installed files
  • RemoveObjs -- remove all local intermediate files
  • Objects -- create all object intermediates
  • Lint -- check code style and coherence

Operations are implemented via a script library, which is easily extended to support project or company specific processes.

The average size of the construction files across 600 directories across 3 projects at Sietec/Toronto was 4 significant lines per directory.

Dependency Expression

Definition: Dependency expression is the specification of the list of prerequisite files for a target (i.e., the files on or from which the targets depend or are produced). The use of the dependency relationships is discussed in the next section.

Make dependencies are static, user generated, and counter-intuitive. Ad-hoc mechanisms, such as makedepends are expensive, and not incorporated into the build automatically.

QEF provides a generalized mechanism (incls) for extracting full transitive closure source implied dependencies, which are recalculated any time relevant source is changed. The mechanism currently supports C, C++, Class, troff, m4, f77, Pascal, bcpl, and SQL; and easily extended for additional languages (without recompiling programs). A standard tool is provided to view the dependencies for a specified file.

Model Of Consistency

Definition: A consistency model is a set of rules applied to determine if a recipe needs to be executed.

Make recipe applied:

  • if any target file does not exist
  • if any of the prereq files is younger than any of the targets and the target is required by explicit request or as a prereq to a requested target

This is insufficient.

QEF's consistency engine applies a recipe given that its targets are requested or required to build a requested target (that needs to be built -- more later):

  • if the recipe has the forced attribute
  • if the recipe has changed since it was last applied
  • if the prereq list for the recipe has changed
  • if the time stamp on any of the prerequisites or the targets have changed (note use of != rather than < relationship and that relationship of prereq's age not compared to that of target but to its age when last used)
  • if the time stamp of the target is more recent than any of the touch files for the recipe (see next page)

The history file used to preserve previously used recipes and time stamps also records the reason that the recipe was last applied.

The qef consistency model also supports:

  • touch files -- represent construction process tools, which can be manually touched to signify use of a new generation of an externally provided tool (e.g. a C compiler recipe will list a file called cc as a touch file)
  • absent intermediates -- for example, /bin/echo could be considered to be ``up-to-date'' even if echo does not exist provided given no other inconsistency for either /bin/echo or echo (i.e., a non-existent file is not necessarily inconsistent) This means that intermediate files can be removed if desirable which can yield a substantial reduction in required disk space
  • parallelism -- parallel executions of recipes; and trivially supports specification of mutual exclusivity (e.g., don't run simultaneous yaccs due to y.tab.c conflict)
  • completeness -- every change to the source, file system, environment or configuration that might produce a change in a product will cause that product to be regenerated
Additional Tools and Features

Many important and powerful features of qef have not been covered as they do not have make equivalents.

QEF supports, indeed encourages, total separation of the source and object trees without changing the construction specification (qeffile). This allows multiple configurations to be built from the same source directories (no need to copy the source to the build tree). The only difference between two different build trees will be in the construction configuration (conf.vrs).

Viewpath support also means that an individual developer's modified source can be completely separated from the shared master source, without copying or duplicating unmodified files.

Caches for information generated during the construction process, such as dependencies, timestamps, recipes, and consistency rationale, mean that subsequent constructions are both efficient and complete.

Powerful tools for use within qef scripts are provided:

  • arupdate is used to maintain archives and has the intelligence to remove members that shouldn't be in the library Also has facilities for dealing with archives that do not have time stamps
  • instal (missing `l' is to avoid name conflict with other implementations) is a pedantic, paranoid installation process with knowledge of the configuration system and an audit trail -- very useful
Build Cycle Time

Due to the improved control QEF offers and parallel execution, QEF builds are almost always at least 30% times faster than makes doing the same builds.

Most organizations have been able to reduce the build cycle time 30 to 50 percent.

However the real payoff occurs due to the improved build management and single pass nature of QEF. At sites that use Imake we have achieved reductions of 80 to 90 percent. This is due to the elimination of the exceedingly slow create Makefile passes that incorporate the snail-like mkdepends. Furthermore the use of QEF's qefdirs means that directories are only processed if there is processing to be done, whereas make managed builds will visit each and every directory for each type of construction whether there is anything to do or not.


Technical Documentation, Overviews, Tutorials, and Papers Software Engineering Definitions QEF Architecture A Comparison of QEF and Make A Cook's Tour of the QEF System An Introductory Guide to QEF The QEF Quick Reference Card
The QEF Specialist's Reference Card Short descriptions of all the QEF tools QEF Papers, Talks, and Presentations
qefvsmake.qh - 1.19 - 03/11/04 QEF home page