Chapter 8: Qvrs: The QEF Variables System

qvrs is the primary configuration tool of the QEF system. This chapter describes its files, syntax, semantics, and variables.

About 40 qvrs variables are set or initialized automatically by qvrs itself. These variables are those that are dependent on the host/platform or the current directory location (e.g., _Hostname_, User, System[*], CurDir), or variables that are initialized to their default values (e.g., LibMatch, CcInclDirs).

The user, project, or host system specific qvrs variables are defined in files collectively known as qvrs files. qvrs's primary purpose is to find and interpret these files and create a database of variable/value tuples that can be consulted by other tools. This database is then accessed by other tools either by invoking qvrs directly and reading the output, or by reading a binary file built by qvrs.

Note:

Due to frequency of access to the qvrs variables, qef's initialization includes invoking qvrs to create /tmp/qv_pid.tmp. qef then sets $QVRS_FILE to the name of this file and all other tools that need qvrs settings load this file directly.

Note:

Environment variables are used within the QEF system, but sparingly as their values are usually volatile and not well controlled or documented. One of the advantages of qvrs over more conventional approaches is that it can be used to capture the environmental controls for a build in simple sources, thereby ensuring that the environment can be consistent for all users and can be resurrected in the future.

qvrs variables are initialized, assigned, overridden, and destroyed in the qvrs files, arranged in a hierarchy. Where you choose to define or override a variable has a significant effect upon its visibility, and therefore upon the construction tools that the generated scripts ultimately call. You control what your production tools know, and when they know it, using the qvrs files.

The scope of most the qvrs files is its containing directory and all that directory's sub-directories as well as the corresponding directories in the other trees. The exceptions are leaf.vrs and qeffile, which apply to the current directory only.

The qvrs files are:

root.vrs	establishes root directory and sets major control and path parameters -- found in the root (by definition) of the current tree. @RootDir is set to the name of the directory containing root.vrs -- defaults to /.
conf.vrs	set user configuration options -- usually found in the @RootDir directory or in the source directory @QefAdm/confvrs.
./leaf.vrs	sets local temporary overrides -- found in the current directory.
qeffile preamble	sets local settings prior to processing higher directory parameters -- found in a directory named by @SrcPath, usually the corresponding baseline tree directory.
branch.vrs	sets tree specific temporary overrides -- found in the current directory or a parent directory, up to @RootDir.
tree.vrs	sets project sub-tree specific parameters -- found in the current directory and parent directories up to @RootDir or a corresponding directory in the @RootPath (established by the root.vrs file).
prereq.vrs	sets the project's prerequisite projects' root paths -- usually found in the project's baseline @QefAdm/confvrs
p_sysvrs.vrs	sets project/system specific parameters -- usually found in the project's baseline @QefAdm/confvrs directory. sysvrs is the value of system v or @System[SysVrs].
sysvrs.vrs	sets system specific parameters -- usually found in <qtree>/lib/sysvrs.
{tree,branch}.vrs redux	any suspended {tree,branch}.vrs resumed in LIFO order to set further sub-tree parameters after processing higher directory files
qeffile	qeffile after any preamble processed up to Begin line to set directory specific parameters
./leaf.vrs redux	suspended leaf.vrs file resumed to set directory specific temporary overrides

The actual names of the files processed and the search mechanisms used to find them are described below in Section 8.5.

Typically, the qvrs files for a directory will be the root.vrs file, a @RootDir/conf.vrs file or the default source file @QefAdm/confvrs/confdefl.vrs, a tree.vrs file in the root of the baseline source tree, the system's <qtree>/lib/sysvrs/`system -v`.vrs, and a qeffile in the corresponding baseline tree. Larger projects will typically include prereq.vrs and p_sysvrs.vrs files from the @QefAdm/confvrs directory. For example:

    % qvrs -f # list current directory's qvrs files
    /m/guide/object/root.vrs
    /m/guide/object/conf.vrs
    /m/guide/baseline/tree.vrs
    /m/guide/baseline/QefAdm/confvrs/prereq.vrs
    /m/guide/baseline/QefAdm/confvrs/p_linux2.vrs
    <qtree>/lib/sysvrs/linux2.vrs
    /m/guide/baseline/guide/qeffile

These files are interpreted by the qvrs program. Any utility can make use of the qvrs program and its output. These other tools also create or modify configuration files:

Creates or modifies the contents of a root.vrs file. Note that mkqtree invokes rootvrs to create a root.vrs file when creating a new tree.

Views the contents of the conf.vrs or the default confdefl.vrs files. It may also be used to view differences between the two mentioned files or to create a fresh conf.vrs file.

Adds or changes values in the conf.vrs file.

qvrs without flags or arguments lists the current settings.

    % qvrs | sed 10q # just list first 10 lines
    AsSuffix	s
    BeginLine	qsg -M
    Branch	guide
    BuildHost	matt
    BuildPath	/m/qtree/linux2_0i/9.1/bin /bin /usr/bin
    BuildSys	Linux-2.0.36-i686
    CcInclDirs	/usr/include
    CcLibDirs	/lib /usr/lib
    ConfVrsFile	/m/guide/o/conf.vrs
    ConfVrsPath	/m/guide/o /m/guide/s/QefAdm/confvrs 

Many of the qvrs variables refer to directory paths for finding files of various kinds. Other variables concern the project (Project, RootDir, RootPath) and user (Logname, User, _Hostname_, System). Some variables contain compiler switches, library mappings etc. The command x-qvrs <variable> will explain most of the standard variables, as in:

% x-qvrs BuildPath
BuildPath : executable path used in builds

This path variable is the list of directories to be assigned
to Path, that is the directories to be searched for
executables. This permits the user to use whatever path
they so choose, but the builds will be done using a somewhat
saner and approved set of directories.

BuildPath is initially set to the Dir/bin, for each Dir named
by $QTREE, followed by the directories named by the traits
variable BuildPath.

Additional directories can be inserted in front of the traits
added directories using:

	addpath BuildPath new-dirs ...

See also: traits $PATH

L

    % x-qvrs -L \*vars | sed 10q # just list first 10
        ARTMPDIR : name of ar(1)'s temporary directory
        AsSuffix : suffix of assembler files
       BeginLine : Begin line arguments
          Branch : path from RootDir to current qvrs file
       BuildHost : name of the host used for remote builds
        BuildSys : type of the system on which build to be done
        CcAsFlag : cc flag to produce assembler file
   CcSed_file[X] : ccsed sed script file
       CcLibDirs : cc's default library search path
      ConfigName : name of the configuration

These variables provide the QEF tools with a reference scheme which is independent of file, directory and even system location. This is the key to QEF's ability to construct and walk complex search paths spanning machines, platforms and even networks. From QEF's point of view, everything is relative. When qef generates the back-end scripts, the myriad variable references are replaced with unambiguous references to files, directories, flags and programs.

The order in which qvrs processes its files takes into account the hierarchical nature of an intelligently arranged tree of directories. The algorithm must walk the directories to find the various qvrs files, and adjust its variables according to the position in the tree where they are initialized, tested, reset, etc. Files are processed according to the following:

1) root.vrs

Look for root.vrs, stopping at the first file found. Start the search in the current directory, working upwards towards `/'.

2) conf.vrs -- the User's Configuration File

If @ConfVrs is defined, search the directories named by @ConfVrsPath for the named file. Otherwise, look for and process @RootDir/conf.vrs if it exists. If there is no such file, search the @SrcRoot directories for @QefAdm/confvrs/confdefl.vrs, using that file if found. To point to a specific conf.vrs file, set the variable @ConfVrs in the root.vrs file to the name of the file you wish to use -- best done using rootvrs C. This causes qvrs to search ConfVrsPath directories for the named file.

3) leaf.vrs -- optional override file

Process ./leaf.vrs if it exists. If a suspend command is encountered, processing of the leaf.vrs file is suspended until step #12.

4) optional qeffile preamble

Search the current @SrcPath for qeffile (or qeffile2 if 2 flag specified). If the first significant line is preamble, lines up to the next line that begins with endpreamble are processed. The balance of the file will be processed in step #11. Note that the file that is processed in step #11 may be different since the search path @SrcPath may have been changed or the variable @QefFile[@RealBranch] may have been set.

5) branch.vrs and tree.vrs files

For each directory starting at the current one, working up the tree to the directory containing the root.vrs file, perform the following: look for branch.vrs and process if found; look for tree.vrs in same directory or corresponding directories in RootPath and process if found. Both branch.vrs and tree.vrs may contain suspend commands.

6) prereq.vrs -- the prerequisite project file

If the top level tree.vrs file invokes the prereqvrs keyword, @ConfVrsPath directories are searched for a file called @PrereqVrs (defaults to prereq.vrs). If found, the file is processed.

7) p_sys.vrs -- the project specific sysvrs file

If @PsysVrs is defined, search the @ConfVrsPath for that file. If it is not found qvrs will abort. If @PsysVrs is not defined, search the @ConfVrsPath directories for the file p_@System[SysVrs].vrs. No special action is taken if the file is not found. This file may also be loaded by the psysvrs keyword in the top tree.vrs file.

8) sys.vrs -- platform specific settings

Search the directories named by SysVarsPath for @SysVrs (default @System[SysVrs].vrs) and process the file found. Alternatively, you may load the file using a sysvrs command within the top level tree.vrs file.

9) <qtree>/data/traits/host/.tab

The host's traits file (see Section 8.9) is processed to incorporate any variables marked qef_imports that aren't already set.

10) resume suspended {branch/tree}.vrs files

Working in reverse order (LIFO), resume processing any suspended branch.vrs and tree.vrs files.

11) the qeffile or its replacement

If @QefFile[@RealBranch] is defined, search the @RootPath directories for the file it names. Otherwise search @SrcPath for qeffile or qeffile2, and process up to and including the Begin line.

12) Resume any suspended leaf.vrs

.

This file anchors the project tree. It provides the point of reference from which all relative directory displacements occur. The file is usually created by the mkqtree utility. Rather than edit the file directly, you maintain it using the QEF program rootvrs.

The following variables are typically set in this file:

RootPath	the root directories of the project. It consists of the current directory plus the source path and the master source path.
RootAlias	alternative name for the root, for example: the symbolic link name.
Project	the one-word name of the project, often used as a directory name
Revision	the current revision number.
TreeType	names the type of tree, for example: baseline, working, object. TreeType[dir] settings are defined for each directory dir named by RootPath.
BuildHost	names the host on which the build is usually done.
BuildSys	names the system (e.g., Linux-2.0.36-i686) of the build.
QremoteEnv	names the environment set (see qremote) used when running a remote build.
_DestDir_	the name of the destination directory.
PrereqList	list of prerequisite projects -- see "The prereq.vrs file" below.
ConfigName	A name for this particular configuration.
ConfVrs	names the conf.vrs file.
SysVrs	names the sys.vrs file.
TreedirsList	names the construction list i.e., the list of all the construction files -- see treedirs.
VCSys	name of the version system to be used, for example: SCCS, RCS.
VCRoot	the root of the version-control tree if it is not equal to SrcRoot.
VCBranch	an offset from the CVSROOT -- required by the CVS system.
OldRoot	name of the directory containing old source.
EditBranches	a boolean flag used by some version-control systems to force editing within branches. That is, any given file requested for editing is edited in a branch of the current directory -- may not be supported by your version system.

This file stores user-specific settings and/or system tasks. Typically, it contains tool names and parameter settings for the tools. This file is usually created by copying a default file (@QefAdm/confvrs/confdefl.vrs), and is usually edited manually or using the qconfset utility.

Two "magic" prefixes are often found here:

_T_tool	This construct names a tool used in constructions. If the variable is not set, the "_T_" prefix is removed and the rest of the name is used as the tool name. For example, _T_cc becomes cc.
_F_tool	this construct specifies a flag list to be passed to a given tool. For example, _F_cc is the flag list to pass to _T_cc, and so on.

The conf.vrs file is sometimes used as a strfix replacement dictionary to configure data files during the build. As such, it should be limited to those keywords that are also supported by strfix.

This file specifies tree-specific settings. It affects the resident directory and all sub-directories beneath it. Like leaf.vrs and branch.vrs, tree.vrs files may use the suspend keyword.

Most projects will have a tree.vrs file in the source root that sets the Project name, standard qsg, library, and include search paths, special semantic options and controls.

tree.vrs may appear in sub-directories, but they are rare and usually small -- one to three lines settings some control that is used within that sub-directory and its tree. For example, a sub-directory may modify @_DestDir_ as in:

    set _DestDir_ @_DestDir_/games

XP

    % qvrs -XP
    PrereqPath	/ph
    PrereqList	tcl(7.4)
    PrereqVrs	prereq.vrs # default value
    PrereqFormat  <P>/<C>/<V> # default value
    PrereqConf	linux2_4i
    PrereqRoot[tcl]  /ph/tcl/linux2_4i/7.4

This file is used to contain project/system specific settings such as system dependent mappings to third party software or application flags. A project employs such a file if there are a more than few system dependent settings. If there is just one or two, it is usually preferable to put the settings in the top level tree.vrs file using a switch statement.

This file, which is initially provided as part of the QEF package, contains system specific settings. These files are located in the $QTREE/lib/sysvrs. The sysnm.vrs files are used to describe platform-level settings such as library mappings and search paths, and special flags and options. In particular, the sysnm.vrs file sets the flags for the @DEBUGGING or @PROFILING options, as in:

    if @(option PROFILING)
	set _CcFlags_index_	Profiling
	prepend LibMatch	lib%_p.a
    elif @(option DEBUGGING)
	set _CcFlags_index_	Debug
	set _Optimize_index_	Debug
	prepend LibMatch	lib%_g.a
    fi
    
    # Set default behaviour for selected options
    cset _CcFlags_value_[*,Profiling]	-pg
    cset _CcFlags_value_[*,Debug]	-g

Every construction directory of a project must have a corresponding qeffile in the directory's SrcPath list or a file named by @QefFile[@RealBranch]. It defines directory-specific settings and controls.

The qeffile has three or four sections:

1. The optional preamble ... endpreamble at the beginning of the file is used to specify settings that are required before processing the sysnm.vrs file or the tree.vrs file, as in:

       preamble
   	options DEBUGGING
       endpreamble
   

2. The qvrs section of the qeffile -- defines flags to be used within the QEF script, search paths specific to this directory, library lists for individual programs, etc. In a large majority of qeffiles, this section is empty or limited to the setting of the @Suffixes and/or @InclPath variables.

3. The Begin line is used to separate the qvrs settings from the qeffile script and to specify the script generator, that is, the command used to generate the back-end input. If there are no arguments on the Begin line the default value, is specified by @DefaultBeginLine, which itself defaults to "qsg M".

4. The section of the qeffile following the Begin line is typically input to the script generator or the back-end.

The files leaf.vrs and branch.vrs are used to override the established values in certain directories and branches of trees. They are intended for strictly temporary use, providing a way to make a quick change without changing the permanent qvrs source files.

To temporarily override or modify a setting in a qeffile or qvrs file, instead of copying the file to the local directory, create a file called leaf.vrs in the local directory, and set the override in this file. For example:

    addpath SrcPath @(home doug)/src/@Branch

which says, in effect, "Add Doug's corresponding directory to the search path for the current directory" -- thus including his maintenance changes along with your own. A leaf.vrs file applies in its containing directory only. A branch.vrs file applies in its containing directory and any of it sub-directory trees.

Here are the most commonly used keywords, along with brief descriptions of their use. All of these are shown in the tutorials. For a complete list, see Appendix F Qvrs Keywords

addpath <pathvar> <newdir>

This command adds path <newdir> to the specified <pathvar>.

cset <var> <value>

This command conditionally sets the specified variable to the value supplied. The setting only takes effect if <var> has not already been set to a value.

set <var> <value>

This command forcibly sets the specified variable to the value supplied. The setting takes effect regardless of the state and contents of <var>.

if <condition> ... [elif <condition 2>]* ... [else ...] fi

The most basic branching mechanism begins with the "if" keyword and ends with "fi". Between these limits may be any number of "elif" and a single "else" statements.

qvrs files consist of zero or more standard text lines.

All white-space at the beginning and end of lines is removed on input. White-space is a non-empty sequence of space and tab characters.

Empty lines or lines that begin with a `#' are ignored, except when they follow a significant (not ignored) line that ends with a `@'.

If a significant line ends with a `@', the `@' and the newline are removed and the next input line, minus any leading white-space, is appended to the previously read line.

Note:

Any white-space immediately before the `@' at the end of the line is preserved. To end a line with a literal `@' use `@@#'.

Note:

When diagnostic output refers to a multiple line input command, the line number used is that of the first line.

There is no limit to the length of a line other than those imposed by the host system.

The resulting line should consist of a keyword and its arguments. The keywords are listed in the keywords(x-qvrs) item and explained in detail in their own individual item. Any embedded `@' characters in the arguments are expanded as explained in the following. A valid qvrs line consists of a keyword followed by arguments. The arguments are processed to replace embedded substrings as follows:

Substring	Replacement
@@	a single `@'
@.	replaced by nothing -- used to terminate variable name when followed by alphas or `['.
@0	obsolete version of @. -- warning produced
# ...	empty string; rest of line ignored
@#	a single `#'
@NotSet	replaced by `@NotSet'
@Var¹	value of Variable, `' if undefined
@Var[str]¹	Any `@'s in str are expanded to yield result. Then entire string is replaced by the value of @Var[result]. The expanded str index must not contain any spaces or tabs.
@( ...)¹	The named function evaluated and result replaces sequence. Any arguments to the function are expanded prior to the call.
@<else>	An `@' followed by any other character just replaced by character.

¹	The @Var; @Var[str], or @(function ...) expressions may be followed by tilde-ops -- see Appendix F Qsg TildeOps. For example, @Var~v is replaced by 1 (0) if @Var is (is not) empty (i.e, void).

The name of a variable (i.e., Var in the above) may be any sequence of alphanumerics or underbars, the first of which may not be a digit.

For a summary of the qvrs command-line options, use the command qvrs x or see the qvrs(1) man page. For developers, the flags often used with qvrs are:

• qvrs: to see all the variables currently defined
• qvrs f: to show the list of qvrs files
• qvrs l: to show library mapping information
• qvrs XI: to show the cc I flags
• qvrs VD: to show value of _DestDir_
• qvrs V?: to get a list of the V and X options
• qvrs W <var>: to see where a variable was set or modified

    % traits | sed 5q # just list first five
    ARFLAGSDASHED   0
    BOGUS_CSH       0
    BOGUS_RENAME    1
    BuildPath       /bin:/usr/bin
    DBLESLASHROOT   0

    % x-qvrs ARFLAGSDASHED
    ARFLAGSDASHED(x-qvrs) specifies ar option preceded by dash

    This boolean traits-var specifies whether or not the
    system's ar(1) program requires the action argument to be
    preceded by a `'. If the setting is 0, the `' is not
    inserted into ar commands.

    See also: traits ar(1)

There are three tools in the traits system:

The <qtree>/lib/traits.vrs file is provided as part of the QEF product and should be left unchanged. In contrast, the <qtree>/data/traits.ext file is used to make local host, site, or platform specific settings.

An important setting is that of QDSRV_HOST, the name of the host on which qdsrv is run, as in:

    cset QDSRV_HOST host

A key feature of the traits system is the automatic importation by qvrs of all the traits that are marked for export or whose name begins with "_[TFD]_".

In many ways qvrs is the second most important tool of the QEF system and understanding its use is a large part of mastering QEF.