QEF home page

Chapter 8: Qvrs: The QEF Variables System

The Guide's Table of Contents Previous Chapter Bottom Of Page Next Chapter

qvrs is the primary configuration tool of the QEF system. This chapter describes its files, syntax, semantics, and variables.


8.1) Introduction to Qvrs

qvrs controls QEF variables used for software production. qvrs variables are used to define search paths, tool names and flags, semantic options, qef controls and miscellaneous application flags and controls. Appendix F Qvrs Variables lists the standard set of qvrs variables.

About 40 qvrs variables are set or initialized automatically by qvrs itself. These variables are those that are dependent on the host/platform or the current directory location (e.g., _Hostname_, User, System[*], CurDir), or variables that are initialized to their default values (e.g., LibMatch, CcInclDirs).

The user, project, or host system specific qvrs variables are defined in files collectively known as qvrs files. qvrs's primary purpose is to find and interpret these files and create a database of variable/value tuples that can be consulted by other tools. This database is then accessed by other tools either by invoking qvrs directly and reading the output, or by reading a binary file built by qvrs.
Note: Due to frequency of access to the qvrs variables, qef's initialization includes invoking qvrs to create /tmp/qv_pid.tmp. qef then sets $QVRS_FILE to the name of this file and all other tools that need qvrs settings load this file directly.
Note: Environment variables are used within the QEF system, but sparingly as their values are usually volatile and not well controlled or documented. One of the advantages of qvrs over more conventional approaches is that it can be used to capture the environmental controls for a build in simple sources, thereby ensuring that the environment can be consistent for all users and can be resurrected in the future.

8.2) The Qvrs Files

qvrs variables are initialized, assigned, overridden, and destroyed in the qvrs files, arranged in a hierarchy. Where you choose to define or override a variable has a significant effect upon its visibility, and therefore upon the construction tools that the generated scripts ultimately call. You control what your production tools know, and when they know it, using the qvrs files.

The scope of most the qvrs files is its containing directory and all that directory's sub-directories as well as the corresponding directories in the other trees. The exceptions are leaf.vrs and qeffile, which apply to the current directory only.

The qvrs files are:
root.vrs establishes root directory and sets major control and path parameters -- found in the root (by definition) of the current tree. @RootDir is set to the name of the directory containing root.vrs -- defaults to /.
conf.vrs set user configuration options -- usually found in the @RootDir directory or in the source directory @QefAdm/confvrs.
./leaf.vrs sets local temporary overrides -- found in the current directory.
qeffile preamble sets local settings prior to processing higher directory parameters -- found in a directory named by @SrcPath, usually the corresponding baseline tree directory.
branch.vrs sets tree specific temporary overrides -- found in the current directory or a parent directory, up to @RootDir.
tree.vrs sets project sub-tree specific parameters -- found in the current directory and parent directories up to @RootDir or a corresponding directory in the @RootPath (established by the root.vrs file).
prereq.vrs sets the project's prerequisite projects' root paths -- usually found in the project's baseline @QefAdm/confvrs
p_sysvrs.vrs sets project/system specific parameters -- usually found in the project's baseline @QefAdm/confvrs directory. sysvrs is the value of system --v or @System[SysVrs].
sysvrs.vrs sets system specific parameters -- usually found in <qtree>/lib/sysvrs.
{tree,branch}.vrs redux any suspended {tree,branch}.vrs resumed in LIFO order to set further sub-tree parameters after processing higher directory files
qeffile qeffile after any preamble processed up to Begin line to set directory specific parameters
./leaf.vrs redux suspended leaf.vrs file resumed to set directory specific temporary overrides
The actual names of the files processed and the search mechanisms used to find them are described below in Section 8.5.

Typically, the qvrs files for a directory will be the root.vrs file, a @RootDir/conf.vrs file or the default source file @QefAdm/confvrs/confdefl.vrs, a tree.vrs file in the root of the baseline source tree, the system's <qtree>/lib/sysvrs/`system -v`.vrs, and a qeffile in the corresponding baseline tree. Larger projects will typically include prereq.vrs and p_sysvrs.vrs files from the @QefAdm/confvrs directory. For example:

    % qvrs -f # list current directory's qvrs files
    /m/guide/object/root.vrs
    /m/guide/object/conf.vrs
    /m/guide/baseline/tree.vrs
    /m/guide/baseline/QefAdm/confvrs/prereq.vrs
    /m/guide/baseline/QefAdm/confvrs/p_linux2.vrs
    <qtree>/lib/sysvrs/linux2.vrs
    /m/guide/baseline/guide/qeffile
In addition to the qvrs files there is another set of significant files and these are the traits files described in Section 8.9 below. qvrs automatically imports all the traits variables that begin with "_T_", "_F_", "_D_", are of the form "_*_" or "_*_[...]", or that are marked as qef_imports.
Note: The qvrs and traits variables that are marked as qef_imports are automatically imported by the qef script pre-processor as macros.

8.3) Configuration Tools

These files are interpreted by the qvrs program. Any utility can make use of the qvrs program and its output. These other tools also create or modify configuration files:

rootvrs

Creates or modifies the contents of a root.vrs file. Note that mkqtree invokes rootvrs to create a root.vrs file when creating a new tree.

confvrs

Views the contents of the conf.vrs or the default confdefl.vrs files. It may also be used to view differences between the two mentioned files or to create a fresh conf.vrs file.

qconfset

Adds or changes values in the conf.vrs file.

qvrsexpr Evaluates and outputs qvrs expressions without executing qvrs itself if a qvrs database (as named by $QVRS_FILE) already exists. Will invoke qvrs to create the database if $QVRS_FILE is not set.
qvrsdmp Processes a qvrs binary database and outputs selected information -- primarily a qvrs debugging tool.
mktraits This tool is actually qvrs but modified to process the traits input files to produce a binary file <qtree>/data/traits/host.tab to facilitate the fast retrieval of traits settings -- see Section 8.9.
traits This tool reads the files created by mktraits to extract requested information, invoking mktraits to create the file if necessary -- see Section 8.9.

8.4) Qvrs Variables

qvrs without flags or arguments lists the current settings.

    % qvrs | sed 10q # just list first 10 lines
    AsSuffix	s
    BeginLine	qsg -M
    Branch	guide
    BuildHost	matt
    BuildPath	/m/qtree/linux2_0i/9.1/bin /bin /usr/bin
    BuildSys	Linux-2.0.36-i686
    CcInclDirs	/usr/include
    CcLibDirs	/lib /usr/lib
    ConfVrsFile	/m/guide/o/conf.vrs
    ConfVrsPath	/m/guide/o /m/guide/s/QefAdm/confvrs 

Many of the qvrs variables refer to directory paths for finding files of various kinds. Other variables concern the project (Project, RootDir, RootPath) and user (Logname, User, _Hostname_, System). Some variables contain compiler switches, library mappings etc. The command x-qvrs <variable> will explain most of the standard variables, as in:

% x-qvrs BuildPath
BuildPath : executable path used in builds

This path variable is the list of directories to be assigned
to Path, that is the directories to be searched for
executables. This permits the user to use whatever path
they so choose, but the builds will be done using a somewhat
saner and approved set of directories.

BuildPath is initially set to the Dir/bin, for each Dir named
by $QTREE, followed by the directories named by the traits
variable BuildPath.

Additional directories can be inserted in front of the traits
added directories using:

	addpath BuildPath new-dirs ...

See also: traits $PATH
The command x-qvrs --L \*vars will list all of the variables documented in the system, as in part:
    % x-qvrs -L \*vars | sed 10q # just list first 10
        ARTMPDIR : name of ar(1)'s temporary directory
        AsSuffix : suffix of assembler files
       BeginLine : Begin line arguments
          Branch : path from RootDir to current qvrs file
       BuildHost : name of the host used for remote builds
        BuildSys : type of the system on which build to be done
        CcAsFlag : cc flag to produce assembler file
   CcSed_file[X] : ccsed sed script file
       CcLibDirs : cc's default library search path
      ConfigName : name of the configuration
Anyone can create and assign variables, the system can only document the ones assigned and used by the tools as shipped. See Appendix F Qvrs Variables for the complete list of documented variables.

These variables provide the QEF tools with a reference scheme which is independent of file, directory and even system location. This is the key to QEF's ability to construct and walk complex search paths spanning machines, platforms and even networks. From QEF's point of view, everything is relative. When qef generates the back-end scripts, the myriad variable references are replaced with unambiguous references to files, directories, flags and programs.

8.5) The Qvrs Files and Their Processing Order

The order in which qvrs processes its files takes into account the hierarchical nature of an intelligently arranged tree of directories. The algorithm must walk the directories to find the various qvrs files, and adjust its variables according to the position in the tree where they are initialized, tested, reset, etc. Files are processed according to the following:

1) root.vrs
Look for root.vrs, stopping at the first file found. Start the search in the current directory, working upwards towards `/'.
2) conf.vrs -- the User's Configuration File
If @ConfVrs is defined, search the directories named by @ConfVrsPath for the named file. Otherwise, look for and process @RootDir/conf.vrs if it exists. If there is no such file, search the @SrcRoot directories for @QefAdm/confvrs/confdefl.vrs, using that file if found. To point to a specific conf.vrs file, set the variable @ConfVrs in the root.vrs file to the name of the file you wish to use -- best done using rootvrs --C. This causes qvrs to search ConfVrsPath directories for the named file.
3) leaf.vrs -- optional override file
Process ./leaf.vrs if it exists. If a suspend command is encountered, processing of the leaf.vrs file is suspended until step #12.
4) optional qeffile preamble
Search the current @SrcPath for qeffile (or qeffile2 if --2 flag specified). If the first significant line is preamble, lines up to the next line that begins with endpreamble are processed. The balance of the file will be processed in step #11. Note that the file that is processed in step #11 may be different since the search path @SrcPath may have been changed or the variable @QefFile[@RealBranch] may have been set.
5) branch.vrs and tree.vrs files
For each directory starting at the current one, working up the tree to the directory containing the root.vrs file, perform the following: look for branch.vrs and process if found; look for tree.vrs in same directory or corresponding directories in RootPath and process if found. Both branch.vrs and tree.vrs may contain suspend commands.
6) prereq.vrs -- the prerequisite project file
If the top level tree.vrs file invokes the prereqvrs keyword, @ConfVrsPath directories are searched for a file called @PrereqVrs (defaults to prereq.vrs). If found, the file is processed.
7) p_sys.vrs -- the project specific sysvrs file
If @PsysVrs is defined, search the @ConfVrsPath for that file. If it is not found qvrs will abort. If @PsysVrs is not defined, search the @ConfVrsPath directories for the file p_@System[SysVrs].vrs. No special action is taken if the file is not found. This file may also be loaded by the psysvrs keyword in the top tree.vrs file.
8) sys.vrs -- platform specific settings
Search the directories named by SysVarsPath for @SysVrs (default @System[SysVrs].vrs) and process the file found. Alternatively, you may load the file using a sysvrs command within the top level tree.vrs file.
9) <qtree>/data/traits/host/.tab
The host's traits file (see Section 8.9) is processed to incorporate any variables marked qef_imports that aren't already set.
10) resume suspended {branch/tree}.vrs files
Working in reverse order (LIFO), resume processing any suspended branch.vrs and tree.vrs files.
11) the qeffile or its replacement
If @QefFile[@RealBranch] is defined, search the @RootPath directories for the file it names. Otherwise search @SrcPath for qeffile or qeffile2, and process up to and including the Begin line.
12) Resume any suspended leaf.vrs
.
The root.vrs file

This file anchors the project tree. It provides the point of reference from which all relative directory displacements occur. The file is usually created by the mkqtree utility. Rather than edit the file directly, you maintain it using the QEF program rootvrs.

The following variables are typically set in this file:
RootPaththe root directories of the project. It consists of the current directory plus the source path and the master source path.
RootAliasalternative name for the root, for example: the symbolic link name.
Projectthe one-word name of the project, often used as a directory name
Revisionthe current revision number.
TreeTypenames the type of tree, for example: baseline, working, object. TreeType[dir] settings are defined for each directory dir named by RootPath.
BuildHostnames the host on which the build is usually done.
BuildSysnames the system (e.g., Linux-2.0.36-i686) of the build.
QremoteEnvnames the environment set (see qremote) used when running a remote build.
_DestDir_the name of the destination directory.
PrereqListlist of prerequisite projects -- see "The prereq.vrs file" below.
ConfigNameA name for this particular configuration.
ConfVrsnames the conf.vrs file.
SysVrsnames the sys.vrs file.
TreedirsListnames the construction list i.e., the list of all the construction files -- see treedirs.
VCSysname of the version system to be used, for example: SCCS, RCS.
VCRootthe root of the version-control tree if it is not equal to SrcRoot.
VCBranchan offset from the CVSROOT -- required by the CVS system.
OldRootname of the directory containing old source.
EditBranchesa boolean flag used by some version-control systems to force editing within branches. That is, any given file requested for editing is edited in a branch of the current directory -- may not be supported by your version system.

The conf.vrs file

This file stores user-specific settings and/or system tasks. Typically, it contains tool names and parameter settings for the tools. This file is usually created by copying a default file (@QefAdm/confvrs/confdefl.vrs), and is usually edited manually or using the qconfset utility.

Two "magic" prefixes are often found here:
_T_tool This construct names a tool used in constructions. If the variable is not set, the "_T_" prefix is removed and the rest of the name is used as the tool name. For example, _T_cc becomes cc.
_F_tool this construct specifies a flag list to be passed to a given tool. For example, _F_cc is the flag list to pass to _T_cc, and so on.

The conf.vrs file is sometimes used as a strfix replacement dictionary to configure data files during the build. As such, it should be limited to those keywords that are also supported by strfix.

The tree.vrs file

This file specifies tree-specific settings. It affects the resident directory and all sub-directories beneath it. Like leaf.vrs and branch.vrs, tree.vrs files may use the suspend keyword.

Most projects will have a tree.vrs file in the source root that sets the Project name, standard qsg, library, and include search paths, special semantic options and controls.

tree.vrs may appear in sub-directories, but they are rare and usually small -- one to three lines settings some control that is used within that sub-directory and its tree. For example, a sub-directory may modify @_DestDir_ as in:

    set _DestDir_ @_DestDir_/games
or modify @InclPath or @LibPath for the entire sub-project.
The prereq.vrs file This file sets and uses @Prereq*[] variables. Once it has completed qvrs searches the @PrereqPath path to set @PrereqRoot[X] for each project X named by @PrereqList. The other @Prereq* variables are used to specify the release, configuration, and directory format for the projects. A full description of the use and setting of the @Prereq* variables may be found in prereqvrs(x-qvrs). The command qvrs --XP flag will list all the Prereq* settings, as in:
    % qvrs -XP
    PrereqPath	/ph
    PrereqList	tcl(7.4)
    PrereqVrs	prereq.vrs # default value
    PrereqFormat  <P>/<C>/<V> # default value
    PrereqConf	linux2_4i
    PrereqRoot[tcl]  /ph/tcl/linux2_4i/7.4
The p_sysnm.vrs file

This file is used to contain project/system specific settings such as system dependent mappings to third party software or application flags. A project employs such a file if there are a more than few system dependent settings. If there is just one or two, it is usually preferable to put the settings in the top level tree.vrs file using a switch statement.

The sysnm.vrs file

This file, which is initially provided as part of the QEF package, contains system specific settings. These files are located in the $QTREE/lib/sysvrs. The sysnm.vrs files are used to describe platform-level settings such as library mappings and search paths, and special flags and options. In particular, the sysnm.vrs file sets the flags for the @DEBUGGING or @PROFILING options, as in:

    if @(option PROFILING)
	set _CcFlags_index_	Profiling
	prepend LibMatch	lib%_p.a
    elif @(option DEBUGGING)
	set _CcFlags_index_	Debug
	set _Optimize_index_	Debug
	prepend LibMatch	lib%_g.a
    fi
    
    # Set default behaviour for selected options
    cset _CcFlags_value_[*,Profiling]	-pg
    cset _CcFlags_value_[*,Debug]	-g
The qeffile

Every construction directory of a project must have a corresponding qeffile in the directory's SrcPath list or a file named by @QefFile[@RealBranch]. It defines directory-specific settings and controls.

The qeffile has three or four sections:

  1. The optional preamble ... endpreamble at the beginning of the file is used to specify settings that are required before processing the sysnm.vrs file or the tree.vrs file, as in:
        preamble
    	options DEBUGGING
        endpreamble
    
  2. The qvrs section of the qeffile -- defines flags to be used within the QEF script, search paths specific to this directory, library lists for individual programs, etc. In a large majority of qeffiles, this section is empty or limited to the setting of the @Suffixes and/or @InclPath variables.

  3. The Begin line is used to separate the qvrs settings from the qeffile script and to specify the script generator, that is, the command used to generate the back-end input. If there are no arguments on the Begin line the default value, is specified by @DefaultBeginLine, which itself defaults to "qsg --M".

  4. The section of the qeffile following the Begin line is typically input to the script generator or the back-end.
Overrides: leaf.vrs and branch.vrs

The files leaf.vrs and branch.vrs are used to override the established values in certain directories and branches of trees. They are intended for strictly temporary use, providing a way to make a quick change without changing the permanent qvrs source files.

To temporarily override or modify a setting in a qeffile or qvrs file, instead of copying the file to the local directory, create a file called leaf.vrs in the local directory, and set the override in this file. For example:

    addpath SrcPath @(home doug)/src/@Branch

which says, in effect, "Add Doug's corresponding directory to the search path for the current directory" -- thus including his maintenance changes along with your own. A leaf.vrs file applies in its containing directory only. A branch.vrs file applies in its containing directory and any of it sub-directory trees.

8.6) Qvrs Keywords

Here are the most commonly used keywords, along with brief descriptions of their use. All of these are shown in the tutorials. For a complete list, see Appendix F Qvrs Keywords

addpath

addpath <pathvar> <newdir>

This command adds path <newdir> to the specified <pathvar>.

cset

cset <var> <value>

This command conditionally sets the specified variable to the value supplied. The setting only takes effect if <var> has not already been set to a value.

set

set <var> <value>

This command forcibly sets the specified variable to the value supplied. The setting takes effect regardless of the state and contents of <var>.

if ... fi

if <condition> ... [elif <condition 2>]* ... [else ...] fi

The most basic branching mechanism begins with the "if" keyword and ends with "fi". Between these limits may be any number of "elif" and a single "else" statements.

8.7) Formal Rules

The following applies to all *.vrs qvrs files and to the lines up to and including the Begin line in a qeffile or qeffile2 (the lines that follow a Begin in a qeffile are not processed by qvrs).

qvrs files consist of zero or more standard text lines.

All white-space at the beginning and end of lines is removed on input. White-space is a non-empty sequence of space and tab characters.

Empty lines or lines that begin with a `#' are ignored, except when they follow a significant (not ignored) line that ends with a `@'.

If a significant line ends with a `@', the `@' and the newline are removed and the next input line, minus any leading white-space, is appended to the previously read line.
Note:Any white-space immediately before the `@' at the end of the line is preserved. To end a line with a literal `@' use `@@#'.
Note:When diagnostic output refers to a multiple line input command, the line number used is that of the first line.
There is no limit to the length of a line other than those imposed by the host system.

The resulting line should consist of a keyword and its arguments. The keywords are listed in the keywords(x-qvrs) item and explained in detail in their own individual item. Any embedded `@' characters in the arguments are expanded as explained in the following. A valid qvrs line consists of a keyword followed by arguments. The arguments are processed to replace embedded substrings as follows:
SubstringReplacement
@@a single `@'
@.replaced by nothing -- used to terminate variable name when followed by alphas or `['.
@0obsolete version of @. -- warning produced
# ...empty string; rest of line ignored
@#a single `#'
@NotSetreplaced by `@NotSet'
@Var¹value of Variable, `' if undefined
@Var[strAny `@'s in str are expanded to yield result. Then entire string is replaced by the value of @Var[result]. The expanded str index must not contain any spaces or tabs.
@( ...)¹The named function evaluated and result replaces sequence. Any arguments to the function are expanded prior to the call.
@<else>An `@' followed by any other character just replaced by character.
¹The @Var; @Var[str], or @(function ...) expressions may be followed by tilde-ops -- see Appendix F Qsg TildeOps. For example, @Var~v is replaced by 1 (0) if @Var is (is not) empty (i.e, void).

The name of a variable (i.e., Var in the above) may be any sequence of alphanumerics or underbars, the first of which may not be a digit.

8.8) Qvrs Command-Line Options

For a summary of the qvrs command-line options, use the command qvrs --x or see the qvrs(1) man page. For developers, the flags often used with qvrs are:

  • qvrs: to see all the variables currently defined
  • qvrs --f: to show the list of qvrs files
  • qvrs --l: to show library mapping information
  • qvrs --XI: to show the cc --I flags
  • qvrs --VD: to show value of _DestDir_
  • qvrs --V?: to get a list of the --V and --X options
  • qvrs --W <var>: to see where a variable was set or modified

8.9) The Traits System

The traits system provides yet another set of variables describing the host system. As the name suggests, these variables specify traits or characteristics of the host system such as special directories, capabilities, and special tool names. To get a list of the traits variables, use traits, as in:
    % traits | sed 5q # just list first five
    ARFLAGSDASHED   0
    BOGUS_CSH       0
    BOGUS_RENAME    1
    BuildPath       /bin:/usr/bin
    DBLESLASHROOT   0
Individual standard traits are described by x-qvrs as in:
    % x-qvrs ARFLAGSDASHED
    ARFLAGSDASHED(x-qvrs) specifies ar option preceded by dash

    This boolean traits-var specifies whether or not the
    system's ar(1) program requires the action argument to be
    preceded by a `--'. If the setting is 0, the `--' is not
    inserted into ar commands.

    See also: traits ar(1)

There are three tools in the traits system:

mktraits Actually a special version of qvrs modified to read the traits input files <qtree>/lib/traits.vrs and <qtree>/data/traits.ext and create the binary file <qtree>/data/traits/host.vrs. This is the file that any program that needs a trait, including traits, will read to retrieve traits settings. These tools will invoke mktraits to build this file if it does not exist.
traits Outputs variables of the traits database or the values for argument variables.
mkalltraits Run mktraits for all hosts for which qdsrv variable host.qtree exists.

The <qtree>/lib/traits.vrs file is provided as part of the QEF product and should be left unchanged. In contrast, the <qtree>/data/traits.ext file is used to make local host, site, or platform specific settings.

An important setting is that of QDSRV_HOST, the name of the host on which qdsrv is run, as in:

    cset QDSRV_HOST host
After changing this file, mktraits should be run to create the binary <qtree>/data/traits/host.tab on all the hosts that would be affected.

A key feature of the traits system is the automatic importation by qvrs of all the traits that are marked for export or whose name begins with "_[TFD]_".

Summary
In this chapter we have explored qvrs and the associated traits package and the ways qvrs controls and configures your software production process using variables to abstract the construction information. We looked at qvrs with respect to its use, syntax, semantics, and files. Using the variables, command keywords and qvrs files, you describe the environment in which your construction is to run. The names of tools, flags, file and directory locations and search paths are all described in qvrs files.

In many ways qvrs is the second most important tool of the QEF system and understanding its use is a large part of mastering QEF.


c070.qh - 9.4 - 03/10/27 QEF Home The Guide's Table of Contents Previous Chapter Top Of Page Next Chapter