ENVSET

The envset files' entries look like the following:

label	description of the entry
    # set export variable Var to value
    eset Var 	value

    # unset Var1 Var2
    unset Var1 Var2

    # include envset path
    +path

    # switch based on host name
    switch $[Host]
    case kent
	set dt_home	/u/dt
    case *
	set dt_home	/home/dt
    endswitch
    function qd qdtmp=$(qds -d $*) && cd "$qdtmp"; unset qdtmp
    alias jo  builtin jobs -l

The envset settings can incorporate environment variables, values from the traits database for the host, and other settings made within the envset database. Of particular importance are the sets used by qremote. When a command is started on a remote host using rsh there is little or no environment variables and the default $PATH is in place. The qremote usually needs to set the variables PATH and QTREE and sometimes LD_LIBRARY_PATH and various licence variables.

CUSH

cush simply runs the argument command, which should consist of a program and arguments to that program. Flags and options allow command line specification of the setting and unsetting of environment variables, selection of an envset set, redirection of I/O, chdirs, special process management, and process attributes often not available or not easily done using the standard shells.

% cush -ddir -uVar -Eset -ofile ls
# run ls after unsetting $Var, setting envset set,
# chdiring to dir and redirecting output to file

cush may seem to just duplicate functionality provided by the shell, but it does facilitate things that cannot be done using the shell. In fact its name is an acronym for cannot use sh. For one thing, the cush may be being run remotely or as another user, thus the chdir and redirections must be done within the environment of the cush process, not the invoking shell. It also avoids the problems of the variations in the shell command syntaxes and limitations. Some shells do not provide a facility to unset a variable.

cush is used extensively within qef scripts to run commands to do special manipulations or redirections without invoking a shell process to do the interpretations. This results in a large saving as invoking a shell just to run a single command is excessively costly.

QREMOTE

qremote is used to run commands remotely, if necessary. Basically qremote creates and executes a shell command that executes the argument command on the appropriate host after the necessary setup (e.g., the setting of $QTREE and $PATH). If the command is to be run locally, qremote will execute:

<Host-Q-Tree>/bin/cush -Q -E env-set -d dir [go -d] command

If the command is to be run remotely, qremote will execute:

<remote-sh> host <Host-Q-Tree>/bin/cush -Q -sHOME=<home-dir>
	-E env-set -X<exit-status-index>
	-d dir [go -d] command

<remote-sh>

will be ~/.qtree/qremote/<host>, $QTREE/data/qremote/<host>, or rsh, rcmd, or remsh (whichever is provided).

<Host-Q-Tree>

the location of the target hosts's QTREE, as named by qremote q argument, the qhost returned value, or /usr/qtree.

<home-dir>

the user's home directory on the remote host as returned by qhost. It is necessary to have cush set this variable as some systems do not set $HOME correctly if rsh is invoked with an argument command.

-E env-set

The named envset should set $PATH, $QTREE and other necessary environment variables. The envset can be specified via qremote's E flag or the qvrs variable QremoteEnv. The default is "qremote".

<exit-status-index>

rsh does not return the exit status of its argument command (a very long-standing bug). To compensate for this deficiency, qdsrv provides a mechanism whereby qremote can request an exit-status index from qdsrv. This index is then specified as a -X flag to cush which will then send the exit status of its argument command to qdsrv. qremote then can retrieve the exit status from qdsrv and exit with the same status as the argument command.

So what's the big deal? Assuming we have logged onto a different machine and we want to build our cook project subdirectory ctoe:

% qremote -v -g -46/ctoe qef  # v verbose; g to go it
# rsh gobo /p/qtree/9.1/linux2_0i/bin/cush -Q -E qremote -d
        /g/dt/cook/obj/ctoe go -d qef
/g/dt/.qtree/goes/gobo/06233a,g
# output is name of go file

% g -d
E 0   2003/07/21 05:03:26 2003/07/21 05:03:27
CWD: /g/dt/cook/obj/ctoe
CMD: qef
#{ @gobo 2003/07/21 05:03:26
# QEFHALTFILE: /g/dt/cook/obj/ctoe/%qef06234b.hlt
+ cc_g -o ctoe ctoe.o ctoey.o ctoel.o -L/g/dt/cook/dest/lib \
        -L/p/qtree/9.1/linux2_0i/lib -ldt \
        /p/qtree/9.1/linux2_0i/lib/libl.a
#} E0 gobo@/g/dt/cook/obj/ctoe 2003/07/21 05:03:27(1)

% qd -46 ctoe  # chdir to project cook ctoe
% rm ctoe
% qremote -v qef
# rsh gobo /p/qtree/9.1/linux2_0i/bin/cush -Q -E qremote -d
        /g/dt/cook/obj/ctoe qef
#{ @gobo 2003/07/21 05:12:12
# QEFHALTFILE: /g/dt/cook/obj/ctoe/%qef06272b.hlt
+ cc_g -o ctoe ctoe.o ctoey.o ctoel.o -L/g/dt/cook/dest/lib \
        -L/p/qtree/9.1/linux2_0i/lib -ldt \
        /p/qtree/9.1/linux2_0i/lib/libl.a
#} E0 gobo@/g/dt/cook/obj/ctoe 2003/07/21 05:12:13(1)

As should be obvious, qremote makes building on remote systems easy. But an even more powerful and potentially important qremote feature is discussed in the next section: Distributed Processing.

JOSH

A josh file consists of lines of the following form:

# comments	# say no more
X=value		# assign a value to variable X
group ~ pats ... # specifies that `group' implies jobs
		# matched by specified patterns.
labels ...: commands
		...
		# the jobs to be done; commands may
		# be continued over multiple lines
		# and incorporate variable values
		# and command line arguments.

One extremely important task performed by josh is to facilitate remote executions of qef jobs. The following are lines from the author's josh file:

# run Xterm on matt
Matt: ! Matt Xterm
	xterm -geom 80x6+520+500 -T "Matt(sun5_4) \
		-e rlogin matt &

# qdsrv path numbers of qtree 9.1 product
Pq84q = 585 586 587 588 589

q84q: ! lash qremote qef for 9.1 qef (not detached)
	lash -o 'qremote -%1 qef -d @D @*' @(Pq84q)

goq84q: ! lash qremote qef for 9.1 qef (detached)
	lash -o 'qremote -g -%1 qef -d @D @*' @(Pq84q)

Given the above:

% josh Matt  # fire up Xterm for matt

% josh -Dcmd goq84q - Install  # build Q-Tree cmd everywhere

The latter command should be explained further. josh runs the lash command which will look like:

lash -o 'qremote -g -%1 qef -d cmd Install' 585 586 587 588 589
# @D replace by -D arg cmd; @* replaced by josh arg Install
# @(Pq84q) replaced by internal setting 585 586 587 588 589

lash will then execute its first argument once for each of the second and subsequent arguments, replacing the `%1' by the argument, thus lash will execute:

qremote -g -585 qef -d cmd Install
qremote -g -586 qef -d cmd Install
qremote -g -587 qef -d cmd Install
qremote -g -588 qef -d cmd Install
qremote -g -589 qef -d cmd Install

Each of those qremote commands will execute:

remote-sh \			# rsh or host specific file
	host \			# named by qvrs BuildHost
	host-q-tree/cush \	# host-q-tree retrieved from qdsrv
	-Q -sHOME=<home> \
	-E env-set \		# env-set named by qvrs QremoteEnv
	-d dir \		# dir retrieved from qdsrv
	go -d qef -d cmd Install

for example:

rsh gobo /p/qtree/9.1/linux2_0i/cush -Q -E qremote9.1
	-d /p/qtree/linux2_0i/o9.1 go -d qef -d cmd Install

Now when you have power like that you must remember not to abuse it. Given qef typically runs three jobs in parallel, the above command is going to somewhat eat into your network and file server making life unpleasant for your co-workers.