Printable version of this document
Testbed Master Control Daemon/Client Reference
Contents
-
Introduction
The Testbed Master Control Daemon (TMCD) is a program that runs
on boss.emulab.net, and provides configuration information to
Testbed nodes when they boot up. The Testbed Master Control
Client (TMCC), is a small program that is installed on each node,
and is used to connect to the TMCD to issue requests and get the
response. In addition, Testbed nodes use the TMCC/TMCD to communicate
events of interest back to the Emulab Database and to the user via the
Web interface. The TMCD interface is text based; clients issue
requests in the form of strings consisting of a command and an
optional argument. The response is also a string, in a very generic
format that can be easly parsed by any C/C++ program or shell
interpreter. For example, to determine how to configure the
experimental interfaces on each testbed node in your experiment when
it boots, you would do the following:
tmcc ifconfig
INTERFACE=1 INET=10.0.0.1 MASK=255.255.255.0
INTERFACE=2 INET=10.0.1.1 MASK=255.255.255.0
-
TMCC
The TMCC is a simple client program that runs on the testbed
nodes and handles the details of connecting to the TMCD,
issuing the request, getting the response, and printing it out. It has
been compiled on FreeBSD 4.x, Redhat Linux 6.2 and 7.1, and Netbsd 1.4, and
should compile on just about any operating system. Alternatively, you
can integrate the TMCC into your own programs. Briefly, the TMCC
connects to port 7777 (UDP or TCP) on boss.emulab.net, writes a
single string to the connection, and then waits for an optional
response, which is a newline separated list of strings. The TMMC exits
when the other side of the connection is closed by the TMCD. The
source code for the TMCC is available upon request by sending email to
Testbed Operations
(testbed-ops@flux.utah.edu)
-
Node Setup Script
The Emulab versions of FreeBSD 4.5, Redhat Linux 7.1, and Netbsd 1.4
all run a setup script at bootup that uses the TMCC client to
configure the node. All of the interfaces are configured, user
accounts for each of the members of the project are created, NFS
mounts are made, etc. These setup scripts are located in
/usr/local/etc/emulab.
-
Command Reference
-
reboot
Report that a node has rebooted to the TMCD. This is an informational
message that is used by the TMCD to determine when a node reboots for
the first time after its disk has been reloaded. No response is
returned.
-
status
Request status information about the project and experiment that the
node is currently part of. Returns the project ID, experiment ID, and
the node name from the NS file that described the topology.
This command is typically used to determine if the setup script needs
to do any further configuration; if the node is free, then no other
information is going to be provided by the TMCD. The format of the
reply is one of:
FREE
ALLOCATED=pid/eid NICKNAME=name
-
ifconfig
Request the configuration information for each of the network
interfaces on the node, as determined by the topology described in the
NS file, and the assignment of IP addresses to interfaces that is
performed when the experiment is configured. The information that is
returned is typically converted into corresponding ifconfig
commands on the node. However, the information can be used in any
manner that is appropriate for the operating system that is running on
the node. The reply to this request is one or more lines in the
following format (in the unlikely case that the topology describes a
node with no network links, the response to this request will be null):
INTERFACE=Z INET=X.X.X.X MASK=Y.Y.Y.Y MAC=AA:BB:CC:DD:EE:FF
iface=`/etc/testbed/findif AA:BB:CC:DD:EE:FF`
/sbin/ifconfig $iface inet X.X.X.X netmask Y.Y.Y.Y
-
accounts
Request group and login account information for each of the project
members of the project that the experiment is running. This
information can be used to generate login accounts for project members
on each of the nodes in an experiment. The Emulab versions of FreeBSD,
Linux, and Netbsd all have stub password/group files that do not
contain any user accounts or groups. When a node first boots after
being allocated to an experiment, this command is used to find out
what accounts to build. The reply to this request is one or more lines of
group information, followed by one or more lines of login account
information:
ADDGROUP NAME=pid GID=XXXX
ADDUSER LOGIN=joe PSWD=ABCD UID=YYYY GID=XXXX ROOT=N NAME="Joe User" \
HOMEDIR=/users/joe GLIST=ZZZ0,ZZZ1
| LOGIN |
|
The user/account name. |
| PSWD |
|
The encrypted password string, suitable for direct
insertion into the password file. |
| UID |
|
The numeric uid. |
| GID |
|
The primary group for the user, as a numeric gid. |
| ROOT |
|
Indicates whether the user should be granted
root access by placing the user into the root group (wheel group on
FreeBSD/NetBSD). |
| NAME |
|
The full name of the user, suitable for
insertion into the gecos field of the user's password entry. |
| HOMEDIR |
|
The absolute path to be used for the home directory. |
| GLIST |
|
A (possibly null) comma separated list of auxiliary
group ids, as numeric gids. |
On Linux, this information would be converted into the following
commands:
groupadd -g XXXX pid
useradd -u YYYY -g XXXX -p ABCD -G root,ZZZ0,ZZZ1 -d /users/joe -c "Joe User" joe
-
mounts
Request the list of remote directories that need to be NFS mounted on
the node when it boots. The reply to this request is one or more lines
in the following format:
REMOTE=fs.emulab.net:/users/joe LOCAL=/users/joe
REMOTE=fs.emulab.net:/proj/testbed LOCAL=/proj/myproj
mkdir /users/joe
/sbin/mount fs.emulab.net:/users/joe /users/joe
mkdir /proj/myproj
/sbin/mount fs.emulab.net:/proj/myproj /proj/myproj
-
rpms
Request the list of RPMs that should be installed on the node when it
boots, as specified in the NS file on a per-node basis. The reply to
this request is null if there are no RPMs to install, or one or more
lines in the following format:
RPM=/path/to/name.rpm
rpm -i /path/to/name.rpm
-
startupcmd
Request the name of the startup script (or program) that should be run
when the node boots, as specified in the NS file on a per-node basis.
The reply to this request is null if a startup script was not
specified, or a single line in the following format:
CMD=/path/to/runme UID=joe
-
startstatus
Report the numeric exit value of the startupcmd back to the TMCD so
that it can be recorded and displayed in the "Experiment Information"
Web page. In fact, this does not have to be the result of the
startupcmd, but can be the result of any application program. The
intent is to report back information that can be used by the
experimentor to determine when the experiment has finished. Each node
reports back status individually. The format of this command
is:
tmcc startstatus XX
-
ready
Report an application level ready status back to the TMCD so
that it can record it. A count of nodes (in your experiment) reporting
ready is maintained by the TMCD, and is made available to nodes via
the readycount request below. There is no response from the
TMCD to this command.
-
readycount
Request the count of nodes that have reported in ready with the
ready command above. This is an application level count;
nodes can use this as a very primitive form of synchronization to
determine when all of the nodes in the experiment have started the
application (say, via the startupcmd above) and have reached
a point where it is necessary to wait until all of the nodes have
reached the same point. The reply to this request is a single line in
the following format:
READY=N TOTAL=M
-
hostnames
Request information about the IP addresses and node names of all of
the nodes in the experiment. The intent it to provide the ability to
easily generate a suitable /etc/hosts file that allows
experiments to operate using the symbolic names of the nodes (as
defined in the NS file), instead of IP addresses, which are generally
assigned by the configuration software, not the experimentor. Since
nodes can use multiple experimental interfaces, the reply gives the IP
address for each interface on each node. Additional aliases can be returned
for nodes that are directly connected to the node making the hostnames
request, or that the requesting node has a route to. Secondary interfaces,
and interfaces that are not directly connected are named with a suffix
indicating the link or LAN that interface belongs to. For historical
reasons, we provide an additional alias in the form 'node-X' where X is
the ordinal number of the interface. The reply to this request is one or
more lines in the following format:
NAME=nodeA-linkX IP=X.X.Y.A ALIASES='nodeA nodeA-0'
NAME=nodeB-linkY IP=X.X.Y.B ALIASES='nodeB nodeB-0'
NAME=nodeC-lanZ IP=X.X.Z.C ALIASES='nodeC-0'
X.X.X.A nodeA-linkX nodeA nodeA-0
X.X.X.B nodeB-linkY nodeB nodeB-0
X.X.Z.C nodeC-lanZ nodeC-0
-
log
The log command can be used by an application to write a
message to a log file on users.emulab.net. This is especially
useful on the Sharks, most of which do not have console serial lines
attached. The argument to the log command is a single string, in
double quotes if operating within the shell:
tmcc log "This is a log message"
|
