The Ohio Network Emulator (ONE) application emulates a network cloud between a pair of network interfaces. Various effects simulating those found in the real world can be applied to traffic flowing between the two interfaces. A report on an earlier version of ONE can be found here. Capabilities have been enhanced since that report was issued. This manual describes version 2.0 of ONE. The emphasis in this document is on how to use ONE.
ONE lives at http://irg.cs.ohiou.edu/one/. Source code, documentation and other information related to ONE can be found at this address.
Requirements are: A Sparc running a recent version of Solaris. Development was done under 5.7, so anything later should be OK. ONE performs best if the host it is running on is fairly fast. Available physical memory will constrain queue sizes - causing the host to swap during a run will likely invalidate any data gathered during that run. General guideline: to emulate 10BaseT order of magnitude speeds, at least an Ultra-2.
The host must have at least two network interfaces, and preferably should have two dedicated interfaces. The network segments attached to each of the interfaces should be private and entirely allocatable to the emulation - otherwise unaccounted for traffic can affect the run
The host should be able to be dedicated to running ONE to the exclusion of all else during a run. ONE is CPU intensive. More importantly, ONE needs to have the best possible chance of transmitting packets as close to the time it calculates they must be transmitted as possible in order that data gathered during a run be correct.
To compile, cd to src and type make. The makefile assumes the presence of gcc. If you are using another compiler, you will have to edit src/makefile accordingly.
Administrative (root) access is required for installation, although not necessarily for running (see below)
The network segments on each of the two interfaces need to be set up to generate traffic according to your research needs.
To assist in correlation, it may be desirable to closely synchronize the times on all the involved hosts prior to a run via a facility like NTP. Running an NTP daemon during a test may be necessary, but also could adversely affect the results (as noted below).
Take steps to ensure other running processes won't interfere. The host should be quiescent. Network services (ntpd, inetd, ...) perhaps disabled.
Other considerations apply when deciding how to maintain good time synchronization. Two ways to do it using the udel ntp suite are: jump the clock by running ntpdate immediately prior to running the test, and running an NTP daemon locally. The latter method works well, particularly if the daemon is allowed to run for extended periods prior to the test. If running the daemon causes undesirable traffic on the test network, this has to be taken into consideration.
We experienced different problems using the two different strategies. When using ntpdate we found drift to be a problem during extended tests. As mentioned before, during a configuration where the test network was used for other purposes, undesired ntpd traffic was a problem. The correct solution is to use dedicated interfaces for the test traffic.
Increasing the default Solaris clock tick rate may improve timing accuracy and reduce jitter. The default rate is 100 Hz. This gives a granularity of 10 ms. The Solaris clock tick rate can be increased to 1000 Hz by adding the line:
set hires_tick = 1
to the /etc/system file and rebooting.
Variable propagation delay configuration is in part done via a perl script. Perl must be present on the host system if variable propagation delay is to be used.
The ONE tar file can be unpacked anywhere convenient to the user. The directory it is unpacked to should be added to $PATH. The executable (named onemust be run as effective UID root, as it must access normally protected system resources (specifically, it must be able to access network interfaces promiscuously, and it needs to place itself on the real time run queue). In order to access these resources one of two events has to happen: the user running the application must be logged in as root, or the program itself must be installed setuid root. The usual security caveats apply.
If variable propagation delay is to be used, the one executable must be able to locate the one-read-raw.pl script. Place the script somewhere in a $PATH directory.
Prior to running ONE, a suitable configuration of the to-be-emulated network must be prepared. The configuration definition is placed in a file, the name of which is specified when ONE is invoked.
ONE is invoked as:
one [-dn] config-file
where -dn is an optionally supplied switch specifying that debugging information be displayed during a run. This option is synonymous with the verbose option that can be specified in the configuration file (as described below). The config-file parameter, which must be supplied, is the name of the file describing the configuration to be used for this run
Items in the configuration file are each specified on a separate line. The format is keyword: value where keyword specifies a configuration item. Comments may be placed in configuration files by starting comment lines with #. The comprehensive list of configuration items is:
- interface: name
- Specify the name of one of the pair of interfaces. Two interfaces must be specified, no more, no less. The interface names must match the names of existing network interfaces on the emulation host (e.g. hme1).
- linespeed: value traffic-units | infinite
- Specify an upper limit on the rate at which data can flow out a given interface. There must be a pair of these, one following each interface specification. The parameter can be either an integer number, representing the maximum speed in traffic-units per second, or the identifier infinite, indicating no upper limit exists for this interface. The identifier traffic-units is described below.
Specifying this limit can affect the flow of data through that interface, causing it to queue as the interface becomes congested. The inherent capabilities of the interfaces still apply - emulating 100BaseT speeds via 10BaseT interfaces isn't likely to yield useful results!
- memunit: value traffic-units
- The internal buffer size (memory allocation granularity) used to store packets in the queue.
- qsize: value traffic-units
- Specify the size of the output queue associated with an interface. There must be a pair of these, one for each interface. The parameters value, an integer, and traffic-units specify the queue size to ONE. ONE allocates an appropriate amount of memory. No checks are done to ensure sufficient physical memory is available, so some attention to this detail is necessary.
- red-threshold: low-value high-value units-designator
- Enable random early detection (RED). RED is an algorithm used to ensure fair access to network bandwidth in situations where congestion is causing packets to be dropped. If not enabled, ONE will discard new traffic if the queue the traffic is bound for is full.
When RED is enabled, a pair of parameters must be supplied. The pair of numbers represent queue fullness percentages and can be thought of as the points at which a zero and a 100 percent chance that a random packet will be dropped from the outbound queue for each incoming packet.
For example: red-threshold 75 95 b is interpreted as: when this output queue is less than 75 percent full do not discard any packets. When the queue increases above 75 percent, using a linear probability (zero percent chance at 75 percent queue full to 100 percent chance at 95 percent queue full) to determine if a packet should be dropped. If a packet is to be dropped, pick one randomly from the output queue.
The units-designator is p or b to indicate packets or bytes respectively.
- ber: value
- Specify a bit error rate (BER). If specified, pseudo-noise will be injected into the traffic stream. The configuration item allows for a simple specification of noise which will cause random single bits to be obscured at the specified probability. The parameter is a number representing the probability (0.0 to 1.0) that a given bit will be obscured by noise. This item is specified per interface.
- propagation: value [ variable file ]
- Specify the propagation delay that is to be applied to a stream of traffic before it can be transmitted through the output interface. In the implementation, traffic is queued on the output queue, but is not transmitted until the specified delay interval has passed. Caveats about appropriate queue size vs. having traffic discarded apply. The parameter value is a floating point number and represents delay time in milliseconds. If the optional parameter variable is specified, the specified file is read to obtain the variable propagation delay values. The variable propagation delay values are added the fixed value. Variable propagation delay is useful in instances like configurations of satellites in a network, where propagation delay is not fixed. If variable propagation needs to be specified, the input format is described below.
- verbose: level
- Enable verbose status reporting. Additional information is displayed and logged if verbose is enabled. The parameter level can range from one to ten, with higher levels resulting in increasing amounts of logged information. A caveat: logging large amounts of information may adversely affect the veracity of a run, as the act of logging consumes resources.
- 0 - dump counters
- 1 - dump cache, interface information, Ethernet cache
- 2 - dump print queue
- 3 - dump qfull
- 4 - dump printdrops
- 5 - dump qfull2
- 8 - dump delays
- drawplots
- logfile: name
- Specify the name of the file to which the logs will be written. The file is overwritten each run, so if the information within the file is to be saved, it will have to be copied elsewhere.
- lockfile: name
- Specify the name of the lock file - a file used to help ensure ONE cannot be run more than once simultaneously on a given host. On occasion, if ONE isn't terminated gracefully, the lock file might have to be manually removed.
- pidfile: name
- ONE writes its PID in this file while running.
- delayfile: name
- Specify the name of the file to which the delay results are to be written. Note that the verbose level must also be set to eight or greater for this to happen.
- drop
- Currently, this item has no effect.
The parameter traffic-units is an identifier comprised of one of the following optional characters:
- K - 1024 (210)
- k - 1000
- M - 1048576 (220)
- m - 1000000
followed by one of:
- B - bytes/octets
- b - bits
A sample configuration file:
# Satellite Emulation
# An example configuration for a host that has two dedicated
# interfaces, le0 and le1. The verbosity of diagnostics is set to a
# relatively quiet level (one); only counters, cache, interface
# information, and the Ethernet cache will be dumped.
#
# propagation: ms
# the propagation delay in milliseconds
# qsize unit: size of the outgoing packet queue
# can use units below, no units=bytes
# linespeed unit: variable
# can use units below, no units=bytes/second
# a linespeed of "infinite" means go as fast as possible
# memunit: memory allocation size for buffering packets, same units
# as linespeed (1K means alloc packets in 1K allocation units)
# verbose: number for 0 to 10, 0 is quiet, 10 is noisy
# --------------------
# values that take args in bits/bytes can use the following units:
# K=1024, k=1000, b=bits, B=bytes, M=1024*1024, m=1000*1000, s=ignored
# so 1 Kbs = 1024 bits/second, and 1 kBs=1000 bytes/second
verbose: 1
logfile: /tmp/bridge.log
pidfile: /tmp/bridge.pid
delayfile: /tmp/bridge.delay
memunit: 1024 B
drawplots: 0
# front side
# The maximum rate at which data can exit this interface is set to
# 768000 bits per second or 96000 bytes per second. The queue size is
# limited to 85 KB (87040 bytes). Propagation delay is set to 50
# milliseconds; traffic entering this interface will be delayed at
# least 50 milliseconds before exiting the other interface. Random
# Early Detection is enabled and is set to start discarding random
# packets when the transmit queue is 70 percent full and to discard on
# a one for one basis if the queue reaches 95 percent full.
interface: le0
linespeed: 768000 bs
qsize: 85 KB
propagation: 50
drop: no_drop
red-threshold: 70 95 p
# back side
# Configured pretty much symmetrically, with the exception RED is not
# enabled. If the queue becomes full, new packets will be dropped.
interface: le1
linespeed: 768000 bs
qsize: 85 KB
propagation: 50
drop: no_drop
A variety of example configurations can be found in the config directory.
The variable propagation delay input file format is that of the output from the Satellite Tool Kit application (STK). The format is a series of records, one per line. Fields within a given record are separated by whitespace.
An example from a variable propagation delay file:
1 Mar 1999 00:26:14.00 228.443 -81.139 44708.216092
1 Mar 1999 00:27:14.00 226.128 -80.972 44413.993235
1 Mar 1999 00:28:14.00 223.845 -80.819 44113.620746
1 Mar 1999 00:29:14.00 221.587 -80.682 43808.004479
...
The first four fields represent the date, and the last (seventh) field represents the distance in kilometers the satellite is from the observer (in this case the antenna the network interface represents). The other two fields are not used by ONE.
Runs using variable propagation delay are based on the start time of the run being treated as the start time of the delay data in the propagation file. Thus, for example, using the above data, the delays imposed on traffic by ONE one minute into a run would be based on the second record, and so on. When ONE reaches the end of the variable delay propagation table the run is terminated. No interpolation is done of delays between data points supplied in the table.
The propagation delay information is stored as a table internally. Each entry consumes around 12 bytes. Choice of an appropriate interval between time data points is a trade off between maximum acceptable jumps in the delay value, and the size of the table (specifically, how the size impacts the amount of available memory). Table size has no significant effect on compute resources
Zones of exclusion (ZOEs) are determined based on discontinuities in the delay table, that is, when there is a step in the time of greater than one minute. During the time covered by a ZOE occurrence, no traffic is forwarded.
An example variable propagation delay data set is the file config/one-delay.raw.
Sniff the test networks. Verify there's no unexpected traffic. Also check that there are no unexpected processes or resource consumption on the ONE host or the traffic generating hosts
Run a test with delay set to zero. In this mode, ONE forwards all traffic immediately. None of the delay algorithms come into play. If the jitter diminishes, some tuning of ONE's internals may help. ONE estimates the clock tick quantum and adds in a fuzz factor of +/- ten percent. Currently this is done by tweaking the source code. If this turns out to be a recurring operation, the capability may be added to the run time configuration.
Run the problematic test with the ONE host configured to route traffic between the two interfaces. Recalling that ONE acts as a bridge, some additional considerations apply. You will need to have the traffic generation peers (and the subnets they reside on) set up such that they can route via the ONE host.
To enable routing, after configuring all the interfaces and building the routing table, issue the command:
ndd /dev/ip ip_forwarding 1
Be aware this will forward traffic through all the interfaces, so be careful in setting up the routing table.
Shawn Ostermann
School of Electrical Engineering and Computer Science
Ohio University
Athens, OH 45701
ostermann@cs.ohiou.edu
Mark Allman
School of Electrical Engineering and Computer Science
Ohio University
Athens, OH 45701
mallman@grc.nasa.gov
Jim McKim
M/S 142-1
NASA Glenn Research Center
Cleveland OH 44135
mckim@grc.nasa.gov
A mail list has been established to facilitate communication among ONE users. Send email to majordomo@lerc.nasa.gov. In the text body include the line:
subscribe one-users
to subscribe to the list.
If you would like to contribute to ONE development, subscribe to the one maintainer's list by requesting:
subscribe one
in the email to majordomo.