TeaCP: a Toolkit for Evaluation and Analysis of Collection Protocols in Wireless Sensor Networks

About TeaCP

TeaCP (pronounced as teacup) is a prototype Toolkit for the evaluation and analysis of Collection Protocols in both simulation and experimental environments running on TinyOS. The toolkit consists of a testing system, which runs a collection protocol of choice, and an optional SD card-based logging system, which stores the logs generated by the testing system. The SD card datalogger allows a wireless sensor network (WSN) to be deployed flexibly in various environments, especially where wired transfer of data is difficult. Using the saved logs, TeaCP evaluates a wide range of performance metrics, such as reliability, throughput, and delay. TeaCP further allows visualization of packet routes and the topology evolution of the network, under both static and dynamic conditions, even in the face of transient disconnections.

TeaCP has the following features:

• Generality — TeaCP is a general toolkit designed to plug in various collection protocols and analyze their performance through post-experiment analysis.
• Configurability — With TeaCP, one can easily configure tests for evaluating a data collection protocol, run the tests and analyze the underlying performance (e.g., reliability, throughput, and delay) via both real experiments and simulations.
• Visualization — TeaCP provides visualization of packet routes, network topology and other statistics, that can be used for understanding, analyzing and diagnosing the behavior of collection protocols.
• Flexibility — The SD card datalogger provides the convenience to test the collection protocol in various environments.

Downloading TeaCP

TeaCP is available on GitHub.

TeaCP has been tested under the following system conditions:

• Ubuntu 12.04
• TinyOS 2.0.1
• Python 2.7

TeaCP also resorts to the following Python packages for statistics calculation and visualization:

• Numpy
• Matplotlib
• PyDot

TeaCP structure

TeaCP operates at the application layer, and consists of the test configuration component, the application layer component for generating and handling packets, and the post-experiment analysis component. The test configuration component and the application layer component run on sensor motes. The post-experiment analysis component runs on a PC.

The configurable test parameters include the packet generation interval, the radio power and the channel for the network. The radio power and channel settings are applied to the radio component of the hardware. The packet generation interval is applied to a periodic timer. The firing signal of the timer (i) increments the packet counter by one, (ii) invokes the packet generator to generate a data message (encapsulated in a packet) with packet ID equal to the packet counter, (iii) drives the log generator to output a log message, and (iv) transfers the packet to the network layer, which moves it towards the root using a collection protocol. Since the collection protocol may use multihop routing to accomplish the delivery, the packet may be received by intermediate nodes and the root. No matter whether the node is a relay or the root, the network layer will notify the packet reception handler when it receives a packet from other sensors. The packet reception handler prepares the information needed by the log generator, which outputs log message to storage through serial communication. Finally, post-experiment analysis is done based on the data saved in storage (e.g., the SD card datalogger).

In the following, we show how to use TeaCP to obtain logs from experiments and simulations and analyze the performance of collection protocols using the TeaCP analysis component.

Using TeaCP in experiments

TeaCP can be used in experiments to evaluate and analyze the performance of collection protocols. In an experiment, a number of sensor nodes are deployed in the testing environment. TeaCP utilizes an activator to send out a broadcast message, which contains the test parameters and establishes clock synchronization. Once the sensors have received the broadcast message, they apply the test parameters to their system. Then they start to generate data packets and the whole data collection network starts to run. While the data packets are routed around in the network, each node will generate log messages, recording information about events happening in the network, such as when a packet is generated or received. These log messages are stored by the data logging system as log files. Finally, TeaCP analysis component can calculate the test statistics based on the log files.

1. Configuring a test

The test parameters are configured in the header file test_config.h under the directory exp/nesc/:

TeaCP provides the following configurable test parameters:

• Traffic type — the traffic pattern in which the packets are generated at each source node. The current version of TeaCP has implemented two kinds of traffic types: one is periodic packet generation (#define PERIODIC_TIEMR) and the other is emulating Poisson packet arrival process (#define EXPONENTIAL_TIMER).
• Packet generation interval — the time interval between two consecutive packet generations at each source node. If the traffic type is periodic packet generation, then the packet generation interval is a constant with value PACKET_INTERVAL. If the traffic type is Poisson packet generation, then the packet generation interval is an exponential random variable with expectation equal to PACKET_INTERVAL.
• Radio power — the transmission power when a node transmits a packet or a beacon message. For the CC2420 chip, radio power ranges from -25 dBm to 0 dBm.
• Radio channel — the IEEE 802.15.4 radio channel on which the collection network operates. The 2.4GHz ISM band is divided into 16 non-overlapping ZigBee channels, with channel ID from 11 to 26.

After we have configured the parameters for a test, compile and install the TinyOS program onto the activator:

2. Installing TeaCP application layer component

Next we compile the TeaCP application layer component and the collection protocol and install onto the nodes.

The last command should be repeated to install the code onto the sensors with different node IDs (by specifying reinstall.[TOS_NODE_ID])

3. Attaching the data logging system

We then attach the data logging system to each node and deploy the nodes in the testing environment. For the data logging system, you can either use our implemented SD card datalogger or connect each node to a PC through USB port for data logging. If you use the USB datalogging system, you need to run the serial listener to receive the serial messages from the sensors:

Here we save the received serial messages into the file with extension .log. This is intentional because our TeaCP analysis component identifies a file as a log file by looking at the file extension.

4. Starting and finishing a test

Press the user button on the activator node to send out a broadcast signal and the experiment starts.

When the experiment finishes, press the user button on the activator node for five more times and the activator will send out a broadcast signal to tell the sensors to reset. Then collect the log files from the dataloggers for post-experiment analysis. If you are using the USB logging system, you need to press CTRL+C on the keyboard to terminate the serial listener.

A log file consists of a list of received serial messages, which contains a log message recording information with regard to network events:

The format of the received serial message is explained in the figure:

The first few bytes of the received serial message are the header added by the TinyOS serial communication protocol. The rest of the serial message is the log message generated by a sensor recording information about a network event. For example, the log message in the figure tells us that the packet (with ID 0x00000a79 generated by source node 6) arrives at node 3 in three hops and the last hop is node 8.

Using TeaCP in simulations

The procedure of using TeaCP to run TOSSIM simulations is basically the same as in experiments. The only difference is that the activator is not needed in the simulation to send out a broadcast message configuring the network. In simulations, the configured test parameters are applied by compiling them into the code of sensors.

1. Configuring a test

The same as experiments, we configure the packet generation interval in test_config.h under sim/nesc/. However, the configurations of radio power and radio channel in the header file are not applicable any more. Instead, the TOSSIM simulation uses a different way to configure the link conditions between nodes. TeaCP includes the configurations in the simulation script, e.g., static.py under the directory sim/test_scripts/:

The file link_filename specifies the link gains between nodes in the network. The file noise_filenamespecifies the noise trace at each node.

2. Compiling TeaCP application layer component

After we finish the configuration, compile the TeaCP application layer component:

This compilation generates the Python code of the TOSSIM simulator, Tossim.py.

3. Running a test

Before running the test, let me explain another two parameters in the configuration list. The parameter protocol denotes the collection protocol that the simulation runs on and is actually the folder name where the script can find the TOSSIM simulator (Tossim.py). By modifying the parameter finish_time, you can specify the total simulated duration of the test in units of seconds.

Now we can run the simulation by:

There are two simulation scripts under sim/test_scripts/: static.py and dynamic.py. The difference between these two scripts is that static.py runs simulations on static link gains while dynamic.py runs simulations on time-varying link gains. For more details, please take a look at the example traces under the directory sim/config/linkgain/.

When the simulation is done, the log file (as specified by log_filename in the parameter list) is generated:

The log messages from simulations have the same fields as the log messages from experiments.

Using TeaCP with the SD card datalogger

We have implemented a SD card datalogger using the Arduino Pro Mini board. The Arduino Pro Mini board plays the role of SPI adapter for transferring the serial messages to the SD card.

The wiring between the TelosB mote and the Arduino board is shown in the figure:

The code that runs on the Arduino board is sd_card_datalogger.ino under the directory exp/arduino. It can be installed onto the Arduino board through IDE.

Using the TeaCP analysis component

Once you have obtained log files from experiments and simulations, you can use the TeaCP analysis component to evaluate collection protocols. The TeaCP analysis component provides a comprehensive set of functions to analyze the performance of collection protocols. These functions include analysis of delay performance, throughput performance and reliability performance. The TeaCP analysis component can also visualize packet routes and network topology.

The TeaCP analysis component is packaged in TeacpAnalysis.py, under both the directories exp/tools/and sim/tools/. Under the directories, you can also find a script, run-analysis.py, which is a simple example using the TeacpAnalysis package.

In run-analysis.py, first we import all the log files (with file name extension .log) from the specified directory:

The second argument specifies whether the log files are obtained from experiments or simulations (since their log file formats are different).

Then we calculate all the statistics and save the results into a file:

 
analysis.calc_all()
analysis.print_result_to_file()

You can find the results in the file result.txt under the directory ../log/example/ctp/output/:

We generate the delay histograms and visualization of network topology by the last two functions:

The output are shown as follows:

TeaCP uses the pydot package to generate the images of network topology. In the network topology images, the locations and colors of the nodes are configured by reading a file named animation_config in the same directory as the log files. If the file animation_config is not found, the TeaCP analysis component will assign values to the locations and colors randomly.

In addition to the statistics and output shown above, the TeaCP analysis component also provides other functionalities such as calculating median delays and analyzing packet paths (even when duplicates of packets exist in the network).

Running a collection of simulations

We have described how to use TeaCP to run a test in simulations step by step. We also provide a script to conveniently run a collection of simulations. For example, you might want to run the simulation with a parameter set to a list of different values. The script is run-simulation.py under the directory sim/.

It provides the following configurations:

The script will iterate over the packet rates in pkt_rate_list. For each packet rate, the script will run the simulation for five times (as specified by simulation_times). This can be useful for obtaining confidence intervals on the statistics. To run the script:

Conclusion

In this tutorial we have shown how to use TeaCP in both experiment and simulations to evaluate performance of collection protocols. The ultimate goal of TeaCP is to let users conveniently compare multiple collection protocols in the same testing environment. If you are a collection protocol developer and you have implemented a collection protocol, you can simply include your library path into the makefile of the TinyOS program and compile. However, TeaCP does require your collection protocol provide the three necessary interfaces: the Send interface, the Receive and the Intercept interface.

The TeaCP analysis component relies on the clock accuracy of sensors to provide accurate statistics. It has been shown that for the same type of sensors, the clock drift is negligible for a short duration of test. However, it would be useful to incorporate a clock synchronization protocol to more accurately record the log messages. This would be our future work to extend the toolkit.

References

• TinyOS Home Page
• TinyOS Documentation Wiki
• Tmote Sky datasheet
• Arduino Pro Mini schematic
• Graphviz – Graph Visualization Software
• matplotlib: python plotting — Matplotlib 1.4.3 documentation
• Wei Si, Morteza Hashemi, Idan Warsawski, Moshe Laifenfeld, David Starobinski, and Ari Trachtenberg, “TeaCP: a Toolkit for Evaluation and Analysis of Collection Protocols in Wireless Sensor Networks,” IEEE COMCAS 2013.