contiki-os/contiki — pull request #52 by adamdunkels (2012-12-10 13:09:23+00:00)
Title: New regression test framework

The travis integration with the current Cooja tests is awesome, but the current testing framework doesn't make it easy to test code before pushing to github.

This pull request contains the regression test framework from Thingsquare Mist (http://www.thingsquare.com/mist/), ported to Contiki, along with a number of new tests that exercise the Rime and IPv6/RPL code.

The regression test framework consists of a new directory, regression-tests/ which contains a bunch of lightweight makefiles that allow you to simply run "make" in the regression-tests/ directory to run through all tests before submitting a pull request.

The regression-tests/ directory contains a set of subdirectories, named as follows:

01-compile/
02-hello-world/
03-base/
04-rime/
05-netperf/
06-shell/
07-elfloader/
08-collect/
09-ipv4/
10-ipv6/
11-rpl/
12-ipv6-apps/

The directory names start with two digits to create an explicit ordering. These subdirectories make it possible to do iterative regression testing. Say that you are working on the RPL code. You can then go into the 11-rpl/ subdirectory and run "make" to ensure that the regression tests for RPL work, without having to run all the other tests that aren't relevant to what you are working on (such as the multithreading test).

Each subdirectory contains either a compile test or a simulation test. Compile test directories contain a Makefile that lists the projects that should be compiled, and the build target platform. The current compile tests can be run in the latest Instant Contiki. Simulation test directories contain Cooja csc files, who's names also start with two digits. For example, the 02-hello-world/ directory contains:

01-multithreading.csc
02-sky-coffee.csc
04-sky-checkpointing.csc
Makefile
x03-crosslevel.csc

The files that start with two digits are run, in order, when running "make" in the directory. The file named x03-crosslevel.csc has been disabled (by putting an x before the digits) since it was unstable when I was testing it.

When run, the regression tests print out either OK or FAIL depending on whether the tests work or not.

This pull request also contains the travis integration files that make it so that all regression tests are run when a pull request is issued.

The pull request also contains a set of new tests for Rime and RPL/IPv6. Specifically, the new RPL tests test:

* Packet forwarding towards the root
* Packet forwarding from a leaf node to another leaf node
* That the RPL network is stable when the root reboots
* That a RPL network works for 28 hours
* That a large (400 nodes) RPL network is able to forward packets to the root even when neighbor and routing tables are exhausted