dgoodwin/tito — pull request #112 by jumanjiman (2014-03-12 21:00:44+00:00)
Title: add containerized test harness

### Docker test harness

To run all tests on all supported platforms,
install docker-io, start the docker service, then
run from the root of the project:

hacking/runtests.sh

If any test fails, the script exits non-zero.
To get a zero exit status, all tests must pass
or be skipped.

The above script runs a test harness based on
docker containers and requires sudo access.
The above script will take several minutes to
run on the first build (or if you remove the
images).

Expected output resembles:

-snip copious output-
=====================
Summary
/tmp/titotest-centos-5.9-python.out : OK (SKIP=1)
/tmp/titotest-centos-6.4-python.out : OK (SKIP=1)
/tmp/titotest-fedora-20-python.out : OK
/tmp/titotest-fedora-20-python3.out : OK

You can then review the output, such as:

$ grep SKIP: /tmp/titotest-*.out
/tmp/titotest-centos-5.9-python.out:... SKIP: git-annex is not available in epel-5
/tmp/titotest-centos-6.4-python.out:... SKIP: git-annex '3.20120522 ' is too old

After you run the test harness the first time,
you can optionally create and enter a container like so:

```
.--- remove container when done (not image)
|
| .--- interactive
| | .--- tty
| | | .---- mount current workdir into container
| | | | .---- name of image
| | | | | .-- get a shell
| | | | | |
sudo docker run --rm -i -t -v $PWD:/home/sandbox titotest-centos-5.9 /bin/bash
```

### future directions

* consider makefile with build, clean, test tasks for containers?
* add optparse for reduced verbosity?

### why shell script?

using shell ensures the test suite runs on a minimal host that has nothing but shell and docker.
e.g., i use coreos to test containers, and it does not have python, ruby, or any such language.

fwiw, i've tested this test harness on fedora 19, coreos, and rhel 6.5 with realtime-kernel.