avocado-framework/avocado — pull request #60 by lmr (2014-05-08 04:20:33+00:00)
Title: RFC - Initial multiplex of configuration

ATTENTION: DO NOT PUSH THIS TO MASTER!

So, my idea of initial implementation of test config multiplexing reuses all of the current cartesian_config present in virt-test, with the objective of extending and fixing any perceived weaknesses as we discuss.

People in avocado could either run tests with it the 'normal' way, through a string with urls, or by providing a 'multiplex file', that creates a test matrix and runs the tests defined there. In my commits I have an example of how this would work nicely in sleeptest, and I do want to implement it in more complex tests, if we think this prototype is a good idea moving forward.

$ cat tests/sleeptest/sleeptest.mplx
variants tests:
- sleeptest:
url = sleeptest
variants conditions:
- short:
sleep_length = 0.5
- medium:
sleep_length = 1
- long:
sleep_length = 5

This file generates the following output:

$ ./scripts/avocado run --multiplex tests/sleeptest/sleeptest.mplx
DEBUG LOG: /home/lmr/avocado/logs/run-2014-05-08-01.17.48/debug.log
TOTAL TESTS: 3
(1/3) sleeptest.short: PASS (0.77 s)
(2/3) sleeptest.medium: PASS (1.27 s)
(3/3) sleeptest.long: PASS (5.27 s)
TOTAL PASSED: 3
TOTAL ERROR: 0
TOTAL FAILED: 0
TOTAL SKIPPED: 0
TOTAL WARNED: 0
ELAPSED TIME: 7.31 s