Merge pull request #61 from ruda/test_runner

avocado.job: Create entry point for  plugable test runners.
avocado.utils: Add params module

Add a utility module that offers a Params class, a
Dict like class that offers some additional advantages
over a regular dictionary for testing purposes.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
avocado: Introduce the multiplex_config module

It is a common occurrence in automated testing
that we want to generate a test matrix, for example,
I want my python program to be tested under

1) Python 2.6
2) Python 2.7

In the following Linux Distributions

1) Fedora 20
2) Fedora 19
3) Ubuntu 12.04
4) Ubuntu 14.04

This commit introduces the multiplex config file
format, a format that lets you define this
test matrix in a compact way:

variants python:
    - py26:
        py_interpreter = 2.6
    - py27:
        py_interpreter = 2.7

variants distros:
    - fedora19:
        distro = Fedora19
    - fedora20:
        distro = Fedora20
    - ubuntu12_04:
        distro = Ubuntu12.04
    - ubuntu14_04:
        distro = Ubuntu14.04

Which generates a set of the 8 tests of this cartesian
product. Each test is in fact a python dict:

dict    1:  fedora19.py26
dict    2:  fedora19.py27
dict    3:  fedora20.py26
dict    4:  fedora20.py27
dict    5:  ubuntu12_04.py26
dict    6:  ubuntu12_04.py27
dict    7:  ubuntu14_04.py26
dict    8:  ubuntu14_04.py27

One given dict can be passed by to avocado tests, so they
can retrieve params from the dict. The next patches
focus on instrumenting avocado tests to make use of said
params dictionaries, and extending the test runner to
accept a multiplex file.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
avocado.multiplex_config: Add unittest

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
avocado.test: Make test classes able to use params dicts

Introduce the possibility of using params dicts to
avocado tests. The params dicts lets tests to set
test tagged names based on the 'shortname' param,
among other features.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
avocado.job: Process multiplex files

If one passes a multiplex file in the runner arguments,
we shall use it and generate test dicts for the entire
test matrix, and pass those dicts to each test instance.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
avocado.plugins.runner: Introduce --multiplex-file option

With this, people will be able to introduce a --multiplex-file
(short -m) option to the run subcommand, that provides the
path to a multiplex file, that will be turned into a test
matrix (list of dicts) that will be run by the test runner.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
sleeptest: Modify to make use of self.params/multiplex file

Modify sleeptest to pick any params from self.params.
Also, introduce a sample multiplex file, sleeptest.mplx,
an example of how to write a multiplex file to fulfill
matrix defining goals.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
avocado.plugins: Add multiplexer plugin

Add a multiplexer parser, so people can try out their
multiplex files. Here is an example of usage

    $ scripts/avocado multiplex tests/sleeptest/sleeptest.mplx
    Dictionaries generated:
        dict 1:    sleeptest.short
        dict 2:    sleeptest.medium
        dict 3:    sleeptest.long

You can display dict contents by providing the flag --contents
(short -c).

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
.travis.yml: Add functional tests for multiplex

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
avocado.settings: Make convert_value_type() more generic

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
avocado.test: Log received test parameters

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
avocado.utils.params: Convert value type when specified in multiplex cfg

When we specify the type of a variable in the multiplex config:

    sleep_length = 1.0
    sleep_length_type = float

Now we'll convert automatically the string 1.0 retrieved
from the multiplex system to a float in a transparent way.
This will save code by making people to avoid to manually
convert values (if not specified, the value is considered
a string).

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
tests/sleeptest: Convert sleep_length type automatically to float

Tell the multiplex config file we're using floats as
the sleep_length value.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
avocado.test: Add convenience accessors to self.params

Make it possible to access param keys through attributes:

    self.params.sleep_length

instead of:

    self.params.get('sleep_length')

Also, introduce a params.set_default() method, that allows
to provide defaults to a given param attribute, in case
the configuration did not provide one (avocado tests running
as stand alone unittests).

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
tests/sleeptest: Make use of the new accessors

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
synctest: Add sample multiplex file

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
avocado.core.data_dir: Fix docstring

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
avocado: Simplify setting test default values

Instead of using self.params.set_default() manually for
each default value, allow test writers to set a
default_params attribute with default params, and
set them at test class init time. This way we keep
things even shorter, while allowing avocado tests
to be used stand alone.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
docs: Update documentation with multiplex config docs

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>