run: qemu - Add support to enable vhost

Make it more convenient to enable vhost=on or vhostforce=on
using the test runner. As of now, bridge access still
requires root, so you'd need to run things as root:

sudo ./run -t qemu --nettype bridge --vhost on

or

sudo ./run -t qemu --nettype bridge --vhost force

CC: Richa Marwha <rmarwah@us.ibm.com>
CC: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
virt: Cartesian config bug repair.

https://github.com/autotest/virt-test/issues/199

There was too aggressive branch config filtering for achieving
better performance.

Before:
  1) config branch not produce any dict.
  2) save filters which block this branch.
  3) if **one** of filter will be in content next time
     then don't use this branch again.
  =>) wrong filtering

Now:
  1) config branch not produce any dict.
  2) save filters which block this branch.
  3) if **all** of filter will be in content next time
     then don't use this branch again.
  4) else check branch again for new dicts.

Signed-off-by: Jiří Župka <jzupka@redhat.com>
virt: Cartesian config speed and memory optimization.

Memory:
  It uses __slots__  instead of dict in class.
Speed:
  Replaces in most of cases very slow operation Template.substitute
  by faster version.
  These changes reduce time for big config file by 1/4 of time.

Signed-off-by: Jiří Župka <jzupka@redhat.com>
run: --no flag is actually a general flag

There is nothing specific to qemu in the --no flag,
but it was on the qemu group. Let's move it to
the general group.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
qemu/cfg/tests.cfg - Exclude multi host tests

Multi host tests can't be executed on the standalone
runner, so we've got to filter them out. This way
we'll have a lot less results displayed when listing
tests for, say, Fedora.18.x86_64, but the list will
be more accurate.

CC: Jiri Zupka <jzupka@redhat.com>
Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
netperf: introduce an agent to launch client sessions

Currently we use many threads/heavy control sessions to launch many
clients, Autotest takes too much of the host resources.

This patch introduces an agent script, it'll be executed inside the
client system to launch multiple sessions, and Autotest only needs
to use one background thread to control clients.

We can consider to process the raw results in this agent script in
future, if it can help to simplify the case.

Signed-off-by: Amos Kong <akong@redhat.com>
utils_misc: Add a function to normalize the data size

Normalize a data size in one order of magnitude to another (MB to GB,
for example). In this function you can also change the factor as you
need, the default factor is 1024.

changes from v1:
  - rename the function and variables
  - update the method for getting a fraction.
  - Add unittests

Signed-off-by: Yiqiao Pu <ypu@redhat.com>
qemu/cfg/tests-example.cfg: Update examples to use current Fedora

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
virt: Make Cartesian config more verbose

Signed-off-by: Jiří Župka <jzupka@redhat.com>
virt: Adds named variants to Cartesian config.

******** example:
variants tests:
  - wait:
       run = "wait"
       variants:
         - long:
            time = short_time
         - short: long
            time = logn_time
  - test2:
       run = "test1"

variants virt_system:
  - linux:
  - windows_XP:

variants host_os:
  - linux:
       image = linux
  - windows_XP:
       image = windows

tests>wait.short:
    shutdown = destroy

only host_os>linux

******** output:
dict    1:  host_os>linux.virt_system>linux.tests>wait.long
    dep = []
    host_os = linux
    image = linux
    name = host_os>linux.virt_system>linux.tests>wait.long
    run = wait
    shortname = host_os>linux.virt_system>linux.tests>wait.long
    tests = wait
    time = short_time
    virt_system = linux
dict    2:  host_os>linux.virt_system>linux.tests>wait.short
    dep = ['host_os>linux.virt_system>linux.tests>wait.long']
    host_os = linux
    image = linux
    name = host_os>linux.virt_system>linux.tests>wait.short
    run = wait
    shortname = host_os>linux.virt_system>linux.tests>wait.short
    shutdown = destroy
    tests = wait
    time = logn_time
    virt_system = linux
dict    3:  host_os>linux.virt_system>linux.tests>test2
    dep = []
    host_os = linux
    image = linux
    name = host_os>linux.virt_system>linux.tests>test2
    run = test1
    shortname = host_os>linux.virt_system>linux.tests>test2
    tests = test2
    virt_system = linux
dict    4:  host_os>linux.virt_system>windows_XP.tests>wait.long
    dep = []
    host_os = linux
    image = linux
    name = host_os>linux.virt_system>windows_XP.tests>wait.long
    run = wait
    shortname = host_os>linux.virt_system>windows_XP.tests>wait.long
    tests = wait
    time = short_time
    virt_system = windows_XP
dict    5:  host_os>linux.virt_system>windows_XP.tests>wait.short
    dep = ['host_os>linux.virt_system>windows_XP.tests>wait.long']
    host_os = linux
    image = linux
    name = host_os>linux.virt_system>windows_XP.tests>wait.short
    run = wait
    shortname = host_os>linux.virt_system>windows_XP.tests>wait.short
    shutdown = destroy
    tests = wait
    time = logn_time
    virt_system = windows_XP
dict    6:  host_os>linux.virt_system>windows_XP.tests>test2
    dep = []
    host_os = linux
    image = linux
    name = host_os>linux.virt_system>windows_XP.tests>test2
    run = test1
    shortname = host_os>linux.virt_system>windows_XP.tests>test2
    tests = test2
    virt_system = windows_XP

For filtering of named variants is used character ">" because there was
problem with conflict with = in expression key = value. The char ">"
could be changed to something better but it should be different from "="
for optimization of speed.

Additionally named variant adds keys to final dictionary in case of
example is it (virt_system = linux). It should reduce size of config file.
Keys defined in config and keys defined by named variants are in same
name space.

Signed-off-by: Jiří Župka <jzupka@redhat.com>
virt: Adds possibility filter defaults variant from variants

If default variant is not filtered by only or no filters then
only default variant is chosen. This behavior was used for optimizing
of speed of Cartesian config.
If variants don't have default variant then everything works as usual.

For calling Cartesian config from command line is used option -d/--defaults:
   ../virttest/cartesian_config.py -d cfg/cc.cfg

For calling Cartesian config from python:
   c = Parser(args[0], defaults=options.defaults, debug=options.debug)

*********  example:
variants tests:
  - wait:
       run = "wait"
       variants:
         - long:
            time = short_time
         - short: long
            time = logn_time
  - test2:
       run = "test1"

variants virt_system:
  - @linux:
  - windows:

variants host_os:
  - linux:
       image = linux
       variants:
            - ubuntu:
            - @fedora:
  - windows:
       image = windows
       variants:
            - @XP:
            - WIN7:

only host_os>windows

In this case is chosen from host_os variants windows variant.
XP will be chosen from host_os>windows variants because XP is
default variant and isn't filtered. Next step is select one
variant from guest_os. There will be chosen only default variant linux.
There is no default variant in tests variants because that all of tests
will be chosen.

********  output:
dict    1:  host_os>windows.tests>wait.long
    dep = []
    host_os = windows
    image = windows
    name = host_os>windows.XP.virt_system>linux.tests>wait.long
    run = wait
    shortname = host_os>windows.tests>wait.long
    tests = wait
    time = short_time
    virt_system = linux
dict    2:  host_os>windows.tests>wait.short
    dep = ['host_os>windows.XP.virt_system>linux.tests>wait.long']
    host_os = windows
    image = windows
    name = host_os>windows.XP.virt_system>linux.tests>wait.short
    run = wait
    shortname = host_os>windows.tests>wait.short
    tests = wait
    time = logn_time
    virt_system = linux
dict    3:  host_os>windows.tests>test2
    dep = []
    host_os = windows
    image = windows
    name = host_os>windows.XP.virt_system>linux.tests>test2
    run = test1
    shortname = host_os>windows.tests>test2
    tests = test2
    virt_system = linux

Signed-off-by: Jiří Župka <jzupka@redhat.com>
virt: Adds possibility to exclude filter from config.

It should be used only for manipulation with parser from
python code. It could bring some problem with understanding
of Cartesian config.

syntax:
  only linux
  exclude only linux    # exclude linux filter

  no linux
  exclude no linux    # exclude linux filter

  only host_s>linux
  exclude only linux    # doesn't work because filter only linux
                        # doesn't match with only host_s>linux

  only host_s>linux
  exclude only host_s>linux    # exclude host_s>linux filter

  only linux..virtio
  exclude only linux           # doesn't work
  exclude only linux..virtio   # works

In other words there is not possibility to exclude only part of filter.
Only whole filter could be excluded!

******* example:
variants host_os:
  - linux:
       image = linux

  - windows:
       image = windows

only host_os>linux

******** output:
dict    1:  host_os>linux
    dep = []
    host_os = linux
    image = linux
    name = host_os>linux
    shortname = host_os>linux

******** with exclude:
variants host_os:
  - linux:
       image = linux
  - windows:
       image = windows

only host_os>linux
exclude only host_os>linux

*********output:
dict    1:  host_os>linux
    dep = []
    host_os = linux
    image = linux
    name = host_os>linux
    shortname = host_os>linux
dict    2:  host_os>windows
    dep = []
    host_os = windows
    image = windows
    name = host_os>windows
    shortname = host_os>windows

Signed-off-by: Jiří Župka <jzupka@redhat.com>
virt: Extends Cartesian config unittests for new features

Signed-off-by: Jiří Župka <jzupka@redhat.com>
virt: Adds formal definition for Cartesian config

Signed-off-by: Jiří Župka <jzupka@redhat.com>