koji: introduce koji_pkgspec tool

A quick and dirty tool for finding out exactly what packages
will be fetched and installed on a system based during a virt
test that uses a KojiInstaller.

This tool can fetch the information directly from a cartesian config
file, provided that it has the *right* configuration keys. Currently
this is hard coded to users of the kvm test, and looks for the
'koji_qemu_kvm_tag' and 'koji_qemu_kvm_pkgs' configuration keys.

Sample usage, query information about build id 123:

./koji_pkg_spec.py 123

Query information about build a given tag and many packages:

./koji_pkg_spec.py -t RHEL-6.3 qemu-kvm spice-server

Query information in a cartesian configuration file:

./koji_pkg_spec.py -c client/tests/kvm/build.cfg

Changes from v1:
 * Better adhere to the autotest import code style, import module
   instead of individual symbols inside a module.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
add 'Category' label to split subtests in results

Use 'Category' to split results to different blocks,
regression.py will compute 'Total Significance' for
each block.

Signed-off-by: Amos Kong <akong@redhat.com>
compute 'Total Significance' for each category

Signed-off-by: Amos Kong <akong@redhat.com>
update netperf test to store results to D.B

Use test.write_perf_keyval() to store results to table in
Autotest database, use test.write_test_keyval() to record
test environment info to database.

Signed-off-by: Amos Kong <akong@redhat.com>
virt performance: store perf result to D.B

Currently we only store performance results to files,
it's simple and convenient. This patch added a new interface,
the raw results would be stored to database of Autotest
server.

Original file interface is reserved, it can work without
Autotest server.

examples:
@ cd client/tools/
@ python regression.py netperf database 98 99
@ python regression.py netperf file /results/98-debug_user/ \
         /results/99-debug_user/

Signed-off-by: Amos Kong <akong@redhat.com>
virt: Add option log_func to QemuIO

Add option log_func to QemuIO to help user to record the
informations in different levels.
Also fix the problem caused by the scripts will try to call
output_func when it is None.

Signed-off-by: Yiqiao Pu <ypu@redhat.com>
client.tests.dacapo: Move cache dir to parent dir of self.srcdir

Currently,in test case dacapo,it put the cache dir,which used to put
the download files in it, down to self.bindir.
Now change the cache dir to parent dir of self.srcdir, that is to
say by default is client/tmp/dacapo/cache

Signed-off-by: Mike Qiu <qiudayu@linux.vnet.ibm.com>
client.tests.lsb_dtk: Move cache dir to parent dir of self.srcdir

Currently,in test case lsb_dtk,it put the cache dir,which used to put
the download files in it, down to self.bindir.
Now change the cache dir to parent dir of self.srcdir,that is to
say by default is client/tmp/lsb_dtk/cache

Signed-off-by: Mike Qiu <qiudayu@linux.vnet.ibm.com>
client.tests.dma_memtest: Move dma_memtest cache dir to parent dir of self.srcdir

Currently,in test case dma_memtest,it put the cache dir,which used to put
the download files in it, down to self.bindir.
Now change the cache dir to parent dir of self.srcdir,that is to
say by default is client/tmp/dma_memtest/cache.

Signed-off-by: Mike Qiu <qiudayu@linux.vnet.ibm.com>
client.shared.test:No need to check self.srcdir for twice

Currently, self.srcdir is checked twice in
shard.test,maybe a waste Now just check once
is enough.

Signed-off-by: Mike Qiu <qiudayu@linux.vnet.ibm.com>
tests.dbt2: Copy pgpool.conf to parent dir of self.srcdir

Currently,srcdir has change in to client/tmp/dbt2/src,but the config
file pgpool.conf still in client/tests/dbt2/
Now,copy the pgpool.conf from client/tests/dbt2/ to client/tmp/dbt2/
to make the test work.

Signed-off-by: Mike Qiu <qiudayu@linux.vnet.ibm.com>
kvm.tests.unittest_kvmctl: Need to check whether the dir exist or not

Currently,there is no file or dir named kvm_userspace/kvm/user,so need
to check,and create it if it doesn't exist.

Signed-off-by: Mike Qiu <qiudayu@linux.vnet.ibm.com>
server: Making sleeptest in server mode pass

Testing sleeptest in server mode caused falling in infinte loop.
Checking value of 'initialized' in _init_on_demand_ executed
__getattr__ because 'initialized' wasn't class attribute.
Setting it as class attribute solve problem.

Signed-off-by: Rafal Mackowski <mackoraf@tieto.com>
server: Run server side sleeptest in the client rather than the server

time.sleep(seconds) was executed on server not a client.
After changes client execute sleep and server wait
until client finish the job.

Signed-off-by: Rafal Mackowski <mackoraf@tieto.com>
client.config.kernel_io_tunable_defs:add newest kernel version support

Currently,The kernel version in  kernel_io_tunable_defs is too old,
so that many tests can't be run in the newest kernel.
Now to make it support 3.3.8/3.4.4/3.4.7

Signed-off-by: Mike Qiu <qiudayu@linux.vnet.ibm.com>
client.config.kernel_io_tunable_values:add newest kernel version support

Currently,The kernel version in  kernel_io_tunable_values is too old,
so that many tests can't be run in the newest kernel.
Now to make it support 3.3.8/3.4.4/3.4.7

Signed-off-by: Mike Qiu <qiudayu@linux.vnet.ibm.com>
client.partition: Change queue for the entire disk, instead of the partition

Currently,it try to return the queue for device,which may cause an
error when run the test about I/O Scheduler,because the device may be
the partition like sdaj12(more than 26+ disks),or whole disk like sda,
for obvious reasons,queue is for the entire disk, not for the partition.
This change try to find out the entire disk of the partition, and get
the queue for the entire disk

Signed-off-by: Mike Qiu <qiudayu@linux.vnet.ibm.com>
client.fsdev_disks: Change anticipatory scheduler to cfq scheduler

Currently,the anticipatory scheduler has been chose for I/O scheduler
test, while it has been removed from the 2.6.33 kernel.
Now change it into the cfq scheduler which is the default in current
kernel.

Signed-off-by: Mike Qiu <qiudayu@linux.vnet.ibm.com>
client.fsdev_disks: Change filesystem to fstype

Currently, FsOptions has changed the filesystem to fstype in
partition, but in fsdev_disks it does not change yet.
Now to change the filesystem to fstype to make the test work.

Signed-off-by: Mike Qiu <qiudayu@linux.vnet.ibm.com>
client.partition: Check the mountpoint before mount

Currently, when mounting a device to a mountpoint,
the code doesn't whether the mountpoint exists,
and this might cause a mount failure. Now check the
mountpoint before mount, if it doesn't exist,
create the mountpoint dir.

Signed-off-by: Mike Qiu <qiudayu@linux.vnet.ibm.com>