client/bin/optparse.py: rework options to improve usability

This patch groups related options for better help display, and removes
the '--debug' command line options.

The grouping of options make the help output of autotest-local easier
to read, IMHO.

By running autotest-local, and omitting '--verbose' we already get the
same behavior of the '--debug' command line option, so ditch it and be
more clear on the '--verbose' help message.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
autotest_local.py: separate parser creation

To make it easier for inherited classes to have their own custom
parsers without instantiating the ones on the parent class.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Fix the tap up fail problem on power arch.

 When we perform the tap up operation, we should pass a 18Byte(16B for the name
 and a 2B short int size for the flag 0x0001) ifreq struct to the ioctl
 function. If we have a 4B int size flag instead, there is no problem on a x86
 machine as it's little-endian arch, the first 2B is [01 00] coincidently the
 same as we have a short int size flag. but on a big-endian arch like power. the
 first 2B is [00 00], which is not correct for the operation. so just change the
 flag size to 2B.

Signed-of-by: Liu Sheng <liusheng@cn.ibm.com>
Make abort on client state mismatch configurable

This patch is a proposed solution for issue #255.

One frequent problem of people using the autotest server
is job aborts due to client state mismatches between
reboots, mainly differences among list of partitions
before and after reboot. In all cases so far the mismatch
is due to simple things like:

 * Presence/absence of cdrom drivers among different
   kernels
 * Difference among device naming. We did actually try
   to fix that in commit 78a07d5e, but not all clients
   may have blkid installed.

So far I haven't seen any case where autotest is aborting
the job in a legitimate context (ie, a context where we
might have a problem in the job itself). So let's make
the abort part optional, with defaults to False.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
apache: fix django rewrite rules with older httpd

httpd on FC16 (version 2.2) has a fairly old version of mod_rewrite and
per-directory rules don't work quite how I expected them to. Fix the changes
from commit a9d1f296fabec223b6e07374ce5ad8281ce151ca by moving the rewrite
rules back to global context, but specifying only the media/ bits we care about
in the django context.

Signed-off-by: Nishanth Aravamudan <nacc@us.ibm.com>
autotest.base_utils: Move virt.utils.Thread to common_lib

Move the class virt.utils.Thread to base_utils.InterruptedThread thus it can be used in general utils.

Signed-off-by: Jiří Župka <jzupka@redhat.com>
autotest.common_lib: Add syncdata class to common_lib

Syncdata class allows to synchronize data between
multiple hosts and guests. It creates a server for
syncing data and hosts uses the server for data
synchronization.

Signed-off-by: Jiří Župka <jzupka@redhat.com>
virt.virt_utils: Add framework for multihost migration

Multihost migration framework makes multi host migration
guest with load easy. This patch also replaces old tests
for multihost migration with versions that use the
framework.

Multihost miration framework takes care of:
  - preparing environment before migration
  - preparing guest for migration on source and dest host
  - start guest
  - start work on guest
  - migration between hosts
  - check work on destination host
  - close guest
  - postprocess environment after migration

The framework also allow start multiple migration
independently in some time with multiple hosts and
different work on guests.

Signed-off-by: Jiří Župka <jzupka@redhat.com>
Make job/client temporary directory

This patch adds test_tmp_dir option in COMMON section in
global_config.ini that, when set, would be assign to tmpdir of base_job
class.

Signed-off-by: Martin Krizek <mkrizek@redhat.com>
client/kernelexpand: add support for kernel mirror at configuration level

Allow system administrators to define kernel mirrors (defaulting at
install time to kernel.org itself) for jobs.

To allow for simpler unittest writing, I've moved the config reading
into a function. This may lead to a slight slow-down, but this is not a
performance-sensitive path.

Signed-off-by: Nishanth Aravamudan <nacc@us.ibm.com>
client/kernexpand: add support for gitweb mirror

Allow system administrators to define gitweb mirrors (defaulting to
none) for jobs.

Signed-off-by: Nishanth Aravamudan <nacc@us.ibm.com>
client/kernelexpand: support for sha1 kernels

When testing against particular kernel trees, it can be useful (i.e.,
during bisection) to build a SHA1 directly. Add support for matching
SHA1s as the kernel to build.

Signed-off-by: Nishanth Aravamudan <nacc@us.ibm.com>
kernelexpand: add sha1 and gitweb unit tests

Signed-off-by: Nishanth Aravamudan <nacc@us.ibm.com>
kernelexpand: Fixup slightly off indentation

Found by the checker script

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
client/tools/scan_results.py: Make program to run from any directory

We actually forgot to fix this a long time ago, but it's
far more convenient to make scan_results.py to execute from
any diretory. Do this by figuring out the path for the script,
and then deducing the path to the results file by relative
positioning.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
Make test source directories configurable

This patch adds test_src_dir option in COMMON section in
global_config.ini that, when set, would be assign to self.bindir of
base_test class.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
Signed-off-by: Martin Krizek <mkrizek@redhat.com>
Change definition of KojiPkgSpec from py list to string

This patch implements the solution for issue #244.

Currently when we define the list of KojiPkgSpec in text format in a
cartesian configuration file, we do so by creating a python list
instruction that gets evaluated by the installer code.

Taking a better look at it, there's no need to have it that way, and a
simple space separated list of plain text would do just fine. This would
make the koji package specification more in line with other content on
cartersian configuration files, and also simplify it's use, such as
adding a new package specification by doing a += :newpkg:subpkg or any
other valid syntax.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
client/tools/scan_results.py: List results from all job tags

As well pointed by Chris Evich <cevich@redhat.com>, scan_results
could get results from all job tags instead of just default.
The fix is easy enough, though.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
KVM test: Repairs multiname flags bug in cpuflag test.

Repairs qemu -cpu command line creation. Instead of
sse4_2|sse4.2 it insert only sse4_2 to qemu -cpu
command line.

And adds work-around aexpect bug to stress.c.
aexpect Expect.read_nonblocking reads data even if timeout expired and
process started in shell produces lots of data to output.

Signed-off-by: Jiří Župka <jzupka@redhat.com>
tko.db: Fix job deletion

As this code wasn't updated in a while, it wasn't taking
into account an extra table that had to be expurged before
removing job information, tko_job_keyvals.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>