Packager.py should take advantage of the parallelism from the package manager

Create the repo directory if it doesn't exist.

Signed-off-by: Jean-Marc Eurin <jmeurin@google.com>
Results parser: Handle errors parsing lines more properly

Use a better error message to help debug what is going on.  I don't see
how that regex could fail off the top of my head, even an empty string should
match and return '' and '' for the groups.

(update: What we're seeing after this is that somehow the parser is getting a "line" that contains newlines at times; not sure how)

Signed-off-by: Gregory Smith <gps@google.com>
Host scheduler refactoring. Move HostScheduler out of monitor_db.

In order to facilitate site extensibility of HostScheduler we need to factor out the dependence on global variables in monitor_db. I modeled this refactoring off of monitor_db_cleanup.

The main changes I've made are as follows:
1. Move BaseHostScheduler, site import, and SchedulerError out of monitor_db. SchedulerError must be moved to prevent a cyclical dependency.
2. Convert staticmethod/classmethods in BaseHostScheduler to normal methods.
3. Fix unit tests and monitor_db to import SchedulerError from host_scheduler.

Reviewable at: http://codereview.chromium.org/6597047/

Signed-off-by: Dale Curtis <dalecurtis@google.com>
KVM test: Add hotplug test to virtio console

Tests:
    Hotplug of virtioserial and console.
    Hotplug of virtio-serial-pci device.

Signed-off-by: Jiri Zupka <jzupka@redhat.com>
scheduler: Prevent the scheduler from logging the stdout return

Prevent the scheduler from logging the large pickled stdout return
value of drone_utility in the scheduler debug logs.  Adds a unittest.

Signed-off-by: Gregory Smith <gps@google.com>
web front end: Change the atomic group pull down menu

Change the atomic group pull down menu displayed in the admin interface
for Labels to a text box in hopes that it won't prepopulate the
selection with a value that then inadvertently gets set when the field
is intentionally NULL (the common case).
Also: display the atomic group in the labels list view.

Signed-off-by: Gregory Smith <gps@google.com>
KVM test: Support qxl and vga parameters in make_qemu_command

The behavior is different between RHEL6.X and RHEL5.X. On RHEL5,
qxl and vag are two parameters. But on RHEL6, qxl is one option
in vga.

This patch works on both RHEL5 and RHEL6 hosts. On RHEL6, if we
set qxl and vga together in tests_base.cfg, vga will be set to qxl,
no matter what you set to vga.

Signed-off-by: Feng Yang <fyang@redhat.com>
Signed-off-by: Amos Kong <akong@redhat.com>
KVM Test: nicdriver_unload.py: Fix some failure when running RHEL4.8 guest

1. readlink in RHEL4.8 doesn't have a '-e' parameter, replace it with '-f'.
2. nic driver parameter path in '/sys' is different, get it from config file.

Signed-off-by: Qingtang Zhou <qzhou@redhat.com>
KVM-test: clock_getres: Compile and execute program in guest

Currently, this test compiles program in host and executes it
in guest. If guest has different OS, execution will always fail.

Signed-off-by: Amos Kong <akong@redhat.com>
KVM test: Extend netperf to support multiple nics

Add the support of executing multiple netperf clients on hosts,
It can be used as a multi-nics stress test.

Signed-off-by: Amos Kong <akong@redhat.com>
Clone the drone setting only if drone sets are enabled

Signed-off-by: Jean-Marc Eurin <jmeurin@google.com>
client job: timeout parameter for job.run_test()

This option allows to set timeout for run_test. If timeout expires
before run_test completes, the test will be killed (test fails).
If no timeout parameter is present or if it is None, no timeout
will be enforced.

Example:
job.run_test('sleeptest', timeout=5, seconds=60)

Signed-off-by: Jan Stancek <jstancek@redhat.com>
client: pass run_test() timeout parameter to harness

pass timeout parameter from run_test() to harness in call
test_status_detail('START',...), so that harness has a chance
to learn about timeout when test is started and react accordingly.

For example: harness could setup an external watchdog,
which would live on server-side and reboot machine when needed.
It is assumed, that harness will typically setup any external
watchdog with a value slightly larger than test timeout, so client
has a chance to terminate failing test by itself.

Signed-off-by: Jan Stancek <jstancek@redhat.com>
KVM-test: Add function to scp files between two remote boxes

This function will help test to scp file between two remote os.
It can be two guest, one guest and one host or other conditions

Changes from v1:
    - docstring and parameter name updated to be password_list
    - function name changed to scp_between_remotes

Signed-off-by: Yiqiao Pu <ypu@redhat.com>
job: Remove non-existant parameter on generate_control_file calls (Fix #47)

Since its introduction on r2440, server/frontend.py has a call to
generate_control_file, present on frontend/afe/rpc_interface.py.
This method apparently never had a do_push_packages parameter,
but for some reason we were passing it in case we provide a specific
kernel with -k. Remove all references to do_push_packages.

With this, bug #47 is fixed, and it is possible to specify -k through
the cli.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
KVM test: Create a verify_kernel_crash() VM method

A method to verify guest kernel panics can be very
useful for a number of tests. Adapted from a function
present on virtio_console test, create VM.verify_kernel_crash()
and use it on unattended_install.

Changes from v1:
 * According to mgoldish's comments, instead of using
read_nonblocking() and read_until_last_line_matches(),
use simply get_output(), and only throw an exception
at the time we have a full trace.

Changes from v3:
 * Reimplementing the method on a less moronic fashion.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
KVM test: Use verify_kernel_crash() on unattended install

With this we can fail the test much earlier if a guest
kernel panic happens, instead of waiting for the full
timeout to complete.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
KVM test: Virtio console - Adding migration test

This patch adds migration test (tmigrate). It also daemonize the virtio_console_guest.py script, which is necessary for the migration test.

[virtio_console_guest.py]
It is splited to 2 parts, first one is client and second one is daemon.
* Daemon part is forked and works as the previous script except it redirects the STD{IN,OUT,ERR} into named pipes.
* Client part connects to named pipes and handle the communication with host OS. Also when the client is executed multiple time it finds the previous client and destrois it. Then it opens the pipes and take over the communication. The daemon is NOT executed twice.

[tmigrate]
This test transfers and checks data through virtio_console loopback while migrating the guest. It's splitted into 2 tests tmigrate_online and tmigrate_offline.
* tmigrate_{offline,online} - parses test parameters and executes the actual test __tmigrate()
* test parameters: virtio_console_migration_{offline,online} = "[{serialport,console}]:$no_migrations:send_buf_len:recv_buf_len:loopback_buf_len;..."  ; multiple tests are allowed and separated by ';'

It sends the data from first {console,serialport} and receive on all remaining ports of the same type. By default it adds 2 {console,serialport}s depends on test parameters but you can override this using no_{consoles,serialports} or the number can be higher according to other tests.

[BUGs]
Tested on: HOST=F13 (2.6.36-0.20.rc3.git4.fc15.x86_64), GUEST=F14 (2.6.35.6-45.fc14.x86_64)
Developer: is noticed simultanously with this patch for explenation.
1) console very often won't survive the migration (it stops transfering data).
virtio_console_migration_offline = "console:5:2048:2048:2048"
2a) offline migration: drops 1-2 send-buffers of data, which is for now accepted as success.
virtio_console_migration_offline = "serialport:5:2048:2048:2048"
2b) Using smp2 the data loss is 1-4 send-buffers of data.
only smp2
virtio_console_migration_offline = "serialport:5:2048:2048:2048"
3) online migration: similar problem as offline but the data-loss is much higher and depends on the console throughput. In this version very high value is accepted as pass. You should check the message "Maximal loss was %s per one migration". If it should be n-degree lower than the transfered data.
virtio_console_migration_online = "serialport:5:2048:2048:2048"
4) ThSendCheck sometimes get stuck in send. Test handle this and the problem is similar to: the "FIXME: workaround the problem with qemu-kvm stall when too much data is sent without receiving". Using migration this occures with even smaller slices.
virtio_console_migration_offline = "serialport:5:4096:1:1"

Signed-off-by: Jiri Zupka <jzupka@redhat.com>
Signed-off-by: Lukas Doktor <ldoktor@redhat.com>
KVM Test: introduce KojiPkgSpec support

KojiPkgSpec is a more flexible way to describe what packages should
be downloaded from a Koji server.

KojiClient supersedes KojiDownloader adding support for KojiPkgSpec.

KojiInstaller is also revamped to support KojiPkgSpec and KojiClient.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
KVM test: Refactor host kernel install into a utils function

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>