Env.destroy: Destroy VMs ungracefully

This is an attempt to fix a problem where ./run is trying to log into a
VM that never booted in the first place, making test runs sometimes take
a long time to start.

I am not sure this is really a good idea, but I am assuming that if we
have reached the VM destruction code inside Env.destroy(), that means
all our mechanisms to shut down the VM gracefully have already failed.

Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
aexpect: Fix fd leaks

An aexpect client, such as Spawn, Tail or other derived
classes keeps open fds to the server reader pipes. once
they get garbage collected, then we have a bunch of fds
that will fall into oblivion, at least 6 fds for each
instance that is garbage collected.

So the solution here is to close all reader fds upon
__del__() calls. This fixes the exploding number of
open files on a simple ./run -t qemu, from 300+ open
files to 15-41 files opened at a given moment. Not
bad.

Yiqiao and Qingtang, please apply this to your internal
tree, it's a simple fix that will end with all the
'too many files opened' problem when running tests, so
I'd say it will be a very welcome fix.

CC: Gerd Hoffmann <kraxel@redhat.com>
CC: Yiqiao Pu <ypu@redhat.com>
CC: Qingtang Zhou <qzhou@redhat.com>
CC: Chris Evich <cevich@redhat.com>
Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
qemu_qtree: Fix the problem with attached cdroms in multi_disk

Previously cdroms were not taken into account while checking
disk params thus VMs with cdroms failed with additional
devices in qtree but not in params.

Now they are skipped from params detection. As this test is not
focused on cdroms it doesn't check the cdrom params and it doesn't
require the cdrom to be present.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
run: Cmdline no_filter option

This patch adds '--no' cmdline option which adds no_filter
to config_parser.

This option is good for virtio_console tests where you can
specify --no virtconsole and still run all unspecifiable tests
without specifying specifically each test.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
virttest/virt_vm.py: Allow login with different credentials

This patch allows logging into (and sending files)  a VM using
credentials that are different than those specified in parameters.

Signed-off-by: Tomas Jamrisko <tjamrisk@redhat.com>
tests.virtio_console: reorganize cfg

In order to be able to run all virtio smoke tests easily I added
virtio_console_smoke variant with all related subtest inside.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
qemu.virtio_console: Add negative boot test with too many ports

Set more than allowed number of ports per virtio-serial-pci.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
qemu.tests.virtio_console: Add interrupted_transfer.suspend test

This patch adds interrupted_transfer suspend variant. This variant
initiates loopback over 2 vio ports and suspends the VM to mem.
After a while it resumes the VM with 'system_wakeup' and verifies
that the data are still transfered in booth directions.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
qemu.tests.virtio_console: Fix loopback test freeze

Loopback test might freeze the whole autotest when sender thread
fills the buffer. During cleanup it tries to send additional data
which is blocked due of full buffer. The only solution I found is
to destroy the VM and inform about this bug.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
virttest.aexpect: Add read_until_any_line_matches() function

This function is similar to read_until_last_line_matches() function
but it search all lines, not just the last ones. It's useful for
example in virtio_console test where additional output might occur
before it's analyzed with the last_line reader.

The match function is prioritizing the patterns so first it searches
the output for the last pattern, than for the -2 so when there
is PASS and FAIL in the output and patterns = (PASS, FAIL) it matches
the FAIL first, even thought PASS is first in the output.

Separate lines are matched so it's recommended to use '^' at patterns
beginning. ('^PASS', '^FAIL') in order to match the beginning of the
line.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
virttest.qemu_virtio_port: Use read_until_any_line_matches()

Use read_until_any_line_matches as another thread might write
some debug info right after the PASS/FAIL statement. Than
the test fails even thought it should PASS. There are still
some problems with non-thread-safe-print, the most important
are solved with sys.stdout.write(...\n).

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
virttest.qemu_virtio_port: Fortify GuestWorker against freezes

Require exact PASS statement for some commands as another PASS/FAIL
line might be matched instead.

Also add some workarounds for stucked ports/python by killing the
worker.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
virttest.qemu_virtio_port: Fortify Send/RecvCheck threads

ThSendCheck and ThRecvCheck might fail or freeze when they use
closed port.

This is a preparation for interrupted_loopback.disconnect test
where the ports are closed while transfering data.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
scripts.virtio_console_guest: Support for port reconnection

When the port is closed, it's completelly removed from the list
of descriptors. When it's opened, the file descriptor might differs
from the original one. Thus it's necessarily to keep the paths
of the used ports in the Switch thread and in case of reconnection
find the new file descriptor.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
tests.virtio_console: Add interrupted_transfer.disconnect subtest

This test closes the used ports while transfering data and after the
intr_time opens them again. It randomly closes ports on guest side
and host side, send threads and recv threads thus in the long run
it should cover all cases.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
qemu.tests.virtio_console: Fix sigio test crashes

When guest script is already running and the test only connects, every
signal destroys the daemon. This is complex problem of python, where
only the main thread accepts the signals. In order to fix this without
much pain I added test_delete_guest_script() at the test beginning thus
fresh instance of guest script is always executed. This solution seems
to work pretty well.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
qemu.tests.virtio_console: Exit background threads on test failure

Force exit background threads on thest failure.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
virttest.env_process: Quit screendump thread when exit_event is None

Sometimes exit_event can be deleted before screendump thread
finishes. Than it hungs in system forever and autotest never
exits.

This problem was spotted after test:
    virtio_console.spread_2.incorrect_options.boot_nr0
where the screendump thread never finished and additional was spawned
at the beggining of the next thread. This caused autotest hang in the and
as only single screendump thread can exit (due of nature of exit fce).

Additionaly set the screendump thread name for future analysis.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
virt-tests: Mass indentation fix

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>