perf: refactor regression.py and add t-test

scipy: http://www.scipy.org/
t-test: http://en.wikipedia.org/wiki/Student's_t-test

scipy and numpy modules are needed.

Use unpaired T-test to compare two samples, user can check
p-value to know if regression bug exists.

Verified by this page: http://graphpad.com/quickcalcs/ttest1.cfm

If the difference of two samples is considered to be not
statistically significant(p <= 0.05), it will add a '+' or '-'
before p-value.
  '+': avg_sample1 < avg_sample2
  '-': avg_sample1 > avg_sample2

The analysis module doesn't care about what's tested,
what's the relationship of each cols/lines.
Currently I describe the sample relationship of cols in perf.conf
This is used to update col results after computing averages.

eg.
| [netperf]
| update_avg = 4,2,3|14,5,12|15,6,13

"normalize"   = $throughput / $cpu
"#tpkt/#exit" = $tx-pkts    / $io_exit
"#rpkt/#irq"  = $rx-pkts    / $irq_inj

Then col 4(normalize),14(#tpkt/#exit),15(#rpkt/#irq) will be re-computed.

Signed-off-by: Amos Kong <akong@redhat.com>
use paired t-test to compute significance of all averages

The paired t-test is computed for averages of same category.
For netperf, we need to compute three paired t-tests for
TCP_RR, TCP_STREAM, TCP_MAERTS.

Code was verified by online tool:
 http://graphpad.com/quickcalcs/ttest1.cfm

Signed-off-by: Amos Kong <akong@redhat.com>
add the raw data that the averages are based on in another file

Users need to reference raw data in analyzing,
so output the raw data and averages to another file.

Signed-off-by: Amos Kong <akong@redhat.com>
display perf result by HTML file

Use HTML syntax to display results by beautiful tables.

Signed-off-by: Amos Kong <akong@redhat.com>
netperf: write version/description in result

fix tiles:
 - Change cpu to %CPU
 - Change normalize to thr/%CPU

write version/description infomation in result file.

Signed-off-by: Amos Kong <akong@redhat.com>
ntttcp: write version/description in result file

Signed-off-by: Amos Kong <akong@redhat.com>
add version info and case description in result

Collect version info and case description from raw results,
add them in main result files.

Signed-off-by: Amos Kong <akong@redhat.com>
tools.regression.py: Add shebang and executable permissions

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
server/kvm.py: remove legacy KVM support

The server side API for KVM has been unmaintaned for ages, and targets
very old versions of KVM. Instead of carrying code that is not relevant
or may not work at all, clean it up.

People interested in KVM testing should look at the better maintained
'kvm' test in client/tests.

If there's demand for pure server side KVM test support, we can consider
creating server side code using the client virt/kvm libraries.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
cient/virt/virt_utils.py: Introduce KojiDirIndexParser

KojiDirIndexParser is a parser for HTML directory index pages,
specialized to look for RPM links.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
client/virt/virt_utils.py: Introduce RPMFileNameInfo

RPMFileNameInfo is a simple parser for RPM based on information
present on the filename itself.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
client/virt/virt_utils.py: Introduce KojiScratchPkgSpec

KojiScratchPkgSpec is a package specification syntax parser for Koji
scratch builds.

This also introduces changes to KojiClient to support downloading
packages from a scratch build.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
virt installer: add support for koji scratch builds

Signed-off-by: Cleber Rosa <crosa@redhat.com>
kvm tests: Update cdrom test

This patch adds the check for tray status and fixes some bugs. Also it
changes the VM handling. The VM restart is now necessarily only when
the test fails.

* tray-open status subtest
* change vm handling
* absolute paths of cdrom locations
* workaround locked cdrom
* workaround eject time

Signed-off-by: Lukas Doktor <ldoktor@redhat.com>
virt multihost_mig: Repairs bug in starting sequence of tests.

Signed-off-by: Jiří Župka <jzupka@redhat.com>
virt: Adds migration over fd for kvm.

Migration over fd:
  source:
    1) Make new descriptor (pipe, socket) and send the descriptor
       to qemu-monitor over unix socket.
    2) Register the descriptor in qemu-monitor with function
          getfd DSC_NAME
       and close the descriptor in main process.
    3) Migrate over descriptor.
          migrate fd:DSC_NAME

  destination:
    1) Start child process with open second side of source descriptor.
    2) Start machine with param -incoming fd:descriptor
    3) Wait for finishing migration.

Signed-off-by: Jiří Župka <jzupka@redhat.com>
kvm-test: Add support for multihost migration over file descriptor.

Test create socket get descriptor from socket and migrate through
the descriptor.

This test allow migration only of one machine at once.

Signed-off-by: Jiří Župka <jzupka@redhat.com>
client/tools/regression.py: usage message fix

Remove the need for the interpreter, since the script has been made
a proper executable (see 05af063b39f50cc2f5b398365e59bb226b829ad7).

Also, use more standard syntax for denoting mandatory arguments.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
virt: move image utility functions from virt_vm to virt_utils

This allows us to break the circular dependency that currently exists
in virt_vm.py/virt_utils.py.

Reasoning and longer explanation: while testing a script that uses some
features of virt_utils.py, it was noticed that a gstreamer
initialization
method was called. That seemed to be unnecessary given that the video
generation code lives in a separate module. A bit more investigation
showed that virt_utils.py imported both virt_vm.py and
virt_env_process.py.

This patch removes the circular dependency for virt_vm.py only. The
dependency on virt_env_process.py is a bit more tricky, but will be
done.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
client: Removing more unused imports

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>