jsonrpc: have simplejson as soft dependency

In commit ec58bca5cb18 we added dependency to
the external simplejson package to gain some speed,
especially on RHEL/Centos 6.x.

We discovered the hard way that python-simplejson
was moved out of the base system in RHEL/CentOS 7,
and put into the EPEL repository.
This is alone makes ec58bca5cb18 not good enough.

We still want to get the JSON codec speedups, so
to this patch
* moves the requires to RHEL/Centos 6.x only
* makes simplejson a soft dependency.

Change-Id: I11c05cff5b1cc3727653fa4ce210a76fcadf5628
Bug-Url: https://bugzilla.redhat.com/1155146
Signed-off-by: Francesco Romani <fromani@redhat.com>
Reviewed-on: http://gerrit.ovirt.org/35354
Reviewed-by: Piotr Kliczewski <piotr.kliczewski@gmail.com>
Reviewed-by: Nir Soffer <nsoffer@redhat.com>
Reviewed-by: Dan Kenigsberg <danken@redhat.com>
sos: remove vdsm pki private keys collection

Change-Id: Iae8371de3c415c946608c2192315cf2f219da630
Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=1162784
Signed-off-by: Sandro Bonazzola <sbonazzo@redhat.com>
Reviewed-on: http://gerrit.ovirt.org/35241
Reviewed-by: Douglas Schilling Landgraf <dougsland@redhat.com>
Reviewed-by: Dan Kenigsberg <danken@redhat.com>
Revert "libnl: make link cache optional for link/route/addr_info"

This reverts commit d977d3ffcabcd2297ff20e7a34dc0f4f15b9cfed.

Needed to make it work again on RHEL/CentOS 6.x.
Revert "libnl: drop libnl-1 dependency"

This reverts commit 56e3784654f2c0c286add4eda4cd8be1024c29fe.

Needed to make it work again on RHEL/CentOS 6.x
tests: mark test broken

mark a consistently failing test (on my CentOS 7) until
it is resolved upstream.
spec: require libvirt 1.2.9

To make use of the bulk stats API, we need libvirt 1.2.9+,
which is found on Fedora 20 in the virt-preview repo and,
(to be confirmed) in RHEL/Centos 7.1.
schedule: Introduce scheduling library

This moudule provides a Scheduler class scheduling execution of
callables on a background thread. This should be part of the new
scalable vm sampling implemntation.

See the module docstring and tests for usage examples.

Change-Id: Ie3764806d93bd37c3b5924080eb5ae4d29e4f4e0
Signed-off-by: Nir Soffer <nsoffer@redhat.com>
executor: introduce the executor library

Executor is a thread pool augmented with the
capability of discard blocked worker threads,
by replacing them with fresh ones.

This is needed to accomodate the needs of the sampling code.
The sampling needs to deal with possibly blocking libvirt
calls which needs to enter into the QEMU monitor,
thus can get stuck due to QEMU being in D state.

Change-Id: Ic06da1ba57868dc2c7db67a1868ad10087a1cff2
Signed-off-by: Francesco Romani <fromani@redhat.com>
sampling: rename variables to make code nicer

Rename internal variables of SampleWindow
methods to make the code nicer and easier
to read.

No changes in behaviour.

Change-Id: I14caeb163552a659d1e7fcc103ac7fc9f1697bad
Signed-off-by: Francesco Romani <fromani@redhat.com>
tests: split SampleWindow tests

Reorganize and rationalize the unit tests after
the introduction of SampleWindow class.

Change-Id: I74f0005eae5466e083002f0d871bcf1fbec54fef
Signed-off-by: Francesco Romani <fromani@redhat.com>
vm: extract monitoring from stats thread

In order to monitor (aka 'sample') the status
of a Vm, VDSM uses the Vm statistics threads.

Each thread is in charge of checking the state
of a Vm, fetching the data from the Vm class
and from the hypervisor.
In order to collect the data from the hypervisor,
the stats thread has no choice but to access
the (private!) domain connection of a Vm,
thus violating the layering and introducing
unnecessary coupling.

This patch introduces the vm.Monitor class,
which is in charge to export the hypervisor
stats needed by VDSM.

This way, we can decouple the source of
hypervisor data from its users, and
establish a stronger (and documented)
interface. This also makes testing and
future changes easier to both sides.

In order to keep this patch minimal,
only the new class is added and mostly
code is moved with minimal renaming.

Future patches in the series will add
documentation and tests for the new class.
virt: move VmStatsThread in its own module

With the introduction of the vm.Monitor class
to provide domain monitoring data in a safe
and controlled way, VmStatsThread isn't
stricly coupled with Vm class anymore,
thus can be moved out of the overcrowded
vm.py module.

Since in a not so distant future virt.sampling
will be organized as subpackages, move
this class into a new module and not
bloat sampling.py.

No code changes besides fixing names and imports.
vm: add docs for vm.Monitor

document the expected format of the samplings,
so noone has to do reverse engineering again.
vm: rename Monitor.{balloon => memory}

rename a method to reflect what it actually does, and not what
it is used for. This concerns the caller.
sampling: move NUMA stats gathering in a method

This is an old pet peeve of mine.
vm: sampling: drop AdvancedStatsFunction

Extract the samplings from VmStatsThread.
In the simplest form, they are a (named) tuple
of callable, interval, so treat them as such.

Doing so, we make the sampling code decoupled
from the Vm - destination sample window is now
a parameter of it.

Moreover, this also makes that samplings may
be done by generic code, not necessarily
by VmStatsThread.

As side effect, AdvancedStatsFunction is
no longer used, so it is dropped.
sampling: reorder methods for clarity

no code changes