Add withDefaultOnIOError

This is a specialized version of try with a constant
handler. The main benefit is the type annotation forcing
the free type variable in 'try' to be IOError.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Look up RPC port

...instead of hard-codedly using the default port.
Ganeti's noded honors /etc/services, so also look
up the port there when trying to contact the daemon.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Objects: Add filterRuleOrder / Ord instance for FilterRule

This is necessary to process filters in this order, as required
by the filtering spec.

Signed-off-by: Niklas Hambuechen <niklash@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Query.Filter: Add `evaluateFilterJSON`

We will need it to evaluate filters on OpCodes for job filtering.

Signed-off-by: Niklas Hambuechen <niklash@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
JSON: Add `nestedAccessByKey` and `nestedAccessByKeyDotted`

We will need them for job filters, where we want to access OpCodes
as a JSON structure.

Signed-off-by: Niklas Hambuechen <niklash@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Implement job filtering

This implements the operational part of the design doc
  "Filtering of jobs for the Ganeti job queue"
(design-optables.rst).

It includes
- respecting filter rules when jobs are scheduled
- cancelling running jobs rejected by filters
- re-running the scheduler when filter rules are changed
- handling of the filter actions ACCEPT, CONTINUE, PAUSE, REJECT
  and RATE_LIMIT
- implementation of the "jobid", "opcode" and "reason" predicates

Signed-off-by: Niklas Hambuechen <niklash@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Unit tests for job filtering

This tests only the filter logic; the QA testing filter management + logic
comes later.

Signed-off-by: Niklas Hambuechen <niklash@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
qa_utils: Make AssertCommand return stdout/stderr as well

This is usefull if the run commands output something that's needed afterwards.

Signed-off-by: Niklas Hambuechen <niklash@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
qa_utils: Allow passing fail=None to AssertCommand

This is for cases where we don't care about the exit code.

Signed-off-by: Niklas Hambuechen <niklash@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
QA for job filtering

Includes QA for RAPI filter management and gnt_filter.

System tests (adding/removing filters, observing their effects)
are done with gnt_filter.

Signed-off-by: Niklas Hambuechen <niklash@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Add nodes field in class Disk

Add a field called "nodes" in the Disk class. This field will be a list
of Ganeti node UUIDs from which this disk is accessible.
Populate disk nodes when creating disks

When creating a Disk using the GenerateDiskTemplate function, fill the
nodes field depending on the disk template. There are three cases:

1) Internally mirrored disks (DRBD):
   These disks are accessible from two nodes, so the nodes field will
   include these. Their children (data, meta) are also accessible from
   two nodes, therefore they will inherit the nodes of the parent.
2) Externally mirrored disks (Blockdev, Ext, Gluster, RBD, Shared File):
   These disks should be accessible from any node in the cluster,
   therefore the nodes field will be empty.
3) Single-node disks (Plain, File):
   These disks are accessible from one node only, therefore the nodes
   field will consist only of the primary instance node.
Maintain consistency of nodes field

There are four scenarios in which the nodes field can change after the
disk's creation; when moving, replacing, converting or recreating disks.
For each scenario, update the nodes field accordingly.
Update cfgupgrade tool

Create an upgrade function that adds a nodes field to each disk that is
attached to an instance. Also, add a downgrade function that removes the
nodes field and brings the config to the default state for 2.12.
Update various tests regarding disk creation

Update the tests that deal with disk creation and check if the nodes
field is populated correctly when creating a disk.