Utils.Monad: Add unfoldrM and unfoldrM'

Taken from monad-loops, with a variable rename to avoid name shadowing.

Signed-off-by: Niklas Hambuechen <niklash@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
TestCommon: Add listOfUniqueBy

This is a Gen for creating elements that are distinct by a certain property,
(e.g. for generating a list of jobs, all of which have different job IDs).

Signed-off-by: Niklas Hambuechen <niklash@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Test.JQScheduler: Fix queue jobs having same job IDs

The `Arbitrary Queue` instance so far would simply generated lists
of jobs, for which it was possible that job IDs are repeated
(inside individual lists or accross all lists).

While this hasn't created any problem so far, real-world Queues
cannot have repeated job IDs inside, and the Gen should reflect that.

Signed-off-by: Niklas Hambuechen <niklash@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Objects: Add filterRuleOrder / Ord instance for FilterRule

This is necessary to process filters in this order, as required
by the filtering spec.

Signed-off-by: Niklas Hambuechen <niklash@google.com>
Query.Filter: Add `evaluateFilterJSON`

We will need it to evaluate filters on OpCodes for job filtering.

Signed-off-by: Niklas Hambuechen <niklash@google.com>
JSON: Add `nestedAccessByKey` and `nestedAccessByKeyDotted`

We will need them for job filters, where we want to access OpCodes
as a JSON structure.

Signed-off-by: Niklas Hambuechen <niklash@google.com>
Implement job filtering

This implements the operational part of the design doc
  "Filtering of jobs for the Ganeti job queue"
(design-optables.rst).

It includes
- respecting filter rules when jobs are scheduled
- cancelling running jobs rejected by filters
- re-running the scheduler when filter rules are changed
- handling of the filter actions ACCEPT, CONTINUE, PAUSE, REJECT
  and RATE_LIMIT
- implementation of the "jobid", "opcode" and "reason" predicates

Signed-off-by: Niklas Hambuechen <niklash@google.com>
Unit tests for job filtering

This tests only the filter logic; the QA testing filter management + logic
comes later.

Signed-off-by: Niklas Hambuechen <niklash@google.com>
qa_utils: Make AssertCommand return stdout/stderr as well

This is usefull if the run commands output something that's needed afterwards.

Signed-off-by: Niklas Hambuechen <niklash@google.com>
qa_utils: Allow passing fail=None to AssertCommand

This is for cases where we don't care about the exit code.

Signed-off-by: Niklas Hambuechen <niklash@google.com>
QA for job filtering

Includes QA for RAPI filter management and gnt_filter.

System tests (adding/removing filters, observing their effects)
are done with gnt_filter.

Signed-off-by: Niklas Hambuechen <niklash@google.com>