Add TestCaseWithParameters to testing_utils

This new subclass of TestCase allows construction of test cases
that multiply themselves to get better coverage for various values
of specified variables.

Implementation is a cross between the well-known TestCaseWithScenarios
and my own TestCaseWithInvariants. It allows the developer to
name variable parameters to a test case and provide all the distinct
values they (as a collection) should take. Using this as the base class
it's easy to construct the more limited TestCaseWithScenarios (I have that
as a separate patch) as well as the special-cased invariants.

The code is heavily documented and seems to work well in causual testing.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
fwts_test now fails based on user specified threshold
accidentally removed the enhanced failure output, so tweaked that and added it back in
suspend/suspend_advanced and suspend/suspend_advanced_auto now only fail on CRITICAL events, while all other tests using fwts_test will still fail on HIGH or CRITICAL events.
removed extra print() statements that were not necessary
Remove unused variables

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
Fix indent on one line

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
Fix mk-interesting-graphs after adding special command

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
Add a simple testing script

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
Add TestCwd to plainbox.testing_utils.cwd

This small class simply creates a context manager for working in
a different directory, temporarily

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
Add support for integration tests

This warrants some explanations. Integration tests won't be running as a
part of normal unit testing, hence the test_*.py patter is not used.
This patch allows integration tests to discover all the checkbox jobs
that should be started by inspecting package data (looking for reference
results).

The actual test just compares the outcome of the job with reference
known-good expected values. Those are stored in the source tree and
are packaged along with everything else. To make discovery painless
and irrelevant of installation path I'm using pkg_resource which matches
python3 packaging evolution and the documented best strategy for accessing
package data.

This works very well but after implementing it and giving it some thought
I cannot help to wonder if we need more than one "set" of reference
answers. (in practice the data directory would get a sub-directory
per each data-set). This could be used to test responses from resource
scripts (so reference data would contain actual resources) and other
generators that don't produce one valid data-set but instead slightly
different data-sets depending on the hardware (or software and hardware)
they run on.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
Add plainbox self-test command.

This command discovers, loads and executes all integration tests

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
Run integration tests when testing for coverage

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>