Avoid pylint errors for members of namedtuples

Pylint isn't smart enough to know about the members of namedtuples, and
it's annoying to be adding pylint "ignore" annotations everywhere.
tests/run-performance-test.sh: Don't measure `stbt run` startup time

To get more accurate measurements of the time spent in the image
processing algorithm.

This also allows testing the speed of the match algorithm when the
template image is/isn't present in the video; uncomment the "if not
m.match" block if you only want to measure the initial frames of the
video that do match the template.
stbt._match_template: Refactor into first pass and `_confirm_match`

This is in preparation for the next commit.
stbt._match: Performance optimisation by using "pyramid" matching

The most expensive part of the image-matching algorithm is the call
to OpenCV's "matchTemplate" function, which compares the template image
against each possible position within the source image.

The new "pyramid" algorithm downsamples (shrinks) the source & template
images to half their width & height (thus 1/4 of the total pixels)
before passing them to the matchTemplate function. If a likely match is
identified, we then call matchTemplate with the source & template images
at their full size, but restricting matchTemplate's search to the areas
identified as potential matches during the previous level. This process
can be repeated more than once, for example starting from images 1/4 of
the original width & height (1/16 the total pixels).

At the smaller image size, if we identify a match, we always check again
at the larger size. So this algorithm won't introduce false positives
(false matches).

At the smaller image size, if we *don't* identify a match, we return a
"no match" decision immediately. This might introduce false negatives
due to differences between the smaller images that weren't present in
the originals; one of the next commits accounts for this by making the
match threshold more tolerant at the smaller levels of the pyramid.

You can disable this optimisation by setting "match.pyramid_levels" (in
your configuration file) to 1.

Once you add the fixes in the next 3 commits, all stb-tester's unit
tests continue to pass; I have also tested the functionality against a
private collection of 60 template images and 100 source images, and all
6,000 comparisons continue to yield the correct result (checking every
template against every source image; some of the combinations are
expected to match, others are expected to not match because every
template is not present in every image).

The performance improvement, as measured by
tests/run-performance-test.sh against a 720p video:

    before: 3.0fps; after: 35fps

But note that every frame in the video was a "no match", so the image
matching algorithm returned early. Measured using the same script and
video, with a template of the same size that does match:

    before: 2.8fps; after: 19fps

Each measurement was from the best of three runs, on my laptop with
dual-core 1.8GHz Intel Core i7, with the next 3 commits also applied,
and with templates sized 160x57 pixels (large enough for all 3 pyramid
levels to be used).
stbt._match: Don't halve the size of very small template images

This fixes some false negatives with small (~25px) template images.

Note that the source image must always be larger than the template
image, so we don't need to check that `image_pyramid` has as many levels
as `template_pyramid`.
stbt._match_template: Fix for large templates

If the source and template images are exactly the same size, the
`roi_mask` will be exactly 1x1. `cv2.findContours` doesn't return *any*
contours from images that are less than 3 pixels wide or high.

This fixes the following unit tests:
* test_wait_for_match_confirm_threshold_affects_match
* test_wait_for_match_erode_passes_affects_match
stbt._match_template: Lower threshold for downsampled images

Downsampling the images for the pyramid templatematch can cause images
that would match at the original size to not match at the smaller size.

I have only seen this happen in the self-test test_press_visualisation,
which has images with very thin, very high contrast text. At the
original size, the template matched with first pass certainty of 1.0; at
half the size, the certainty was 0.799, just below the default threshold
of 0.8.

To avoid matching images that didn't match before, we still use the same
threshold when checking the images at the full size (i.e. the last step
of the pyramid algorithm); but at the intermediate steps dealing with
downsampled images, we lower the threshold by 0.20. This number is
purely based on the result from the test_press_visualisation self-test,
which is the only failure of its type I have observed with the new
pyramid algorithm, after checking many images in a private collection.

In theory this has a performance impact, because the region of interest
processed at subsequent levels in the pyramid algorithm will be larger.
But I couldn't measure any difference with tests/run-performance-test.sh
(perhaps because the particular source video & template image I was
using don't present many matches between the thresholds of 0.6 and 0.8).
stbt._find_match: Nicer error if `match.pyramid_levels` misconfigured
"stbt run -vv" output includes html guide to understanding the debug images

"stbt run -vv" generates debug images to
"./stbt-debug/detect_match/<frame_number>/".

These images are numerous, and difficult to understand. Now we also
generate "./stbt-debug/detect_match/<frame_number>/index.html" which
presents the images in the correct order, notes the decision taken at
each point of the algorithm, and explains the images somewhat.

This uses Jinja2, a template engine implemented in python. The
"python-jinja2" package is available in the main Fedora repositories.
Note that the VM in `extra/vm` already installs Jinja2 (for the runner).
Merge branch 'match-performance'

An implementation of "pyramid" template matching -- a performance
optimisation for `stbt.wait_for_match` and `stbt.detect_match`.

The main change is ac97d6dd; see that commit message for details.

Also of interest is b4b59f5c, which extends the `stbt run -vv` debug
(which dumps to disk intermediate images from the image processing
algorithm) to also generate an `index.html` file that presents the
images in the correct order, notes the decision taken at each point of
the algorithm, and explains the images somewhat. See an example at
http://stb-tester.com/stbt-debug-example/detect_match/00001/index.html

* match-performance:
  "stbt run -vv" output includes html guide to understanding the debug images
  stbt._find_match: Nicer error if `match.pyramid_levels` misconfigured
  stbt._match_template: Lower threshold for downsampled images
  stbt._match_template: Fix for large templates
  stbt._match: Don't halve the size of very small template images
  stbt._match: Performance optimisation by using "pyramid" matching
  stbt._match_template: Refactor into first pass and `_confirm_match`
  tests/run-performance-test.sh: Don't measure `stbt run` startup time
  Avoid pylint errors for members of namedtuples
stbt.argparser: Document seemingly-unnecessary line of code

This line isn't strictly necessary because:

* When no `-v` command-line argument is specified, we set the
  default `_debug_level` 3 lines above (see 9aa90c16).
* When the `-v` command-line argument *is* specified, we increment
  `_debug_level` regardless of any default value retrieved by this line.

However, `stbt-run` prints out all the configuration options at startup:

    stbt.debug("Arguments:\n" + "\n".join([
        "%s: %s" % (k, v) for k, v in args.__dict__.items()]))

which includes a line of output like this:

    verbose: 2

Without this line of code, the output would look like this:

    verbose: None

even though verbose was set to 2 in the config file.

Hopefully adding this comment will stop people from trying to remove
that line, finding out that the unit test
`test_that_verbosity_level_is_read_from_config_file` fails, and
generally wasting time.
stbt.get_config: Take `type_` parameter, for better error messages

Instead of:

    >>> int(get_config(section, key))
    ValueError: invalid literal for int() with base 10: 'xyz'

we now get:

    >>> get_config(section, key, type_=int)
    ConfigurationError: 'section.key' invalid type (must be int)
stbt.wait_for_match etc: Nicer error message if template larger than source

This used to raise a very obscure exception somewhere deep in the
implementation details. It's a common error when you use the `stbt
templatematch` command-line tool and you give the arguments in the
wrong order.
self-tests for stbt.press_until_match

In the next commit I'm about to make some changes to
`press_until_match`, so it seemed a good idea to write some unit tests
that really test its behaviour under various conditions.
stbt.press_until_match: Default interval_secs & max_presses from stbt.conf

`press_until_match` is particularly sensitive to timing issues with
certain hardware configurations that introduce a lot of latency.
To account for different latencies in your particular setup (some
systems-under-test, infrared emitters, or video-capture devices might be
consistently slow to respond), the default value for the `interval_secs`
parameter to `stbt.press_until_match` is now configurable in stbt.conf.

Specifically, we have seen the following behaviour with a USB lirc-based
infrared emitter + the Hauppauge HDPVR USB video-capture device + stbt
running in a VM on an underpowered laptop:

1. The set-top box is displaying a menu or carousel; entry "A" is
   selected. The stbt script says:

       press_until_match("RIGHT", "menu-B.png")
       press("OK")

2. stbt presses "RIGHT".
3. stbt doesn't see the selection change in 3 seconds of video.
4. stbt presses "RIGHT".
5. In the very next frame, stbt sees the selection change to "B".
6. In the meantime, the set-top box changes the selection to "C",
   because of (4).
7. stbt presses "OK".
8. The set-top box opens menu entry "C", instead of "B" that we
   expected.

I'm not sure how much of the latency is introduced by lirc or the
infrared emitter, vs. the HDPVR, vs. stbt, vs. the system under test
itself. But the failures we have observed are all consistently a single
frame too late (the new menu selection is seen in the very next frame
after the last `press`; this is on a VM/laptop combination that only
manages < 2 frames per second).

The unit-test
`test_press_until_match_reads_interval_secs_from_config_file`
tests that `press_until_match` takes >=3s at the default configuration
(this should always be true, no matter what), and that it takes <3s when
configured with `interval_secs = 1`. Hopefully even on a heavily-loaded
PC (I'm looking at you, travis-ci.org continuous integration service),
2 seconds will be enough time (in that time `press_until_match` only
needs to process a single frame).
make doc: API docs should show default values from source config file

...not from the user's ~/.config/stbt/stbt.conf or $STBT_CONFIG_FILE.

315fb3a0 changed `press_until_match`'s default value for `interval_secs`
and `max_presses` to be read from the configuration file. This is done
at module load time, so the config file's value is reflected in the
output of `pydoc stbt.press_until_match`.

When the user runs `make doc`, the values should come from the stbt.conf
file distributed with stb-tester, not with whatever values happen to be
in the user's configuration file at that time.
stbt._log_image_descriptions [bug]: fix link to image in index page
tests/test-stbt-run.sh: Simplify tests

There's no reason for these tests to run "press" or "wait_for_match" --
all it does is increase the test runtime. And we can use "assert"
directly from python, instead of writing to a log and later grepping
that log.
test_extra_arguments: Test "--" argument to stbt-run

Commit bf393c1d allowed users to pass arguments to the test script on
the stbt-run command line. You can put a "--" between the normal
stbt-run flags and these extra arguments, and the "--" isn't passed on
to the user script. This wasn't documented in bf393c1d, but it's just
the behaviour of python's argparse. This test documents that behaviour,
which is relied on by the next commit.
runner/run: Allow test arguments starting with "-"

Although `stbt run` allows us to specify test arguments, if these start
with a dash, these will be processed as `stbt run` arguments instead of
test ones, i.e.:

    stbt run mytest.py --my-flag

In order to get that working we need to use bash's double dash to force
it to stop taking arguments:

    stbt run mytest.py -- --my-flag

Because of this, when using the runner, it's not possible to do:

    run mytest.py --my-flag

And `run mytest.py -- --my-flag` wouldn't be acceptable because the
double dash is used to separate one test and its arguments from another.