virtual-stb/service/Dockerfile: This is a combination of 85 commits.

fixup! Add Dockerfile

fixup! Makefile: Add docker-build target

extra/virtual-stb: Add unit test for netstat output parsing

This test doesn't run the real netstat so it doesn't prove that the
function works in the real world, but at least it documents the parsing
code and what the expected input looks like.

Dockerfile: git is now a runtime dependency of stb-tester

Dockerfile: Set stb-tester user's $HOME

I don't know why it isn't set properly inside the container.

fixup! extra/virtual-stb: Work around netstat limitations in Docker container

WIP: Fix virtual-stb shutdown

TODO:
* Better commit message

extra/virtual-stb: Rename README => README.md

So that the github web interface displays it nicely.

fixup! virtual-stb: Add readyness notification

virtual-stb: Pull stbt config out into seperate Python file

Add make install-virtual-stb

Very few dependencies

fixup virtual-stb: Ensure .X11-unix exists and has right permissions

Fixes virtual-stb on systems not already running X

WIP: Add stbt-ssh-endpoint

Intended to allow creating a stateless machine which can be used by other
machines to run tests to help integrate stb-tester into other CI systems.

TODO:
* Document
* Test
* Combine with main Makefile
* Package-up

virtual-stb & service: Share timezone with Docker containers

Otherwise the time is incorrect.  I'm not certain whether this is a good
idea.  We should really be storing all timestamps as UTC and convert at
the edges via javascript or dynamically generated web-pages.  It still
leaves open the question of should we be sending a timezone to the test-
packs at all?

Also: perhaps a virtual-stb would want to have a different timezone.  This
needs to be solved before the first release of the service stuff as any
incompatible change there would be strictly forbidden.

fixup! WIP: Add stbt-ssh-endpoint

fixup! extra/service: Add web server container for displaying results

fixup! WIP: Add stbt-ssh-endpoint

fixup! fixup! WIP: Add stbt-ssh-endpoint

fixup! WIP: Add stbt-ssh-endpoint

REMOVEME: stbt-ssh-endpoint: Add command setup-lirc

This command should be removed once we figure out how to handle node
configuration properly and should definitely not make it into any
official releases.

fixup! REMOVEME: stbt-ssh-endpoint: Add command setup-lirc

extra/service/run: Fix pylint & nose errors

virtual-stb: Add run a fake-lircd in the virtual stb to inject key events

Allows customising the mapping between the keycode names that stbt sees and
the XKeySyms that X and the program under test understands.

Dockerfile: Load config file from tests repo

service: The system configuration should come from the host

service: Use the volumes from the virtual-stb

virtual-stb: Attempt to fix volumes

Fix add with-lircd to service

virtual-stb: Attempt to fix volumes

service: fixup

fix notification

vstb-lircd fixes

fix lirc socket path

vstb: Fix /proc/unix parsing code

Fixup lirc in virtual-stb

virtual-stb: Refactor so it can be used as a library by "service"

virtual-stb: Rename so it can be imported by other python files

service: fix using virtual_stb as a library

virtual-stb fixes

Add example virtual-stb

Build out example test

A better name for new example virtual-stb

Better name for test pack

Fix service/virtual-stb/Dockerfile and add test

service: Tidy-up

service: Fix Makefile/installation path

service: Fix make install-service

service: Fix paths

service: Fix direcotry

Fix service install direcotry

service: Fix PYTHONPATH

service: Another fix

service: Fix PYTHONPATH

virtual-stb: Fix and add test for custom key mappings

service tests: Move html5 test pack to make room for more

Add test for non-virtual-stb service run

service: Support running against system configured video capture

stbt virtual-stb: Fix handling of "--docker-image" arg

This fixes an error introduced in recent refactorings related to
ssh-endpoint / service.

stbt virtual-stb: Expose lirc via IP, not Unix-domain socket

This way it's easier to run "stbt run" on the host against a virtual-stb
running in a docker container (as opposed to when "stbt run" is also
running inside a docker container, which worked because the 2 containers
shared the /run/lirc/lircd socket).

fixup service/virtual-stb

Fixup service: Allow different paths to tests-pristine between host and container

Dockerfile/service: Add test-pack-entrypoint

test-pack-entrypoint works around some issues with running test-packs in
docker.

service: fixup stbt-ssh-endpoint

virtual-stb: fix running without docker

extra/service/serve-results: Fix .log mimetype, enable directory listing

Dockerfile: Refactor `make install`

Dockerfile: Don't `ENV DEBIAN_FRONTEND noninteractive`

It's better to export it in the individual rules as otherwise it ends up
in the environment of the commands run.

service: Add command to regenerate authorized_keys file for ssh login

Add Dockerfile for service

Fixup service

service: The docker command is `docker` not `docker.io`

We require docker >1.0 which even on Debian/Ubuntu uses the command
`docker`.

Fixup stbt service running in docker

fix make docker-build

rename service-users-volume

Fix make docker-publish

fixup docker

service: Add documentation

service README: Fix image

service README: Fix whitespace

service README: HTMLise table

github can't do multi-line table cells

fixup service & virtual-stb stuff (validating service installation instructions)

service run: Avoid `git checkout` every run

fixup service

service run: Avoid unnecessary git checkout

Dockerfile.service: Speed up docker build

...by caching build dependencies. This comes at the cost of a larger
docker image.

service run: TODO: pull/reset latest changes if "revision" is a branch

fixup service

service: Fix concurrent running of tests

service: Add a bunch of tests

fixup service

fixup service

extra/virtual-stb: Don't hardcode python location

So that when I run `stbt virtual-stb` on my Mac, it uses the python from
macports instead of the system python (otherwise python won't be able to
import all my third-party python libraries that I installed with
macports).

virtual-stb: Show virtual-stb.log if it failed to start

virtual-stb: Document why we use `os._exit`

virtual-stb: Flush std{out,err} before exiting

A typical usage pattern is:

    export $(stbt virtual-stb --daemonize --docker-image xxx)

to export the environment variables `VSTB_DOCKER_CID` etc, which are
printed to stdout by `stbt virtual-stb`. However, since with
`--daemonize` we use `_exit` instead of `exit`, this would
intermittently fail to work because `stbt` exited before anything was
printed.

docker: Fix `make docker-publish` and make more flexible

This allows you to customise the prefix and tag of the docker build
allowing publishing docker images to the docker hub under your username.
e.g.:

    make docker-publish docker_prefix=wmanley/ docker_tag=next

docker: Use stbtester as the docker prefix rather than stb_tester

I have registered the stbtester organisation on the docker registry.
Underscores are not allowed in the names.
WIP: Add "next" and "prev" links to debug HTML pages

TODO:
* Write tests
* Make it look better by right aligning next and left aligning prev
WIP: Report: Strip HTML from notes in table

Come up with a better idea.
WIP: HTML in notes can use the failure and success classes
Packaging: Add run-time dependency on git

For `stbt batch run`, which puts the git version of the test script
in the reports.
report: Add video to the top of the testrun page

It looks great and makes the awesome video recording feature more
visible.
report: Move toward a more data-oriented design

This will make it easier to change to a database backed data-store from
the current directory-backed one, but the reason I'm doing it now is to
make it easier to customise the extra-columns, as will be necessary to add
invisible extra columns which are search-able, but not visible.  This will
be useful so we can search by full git SHA, but only display the git
version description.
stbt batch report: Generate what data we can at `run-time`

The intention is that in the future the `data` member of `Run` will be
stored in a database.  The intention is that the database stores *facts*
and we wish to perform interpretation on these facts at run-time where
possible.  In this case we move the interpretation of what to do with each
of the files until template rendering time.
report: More data-oriented refactoring
report: Refactor: Declare columns in the code rather than in the template

This gives us more flexibility to customise the columns in the report
as is required to do custom formatting on extra-columns.
WIP: report: Allow customising the columns
stbt batch report: Don't HTML escape files we read

Jinja will escape them when writing HTML out and we don't want them
double escaped.  We also want the ability to include HTML in notes.
REMOVEME: stbt-control: Take screenshot on "s"

This should be replaced by a refactoring of stbt-control to include
stbt tv by

    import stbt
    stbt.init(...)

etc. and taking a screenshot and asking for a filename with ncurses.
stbt.py: Hide "libdc1394 error" when importing cv2 (OpenCV)

This happens on Ubuntu 12.04 and 14.04, at least.

The downside of this approach is that if a genuine error happens (such
as the cv2 module isn't installed) then useful output messages will be
hidden.
stbt.ocr: Show tesseract output when it fails
DON'T MERGE! HACK! Fix graph for report without "notes" and "failure reason" columns

Without this the graph doesn't populate, and javascript is broken so the
search doesn't work properly.
stbt batch instaweb: Allow editing testrun's exit status

As an easy way of changing red results (failures) to yellow (errors).
This doesn't update the main results table instantly, but it does invoke
`stbt batch report` so if you wait a few seconds and then press
"refresh" you should see the change reflected in the results table.
stbt-batch.d/templates/index.html: Remove unused code

Unused since b784a841.
stbt batch: summary report: List pass/fail/error counts per testcase

The code is ugly, but it works. I had initially tried to use
underscore.js's `_.groupBy` but I can't use underscore.js because of
https://github.com/asimihsan/tablequeryjs/issues/20
stbt batch run, stbt templatematch: Hide OpenCV "libdc1394 error" warning

Instead of importing stbt's `util.hide_stderr`, just use `stbt.cv2` and
let stbt.py do the error-hiding for us.