test: support voluptuous 7.0+

Voluptuous 0.7.0 introduced a backward API incompatible change in
upstream commit 475adebc:

  https://github.com/alecthomas/voluptuous/commit/475adebc

The schema are precompiled and the validate_X have been removed,
voluptuous is now smart enough to detect the type of value being
validated and would call an internal validation method matching the
type.

Commit is contained since 0.7.0:

 $ git tag --contains 475adebc
 0.7.0
 0.7.1
 0.7.2
 0.8.1
 0.8.2
 $

I tested it using etc/layout.yaml-sample and added an inexistent
pipeline to the project. The test yield:

  voluptuous.MultipleInvalid: extra keys not allowed @
  data['projects'][0]['nonpipe']

Debian has recently accepted voluptuous 0.8.2 in testing, so if we want
to package Zuul, we better have to upgrade our voluptuous requirement as
well. Ref: http://packages.qa.debian.org/v/voluptuous.html

Change-Id: I117ea644863b2e4a4dc3429aa81e868573382877
register reporters on test config

When doing the layout validation, I ended up spammed with a few errors:

  ERROR:zuul.Scheduler:Invalid reporter name gerrit

The issue is that my pipelines use 'gerrit' as reporter while it is not
registered when testing the config.  I registered the 'smtp' reporter as
well and the error is gone.

Side effect: the layout validation output now dumps actions for
start/success/failure.

Change-Id: I271a2943fa3e846ae60d9b615cd3a1ac3815bb1b
Doc: enhance gating reference with diagrams

ASCII art is fun but not savvy for project managers and directors. This
patch slightly enhance the 'gating' documentation with colored diagrams.

This is made possible via http://blockdiag.com/ by Takeshi KOMIYA who
even took the time to write a sphinx extension.  The version dependency
is at the very least 0.5.5, but might be higher :/

Change-Id: Ibe3c2674a5dff2114c40a84ffdec8a8886b1b21b
Merge "test: support voluptuous 7.0+"
Merge "register reporters on test config"
Clone repo if it is not on disk

* zuul/merger.py: Previously zuul's merger only checked an initialized
flag to determine if a repo should be cloned. If the state of the disk
changes under zuul the repo may need to be recloned even if the
initialized flag is set. Check if the repo exists on disk to determine
if the repo should be cloned to deal with external state changes. Keep
initialized flag in order to preserve ability to change git repo
settings for user and email.

* tests/test_scheduler.py: Add a test that checks a repo is recloned
when removed from the filesystem under zuul.

* tests/fixtures/layout-repo-deleted.yaml: Layout fixture for new test
added above. This creates a new repo unused by any other tests so that
failures of this test do not interfere with other tests, they will run
independent of each other.

Change-Id: I14fb34a2916002cefef73e41ec9182a073d59ef3
GC git.Repo objects in merger.Repo operations.

* zuul/merger.py: Zuul's merger.Repo is holding on to repo objects that
keep many file descriptors open. Turn all git.Repo objects into method
local references so that the git.Repo objects can be garbage collected
once these methods are popped off the stack. GCing these objects frees
the file descriptors associated with that repo. This adds overhead to
repo operations as each operation must do initial setup IO.

* tests/test_scheduler.py: Assert that no instances of git.Repo are
known to the python garbage collector. Do the assertion after running a
fully garbage collection run. This is a cheap check that repo objects
are being garbage collected properly at the end of each test.

Change-Id: I1d67981f32708a85af62ca622402de9fac0e1842
Merge "Doc: enhance gating reference with diagrams"
Merge "Clone repo if it is not on disk"
Merge "GC git.Repo objects in merger.Repo operations."
Add the possibility to pass the zuul url

This patch adds the zuul_url option in zuul conf file
in order to pass ZUUL_URL to Jenkins, which will be used
by devstack-vm-gate-wrap.sh.
Documentation added in launchers.rst and zuul.rst
explaining how this new option works.

Change-Id: I840423cc06fdfdacd301d30be3e0b3e589e563e9
Add a zuul client

Add a command line client called 'zuul' that supports one command
to start with: 'enqueue'.  It allows an operator (one with access
to the gearman server) to enqueue an arbitrary change in a specified
pipeline.  It uses gearman to communicate with the Zuul server, which
now has an added RPC listener component to answer such requests via
gearman.

Add tests for the client RPC interface.

Raise an exception if a Gerrit query does not produce a change.  Unlike
events from Gerrit, user (or admin) submitted events over the RPC bus
are more likely to reference invalid changes.  To validate those, the
Gerrit trigger will raise an exception (and remove from its cache) changes
which prove to be invalid.

Change-Id: Ife07683a736c15f4db44a0f9881f3f71b78716b2
Fix merging transitively-connected change queues.

When combining change queues, given 3 projects that were transitively
connected by shared jobs, depending on the order of processing, it
was possible for them not to be combined.  To correct this, repeat
the combining operation until the resulting set can be combined no
further.

In order to make the test (and actual usage) behavior more deterministic,
the list of projects returned by the pipeline is now sorted by name.

A test is added for this.

Change-Id: If1386cad4118257efee9aa9918ad12a626927038
Ignore .venv directory

Change-Id: I477ef147d2f1d04487613049139107f6b8bb0a3b
Signed-off-by: Paul Belanger <paul.belanger@polybeacon.com>
Document zuul_url breaking change

The change to add zuul_url breaks backwards compatibility, so
document it in NEWS.

Change-Id: Iac4e6540f5383a570961b157b15ad4707c520414
Merge "Fix merging transitively-connected change queues."
Merge "Add a zuul client"
Merge "Document zuul_url breaking change"
Remove unecessary debug message

Change-Id: I7d1c22c022913994e68c651e21ff127eac61f670
Document zuul client

Adds programoutput sphinx extension as a test dependency so doc
builds can include the program help text.

Change-Id: Iec2f09f710162614cbb393a5628204ddebe2e29f