register reporters on test config

When doing the layout validation, I ended up spammed with a few errors:

  ERROR:zuul.Scheduler:Invalid reporter name gerrit

The issue is that my pipelines use 'gerrit' as reporter while it is not
registered when testing the config.  I registered the 'smtp' reporter as
well and the error is gone.

Side effect: the layout validation output now dumps actions for
start/success/failure.

Change-Id: I271a2943fa3e846ae60d9b615cd3a1ac3815bb1b
Doc: enhance gating reference with diagrams

ASCII art is fun but not savvy for project managers and directors. This
patch slightly enhance the 'gating' documentation with colored diagrams.

This is made possible via http://blockdiag.com/ by Takeshi KOMIYA who
even took the time to write a sphinx extension.  The version dependency
is at the very least 0.5.5, but might be higher :/

Change-Id: Ibe3c2674a5dff2114c40a84ffdec8a8886b1b21b
Merge "register reporters on test config"
Clone repo if it is not on disk

* zuul/merger.py: Previously zuul's merger only checked an initialized
flag to determine if a repo should be cloned. If the state of the disk
changes under zuul the repo may need to be recloned even if the
initialized flag is set. Check if the repo exists on disk to determine
if the repo should be cloned to deal with external state changes. Keep
initialized flag in order to preserve ability to change git repo
settings for user and email.

* tests/test_scheduler.py: Add a test that checks a repo is recloned
when removed from the filesystem under zuul.

* tests/fixtures/layout-repo-deleted.yaml: Layout fixture for new test
added above. This creates a new repo unused by any other tests so that
failures of this test do not interfere with other tests, they will run
independent of each other.

Change-Id: I14fb34a2916002cefef73e41ec9182a073d59ef3
GC git.Repo objects in merger.Repo operations.

* zuul/merger.py: Zuul's merger.Repo is holding on to repo objects that
keep many file descriptors open. Turn all git.Repo objects into method
local references so that the git.Repo objects can be garbage collected
once these methods are popped off the stack. GCing these objects frees
the file descriptors associated with that repo. This adds overhead to
repo operations as each operation must do initial setup IO.

* tests/test_scheduler.py: Assert that no instances of git.Repo are
known to the python garbage collector. Do the assertion after running a
fully garbage collection run. This is a cheap check that repo objects
are being garbage collected properly at the end of each test.

Change-Id: I1d67981f32708a85af62ca622402de9fac0e1842
Merge "Doc: enhance gating reference with diagrams"
Merge "Clone repo if it is not on disk"
Merge "GC git.Repo objects in merger.Repo operations."
Add the possibility to pass the zuul url

This patch adds the zuul_url option in zuul conf file
in order to pass ZUUL_URL to Jenkins, which will be used
by devstack-vm-gate-wrap.sh.
Documentation added in launchers.rst and zuul.rst
explaining how this new option works.

Change-Id: I840423cc06fdfdacd301d30be3e0b3e589e563e9
Add a zuul client

Add a command line client called 'zuul' that supports one command
to start with: 'enqueue'.  It allows an operator (one with access
to the gearman server) to enqueue an arbitrary change in a specified
pipeline.  It uses gearman to communicate with the Zuul server, which
now has an added RPC listener component to answer such requests via
gearman.

Add tests for the client RPC interface.

Raise an exception if a Gerrit query does not produce a change.  Unlike
events from Gerrit, user (or admin) submitted events over the RPC bus
are more likely to reference invalid changes.  To validate those, the
Gerrit trigger will raise an exception (and remove from its cache) changes
which prove to be invalid.

Change-Id: Ife07683a736c15f4db44a0f9881f3f71b78716b2
Fix merging transitively-connected change queues.

When combining change queues, given 3 projects that were transitively
connected by shared jobs, depending on the order of processing, it
was possible for them not to be combined.  To correct this, repeat
the combining operation until the resulting set can be combined no
further.

In order to make the test (and actual usage) behavior more deterministic,
the list of projects returned by the pipeline is now sorted by name.

A test is added for this.

Change-Id: If1386cad4118257efee9aa9918ad12a626927038
Ignore .venv directory

Change-Id: I477ef147d2f1d04487613049139107f6b8bb0a3b
Signed-off-by: Paul Belanger <paul.belanger@polybeacon.com>
Document zuul_url breaking change

The change to add zuul_url breaks backwards compatibility, so
document it in NEWS.

Change-Id: Iac4e6540f5383a570961b157b15ad4707c520414
Merge "Fix merging transitively-connected change queues."
Merge "Add a zuul client"
Merge "Document zuul_url breaking change"
Remove unecessary debug message

Change-Id: I7d1c22c022913994e68c651e21ff127eac61f670
Document zuul client

Adds programoutput sphinx extension as a test dependency so doc
builds can include the program help text.

Change-Id: Iec2f09f710162614cbb393a5628204ddebe2e29f
Merge "Remove unecessary debug message"
Refactor reconfiguration into a management event

Create a new management event queue to handle external requests
for actions that are not related to triggers or results.

Use this for reconfiguration events.  Subsequent patches will use
it for other kinds of events.

Change-Id: Ia018d9a7acc35aaf615ca85f75c9f4c630a5287f