Clone repo if it is not on disk

* zuul/merger.py: Previously zuul's merger only checked an initialized
flag to determine if a repo should be cloned. If the state of the disk
changes under zuul the repo may need to be recloned even if the
initialized flag is set. Check if the repo exists on disk to determine
if the repo should be cloned to deal with external state changes. Keep
initialized flag in order to preserve ability to change git repo
settings for user and email.

* tests/test_scheduler.py: Add a test that checks a repo is recloned
when removed from the filesystem under zuul.

* tests/fixtures/layout-repo-deleted.yaml: Layout fixture for new test
added above. This creates a new repo unused by any other tests so that
failures of this test do not interfere with other tests, they will run
independent of each other.

Change-Id: I14fb34a2916002cefef73e41ec9182a073d59ef3
GC git.Repo objects in merger.Repo operations.

* zuul/merger.py: Zuul's merger.Repo is holding on to repo objects that
keep many file descriptors open. Turn all git.Repo objects into method
local references so that the git.Repo objects can be garbage collected
once these methods are popped off the stack. GCing these objects frees
the file descriptors associated with that repo. This adds overhead to
repo operations as each operation must do initial setup IO.

* tests/test_scheduler.py: Assert that no instances of git.Repo are
known to the python garbage collector. Do the assertion after running a
fully garbage collection run. This is a cheap check that repo objects
are being garbage collected properly at the end of each test.

Change-Id: I1d67981f32708a85af62ca622402de9fac0e1842
Merge "Clone repo if it is not on disk"
Merge "GC git.Repo objects in merger.Repo operations."
Add the possibility to pass the zuul url

This patch adds the zuul_url option in zuul conf file
in order to pass ZUUL_URL to Jenkins, which will be used
by devstack-vm-gate-wrap.sh.
Documentation added in launchers.rst and zuul.rst
explaining how this new option works.

Change-Id: I840423cc06fdfdacd301d30be3e0b3e589e563e9
Add a zuul client

Add a command line client called 'zuul' that supports one command
to start with: 'enqueue'.  It allows an operator (one with access
to the gearman server) to enqueue an arbitrary change in a specified
pipeline.  It uses gearman to communicate with the Zuul server, which
now has an added RPC listener component to answer such requests via
gearman.

Add tests for the client RPC interface.

Raise an exception if a Gerrit query does not produce a change.  Unlike
events from Gerrit, user (or admin) submitted events over the RPC bus
are more likely to reference invalid changes.  To validate those, the
Gerrit trigger will raise an exception (and remove from its cache) changes
which prove to be invalid.

Change-Id: Ife07683a736c15f4db44a0f9881f3f71b78716b2
Fix merging transitively-connected change queues.

When combining change queues, given 3 projects that were transitively
connected by shared jobs, depending on the order of processing, it
was possible for them not to be combined.  To correct this, repeat
the combining operation until the resulting set can be combined no
further.

In order to make the test (and actual usage) behavior more deterministic,
the list of projects returned by the pipeline is now sorted by name.

A test is added for this.

Change-Id: If1386cad4118257efee9aa9918ad12a626927038
Ignore .venv directory

Change-Id: I477ef147d2f1d04487613049139107f6b8bb0a3b
Signed-off-by: Paul Belanger <paul.belanger@polybeacon.com>
Document zuul_url breaking change

The change to add zuul_url breaks backwards compatibility, so
document it in NEWS.

Change-Id: Iac4e6540f5383a570961b157b15ad4707c520414
Merge "Fix merging transitively-connected change queues."
Merge "Add a zuul client"
Merge "Document zuul_url breaking change"
Remove unecessary debug message

Change-Id: I7d1c22c022913994e68c651e21ff127eac61f670
Document zuul client

Adds programoutput sphinx extension as a test dependency so doc
builds can include the program help text.

Change-Id: Iec2f09f710162614cbb393a5628204ddebe2e29f
Merge "Remove unecessary debug message"
Refactor reconfiguration into a management event

Create a new management event queue to handle external requests
for actions that are not related to triggers or results.

Use this for reconfiguration events.  Subsequent patches will use
it for other kinds of events.

Change-Id: Ia018d9a7acc35aaf615ca85f75c9f4c630a5287f
Add a promote client command

Takes one or more changes and promotes them to the head of the queue.

Also, change the command line syntax for the enqueue command to accept
change IDs in the form 'change,patchset' in order to match the syntax
of promote, as well as be potentially more compatible with future
triggers.

Change-Id: Ic7ded9587c68217c060328bf4c3518e32fe659e3
Pin Sphinx to <1.2

1.2 was just released and has a bug that causes builds to fail with
the error:

  error: 'source_dir' must be a directory name (got `...doc/source`)

See: http://bugs.python.org/issue19570
and: https://bitbucket.org/birkenfeld/sphinx/pull-request/193/builddoc-shouldnt-fail-on-unicode-paths/diff

Change-Id: I53b49cdc00851f0bddfdd18d0cf4e095a917aaa4
Merge "Refactor reconfiguration into a management event"
Merge "Add a promote client command"