Generate AUTHORS and ChangeLog for zuul.

Bug: 1027347

Inline with changes in other projects, generate AUTHORS
and ChangeLog to include in release package.

* .gitignore
  Ignore these 2 files.

* setup.py
  Command class to generate these files, from openstack.common.

* zuul/openstack/common/setup.py
  Import from openstack.common project.

* openstack-common.conf
  Configuration file to sync changes using the update script.

Change-Id: I221366b9ad183a463349b4b3093f843fba6c9195
Test changes in dependency order.

For dependent change queues, test the changes in dependency order,
as reported by gerrit.

Effectively, this means that a change is enqueued only if:
  * it can be merged
  * the change it depends on has been merged or:
    * can be merged
    * is the current patchset

Also, once a change is enqueued, changes that depend on it are
examined to see if they can be merged (assuming this one succeeds).
If so, they are enqueued.

Change-Id: I917b2a2d1fc94c3aa2de406ed4b9f795a8abb079
Add functional tests.

Add support for testing, and some initial functional tests.

Some threads in Zuul now have proper shutdown methods, since
that's needed for testing.  The threads that are faked in the
testing infrastructure still lack shutdown methods.

Fix some bugs uncovered by tests:

Reverse the order of commits that are given to Jenkins (the
commits that preceded the one being tested were listed in
revers order, this corrects that).

Return 0 in reportChange if the change has already been
reported.  The old return value of True was misinterpreted
as an error.

Change-Id: I4979d0efd8581061a2b856b892d4ecdb75284a1b
Reviewed-on: https://review.openstack.org/10572
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Add more tests.

Add two tests:
 * Changes behind failed changes get retested
 * Shared change queues

Also, add the ability to simulate that a job for a change failed.

Change-Id: Ia266a9a6a8920ef49b24025f8109c4414c563e8c
Reviewed-on: https://review.openstack.org/10578
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Add Sphinx to test-requires.

Needed for the gate-zuul-docs job.

Change-Id: Ib330aeaf322543ab1f8b92e48672b0b0b4a4f692
Reviewed-on: https://review.openstack.org/10681
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Add more tests.

Add tests for canceling/aborting jobs behind a failed change.

Change-Id: I589f5eda64223ad8ab0e00d936456b568b5aa4c9
Reviewed-on: https://review.openstack.org/10613
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Add more tests.

 * Test that patches are queued in gerrit dependency order.
 * Test that we correctly decide when a change is able to merge.

Change-Id: Ib541314e7c956202a3f1f33a6b2185dd22f83f73
Reviewed-on: https://review.openstack.org/10633
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Wait for a commit to appear in the repo.

This fixes a potential race condition where a change may be reported
as merged in gerrit, but due to replication, the commit may not
yet show up in the repo.  If a job is started during this time frame,
then the git repo will not be in the expected state for testing.

This now waits for the commit associated with the latest patchset
for the change to become the commit at ref/heads/branch.  This
check obviously only works to verify that the merged commit is the
_most recently_ merged commit (ie, it does not perform the more
general check that the commit was merged at some time in the past).
That's okay here because zuul's merge commands are issued
synchronously from a single thread, so if zuul instructs gerrit
to merge a change, it should become that head, and no other changes
should be able to merge during that time.

Change-Id: I2213916c4850e7be80795cbed2dac29eacdb82bf
Reviewed-on: https://review.openstack.org/10673
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Fix the zuul-coverage job (test-requires update).

The zuul-coverage job depends on the coverage tool. Add it to
tools/test-requires list.

Change-Id: I4c5609f7992dd6dae649f5c059a69bb06b021831
Reviewed-on: https://review.openstack.org/10684
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Add pipelines to data model.

This is a refactoring of the data model with the following goals:

 * Call top-level queues pipelines -- because too many other things
   are already called queues.  Pipelines convey the idea that there
   are a number of tasks to be performed (jobs), and that those
   tasks can be applied to different changes in parallel.
 * Eliminate references to queue_name from within a Change.
   Instead, methods that need to understand the pipeline that were
   accessed previously via the change are now located in the
   Pipeline class, taking a change as an argument.  Essentially,
   many methods involving changes (and builds, jobs, etc) must now
   be called in the context of a pipeline.
 * Add a changeish object to encompass the things that change and
   ref events have in common.

Change-Id: Iaf8ed0991f3c5b2bf7ded2c340a60725f7f98eaf
Reviewed-on: https://review.openstack.org/10757
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Just check that ref has changed when merging.

Record the sha at refs/head/branch before each report.  Then if
isMerged is called, just check that the sha has _changed_ since
then, and if it has, assume that means it has reached the replica.

This handles the case of a merge or cherry-pick commit being the
actual commit that ends up at the ref, without the overhead of
fetching and parsing that commit to see if it's really the change.
This should be sufficient as long as Zuul is the only thing
triggering merges.

Change-Id: I50cfa7b4c011d9fd108046914c7bd236858ff5d3
Reviewed-on: https://review.openstack.org/10970
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Fix bug with looking up a job tree.

If there are no jobs for a project in a pipeline, an exception
would be raised.  This change ensures that case is checked.

The test fixtures are updated to simulate this condition as well.

Change-Id: Id8e5bdef6288e5574d8553c11f5ac8e8c6c61704
Reviewed-on: https://review.openstack.org/10973
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: James E. Blair <corvus@inaugust.com>
Fix typo in status format.

Change-Id: I4e8d5ca2a91595ba1ae3aaba4e64dd1b75665027
Reviewed-on: https://review.openstack.org/10972
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: James E. Blair <corvus@inaugust.com>
Fix HTML status for dependend change queue.

Change-Id: If14af44d13a0506501a88af0c12fca76862e1b8c
Reviewed-on: https://review.openstack.org/10974
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: James E. Blair <corvus@inaugust.com>
Fix problems with ref pipelines.

Bugs introduced by refactoring.

Change-Id: I4bc2440317f2def84308d1337649a8cdebf0ac34
Reviewed-on: https://review.openstack.org/10980
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Return correct pipeline name in str method.

Change-Id: I03e9ec477b1d7493984fc6082bf30425ea8f5ac2
Reviewed-on: https://review.openstack.org/10981
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: James E. Blair <corvus@inaugust.com>
Log exceptions formatting HTML status.

Change-Id: I13a5f5f8ab8697c8274eab541a8eba1604e143fa
Reviewed-on: https://review.openstack.org/10984
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Fix status output for ref changes.

Change-Id: I75b9f9b2c059fae776ab395951242cda0153c7ab
Reviewed-on: https://review.openstack.org/10996
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Merge branch 'master' of https://github.com/openstack-ci/zuul