Update URL for openstack zuul example

Change-Id: Iae253977eafaff1a7cbdcaf1b873388487c4e36e
Signed-off-by: Paul Belanger <paul.belanger@polybeacon.com>
Reviewed-on: https://review.openstack.org/21730
Reviewed-by: Elizabeth Krumbach <lyz@princessleia.com>
Approved: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Tested-by: Jenkins
Dequeue changes when new patchset created.

When a new patchset is created for a change that is in a pipeline,
cancel running builds and dequeue that change (and possibly
dependent changes that can no longer merge).

Make this optional (and document the option).

Fixes bug 1022643.

Change-Id: I8f591956cf86645443e4b6075b8cdfc95a939e4f
Reviewed-on: https://review.openstack.org/20948
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Dequeue changes if they have no hope of merging.

In the DependentPipeline remove changes if they have absolutely no
chance of merging (all possible merge combinations fail). This allows
zuul to remove changes as soon as possible when it is known that they
cannot merge and continue testing the remaining changes.

Fixes bug 1123329

Change-Id: I97af03cca997541f6a8b3fb519d2a44825dbd937
Reviewed-on: https://review.openstack.org/21723
Reviewed-by: James E. Blair <corvus@inaugust.com>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
isUpdateOf check field presence before comparison.

isUpdateOf is passed an "other" changeish that may be a Ref which does
not have number and patchset fields which are compared against in
Change's isUpdateOf method. Check that fields exist before comparisons
are made.

Change-Id: Ifb5a36dcb0377f496e6b0707ae27810b96f2c752
Reviewed-on: https://review.openstack.org/22159
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
Revert "Dequeue changes if they have no hope of merging."

This reverts commit dd26978d5da1e37f75e7fc38201c0d5427417d72.

The performance of mergability checking is poor and starves zuul from
processing other things. revert the change until we can come up with a
better performing solution to this problem.

Change-Id: I744031159572236f61b3fca45bb22492135ebcb2
Reviewed-on: https://review.openstack.org/22251
Reviewed-by: James E. Blair <corvus@inaugust.com>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: James E. Blair <corvus@inaugust.com>
Make canceling a queue item a POST request.

Needed by Jenkins 1.502.

(Also don't print the results of error messages so they don't
clutter stdout, but leave them there as a debug option
comments because they are very handy when debugging
Jenkins API issues.)

Change-Id: I2e1dcc1ac2344b1eb9f66292b6fd30900cb45e6d
Reviewed-on: https://review.openstack.org/22579
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Update Voluptuous requirement to version 0.6.

Voluptuous has a version 0.6 now which deprecates the any, match, and
required predicates and replaces them with Any, Match, and Required.
Update the minimum required version of Voluptuous and stop using the
deprecated predicates.

Change-Id: I554a6fda2c622a0b63f9247b1eae5eb494af225b
Reviewed-on: https://review.openstack.org/22167
Reviewed-by: Khai Do <zaro0508@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Test that Zuul refs end up in the correct repos.

Change-Id: I5a0040e6cf1076dfb21e355b7c021fdfc1914ca5
Reviewed-on: https://review.openstack.org/22738
Reviewed-by: Anita Kuno <akuno@lavabit.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Correct two links.

The github project has changed from `openstack-ci` to `openstack-infra`,
but two links in the documentation do not reflect this change.

Change-Id: I9359d1071f33719da39fdefd14deea728b9dbfff
Reviewed-on: https://review.openstack.org/23033
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Remember merges to save time.

Previously Zuul had to merge the entire change tree for each change
in the DependentPipeline. Remember previous merges to save time and
only merge changes against their project and branch.

Co-Authored-By: James E. Blair <jeblair@openstack.org>

Change-Id: I8105ac3aa4e4697e90e8d7fac1e54c5614f79daf
Reviewed-on: https://review.openstack.org/22504
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Restart jobs behind a removed change.

Even though we're removing from (potentially) the middle of the
queue, restart jobs behind this change since the situation has
changed.

Change-Id: Iab44898bda07d991a3d4c53715bf7a04ff58c9d1
Reviewed-on: https://review.openstack.org/23038
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Make trigger/result queue lengths available to status page.

Change-Id: I79fc7d1df0612fbe87e9b61499111c2d08d78dd0
Reviewed-on: https://review.openstack.org/23046
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
support to update a repository

We sometime want to update a repository without fetching an actual
refspec. A typical example is to fetch tags, since I did not manage to
pass -t to git.fetch(), I am using instead the GitPython repo.update()
calls which binds to `git remote update`.

reset() add a clearing cache to make sure PyGit does not fail whenever
the remote repository has been updated. Instead of duplicating code
between reset() and update(), I have moved the recache command to
update() and make reset() to call this new update().

Change-Id: I0d989dfc1f31ede6c79ccdd1d5b3968978a418ab
Reviewed-on: https://review.openstack.org/19927
Reviewed-by: James E. Blair <corvus@inaugust.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
update remote on ref-update

Whenever receiving a ref-update event, for example when a tag is
created, deleted or changed, the local Zuul git repository would not
reflect the change.

This patch force a git remote update on the local git repository so the
Jenkins git plugin can actually find the new tag. This is done before
enqueing the change in the pipeline to only update the remote once.

Change-Id: I87a29aa15855bfb367f03da3e15cd8a733b757fd
Reviewed-on: https://review.openstack.org/19928
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Gerrit 2.5/2.6 stream-events has new event types

We run Gerrit 2.5 with most of 2.6 cherry picked in. At some point
Gerrit has added some new events to the `stream-events` command:

 - draft-published
 - merge-failed
 - reviewer-added

Each event comes with a field holding the account of the user performing
the action. Since Gerrit uses different names for that field, we have to
maintain a map to find out which key holds the account information.

This patch update the map for the three new events.

Also fixed a minor typo in an error message.

Change-Id: I79032280884e50edf83a3ddf60af7a2f62bde971
Reviewed-on: https://review.openstack.org/22690
Reviewed-by: James E. Blair <corvus@inaugust.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Give the result event queue priority.

Processing result events sooner means that we can potentially
stop builds or at least stop launching builds sooner.

New trigger events are only likely to cause us to start new
builds (which may need to be stopped).

Change-Id: I5c33ed74cf4af84900c96c2155b3cec5252248d5
Reviewed-on: https://review.openstack.org/23117
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Treat any non-success result as a failure.

Aborted, lost, skipped, unstable will all cause a change not to
merge, so treat them the same when deciding if any job for a
change failed.  This should stop new launches and cancel builds
faster when these alternate failures happen.

Also, make the output of the completed jobs in the test suite
more comprehensible by including the change numbers, and use
a new object with its own string formatting for printing.

Change-Id: I2826b0eb5d26a019fed7d20bb1296624588c9fac
Reviewed-on: https://review.openstack.org/23123
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Fix test race condition.

The fake jenkins queue may have been modified during the call to
get_queue_info, so operate on a copy of the queue (which is more
like reality than locking the queue because we get a copy of the
queue from the jenkins api).

Also, fix a string formatting error which for some reason was
not raising an exception until we tried running the tests on
quantal.

Change-Id: I51e588fb147515c826395439516cf843c2da83f3
Reviewed-on: https://review.openstack.org/23781
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: Jeremy Stanley <fungi@yuggoth.org>
Tested-by: Jenkins
Add a utility script for mass-comments.

The script prints a gerrit review command for every change in
a given zuul pipeline.

Change-Id: Ibdee87ea64c977e7589cbc3ee9567e866cb78341
Reviewed-on: https://review.openstack.org/23784
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Tested-by: Jenkins
Document zuul-changes.py tool script.

* doc/source/zuul.rst: Mention zuul-changes.py and provide sample
syntax in the restarting section.

Change-Id: I05888ca40e16681111e56b9ebf92acc3dc36175b
Reviewed-on: https://review.openstack.org/23938
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Tested-by: Jenkins