Make smtp test more robust

It may have ended up with changes still in the pipeline because
it uses the timer trigger.

Change-Id: Ieb46d7b5b7020e62e7d3baa704e3f75f9f87dca1
Maintain the trigger cache after reconfiguring

A previous change removed the trigger cache cleanup.  When Zuul
is live-reconfigured, it creates new Project objects and
attaches them to all changes currently in pipelines.  Because
the cache contained changes outside of pipelines, those changes
were not being updated with new Project objects.

This restores the maintainCache method and uses it during the
reconfigure event to clear the cache of all changes not currently
in pipelines.  This corrects the error introduced by the previous
change as well as gives Zuul an opportunity to release unused
memory (as now the cache will at least be emptied on each
reconfiguration).

The project-change-merged test is updated to explicitly test this
situation.

Change-Id: I67736fca08f2e14ab733bd9f143820da19839ef9
Merge "Make smtp test more robust"
BuildSet now has a representation

I had Zuul Scheduler report warning messages such as:

zuul.Scheduler: Build set <zuul.model.BuildSet object at 0x1a9a310> is
not current

Which is not very meaningful.  This adds a __repr__ to BuildSet which
shows up:
- the QueueItem representation
- number of builds in the set
- merge state

The merge states are internally stored using integer constants.  I have
duplicated the information in a dict to lookup the int as a meaningful
string.  That is not pretty :(

Change-Id: I0e087e4cb3f4c1da4ef40d0805793f737af1765e
Fix zuul javascript cookie load

The default filter value for the status page was not being set correctly.
As such the filter would incorrectly remove all results. When no cookie
is set, make sure the filter is blank.

Change-Id: I4868db8b357373d1fd94dc6ffc7528692b75dd8d
Fix status page overlap

If the job name was too long the progress bar would float under the
next job. Fix this be clearing the flots in each job line.

Change-Id: Ie476cd97ee6207fac84051e0580871590bccbc0f
Handle random PYTHONHASHSEED values.

Voluptions does a depth first in order traversal of the data structures
it is checking and fails fast. This means it will die when it hits the
first error and return only that error. The test_layouts test is
checking that different bad layout files fail in unique ways. This check
breaks when the python hashseed is random resulting in different depth
first traversals which can result in returning the same error twice if
bad layout files have common errors.

Fix this by trimming down the errors in each layout file to a single
error this way the order of the errors hit never matters.

Fix a string typo in test_layouts while we are in there.

Change-Id: Ie7587ebac4d22063a6819c68011f7fb67a49fcc0
Clean up bad layout files in zuul tests

Clean up the bad layout files in the zuul tests by removing a redundant
file (bad_pipeline and bad_pipeline1.yaml were identical). Annotate the
reason the layout is bad in each file. Finally remove a redundant swift
section. Both bad switf sections were failing for the same reason, a
missing name entry.

Change-Id: I0fd09da1fb0ec7443fa7d12a3b31809546a01afd
Merge "Maintain the trigger cache after reconfiguring"
Merge "BuildSet now has a representation"
Handle latest testtools tearDown checks

Latest testtools checks that you only call tearDown once (it also checks
that setUp is only called once). Zuul's test framework was calling
tearDown from a cleanup which resulted in tearDown being called twice.
Once by the test framework and once by zuul's testclasses cleanups.

Just remove the explicit tearDown call in the cleanups and let the
larger python tooling call tearDown for us.

Change-Id: Id8eaa9da5153fdfe9b9f852d2c57838736cb7040
Fix cloner test

The cloner test was incorrectly asserting that project2 should
always be at the commit specified for change B.  However, when
testing change A, project2's repo should be at master.

The cloner actually works as intended, the reason the assertion
passed is that the cloner was being run _after_ the gate jobs
finished, meaning that the master branch of project2 actually
was the commit specified for change B -- because it had merged.

Correct this by moving the build release to the end of the test,
so that the test operates entirely within the time that the jobs
are building (before anything merges).  Change the checks that
the commits are correct so that they assert the different and
correct values for each of the builds being tested.

Additionally, remove some unecessary actions from the test (adding
additional patchsets and specifying a clone map that is not used).

Change-Id: I7575e8b84925115393cef81d6b02a3532913ffed
Add check for ref being a string before applying regex

Incorrect filter by ref in zuul layout could end on
applying a regex to a None value, causing an error and
zuul crashing. That is not an usual thing but could happen
and zuul crashes because of that.

Add the comparison for ref not being None
before applying the regex, as it was done for comment
comparison.

Change-Id: Ie58bb288a1f7609564283a0062e08ecc45b28cd0
Cloner: default to ZUUL_BRANCH

The cloner takes an argument for the "indicated branch", however,
that argument has a default value of None, rendering it optional.

In the original setup method from devstack gate, that argument
was not optional, instead, it was typically set to ZUUL_BRANCH
unless it needed to be specified otherwise, for example for grenade.

Effectively, if omitted the (optional) branch argument to the cloner
it would use the 'master' branch in preference to the ZUUL_BRANCH
for projects that do not have a zuul ref.  This is not the desired
behavior in the simple case (or quite likely any case), and therefore
should not be the default behavior for the cloner.

Resolve this by having the cloner treat the indicated branch as
ZUUL_BRANCH unless it is specified in the branch argument.  In doing
this, the logic in the cloner is slightly simplified and now more
closely matches that in the setup_project function in devstack-gate.

Rename the cloner test to 'test_one_branch' to match the pattern from
devstack-gate tests.  This test encompases the 'test_one_on_master'
and 'test_two_on_master' tests from d-g.  They are combined because
in these tests we are able to iterate over the Zuul builds and check
the state for each build in turn, which is a more complete form of
testing than what was employed in d-g.

Add the 'test_multi_branch' test to the cloner.  This encompases
'test_multi_branch_on_master' (with an additional fourth project
with no changes), as well as 'test_multi_branch_on_stable'.

Change-Id: Ib26bced46073bd61298c52c836a20085511201f3
Cloner: add upgrade test

This is the 'test_grenade_forward' and 'test_upgrade_backward'
tests from d-g.  Both scenarios are covered because they both
simulate havana->master upgrades, just changing which of them
is the 'current' change.  Since we examine the states of all
changes, in the course of running the test we check the results
for both when a current change is havana and master.

This also covers the 'test_branch_override' case from d-g.

Change-Id: I95abd62279bd44e88a608ad908f4876d11d50ec2
Cloner: add project-specific branch overrides

Add a facility to override the indicated branch for a specific
project.  In OpenStack, this is used to specify that tempest should
always use its master branch even though it has stable branches
(we no longer use stable branches for tempest).  It may also be
useful in some kinds of compatibility cross-testing.

The test for this is taken from 'test_multi_branch_project_override'
in devstack-gate.

Change-Id: I958af233d1def3b1c7362e1b2ddc77c108766358
Add cache-dir argument to cloner

Clone repos from cache-dir if they exist, then reset their origins
to be the git base url and update from there.

Change-Id: Ic42961aa5da9e4a66524ec14758d421944403f91
Clarify docs for optional template pipelines

The documentation could be read to imply a check, gate, et cetera
pipeline must contain a job, when in fact pipeline definition is optional
in templates.
.
Inform the reader of the documentation as it isn't entirely clear for
folks maintaing projects where the developer is not a zuul developer.

Change-Id: If386e9431d486f8cea2ebde52c555305e1278c7d
Merge "Fix cloner test"
Merge "Cloner: default to ZUUL_BRANCH"