Use the full release string when showing version

My Zuul version shows up as:

 $ zuul --version
 2.0.0.55

Since I have a local hack to slightly adjust the requirements, it is
really version 2.0.0.54 with my hack.  To help disambiguate this change
make the version string to report the full release string, ie including
the git commit.  Yields:

 $ zuul --version
 2.0.0.55.g1395090
 $

Where 1395090 is the commit sha1 of my cherry pick.  Ideally it would be
something like:  2.0.0.54.1.g1395090 to show up that one change is
diverging from upstream.

Change-Id: I32f5e8a1a84943e920f91ac1fa90784a4ac1fe6b
cloner to easily clone dependent repositories

The intent is to replace the devstack shell script with a python utility
which would be easy to reuse.

Change-Id: I3c8748e2544af80e72fdd0b2e3627b4fc2212c01
Add toggleable yappi profiling to zuul

Add a bit more functionality to the SIGUSR2 handler in zuul. When that
handler is run toggle the state of the yappi profiler and when we turn
the profiler off log the profiling data. Do this in addition to the
normal thread dumps.

Change-Id: I127cfb924f0576072d26bde0b4204f8636195d97
Merge "Use the full release string when showing version"
Stop gzipping zuul status json in zuul

It is expensive for zuul to do gzipping of data in zuul. Stop gzipping
here and have apache (or some other proxy) do it for you in cases where
smaller json is helpful. This offloads the work from zuul so that it can
focus on scheduling tests.

partial-bug: 1326170
Change-Id: Ieef0a19b4b36b28118a6a93ebd595a5287848034
Cache zuul status.json for one second.

Serve previously rendered zuul status.json if it is was formatted less
than one second prior. This caches the status.json reducing expensive
calls to format the status.json.

The cache ttl is configurable using the zuul.status_expiry configuration
option in the zuul config.

Change-Id: Ib3fcd75f30bc4d4ef5f09d2adfd566d10c58a991
Merge "Stop gzipping zuul status json in zuul"
Merge "Cache zuul status.json for one second."
Merge "Add toggleable yappi profiling to zuul"
Set Last-Modified header when serving status.json

Apache won't cache anything without a Last-Modified header. Be a good
citizen and set the Last-Modified header when serving status.json data.
Set the value to the cache_time timestamp.

Update the status checking test to make sure that this and other headers
are properly present so that this does not regress.

Change-Id: I2215d9c9694bd9ef7b94f29f654da77ae85a9959
wmf: soften requirements

WebOb and paramiko have a minimal version which is enforced by OpenStack
but is most probably not needed.

ordereddict is for python 2.6, the feature is included in python 2.7.

Change-Id: I1aa1e3ae496d68d25dd64e8b21d2057c473342f4
wmf: integrates Zuul cloner
wmf: disable swift dependencies

We are not going to use swift at all. So get rid of the deps.

Change-Id: I0bb5d07d61a2f35df913bad9d791fa345a426c55
Make swiftclient an optional dependency

Swift is only used when there is a [swift] configuration section, but we
always import the swiftclient module.

I can not use the switclient on my setup, so I thought about loading it
via extras just like we do for statsd to transparently ignore the
missing dependency.

Another way would be only import it when we have a [swift] section but
that makes it harder to find out that is an optional dependency.

Change-Id: Ie2ef84f86f9f57796a85758147ba87d8b22c3e55
wmf: Make swiftclient an optional dependency

Drop the dependencies:
  python-swiftclient
  python-keystoneclient

Adjust Zuul to not load swiftclient by cherry picking a change pending
upstream: https://review.openstack.org/#/c/97933/