Archive Ironic VM nodes console logs

Set log directory in localrc for Ironic deployment and archive these
logs as part of cleanup_host().

Change-Id: Ia99e5b22cc43124c582369b7ddaf0455d2d606e4
Use the DEVSTACK_GATE_USE_APACHE flag to disable

Instead of conditionally enabling Apache based services with the
DEVSTACK_GATE_USE_APACHE flag, if set to '0' now prevent any
services with alternative run configurations from running under
HTTPD + mod_wsgi (e.g. Keystone, Swift). This inversion of logic
is being used to allow for enabling services to run under
HTTPD + mod_wsgi by default.

Change-Id: Ia8e4a1dde9dbebcb522110ff5e9522d29c7fdc6f
Support for alternate trunk name (not master)

Commit 537a2926745a2acbacfa682295f06fa44c42716a introduced support
for having trunk named something other than master.

The name "master" is still hardcoded in the test-matrix, and in
the list of allowed branches. Adding support for an arbitrary
features file via DEVSTACK_GATE_FEATURE_MATRIX env variable.

The list of allowed branches is now parsed from the feature matrix.

Change-Id: Ic07c958e1630f7e62d9ff35b0add40a5d2b1548b
Merge "Support for alternate trunk name (not master)"
Fix typo in DEVSTACK_GATE_TEMPEST_NOVA_V3_API

This commit fixes a small oversight in setting the nova v3 api tempest
tests. When this env variable was added to set whether the testing was
enabled the check used a different variable name than the wrap script.
Also neither of these variables matched what was being set by jenkins,
this meant that the v3 api tests would never be enabled. This fixes
that by ensuring the variables are the same and matches the variable
set in the JJB template for the v3 api jobs.

Change-Id: I54925a468c4dc7eb9390a78d509b9385caa587d4
Add keystonemiddleware to devstack-gate

Add the keystonemiddleware project to devstack-gate

Change-Id: I1fb6a238593dc8620a97fad6a6478c8e0bc11fec
add network_sanity_check

a couple of quick early tests of the network sanity to try to help
us categorize the network quality from various zones at various
times of the day. As connectivity to pypi.openstack.org is one of
our biggest issues, use that as the test to see if this is useful.

Also test mirror.rackspace.com. Even though it's not always the
apt mirror we use, it does fail a bunch, and would be interesting
to know what it's base connectivity looks like when it does.

Change-Id: I71e4e732c8222e7cfe4775c32038eddd65f40b1b
double up [ for safety

The construct [ "$UNSET_VAR" -eq "1" ] will throw a bash error of
'integer expression expected' if UNSET_VAR doesn't exist. In bash
we can protect this by doubling up the brackets.

Change-Id: I261212c9d225b756a82214aa28d9e43e5d89fe55
Capture QEMU logs

This commit adds support for capturing any qemu logs. We were previously
capturing the libvirt logs, but for debugging it is useful to also have
the qemu logs.

Change-Id: I0d14074430a6a8ec6d99722646225f4ca3262080
Capture apache configuration files

Capture the apache vhost files in the output from ds-g. Since we
have more than just horizon's configuration, it is useful to know
how the Apache server is running / errors in the vhost when a
check or gate job finishes.

Change-Id: I1e5a7b5086566c0cbd18dee92c5062773ce8a8e5
Merge "add network_sanity_check"
Merge "Add keystonemiddleware to devstack-gate"
Merge "double up [ for safety"
Merge "Capture QEMU logs"
Make one copy of grenade log files

Change 709f2349 removed the blanked 'rm' of grenade log files, but
that leaves behind several confusing entries in the final log output;
e.g [1].

---
 grenade.sh.log.2014-06-16-014311
 grenade.sh.log.2014-06-16-014311.2014-06-16-014311.summary
 grenade.sh.log.summary
 grenade.sh.txt
---

grenade.sh.txt is greande.sh.log renamed, which was a symlink to the
time-stampped file, which is no longer removed.  The summary files are
also duplicated and not directly visible because they do not have a
.log extension.

This change just keeps the grenade log and summary files.  It renames
the summary so it can be viewed, and removes the unnecessary files.
It also puts grenade's localrc in the top-level directory with the
logs, renamed to grenade_localrc

[1] http://logs.openstack.org/83/98283/2/gate/gate-grenade-dsvm/8ba1473/logs/

Change-Id: I932ee1583f04756579abf63c792d336ae937f64d
Merge "Archive Ironic VM nodes console logs"
Merge "Make one copy of grenade log files"
Handle proposed/ branches in test-matrix.py

We are now cutting proposed/* branches on projects. Test matrix needs to
know about them. Test proposed/foo* as if it were a stable/foo branch
when we know what foo is. If we do not know about foo fallback onto
testing with a master setup as this branch is an intermediate non
integrated release (eg swift proposed/2.0.0) which is released while
other projects are still developing on master.

As part of this change update features.yaml to teach it about juno in
prep for the upcoming juno release.

Change-Id: I3f72d3171d0a1afc00298b1761c06b77bb282a6b
Merge "Capture apache configuration files"
Update toggle variable for HTTPD + mod_wsgi services

Update the devstack-gate toggle variable to match the new global
toggle for running services under alternatives to HTTPD + mod_wsgi.
This toggle only affects services that default to running under
HTTPD + mod_wsgi.

The new toggle was added in: https://review.openstack.org/#/c/101611/

Change-Id: I7ebaebf7806049fee2a6e57df628ad934b6d494c