Fix up tempest conf settings

The tempest api.volume.test_volume_types test won't
work with non-default drivers configured for cinder's backend
any more.  The reason is that we create a type using capability
scheduler keywords in the extra-specs for the test;
(vendor_name and storage_protocol).  The result is the extra-spec
uses the filters: "vendor_name=Open Source" and
"storage_protocol=iSCSI", but for example if you have another backend
say SolidFire, EMC, NetApp, IBM etc the capabilities filter will fail
the create with a "No valid host available".

This is intended to work by simply setting these values in your
tempest.conf file.  That's fine, however upon setting this up
in my localrc I found that the tempest config variables being
set via devtsack were never picked up

Currently devstack doesn't use the same variable names for
configuration variables as tempest expects. Devstack is using
the variable "TEMPEST_CONF" however the Tempest project is
expecting the variable "TEMPEST_CONFIG", so currently the
devstack lib/tempest rc variables are never picked up by
tempest properly.

This change modifes devstack's naming of TEMPEST_CONF, my though
being that since this doesn't work in devstack currently
that changing it here would be better than changing it in Tempest
where it's possible people had their own custoizations already
outside of devstack.

In addition this change creates rc variables in devstack to actually
set these via devstack.  The idea here is that Cinder 3'rd party testing
needs to be a simple devstack config and run stack.sh.  By fixing up
the configuration file variable naming and adding the variables for
the vendor and protocol settings that's now possible.

An example localrc for a custom config is shown below.  The example
sets the tempest config file to /etc/tempest/tempest.conf, and
configures tempest to use the SolidFire driver as the cinder backend.

TEMPEST_VOLUME_VENDOR ==> tempest.conf.volume_vendor
TEMPEST_STORAGE_PROTOCOL ==> tempest.conf.storage_protocol

relevant example localrc entries:
  TEMPEST_CONFIG=/etc/tempest/tempest.conf
  TEMPEST_CONFIG_DIR=/etc/tempest
  TEMPEST_VOLUME_DRIVER=solidfire
  TEMPEST_VOLUME_VENDOR="SolidFire Inc"

***NOTE***
storage_protocol and vendor_name MUST match what the backend device reports from
get capabilities.

Change-Id: I28dfa90c877b27f5d4919f2748fae092bb2f87fa
Closes-Bug: 1271781
Use service postgresql initdb with el6

postgresql-setup does not exists on el6,
the service postgresql initdb is the documented db init command.

Change-Id: I2b92a3c8e7db603eb13378e46893fc81f507405b
Include SAVANNA_CONF_DIR into SAVANNA_CONF_FILE

It's the commom way of using X_CONF_FILE variable.

Change-Id: Ibc284be44ffdd25be3191913c78424cbf06b2bb0
Stop all neutron-ns-metadata-proxy with stop_neutron

Process name is actually python therefore neutron-ns-metadata-proxy
pattern didn't match wanted process.

Closes-bug: #1269982
Change-Id: Ib4439b0d32f103253b461841fa903c65763ff280
Copy container-sync-realms.conf in /etc/swift

We need the new container-sync realms configuration or we will get a
nasty harmless error opening file at swift proxy startup.

Change-Id: If939da305dcb9403c418219032ac6b50b0099bd3
Closes-Bug: 1274295
Generate Tempest service list rather than hard-code it

The list of services that Tempest used to set its 'service_available'
config values was hard-coded. To be plugin-friendly have each
service (project) add its name to the TEMPEST_SERVICES variable
and use that for setting the 'service_avilable' values.

Change-Id: I208efd7fd0798b18ac2e6353ee70b773e84a2683
Set keystone admin_bind_host to KEYSTONE_SERVICE_HOST

On Linux ports 32768-61000 can be used by just about
anything needing a socket. Keystone's IANA assigned port is 35357.
Occasionally something else will be using port 35357 first because Linux
allows this. Workaround is to bind to port 127.0.0.1 instead of 0.0.0.0.
$KEYSTONE_SERVICE_HOST gets its value from $SERVICE_HOST which is set to
127.0.0.1 in the gate.

"Ephemeral (client) ports will *never* be sourced from 0.0.0.0, and are
uniquely identified by the full connection five-tuple (proto, src IP,
src port, dst IP, dst port) anyway, allowing them to overlap src IP/src
port as long as proto/dst IP/dst port are different. Thus it is up to
keystone/devstack to bind more appropriately and not use wildcard bind
addresses unless explicitly necessary for some reason. For example, in
the log output, the URLs are configured with dst IPs of 127.0.0.1
anyway, so binding explicitly to localhost would change nothing, while
skirting this particular edge case nicely." ~Evan Callicoat

This doesn't fix bug 1253482 it works around it while a better solution
is prepared (running keystone behind apache in devstack).

Co-Authored-By: Joe Gordon <joe.gordon0@gmail.com>
Change-Id: I112309661dadf8b753c3311182f82464d9d3595e
Related-bug: #1253482
Merge "Include SAVANNA_CONF_DIR into SAVANNA_CONF_FILE"
Merge "Stop all neutron-ns-metadata-proxy with stop_neutron"
Merge "Generate Tempest service list rather than hard-code it"
Retry rabbitmq password change

Due to the bug referenced below, on Fedora it is possible for
the rabbitmq password change to fail the first time rabbitmq is
started.  This change adds a retry loop to avoid the problem in
devstack.  One retry should be enough in most (all?) cases, but
this will retry up to ten times just to be safe.

Note that just retrying the password change is not enough.  The
rabbitmq-server service must be restarted as well.

Change-Id: I403dcd503aa8e74e2ba6312a0decf0d4fd0d8795
bz: https://bugzilla.redhat.com/show_bug.cgi?id=1059028
Add support for Gantt

Gantt is the new breakout of the scheduler code from the Nova
source tree.  These changes allow devstack to install/configure/startup
gantt as the scheduler service for openstack.

Change-Id: Ia2b6001f5ccf2469ee9fdee67564c9a915a13862
Merge "Set keystone admin_bind_host to KEYSTONE_SERVICE_HOST"
Pull docker images from global registry

The global docker registry is where images are
being built and uploaded. It's effectively docker's
version of, say, 'pip'. The static tarballs are
not only an extra maintenance burden as they're outside
the standard build and publishing process, but are presently
outside the scope of an open development / release
process as well.

While this process does cause some trouble with
network-independence for CI purposes, the fetching
is still done from install-docker.sh; Additionally,
this driver is not currently tested via the community CI effort.

Change-Id: I3ee6bfee9c273cd3aabe1e00a1d1a8856a466189
glance: stop using deprecated notifier_strategy

Change-Id: Ic796f0ad57db45bf053312ad10815461528030b3
fix sar reporting in the gate

the sar filter made an assumption of time display including an
AM/PM... which isn't true in all environments. Hence the blank
sysstat screen in the gate runs of late.

This fixes that, and displays the first line which includes header
version to make sure we are functioning.

Change-Id: I537e0bf2127efaf337c4792bc23d938145c8990d
Install libguestfs for nova-compute on Ubuntu

We were already installing this for n-cpu on rpm distros, but not
Ubuntu.  Install it so that nova-compute can use it for file injection,
which is the preferred method over nbd.

Set CONF.libvirt.inject_partition to -1.  This enables using libguestfs to
determine the proper partition to inject into.

Don't bother trying to load the nbd kernel module anymore.  It won't be used
since we know always expect libguestfs to be installed.

Change-Id: Ifa9d95bf759f1dad8685590a2df242d852dd2cb0
Merge "Copy container-sync-realms.conf in /etc/swift"
Merge "fix sar reporting in the gate"
Merge "Add support for Gantt"