Ensure upgrade testing tests same deployment methods

During Icehouse Keystone was only ever deployed (gate/check) under
eventlet. With the ability to deploy Keystone under mod_wsgi in
gate/check it is important to handle the upgrade appropriately
within grenade. During the upgrade, use the new global method
to disable services being deployed under HTTPD + mod_wsgi.

This change uses the new flag(s) provided in
https://review.openstack.org/#/c/101611/

Change-Id: I0179f16a9074678166bcb6ea1a883de5750e520e
Leave the VG alone

The configured volume group does not change during an upgrade.

This allows the generalized Cinder backend support to simply
utilize the existing LVM back-end in place rather than re-create
it after an upgrade.

Change-Id: I92d2b0c2dc6ea05dda3c8cf27e686a8eb98f326c
check-sanity skip checking q-fwaas and q-l3

FWaaS is running as a service plugin in neutron-server and doesn't have
any dedicated process. When Neutron is deployed with VPNaaS it's running
instead of l3-agent with the same capabilites as L3 agent has. Thus
checking whether processes for these services are running is not
necessary.

Change-Id: I8f438cfccc84be801d4064bdb564a95c6621bf55
make grenade logdir creation work on clean system

On a completely fresh system, /opt/stack is something which doesn't
yet exist, and the directory creation for it will fail on the first
try. By reordering the directory creation it will be easier for
people to get this working with completely fresh systems.

Change-Id: I5183ce091d0c3d417e700aa2c8b932507e51c16a
Only upgrade services we are actually running

Only upgrade the services that are actually active in grenade,
making it runnable outside of the gate environment where ceilometer
isn't enabled.

Change-Id: I3aeff82ceb99fbfa3689141f0dae974189c7370a
first pass of integrating javelin2 to grenade

This provides support for creating users, images, servers, and
objects and checking on the far side.

Depends-On: If5f2674179eea4972ad43a1581073d483bb75d79 from
tempest to fix some javelin issues to make it usable.

Change-Id: I3634fd1c1db08a9a03e2dce56ebe5424fe2f27d5
move from screen dumping to using worlddump tool

This makes it easier to collect this information for archival
purposes.

Change-Id: I8db9dc99da8bb486a07d7ead77031e1c0b419857
Remove deprecated glance commands from javelin setup

There are some deprecated commands being used in the javelin setup
they should be removed since we are removing all the deprecated
glance client commands in this patch
http://review.openstack.org/#/c/108804/

Change-Id: Ieb05c7ba2959c26c987667dba751e94b899772af
Write javelin2 logs to stderr

We currently don't keep the oslo-logging based javelin2 logs in gate
runs, making it very hard to debug. Fix that by logging to stderr.

There is a bug in oslo that is causing stdout to not work, so use
stderr.

Specify config file via CLI (added in
I1e82e0a8e72117ba55048b2dab7aa9a8ba00c1cb)

Change-Id: Idbdac27392121e348ae0ab93477c98a904f8c001
Co-Authored-By: Sean Dague <sean@dague.net>
Consult target when deciding to upgrade service

The target side's enabled services should be queried when determining
whether or not to upgrade a service.  This allows sideways upgrades
that involve adding or removing services during the upgrade/migration.

Change-Id: I94569b35342d20fd5ef4d92d3f97cee6cfc42291
Merge "Remove deprecated glance commands from javelin setup"
Merge "move from screen dumping to using worlddump tool"
Merge "Consult target when deciding to upgrade service"
Log javelin into in its own  logfile.

Use a single log file for both javelin create and check

Related devstack-gate change: I787c08506d9a4d12081b5b5b16d752d5147f8e72

Change-Id: Id5c0dfeaf8e7e9ed1d54f51c54e6e5c65b1361fa
Merge "Write javelin2 logs to stderr"
Rename bash8 to bashate

bash8 has been renamed to bashate.

Change-Id: Id4781f8174ae34cf4cc69da607806583f0056242
Save base and target release state correctly

If there are both a neutron and neutron_ml2 database, saving state
would fail in both save-state and grenade.sh.

neutron databases were being saved to BASE not TARGET_RELEASE
in grenade.sh

ceilometer databases were not being saved in grenade.sh

Clean up some whitespace.

Change-Id: I638c6923af65ad612d1329c8c25f534bfdad2d02
Closes-bug: 1358332
Merge "Log javelin into in its own  logfile."
Source all devstack service libs for Tempest

Devstack's tempest library relies on TEMPEST_SERVICES when
configuring the service_available section of tempest.conf. This
is built as various service libraries are sourced into the environment.
This imports all relevant devstack service libraries to ensure the
services_available section of tempest.conf is accurate.

Change-Id: Iea9c08f5ce374c4343ae56adb49221737c7a55d7
Add within-juno dir and within-master symlink

This adds logic to upgrade_project() to call project upgrade
scripts from the within-$release directory if performing a
sideways upgrade within the same release.  A symlink named
within-master is added and linked to the current development
release to allow sideways migration testing of trunk.  When Juno
tags are cut and K opens, a from-kilo directoy should be created
and this symlink updated.

Change-Id: I770075442d05f38515e5dadf24f2c642d14d7845