make grenade logdir creation work on clean system

On a completely fresh system, /opt/stack is something which doesn't
yet exist, and the directory creation for it will fail on the first
try. By reordering the directory creation it will be easier for
people to get this working with completely fresh systems.

Change-Id: I5183ce091d0c3d417e700aa2c8b932507e51c16a
Only upgrade services we are actually running

Only upgrade the services that are actually active in grenade,
making it runnable outside of the gate environment where ceilometer
isn't enabled.

Change-Id: I3aeff82ceb99fbfa3689141f0dae974189c7370a
first pass of integrating javelin2 to grenade

This provides support for creating users, images, servers, and
objects and checking on the far side.

Depends-On: If5f2674179eea4972ad43a1581073d483bb75d79 from
tempest to fix some javelin issues to make it usable.

Change-Id: I3634fd1c1db08a9a03e2dce56ebe5424fe2f27d5
move from screen dumping to using worlddump tool

This makes it easier to collect this information for archival
purposes.

Change-Id: I8db9dc99da8bb486a07d7ead77031e1c0b419857
Remove deprecated glance commands from javelin setup

There are some deprecated commands being used in the javelin setup
they should be removed since we are removing all the deprecated
glance client commands in this patch
http://review.openstack.org/#/c/108804/

Change-Id: Ieb05c7ba2959c26c987667dba751e94b899772af
Write javelin2 logs to stderr

We currently don't keep the oslo-logging based javelin2 logs in gate
runs, making it very hard to debug. Fix that by logging to stderr.

There is a bug in oslo that is causing stdout to not work, so use
stderr.

Specify config file via CLI (added in
I1e82e0a8e72117ba55048b2dab7aa9a8ba00c1cb)

Change-Id: Idbdac27392121e348ae0ab93477c98a904f8c001
Co-Authored-By: Sean Dague <sean@dague.net>
Consult target when deciding to upgrade service

The target side's enabled services should be queried when determining
whether or not to upgrade a service.  This allows sideways upgrades
that involve adding or removing services during the upgrade/migration.

Change-Id: I94569b35342d20fd5ef4d92d3f97cee6cfc42291
Merge "Remove deprecated glance commands from javelin setup"
Merge "move from screen dumping to using worlddump tool"
Merge "Consult target when deciding to upgrade service"
Log javelin into in its own  logfile.

Use a single log file for both javelin create and check

Related devstack-gate change: I787c08506d9a4d12081b5b5b16d752d5147f8e72

Change-Id: Id5c0dfeaf8e7e9ed1d54f51c54e6e5c65b1361fa
Merge "Write javelin2 logs to stderr"
Rename bash8 to bashate

bash8 has been renamed to bashate.

Change-Id: Id4781f8174ae34cf4cc69da607806583f0056242
Save base and target release state correctly

If there are both a neutron and neutron_ml2 database, saving state
would fail in both save-state and grenade.sh.

neutron databases were being saved to BASE not TARGET_RELEASE
in grenade.sh

ceilometer databases were not being saved in grenade.sh

Clean up some whitespace.

Change-Id: I638c6923af65ad612d1329c8c25f534bfdad2d02
Closes-bug: 1358332
Merge "Log javelin into in its own  logfile."
Source all devstack service libs for Tempest

Devstack's tempest library relies on TEMPEST_SERVICES when
configuring the service_available section of tempest.conf. This
is built as various service libraries are sourced into the environment.
This imports all relevant devstack service libraries to ensure the
services_available section of tempest.conf is accurate.

Change-Id: Iea9c08f5ce374c4343ae56adb49221737c7a55d7
Add within-juno dir and within-master symlink

This adds logic to upgrade_project() to call project upgrade
scripts from the within-$release directory if performing a
sideways upgrade within the same release.  A symlink named
within-master is added and linked to the current development
release to allow sideways migration testing of trunk.  When Juno
tags are cut and K opens, a from-kilo directoy should be created
and this symlink updated.

Change-Id: I770075442d05f38515e5dadf24f2c642d14d7845
Add network resources to javelin

There are efforts in tempest to implement support for network and
security group resources - https://review.openstack.org/#/c/100169/

Change-Id: I18174cf2850b9254d3fa988d42aebbcd704150b8
Fix check-sanity + save-state /w disabled services

This adds a mapping for all process names described in BASE_SERVICES
and TARGET_SERVICES, allowing check-sanity to skip checking for running
processes of services that are disabled according to ENABLED_SERVICES.
It also updates grenade.sh and save-state to only dump databases that
exist.  These changes allow grenade to pass against deployments that
have disabled non-essential services in devstack.

Change-Id: Ic4b2bab88b12f33f939b9bf7aca47e930d799f3f
Closes-bug: #1355499
Merge "Save base and target release state correctly"