Add identity_feature_enabled.xml_api config

This change allows us to continue to test identity XML support for
stable icehouse. XML support in Keystone has been deprecated and staged
for removal in Kilo.

Change-Id: Ibb402640c5b744f1ea38ce548994079996c72508
Reserve Keystone ports from the ephemeral range

Reserve Keystone ports from the ephemeral range as early as reasonably
possible in the fixup_stuff.sh process to reduce the likelihood that the
port will be in use. This does not completely resolve the issue
where Keystone's IANA assigned port falls into Linux's ephemeral
range, but this should reduce the occurrences. The default ports
are 35357 and 35358.

Change-Id: I8cfb53d8f90c1ff1fb1083c59fefabca3d14323b
Partial-Bug: #1253482
(cherry picked from commit 6cae83efd72133adae891da0665f51f526705592)
Move to download.fedoraproject.org

download.fedoraproject.org is the mirror system, while
dl.fedoraproject.org is just one system. We have seen performance issues
with dl.fedoraproject.org and using download.fedoraproject.org was a
recommended as a solution by the folks at #fedora-admin.
Local testing on a rackspace server with dl.fedoraproject.org got about
70K/s while download.fedoraproject.org got over 1M/s

Note this is not a cherry-pick of I318d8844cfcfbc42c1e552e443c6b793d20f321f, because that wouldn't work

Change-Id: Ic68efcef0ed62f7867114d6992aee1610d6e8eb9
Related-Bug: #1383928
Merge "Add identity_feature_enabled.xml_api config" into stable/icehouse
Enable Swift's newer feature of container-sync

Newer version of container-sync feature is introduced in Swift ver. 1.12.0.

The spec:
http://docs.openstack.org/developer/swift/overview_container_sync.html

Before this commit, Devstack does not configure any realm used in
container-sync, therefore this feature does not work.
To test this feature in CI system, moreover to show the sample
configuration of realms, Devstack now edits realms configuration file.

Change-Id: I9f1e3224403e08e725a989162729470357fe90b0
Closes-Bug: 1378646
(cherry picked from commit d03915f9c08ff91c5d39f8cadb064718079bdb4b)
Merge "Reserve Keystone ports from the ephemeral range" into stable/icehouse
Merge "Enable Swift's newer feature of container-sync" into stable/icehouse
Set xml_api_v2 for Compute tests

Nova XML V2  api support is being removed in Kilo, so we need to
enable here to ensure that the API is still tested in this stable release.

Change-Id: I85ca8f5a7140a2637e64e980015ee30b2a23a46d
make run_tests.sh run devstack unit tests

run_tests.sh was created as an entry point for bashate. We stopped
using it now that we know how to run bashate directly in the gate.
We lost running an unrelated bashate test when that happened.

Devstack does have unit tests. We don't run them. We should.
Especially when working on things like the LIBS_FROM_GIT which is
sufficiently tricky that we really need to add unit tests to ensure
that we don't break it.

Conflicts:
	run_tests.sh

Change-Id: Ife067855569e2eae4c085471d326e8086de37332
(cherry picked from commit 9f20ea13826234ac2c508ea1d50a0822b68d42dc)
use released library versions by default

This patch provides a new path for installing libraries in devstack so
that it's possible to either test with upstream released libraries, or
with git versions of individual libraries.

Libraries are added by name to 3 associative arrays GITREPO,
GITBRANCH, GITDIR. When we get to the library install phase we inspect
LIBS_FROM_GIT and look for libraries by name (i.e. "oslo.config") and
if they exist we'll clone and install those libraries from
git. Otherwise we won't, and just let pip pull them as dependencies
when it needs them.

This patch provides the conversion of the oslo libraries, including
pbr.

Devstack-gate jobs for these libraries will need to change to support
actually forward testing their content.

restructure stackrc into groupings

in order to support installing from stable libraries we first need to
actually sort out all the categories our giant list of git repos fit
into. This will make it much easier to not lose one in the process.

support installing clients and supporting libraries at released versions

expand the devstack support for libraries from released versions to
support python-* clients and supporting libraries. This is a collapse
of the master patch series that took a while to get correct.

(This is a collapsed patchset for backport to make backport fixups easier)

add unit tests for GIT* definitions

This adds unit tests for all the GIT* definitions, ensuring that for
libraries we think should be defined, they are. It exposed a bug in
glance_store definitions in the process.

The GITDIR definition for python-openstackclient is moved to stackrc
for testability.

Conflicts:
	lib/glance

Change-Id: I26fac0ccf8fd4818e24618d56bf04b32306f88f6
Source functions from stackrc

stackrc now requires GITREPO, GITBRANCH, GITDIR and has been
dependent on functions for a while (is_package_installed).  Ensure
we source the required functions file when stackrc is loaded.  Avoids
unexpected issues in grenade where they may or may not have been loaded
depending on the configuration.

Closes-bug: #1377274

Change-Id: I5027cfad07af0de7ff39f424601d6f7ec5dcadae
(cherry picked from commit 0e4cd038287bcf36ff31c4e7b22266051198b44c)
Actually enable new feature of container-sync

I modified devstack to run newer version of container-sync in
commit d03915f, and cherry-picked to stable/icehouse.
However, $SWIFT_SERVICE_PROTOCOL variable is introduced after
stable/icehouse (commit 18d4778), therefore my cherry-pick patch did
not enable container-sync feature.

This patch replaces this variable with string value "http", so now
icehouse swift deployed by devstack works appropriately.

Change-Id: I3b509312846d29a5d14607d5eb561eb9545b28a5
Closes-Bug: 1378646
Fix lib/oslo to only reference things which exist

The backport of the install from libs infrastructure left some
unwanted bits in lib/oslo that cause cryptic build issues. Remove
that.

Change-Id: If56f64f1305a206e5360ad9b5e3e41db4d60d576
Merge "Actually enable new feature of container-sync" into stable/icehouse
Hard code icehouse service extension lists

This commit adds the hard coded service extension lists for icehouse
in the tempest config files. To enable gating with tempest master on
all supported branches the extension list has to be explicit on stable
branches to ensure we don't attempt to run tests for new extensions on
stable branch services. This commit adds these lists for the icehouse
branch.

Implements blueprint branchless-tempest-extensions

Change-Id: I14b4301f9211529e5ff6fe56f65392c1d198a18a
Install setuptools<8.0

New setuptools (>=8.0) breaks with versions used in older versions of
things. Specifically .gshortsha1 source identifiers do not parse and are
sorted as coming before all other versions. Using <8.0 at least unwedges
us then we can work on backporting changes to use newer pbr which will
use +gshortsha1 which sorts properly.

Change-Id: Iae98d9d7e97b81955cafe1b88022133e6902e551