Configurable python-keystoneclient repo

The URL for the python-keystoneclient repo couldn't be configured.
With this change, the user can set the OPENSTACK_REPO environment
variable to set the URL for where the python-keystoneclient repo
should be cloned from. So for example you can use a local clone.
The default OPENSTACK_REPO is https://git.openstack.org/openstack
which gets python-keystoneclient.git added ot it to make the clone
come from
https://git.openstack.org/openstack/python-keystoneclient.git.

Change-Id: I936713af3c11e8d68d8739dc68eefabd645df294
Remove fixture from openstack-common.conf

Keystone doesn't use anything in fixture, so remove it from
openstack-common.conf and re-sync with oslo-incubator 5fa2da to
remove unused parts.

Change-Id: Icec01fc7e3bcf3df03c1f8a589d13f8955c64380
Issue multiple SQL statements in separate engine.execute() calls

Some sqlalchemy drivers (eg: mysqlconnector) don't support
engine.execute("sql stmt 1; sql stmt 2;") and require a separate
execute() call for each SQL statement.  After discussions with
sqlalchemy author, he confirmed it would be better to fix callers rather
than attempt to patch in support for multiple statements.

With this fix, keystone-manage db_sync succeeds using mysqlconnector.

Change-Id: I658a4dd479dbdcd07345c97d810c85aada3fa35b
Set default token provider to UUID

This changes the default token provider to UUID, which affords a much
better deployer experience (no external dependencies and no additional
setup complexity) for deployers. It also provides a better end-user
experience (smaller, more manageable tokens) and appears to be the more
popular deployment option today, despite the current default to PKI.

DocImpact
Closes-Bug: 1350000

Change-Id: I7fb2b191cce7a9762c33fee09e7e8d48a71a297b
Trust unit tests should target additional threat scenarios

This adds unit tests for two threat scenarios around the trust functionality
that are not currently tested.

The first scenario is related to deletion of a grant that has been previously
delegated via a trust. We need to ensure that executing a trust for a role that
the trustor no longer has is rejected.

The second scenario is related to an attempt to use a trust token with
impersonation to execute another trust as the impersonated user. We need to
ensure that a trust token can't be used to execute another trust.

SecurityImpact
Closes-Bug: #1347909
Change-Id: Ie1a0c286ff7e513cd964d4a93855230c78b98c6c
Filter List Regions by 'parent_region_id'

List Regions (GET /regions) has an optional query parameter
'parent_region_id', but the API still returns all the regions.

This fix adds support of filtering regions by an optional query
parameter 'parent_region_id'.

Closes-Bug: #1350192

Change-Id: I59f8d852870bd3ad26622bfcd00dfadfdf9d5b42
Class for V3 router packages

The V3 routes were being added by the append_v3_routers function in
each controller package, so the only way state could be stored is in
a global variable which makes unit testing difficult. With this
change, the append_v3_routers functions are put into a class in each
package. This will eventually be used to store JSON Home data.

bp json-home

Change-Id: I744a4c82dc84bb1a8d29d0314e3c51cc43c077a2
Enhance V3 router class for resources

RESTful APIs are resource-oriented, and Keystone provides a
RESTful API, but the current way that routes are being mapped is
based on URLs rather than resources. With this change, the base
Routers class is enhanced to support a resource-oriented way to
map paths to controllers and methods.

bp json-home

Change-Id: I0ba856845116b3ddd3dfed4608be65a87d9c01dc
Change V3 router classes to use resources

The V3 router classes are changed to use the resource-oriented
method to map paths.

bp json-home

Change-Id: I81957c5b2fc41d6f46e4913afa1073c58a040c45
V3 Extension class

There was no base class specific to V3 extensions. This adds a base
class for V3 extensions and also changes the current V3 extensions
to use it.

bp json-home

Change-Id: I9a65867ced1bc116e3395baf4abfa35796511788
Enhance V3 extension class to use resources

RESTful APIs are resource-oriented, and the V3 extensions each
provide a RESTful API, but the current way that extensions map their
routes is based on URLs rather than resources. With this change, the
base V3 extensions class is enhanced to support a resource-oriented
way to map paths to controllers and methods.

bp json-home

Change-Id: Ic82fc9d03b0073a8a1fcec5d03529c84e22f4d69
Change V3 extensions to use resources

The V3 extension classes are changed to use the resource-oriented
method to map paths.

bp json-home

Change-Id: I4286f0affee7570b509c215f3375b2a517a5af4f
Allow LDAP lock attributes to be used as enable attributes

Some LDAP servers support disabling accounts via a boolean "lock"
attribute. For these servers, a value in LDAP of "True" means that
the account is locked, while a value of "False" means the account
is active. When the "user_enabled_mask" and "user_enabled_emulation"
options are not in use, Keystone currently expects a boolean
"enabled" attribute where "True" means the account is enabled and
"False" means the account is disabled.

To support LDAP account lock attributes, we need a way to tell
Keystone that the boolean values from LDAP are inverted. This
adds a new "user_enabled_invert" setting that allows the enabled
boolean logic to be inverted in the resource (LDAP), while leaving
the logic as-is in the model (Keystone user object).  The existing
default behavior remains as-is.

DocImpact
Change-Id: I2a89d4b98c854e68e1bb10f53b8b29d92f945f60
Closes-bug: #1337029
Add oslo.utils requirement

This will allow Keystone to use functions out of oslo.utils.

Change-Id: I0467ed68f894e7075bc25ff3e779351fab9ed4c3
Remove S3 middleware tests from tox.ini

The middleware was moved to the keystonemiddleware project and no longer
needs to be tested here.

Change-Id: Ic4ac0d2558ced56eeadd6241159188c49fcceb6e
Make token_provider_api contain token persistence

The token_provider_api now houses the token persistence manager
directly. A new token_api provider has been created that just
proxies to token_provider_api.persistence.

This is in support of deprecating use of token_api internal to
Keystone and migrating direct uses of token_api to gate through
the token_provider_api methods.

Uses of the token_api within the provider and provider drivers
has been lifted up to the token_provider_api manager and now
utilize the persistence manager directly.

Change-Id: I62cc1d5efbe0869fd0e501aaa9405ff0b0da0e5e
bp: non-persistent-tokens
Fix documentation link

Without the :py: prefix the mod part of the string ends up in the
documentation.

Change-Id: I652eb6c545deede33622b64fb4810c319d480f7a
Delete intersphinx mappings

The documenatation does not use intersphinx mappings so remove these so
as not to download them.

Related-Bug: #1353817
Change-Id: I0b0e48fa04d08b1023984b6d2169322060fcee9e
Refactor serializer import to XmlBodyMiddleware

Previously, the keystone/middleware/core.py module would import
keystone/common/serializer.py regardless of the XmlBodyMiddleware class
being used. This change modifies keystone/common/serializer.py to raise
an error if there is an issue with using lxml at runtime.

Change-Id: I977275d0df8666b49ddbfadd4f5606d7c2292b1f
Closes-Bug: 1351016
Remove assignment controller dependency on token_api

The assignment controller now uses the `validate_token` method on
the `token_provider_api` instead of referencing `get_token` directly
from the token persistence service.

The user_crud contrib (v2.0) extension now will emit an actual
token to the wire instead of the raw-internal data structure from
the token persistence backend. The intent here was clear by the
fact that the structure was built to look like a v2.0 token,
however, it was retriving the data from the persistence backend
and just updating the token_id then resaving the data before
emitting the raw persistence data (incorrect). The extension
now uses the issue_v2_token method from the token_provider_api.

This change is to ensure interactions with the token persitence
are consistent so that it is possible to support toggling
persistence of tokens.

Change-Id: I22b6b837731566bfd4dc61a47eec2eb0d336d4d1
bp: non-persistent-tokens