Documents keystone v3 API usage - part 1

docs/source/using-api.rst documents how to use the V2 API. This patch
starts the equivalent documentation (using-api-v3.rst) about how to
use Keystone v3 API. Introduction and Authentication sections have
been written.

Change-Id: I5a2d45d80ba712492717a75ac5901f5ca775daa4
Partial-Bug: #1260527
Do not try to call decode() on a text string

This fixes a call to 'isinstance' that only worked on Python2.

Closes-Bug: #1265879
Change-Id: I886da95f1ad2a8314de92be8aecb354ee3124e6d
Prevent dictionary size from changing while iterating over its items

In Python 3, dict.items() returns 'a dict_item'. Iterating over it while
deleting some of the dictionary elements is forbidden. We have to iterate over
a list to avoid getting this error:

    RuntimeError: dictionary changed size during iteration

Change-Id: I43401e6eb9a31148fda4677644bf99e1b739d0dd
Debug env for tox

Running a test with pdb was difficult because testr captures
output and causes pdb prompt to quit.

Tips for how to run testr with debug are provided here:
  https://wiki.openstack.org/wiki/Testr#Debugging_.28pdb.29_Tests

This change puts these commands into a debug env in tox.ini so
you can do a command like

 tox -e debug test_auth_token_middleware

or, for more granularity:
 tox -e debug keystone.tests.test_sql_migrate_extensions.EndpointFilterExtension
 tox -e debug keystone.tests.test_sql_migrate_extensions.EndpointFilterExtension.test_downgrade

and when it hits your breakpoint you'll get the debug prompt.

This same change was made to keystone in commit c8302509 .

Change-Id: I2b5dc81f4652dc89ecbcc7c785245a81cf77a1ca
Rename using-api.rst to using-api-v2.rst

Renaming file for consistency, since using-api-v3.rst is being written
there: https://review.openstack.org/#/c/63408

Minor changes have been made to indicate that the file documents the
keystone v2 API.

Change-Id: I694b658a8b59d21615af5d88edc0f7b394ebbe7b
Partial-Bug: #1260527
Add unit tests for generic/shell.py

blueprint generic-shell-unittest
Change-Id: I234c58d6e0dfa6402633f5f4f2149e7b1b9d36f2
Fix auth_token middleware test invalid cross-device link issue

Fixes path of temporary files, containing token revocation lists,
which weren't in the appropriate directory. File renaming now works
properly since source and destination files are in the same directory,
and therefore on the same partition.

Added to .gitignore temporary files generated during tests, that are
not being removed at the end of test.

Change-Id: I21ff81419313f4205ebe552556e27b86135fd096
Closes-Bug: #1265190
Verify token binding in auth_token middleware

The server side has had token bind checking implemented for a while. Now
that it is possible to generate bound tokens auth_token middleware
should be capable of verifying them.

Change-Id: I4f9c5855ab3102333b0738864c506e2501bf9c7e
Sync strutils from oslo

Sync strutils from oslo which fix safe_encode under py3, this gets
keystone CLI to properly output results.

84d461 Fix a bug in safe_encode where it returns a bytes object in py3

Closes-Bug: 1260824
Change-Id: Idf0f3a51f5cfe3077395b53da66a12955449861f
Revert "Whitelist external netaddr requirement"

This reverts commit 01dc81aa148adaa127281f197047636cf2d2c898.

The netaddr package is now hosted on PyPI, so this workaround is
not needed anymore.

Change-Id: I76957aa5e276036921696f27e18efb5494b33120
Closes-Bug: #1266513
Controllable redirect handling

The requests library handles redirects as a browser would, which has the problem
that forwarding a POST will convert it to a GET. This is not necessarily
intuitive for an API. Handle all redirection manually and provide some
control over how far redirection can go.

Closes-Bug: 1267286
Change-Id: I24596a9c39cc29db28a66c3053ef3954b33abf90
Saner debug log message generation

For some reason building the debug log would include spaces in the
message elements and join on the empty string. It makes much more sense
to just build the list and join on the space.

Change-Id: Idd82787b87518c56122d0b13551f84529306337c
Merge "Prevent dictionary size from changing while iterating over its items"
Removes use of timeutils.set_time_override

The set_time_override function in timeutils was written as a
helper function to mock utcnow for unittests before 'mock' was
generally used. Now that we have mock and fixture, we no longer
need to use it.

Partial-Bug: #1266962

Change-Id: I56935911a08ccd9847419eb118af2d2c9dd6c1cd
Merge "Rename using-api.rst to using-api-v2.rst"
Use install_venv from oslo to fix no post_process issue

In a recent oslo sync commit Ieb73d5f799423fa8abf71634c86601ba6d32df2f
post_process() has been removed from tools/install_venv_common.py, which
is still called in tools/install_venv.py

Further investigation shows install_venv.py has some legacy distro
specific code, which should be removed at the time introducing
install_venv_common from oslo.

This change switch tools/install_venv.py to use oslo code to align the
effort as well as fixing original no 'post_process' issue.

Changes being pulled into in install_venv are:
12bcdb7 - 2013-11-25 12:16:11 +0800 - Remove vim header
b5cca99 - 2013-10-09 21:34:00 +0100 - Remove obsolete
redhat-eventlet.patch
9378d3c - 2013-08-13 18:06:11 +0300 - Move `./run_tests.sh` to Oslo

Note: openstack-common.conf is not updated, because the project name
variable in install_venv has to be modified after syncing from oslo.
That makes it not an exact sync but modification after sync. Besides
other openstack projects using install_venv from oslo also don't change
openstack-common.conf. So this patch simply follow the common practice
here.

Change-Id: I78a7563fbbde70cfd98cf1bd7a5a4eec5370d31d
Closes-Bug: #1263940
Merge "Do not try to call decode() on a text string"
Merge "Add unit tests for generic/shell.py"
Merge "Fix auth_token middleware test invalid cross-device link issue"
Tests cleanup temporary files

Some tests in test_auth_token_middleware weren't cleaning up
temporary files because the original filename was lost when
set_middleware() was called twice or tearDown() didn't clean up
the files.

The fix strategy is to use cleanUp(), because it will delete all
files even if called multiple times, and it doesn't rely on
tearDown() being called in subclasses.

Change-Id: Ib78150378ba6b622c9bc451ef4b331a87c17c56a