Prevent early auth deny in tempauth when using swift3 middleware.

When tempauth is used together with swift3 and keystone, groups are
empty and tempauth denies the request too early without a chance for
keystone to authenticate the request.

Change-Id: I21d9b22ecbd18a5f1fba901abd94221a332c45ea
Closes-Bug: 1244545
GreenDBConnection no longer assumes that arguments are kwargs

Previous this code assumed that almost all arguments would be passed
as kwargs. Now the code also handles ``timeout`` occurring in
``*args``. This is needed because on PyPy's sqlite module, they are
passed in ``*args``.

Change-Id: I7ce25e7b262c6f46680e3f0eb391094de8051946
Fixed DB test cases on PyPy

They weren't working because of the intersection between some monkey
patching in the tests and the fact that ``sqlite3.Connection`` and
``Cursor`` are Python classes on PyPy, but C-extension classes on
CPython. The result was that trying to call the superclass's init
didn't work because the name ``sqlite3.Connection`` pointed to a
different class that wasn't actually the real superclass of
``GreenDBConnection``. Using ``super`` fixes this because the method
called is now always the real superclass's.

Change-Id: Ib45b16c7499883e560877788f13285d0a47cc2cd
Remove swift-bench

swift-bench has moved to :

http://github.com/openstack/swift-bench

DocImpact

Change-Id: Id09765f53d1e493fb0eeae0dba57879ba9dd0ade
Update handoff algorithm to use IP/port pairs

The replica placement algorithm works on regions, then zones, then
IP/port, then device ID. The handoff algorithm worked on regions, then
zones, then device ID, completely skipping IP/port. It's now been
updated to take IP/port into consideration.

This means you get one handoff on each machine in the cluster before
you start getting handoffs that share a machine with a previous
one. In small clusters, this can help with durability.

Because this is performance-critical code, here are some quick
benchmark results:

Run time averages over 25000 trials on a 1200-device ring (20 part
power, 3 replicas, 2 regions, 10 zones, 120 nodes):

		   |   master    |   branch
===================+=============+============
get 1 more node	   |  2.727e-05	 |  3.076e-05
get 6 more nodes   |  3.55e-05   |  4.214e-05
get all more nodes |  0.002247   |  0.002691

There's a small slowdown from the additional bookkeeping, but nothing
too awful. The time to get 6 more nodes (for handoff checks on 404,
it's 2x replica count by default, hence 6) went from 35 to 42
microseconds, so it remains small.

Change-Id: Ie7da4dfcb0fcf1a38e2fb13f60c204540fadbf06
Add option to return disk usage report with byte suffix.

Disk usage is currently returned as bytes. This patch adds the option
to return the value with a more human-friendly suffix, eg. MB/GB/TB/PB.

Change-Id: I3d55181f7e6085e711bd6e09ea2ce0a12b290cc5
Merge "GreenDBConnection no longer assumes that arguments are kwargs"
Merge "Add option to return disk usage report with byte suffix."
Preserve closeability of app iterables

PEP 333 (WSGI) says that if your iterable has a close() method, the
framework must call it.

WSGIContext._app_call pulls the first chunk off the returned iterable
to make sure that it gets status and headers, and then it would
itertools.chain() that first chunk back onto the iterable so the whole
body went out. swob.Response.call_application() does it too.

The problem is that an itertools.chain object doesn't have a close()
method, so your iterable's fancy-pants close() method has no chance of
getting called.

This patch adds a slightly smarter CloseableChain that works like
itertools.chain, but has a close() method that calls the underlying
iterables' close() methods, if any.

Change-Id: If975c93f53c27dfa0c2f52f4bbf599af25202f70
Remove ssync iter_hook usage

Apparently that disappeared and now the whole thing does on its own what this
was trying to achieve.

Change-Id: If20016cb80c855e3e8373de5d56926ef8f8e7dac
Merge "Fixed DB test cases on PyPy"
use diskfile in ssync_sender tests

Change-Id: I7993de98ce3eb4839fa5d72d1b6ce08e4a7c1451
Opt out of the service catalog

Set include_servce_catalog=False in Keystone's auth_token
example configuration. Swift does not use X-Service-Catalog
so there is no need to suffer its overhead. In addition,
service catalogs can be larger than max_header_size so this
change avoids a failure mode.

DocImpact
Relates to bug 1228317

Change-Id: If94531ee070e4a47cbd9b848d28e2313730bd3c0
Merge "Remove ssync iter_hook usage"
fix expired object deletion

fixes bug #1257330

Change-Id: I49f645abdeba97eafb3ae42ef9e3684c912cfdc6
Merge "use diskfile in ssync_sender tests"
Allow access to the in-memory object server

This will allow functional tests to be used against it, and can then
be used for the storage-policy work as an example diskfile
implementation associated with a storage policy.

Change-Id: I47a88e70cee99225779baaed379b0c5d4c73611a
Fixed a failing test on my saio

The added sleep makes this test pass on my saio. I have not heard of
it failing for anyone else, but I figured I'd post this up just in
case someone does have the same problem and this fixes it for them.

Change-Id: Ia0bb09d36d0b531ade7c6a6034bbe31dd6c90a98
Merge "fix expired object deletion"
Override spelling of tempauth test name

Change-Id: Ife571bbeb6dd4ba5c552a2f0725c36e133e64cf4