Prevent early auth deny in tempauth when using swift3 middleware.

When tempauth is used together with swift3 and keystone, groups are
empty and tempauth denies the request too early without a chance for
keystone to authenticate the request.

Change-Id: I21d9b22ecbd18a5f1fba901abd94221a332c45ea
Closes-Bug: 1244545
Return an exit code for configuration errors

Red Hat's QA noticed that in case of the infamous "xattr>=0.4"
error, swift-init exits with a zero error code, which confuses
startup scripts (not Systemd though -- that one knows exactly
if processes run or not).

The easiest fix is to return the error code like Guido's blog
post suggested.

Change-Id: I7badd8742375a7cb2aa5606277316477b8083f8d
Fixes: rhbz#1020480
Remove swift-bench

swift-bench has moved to :

http://github.com/openstack/swift-bench

DocImpact

Change-Id: Id09765f53d1e493fb0eeae0dba57879ba9dd0ade
Update handoff algorithm to use IP/port pairs

The replica placement algorithm works on regions, then zones, then
IP/port, then device ID. The handoff algorithm worked on regions, then
zones, then device ID, completely skipping IP/port. It's now been
updated to take IP/port into consideration.

This means you get one handoff on each machine in the cluster before
you start getting handoffs that share a machine with a previous
one. In small clusters, this can help with durability.

Because this is performance-critical code, here are some quick
benchmark results:

Run time averages over 25000 trials on a 1200-device ring (20 part
power, 3 replicas, 2 regions, 10 zones, 120 nodes):

		   |   master    |   branch
===================+=============+============
get 1 more node	   |  2.727e-05	 |  3.076e-05
get 6 more nodes   |  3.55e-05   |  4.214e-05
get all more nodes |  0.002247   |  0.002691

There's a small slowdown from the additional bookkeeping, but nothing
too awful. The time to get 6 more nodes (for handoff checks on 404,
it's 2x replica count by default, hence 6) went from 35 to 42
microseconds, so it remains small.

Change-Id: Ie7da4dfcb0fcf1a38e2fb13f60c204540fadbf06
Increases the UT coverage of db_replicator.py

Adds 20 unit tests to increase the coverage of db_replicator.py
from 71% to 90%

Change-Id: Ia63cb8f2049fb3182bbf7af695087bfe15cede54
Closes-Bug: #948179
Preserve closeability of app iterables

PEP 333 (WSGI) says that if your iterable has a close() method, the
framework must call it.

WSGIContext._app_call pulls the first chunk off the returned iterable
to make sure that it gets status and headers, and then it would
itertools.chain() that first chunk back onto the iterable so the whole
body went out. swob.Response.call_application() does it too.

The problem is that an itertools.chain object doesn't have a close()
method, so your iterable's fancy-pants close() method has no chance of
getting called.

This patch adds a slightly smarter CloseableChain that works like
itertools.chain, but has a close() method that calls the underlying
iterables' close() methods, if any.

Change-Id: If975c93f53c27dfa0c2f52f4bbf599af25202f70
Opt out of the service catalog

Set include_servce_catalog=False in Keystone's auth_token
example configuration. Swift does not use X-Service-Catalog
so there is no need to suffer its overhead. In addition,
service catalogs can be larger than max_header_size so this
change avoids a failure mode.

DocImpact
Relates to bug 1228317

Change-Id: If94531ee070e4a47cbd9b848d28e2313730bd3c0
Allow access to the in-memory object server

This will allow functional tests to be used against it, and can then
be used for the storage-policy work as an example diskfile
implementation associated with a storage policy.

Change-Id: I47a88e70cee99225779baaed379b0c5d4c73611a
Fixed a failing test on my saio

The added sleep makes this test pass on my saio. I have not heard of
it failing for anyone else, but I figured I'd post this up just in
case someone does have the same problem and this fixes it for them.

Change-Id: Ia0bb09d36d0b531ade7c6a6034bbe31dd6c90a98
Override spelling of tempauth test name

Change-Id: Ife571bbeb6dd4ba5c552a2f0725c36e133e64cf4
Merge "Remove swift-bench"
Merge "Fixed a failing test on my saio"
Merge "Override spelling of tempauth test name"
Fix HEAD tempurls

HEAD-only tempurls didn't work; tempurl only allowed a HEAD request if
the tempurl was generated for GET or PUT (that is, the method in the
HMAC-signed string was "GET" or "PUT").

The intent of the code was to allow a user with a GET or a PUT tempurl
to also perform a HEAD request; I think the breaking of HEAD tempurls
is just a bug.

Change-Id: I621ddaac03e0d058dd9e7c7c374cb5c4b6386d36
Merge "Fix HEAD tempurls"
Make ssync_receiver drop conn on exc

Before, the ssync receiver wouldn't hang up the connection with all
errors. Now it will. Hanging up early will ensure the remote end gets
a "broken pipe" type error right away instead of it possible sending
more data for quite some time before finally reading the error.

Change-Id: I2d3d55baaf10f99e7edce7843a7106875752020a
Release notes for Swift 1.11.0

Change-Id: Iff13ce1fb9ab662fc4965f8f2a420f58535fb5d3
Stop mutating PATH_INFO in proxy server

The proxy server was calling swob.Request.path_info_pop() prior to
instantiating a controller so that req.path_info was just /a/c/o (sans
/v1). The version got moved over into SCRIPT_NAME.

This lead to some unfortunate behavior when trying to re-use a request
from middleware. Something like this:

    # Imagine we're a WSGIContext object here.
    #
    # To start, SCRIPT_NAME = '' and PATH_INFO='/v1/a/c/o'

    resp_iter = self._app_call(env, start_response)
    # Now SCRIPT_NAME='/v1' and PATH_INFO ='/a/c/o'

    if something_special in self._response_headers:
        env['REQUEST_METHOD'] = 'GET'
        env.pop('HTTP_RANGE', None)

	# 404 SURPRISE! The proxy calls path_info_pop() again,
        # and now SCRIPT_NAME='/v1/a' and PATH_INFO='/c/o', so this
        # gets treated as a container request. Yikes.
        resp_iter = self._app_call(env, start_response)

Now we just leave SCRIPT_NAME and PATH_INFO alone. To make life easy
for everyone who does want just /a/c/o, I defined
swob.Request.swift_entity_path, which just strips off the /v1.

Note that there's still one call to path_info_pop() in tempauth, but
that's only for requests going to /auth, so it won't affect Swift API
requests. It might be a good idea to remove that one later, but let's
do one thing at a time.

Change-Id: I87557a11c01f3f3889b610578cda6ba7d3933e7a
make test tooling less opinionated

Change-Id: I709afcec998795794a9ef13bbe7493ddd46c59b5
Merge "Preserve closeability of app iterables"