dispersion report option to output missing parts

Adds a -p option to swift-dispersion-report that will output the
partitions missing copies to standard error. Another thing we've been
meaning to add for forever. It's useful when you want to do some
further research on whether a partition truly has fewer copies or if
they're just somewhere else in the cluster due to a ring change, for
instance.

DocImpact

Change-Id: I5e47aa5818483ecc34b39ef6f8cd83ad312ed9a0
Handle down object servers in container-sync

If an object server is down, container-sync stops syncing the container
even if the it gets object copies from "up" obejct servers.

Bug 1069910

In case the git history gets mangled, this fix was done almost entirely
by Donagh McCabe <donagh.mccabe@hp.com>.

Change-Id: Ieeadcfeb4e880fe5f08e284d7c12492bf7a29460
blueprint Multi-range support implementation

This change adds multi range retrieval to OpenStack Swift. For non-
segmented data object, a client can use HTTP Range header to specify
multiple ranges to retrieve sections of the data object.

This implementation currently does not support segmented data object
multi range retrieval. When a client sends a multi range request
against a segmented data object, Swift will return HTTP status code
200. Support for segmented data multi range retrieval will be added
in near future.

This implementation is to bring Swift closer to CDMI multi range
data retrieval standard. Once support for segemented data multi
range is added, Swift will be compliant with CDMI standard in
this area.

DocImpact

Change-Id: I4ed1fb0a0a93c037ddb2f551ea62afe447945107
re-use headers_to_container_info on container GET

Currently, a container's info can be cached without cors data intact after
a container GET.

I made headers_to_container_info a function instead of a method and I crammed
all container metadata into container_info.
This is so e.g. staticweb can eventually re-use the same container info cache.

Fix pep8 in swift/proxy/controllers/container.py

Change-Id: I4bbb042dde79afac48395efc38bd80f0ff240e1f
Merge "Handle down object servers in container-sync"
Separate wsgi init from web-front-end activation

Separate the request processor initialization steps
    Load common settings from conf
    Set the logger
    Load the request processor
from the web front end server application step

On branch bp/wsgi-application-interface-7
   modified:   swift/common/wsgi.py

Implements Blueprint:   wsgi-application-interface

Change-Id: I25fabad8846cd644e4bf1e1bd9c2e568512e12e5
Merge "blueprint Multi-range support implementation"
Make multipart/byteranges boundary always length 32.

Due to a lack of zero-padding in a format string, there was a 1/16
chance that the length of the randomly-generated boundary string was
less than 32. This didn't affect client downloads, but it did give the
unit tests a 1/16 chance of failing (there's a [0-9a-f]{32} regex in a
test).

Now we left-zero-pad the boundary string to length 32, and the test
consistently passes.

Change-Id: I4ee98dab127e5674e59b078d8090470ec783e0b0
Merge "Separate wsgi init from web-front-end activation"
Allow object-expirer to delete the last version of an object

Fix bug 1067677

When we delete a versioned object, the last version will be poped out from the
versiones container. When a versioned object is expired and deleted by object-
expirer, the last version is restored but remains in the versions container
instead of getting deleted. The reason is object-expirer will set
'X-If-Delete- At' header when deleteing an object. However this is for the
current version - not for the last version. When the object-server is trying
to delete the last version, the transaction will fail with error:
X-If-Delete-At and X-Delete-At do not match. Delete the 'X-If-Delete-At'
field in the later built delete request would help to solve this issue.

This patch, without the test, was first proposed by
Zhou Yuan <yuan.zhou@intel.com>.

Change-Id: I62c97e6a0f140888497e189129825865fb6f7966
Merge "dispersion report option to output missing parts"
Merge "Make multipart/byteranges boundary always length 32."
Merge "Allow object-expirer to delete the last version of an object"
Remove dead code in swob.Response.__init__.

self.status would have been "200 OK", not 200, so the branch was never
taken. self.status_int would have been 200.

This looked like part of an attempt to honor the Range header in GET
requests. However, when I changed the code to check if self.status_int
== 200, the behavior gets weird. In the case of a GET request w/Range
header, the status gets changed from a 200 (from the underlying
app/middleware stack) into a 206. However, the response body isn't
actually changed to be of the right length. The net result is a
response w/status "206 Partial Content" that actually contains the
full content.

RFC 2616 section 14.35.2 says "A server MAY ignore the Range header."
The right thing to do is either (a) return a 206 with actual partial
content, or (b) ignore the Range header. For object GETs, there's
already code to do (a) [see commit ce274b3]. For other GETs, we should
ignore the Range header instead of returning a Frankenresponse.

This code is dead, and rightly so; let's bury it six feet deep in the
bit bucket.

Fixes bug 1068279.

Change-Id: Id648dfda9cba45012b5da097a235598fbd97c8b0
Merge "re-use headers_to_container_info on container GET"
Split swob.content_range_header() into two methods.

content_range_header() took an optional parameter 'value_only'. If
true, it returned just the header value; if false, the whole header.
At all the call sites, value_only was either a hardcoded True or
False.

Instead, let's have two methods (one for the full header, one for the
header value) and just call the right one.

Change-Id: I3aa03aca233437e9bdbed7edf9622756e19b4c49
Merge "Remove dead code in swob.Response.__init__."
Use pypi for python-swiftclient dependency.

Using github as a package source is less reliable in the Jenkins test
framework than using pypi. This is because the Jenkins test framework
has a local pypi mirror.

Change-Id: Ia904d39ce70f5c2bd7d6edb9172f8226131074e2
Merge "Split swob.content_range_header() into two methods."
Set content-type on account/container head

Change-Id: Ib54fa5adb7539bebfbd6644064be9d7f4d7af9db