Check object segment MD5s

Now if a client is downloading a large object and one of the segment
fetches gets bad data, we'll abort the request right away instead of
letting the client carry on. We do this by checking each segment's
body against the etag in the GET response's headers, like we tell
clients to do.

To make sure that the etag is the MD5, we add ?multipart-manifest=get
to all segment GET requests. That way, a segment won't be another
large object, because a large object's etag isn't its MD5. This does
mean that clients can't make, say, a DLO manifest that points to a
bunch of SLO manifests. However, that capability was only introduced
in Swift 1.13.0 (post-Havana); prior to that, clients couldn't mix
large-object types. I'm more than happy to trade that capability away
in exchange for data-integrity checks.

Change-Id: I98c78a291ee1a863a36d54a22867bd01c126644e
Support If-[Un]Modified-Since for object HEAD

We already supported it for object GET requests, but not for
HEAD. This lets clients keep metadata up-to-date without having to
either fetch the whole object when it's changed or do their own date
parsing. They can just treat Last-Modified as opaque and update their
idea of metadata when they get a 200.

Change-Id: Iff25d8989a93d651fd2c327e1e58036e79e1bde1
Fix TestGreenThreadSafeIterator test

Change-Id: Ic121ae5c99b4e5c5e3d1efc432b55b762d50f3fb
Closes-Bug: 1190244
Update swift-get-nodes to use storage_directory util func

Change-Id: I074d6c5f62e5218427c72b3e11de40da4ecfc0ab
additional functional tests for account acls

Change-Id: I7d8a8e48066281707862cd07f6b5a012476e8f24
minor cleanups for swift-container-info

Change-Id: I4d0687f64524093e2ffaa4f8d31bcff70dca145e
Invert which proxy logging middleware instance logs

Prior to this patch the proxy_logging middleware always prepared to
read the response and process it for access logging, where the
proxy_logging instance first to handle the response bytes marked the
request as "logged".

This meant that the proxy_logging instance immediately before the
proxy-server app (right most in the pipeline, if properly setup) would
always log the responses for all the client requests, regardless if
middleware to the left of it still had processing to do for the
response. This would break SLO, where the slo middleware passed along
the GET of the manifest, unchanged from the client, so that the right
most proxy logging middleware would log the sponse for just the
manifest, but then the slo middleware would fetch all the segments of
the manifest, feeding them out in the response to the client, the
request now marked as "logged" so that the left most proxy logging
middleware would not log it.

This patch inverts this behavior so that now the first proxy_logging
middleware instance to receive a request in the pipeline marks that
request as handling it. So now, the left most proxy_logging middleware
handles logging for all client reaquests, and the right most
proxy_logging middleware handles all other requests initiated from
within the pipeline to its left.

Closes bug: 1297438

Change-Id: Ia3561523db76c693e4e0b2c38f461544dfcee086
Write out ring.gz's in a safer fashion

Write the ring file to a temporary location first and then move it into
place. This should help prevent issues like reading a partial ring file
while a new instances is being written out.

Change-Id: I28357a2f038a51cb9d823822b92f736dff033a9e
Only run flake8 on Swift code

In particular, this skips the "build" directory, which is prone to
containing third-party stuff that doesn't comply with all the various
hacking/flake8/pep8 checks.

Change-Id: Ibe2240d8c6ca5947b2cac259cd7d0a1154f8a9cd
Move like unit tests together; add comments

Some simple code movement to move the utils.ratelimit_sleep() unit
tests together so that they can be viewed all at once.

We also add some comments to document the behavior of
utils.ratelimit_sleep(); small modification to max_rate parameter
checking to match intended use.

Change-Id: I3b11acfb6634d16a4b3594dba8dbc7a2d3ee8d1a
Merge "Write out ring.gz's in a safer fashion"
Mock out time and sleep to avoid races

We mock out time.time(), time.sleep() and eventlet.sleep() so that we
avoid test problems caused by exceedingly long delays during the
execution of the test.

We also make sure to convert the units used in the tests to
milliseconds for a bit more clarity.

Closes bug: 1298154

Change-Id: I803d06cbf205a02a4f7bb1e0c467d276632cd6a3
Added swift-account-info tool.

This is a very simple swift tool to retrieve information
of an account that is located on the storage node.
One can call the tool with a given account db file
as it is stored on the storage node system.
It will then return several information about that account.

Change-Id: Ibfeee790adc000fc177b4b3c03d22ff785fda325
Merge "minor cleanups for swift-container-info"
Merge "Move like unit tests together; add comments"
Merge "Mock out time and sleep to avoid races"
Merge "Added swift-account-info tool."
Fixes SUPER important typo in CHANGELOG.

Change-Id: Id1ecc425cd9e32fef4d70e97d3a705a695eb73a2
Merge "Update swift-get-nodes to use storage_directory util func"
Merge "Fixes SUPER important typo in CHANGELOG."