Disable case-changing behavior in Eventlet

RFC 2616 says that HTTP header fields are case-insensitive.  However, there are
some S3 clients who don't accept normalized header by Swift and Eventlet.  For
example, AWS Java SDK expects that an etag header is 'ETag', not 'Etag'.

This patch disables Eventlet's header capitalization so that the swift3
middleware can normalize the response headers as those clients expect.

Note that this change requires a fix for Eventlet, which will be included in
the next Eventlet release (v0.15).

Change-Id: I6d3428b0dafef776bdb3ebac7639b3126fa5e60d
Add X-Backend-Timestamp on more object server responses

It's particularly interesting on writes (PUT, POST, DELETE) where the
current on-disk timestamp would prevent the object server from serving
the incoming request and returns 409 Conflict.

The FakeConn has also been updated to respond in kind for 409's on
expect and all responses generaly just cause it's good to keep fakes in
line with the reals - not that I expected any existing tests to break
because of the new headers.

Change-Id: Iac6fbd2f872a9521bb2db84a333365b69f54fb6c
Object services user-agent string uses full name

It does not appear that, aside from the user-agent string, the strings
"obj-server", "obj-updater", or "obj-replicator" (or "obj-<anything>"*)
appear in the swift code base, aside from the directory containing the
object services code being named "obj".

Furthermore, the container, account, and proxy services construct their
user-agent string, as reported in the logs, using their full name. In
addition, this full name also shows up as the name of the process via
"ps" or "top", etc., which can make it easier for admins to match log
entries with other tools.

For consistency, we update the object services to use an "object-"
prefix rather than "obj-" in its user agent string.

* obj-etag does appear in a unit test, but not part of the regular
code.

Change-Id: I914fc189514207df2535731eda10cb4b3d30cc6c
Container PUT requests and max container per account

If container counter per account is equal or greater than
max_container_per_account, then all PUT requests are failed and
403 is returned.
This is correct behaviour if the request is to create a new
container, however if container already exists PUT should be
allowed, even the max_container_per_account condition has met.

This patch allows to process PUT requests for existing containers,
even if max_container_per_account > = container count.

It indirectly resolve the bug 1306711, since swift-client
uses internally PUT requests for container, prior it upload an
object there.

Change-Id: I2dcf20b6feb27e346111466a565695eba4b4b1da
clean up saio env. after runnning func tests

Added call to delete container in teardown_package().
Tests are leaving objects and containers in saio environment
after finishing running the tests.

Currently, delete_containers() is called in the setUp of each test
class, which cleans up the containers of the previous test,
but leaves the containers of the last test in the saio environment.

Change-Id: I643d7083a2a310fc3d24eab48c565f3798cff25f
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Allow HEAD for POST tempurls

HEAD requests are already allowed for GET and PUT tempurls; this
commit adds that for POST tempurls. Since POST replaces all the
object's metadata, it's quite useful to be able to HEAD the object in
order to fetch the old metadata and do a client-side merge of the new
metadata (like with normal, token-authenticated POST requests).

Change-Id: I603c7822cd27f0e304fd27024f83f95114eb0aef
Merge "clean up saio env. after runnning func tests"
Merge "Container PUT requests and max container per account"
Merge "Object services user-agent string uses full name"
Merge "Disable case-changing behavior in Eventlet"
Add v2 API to list endpoints middleware

The new API adds better support for storage policies and changes the
response from a list of backend urls to a dictionary with a key
"endpoints" that's a list of of the backend urls and a new key headers
that's a dictionary of headers to send along with the backend request.
In the v2 response format for object requests, there the headers key
includes "X-Backend-Storage-Policy-Index" which indicates the storage
policy index for the endpoints returned in the response.

Change-Id: I706a5b5be8002c633fe97b2429256735800a902e
Pass disk usage options when using --all

This commit allows the arguments --human-readable and --top to be
used with --all to customize the output of the disk usage portion.

There are no unit tests attached to this commit, I have created and
referenced bug #1344200 which shows there is a larger bug to address
to create several missing tests.

Change-Id: I46b8359533989efc3067971acaafec1d0cbc2b9f
Closes-Bug: #1344178
Related-Bug: #1344200
Fix small typo in deployment guide

Change-Id: Ifd86ce5b36c2588b57fd44c633b095d1f49fcd5b
Add X-Delete-At/After support to FormPost

This change adds the ability to specify a X-Delete-At or
X-Delete-After attribute when using the FormPost middleware.

Here is an example of what you need to add to the form:

  <input type="hidden" name="x_delete_at" value="<unix-timestamp>"/>
  <input type="hidden" name="x_delete_after" value="<seconds>"/>

To be inline with the other form imput names, x-delete-at/after has
changed to x_delete_at/after.

DocImpact
Change-Id: Ib1cc0bcf1dd7d2b689f2f26d100f9bab36880c81
Closes-Bug: #1065522
lock_file race fixes

I attempted to use this function and found a few problems.

We shouldn’t unlink the file after closing it, because someone else could lock
it in between.  Switch to unlink before close.

If someone else locked the file between our open and flock, they are likely to
unlink it out from underneath us.  Then we have a lock on a file that no longer
exists.  So stat the filename after locking to make sure the inode hasn't
changed or gone away.

We probably shouldn’t unlink the file if we time out waiting for a lock.  So
move that to before the finally block.

Change-Id: Id1858c97805d3ab81c584eaee8ce0d43d34a8089
Fix a couple small things in formpost doc string

Change-Id: Idb8d4707180396662493aacbd1f125c4ca43a1ca
Merge "Add X-Delete-At/After support to FormPost"
typo tradeoff -> trade-off

Change-Id: Iad5eb611bc727991606f8a2d4dbb08dcc8fa3ddb
Merge "Add v2 API to list endpoints middleware"
Merge "typo tradeoff -> trade-off"