Add delay in swift-bench

With enable write affinity, it's necessary to wait until
replication has moved things to their proper homes before
running delete request. With write affinity turned on, only
nodes in local region will get the object right after PUT request.

Fix bug #1198926

Change-Id: I3aa8933d45c47a010ae05561e12176479e7c9bcc
Fix unbalanced rings after initial creation.

Sometimes you'd make a ring that should have a good balance (~= 1),
but it would come out with a poor balance (>= 90). Rebalancing a
couple times would fix it, but it shouldn't be necessary to do that.

This was due to a bug in the ring builder's rebalance() method where,
when placing a replica for a partition, it would pick *one of* the
tiers with the lowest number of replicas, but the choice was made
without regard to parts_wanted.  Now rebalancing pays attention to the
drives' parts_wanted attributes, so the tier with the emptiest drive
gets the replica.

(Also fixed a typo in a completely unrelated comment.)

Change-Id: I1821f15170f0ed212b0ad7c81b97031540ab4460
[tempauth] update token if its value changes or expires

In current codes, tempauth token will be updated only when it expires. In
token's lifetime, any changes in tempauth config file doesn't update token,
which means if you update your config file and restart proxy server, the
role in runtime might not change. So a good fix is updating token when
it changes or expires.

fixes bug #1201678

Change-Id: Ieb080e87268e16d0a2e5896431aa41fcedc7a900
Refactor how we pick listings' content type.

There were a few different places where we had some repeated code to
figure out what format an account or container listing response should
be in (text, JSON, or XML). Now that's been pulled into a single
function.

As part of this, you can now raise HTTPException subclasses in proxy
controllers instead of laboriously plumbing error responses all the
way back up to swift.proxy.server.Application.handle_request(). This
lets us avoid certain ugly patterns, like the one where a method
returns a tuple of (x, y, z, error) and the caller has to see if it
got (value, value, value, None) or (None, None, None, errorvalue). Now
we can just raise the error.

Change-Id: I316873df289160d526487ad116f6fbb9a575e3de
get rid of unlinkold

We currently have two methods for cleaning up object directories - unlinkold in
the object server and hash_suffix in the replicator.

This modifies the object server to reuse the more complete hash_suffix logic.

Change-Id: Id2fd83caad979bfd446d52a775a1fa373a670cf1
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Improve swift-bench performance

Single-host bench controller should be monkey-patched to achieve
maximum concurrency.

Change-Id: I77c63773edc7427d06c2fbfb67ededf36d9913d3
remove unused reference to resellers_conf

Change-Id: I956f25f44b20f65c72beddb2cd49225bdecc8430
Fix setdefault() for swob's HeaderKeyDict.

You'd think this would just work, given that HeaderKeyDict inherits
from dict and overrides the usual __thingy__ methods, but it
doesn't. It would work if you title-cased the key, but the whole point
of HeaderKeyDict is to do that for you.

Change-Id: If5c22df0690a245d1dd02fa3a52fa135235fe60d
Merge "Refactor how we pick listings' content type."
Merge "Fix setdefault() for swob's HeaderKeyDict."
Refactor path splitting and validation.

The account, container, and object servers all do the same thing at
the top of GET/PUT/etc.: they split the path, validate that the first
two components are a good (device, partition) pair, and return a 400
if they're not. The object server already had a module-local helper
function for this, but the account and container servers just had
duplicate boilerplate code. Now it's all in one common helper.

Change-Id: I9a20d37fc9e1a68b10149a7aa78cb9691fc04ea9
Don't use float() when normalize_timestamp() does it.

And add a comment to note for the curious when the current timestamp
will bump to 11 digits of seconds (which implies that one would no
longer be able to use string compare on timestamps of differing
length).

Change-Id: I823e7659c41a54915d7c9c64c9161a29e79cd48c
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Fix regression that causes Timeouts to return 500

When the hacking changes were put in, one of the exceptions that were
fixed didn't also add a catch for Timeout, which causes certain timeouts
to cause a 500 return

Change-Id: I7a9db200528303868d8a9ab05f35a70d99bfb118
Added test for timeout regression

Change-Id: Ia0e5f1595d1b1f58e6c415d7397f33496576c944
Merge "Don't use float() when normalize_timestamp() does it."
Clean up some of the proxy unit tests.

Since we've got machinery to catch HTTPException errors that happens
after the proxy server's __call__ method but before the individual
GET()/POST()/etc. methods, we shouldn't just call
GET()/POST()/etc. Instead, we should use swob.Request's get_response()
method, or otherwise actually call the proxy application. That way we
can have the code raise HTTPBadRequest, but the test just gets a
plain old response with a 400 status code, just like you'd expect.

Earlier changes fixed some of the tests; this commit fixes some
more. In the interest of not having a gigantic diff, this only gets
about half the proxy's unit tests, and the others will be done in a
subsequent commit.

Change-Id: Iaca32e74a8f4499440e655c207a1012e14486c74
Remove duplicate code for update_put_timestamp().

Change-Id: I035283f30257961d6efb98f1bb65d27bf6a5bbf5
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Clean up utf8ness quote

Move common codes on utf8ness and quote to common/utils.py

Change-Id: I91d98a06fa94ac608119a0d70adedc6d73337c64
Merge "get rid of unlinkold"
Merge "Added test for timeout regression"