Add delay in swift-bench

With enable write affinity, it's necessary to wait until
replication has moved things to their proper homes before
running delete request. With write affinity turned on, only
nodes in local region will get the object right after PUT request.

Fix bug #1198926

Change-Id: I3aa8933d45c47a010ae05561e12176479e7c9bcc
Fix unbalanced rings after initial creation.

Sometimes you'd make a ring that should have a good balance (~= 1),
but it would come out with a poor balance (>= 90). Rebalancing a
couple times would fix it, but it shouldn't be necessary to do that.

This was due to a bug in the ring builder's rebalance() method where,
when placing a replica for a partition, it would pick *one of* the
tiers with the lowest number of replicas, but the choice was made
without regard to parts_wanted.  Now rebalancing pays attention to the
drives' parts_wanted attributes, so the tier with the emptiest drive
gets the replica.

(Also fixed a typo in a completely unrelated comment.)

Change-Id: I1821f15170f0ed212b0ad7c81b97031540ab4460
[tempauth] update token if its value changes or expires

In current codes, tempauth token will be updated only when it expires. In
token's lifetime, any changes in tempauth config file doesn't update token,
which means if you update your config file and restart proxy server, the
role in runtime might not change. So a good fix is updating token when
it changes or expires.

fixes bug #1201678

Change-Id: Ieb080e87268e16d0a2e5896431aa41fcedc7a900
get rid of unlinkold

We currently have two methods for cleaning up object directories - unlinkold in
the object server and hash_suffix in the replicator.

This modifies the object server to reuse the more complete hash_suffix logic.

Change-Id: Id2fd83caad979bfd446d52a775a1fa373a670cf1
Signed-off-by: Peter Portante <peter.portante@redhat.com>
remove unused reference to resellers_conf

Change-Id: I956f25f44b20f65c72beddb2cd49225bdecc8430
Refactor path splitting and validation.

The account, container, and object servers all do the same thing at
the top of GET/PUT/etc.: they split the path, validate that the first
two components are a good (device, partition) pair, and return a 400
if they're not. The object server already had a module-local helper
function for this, but the account and container servers just had
duplicate boilerplate code. Now it's all in one common helper.

Change-Id: I9a20d37fc9e1a68b10149a7aa78cb9691fc04ea9
Don't use float() when normalize_timestamp() does it.

And add a comment to note for the curious when the current timestamp
will bump to 11 digits of seconds (which implies that one would no
longer be able to use string compare on timestamps of differing
length).

Change-Id: I823e7659c41a54915d7c9c64c9161a29e79cd48c
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Fix regression that causes Timeouts to return 500

When the hacking changes were put in, one of the exceptions that were
fixed didn't also add a catch for Timeout, which causes certain timeouts
to cause a 500 return

Change-Id: I7a9db200528303868d8a9ab05f35a70d99bfb118
Added test for timeout regression

Change-Id: Ia0e5f1595d1b1f58e6c415d7397f33496576c944
Merge "Don't use float() when normalize_timestamp() does it."
Clean up some of the proxy unit tests.

Since we've got machinery to catch HTTPException errors that happens
after the proxy server's __call__ method but before the individual
GET()/POST()/etc. methods, we shouldn't just call
GET()/POST()/etc. Instead, we should use swob.Request's get_response()
method, or otherwise actually call the proxy application. That way we
can have the code raise HTTPBadRequest, but the test just gets a
plain old response with a 400 status code, just like you'd expect.

Earlier changes fixed some of the tests; this commit fixes some
more. In the interest of not having a gigantic diff, this only gets
about half the proxy's unit tests, and the others will be done in a
subsequent commit.

Change-Id: Iaca32e74a8f4499440e655c207a1012e14486c74
Remove duplicate code for update_put_timestamp().

Change-Id: I035283f30257961d6efb98f1bb65d27bf6a5bbf5
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Clean up utf8ness quote

Move common codes on utf8ness and quote to common/utils.py

Change-Id: I91d98a06fa94ac608119a0d70adedc6d73337c64
Merge "get rid of unlinkold"
Merge "Added test for timeout regression"
Merge "Clean up some of the proxy unit tests."
Add constructor args to swob.Request.blank

I knew webob.Request.blank could take most of the attributes on the class as
kwargs to blank, so I went and looked how.  It seems to work ok and is pretty
nice.

Change-Id: I72fae7c28f81c97768ee98b8ebcd69789a4c2e84
Merge "Clean up utf8ness quote"
Refactor DiskFile init logic into multiple methods

The DiskFile constructor needs to grow a bit in the future it is was
already considered "large" by some. This breaks the logic up into
three main parts:

  1. Determine if the object exists, fetching the on-disk file(s) to
     use
  2. Finish the object construction when we have a tombstone file
  3. Finish the object construction when we have a regular data file,
     and optionally a fast-POST metadata file

We added a number of assertions as a way of documenting the expected
behavior of the loop that figures out the on-disk file to use, and a
set of unit tests to verify the loop works as expected.

Change-Id: I1503b7561e280ded3222a09514edc59ac26dbc65
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Merge "Add delay in swift-bench"