Add delay in swift-bench

With enable write affinity, it's necessary to wait until
replication has moved things to their proper homes before
running delete request. With write affinity turned on, only
nodes in local region will get the object right after PUT request.

Fix bug #1198926

Change-Id: I3aa8933d45c47a010ae05561e12176479e7c9bcc
Fix range GET w/If-None-Match.

An object GET request with both If-None-Match: <Etag> and
Range: bytes=N-M should result in a 304 Not Modified if the Etag
matches. However, it was resulting in a 416 Requested Range Not
Satisfiable. This commit fixes that.

The fix was to remove conditional_response=True from a request in the
proxy. As far as I can see, the only effect that
conditional_response=True has in swob is to enable range
checking. Since the object server handles all the range stuff, the
proxy shouldn't do it. Otherwise, you wind up with double-application
of the Range header, and that doesn't work very well.

This is probably left over from the WebOb days.

Fixes bug #1196805.

Change-Id: I207f3839731b7503baa3fdbd7cd1cfe3e1b46f62
fix gettext calls to operate on uninterpolated strings

The strings need to be interpolated after the gettext call,
so that the gettext() lookup works correctly

Change-Id: Ibdb9044813f42039f7a87ca7e8fec20583aa61ae
Added container listing ratelimiting

Change-Id: If4e9cfe4e4c743de1f39704acf849164cf3f0bd0
Fix unbalanced rings after initial creation.

Sometimes you'd make a ring that should have a good balance (~= 1),
but it would come out with a poor balance (>= 90). Rebalancing a
couple times would fix it, but it shouldn't be necessary to do that.

This was due to a bug in the ring builder's rebalance() method where,
when placing a replica for a partition, it would pick *one of* the
tiers with the lowest number of replicas, but the choice was made
without regard to parts_wanted.  Now rebalancing pays attention to the
drives' parts_wanted attributes, so the tier with the emptiest drive
gets the replica.

(Also fixed a typo in a completely unrelated comment.)

Change-Id: I1821f15170f0ed212b0ad7c81b97031540ab4460
use testCase.assert instead of assert and try...except

In unit tests, we should use: self.assertTrue or self.assertFalse
instead of 'assert xxx is True/False', and use self.assertRaises instead
of 'try ... except ...'

fixes bug #1212602

Change-Id: I25318eb59cd26a98afdce5495273e3b3266655a4
[tempauth] update token if its value changes or expires

In current codes, tempauth token will be updated only when it expires. In
token's lifetime, any changes in tempauth config file doesn't update token,
which means if you update your config file and restart proxy server, the
role in runtime might not change. So a good fix is updating token when
it changes or expires.

fixes bug #1201678

Change-Id: Ieb080e87268e16d0a2e5896431aa41fcedc7a900
clean temp dir in /tmp

In unit tests, many test class use tempfile.mkdtemp in SetUp and
shutil.rmtree in tearDown.

A common wrong case is putting test data in `testdir = os.join(mkdtemp() +
'some_class')` and `rmmtree(testdir)`. As a result, /tmp/xxx/some_class
was deleted, but /tmp/xxx is still leaved in system. So a proper fix is
`rmtree(os.path.dirname(testdir), ignore_errors=1)`

fixes bug #1212583

Change-Id: Iafbe3e11f16b51bdf49abce9e68eb01f25bc5df2
Refactor how we pick listings' content type.

There were a few different places where we had some repeated code to
figure out what format an account or container listing response should
be in (text, JSON, or XML). Now that's been pulled into a single
function.

As part of this, you can now raise HTTPException subclasses in proxy
controllers instead of laboriously plumbing error responses all the
way back up to swift.proxy.server.Application.handle_request(). This
lets us avoid certain ugly patterns, like the one where a method
returns a tuple of (x, y, z, error) and the caller has to see if it
got (value, value, value, None) or (None, None, None, errorvalue). Now
we can just raise the error.

Change-Id: I316873df289160d526487ad116f6fbb9a575e3de
Run a more GC iterations to make sure weakrefs are collected

This is needed to make sure the weakref callback is fired under PyPy.

Change-Id: I5d1b83186780ee6130463fe42fac58e411ad9f79
Make the length of a line logged configurable

Failed calls to rysnc can
  result in very long log lines.

These lines are
  mostly made up
of file paths and are
  not always useful.

This change will
  allow for reducing the
length of these
  lines logged if desired.

Change-Id: I9a28f19eadc07757da9d42b0d7be1ed82170d732
Don't silence errors in spawning processes

Change-Id: I85eca76b5eef6b8eb04dfe070a24a9f05e19baa2
Merge "fix gettext calls to operate on uninterpolated strings"
change .data vrs .meta file metadata filtering in obj.diskfile

Add DATAFILE_SYSTEM_META to diskfile.py which is a set of
system-set metadata keys that cannot be changed with a POST.

Fixes: bug #1214607

Change-Id: I4bdfc1e4813a1d27fe726ba83481c6e7194aab7f
Merge "clean temp dir in /tmp"
get rid of unlinkold

We currently have two methods for cleaning up object directories - unlinkold in
the object server and hash_suffix in the replicator.

This modifies the object server to reuse the more complete hash_suffix logic.

Change-Id: Id2fd83caad979bfd446d52a775a1fa373a670cf1
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Merge "Fix range GET w/If-None-Match."
Merge "Make the length of a line logged configurable"
Merge "Don't silence errors in spawning processes"
Merge "change .data vrs .meta file metadata filtering in obj.diskfile"