Adds a retry mechanism when deleting containers

Bulk middleware now has a mechanism to retry a delete when the
HTTP response code is 409. This happens when the container still
has objects. It can be useful in a bulk delete where you delete
all the objects in a container and then try to delete the container.
It is very likely that at the end it will fail because the replica
objects have not been deleted by the time the middleware got a
successful response.

Change-Id: I1614fcb5cc511be26a9dda90753dd08ec9546a3c
Closes-Bug: #1253478
Some functional tests for object versioning

Also fix some exception-handling stuff in functional tests; at some
point, ResponseError() started needing two more parameters, but the
functional tests (not swift_test_client, just tests.py) still had a
couple spots that were not passing in the new params. Now they're
optional again, and if you omit them, then the stringification of the
ResponseError is just a little less useful than it could be.

Change-Id: I38968c4b590fc04b97b85c5f974c8648291a6689
Sanify handoff search depth with non-integer replica counts

On GET, the proxy will go search the primary nodes plus some number of
handoffs for the account/container/object before giving up and
returning a 404. That number is, by default, twice the ring's replica
count. This was fine if your ring had an integral number of replicas,
but could lead to some slightly-odd behavior if you have fractional
replicas.

For example, imagine that you have 3.49 replicas in your object ring;
perhaps you're migrating a cluster from 3 replicas to 4, and you're
being smart and doing it a bit at a time.

On an object GET that all the primary nodes 404ed, the proxy would
then compute 2 * 3.49 = 6.98, round it up to 7, and go look at 7
handoff nodes. This is sort of weird; the intent was to look at 6
handoffs for objects with 3 replicas, and 8 handoffs for objects with
4, but the effect is 7 for everybody.

You also get little latency cliffs as you scale up replica counts. If,
instead of 3.49, you had 3.51 replicas, then the proxy would look at 8
handoff nodes in every case [ceil(2 * 3.51) = 8], so there'd be a
small-but-noticeable jump in the time it takes to produce a 404.

The fix is to compute the number of handoffs based on the number of
primary nodes for the partition, not the ring's replica count. This
gets rid of the little latency cliffs and makes the behavior more like
what you get with integral replica counts.

If your ring has an integral number of replicas, there's no behavior
change here.

Change-Id: I50538941e571135299fd6b86ecd9dc780cf649f5
Various optimizations to RingBuilder.rebalance()

Overall gain is about 20-22% of time on my laptop. This includes:

* replacing string sort_key with a tuple (because we can and because
  compairing two 3-tuples is faster than comparing two 26-byte strings);
* keeping track of hungriest dev in tier (allows us to use built-in
  dict.__getitem__ as a key instead of lambdas in couple of places);
* remove unnecessary sorted() call in the innermost loop (because we
  don't need to sort all of them or better we don't need to sort
  anything there);
* memoize tiers for each dev (saves just a couple of percents but why
  not).

Performance measurments have been done using this script:
http://paste.openstack.org/show/55609/

Related-Bug: #1262166
Related-Bug: #1261659
Change-Id: If38bd9fe82efc12b01e9aa146e0f4ab565fb6bea
Make POST for bulk delete actually work

Change-Id: I568e7e31df3dcbeac20dba6d543a13c0409de00e
Closes-Bug: 1232787
Documentation of the logs formats used in Swift

Change-Id: I402014627bab7113105a6a63c0b0ce1267735ca0
Refactor to share on-disk layout with hash cleanup

Closes-Bug: 1260132
Change-Id: Iaa367c686b8dc49dd53c55a7cca661d9611044f8
Generic means for persisting system metadata.

Middleware or core features may need to store metadata
against accounts or containers. This patch adds a
generic mechanism for system metadata to be persisted
in backend databases, without polluting the user
metadata namespace, by using the reserved header
namespace x-<server_type>-sysmeta-*.

Modifications are firstly that backend servers persist
system metadata headers alongside user metadata and
other system state.

For accounts and containers, system metadata in PUT
and POST requests is treated in a similar way to user
metadata. System metadata is not yet supported for
object requests.

Secondly, changes in the proxy controllers ensure that
headers in the system metadata namespace will pass through
in requests to backend servers.

Thirdly, system metadata returned from backend servers
in GET or HEAD responses is added to the cached info
dict, which middleware can access.

Finally, a gatekeeper middleware module is provided
which filters all system metadata headers from requests
and responses by removing headers with names starting
x-account-sysmeta-, x-container-sysmeta-. The gatekeeper
also removes headers starting x-object-sysmeta- in
anticipation of future support for system metadata being
set for objects. This prevents clients from writing or
reading system metadata.

The required_filters list in swift/proxy/server.py is
modified to include the gatekeeper middleware so that
if the gatekeeper has not been configured in the
pipeline then it will be automatically inserted close
to the start of the pipeline.

blueprint cluster-federation

Change-Id: I80b8b14243cc59505f8c584920f8f527646b5f45
Move the tests from functionalnosetests

Move the tests from functionalnosetests under functional, so we no
longer have two seperate trees for functional tests. This also drops
the 'nose' name from the directory, so that it doesn't end up with
confusion if we move to testr. Further, since there are no longer two
test runs in .functests, it nows looks very close to the other two.

Change-Id: I8de025c29d71f05072e257df24899927b82c1382
Add Swift-account-stats as a related project.

Change-Id: If2620a6b448581697d0d049578ea28faead6fbcf
Mute hash_suffix rmdir failures

Change-Id: I10acfb1a3005ef988ac8a88819bf754b98cea63a
Merge "Adds a retry mechanism when deleting containers"
Log when quarantining an object

When you're trying to troubleshoot why all your objects are getting
quarantined, it's really nice to have some logging. If nothing else,
it's nice to know which process did it.

Change-Id: I6e8be6df938659f7392891df9336ed70bd155706
Warn if read_affinity is configured but not enabled

To get the proxy's read affinity to work, you have to set both
"read_affinity = <stuff>" and "sorting_method = affinity" in the proxy
config. If you set the first but not the second, then you don't get
read affinity, and Swift doesn't help you determine why not.

Now the proxy will emit a warning message if read_affinity is set but
sorting_method is a value other than "affinity", so if you check your
logs to see why it isn't working, you'll get a hint.

Note that the message comes out twice per proxy process, so with 2
workers you'll see the warning 6 times on startup (2 for master + 2 *
2 per worker). It's sort of annoying, but at least it's not
per-request.

Bonus docstring fix: remove a sentence that's not true

Change-Id: Iad37d4979a1b7c45c0e3d1b83336dbcf7a68a0c9
Raise diskfile.py module coverage to > 98%

We attempt to get the code coverage (with branch coverage) to 100%,
but fall short because due to interactions between coverage.py and
CPython's peephole optimizer. See:

    https://bitbucket.org/ned/coveragepy/issue/198/continue-marked-as-not-covered

In the main diskfile module, we remove the check for a valid
"self._tmppath" since it is only one of a number of fields that could
be verified and it was not worth trying to get coverage for it. We
also remove the try / except around the close() method call in the
DiskFileReader's app_iter_ranges() method since it will never be
called in a context that will raise a quarantine exception (by
definition ranges can't generate a quarantine event).

We also:

* fix where quarantine messages are checked to ensure the
  generator is actually executed before the check
* in new and modified tests:
  * use assertTrue in place of assert_
  * use assertEqual in place of assertEquals
* fix references to the reserved word "object"

Change-Id: I6379be04adfc5012cb0b91748fb3ba3f11200b48
Merge "Refactor to share on-disk layout with hash cleanup"
Merge "Move the tests from functionalnosetests"
Merge "Add Swift-account-stats as a related project."
Update account quota doc

A note was added stating that the same limitations apply to
account quotas as for container quotas. An example on uploads
without a content-length headers was added.

Related-Bug: 1267659
Change-Id: Ic29b527cb71bf5903c2823844a1cf685ab6813dd
Merge "Raise diskfile.py module coverage to > 98%"