Make POST for bulk delete actually work

Change-Id: I568e7e31df3dcbeac20dba6d543a13c0409de00e
Closes-Bug: 1232787
Log when quarantining an object

When you're trying to troubleshoot why all your objects are getting
quarantined, it's really nice to have some logging. If nothing else,
it's nice to know which process did it.

Change-Id: I6e8be6df938659f7392891df9336ed70bd155706
Warn if read_affinity is configured but not enabled

To get the proxy's read affinity to work, you have to set both
"read_affinity = <stuff>" and "sorting_method = affinity" in the proxy
config. If you set the first but not the second, then you don't get
read affinity, and Swift doesn't help you determine why not.

Now the proxy will emit a warning message if read_affinity is set but
sorting_method is a value other than "affinity", so if you check your
logs to see why it isn't working, you'll get a hint.

Note that the message comes out twice per proxy process, so with 2
workers you'll see the warning 6 times on startup (2 for master + 2 *
2 per worker). It's sort of annoying, but at least it's not
per-request.

Bonus docstring fix: remove a sentence that's not true

Change-Id: Iad37d4979a1b7c45c0e3d1b83336dbcf7a68a0c9
Update account quota doc

A note was added stating that the same limitations apply to
account quotas as for container quotas. An example on uploads
without a content-length headers was added.

Related-Bug: 1267659
Change-Id: Ic29b527cb71bf5903c2823844a1cf685ab6813dd
Add missing license header to test file

All the other tests have license headers, so this one should too.

I picked 2013 for the copyright year because that's when "git log"
says it was first and last touched.

Change-Id: Idd41a179322a3383f6992e72d8ba3ecaabd05c47
New container sync configuration option

Summary of the new configuration option:

The cluster operators add the container_sync middleware to their
proxy pipeline and create a container-sync-realms.conf for their
cluster and copy this out to all their proxy and container servers.
This file specifies the available container sync "realms".

A container sync realm is a group of clusters with a shared key that
have agreed to provide container syncing to one another.

The end user can then set the X-Container-Sync-To value on a
container to //realm/cluster/account/container instead of the
previously required URL.

The allowed hosts list is not used with this configuration and
instead every container sync request sent is signed using the realm
key and user key.

This offers better security as source hosts can be faked much more
easily than faking per request signatures. Replaying signed requests,
assuming it could easily be done, shouldn't be an issue as the
X-Timestamp is part of the signature and so would just short-circuit
as already current or as superceded.

This also makes configuration easier for the end user, especially
with difficult networking situations where a different host might
need to be used for the container sync daemon since it's connecting
from within a cluster. With this new configuration option, the end
user just specifies the realm and cluster names and that is resolved
to the proper endpoint configured by the operator. If the operator
changes their configuration (key or endpoint), the end user does not
need to change theirs.

DocImpact

Change-Id: Ie1704990b66d0434e4991e26ed1da8b08cb05a37
Allow programmatic reloading of Swift hash path config

New util's function validate_hash_conf allows you to programmatically reload
swift.conf and hash path global vars HASH_PATH_SUFFIX and HASH_PATH_PREFIX
when they are invalid.

When you load swift.common.utils before you have a swift.conf there's no good
way to force a re-read of swift.conf and repopulate the hash path config
options - short of restarting the process or reloading the module - both of
which are hard to unittest.  This should be no worse in general and in some
cases easier.

Change-Id: I1ff22c5647f127f65589762b3026f82c9f9401c1
Merge "Log when quarantining an object"
Merge "Add missing license header to test file"
Merge "Make POST for bulk delete actually work"
Merge "Allow programmatic reloading of Swift hash path config"
Merge "Update account quota doc"
Move all SLO functionality to middleware

This way, with zero additional effort, SLO will support enhancements
to object storage and retrieval, such as:
 * automatic resume of GETs on broken connection (today)
 * storage policies (in the near future)
 * erasure-coded object segments (in the far future)

This also lets SLOs work with other sorts of hypothetical third-party
middleware, for example object compression or encryption.

Getting COPY to work here is sort of a hack; the proxy's object
controller now checks for "swift.copy_response_hook" in the request's
environment and feeds the GET response (the source of the new object's
data) through it. This lets a COPY of a SLO manifest actually combine
the segments instead of merely copying the manifest document.

Updated ObjectController to expect a response's app_iter to be an
iterable, not just an iterator. (PEP 333 says "When called by the
server, the application object must return an iterable yielding zero
or more strings." ObjectController was just being too strict.) This
way, SLO can re-use the same response-generation logic for GET and
COPY requests.

Added a (sort of hokey) mechanism to allow middlewares to close
incompletely-consumed app iterators without triggering a warning. SLO
does this when it realizes it's performed a ranged GET on a manifest;
it closes the iterable, removes the range, and retries the
request. Without this change, the proxy logs would get 'Client
disconnected on read' in them.

DocImpact

blueprint multi-ring-large-objects

Change-Id: Ic11662eb5c7176fbf422a6fc87a569928d6f85a1
Merge "Warn if read_affinity is configured but not enabled"
Merge "Move all SLO functionality to middleware"
Merge "New container sync configuration option"