Make POST for bulk delete actually work

Change-Id: I568e7e31df3dcbeac20dba6d543a13c0409de00e
Closes-Bug: 1232787
Mute hash_suffix rmdir failures

Change-Id: I10acfb1a3005ef988ac8a88819bf754b98cea63a
Warn if read_affinity is configured but not enabled

To get the proxy's read affinity to work, you have to set both
"read_affinity = <stuff>" and "sorting_method = affinity" in the proxy
config. If you set the first but not the second, then you don't get
read affinity, and Swift doesn't help you determine why not.

Now the proxy will emit a warning message if read_affinity is set but
sorting_method is a value other than "affinity", so if you check your
logs to see why it isn't working, you'll get a hint.

Note that the message comes out twice per proxy process, so with 2
workers you'll see the warning 6 times on startup (2 for master + 2 *
2 per worker). It's sort of annoying, but at least it's not
per-request.

Bonus docstring fix: remove a sentence that's not true

Change-Id: Iad37d4979a1b7c45c0e3d1b83336dbcf7a68a0c9
Update account quota doc

A note was added stating that the same limitations apply to
account quotas as for container quotas. An example on uploads
without a content-length headers was added.

Related-Bug: 1267659
Change-Id: Ic29b527cb71bf5903c2823844a1cf685ab6813dd
New container sync configuration option

Summary of the new configuration option:

The cluster operators add the container_sync middleware to their
proxy pipeline and create a container-sync-realms.conf for their
cluster and copy this out to all their proxy and container servers.
This file specifies the available container sync "realms".

A container sync realm is a group of clusters with a shared key that
have agreed to provide container syncing to one another.

The end user can then set the X-Container-Sync-To value on a
container to //realm/cluster/account/container instead of the
previously required URL.

The allowed hosts list is not used with this configuration and
instead every container sync request sent is signed using the realm
key and user key.

This offers better security as source hosts can be faked much more
easily than faking per request signatures. Replaying signed requests,
assuming it could easily be done, shouldn't be an issue as the
X-Timestamp is part of the signature and so would just short-circuit
as already current or as superceded.

This also makes configuration easier for the end user, especially
with difficult networking situations where a different host might
need to be used for the container sync daemon since it's connecting
from within a cluster. With this new configuration option, the end
user just specifies the realm and cluster names and that is resolved
to the proper endpoint configured by the operator. If the operator
changes their configuration (key or endpoint), the end user does not
need to change theirs.

DocImpact

Change-Id: Ie1704990b66d0434e4991e26ed1da8b08cb05a37
Allow programmatic reloading of Swift hash path config

New util's function validate_hash_conf allows you to programmatically reload
swift.conf and hash path global vars HASH_PATH_SUFFIX and HASH_PATH_PREFIX
when they are invalid.

When you load swift.common.utils before you have a swift.conf there's no good
way to force a re-read of swift.conf and repopulate the hash path config
options - short of restarting the process or reloading the module - both of
which are hard to unittest.  This should be no worse in general and in some
cases easier.

Change-Id: I1ff22c5647f127f65589762b3026f82c9f9401c1
use specific hacking rules

Change-Id: I91cb5e30ec081262e2a590d64ba46fddadc7590f
Merge "Make POST for bulk delete actually work"
Merge "Allow programmatic reloading of Swift hash path config"
Merge "Update account quota doc"
Move all SLO functionality to middleware

This way, with zero additional effort, SLO will support enhancements
to object storage and retrieval, such as:
 * automatic resume of GETs on broken connection (today)
 * storage policies (in the near future)
 * erasure-coded object segments (in the far future)

This also lets SLOs work with other sorts of hypothetical third-party
middleware, for example object compression or encryption.

Getting COPY to work here is sort of a hack; the proxy's object
controller now checks for "swift.copy_response_hook" in the request's
environment and feeds the GET response (the source of the new object's
data) through it. This lets a COPY of a SLO manifest actually combine
the segments instead of merely copying the manifest document.

Updated ObjectController to expect a response's app_iter to be an
iterable, not just an iterator. (PEP 333 says "When called by the
server, the application object must return an iterable yielding zero
or more strings." ObjectController was just being too strict.) This
way, SLO can re-use the same response-generation logic for GET and
COPY requests.

Added a (sort of hokey) mechanism to allow middlewares to close
incompletely-consumed app iterators without triggering a warning. SLO
does this when it realizes it's performed a ranged GET on a manifest;
it closes the iterable, removes the range, and retries the
request. Without this change, the proxy logs would get 'Client
disconnected on read' in them.

DocImpact

blueprint multi-ring-large-objects

Change-Id: Ic11662eb5c7176fbf422a6fc87a569928d6f85a1
Merge "Warn if read_affinity is configured but not enabled"
Merge "Move all SLO functionality to middleware"
Merge "New container sync configuration option"
Add missing setting to SLO sample conf

Change-Id: I3b3a306e855509d6d03223ea59dfbd109647733c
Consolidating and standardizing x-delete-at format

Change-Id: Idc916da1c7fe1cc43a2c26f7f7ee1d4fcdd52c89
Expose some SLO constraints via /info

These are the ones that affect what requests a client can make; the
others are just time and speed limits, so they weren't as interesting.

Change-Id: I21c19c950227f02725aafc309a3996fc6749a383
Mute hash_cleanup_listdir common failures

Change-Id: I0fffa387b5fae793de91870e20cab4e32e94387a
Merge "Add missing setting to SLO sample conf"
Use constant time comparison in tempURL

Use constant time comparison when evaluating tempURL to avoid timing
attacks (CVE-2014-0006).

Fixes bug 1265665

Change-Id: I11e4ad83cc4077e52adf54a0bd0f9749294b2a48