Allow specification of object devices for audit

In object audit "once" mode we are allowing the user to specify
a sub-set of devices to audit using the "--devices" command-line
option.  The sub-set is specified as a comma-separated list.  This
patch is taken from a larger patch to enable parallel processing
in the object auditor.

We've had to modify recon so that it will work properly with this
change to "once" mode.    We've modified dump_recon_cache()
so that it will store nested dictionaries, in other words it will
store a recon cache entry such as {'key1': {'key2': {...}}}. When
the object auditor is run in "once" mode with "--devices" set the
object_auditor_stats_ALL and ZBF entries look like:
{'object_auditor_stats_ALL': {'disk1disk2..diskn': {...}}}. When
swift-recon is run, it hunts through the nested dicts to find the
appropriate entries.  The object auditor recon cache entries are set
to {} at the beginning of each audit cycle, and individual disk
entries are cleared from cache at the end of each disk's audit cycle.

DocImpact

Change-Id: Icc53dac0a8136f1b2f61d5e08baf7b4fd87c8123
Allow pre-1970 dates in If-[Un]Modified-Since

If I want to fetch an object only if it is newer than the first moon
landing, I send a GET request with header:

    If-Modified-Since: Sun, 20 Jul 1969 20:18:00 UTC

Since that date is older than Swift, I expect a 2xx response. However,
I get a 412, which isn't even a valid thing to do for
If-Modified-Since; it should either be 2xx or 304. This is because of
two problems:

(a) Swift treats pre-1970 dates as invalid, and

(b) Swift returns 412 when a date is invalid instead of ignoring it.

This commit makes it so any time between datetime.datetime.min and
datetime.datetime.max is an acceptable value for If-Modified-Since and
If-Unmodified-Since. Dates outside that date range are treated as
invalid headers and thus are ignored, as RFC 2616 section 14.28
requires ("If the specified date is invalid, the header is ignored").

This only works for dates that the Python standard library can parse,
which on my machine is 01 Jan 1 to 31 Dec 9999. Eliminating those
restrictions would require implementing our own date parsing and
comparison, and that's almost certainly not worth it.

Change-Id: I4cb4903c4e5e3b6b3c9506c2cabbfbda62e82f35
Fix race on container recreate

There was a path on container recreate that would sometimes allow db to get
reinitialized without updating put_timestamp.  Replication would of course fix
it up, but that node would think it's database was deleted till then desipite
just ok'ing a request with a newer X-Timestamp than the deleted_timestamp on
disk.

Change-Id: I8b98afb2aac2e433b6ecb5c421ba0d778cef42fa
Closes-Bug: #1292784
Make backend container requests use the same X-Timestamp

The proxy.controllers.base's generate_request_headers will set an X-Timestamp
header for you if it didn't get populated by additional kwarg or the
transfer_headers method.  This works fine if you only call it once per
request, but because of how proxy.controllers.obj and
proxy.controllers.container fill in the backend update header chains in
_backend_requests we need multiple independent copies and call the base
controllers generate_request_headers once of each backend request - which left
the ContainerController sending down different X-Timestamp values
(microseconds apart) for PUT and DELETE.

The ObjectController skirts the issue entirely because it always preloads a
X-Timestamp on the req used to generate backend headers, and it allows it to
be copied over via transfer_headers by including 'x-timestamp' in it's
pass_through_headers attribute.

Because the container-replicator is already does merge_timestamps the
differences would always eventaully even out and there is no consistency bug,
but this seems cleaner since they put_timestamp being stored on the three
replicas during a container PUT were all coming from the same client request.

Since both PUT and DELETE were effected, and the ContainerController doesn't
need to allow X-Timestamp to pass_through like the ObjectController does for
container-sync, it seemed cleanest to fix the issue in _backend_requests via
the additional kwarg to generate_request_headers.

There's a driveby fix for FakeLogger and update to the proxy_server's
ContainerController tests.

Change-Id: Idbdf1204da33f8fb356ae35961dbdc931b228b77
move cors-test-page to literal include

This makes it so test-cors.html is a real file in doc/source so it's easy for
those in the know to jump in there with a `python -m SimpleHTTPServer` and
point their webbrowser to `http://localhost:8000/test-cors.html`.

The example html and javascript still appear in the docs in their entirety
using the Sphinx literal include directive.

Change-Id: Ia0ba36df6c58795e3764fa53b7f585dcc1b3be07
Enable object versions for SAIO by default

Change-Id: I4a20cacd6c8eb4d82a1f0b59a6b26340bc57184a
Merge "move cors-test-page to literal include"
Merge "Allow pre-1970 dates in If-[Un]Modified-Since"
Merge "Enable object versions for SAIO by default"
Merge "Fix race on container recreate"
Merge "Make backend container requests use the same X-Timestamp"
Merge "Allow specification of object devices for audit"
Add Storage Policy Support

The basic idea here is to replace the use of a single object ring in
the Application class with a collection of object rings. The
collection includes not only the Ring object itself but the policy
name assocaited with it, the filename for the .gz and any other
metadata associated with the policy that may be needed. When
containers are created, a policy (thus a specific obj ring) is
selected allowing apps to specify policy at container creation time
and leverage policies simply by using different containers for object
operations.

The policy collection is based off of info in the swift.conf file.
The format of the sections in the .conf file is as follows:

swift.conf format:

    [storage-policy:0]
    name = chicken

    [storage-policy:1]
    name = turkey
    default = yes

With the above format:

- Policy 0 will always be used for access to existing containers
  without the policy specified. The ring name for policy 0 is always
  'object', assuring backwards compatiblity. The parser will always
  create a policy 0 even if not specified

- The policy with 'default=yes' is the one used for new container
  creation. T allows the admin to specify which policy is used without
  forcing the application to add the metadata.

This commit simply introduces storage policies and the loading
thereof; nobody's using it yet. That will follow in subsequent
commits.

Change-Id: Ica05f41ecf3adb3648cc9182f11f1c8c5c678985
Storage policy support for REST API

There are two major pieces in this diff: the container piece and the
object piece.

Objects now have a storage policy index associated with them as well;
this is determined by their filesystem path. Like before, objects in
policy 0 are in /srv/node/$disk/objects; this provides compatibility
on upgrade. (Recall that policy 0 is given to all existing data when a
cluster is upgraded.) Objects in policy 1 are in
/srv/node/$disk/objects-1, objects in policy 2 are in
/srv/node/$disk/objects-2, and so on.

Containers now have a storage policy index associated with them,
stored in the container_stat table. This index is only at container
creation time, and cannot be changed without deleting and recreating
the container. This is because a container's policy index applies to
all its objects, so changing a container's policy index would require
moving large amounts of object data around.  Thus, if it is desired to
change the policy for data in a container, the supported method would
be to create a new container with the desired policy and move the data
over.

This commit does not address replicators, auditors, or updaters except
where method signatures changed. They'll still work if your cluster
has only one storage policy, though.

Change-Id: I983c7fb0cc39df002600586aa990f10d277a4b67
Add Storage Policy Support to list_endpoints

This patch makes list_endpoints policy aware so that the object is
looked up in the right ring and its actual locations returned to the
client.

Change-Id: Ic33f54d806e85c969f65e6feab8ab45b16648975
Add storage policy support to object updates

The object server will now send its storage policy index to the
container server synchronously and asynchronously (via async_pending).

Each storage policy gets its own async_pending directory under
/srv/node/$disk/objects-$N, so there's no need to change the on-disk
pickle format; the policy index comes from the async_pending's
filename. This avoids any hassle on upgrade. (Recall that policy 0's
objects live in /srv/node/$disk/objects, not objects-0.)

Also clean up a couple little things in the object updater. Now it
won't abort processing when it encounters a file (not directory) named
"async_pending-\d+", and it won't process updates in a directory that
does not correspond to a storage policy.

That is, if you have policies 1, 2, and 3, but there's a directory on
your disk named "async_pending-5", the updater will now skip over that
entirely. It won't even bother doing directory listings at all. This
is a good idea, believe it or not, because there's nothing good that
the container server can do with an update from some unknown storage
policy. It can't update the listing, it can't move the object if it's
misplaced... all it can do is ignore the request, so it's better to
just not send it in the first place. Plus, if this is due to a
misconfiguration on one storage node, then the updates will get
processed once the configuration is fixed.

Change-Id: I07c730bebaee068f75024fa9c2fa9e11e295d9bd
Add Storage Policy Support to the Auditor

This patch makes the object auditor policy-aware, so it'll audit
objects in any storage policy.

Change-Id: I94e3a7937d9814b9ecef6ca35371e245a43513d3
Add storage policy support for the Replicator

This makes it so that objects stored in all policies get replicated
properly. This is only for rsync replication, not ssync.

Change-Id: Ifdb4624841f35953ba80189e669d3ef15d5563fd
Add Storage Policy Support to ssync

This patch makes ssync policy aware so that clusters using storage
policies and ssync replication will replicate objects in all policies.

Change-Id: I64879077676d764c6330e03734fc6665bb26f552