Updated probe tests

The probe tests were woefully out of date with all the changes that
have ocurred since they were written. I've updated most of them and
removed some that are hopeless outdated.

I also greatly improved the timing issues (hopefully completely
solved them? I ran them 25 times with no problems) and made them pep8
1.3.1 safe.

Change-Id: I8e9dbb6e7d6e04e293843b1dce1ded99d84e0348
Do not use pickle for serialization in memcache, but JSON

We don't want to use pickle as it can execute arbitrary code. JSON is
safer. However, note that it supports serialization for only some
specific subset of object types; this should be enough for what we need,
though.

To avoid issues on upgrades (unability to read pickled values, and cache
poisoning for old servers not understanding JSON), we add a
memcache_serialization_support configuration option, with the following
values:

 0 = older, insecure pickle serialization
 1 = json serialization but pickles can still be read (still insecure)
 2 = json serialization only (secure and the default)

To avoid an instant full cache flush, existing installations should
upgrade with 0, then set to 1 and reload, then after some time (24
hours) set to 2 and reload. Support for 0 and 1 will be removed in
future versions.

Part of bug 1006414.

Change-Id: Id7d6d547b103b4f23ebf5be98b88f09ec6027ce4
Fixed bug where expirer would get confused by...

Fixed bug where expirer would get confused by odd deletion times.
Since this has already rolled out, I just capped things at ten 9s, or
Sat Nov 20 17:46:39 2286. I can't wait for the Y2286 world panic. :/

Change-Id: Iba10963faa344a418a1fa573d5c85f4ff864b574
A few misc. code cleanups.

Fixed a couple config gets which weren't using TRUE_VALUES (hopefully
the last ones?).

Removed a deprecated parameter from
swift.common.db.ContainerBroker.list_objects_iter() a "TODO" which dates
back to the original import of Swift into github.

Also made AccountBroker.reclaim()'s docstring match its arg names.

Change-Id: I646bdd00a3693763b55a50ba210c7ed2ba1df4cd
Retuen a reasonable response to client.

As we use the 'account_autocreate' for auto-create accounts, we send a
request to HEAD an account which is just deleted(the account-reaper not
reclaim it immediately), the proxy-server will return 'HTTP 500 Internal
Server Error'.
In my opinion, this is unreasonable, so I change the code in swift/proxy
/server.py for returning a reasonable response. I modified the code in
POST, GETorHEAD and account_info. At last, I modified some code in the
unittest test/unit/proxy/test_server.py.

Change-Id: Ib057b387c9da073d707ffae49ead54206a8fb7dd
Fixes: Bug #1037889
Update docs for new ring serialization.

The Admin Guide now contains information about the ring serialization
change (and importantly, how to downgrade, if necessary).

Also added container-server conf var, "allow_versions" to the Deployment
Guide.

Also changed description of proxy-server conf var,
"max_containers_whitelist" to say it contains "account names" not
"account hashes".

Change-Id: Ib23c6118cc5195cc04765afd28e442e4c735f0d4
Merge "Updated probe tests"
Add nosehtmloutput as a test dependency.

Adding nosehtmloutput as a test dependency allows nose to output its
results to an html file. This will be used by Jenkins to save logs on
a different server.

Change-Id: I6bd4736e8cc4132db65bfe8cdfe1f3a388369fa8
Merge "Fixed bug where expirer would get confused by..."
A couple minor tweaks to the SAIO guide.

Added "-p" to mkdir cmds in a "set -e" script (/etc/rc.local).

/var/log/swift gets owned by the group "adm", but isn't group writeable
by default, so the find/rm in "resetswift" fails.  So I added a chmod to
add group-write for that directory tree.

Added commands for installing tox.

Change-Id: Ic141491272a83a9233cd6a6a70c3ef74411f3f51
Merge "A few misc. code cleanups."
Merge "A couple minor tweaks to the SAIO guide."
Merge "Add nosehtmloutput as a test dependency."
Fix PEP8 issues.

Fix a PEP8 issues in doc/source/conf.py to make the code pretty.

Change-Id: I71d36ebb59d44ba145ca13e4f4827139f0466d78
Merge "Update docs for new ring serialization."
Merge "Fix PEP8 issues."
make obj replicator locking more optimistic

Basically, do all hashing in the replicator without a lock, then
lock briefly to rewrite the hashes file. Retry if someone else has
modified the hashes file in the mean time (which should be rare).

Also, a little refactoring.

Change-Id: I6257a53808d14b567bde70d2d18a9c58cb1e415a
Add device name to *-replicator.removes for DBs

To tell when replication for a device has finished, it's important to
know when the replicator is removing objects.  This was previously
handled for the object-replicator
(object-replicator.partition.delete.count.<device> and
object-replicator.partition.update.count.<device> metrics) but not the
account and container replicators.

This patch extends the existing DB removal count metrics to make them
per-device.  The new metrics are:
 account-replicator.removes.<device>
 container-replicator.removes.<device>

There's also a bonus refactoring and increased test coverage of the DB
replicator code.

Change-Id: I2067317d4a5f8ad2a496834147954bdcdfc541c1
Merge "Retuen a reasonable response to client."
Merge "Add device name to *-replicator.removes for DBs"