Updated probe tests

The probe tests were woefully out of date with all the changes that
have ocurred since they were written. I've updated most of them and
removed some that are hopeless outdated.

I also greatly improved the timing issues (hopefully completely
solved them? I ran them 25 times with no problems) and made them pep8
1.3.1 safe.

Change-Id: I8e9dbb6e7d6e04e293843b1dce1ded99d84e0348
To simulate the real workloads, the objects to be uploaded could be created in the random sizes, which are bounded (lower_object_size and upper_object_size) by the user inputs.

Change-Id: I64647c5d613a0794f0807886a4833cd5c31c0c5e
Use custom encoding for RingData, not pickle.

Serialize RingData in a versioned, custom format which is a combination
of a JSON-encoded header and .tostring() dumps of the
replica2part2dev_id arrays.  This format deserializes hundreds of times
faster than rings serialized with Python 2.7's pickle (a significant
performance regression for ring loading between Python 2.6 and Python
2.7).  Fixes bug 1031954.

swift.common.ring.ring.RingData is now responsible for serialization and
deserialization of its data via a new load() class method and save()
object method.  The new implementation is backward-compatible; if a ring
does not begin with a new-style magic string, it is assumed to be an
old-style pickle-dumped ring and is handled as before.  So new Swift
code can read old rings, but old Swift code will not be able to read
newly-serialized rings. THIS SHOULD BE MENTIONED PROMINENTLY IN THE
RELEASE NOTES.

I didn't want to bite of more than necessary, so I didn't mess with
builder file serialization.

Change-Id: I799b9a4c894d54fb16592443904ac055b2638e2d
Fixed bug where expirer would get confused by...

Fixed bug where expirer would get confused by odd deletion times.
Since this has already rolled out, I just capped things at ten 9s, or
Sat Nov 20 17:46:39 2286. I can't wait for the Y2286 world panic. :/

Change-Id: Iba10963faa344a418a1fa573d5c85f4ff864b574
update object versions docs

Change-Id: I00bf2e20c2683d0440ca425461e0ab87197ee720
handle exception correctly in _make_app_iter_reader

_make_app_iter_reader uses a queue to pass read data to
_make_app_iter, and uses an empty string as a terminator.  The problem
is that it pushes the same terminator even if it fails to reads the
data.  If an exception happens before pushing all data to the queue,
_make_app_iter exits without yielding all data.  In that case,
proxy-server stops in the middle of sending data, and a client waits
for all data to come infinitely.

This patch uses a boolean value for the terminator instead of an empty
string to distinguish the results, and raises an exception in the
error cases.  Fixes bug 890888.

Change-Id: Ie4d48025843aa1bb600446da6f4fc49f82a440bf
Merge "Use custom encoding for RingData, not pickle."
ring.py: Uses simplejson if avail and pep8 1.3.1

Change-Id: I1f91a82da064c9f0e64779f4fa8951962678831b
Cleanup after LockTimeout

- If lock_file encounters a LockTimeout a fd can be left behind
- Fixes bug 1038191

Change-Id: Ib3f45d18aeeec91eedf0b4f0492664d33b4f57fa
Merge "update object versions docs"
Merge "To simulate the real workloads, the objects to be uploaded could be created in the random sizes, which are bounded (lower_object_size and upper_object_size) by the user inputs."
Merge "handle exception correctly in _make_app_iter_reader"
A few misc. code cleanups.

Fixed a couple config gets which weren't using TRUE_VALUES (hopefully
the last ones?).

Removed a deprecated parameter from
swift.common.db.ContainerBroker.list_objects_iter() a "TODO" which dates
back to the original import of Swift into github.

Also made AccountBroker.reclaim()'s docstring match its arg names.

Change-Id: I646bdd00a3693763b55a50ba210c7ed2ba1df4cd
Retuen a reasonable response to client.

As we use the 'account_autocreate' for auto-create accounts, we send a
request to HEAD an account which is just deleted(the account-reaper not
reclaim it immediately), the proxy-server will return 'HTTP 500 Internal
Server Error'.
In my opinion, this is unreasonable, so I change the code in swift/proxy
/server.py for returning a reasonable response. I modified the code in
POST, GETorHEAD and account_info. At last, I modified some code in the
unittest test/unit/proxy/test_server.py.

Change-Id: Ib057b387c9da073d707ffae49ead54206a8fb7dd
Fixes: Bug #1037889
Update docs for new ring serialization.

The Admin Guide now contains information about the ring serialization
change (and importantly, how to downgrade, if necessary).

Also added container-server conf var, "allow_versions" to the Deployment
Guide.

Also changed description of proxy-server conf var,
"max_containers_whitelist" to say it contains "account names" not
"account hashes".

Change-Id: Ib23c6118cc5195cc04765afd28e442e4c735f0d4
Merge "Updated probe tests"
Add nosehtmloutput as a test dependency.

Adding nosehtmloutput as a test dependency allows nose to output its
results to an html file. This will be used by Jenkins to save logs on
a different server.

Change-Id: I6bd4736e8cc4132db65bfe8cdfe1f3a388369fa8
Merge "Fixed bug where expirer would get confused by..."
A couple minor tweaks to the SAIO guide.

Added "-p" to mkdir cmds in a "set -e" script (/etc/rc.local).

/var/log/swift gets owned by the group "adm", but isn't group writeable
by default, so the find/rm in "resetswift" fails.  So I added a chmod to
add group-write for that directory tree.

Added commands for installing tox.

Change-Id: Ic141491272a83a9233cd6a6a70c3ef74411f3f51
Merge "A few misc. code cleanups."