Fix object auditor recon and logging

1. Nothing is logged until at least one audit needs more than log_time.
   If the audit runtime never exceeds this value (which is 3600 seconds
   by default) nothing is logged and the recon entry is never updated.
   It happens especially on very fast disks with a low usage and/or if
   only a few disks are audited (for example, using the --devices
   parameter on the command line).

   This patch changes this to log and update the recon cache entry
   at least one time after the first device audit.

2. If device_dirs is set the recon entry will be deleted after all
    devices are audited.

Change-Id: Ifa504d21389b3a5f7eaf914b19d6e26543dac121
Increase Account Auditor Test Code Coverage

Some simple additional test code to bring coverage from 82% to 99%

Change-Id: I561bc0a043f3451bb0a84cad274fedc907e61ee8
Add a env var to use in-memory obj server in func

Add an environment variable to enable the use of the in-memory object
server during in-process functional test runs.

It might be worth-while to just run under both object servers in-tree,
but this at least enables it, without having to figure out how to make
two test runs in two different environments.

DocImpact

Change-Id: Id76b008e1f273c639ae61550affddc32c5d7c419
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Add POST and DELETE to tempurl default methods

The tempurl middleware supports any configured HTTP methods, but the
default set was only GET, PUT, and HEAD, so cluster operators had to
take action to enable POST and DELETE. This commit changes the
defaults to include POST and DELETE.

Note that this doesn't affect any existing temporary URLs at all; the
method is baked into the signature (temp_url_sig query param), so no
new access is granted to a holder of a temporary URL by this
change. It simply gives more flexibility to creators of temporary
URLs.

Change-Id: I5bc15bbd2968ab7bedcd7c0df10f2ec825537191
Update tempurl docstring with methods config option

Adds a description of the methods config option to the
docstring.

Also fixes description of swift.source value and a
couple of other typos.

Change-Id: If3489087df020536ff663ffe4e249c300ea2d506
fix to handle object-expiration headers in one method

There was some duplication of code in both POST and PUT
methods to handle object-expiration headers.
A method was created to remove this duplication,
which should help with maintainability of code.

Change-Id: I85cc4a7b0d688760c97598d80b9e9a39288c5f34
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Add timestamp checking in AccountBroker.is_status_deleted

Account-reaper works only at account-server with the first replica, and reaps
account with "deleted" status.

On the other hand, account-replicator doesn't replicate the status, only
replicates *_timestamp.
When swift fails to delete the first account replica, account-reaper never
reaps the account, because the first replica never gets marked as "deleted".

This patch adds a timestamp checking into is_status_deleted method, and
account-reaper will start to reap the account after account-replicator
replicates *_timestamp.

Change-Id: I75e3f15ad217a71b4fd39552cf6db2957597efca
Closes-Bug: #1304755
Catch permissions errors when writing StatsD packets

Closes-Bug: #1183152

Change-Id: I4b2c6e947241c987779a385fdff270d037470a57
Merge "Add POST and DELETE to tempurl default methods"
Merge "Catch permissions errors when writing StatsD packets"
Sleep for longer at a time in lock_path.

When lock_path is called and the lock goes for the whole 10 seconds,
the flock is called 1000 times. With this patch, the short 0.01 sleep
is used for the first 1% of the total lock time and then 1% of the
total lock time is used.

Change-Id: Ibed6bdb49bddcdb868742c41f86d2482a7edfd29
Merge "Increase Account Auditor Test Code Coverage"
Merge "Add a env var to use in-memory obj server in func"
Shard expiring object container

All the expiring objects for a given X-Delete-At are funnelled into the
same expiring object container- this can act as a bottleneck.

Change-Id: I288a177a7ae3e213c727a2a81fa76d4ef9cf7eb3
Merge "Add timestamp checking in AccountBroker.is_status_deleted"
Merge "Sleep for longer at a time in lock_path."
Merge "Fix object auditor recon and logging"
add the account management config values to swift info

added account_autocreate and allow_account_management to
the /info endpoint

Change-Id: I4b239c9cefb728c3c93bf75cad065c72edf2fc0a
Merge "Update tempurl docstring with methods config option"
Merge "Shard expiring object container"