Allow empty reseller prefixes in domain_remap

Change-Id: I2013e4686fce1c06626ec974f8d721721e6ca6e7
Changed several files to be Python3 friendly

The print syntax has been changed to be Python3 compatible.

Change-Id: Id3fcbab390f08dcb3380cfda16d7b820b03ef683
Fix object auditor recon and logging

1. Nothing is logged until at least one audit needs more than log_time.
   If the audit runtime never exceeds this value (which is 3600 seconds
   by default) nothing is logged and the recon entry is never updated.
   It happens especially on very fast disks with a low usage and/or if
   only a few disks are audited (for example, using the --devices
   parameter on the command line).

   This patch changes this to log and update the recon cache entry
   at least one time after the first device audit.

2. If device_dirs is set the recon entry will be deleted after all
    devices are audited.

Change-Id: Ifa504d21389b3a5f7eaf914b19d6e26543dac121
Add X-Backend-Timestamp on more object server responses

It's particularly interesting on writes (PUT, POST, DELETE) where the
current on-disk timestamp would prevent the object server from serving
the incoming request and returns 409 Conflict.

The FakeConn has also been updated to respond in kind for 409's on
expect and all responses generaly just cause it's good to keep fakes in
line with the reals - not that I expected any existing tests to break
because of the new headers.

Change-Id: Iac6fbd2f872a9521bb2db84a333365b69f54fb6c
Increase Account Auditor Test Code Coverage

Some simple additional test code to bring coverage from 82% to 99%

Change-Id: I561bc0a043f3451bb0a84cad274fedc907e61ee8
Remove fake _get_part and use the real thing

The get_part method is fast and stable given a consistent hash path
suffix/prefix, so there's no absolute requirement for the fake
implementation other than convenience.  OTOH, removing the fake
implementation and fixing the tests that were relying on it should make
it easier to write better tests going forward and harder to hide bugs
that don't show up when using the fakes.

There may be some overhead when writing new tests that use the ring if
you're making assertions on partitions or paths, but with a part power
of zero it's normally trivially obvious when a 1 needs to be a 0 or vice
versa.  Or you can just drop the assertions around the parts you were
faking anyway.

Change-Id: I8bfc388a04eff6491038991cdfd7686c9d961545
Setup localization properly

To start translation of swift, we need to initially import the
translation file - and place it at the proper place so that
the usual CI scripts can handle it.

The proper place is for all python projects
$PROJECT/locale/$PROJECT.pot, so move locale/$PROJECT.pot to the new
location and regenerate the file.

Update setup.cfg with the new paths.

Further imports will be done by the OpenStack Proposal bot.

Change-Id: Ide4da91f2af71db529f4a06d6b1e30ba79883506
Partial-Bug: #608725
Closes-Bug: #1082805
Allow HEAD for POST tempurls

HEAD requests are already allowed for GET and PUT tempurls; this
commit adds that for POST tempurls. Since POST replaces all the
object's metadata, it's quite useful to be able to HEAD the object in
order to fetch the old metadata and do a client-side merge of the new
metadata (like with normal, token-authenticated POST requests).

Change-Id: I603c7822cd27f0e304fd27024f83f95114eb0aef
Update doc for list_endpoints v2 API

Adds some v2 doc to list_endpoints module.

Also adds LE to list of swift sources in logs.rst

Change-Id: I6ccbcf677b12f5aa7db0f9d81e43ce5ca990cc57
Make swift-form-signature testable

Moved the body of bin/swift-form-signature into
swift/cli/form_signature.py, like was done with swift-ring-builder and
others. Added a couple basic tests; there's not 100% coverage, but
it's better than the 0% coverage we had before.

It's almost a straight forklift, but I changed exit() calls to return
statements.

Change-Id: Ie2f702c070da24d9cdface83b9e838e9e2965085
Fix "Handoff requested (N)" log spam

A long, long time ago, on a GET request, the proxy would go look on 3*
nodes for the requested thing. If one of the primary nodes was
error-limited, it'd look on two primaries and a handoff. Since this
indicated some failure somewhere, the proxy would emit a warning:
"Handoff requested (1)". If two primaries were down, there'd be a
second message "Handoff requested (2)", and so on.

Some StatsD messages were emitted too.

A somewhat shorter time ago (commit d79a67eb), the proxy started
looking into handoffs if it got 404s from the primaries. While this
was a good idea, it resulted lots of "Handoff requested (N)" log spam;
you'd see these messages on every single 404. Also, the StatsD
handoff_count and handoff_all_count metrics shot way up and turned
into noise.

This commit restores the original intent (and usefulness) of the log
messages and StatsD metrics: if the proxy only looks at the normal
number of handoff nodes, nothing is logged. However, if a primary is
down, then the message "Handoff requested (1)" will be logged,
indicating that the proxy looked at one more handoff than it normally
would, and this happened because a primary node was error-limited.

Closes-Bug: 1297214

* or whatever the replica count was

Change-Id: If1b77c18c880b096e8ab1df3008db40ce313835d
Avoid usage of insecure mktemp() function

This patch eliminates the use of the deprecated and insecure
tempfile.mktemp() function.  It has been replaced with secure
alternatives where temporary files are actually required.

Change-Id: I0a13d6d44cd1abc4b66fa33f39eea407617a01d5
SecurityImpact
Closes-bug: #1348869
Merge "Fix "Handoff requested (N)" log spam"
Multithread optimization for object updater

This change makes the object updater work the same
as the container updater, which is that it now
spawns one thread for each node that it needs to
update. This should help optimize its
performance/efficiency.

Change-Id: If740b8c67c0d953e4b510913bd8594570d868da6
Put proxy-logging in object expirer's sample config

It's generally better to have logs for something than to not have
logs. This way, the object expirer (if using the sample config as a
starting point) will log what it does.

Note that the container reconciler's sample config already contains
proxy-logging, as does the proxy server's. The object expirer is the
odd man out.

Change-Id: I32aac99131746501820319b94405440c1934a694
Fix swift-dispersion-populate bug when using SimpleClient class

Commit 7fcbbebb changed the internal client a little bit and removed the
class variable self.attempts in SimpleClient. This is currently still
needed in swift-dispersion-populate to track the number of retries, thus
re-adding it again.

Closes-Bug: 1351323
Change-Id: I98ba6441607158b34708b2dbb2d1a63788681d63
Enable object system metadata on PUTs

This patch takes a first step towards support
for object system metadata by enabling headers
in the x-object-sysmeta- namespace to be
persisted when objects are PUT. This should be
useful for other pending patches such as on
demand migration and server side encryption
(https://review.openstack.org/#/c/64430/ and
https://review.openstack.org/#/c/76578/1).

The x-object-sysmeta- namespace is already
reserved/protected by the gatekeeper and
passed through the proxy. This patch modifies
the object server to persist these headers
alongside user metadata when an object is
PUT.

This patch will preserve existing object
system metadata and ignore any new system
metadata when handling object POSTs,
including POST-as-copy operations. Support
for modification of object system metadata
with a POST request requires further work
as discussed in the blueprint.

This patch will preserve existing object
system metadata and update it with new
system metadata when copying an object.

A new probe test is added which makes use of
the BrainSplitter class that has been moved
from test_container_merge_policy_index.py to
a new module brain.py.

blueprint object-system-metadata

Change-Id: If716bc15730b7322266ebff4ab8dd31e78e4b962
fix expirer bug with unicode container listings

If the container names in the expirer's account are returned as
unicode strings (as is the case with some json libraries), the
expirer compared eg u'1' == '1', which is problematic. This patch
ensures that the unicode is coerced to ascii so the comparison
is correct.

Change-Id: I72b322e7513f7da32e8dc75c6bf0e7e016948c88
Merge "Enable object system metadata on PUTs"
Merge "fix expirer bug with unicode container listings"