Implementation of replication servers

Support separate replication ip address:
- Added new function in utils. This function provides ability
  to select separate IP address for replication service.
- Db_replicator and object replicators were changed.
  Replication process uses new function now.

Replication network parameters:
- Replication network fields (replication_ip, replication_port)
  support was added to device dictionary in swift-ring-builder script.
- Changes were made to support new fields in search, show and set_info
  functions.

Implementation of replication servers:
- Separate replication servers use the same code as normal replication
  servers, but with replication_server parameter = True.  When using a
  separate replication network, the non-replication servers set
  replication_server = False.  When there is no separate replication
  network (the default case), replication_server is not included in the config.

DocImpact
Change-Id: Ie9af5bdcdf9241c355e36053ca4adfe49dc35bd0
Implements: blueprint dedicated-replication-network
Add an explicit unit test for handling content-length: 0

Change-Id: I3568d4dc1900e6ddb4860589ca6a7b7039cc8c2d
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Fix rebalance for zero weighted devices.

If we set device's weight to zero, currently balance will be set
special value(999.99) until zero weighted device return all
its partitions. So we cannot check balance has changed.
Thus we need to check balance or last_balance is special value.

Change-Id: I5b7db8b8e48db0c4771c51a764bda689869817d5
Fixes: bug #1171731
conf.d support

Allow Swift daemons and servers to optionally accept a directory as the
configuration parameter.  Directory based configuration leverages
ConfigParser's native multi-file support.  Files ending in '.conf' in the
given directory are parsed in lexicographical order.  Filenames starting with
'.' are ignored.  A mixture of file and directory configuration paths is not
supported - if the configuration path is a file behavior is unchanged.

 * update swift-init to search for conf.d paths when building servers
    (e.g. /etc/swift/proxy-server.conf.d/)
 * new script swift-config can be used to inspect the cumulative configuration
 * pull a little bit of code out of run_wsgi and test separately
 * fix example config bug for the proxy servers client_disconnect option
 * added section on directory based configuration to deployment guide

DocImpact

Implements: blueprint confd
Change-Id: I89b0f48e538117f28590cf6698401f74ef58003b
Update to flake8 instead of pep8.

Change-Id: I3d4a31111c3044da06611405ce80f208ef8a0ce3
Adds chunk_size to CompressingFileReader.

Before when iterating using this object, the call to the underlying file object
was made without a chunk_size.  This is bad because this would cause the entire
contents of the file to be read.

Change-Id: I9956e5d2d693a6260252fff331d4f78f70179a6c
Merge "Update to flake8 instead of pep8."
Rename "file" to "disk_file"

Broke this out into a separate commit since adding five characters
caused a few lines to get re-wrapped, adding more diffs.

See John's comment from Patch Set #3 for the motivation for this
change in review https://review.openstack.org/27149.

Change-Id: I1edf2cb468dcda0b781569161ada6e4016c1141c
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Merge "conf.d support"
Merge "Add an explicit unit test for handling content-length: 0"
Skip cname_lookup when host is an IP address

cname_lookup cannot resolve IP addresses as CNAME records, and therefore
should not attempt to resolve the host in that case. The middleware is
skipped when the host is an IP address.

Change-Id: I6961ec205e771116ace1ebcb8c088f3116eb38f0
Fixes: bug #1172289
Skip IPv6 addresses in cname_lookup middleware.

It already skips IPv4 addresses, and since IPv6 is the future of the
Internet*, we should probably do the right thing with those too.

* IPv6: Just two years away for over fifteen years!

Change-Id: I54f1db4e936fd38d05ac8b5c709efba76525b9d2
Merge "Fix rebalance for zero weighted devices."
Merge "Adds chunk_size to CompressingFileReader."
Doc changes for new pip-requirements

dnspython was added recently to the list of requirments.

This patch completes the change:
1. The new requirement is documented
2. The need for tox --recreate is documented
(assuming there is no way to automate 2 above)

Change-Id: I49f0cb4b01b79dd7cf4329984a2f8416378ebe19
Fixes: Bug #1176627
Fixed issue with use of delimiter in container queries.

Currently if you have a container with objects named
x/y and x0 and you use delimiter=/ in your query, x0
will be excluded from the results. This patch resolves
this problem.

To replicate the original issue, create a container
called 'test' with the following objects:

x/y
x0
x1

Then do a GET with 'accnt/test?delimiter=/' and the
following is returned:

x/
x1

i.e. x0 is missing from output.

Change-Id: If196e3075612b121ef8da4a9128167d00a248c27
have tempurl allow OPTIONS requests

Change-Id: I431cac1e9818a335ff8d20288ab6acf39d6b6d5e
Merge "have tempurl allow OPTIONS requests"
Allow 2 TempURL keys per account.

This allows users to rotate their TempURL keys without invalidating
all existing signed URLs. This is handy if you have multiple systems
generating signed URLs, but you want to change your keys for some
reason (e.g. keys compromised, company policy, general paranoia).

Both the first and second keys are optional, so existing accounts'
signed URLs will continue to work as before.

This commit does change the memcache key used to store the fetched
TempURL keys. This is because we were storing the old key as a string
in memcached, but the new one is a list of keys. Since the key cache
lifetime here is only 60 seconds, it doesn't seem like too big a deal
to completely flush the TempURL cache.

Also, this commit adds caching of a negative TempURL result. If the
account HEAD reveals no TempURL keys at all, that result is now stored
for 60 seconds the same way that a positive result would be.

Change-Id: I40a02bd607283fbce11aa52a9bb8a5846ab17f5e
Refactor Bulk middleware to handle long running requests

Change-Id: I8ea0ff86518d453597faae44ec3918298e2d5147