replace string format arguments with function parameters

There are files containing string format arguments inside
logging messages. Using logging function parameters should
be preferred.

Change-Id: I1c2a4f43463932e85c75d576241d0c0455c38492
Container sync no longer sending swift_bytes value

Container sync had a bug where it'd send out the trailing
"; swift_bytes=xxx" part of the content-type header. That trailing part
is just for internal cluster usage by SLO. Since that needed to be
stripped in two places now, I separated it out to a function that both
spots call.

Change-Id: Ibd6035d7a6b78205344bcc9d98bc1b7a9d463427
Add debug logging to container sync requests

Also fixed a bug where SimpleClient would send ?format=json for object
requests, which is not necessary.

Change-Id: If06a7dcebc9de2d7c8b28a046d60b902dae821c1
Remove unused variable

The tempurl functional tests would fail if you put tempurl.methods in
the proxy server's disallowed_sections. The test assumed that the
presence of the key 'tempurl' implied the presence of the subkey
'methods', but since 9cbf8a3 landed, operators can remove arbitrary
things from /info.

Normally I'd just change x['methods'] to x.get('methods', []), but it
turns out this was setting up a class variable that nobody ever looked
out again, so removing the code also works.

Change-Id: Ie899d146bc6fff81a5fae77815897244e8ec6bec
isolate representation from content in GET request

Container server representation of data (received from backend) is isolated
into `create_listing()` call. GET() method will call `create_listing()`
when it's ready to create a serialized object listing. It will allow us to
customize that representation: get different listing, if backend supports
more interesting queries, for example.

Change-Id: I4fa03357d3d9e697bc3e1fb5da42d7572c6ae9b3
replaced e.message with a bare "e" object ref

Change-Id: Ia4fa7ab0860532a71a531c13ee2125666547136e
Fixes typo in man page

There is a simple typo in the man page of proxy-server.conf,
"client_timeout" is written as "client_timeoutt".
This commit fixes it.

Closes-Bug: #1326237
Change-Id: I98777f523906e4ed625de8f20a96979ea627aa1f
Fix file uploads > 2 GiB in formpost middleware

Formpost middleware fails to upload files larger then 2 GiB due to
an Overflow error. The reason is that the readline() will use a
readline(int(content_length)) later on and fail if it is larger than
2GiB.  Since it is not required to read the whole content into memory
to detect the boundary only read the amount of required bytes.

The underlying error is located in Python 2.7 and is related to
cStringIO: http://bugs.python.org/issue7358

Closes-Bug: #1326429
Change-Id: I196edda647921c2691d278cebd1cca80ebd360f2
xLO bug with auth tokens expiring during download.

Just put SLO and DLO after any auth middleware. This works because when
the request goes through that middleware in the pipeline the
authentication takes place: validation of the token, setting up who the
user is, and setting the authorization call back. Each subrequest made
for the segments will be subjected to that authorization call back which
verifies the user has access to the individual segments.

To get this to work with keystone, the keystone identity is set up
during __call__ and applied to the authorize function using a
functools.partial. When the authorize function is later called from the
environ by the proxy server the idenity that was set up when the request
passed through the auth middleware is used, not what can be pulled out
of the possibly altered state of the request's environment.

DocImpact
fixes bug: 1315133

Change-Id: I7827dd2d9dfbb3c6424773fb2891355d47e372ba
Merge "Container sync no longer sending swift_bytes value"
Merge "replace string format arguments with function parameters"
Merge "xLO bug with auth tokens expiring during download."
Merge "Fix file uploads > 2 GiB in formpost middleware"
Merge "Add debug logging to container sync requests"
Merge "isolate representation from content in GET request"
Merge "Fixes typo in man page"
Merge "replaced e.message with a bare "e" object ref"
Merge "Remove unused variable"
properly quote www-authenticate header value

HTTP header values should be quoted. Since the WWW-Authenticate
header value contains user-supplied strings, it's important to
ensure it's properly quoted to ensure the integrity of the protocol.

Previous to this patch, the URL was unquoted and then the unquoted
value was returned in the header. This patch re-quotes the value
when it is set on the response.

This is filed as CVS-2014-3497

Fixes bug 1327414

Change-Id: If8bd8842f2ce821756e9b4461a18a8ac8d42fb8c
Add Storage Policy Support

The basic idea here is to replace the use of a single object ring in
the Application class with a collection of object rings. The
collection includes not only the Ring object itself but the policy
name associated with it, the filename for the .gz and any other
metadata associated with the policy that may be needed. When
containers are created, a policy (thus a specific obj ring) is
selected allowing apps to specify policy at container creation time
and leverage policies simply by using different containers for object
operations.

The policy collection is based off of info in the swift.conf file.
The format of the sections in the .conf file is as follows:

swift.conf format:

    [storage-policy:0]
    name = chicken

    [storage-policy:1]
    name = turkey
    default = yes

With the above format:

- Policy 0 will always be used for access to existing containers
  without the policy specified. The ring name for policy 0 is always
  'object', assuring backwards compatiblity. The parser will always
  create a policy 0 even if not specified

- The policy with 'default=yes' is the one used for new container
  creation. This allows the admin to specify which policy is used without
  forcing the application to add the metadata.

This commit simply introduces storage policies and the loading
thereof; nobody's using it yet. That will follow in subsequent
commits.

Expose storage policies in /info

DocImpact
Implements: blueprint storage-policies
Change-Id: Ica05f41ecf3adb3648cc9182f11f1c8c5c678985