Plumb insecure flag to functests

The problem arises when running functests against pre-existing
clusters configured with self-signed certificates. As far as
security goes, doing that is rather useless, but it happens.
Such testing worked previously, but with the switch to Requests
in the client, we must plumb "insecure" flag. There's no other
way to do it in Requests, no environment variable, anything.

Change-Id: I94a61b60de8ee0529652c8c4a1ab9894c7cdb589
Add --quoted option to swift-temp-url

If you have a path with special characters it may be easier to hand them to
swift-temp-url prequoted than try to escape them on the command line.  By the
time common.middleware.tempurl gets ahold of the path it's unquoted so we do
the same before calculating the hmac but still use the pre-quoted path output
to the commandline.

Change-Id: Ia1a9666e487b1e70e4db7cd597bc6a1027e3e918
Print pipeline names in the reported pipeline

When current code modifies the pipeline, it prints the entry point
names instead of the names used to construct the pipeline. This is
inconvenient because a sysadmin cannot copy and paste from the log.

We already save the pipeline name into contexts in most cases, so
the fix simply reuses that to provide friendly names.

Fixes bug: 1311802

Change-Id: Ic76baf1360cd521f140fa1980029ccbce58f1717
replace string format arguments with function parameters

There are files containing string format arguments inside
logging messages. Using logging function parameters should
be preferred.

Change-Id: I1c2a4f43463932e85c75d576241d0c0455c38492
Container sync no longer sending swift_bytes value

Container sync had a bug where it'd send out the trailing
"; swift_bytes=xxx" part of the content-type header. That trailing part
is just for internal cluster usage by SLO. Since that needed to be
stripped in two places now, I separated it out to a function that both
spots call.

Change-Id: Ibd6035d7a6b78205344bcc9d98bc1b7a9d463427
Add ability to remove subsections from /info

Change-Id: Ic881065962cf5f69f7a5b64f6e38d9e6e1f8fd18
Fixed a handful of typos

Change-Id: I8cb2de27818332be587df7321ef567f8399910a9
Merge "Print pipeline names in the reported pipeline"
Add Ratelimit parameters to /info

We previously registered with no parameters, added parms so they
are displayed as follows (example):

"ratelimit": {"max_sleep_time_seconds": 60.0,
"container_listing_ratelimits": [[0, 100.0], [10, 50.0], [50, 20.0]],
"container_ratelimits": [[0, 100.0], [10, 50.0], [50, 20.0]],
"account_ratelimit": 1.0}

Note that not all parameters are exposed (intentionally) via /info

Change-Id: I36c7ef15af17e3eb8ebb93429035bd06d089a945
Closes-Bug: 1308989
Add debug logging to container sync requests

Also fixed a bug where SimpleClient would send ?format=json for object
requests, which is not necessary.

Change-Id: If06a7dcebc9de2d7c8b28a046d60b902dae821c1
Merge "Add ability to remove subsections from /info"
Merge "Plumb insecure flag to functests"
Merge "Add Ratelimit parameters to /info"
Remove unused variable

The tempurl functional tests would fail if you put tempurl.methods in
the proxy server's disallowed_sections. The test assumed that the
presence of the key 'tempurl' implied the presence of the subkey
'methods', but since 9cbf8a3 landed, operators can remove arbitrary
things from /info.

Normally I'd just change x['methods'] to x.get('methods', []), but it
turns out this was setting up a class variable that nobody ever looked
out again, so removing the code also works.

Change-Id: Ie899d146bc6fff81a5fae77815897244e8ec6bec
isolate representation from content in GET request

Container server representation of data (received from backend) is isolated
into `create_listing()` call. GET() method will call `create_listing()`
when it's ready to create a serialized object listing. It will allow us to
customize that representation: get different listing, if backend supports
more interesting queries, for example.

Change-Id: I4fa03357d3d9e697bc3e1fb5da42d7572c6ae9b3
Merge "Add --quoted option to swift-temp-url"
replaced e.message with a bare "e" object ref

Change-Id: Ia4fa7ab0860532a71a531c13ee2125666547136e
Fixes typo in man page

There is a simple typo in the man page of proxy-server.conf,
"client_timeout" is written as "client_timeoutt".
This commit fixes it.

Closes-Bug: #1326237
Change-Id: I98777f523906e4ed625de8f20a96979ea627aa1f
Fix file uploads > 2 GiB in formpost middleware

Formpost middleware fails to upload files larger then 2 GiB due to
an Overflow error. The reason is that the readline() will use a
readline(int(content_length)) later on and fail if it is larger than
2GiB.  Since it is not required to read the whole content into memory
to detect the boundary only read the amount of required bytes.

The underlying error is located in Python 2.7 and is related to
cStringIO: http://bugs.python.org/issue7358

Closes-Bug: #1326429
Change-Id: I196edda647921c2691d278cebd1cca80ebd360f2
xLO bug with auth tokens expiring during download.

Just put SLO and DLO after any auth middleware. This works because when
the request goes through that middleware in the pipeline the
authentication takes place: validation of the token, setting up who the
user is, and setting the authorization call back. Each subrequest made
for the segments will be subjected to that authorization call back which
verifies the user has access to the individual segments.

To get this to work with keystone, the keystone identity is set up
during __call__ and applied to the authorize function using a
functools.partial. When the authorize function is later called from the
environ by the proxy server the idenity that was set up when the request
passed through the auth middleware is used, not what can be pulled out
of the possibly altered state of the request's environment.

DocImpact
fixes bug: 1315133

Change-Id: I7827dd2d9dfbb3c6424773fb2891355d47e372ba