Prevent early auth deny in tempauth when using swift3 middleware.

When tempauth is used together with swift3 and keystone, groups are
empty and tempauth denies the request too early without a chance for
keystone to authenticate the request.

Change-Id: I21d9b22ecbd18a5f1fba901abd94221a332c45ea
Closes-Bug: 1244545
allow bare excepts in flake8

Change-Id: I4751f9f65712960019f9a303b7d7fe017c6e85c7
Some functional tests for static large objects

There's some sort-of-hacky code in there to detect SLO support in
order to skip tests when SLO is off so that the functests won't fail
on older clusters.

Change-Id: I6ad5974a0db7213747b0f4497d08ffc706d3f220
Reorganize SLO unit tests

This brings some sanity to the SLO test app (the thing the middleware
wraps in the unit tests) as well as splits things into multiple test
classes.

This is part of the effort to move all SLO functionality to
middleware, but it's a separate commit to prove that these test
changes don't harm anything when running against the old code.

Change-Id: I52a16f15a80dfaf9b3c595b0e634d52f418caf6c
in case you lose your builder backups

Change-Id: Ica555be2be492c3ec5fdeab738058ff35989a603
workaround probetest race from early response

Change-Id: I594633887c86fc2212850409a37ee2257633a23c
Add a DebugLogger for wsgi server tests

Change-Id: Ifd2528be443ba3879bf4921f6c5f4ef31f29044b
test: improve db_replicator coverage

This patch adds a test for ReplicatorRpc.complete_rsync()
and complete extract_device() coverage.

test_extract_device:
  test the case the parameter is invalid

test_complete_rsync_with_bad_input:
  ensure the use of invalid parameters return a 404 erro

test_complete_rsync:
  validate the returned code in case of success

Change-Id: I59e0d26a1efe59d8beff1e81c2a7edc6de0872e9
Added discoverable capabilities.

Swift can now optionally be configured to allow requests to '/info',
providing information about the swift cluster.  Additionally a HMAC
signed requests to
'/info?swiftinfo_sig=<sign>&swiftinfo_expires=<expires>' can be
configured allowing privileged access to more sensitive information
not meant to be public.

DocImpact
Change-Id: I2379360fbfe3d9e9e8b25f1dc34517d199574495
Implements: blueprint capabilities
Closes-Bug: #1245694
Return an exit code for configuration errors

Red Hat's QA noticed that in case of the infamous "xattr>=0.4"
error, swift-init exits with a zero error code, which confuses
startup scripts (not Systemd though -- that one knows exactly
if processes run or not).

The easiest fix is to return the error code like Guido's blog
post suggested.

Change-Id: I7badd8742375a7cb2aa5606277316477b8083f8d
Fixes: rhbz#1020480
Use a more portable errno in tests.

ELIBBAD doesn't exist on OS X. The exact value we use here isn't
important, so use something more portable.

Change-Id: Id03dc1773f416a94bbd14ad31b2b2a70f16b9a51
Use upstream patched Pool.get

This works around an eventlet bug in eventlet 0.9.16.
This version properly keeps track of pool size accounting, and
therefore doesn't let the pool grow without bound. This patched
version is the result of commit
f5e5b2bda7b442f0262ee1084deefcc5a1cc0694 in eventlet and is
documented at https://bitbucket.org/eventlet/eventlet/issue/91

This patch includes full test coverage of the back-ported code, even
when the actually-installed eventlet is newer.

This fixes bug #1254119

Change-Id: I075bb5e40e08571d52fe17fcc3fa0e25be5befed
slightly less early quorum

The early quorum change has maybe added a little bit too much
eventual to the consistency of requests in Swift, and users can
sometimes get unexpected
results.

This change gives us a knob to turn in finding the right balance,
by adding a timeout where pending requests can finish after quorum
is achieved.

Change-Id: Ife91aaa8653e75b01313bbcf19072181739e932c
Merge "allow bare excepts in flake8"
Merge "Add a DebugLogger for wsgi server tests"
Protect against hash cleanup errors on PUTs

In http://launchpad.net/bugs/1254405 an exception occurs when
finalizing the PUT of an object, but it is obscured by the thread pool
code, so we don't see the actual line where it originated. However,
there are two possible functions where this exception could originate:

    1. renamer()
    2. hash_cleanup_listdir()

If this is an error in renamer(), there is some other waky problem
where the temporary file has been removed. It is likely that this is a
problem where a file name from os.listdir ends up disappearing (but
even this is a rare occurence). Regardless, it is not clear that we
really want an error from hash_cleanup_listdir() from affecting the
return result of the PUT.

To that end, we squelsh OSErrors from hash_cleanup_listdir() for
now. One might argue for all errors, but since os.unlink() and
os.listdir() raise OSError today, that is probably sufficient.

Even that we use "Closes-Bug" below, it is not clear it can even be
determined that this closes that bug report.

Change-Id: I2f55df835c387e4d17cffda74c04c9994aebbe1f
Closes-Bug: 1254405
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Remove unnecessary "is not None" check

From a review comment on https://review.openstack.org/30051 remove
the "is not None" check, as the assignment in the try block will
never assign None as value as the int() built-in will not return it.

There was some concern of long-term maintenance of the DiskFile
class's _quarantine method raising exceptions. If that routine were
ever mistakenly changed to NOT raise an exception, subtle problems
could creep into the code (see https://review.openstack.org/53237).
We address this concern by raising an exception explicitly at the
call sites of DiskFile._quarantine().

Change-Id: I1729a2d77a6b72b4494b24a8838b47ad5272c075
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Fix for memcache middleware configuration

The documentation rightly said to use "memcache_max_connections", but
the code was looking for "max_connections", and only looking for it in
proxy-server.conf, not in memcache.conf as a fall back.

This commit brings the code coverage for the memcache middleware to
100%.

Closes-Bug: 1252893
Change-Id: I6ea64baa2f961a09d60b977b40d5baf842449ece
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Push cooperative sleep call down into ThreadPool

The PUT REST API has no idea how writes are performed, so when thread
pools are in use, the sleep is not necessary, though it is still
necessary when thread pools are not in use. Since the ThreadPool
object knows when threads are actually in use, it can take care of
being cooperative with the eventlet hub.

In addition, we can hide the cooperative iterator hook, given that the
only other consumer of this was the auditor, which does not need it
any longer.  The only consumer of the DiskFile class that wants the
cooperative behavior is the REST API layer of the object server, which
is also using thread pools.

Change-Id: Ibc4ac672899f9a35fd68c85d7f56403c19b4f991
Signed-off-by: Peter Portante <peter.portante@redhat.com>
add bare except to catch errors

Change-Id: Ibe78912cf923591bddd6a8cf0e683cd028c9c4e8