Add support for default_region

Changes to actually use the
'default_region' setting in the
S3TEST_CONF file.
Previously, the first connection set
was assumed to be the default.
This means that the parse order
of the S3TEST_CONF file was dictating
how the tests behaved, resulting in
failures in some cases.

Signed-off-by: Joe Buck <jbbuck@gmail.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Improve post-test cleanup

The multi-region tests were not being diligent
about cleaning up their buckets / keys
and making sure that those deletes were synced.
This was causing the nuke_prefixed_buckets()
method to run into errors.
This patch adds more cleanup code and
validation of that cleanup within
the pertinent tests.

Signed-off-by: Joe Buck <jbbuck@gmail.com>
Rework nuke_buckets

Nuke all buckets on the master, sync, and then
nuke any remaining buckets on non-master regions.

Signed-off-by: Joe Buck <jbbuck@gmail.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Moving region_sync to utils

Since other classes need to do syncs,
let's move it to the utils file.

Signed-off-by: Joe Buck <jbbuck@gmail.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
fix nuke_buckets for non-region tests

Signed-off-by: Joe Buck <jbbuck@gmail.com>
Merge branch 'next'
Added pip 1.5 workaround.

Setuptools needs to be upgraded with --no-use-wheel. A change we
already were using in other projects.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>
test_s3: extend multi-delete test

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
test_s3: extend multipart test

Make test use a multipart upload with different part sizes, to exercise
code that triggered #7935.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
test_s3: add some debug prints for multi region test

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>