Allow use of S3 buckets in regions other than us-east-1.

Work around region issue in S3Backup.new_connection and
boto.storage_uri.  When attempting to connect to a bucket in a
non-standard (non us-east-1) region, boto complains

  S3ResponseError: 301 Moved Permanently
  The bucket you are attempting to access must be addressed using the
  specified endpoint. Please send all future requests to this endpoint.

Naming collision fix contributed by Toby Collier (@tobyw4n)

Add a unit test and integration test.  This adds a testing dependency on
unittest2 which is a Python 2.6 backport of the unittest module
improvements in Python 2.7.

Having AWS credentials in your environment along with the
WALE_S3_INTEGRATION_TESTS (any arbitrary value) will trigger inclusion
of the integration tests.

References:

  https://github.com/boto/boto/issues/443
  https://github.com/heroku/WAL-E/issues/24

Signed-off-by: Bo Shi <bs@alum.mit.edu>
Signed-off-by: Daniel Farina <drfarina@acm.org>
pyflakes and pep8 cleanup

Signed-off-by: Daniel Farina <daniel@heroku.com>
Add Contributor: Toby Collier

Signed-off-by: Daniel Farina <daniel@heroku.com>
Include VERSION in package as a resource

With this, wal-e can (perhaps?) finally be packaged as a cheeseshop
package, and can learn the long-missing 'version' action.

Signed-off-by: Daniel Farina <daniel@heroku.com>
Increment version number

v0.6 now has a maintenance branch.  This should have been done in the
beginning.

Signed-off-by: Daniel Farina <daniel@heroku.com>
setup.py: downcase WAL-E to wal-e

Continue the general conversion of machine-intended references to
WAL-E to be lower case, much like the executable, package name,
repository, and so on.

It is still handy to have capitalized name (and WAL is an acronym
anyway) because it makes references to WAL-E stand out in human-read
text without quoting or special formatting.

Signed-off-by: Daniel Farina <daniel@heroku.com>
setup.py: prepare metadata for cheeseshop upload

Also refactor the way short files are read into cheeseshop metadata.

Signed-off-by: Daniel Farina <daniel@heroku.com>
tests: excise unittest2 dependency

Given that pytest is already pretty much here to stay without
significant work, write these tests to use pytest's functionality to
be able to avoid unittest/unittest2.

Unfortunately this seems to require a change of behavior in how the
cause of the skip will be reported.

Signed-off-by: Daniel Farina <daniel@heroku.com>
s3_worker: don't capture all exceptions

Problematic exceptions that can be caught by this except: include
KeyboardInterrupt.  It seems like this wouldn't matter much, but in
general 'except:' without re-raising is probably not good form.

Signed-off-by: Daniel Farina <daniel@heroku.com>
log_help: ignore syslog connection problems

Signed-off-by: Daniel Farina <daniel@heroku.com>
retries: pull out retry helpers from wal_e.s3_worker

These are useful in their own right (even though the abstraction
itself is kind of lousy, and the code convoluted), having little to do
with S3 work in particular.

Signed-off-by: Daniel Farina <daniel@heroku.com>
retries: avoid accidental livelocks

Putting in an extra yield doesn't seem like it could hurt, but the
alternative sure could.

This was a defect of the pre-refactored version, too.

Signed-off-by: Daniel Farina <daniel@heroku.com>
retries: do not ignore KeyboardInterrupt

Unlike other exceptions, this one is triggered by the user -- don't
ignore it.

Signed-off-by: Daniel Farina <daniel@heroku.com>
s3: multi-delete optimization

Introduce asynchronously triggered bulk-deletion.  This makes
deletions of many keys considerably (over ten times) faster, for large
cases.

The minimum boto version has been incremented to 2.2, the first
version to support multi_delete.

Signed-off-by: Daniel Farina <daniel@heroku.com>
Add build and packaging tools for PyPI

Editorialized (mostly, just squashed) by Daniel Farina.

Signed-off-by: Daniel Farina <daniel@heroku.com>
Add contributor: Kirill Klenov

Signed-off-by: Daniel Farina <daniel@heroku.com>
Fix license placeholder expansion

Signed-off-by: Daniel Farina <daniel@heroku.com>
Fix upper case bucket names regression (from #3).

Evidently, boto.storage_uri called without extra connection arguments
constructed a connection object compatible with uppercase bucket names
(which I discovered today is only allowed in us-east-1; one gets an
error attempting to create a bucket with an uppercase name in all other
regions).  Issue #3 added a 'host' connection argument that appears to
have changed the configuration.  Explicitly request the appropriate
calling format.

Signed-off-by: Daniel Farina <daniel@heroku.com>
ImportError: No module named Bucket

A --dry-run on delete fails with
        DETAIL: Traceback (most recent call last):
          File "/usr/local/lib/python2.7/dist-packages/wal_e-0.7.dev-py2.7.egg/wal_e/cmd.py", line 365, in main
            import boto.s3.Bucket
        ImportError: No module named Bucket

This fixes the import
Add Christian Pedersen to CONTRIBUTORS

Signed-off-by: Daniel Farina <daniel@heroku.com>