boto/boto — pull request #994 by jamesls (2012-09-18 21:59:26+00:00)
Title: Add threaded multipart archive upload for glacier

This will internally create a thread pool that concurrently
uploads the various chunks of a file using the multipart
API for glacier.

When testing upload speeds, this was approximately 13 times faster
than the single threaded archive upload. It also is much better
at sustaining a constant upload rate, in the single threaded
upload the upload rate fluctuated quite a bit (which makes sense
given that it part of the time it's reading from disk and computing
hashes instead of sending data).

For comparison, for a 1GB file:

```
Operation | Time (secs) | Average Upload rate (MB/s) |
---------------------------------------------------------------
s3multiput 216 6.0
glacier-single 1291 2.0
glacier-concurrent 98 11.5
```

I added this functionality as a new method to the vault
class as well as a separate module. Wasn't sure if it made sense
to also make this the default for the Vault.upload_archive method.