bazaarvoice/crunch — pull request #1 by shawnsmith (2013-01-17 15:11:11+00:00)
Title: Work around S3 upload errors in Elastic Mapreduce

I encountered an issue where the final step of the EmoDB bulk ETL step failed while attempting to upload the results to S3. Some files uploaded successfully, but most did not. The error condition closely matches the issues reported in the following forum threads:

https://forums.aws.amazon.com/thread.jspa?threadID=114363&tstart=0
https://forums.aws.amazon.com/thread.jspa?threadID=105345&tstart=0

In the absence of a useful response from Amazon, this workaround bypasses the EMR S3 native file system upload code and uses the aws-java-sdk and TransferManager directly. With this change, I haven't seen the same "request signature we calculated does not match the signature you provided" error message. reoccur. My best guess as to the reason this workaround works is that it limits the number of simultaneous uploads to 5 threads (by default TransferManager uses 10, which I've seen cause errors elsewhere). Or, the EMR MultipartUploadOutputStream stream has a bug in its multipart upload handling, since it doesn't use the TransferManager and uses a lower-level aws-java-sdk API for multipart uploads.