thoughtbot/paperclip — pull request #1626 by jebw (2014-08-15 10:57:03+00:00)
Title: Download S3 files by reading in chunks, not all at once.

This changes the downloading of objects from S3 to avoid reading the entire object into memory.

Anything which results in a reprocessing of an attachment will need to download the file from S3. In our case we're dealing with large files (video, sometimes several hundred megs), and we're using the delayed_paperclip gem to reprocess the files using delayed job. This results in the original files getting downloaded, reprocessed then uploaded back to S3. The upload gets chunked automatically by the S3 gem, but the download part doesn't ballooning up Ruby's memory usage - this is changes the download behaviour to follow the pattern recommended in the aws-sdk docs for S3Object#read