pifantastic/grunt-s3 — pull request #34 by jesseditson (2013-02-11 22:16:30+00:00)
Title: Better file handling options

Hey!

I was using this lib to clone a directory with a bunch of files, including licenses and junk - I ran into a few problems I've resolved with these commits:

- when uploading many files at once, EMFILE is hit pretty quickly. I added a maxOperations that will throttle the max number of concurrent transfer operations. This solves hitting an EMFILE error in a lots of files and low ulimit situation.

- I also had some files with special characters that S3 was refusing to upload with error 505. (even spaces cause this issue...) I added an encodePaths option that will run encodeURIComponent on destination paths when uploading.

Both new options are documented, and the library will default to performing as it did before these commits (maxOperations defaults to 0, which is equivalent to ∞, and encodePaths defaults to false).

The only other changes is that options don't try to parse templates on non-string values, and instead of a native forEach when processing ∞ parallel transfers, it will use async.forEach.