gruntjs/grunt-contrib-compress — pull request #36 by jpommerening (2013-04-02 11:44:03+00:00)
Title: Open streams lazily to avoid "Error: EMFILE"

In `tasks/lib/compress.js` around Line 115, there is a loop that opens all of the input files at once, or at least faster than `archiver` can consume them.

When compressing a lot of files this reliably triggers `EMFILE` errors (too many open files).

`archiver.append()` can either take readable streams, buffers or plain Strings. So, assuming we don't want to read all files to buffers, we are left with streams.

I added a simple stream class that creates the `fs.ReadStream` on demand ("demand" meaning when `resume()` is called). If waiting for `resume` to be called is too fragile, maybe it is safer to wait for someone to subscribe to the `data` event?

Additionaly, creating a test for that is somewhat tricky - the number of files that can be opened by a process varies greatly between systems. If we *really* need a test for that, we could patch the stream class to keep track of the number of open files and check that it never exceeds "one".

Let me know if there's anything else to be done to get this pulled, or if there's a nicer way to solve this.

Thanks in advance!