wowus/stm-conduit — pull request #15 by jwiegley (2014-01-21 09:50:49+00:00)
Title: Add a bufferToFile function in Data.Conduit.Async

This patch adds a single function, `bufferToFile`, which is very similar to the existing `buffer` function in that it allows for asynchronous communication between producers and consumers -- except now you can buffer any overflow to a disk cache.

This can be useful if you have expensive (i.e., slow) consumers and/or producers, with variable latency on either side, yet where local disk space is cheap. You can allow the disk cache fill up with results from the producer while the consumer is waiting; or if the producer suddenly stalls, you might then have ample inputs for the consumer to process from out of the disk cache.

Many, many thanks to @snoyberg for helping make this function what it is now. He was invaluable in suggesting simplifications, and better use of conduit within the function itself.