taskrabbit/elasticsearch-dump — pull request #24 by evantahler (2014-05-27 05:53:31+00:00)
Title: Throttle elasticsearch writes

For node v0.11.x (and eventually node v0.12.x), the semantics of http.agent have changed, and it looks like the http pool behavior will default to `Infinity` rather than defaulting to `5`. You probably don't want to make Infinite HTTP requests to your elasticsearch server.

Testing locally, you can see that any number of requests over the `ulimit` (sometimes /4, depending on your OS) will fail to connect. This will lead to lost data on writes.

This PR begins an attempt to re-introduce a http agent/throttle to the ES.write transport