mikeal/request — Issue #195: Occasional errors piping CouchDB via node.js
Author: jmarca
Created: 2012-02-28 22:34:35+00:00
State: open

I apologize for not having a better bug report, but I am noticing an occasional error using the `x.pipe(y)` pattern to push couchdb data out in response to web requests.

The version of request that I am using is request@2.9.152

The error has two facets: a crash with an error report, and a silent fail with no further activity from Request.

The crashing error is as follows, but I think this is just a symptom of the silent fail, not the actual problem:

Error: This request has been piped before http.request() was called.
at Request.resume (/home/james/repos/jem/carbserver/node_modules/tracker_rewriter/node_modules/request/main.js:702:29)
at ServerResponse.ondrain (stream.js:48:14)
at ServerResponse.emit (events.js:64:17)
at Socket.ondrain (http.js:1322:27)
at Object.afterWrite [as oncomplete] (net.js:488:28)

The code that triggers this error is as follows. (BTW, the code is borrowed shamelessly (but with attribution) from maxogden/rewriter, so it should be easy to reproduce.)

~~~ javascript
var couch = "http://127.0.0.1:5984";
exports.rewriter = function(req,res,next){
var to = couch + '/vdsdata%2ftracking/'+req.params.id;
if(req.params.attachment !== undefined){
to += '/'+req.params.attachment;
}
request(to).pipe(res);
}
~~~

My app does two things. First it asks for a document, and then pulls out all of the document's attachments (png files) and displays them. that part of the app uses the above service and when it crashes produces the above error.

The app also gets a list of the 5,000 or so document ids in a separate request, asking for 1000 docs at a time. This part of the app is the part that triggers the silent fail (Request freezes up). It uses the following service (very similar to the above).

~~~ javascript
exports.vdsdetectors = function(req,res,next){
var to = couch + '/vdsdata%2ftracking/_design/vdsml/_view/mainline';
var query = req.query
if(query)
to += "?" + toQuery(query)
request(to).pipe(res);
}
~~~

As I said, the bug seems to have two manifestations. The most obvious one is the crash, and I have found that it only happens *after* calling the listing service, but it also only happens from the one-at-a-time service (get a doc or get the doc's attachments). I *cannot* get that crash to happen by holding down control-r on a stripped down version of my app that just gets one document and its attachments.

The silent fail, in which Request freezes and refuses to work anymore (couchdb is fine) only occurs when triggering the request to get 1000 documents a bunch of times (I hold down control-r in ffox). I can get that one to happen every time, and it will usually fail after working only about 6 times (it varies). If I hit refresh at a more modest pace, the crash doesn't happen.

In case it is relevant, I am using these services within the connect framework, with a bunch of other things turned on...logging requests, session cookies, etc.

I hope this bug report makes sense, but if it doesn't or if you have any suggestions on what else to try, please let me know.