joyent/node — Issue #2809: parallel requests to the same site with the http client result in compounding latency
Author: mcavage
Created: 2012-02-22 23:35:05+00:00
State: open

We observed this in production (i.e., a portal making N concurrent calls to a REST api to populate page details), and ultimately distilled this down to reproducible behavior with node "hello world".

The details can be found here: https://gist.github.com/f0403429922ba995ac4f

The explanation to that is there's an extremely basic node http server, and a client, that issues N parallel requests to it. This behavior can be seen in both 0.4 and 0.6, and the output illustrates that latency for the Nth request is some multiplied factor of time for latency for the first request (this looks a lot like queuing client-side). I've tried tuning the Agent to ensure that there is never queuing on sockets, but that doesn't seem to help.

Also, while obviously not apples -> oranges, I whipped up a C client that runs parallel requests in N threads (it does reuse the connection, but that's what node's agent is supposed to be doing anyway), and you can see a flat latency profile (note the time for C is in microseconds, whereas JS is milliseconds). I largely did this to rule out any server problems; also, this client behavior can be reproduced against non-node servers, but it seemed relevant given the test case here was a node server.

Thanks,
~Mark