christkv/node-mongodb-native — Issue #386: Slow deserialization performance under heavy loads
Author: nikhilbelsare
Created: 2011-10-29 12:44:10+00:00
State: open

Hi,

I am experiencing unusually slow deserialization performance with concurrent load. I have a simple collection of items no more than 1000 items. I'm using mongoose infront of the mongo driver to fire queries like {itemId: {$in: [ array of ids] }. The problem is for such a simple operation the db is turning out to be the bottleneck. Under heavy loads such as 100 concurrent users this simple fetch is taking anything between 1.5 to 2 seconds on sufficiently beefy hardware.

At first I thought mongoose wrapper or even the db could be causing problem. To find the root cause I enabled mongodb profiling but nothing showed up in slow queries confirming db was not underperforming. I also profiled mongoose overhead and found that it was negligible. Finally, I tried a similar workflow with Java driver of MongoDB under similar loads to find it performing noticeably better - low 200ms.

I instrumented cusror.js and db.js to find the bottlenecks and finally reached function "executeCommand" where the bulk of the time was being taken. I figure the time spent between firing actual command and returning to "handleBothCommandsFunction" should include time spent in talking to server and deserialization. Not sure if it's slow deserilization but that would be my first guess. Hope this helps.

node.js version: 0.4.8
mongodb driver version: 0.9.6-22
mongodb version: v1.8.2
mongoose version [just in case you need]: 2.3.8

Any help is appreciated.

TIA
~ Nikhil