pouchdb/pouchdb — pull request #3015 by yaronyg (2014-11-19 23:10:02+00:00)
Title: (#2940) - Issues GET requests on pull replication in parallel for each batch.

In node.js using default settings (5 connections in the pool + 100 docs per batch) over a HTTP connection to a local PouchDB server perf increased by 8x over previous perf.

When replicating down the 0.6 gig NPM CouchDB database over HTTPS using CouchDB locally it took CouchDB about 0.69 seconds/100 records. Without this change it took PouchDB around 30 seconds/100 records. With this change on default settings we reduced time to 4 seconds/100 records. By changing the number of connections to 15 and the batch size to 1000 we got perf down to 0.9 seconds/100 records. So perf got down to only 30% worse than CouchDB.

I suspect that when the node.js connection harvesting bug is fixed (so we don't lose our existing connections between batches) and with a little more tweaking (it's clear that we are a bit more 'chatty' than CouchDB's replication client) we should be able to match CouchDB's perf.