pouchdb/pouchdb — pull request #3174 by daleharvey (2014-12-14 14:46:26+00:00)
Title: (#3015) - Issues GET requests on pull replication in parallel for each b...

...atch.

In node.js using default settings (5 connections in the pool + 100 docs per batch) over a HTTP connection to a local PouchDB server perf increased by 8x over previous perf.

When replicating down the 0.6 gig NPM CouchDB database over HTTPS using CouchDB locally it took CouchDB about 0.69 seconds/100 records. Without this change it took PouchDB around 30 seconds/100 records. With this change on default settings we reduced time to 4 seconds/100 records. By changing the number of connections to 15 and the batch size to 1000 we got perf down to 0.9 seconds/100 records. So perf got down to only 30% worse than CouchDB. I strongly suspect that when the node.js connection harvesting bug is fixed (so we don't lose our existing connections between batches) and with a little more tweaking (it's clear that we are a bit more 'chatty' than CouchDB's replication client) we should be able to match CouchDB's perf.