membase/ep-engine — pull request #17 by traqy (2012-02-06 07:17:48+00:00)
Title: Bug fixed on the multi-threading. 

The current code base does not actually propagate the value assigned to opts.threads (-t). Even the DEFAULT_THREADS is also not propagated. The script "cbrestore" always create one thread instance for worker function. As a result, it can restore only upto 700 items/sec.

This bug fix is tested in our production platform and we've managed to restore db files at 3.5k to 5k items/sec on each node running cbrestore process to all nodes in parallel. For nine node-cluster, we were able to restore at 30k to 36k items/sec. At 100 million items, we were able to restore it within an hour.