jssjr/carbonate — pull request #40 by bruce-lyft (2014-11-11 18:41:41+00:00)
Title: --parallel option to use gnu parallel for the merge step

This PR introduces a "--parallel" option to carbon-sync. "--parallel" will shell out to gnu parallel in order to parallelize the whisper-fill invocations.

For our workloads, the speedup is about 5x during the merge phase, and about 2x overall.
Obviously YMMV depending on number of cores available, etc.

TODO - would be nice to pass arguments to gnu parallel.