columbia-applied-data-science/rosetta — pull request #29 by mdeland (2014-03-30 22:49:43+00:00)
Title: Parallel apply

Introduces two ways to apply a function over a stream, one using threads and one using processes, without having to read the entire stream.
We altered the default implementation of '''.to_vw()''' to use the parallel process approach. Benchmark tests have shown a 2-4x speedup over using a single process. The more CPU bound the function is, the more improvement you can realize.
The threading functions are not currently used by the project, but could be useful in the future.