scikit-learn/scikit-learn — pull request #1401 by cjauvin (2012-11-23 20:31:39+00:00)
Title: Idea: speed up the parallelization of CountVectorizer by adding a batch mechanism to joblib.Parallel

A couple of weeks ago, I submitted this experimental idea to the joblib mailing list, but it didn't receive much attention:

As I was studying the implementation of the sklearn CountVectorizer, my attention was drawn to a comment in the code saying that its main loop could not be efficiently parallelized with joblib:

https://github.com/scikit-learn/scikit-learn/blob/33a5911b55617e919fcfabc283c24784deaed686/sklearn/feature_extraction/text.py#L469

I don't know much about the internals of multiprocessing, but I imagined that there might be a tradeoff between the size of individual jobs and the number of times that a process in the pool is dispatched a new job. For instance, if the vectorizer is passed a very long list of very short documents, then it would seem possible that the dispatching overhead makes it very suboptimal. Perhaps a smaller number of longer jobs would work better in that case?

To explore this hypothesis, I had the simple idea of chaining together jobs as batches (to be executed by a single process) and dispatch those, instead of individual jobs. The user can then experiment with different batch sizes, trying to find the sweet spot.

---

Today I have decided to try my idea within a more realistic setting, by implementing a minimal version of it for CountVectorizer (it's not a full solution, see caveats in the code comments). Using a 1M-line text corpus, I have obtained a 3X speedup on a 24-core Linux machine:

https://gist.github.com/4137131

I don't know if the idea makes much sense, but I thought I would submit it here anyway, just to see what people thinks.