I think you're broadly right, Lars, but I'm trying to do a benchmark with a larger corpus, without anything fancy like stemming. I'm currently working with 70000 articles published by the New York Times in 2000 (from LDC2008T19‎, with some filtering). I'm varying `n_jobs` (1, 2, 16), batch size (1, 10, 100), and lookup/transmission method (keyed to file; text transmitted to subprocesses directly). Apologies for my lack of formatting effort.


Text in memory
```
n_jobs batch_size time
16 100 168.387213945
16 10 171.856986046
16 1 179.449703217
2 100 174.612135172
2 10 165.858082056
2 1 173.085582972
1 1 99.9582018852
```
In memory text doesn't just work with `n_jobs>1`. 

Files
```
n_jobs batch_size time
16 100 2.69259810448
16 10 7.49362397194
16 1 64.7444419861
2 100 2.81486201286
2 10 7.29099702835
2 1 64.1586031914
1 1 28.4908490181
```
With files the benefit is substantial as long as batch_size is big enough, but as we move from 2 jobs to 16 we get only marginal gain, suggesting it's I/O-bound. But the fact that `batch_size` affects things suggests that it's not all about I/O, and that there's something to be gained by parallelising even the default analyzer, as long as the batch size is *big enough*, and the corpus much bigger. In sum, a fixed batch size of 10 will give us a 4x speed-up in multiprocessing (relative to a single process), and we can get >10x speed-up using a batch size of 100.

(I'm also impressed to see @syllog1sm's POS tagger, and yet something like the C&C Tools tagger, which was created by syllog1sm and my common PhD supervisor still seems to be much faster. Tagging Brown takes 4.3s on my system, or 8e-5s per sentence.)