However (and, I realize this is subject to internal API changes) we
observed that changing the size of the batch had a pretty drastic effect on
naive bayes because of the overhead of the finalize method.  Is this worth
mentioning?


On Wed, Jul 31, 2013 at 12:15 PM, Olivier Grisel
<notifications@github.com>wrote:

> In doc/modules/scaling_strategies.rst:
>
> > +explicit care by using a stateless feature extractor. A good example is text
> > +classification where unknown terms are likely to be found during training.
> > +Currently the preferred way to do this is to use the so-called
> > +:ref:`hashing trick<feature_hashing>` as implemented by
> > +:class:`sklearn.feature_
> extraction.FeatureHasher`.
> > +
> > +Online learning
> > +---------------
> > +Finally, for \3. we have a number of options inside Scikit-learn. Although all
> > +learning algorithms can not learn online (i.e. without seeing all the instances
> > +at once), all estimators implementing the `partial_fit` API are candidates.
> > +Actually, the ability to learn iteratively from a mini-batch of instances is key
> > +to out-of-core learning as it guarantees that at any given time there will be
> > +only a small amount of instances in the main memory. Choosing a good size for
> > +the mini-batch that balances relevancy and memory footprint could involve some
> > +tuning.
>
> For some model the batch size has no impact on the relevancy as internally
> it's implemented with true, one sample at a time cython loop. For instance
> SGD[Classifier|Regressor] based models (Perceptron, passive aggressive) and
> discreete naive bayes do true online learning internally.
>
> For MiniBatchKMeans the batch size has an impact on the convergence rate
> though.
>
> Maybe this should be added as a note at the end of the section instead.
>
> —
> Reply to this email directly or view it on GitHub<https://github.com/scikit-learn/scikit-learn/pull/2222/files#r5499990>
> .
>