scikit-learn/scikit-learn — comment on commit 1694a8f2d802c006d35c842d37de5d0610f9f57a
File:  line 0
By GaelVaroquaux:

----- Original message -----
> On the 20newsgroup dataset, the estimator
> object for a `LinearSVC` with L1 penalty went from 9.5M to 180K.

That's really cool. Definitely usefull.

> This just gave me an idea: one could create a smart CSR matrix
> implementation that would uncompress rows on demand (possibly combined
> with mmap to load the compressed rows).

You can have a look at the carray package, that does this and can be usefull to work on really big data. We cannot depend on it in the scikir, but we can try and make the scikit work as well as possible with it.