scikit-learn/scikit-learn — pull request #1702 by pprett (2013-02-22 10:14:11+00:00)
Title: WIP: LinearClassifierMixin support for sparse coef_

In the case of high-dimensional feature spaces and/or large number of classes it might be wise to store the coef_ array of a linear model as a scipy sparse matrix rather than a dense numpy array.

We might even do this automagically e.g. if the sparsity of coef_ reaches a certain level, however, in the meanwhile I'd suggest that the user is in charge of making the decision whether to convert coef_ to a sparse representation.

This popped up on the mailing list recently: http://sourceforge.net/mailarchive/message.php?msg_id=30515870

This PR is just a minor modification of LinearModelMixin.decision_function that enables the use of sparse ``coef_``.

NOTE: this PR is work in progress - no tests yet.

I was suspicious that adding ``dense_output=True`` might cause a performance regression, however, it seems that this is not the case::

The following test is on the test set of RCV1 ccat; shape: (23149, 47152)

Original
======

Dense
---------

In [7]: %timeit clf.predict(x)
10000 loops, best of 3: 51.6 us per loop

In [8]: %timeit clf.predict(X)
100 loops, best of 3: 6.55 ms per loop

This PR
=======

Dense
---------

In [16]: %timeit clf.predict(x)
10000 loops, best of 3: 54 us per loop

In [15]: %timeit clf.predict(X)
100 loops, best of 3: 6.83 ms per loop

CSC coef_

In [12]: %timeit clf.predict(x)
1000 loops, best of 3: 324 us per loop

In [13]: %timeit clf.predict(X)
10 loops, best of 3: 29.6 ms per loop

CSR coef_

In [18]: %timeit clf.predict(x)
1000 loops, best of 3: 566 us per loop

In [19]: %timeit clf.predict(X)
10 loops, best of 3: 30.3 ms per loop