scikit-learn/scikit-learn — pull request #650 by fcostin (2012-02-23 14:59:36+00:00)
Title: optimisations to Ridge Regression GCV

Hi all,

I've made a few optimisations to the ridge regression generalised cross validation (GCV) code:

These changes eliminate unnecessary construction of the explicit matrix G=dot(Q,dot(diag(1/(v+alpha)),Q.T)) when we just want dot(G, y) and diag(G) inside the _error method. Similarly inside the _values method, we can do the same thing to avoid explicitly constructing dot(K,G) = dot(Q, dot(diag(v/(v+alpha)), Q.T)).

I've tested the speedup on the "communities and crime" dataset (http://archive.ics.uci.edu/ml/datasets/Communities+and+Crime), and I get a ~ 11x speedup when using a small training set consisting of the first 398 rows. I expect the speedup is even more convincing for larger training sets.

The scripts I'm using to benchmark the code are here: https://gist.github.com/1893136

I should probably add a new test for this too.