scikit-learn/scikit-learn — pull request #1914 by fabianp (2013-05-02 09:46:56+00:00)
Title: Add SVD-based solver to ridge regression.

I added a solver based on the thin SVD of X to compute the ridge
coefficients. This is much more stable than the cholesky decomposition
for singular matrices.

While in the Ridge regression the Gram matrix becomes nonsingular
because of the regularization, I've come across problems when the
regularization parameter is very small. This pull request remedies it,
making the algorithm defined and stable even for the border case
alpha=0 (useful for cross validating over a grid of parameters)

I took the liberty to change the default algorithm of to this one
(when there are no sample weights). I haven't done any benchmarks.
It should be slower than dense_cholesky but not much ...