scikit-learn/scikit-learn — pull request #3034 by eickenberg (2014-04-03 07:39:34+00:00)
Title: [ENH] Ridge can use sample weights in feature space (X.T.dot(X) gram matrix)

Hello everybody,

here is an enhancement to the ridge_regression function: Up until now, if sample_weight is given, the decision tree imposes a treatment in sample space, which can lead to Memory errors if the number of samples is large even though the number of features is small. See http://stackoverflow.com/questions/22766978/sklearn-ridge-and-sample-weight-gives-memory-error for an external issue on this.

All I added was the possibility to treat sample weights in _solve_dense_cholesky and not only in _solve_dense_cholesky_kernel.

I provide a batch of tests checking that the results of these two functions are the same given sample weights.

Further I provide a test to ensure that the correct branch is chose in ridge_regression (i.e. use _solve_dense_cholesky if n_samples > n_features even if there are sample weights). The way I implemented this is that it will cause a MemoryError if the wrong branch is taken. By TDD I made sure it fails first - now it passes. Please comment on this practice: Evidently in this current, corrected case, the MemoryError will not be raised, but if ever the branching is wrong, and the computer tested has 2TB of memory available, this test will no longer raise a MemoryError. If there are better ways of checking the right branching, please let me know.

PS: There are some inconsistencies in function signature default values: _solve_dense_cholesky_kernel has sample_weight=None as default, which will, if called directly, be interpreted as though sample_weights are actually present. This has never caused an error, because the public function ridge_regression always calls with sample_weight=1. if no sample weights are given. Should I correct this in the present PR or elsewhere?