scikit-learn/scikit-learn — pull request #3302 by eickenberg (2014-06-19 20:48:09+00:00)
Title: [BUG]  _BaseRidgeCV doesn't pass scoring if cv is not None / RidgeClassifierCV shouldn't use continuous setting in inner CV

I could not create a failing test for the failure to pass scoring through to the `GridSearchCV` which is involved, since these objects are not persisted in the `_BaseRidgeCV`.

Passing the the `scoring` to the `GridSearchCV` within `_BaseRidgeCV` creates a new problem, since the internal CV of this object seems to be exclusively working in the continuous setting, thus making e.g. `f1_score` raise an exception due to discrepancy of `y_true` and `y_pred` types.

While there may exist consistency proofs for accuracy score wrt continous Ridge, I am unsure if this is the case for f1_score (?). So the solution would be to have `RidgeClassifierCV` use an inner CV evaluated in a classifier setting, not a regression approximation to this setting. This looks like a slightly bigger change than I thought.

The diff I am pushing shows the situation where `RidgeClassifierCV` breaks due to `scoring='f1'` once it is passed through to the `GridSearchCV` within `_BaseRidgeCV`.

I have some ideas for a general refactoring of this code, but will first collect those in a gist for discussion.