scikit-learn/scikit-learn — pull request #2759 by mblondel (2014-01-16 16:36:40+00:00)
Title: [WIP] Multiple-metric grid search

This PR brings multiple-metric grid search. This is important for finding the best-tuned estimator on a per metric basis *without* redoing the grid / randomized search from scratch for each metric.

* Updates `cross_val_score` so as to support lists as `scoring` parameter. In this case, a 2d array of shape `(n_scoring, n_folds)` is returned instead of a 1d array of shape `(n_folds,)`.
* Introduces `grid_search_cv` and `randomized_search_cv` functions which also accept lists as `scoring` parameter. `GridSearchCV` and `RandomizedSearchCV` are reserved for single-metric hyper-parameter tuning since they need to behave like a predictor.
* Introduces `_evaluate_scorers` for computing several scorers without recomputing the predictions every time.
* Fixes #2588 (regressors can now be used with metrics that require `need_threshold=True`.

AFAIK, no public API has been broken and I didn't introduce `decision_function` into `RegressorMixin`.

Tagging this PR with the v0.15 milestone.