scikit-learn/scikit-learn — pull request #1036 by pprett (2012-08-18 10:41:50+00:00)
Title: WIP GBRT with built-in cross-validation

Two new classes ``GradientBoostingClassifierCV`` and ``GradientBoostingRegressorCV`` which pick ``n_estimators`` based on cross-validation.

``GradientBoostingClassifierCV`` fits a ``GradientBoostingClassifier`` with ``max_estimators`` for each fold; it picks ``n_estimators`` based on the min deviance averaged over all test sets. Finally, it trains the model on the whole training set using the found ``n_estimators``.

``GradientBoostingClassifierCV`` is implemented as a ``GradientBoostingClassifier`` decorator. It soley implements ``fit``, otherwise it delegates to ``GradientBoostingClassifier`` (see ``__getattr__`` and ``__setattr__``).
The current implementation might pose some problems if the client uses ``isinstance`` rather than duck typing: a ``GradientBoostingClassifierCV`` instance is not an instance of ``GradientBoostingClassifier``. I would really appreciate any remarks/feedback to this issue.

I tried to adhere the interface of ``RidgeCV``.

Additionally, I refactored the prediction routines in order to remove code duplication. ``staged_predict`` and ``staged_predict_proba`` has been added to ``GradientBoostingClassifier``.

Limitations:

* It is currently hard-wired to pick ``n_estimtors`` based on deviance (no support for custom loss function yet) - is this needed?
* No joblib support yet
* Only cross-validation support - should we add held-out and OOB estimation too?