scikit-learn/scikit-learn — pull request #2570 by pprett (2013-11-03 07:49:34+00:00)
Title: [WIP] Gradient Boosting enhancements

This PR contains a number of enhancements for GBRT:

1. partial_fit now allows to add additional trees to an already trained model.

2. The fit parameter ``monitor`` allows to inject a callback into the learning procedure that can implement early stopping, evaluate training progress on held-out, creating snap-shots, ...

3. Trees can now be grown using ``max_leaf_nodes`` as a stopping criterion.
Setting ``max_leaf_nodes > 0`` will grow the tree in a best-first fashing. Nodes are pushed on a PriorityQueue (impl. as a binary heap) and the node with the maximum *impurity* (note: not relative impurity) is expanded next.
If ``max_leaf_nodes < 0`` then trees are grown in depht-first fashion by using a stack instead of the PriorityQueue.
Trees grown with ``max_leaf_nodes > 0`` resemble tree branches of depth ``max_leaf_nodes - 1`` and can model interactions of ``max_leaf_nodes - 1`` features. Individual trees might, however, occasionally be shallower then ``max_leaf_nodes - 1``.

4. New estimator ``ZeroEstimator`` if you want to start your GBRT model from scratch.