apache/spark — pull request #2607 by manishamde (2014-10-01 01:36:58+00:00)
Title: [MLLIB] [WIP] SPARK-1547: Adding Gradient Boosting to MLlib

Given the popular demand for gradient boosting and AdaBoost in MLlib, I am creating a WIP branch for early feedback on gradient boosting with AdaBoost to follow soon after this PR is accepted. This is based on work done along with @hirakendu that was pending due to decision tree optimizations and random forests work.

Ideally, boosting algorithms should work with any base learners. This will soon be possible once the MLlib API is finalized -- we want to ensure we use a consistent interface for the underlying base learners. In the meantime, this PR uses decision trees as base learners for the gradient boosting algorithm. The current PR allows "pluggable" loss functions and provides least squares error and least absolute error by default.

Here is the remaining task list:
- [ ] Stochastic gradient boosting support – Re-use the BaggedPoint approach used for RandomForest.
- [ ] BaggedRDD caching -- Avoid repeating feature to bin mapping for each tree estimator. Will require minor refactoring of RandomForest code.
- [ ] Checkpointing – This approach will avoid long lineage chains. Need to conduct experiments to verify good default settings.
- [ ] Unit Tests – I have performed some basic tests but I need to add them as unit tests.
- [ ] Create public APIs
- [ ] Tests on multiple cluster sizes and datasets – require help from the community on this front.

Note: Classification is currently not supported by this PR since it requires discussion on the best way to support "deviance" as a loss function.

cc: @jkbradley @hirakendu @mengxr @etrain @atalwalkar @chouqin