scikit-learn/scikit-learn — pull request #923 by glouppe (2012-06-29 12:06:06+00:00)
Title: WIP: Multi-output decision trees

Hi folks!

Just to let you know, I am currently working on a multi-output extension of our decision trees.

Basically, this will make our implementation capable of handling classification or regression problems with several outputs. As I was discussing with @pprett, a very simple way to solve this kind of problems is to build n independent models, i.e. one for each output. However by doing that you lose the (likely) correlation between the outputs (classes or regression values). Hence an often better way is to build a single model to predict simultaneously all n outputs. With regard to decision trees, this amounts to store n output values in a leaf and to use splitting criteria that compute the average reduction among all outputs.

This PR includes a working prototype of multi-output decision trees. I tried as much as possible not to impair training time on single-output problems.

A lot of things still need to be done:
1) Write multi-output unit tests
2) Patch RandomForest* and ExtraTrees* to account for the API changes
3) Patch GradientBoosting* to account for the API changes
4) Benchmarks