scikit-learn/scikit-learn — pull request #3346 by MatthieuBizien (2014-07-05 21:27:44+00:00)
Title: Categorical split for decision tree

Contrary to many algorithms, that can only use dummy variables, decision trees can behave differently for categorical data. The leaves will partition the categories. We could expect a better accuracy in some cases, and to limit the number of dummy columns. This is the default behavior of R randomForest package.

I am currently implementing that in sklearn, using the Cython classes. I propose to ask a categorical_features option to the decision trees classes (DecisionTreeClassifier, DecisionTreeRegressor, ExtraTreeClassifier, ExtraTreeRegressor). This option could be, like in other modules of sklearn, 'None', 'all', a mask or a list of features.

Each feature could have up to 32 classes, because we will have to test all the combinaisons, so 2**31 cases. This limit allows us to use a binary representation of a split. The same limit exists in R, I think for the same reasons.

This is a work in progress, and not ready to be merged. I prefer to release it early, so I could have feedbacks.