scikit-learn/scikit-learn — pull request #2829 by arjoly (2014-02-05 13:30:43+00:00)
Title: Uniformize max_features semantics of extra trees and random forest

The goal of this pr is to uniformize the semantics of max_features:
- random forest: currently draws features until it found a non constant features, then it begins to count the number of features drawn
- extra trees: draws max_features non constant features

This pr uniformize all behavior as discussed in #2813: we draw max_feature and at least one must be one must valid (i.e. non constant).

Benchmarks on covtype:
```
this pr train-time test-time error-rate
--------------------------------------------
RandomForest 70.6019s 0.3386s 0.0344 (max_features=sqrt)
RandomForest 109.0293s 0.2501s 0.0263 (max_features=15)
ExtraTrees 53.3487s 0.4981s 0.0381 (max_features=sqrt)
ExtraTrees 93.1047s 0.3927s 0.0277 (max_features=15)
CART 16.5673s 0.0155s 0.0425
```

```
master train-time test-time error-rate
--------------------------------------------
ExtraTrees 103.1957s 0.3149s 0.0212 (max_features=sqrt)
RandomForest 73.9088s 0.3621s 0.0309 (max_features=sqrt)
CART 17.6760s 0.0194s 0.0426
```

There is a decrease of performance du to the change in semantics. Increasing max_features allows to get back to previous error-rate.