scikit-learn/scikit-learn — pull request #2146 by ianozsvald (2013-07-10 12:39:41+00:00)
Title: clearer decision surface plots and classifier final predictions for the ...

I've made some changes to plot_forest_iris.py to make the demo more useful when teaching, this affects the image shown here:
http://scikit-learn.org/dev/auto_examples/ensemble/plot_forest_iris.html

This is connected with this issue https://github.com/scikit-learn/scikit-learn/issues/2133

I've attached a preview image of the output to this report. Roughly in order of significance I discuss the changes. I have *not* changed the AdaBoost plot with respect to estimator weights - see below.

1) Changed estimator_alpha from a hardcoded 0.1 to 1/nbr_estimators. At 0.1 it was common for the red channel to dominate the plot, leading to the erroneous visual output in the original diagram (seen at the link above) where e.g. the bottom-right plot in the region -1,1 to 3,2.5 looks like it has a strong red decision surface but has yellow samples plotted on top. The model is correctly predicting yellow in this region but the 0.1 alpha and the red channel obscure this. After this change the same region is still coloured Red but with a much lighter shade (but this still looks wrong). A benefit is that if the user experiments with the size of the ensembles, the plot will still be drawn sensibly.

2) Changed the colour map from Paired (yielding BlueRedOrange) to RedYellowBlue. The distinction between red/orange is weaker than red/yellow, the new colour map shows the erroneous region mentioned in 1) with its correct colour (now blue). The side effect is that the original Blue/Red/Orange colours for the 3 classes change to Red/Yellow/Blue. I don't know if the colour map change is acceptable? Maybe there's a standard for the Iris set that I'm not aware of? This choice does not conflict with the other examples in the /examples/ensemble/ directory.

3) Added a regular coarse grid overlay of predictions in 2 dimensions using a scatter plot, these show as small dots in a grid over each plot. The purpose is to show how the predictor behaves at each plot location, which might *not* be the same as the average colour that comes through via the decision surfaces and the alpha values. An example is plot 8 (right column, second row) where the yellow region is not a uniform rectangle but is narrower near the red points (on the left) and wider in the blue points (on the right), although to my eye the alpha-blended decision surfaces do not look very different. In the same plot there is a yellow 'spur' from 0,0.5 up to 0,2.5 which runs through the blue region, this region would probably be missed by a student.

4) I did *not* vary the AdaBoost column's alpha value with respect to the underlying weights. Visually I could see very little difference to the output when using the weights to vary the alpha value. I found the regular coarse grid overlay to be far more informative (see 3).

5) Changed AdaBoost to use a tree max_depth of 3 rather than the usual None (which in this dataset is equivalent to a max_depth of 5) to show that the method combats increased bias by combining votes from weak classifiers (as per OG's note in the forum on 7th July)

6) Added column titles rather than using a long suptitle.

7) Added more output to the console to show the number of estimators (noting that AdaBoost can use fewer estimators if it fits well early on - a student might experiment and not realise this) and the score on the dataset.

![plot_forest_iris_new](https://f.cloud.github.com/assets/273210/774555/405f873a-e95d-11e2-9f5b-9271572c8a5c.png)
...ensembles