I did quite extensive [parameter search](http://nbviewer.ipython.org/urls/raw.github.com/ogrisel/notebooks/master/Parameter%20search%20for%20Extra%20Trees%20on%20the%20MNIST%20classificationt%20task.ipynb) on a EC2 cluster. The default parameters are almost as good as those I find by parameter search (overlapping mean +/- standard errors of the mean).

The final model, even with 1000 trees is not as good as the MLP model (see the linked notebook). The MLP's smoother decision function must be a better bias to encode small translation and rotation invariance of the data that pixel-wise decision function of the extra trees cannot capture. Maybe extra oblique trees would do very well on this task.