scikit-learn/scikit-learn — pull request #941 by sgenoud (2012-07-10 11:10:26+00:00)
Title: Adding a pruning method to the tree

I have added a pruning method to the decision trees method. The idea with decision tree is to build a huge one, prune it via a weakest link algorithm until it reaches a size that is reasonable (neither overfitting nor under fitting).

I also have build a helper function, cv_scores_vs_n_leaves, that computes the cross validated scores for different sizes of the tree. This can be plotted with a function such as
```python
def plot_cross_validated_scores(scores):
"""Plots the cross validated scores versus the number of leaves of trees"""
import matplotlib.pyplot as plt
means = np.array([np.mean(s) for s in scores])
stds = np.array([np.std(s) for s in scores])

x = range(len(scores)+1, 1, -1)

plt.plot(x, means)
plt.plot(x, means+stds, lw=1, c='0.7')
plt.plot(x, means-stds, lw=1, c='0.7')

plt.xlabel('Number of leaves')
plt.ylabel('Cross validated score')

```
Then we choose which size is the best for the data.

Just a couple of notes:
* I need to add some tests (I am not sure how to make them)
* This also needs to be documented

Before doing that work, I would gladly have some feedback on these modifications