scikit-learn/scikit-learn — pull request #882 by mblondel (2012-06-02 17:53:38+00:00)
Title: WIP: Calibration plot

I've started to play with calibration plots. I added an example: `examples/plot_calibration.py`.

I would like to get feedback from people who are familiar with the subject (especially @alextp and maybe @paolo-losi?).

On small and even not so small datasets, I'm often getting empty bins (meaning that some ranges of probabilities are never predicted by the classifier). To get smoother plots, I was thinking I could use polynomial regression but there's a risk of overfitting since the number of bins is small.

This is a work-in-progress: I won't have time to add tests and documentation anytime soon.