scikit-learn/scikit-learn — pull request #4034 by borjar (2014-12-30 17:58:58+00:00)
Title: Implemented the H-Measure metric for binary classification.

Hi,

I added an implementation of the H-Measure, an alternative performance metric to the AUC for binary classifiers.

The AUC uses different misclassification costs (false positive - false negative) for different classifiers, which means it effectively compares different things when used on two different classifiers even on the same problem/dataset (see the following paper for proof: http://engr.case.edu/ray_soumya/mlrg/measuring_performance_hand.mlj09.pdf). The H-Measure solves this issue by explicitly specifying the relative costs of false positives and false negatives as a probability distribution.

This new method uses a beta distribution with parameters alpha and beta for the relative misclassification costs, and its implementation follows the paper mentioned above. If the parameters alpha and beta are not provided then the default distribution will be the one explained in the following paper:
http://arxiv.org/pdf/1202.2564v2.pdf

This new method was added to the metrics module in ranking.py. More information on the H-Measure can be found at the following website: http://www.hmeasure.net/

Kind regards,

Borja