SimTk/pylearn2 — pull request #17 by skearnes (2014-05-20 02:14:46+00:00)
Title: Calculate ROC AUC as a monitor channel

I wrapped sklearn.metrics.roc_auc_score as a Theano Op and wrote a TrainExtension class that calculates ROC AUC on each monitoring dataset. It's recommended to use monitoring_batches=1 with this extension, since ROC AUC cannot be calculated unless both classes are represented in a batch (but it fails gracefully, returning nan for that channel value).

See test_roc_auc.py for a simple example.