bytefish/libfacerec — pull request #19 by pickle27 (2013-07-26 15:58:31+00:00)
Title: finished idea for ROC, code works but the result isn't useful

The result isn't useful because the best roc is at the lowest confidence filtering thresh.

The problem is that regular ROC only works for binary classifiers and face rec is a multiclass problem. There are ways of doing multiclass roc but in my brief research I found that they looked quite complicated (thus a lot of code work) and because of the complexity I don't think they really meet our goal here of simply demonstrating (with a single plot?) that some of the parameter choices can have an effect on your performance and its worth taking some time to set them properly for your application.

I decided to push this as is so everyone can see my typical framework for ROC when the problem is binary. I had a bit of a different idea that I thought might work that is just a simple modification where at each thresh I just print the % correct and the number of mistakes but again the best performance is to no reject anything (unless your application needed 0 mistakes but in order to get 0 the correctness dropped to 20% due to some stubborn faces that must be really close in eigenspace).