Although the conventional performance indexes, such as accuracy, are commonly used in classifier selection or evaluation, information-based criteria, such as mutual information, are becoming popular in feature/model selections. In this work, we analyze the classifier learning model with the maximization normalized mutual information (NI) criterion, which is novel and well defined in a compact range for classifier evaluation. We derive close-form relations of normalized mutual information with respect to accuracy, precision, and recall in binary classifications. By exploring the relations among them, we reveal that NI is actually a set of nonlinear functions, with a concordant power-exponent form, to each performance index. The relations can also be expressed with respect to precision and recall, or to false alarm and hitting rate (recall).
Wang and Hu REF was the first to derive the analytical relations of mutual information with respect to accuracy, precision, and recall, and their analytical bounds.
Derivations of Normalized Mutual Information in Binary Classifications
2009 Sixth International Conference on Fuzzy Systems and Knowledge Discovery
2009 Sixth International Conference on Fuzzy Systems and Knowledge Discovery
Mathematics Computer Science