On the Feature Selection Criterion Based on an Approximation of Multidimensional Mutual Information
We derive the feature selection criterion presented in [1] and [2] from the multidimensional mutual information between features and the class. Our derivation: 1) specifies and validates the lower-order dependency assumptions of the criterion and 2) mathematically justifies the utility of the criterion by relating it to Bayes classification error.
feature selection mutual information bayes classification error entropy entropy estimation