criterion to evaluate a set of candidate features and to select an informative subset to be used as input data for a neural network classifier. Because the mutual information measures arbitrary dependencies between random variables, it is suitable for assessing the "information content" of features in complex classification tasks, where methods bases on linear relations (like the correlation) are prone to mistakes. The fact that the mutual information is independent of the coordinates chosen permits a robust estimation. Nonetheless, the use of the mutual information for tasks characterized by high input dimensionality requires suitable approximations because of the prohibitive demands on computation and samples. An algorithm is proposed that is based on a "greedy" selection of the features and that takes both the mutual information with respect to the output class and with respect to the already-selected features into account. Finally the results of a series of experiments are discussed. Index Terms-Feature extraction, neural network pruning, dimensionality reduction, mutual information, supervised learning, adaptive classifiers.
For this problem, the relationship between candidate features and the already selected features was also studied in MIFS REF .
Using mutual information for selecting features in supervised neural net learning
IEEE Transactions on Neural Networks
Computer Science Medicine