Accuracydiversity based pruning of classifier ensembles
Classification ensemble methods have recently drawn serious attention due to their ability to appreciably pull up prediction performance. Since smaller ensembles are preferred because of storage and efficiency reasons, ensemble pruning is an important step for construction of classifier ensembles. In this paper, we propose a heuristic method to obtain an optimal ensemble from a given pool of classifiers. The proposed accuracydiversity based pruning algorithm takes into account the accuracy of individual classifiers as well as the pairwise diversity amongst these classifiers. The algorithm performs a systematic bottom-up search and conditionally grows sub-ensembles by adding diverse pairs of classifiers to the candidates with relatively higher accuracies. The ultimate aim is to deliver the smallest ensemble with highest achievable accuracy in the pool. The performance study on UCI datasets demonstrates that the proposed algorithm rarely misses the optimal ensemble, thus establishing confidence in the quality of heuristics employed by the algorithm.
diversity classifier ensemble ensemble pruning heuristic optimal ensemble accuracy brute force search