We define a new feature selection score for text classification based on the KL-divergence between the distribution of words in training documents and their classes. The score favors words that have a similar distribution in documents of the same class but different distributions in documents of different classes. Experiments on two standard data sets indicate that the new method outperforms mutual information, especially for smaller categories.
Recently, instead of using tf-idf, REF defines a new feature selection score for text classification based on the KL-divergence between the distribution of words in training documents and their classes.
A New Feature Selection Score For Multinomial Naive Bayes Text Classification Based On KL-Divergence
Annual Meeting Of The Association For Computational Linguistics - Interactive Poster And Demonstration Sessions
Computer Science