Learning Multi-Label Topic Classification of News Articles

We consider the problem of learning to classify the topic of news articles for which there are multiple relevant topic labels. We analyze the shortcomings of a number of algorithmic approaches, including naive bayes, and develop two alternate approaches to solving the problem. In particular, we assess the performance of a binary classifier approach in which we learn a set of oneversus-all naive bayes binary classifiers, one for each label class. We also develop and analyze the performance of two novel algorithms derived from the popular tf-idf weighting scheme, and motivated by the goal of constructing a learning model that is more similar to the way human readers may classify article topics. I. TOPIC CLASSIFICATION OF NEWS ARTICLES Classifying the semantic content, or topic, of text is one of the critical problems in natural language processing, information retrieval, artificial intelligence and machine learning more broadly. Newspaper articles provide a particularly good opportunity for learning such classifications, as the semantic content of articles is generally coherent, and large, open source corpuses of labelled news articles exist and are easily accessible. There is also a fair deal of interest in classifying news article topic for specific applications and research. News article topic classification can enable automatic tagging of articles for online news repositories and the aggregation of news sources by topic (e.g. google news), as well as provide the basis for news recommendation systems. More broadly, given the social and political impact of news and media, communications specialists and other social scientists including our collegue Rebecca Weiss at the Stanford Communications department, who suggested this project to us are particularly interested in analyzing news data programmtically to uncover patterns and biases in news production, content and dissemination. A complicating factor is that news articles often fall under multiple topic labels. An article about a transfer in ownership of the Chealsea football club, for instance, might be labelled under Sports, Business, and World News. Humans can recognize and correctly provide multiple relevant labels for an article, but can a machine learning system get similar results? II. DATA The New York Times has made a large corpus of its archive publicly available, with 5 years of articles hand tagged for various attributes of an article. Each article is provided along with a set of relevant taxnomic classifiers, which are essentially content labels. We chose to focus on learning only a small subset of the labels, 9 in total, corresponding to a number of major news topics: Business, Arts, Technology, Style, Books, Home and Garden, Sports, Science, and Health. Even when considering only 11 of many labels, most of the articles in our dataset had multiple tags, and some had many: the average number of articles per tag was 1.65, while the max was 6. Note that we also found that a fair number of the articles in the corpus were mislabelled see analysis for more detail. III. THE MULTI-LABEL PROBLEM A. Does it make sense to use Naive Bayes? Consider the following corpus of articles, corresponding to the following priors: label article content