A Two-Dimensional Topic-Aspect Model for Discovering Multi-Faceted Topics

This paper presents the Topic-Aspect Model (TAM), a Bayesian mixture model which jointly discovers topics and aspects. We broadly define an aspect of a document as a characteristic that spans the document, such as an underlying theme or perspective. Unlike previous models which cluster words by topic or aspect, our model can generate token assignments in both of these dimensions, rather than assuming words come from only one of two orthogonal models. We present two applications of the model. First, we model a corpus of computational linguistics abstracts, and find that the scientific topics identified in the data tend to include both a computational aspect and a linguistic aspect. For example, the computational aspect of GRAMMAR emphasizes parsing, whereas the linguistic aspect focuses on formal languages. Secondly, we show that the model can capture different viewpoints on a variety of topics in a corpus of editorials about the Israeli-Palestinian conflict. We show both qualitative and quantitative improvements in TAM over two other state-of-the-art topic models. Probabilistic topic models such as LDA (Blei, Ng, and Jordan 2003) have emerged in recent years as a popular approach to uncovering hidden structures in text collections, and offer a powerful way to represent the content of documents. These models, however, typically learn distributions over words along only a single dimension of topicality, and ignore the fact that words may fall along other dimensions such as sentiment, perspective, or theme. Some work has been done to simultaneously model both topics and other types of groupings. For example, in the topic and perspective model (Lin, Xing, and Hauptmann 2008), each word is modeled as having some weight of topicality and perspective (e.g., liberal or conservative), however, this model assumes that all documents are about the same topic. The topic-sentiment mixture model (Mei et al. 2007) models each document as both a mixture of topics and a mixture of different sentiments (i.e. negative/positive), however, words come from either the topic model or the sentiment model rather than a combination of both. In these approaches, there is no inter-dependency of topics and perspectives, and they cannot capture how these perCopyright c  Recently we have presented a new model, cross-collection latent Dirichlet allocation (ccLDA) (Paul and Girju 2009a), which can both discover topics among multiple text collections as well as differences between them, and we used this to capture the perspectives of different cultures in blogs. Each topic is associated with a probability distribution over words that is shared across all collections, as well as a distribution that is unique to each collection. For example, the topic of FOOD found in documents from different countries might contain the words food and eat among all collections, but curry would be more likely to appear in the India collection. In many settings, however, it is more realistic to assume that a document is a mixture of such aspects, rather than belonging to exactly one aspect. For example, a recipe for curry pizza would contain elements from both Indian and American food, rather than strictly one or the other. In this paper we introduce a novel topic model, TAM, which not only allows documents to contain multiple aspects, but it can learn these aspects automatically. Unlike ccLDA, the model can be applied to a single collection and can discover patterns without document labels. A common application of topic models is topic discovery in scientific literature (Griffiths and Steyvers 2004), which is useful for browsing large collections of literature. Topic models can also be used to assign research papers to reviewers (Mimno and McCallum 2007). In computational linguistics, (Hall, Jurafsky, and Manning 2008) and (Paul and Girju 2009b) model topics in this field and study their history. These studies, however, have ignored the multi-faceted and interdisciplinary nature of many scientific topics. The only work in this direction we are aware of is our recent work (Paul and Girju 2009b) where we model scientific literature from multiple disciplines such as computational linguistics and linguistics. However, in that approach the fields are modeled independently, whereas TAM incorporates this directly into the model. In this paper we show how TAM can be used for the discovery of multi-faceted scientific topics. Additionally, we model a corpus of editorials on the Israeli-Palestinian conflict. We improve upon studies of this corpus (Lin et al. 2006; Lin, Xing, and Hauptmann 2008) by modeling how different perspectives on this issue affect multiple topics within the data.