The syntactic topic model (STM) is a Bayesian nonparametric model of language that discovers latent distributions of words (topics) that are both semantically and syntactically coherent. The STM models dependency parsed corpora where sentences are grouped into documents. It assumes that each word is drawn from a latent topic chosen by combining document-level features and the local syntactic context. Each document has a distribution over latent topics, as in topic models, which provides the semantic consistency. Each element in the dependency parse tree also has a distribution over the topics of its children, as in latent-state syntax models, which provides the syntactic consistency. These distributions are convolved so that the topic of each word is likely under both its document and syntactic context. We derive a fast posterior inference algorithm based on variational methods. We report qualitative and quantitative studies on both synthetic data and hand-parsed documents. We show that the STM is a more predictive model of language than current models based only on syntax or only on topics. When we read a sentence, we use two kinds of reasoning: one for understanding its syntactic structure and another for integrating its meaning into the wider context of other sentences, other paragraphs, and other documents. Both mental processes are crucial, and psychologists have found that they are distinct. A syntactically correct sentence that is semantically implausible takes longer for people to understand than its semantically plausible counterpart (Rayner et al. 1983) . Furthermore, recent brain imaging experiments have localized these processes in different parts of the brain (Dapretto and Bookheimer 1999) . Both of these types of reasoning should be accounted for in a probabilistic model of language. To see how these mental processes interact, consider the following sentence from a travel brochure, Next weekend, you could be relaxing in ____. How do we reason about filling in the blank? First, because the missing word is the object of a preposition, it should act like a noun, perhaps a location like "bed," "school," or "church." Second, because the document is about travel, we expect travel-related terms. This further restricts the space of possible terms, leaving alternatives like "Nepal," "Paris," or "Bermuda" as likely possibilities. Each type of reasoning restricts the likely solution to a subset of words, but the best candidates for the missing word are in their intersection. In this article we develop a probabilistic model of language that mirrors this process. Probabilistic modeling has emerged as a powerful formalism for expressing assumptions * 3155 AV Williams,
Also, REF proposed the Syntactic Topic Model whose goal is to integrate the text semantics and the syntax in a non-parametric topic model.
Syntactic Topic Models