A Bayesian Model of Human Sentence Processing

Language comprehension is a classic problem of reasoning under uncertainty . La guage comes to us as a noisy, unsegmented, ambiguous mass of auditory waveforms or visual stimul i. Humans must somehow combine this input with other knowledge we have to come up with reasonable interpretation s and actions. How might humans address this problem of decision-making under uncertainty? The best normative model we have for solving problems of this sort is probability theory, which offers a principled method with a co herent semantics for weighing and combining evidence. Whether this normative model is the correct descriptive model fo r al of human behavior has recently been the subject of much debate (Kahnema; Gigerenzer). While this debate is not res olved for all areas of human cognition and reasoning, the last decade or so had produced emerging consensus through out the cognitive sciences that in some areas human cognition is likely to make use of probabilistic models. Th e seminal work of Anderson (1990) gave Bayesian underpinnings to cognitive models of memory, categorization, and causation, and recent Bayesian models of human cognition include work in human visual processing (Rao et al. 20 01; Weiss & Fleet 2001), categorization (Tenenbaum, 2000; Tenenbaum & Griffiths, 2001b, 2001a), and the human u nderstanding of causation (Rehder, 1999; Glymour & Cheng, 1998). Together, these ideas suggest that perhaps the p rocess of human language comprehension is also best modeled as a process of probabilistic, Bayesian reasoning. This idea that human processing of language draws on probabilistic mo dels is hardly novel. (Schuchardt, 1885), in his arguments against the 19th century Neogrammarians, point out that ke y role of frequency in language production and language change. Schuchardt noted that word frequency is a good predictor of which words are phonologically weakened or ‘lenited’. Words which are more frequent tend to be shorter and p honologically simplified; (Zipf, 1929) pointed out that this reduction of frequent forms also happened for frequ ent phones. (Jespersen, 1922) expanded Schuchart’s idea from pure frequency to predictability or probability . Jespersen pointed out that the predictability of the word in its context, in addition to its raw frequency, must play a factor in the phonological form of the word. These early intuitions about frequency and probability were all related to language production. Evidence for the role of frequency and probability specifically in language comprehensi on processing dates quite a bit later, from the mid 20th century. In the 1950’s, for example, Davis Howes showed that word frequency plays a key role in comprehension in both the visual and auditory domains (Howes & Solomo n, 1951; Howes, 1957). Throughout the second half of the 20th century, evidence amassed that high frequency words are accessed more quickly, they are accessed more easily, and they are accessed with less input signal than low-frequ ency words. This is a very robust effect, supported by tachistoscopic recognition Howes and Solomon (195 1), naming (Forster & Chambers, 1973), lexical decision (Rubenstein, Garfield, & Millikan, 1970; Whaley, 1978 ; Balota & Chumbley, 1984), recognition accuracy and errors in noise (Howes, 1957; Savin, 1963), and gating (Gr osjean, 1980). The last two decades of behavioral research have extended these lexical results to other areas of psycholinguistics such as sentence processing. We know that many kinds of probabilistic kn owledge play a role in the comprehension of sentences. One such factor is the probability of the different lexical cat egories of a word. For example the a priori probability that the ambiguous word fires is a noun, or alternatively a verb, plays a role in sentence comprehension, as does the probability that the word selectedis a preterite or a participle (Burgess & Hollbach, 1988; Trueswell,