Error-Driven Adaptation of Higher-Level Expectations During Reading

Error-Driven Adaptation of Higher-Level Expectations During Reading Thomas A. Farmer 1 (thomas-farmer@uiowa.edu), Alex B. Fine 2 (abfine@illinois.edu), 1 Shaorong Yan (shaorong- yan@uiowa.edu), 3 Spyridoula Cheimariou (spyridoula-cheimariou@uiowa.edu) & T. Florian Jaeger (fjaeger@bcs.rochester.edu) Department of Psychology, University of Iowa, Iowa City, IA 52242 USA Department of Psychology, University of Illinois at Urbana-Champaign, Champaign, IL 61820 USA Department of Communication Sciences and Disorders, University of Iowa, Iowa City, IA 52242 USA Department of Brain & Cognitive Sciences, University of Rochester, Rochester, NY 14627 USA Abstract probabilistically associated with words from an expected category) of an upcoming word. As the incoming signal flows from lower-level sensory processes to progressively higher levels of analysis (more abstracted away from the physical input), it is assessed with respect to progressively higher-level expectations that are generated via context. Any mis-match between the properties of the signal and the prediction (at any level) will generate a prediction error. These “error signals” contain information about the difference between what was predicted and the structure of the input contained in the arriving signal. The error signal feeds forward to higher levels of representation, potentially facilitating an adjustment of higher-level expectations such that the predictions generated in the future may be more precise (see also Chang, Dell, & Bock, 2006; Fine & Jaeger 2013; Jaeger & Snider, 2013). 1 Fine et al. (2013) recently demonstrated that readers continually adapt their syntactic expectations in order to accurately approximate the distributions of syntactic structures in a given communicative context. Here, we examine patterns of eye movements as subjects read sentences that contain an atypical distribution of syntactic structures to gain more fine-grained insight into the time-course and nature of this adaptive process. An adaptation effect was only elicited on a late measure— second-pass reading times—consistent with the claim that expectation adaptation to an atypical distribution of syntactic structures occurs at a higher level that is abstracted away from the physical properties of the visual input. Keywords: Language Comprehension; Eye Movements; Prediction; Adaptation; Learning; Reading Introduction Based on a lifetime of experience, adults possess a wealth of knowledge about the structure of language and about how linguistic events are typically distributed within a specific communicative context. Language comprehension arises by synthesizing this stored knowledge with the information available in the linguistic stimulus as it unfolds over time. Stated alternatively, prior knowledge about contextualized language facilitates the generation of expectations about what types of information are likely to be contained in the to-be-encountered portion of an unfolding linguistic signal. Consistent with such a knowledge-driven model, it is widely recognized that readers and listeners use linguistic, visual, and social contexts to generate predictions for many aspects of the incoming linguistic input during on-line comprehension, from grammatical properties of the signal (e.g. Arai & Keller, 2013) to semantic properties of a word (e.g. DeLong, Urbach, & Kutas, 2005; Kamide, Altmann, & Haywood, 2003), all the way down to very low-level form- based perceptual properties of the physical input (e.g. Dikker, Rabagliati, Farmer, & Pylkkanen, 2010). For example, upon encountering a single decontextualized sentence such as, The child saved the … a reader isn’t likely to accurately anticipate the precise word-form that they are about to next encounter (the context provided by this isolated sentence is not constraining enough), but can generate expectations for, and thus pre-activate representations for, syntactic information (a noun is likely to appear), semantic features (savable things), and physical form-based properties (physical information Syntactic Expectation Adaptation One consequence of this error-driven fine-tuning process is that it affords communicators the ability to rapidly adapt their expectations about properties of the multi-dimensional linguistic signal. Why is this important? Language use is highly variable, from properties of the speech signal, to lexical use, to the distribution of syntactic structures across, for example, different genres and regions. Error-driven fine- tuning of expectations affords readers and listeners the ability to deal with variability in the input. It fosters adaptation by providing a means through which to continuously update the expectations that are generated within a communicative context, such that comprehenders can more accurately anticipate aspects of the signal across multiple levels of analysis over the course of experience. Expectation adaptation has been demonstrated across a wide range of perceptual and motoric tasks (Koerding & Wolpert, 2004; Kohn, 2007). From the hierarchical predictive processing account detailed above, however, expectations, and prediction errors associated with violations of them, should also occur at higher levels of analysis. As a result, we should be able to observe evidence of adaptation to those higher-level error signals as well. Indeed, Fine, Jaeger, Farmer, & Qian (2013) recently See Clark (2013) for a detailed perspective on “hierarchical predictive processing” (see also Farmer, Brown, & Tanenhaus, 2013, for a discussion of on-line language comprehension within such a framework).