Interactive alignment during dialogue : Implications for language processing architecture

We used event-related brain potentials (ERPs) to keep track of discourse-semantic computations involved in processing progressive constructions. The present study is based on the ‘recomputation hypothesis' discussed in Baggio & van Lambalgen (Journal of Semantics, forthcoming), where it was argued that sentences such as ‘The girl was writing a letter when her friend spilled coffee on the paper' are processed in two steps: (1) when the progressive clause is encountered, a model in which the goal state is attained (a complete letter) is computed; (2) when the subordinate clause is encountered, the initial discourse model is recomputed to the effect that, in the final discourse structure, the goal state does not hold. Critical sentences evoked larger sustained anterior negativities (SANs) compared to controls, starting around 400 ms after the onset of the sentence-final word and lasting for a few hundreds of milliseconds. The implications of our results for incrementality and monotonicity in language processing are discussed. ************************** Embodied sentence processing in the left inferior frontal lobe Annette Baumgaertner & Ferdinand Binkofski Apart from being increasingly implicated in higher motor control, Broca’s area is considered to play an important role in action understanding by coding the motor goal of an action. Moreover, recent findings suggest that the pars opercularis of Broca’s area may be able to code action content in a more abstract fashion, independently of specific movement parameters or effectors used. We performed event-related functional MRI to examine whether the processing of spoken sentences representing object-directed hand actions led to specific increases of BOLD signal change in left inferior frontal cortex. To control for action specificity, we presented sentences denoting inanimate motion events. In order to induce conceptual processing of the sentences, we interspersed few incongruent stimuli and asked participants to judge the acceptability of each sentence. There was a highly significant effect of sentence congruence: incongruent compared to congruent sentences resulted in an increase of relative BOLD signal change in pars triangularis extending to the anterior insula bilaterally. The magnitude of this effect did not differ between action and non-action sentences. Contrasting the (congruent) action with the (congruent) non-action sentences directly, we found left-lateralized activation predominantly in the pars opercularis of Broca’s region. These findings are consistent with accounts proposing that the formation of an action concept is closely linked to sensory and motor characteristics experienced during a particular action ('embodied neural theory of concepts', Gallese & Lakoff, 2005). Baumgaertner A, Buccino G, Lange R, McNamara A, Binkofski F (in press, OnlineEarly). Polymodal conceptual processing of human biological actions in the left inferior frontal lobe. European Journal of Neuroscience. Gallese, V. & Lakoff, G. (2005). The brain's concepts: the role of the sensory-motor system in conceptual knowledge. In Rumiati, R. & Caramazza, A. (eds), The multiple functions of sensory-motor representations. Psychology Press, New York, pp. 455-479. ************************** So how do you feel about this? An ERP study on opinion poll comprehension Jos van Berkum, Bregje Holleman, Jaap Murre, Mante Nieuwland, Marte Otten What happens in the brain when people have strong feelings about the text they read? To reduce complexity, neuro-imaging (EEG, fMRI) studies on text comprehension virtually always use decontextualized neutral bits of text, carefully crafted for listwise presentation to a cooperative but unavoidably also somewhat indifferent participant in the lab. Furthermore, although neurocognitive studies on attitudes and emotions often use language, the stimuli are usually single value-laden words presented as primes or targets in a word list. Therefore, very little is known about how the brain supports the comprehension of natural text touching upon the reader’s value system. We recorded EEG as people of radically different political orientation filled out a realistic opinion poll on societal matters (e.g., abortion, immigration, drugs, euthanesia, religion, premarital sex, gay marriage). The poll statements had been designed to be compatible or incompatible with average societal attitudes of members of a small Dutch Christian party. Examples: “A society that allows for abortion is a bad / good society”, “If my child were homosexual, I’d find this hard / easy to accept”, “I think the increasing emancipation of women is a negative / positive development”, or “Watching TV to relax is wrong / fine in my eyes”. In the EEG experiment, we presented these statements to members of this group as well as to people with on average very opposing attitudes, and we asked respondents to indicate their agreement on a four-point “agree” – “don’t agree” scale. Relative to an appropriate control, words that rendered the poll statement inconsistent with the respondent’s attitude elicited a positivity between 400-700 ms in ERPs, as people were reading the statement. This reveals that opinion poll statements incrementally unlock the relevant attitudes, and furthermore, that the neural processing of attitude-inconsistent words goes beyond what usually happens when words collide with the ‘cold’ interpretive context (as indexed by N400 effects). Our study also demonstrates that there is no reason to refrain from neuro-imaging language comprehension processes in situations where language matters. ************************** Predicting events according to a counterfactual world or the beliefs of others: Evidence of a gender bias in processing Heather J. Ferguson, Anthony J. Sanford and Christoph Scheepers The ability to update our current knowledge using contextual information is a vital process during every-day language comprehension. Counterfactual reasoning, an understanding of events that are counter to reality, is an essential ingredient of our everyday cognition. Similarly, the ability to understand and predict other peoples’ behaviour by attributing independent mental states to them is a uniquely human ability known as ‘theory of mind’. Numerous experiments have used the visual world paradigm to demonstrate that discourse processing is driven by predictive relationships involving syntax, semantics and real-world expectations. However, it is not clear whether these fundamental processing strategies can be influenced by introducing either a counterfactual context or by manipulating the beliefs of others. The issue we examined, therefore, was whether people can use their knowledge of the wider discourse to over-ride realworld knowledge to predict specific upcoming words as the current sentence is unfolding. Here we report two visual world studies investigating the extent to which the constraints of realworld knowledge and prior context influence eye-movements directed towards objects in a visual field. Experimental passages were presented aurally. In the counterfactual study, linguistic contexts were either counterfactual-world (CW), or real-world (RW). In the theory of mind study, contexts could involve an understanding of the beliefs of others or a reality judgement. In a final sentence, the target noun was manipulated to create RW-anomalous continuations, and RW-congruent continuations. Note that RW violations are congruent in a CW/ belief context, and vice-versa. The critical sentence in each passage was presented together with a picture showing, amongst other things, a RW-congruent referent and a RWincongruent referent. Relative proportions of anticipatory eye-movements to either of these referents (i.e. eyemovements that started reliably before the onset of the target word) indicated an initial contextdependent bias in both studies towards the contextually constrained referent. Shortly after the onset of the target word, however, the relevant ‘appropriate’ referent was visually favoured. This suggests that when visual information is available to reinforce linguistic input, participants expect a context-relevant continuation. Further analysis revealed a significant gender bias for anticipating the beliefs of others that was not present when predicting events in a counterfactual world. Females show very early context-biased expectancies on the theory of mind task, while males show no significant anticipation in this task. The results present the first on-line evidence to support a theory of a gender bias in cognition and provide the first evidence that real-world biases in the visual world paradigm can be ‘neutralised’ by a prior context. ************************** Detecting violations in realand counterfactualworld contexts: Eyemovements and ERP analysis Heather J. Ferguson, Anthony J. Sanford and Hartmut Leuthold During language comprehension, information about the world is exchanged and processed. Counterfactual reasoning, an understanding of events that are counter to reality, or false, is an essential ingredient of our everyday cognition. Counterfactual situations are frequently depicted through language, yet surprisingly little is known of how they are processed during reading or listening. Here we report a series of studies using eye-movements and event-related brain potentials (ERP) to investigate the comprehension of counterfactual information, using materials like those shown below. Experimental passages were created as shown below and were used in all experiments. Linguistic contexts were either counterfactual-world (CW) [1a], or real-world (RW) [1b] and the critical noun in sentence two was manipulated to create RW-anomalous continuations [2a], and RW-congruent continuations [2b]. Note that RW violations are congruent in a CW context, and vice-versa. [1a]. If cats were vegetarians they would be cheaper for owners to look after. [1b] If cats are hungry they usually pester