Language and cognitive load in a dual task environment

Language and cognitive load in a dual task environment Nikolaos Engonopoulos (nikolaos.engonopoulos@uni-potsdam.de) Potsdam University, Department of Linguistics Karl-Liebknecht-Str. 24-25, 14476 Potsdam, Germany Asad Sayeed (asayeed@coli.uni-saarland.de) and Vera Demberg (vera@coli.uni-saarland.de) Cluster of Excellence, Saarland University, Campus C7.4, 66123 Saarbr¨ucken, Germany Abstract Background and Related Work We investigate the effect of linguistic complexity on cogni- tive load in a dual-task scenario, namely simultaneous driv- ing and language use. To this end, we designed an experiment where participants use a driving simulator while listening to spoken stimuli and answering comprehension questions. On- line physiological measures of cognitive load, including the re- cently established Index of Cognitive Activity, as well as mea- sures of performance in both tasks have been collected with high temporal resolution. The resulting aligned data streams can be used to test a vast array of different hypotheses about the relationship between performance, difficulty, and cognitive load in dual tasks at various levels of temporal resolution and linguistic structure. We present results of the data analysis, in- cluding evidence that different linguistic structures may cause measurable changes in cognitive workload on a very fine tem- poral scale in cases of increased primary task difficulty. Keywords: relative clause; dual task; cognitive load; pupil- lometry; skin conductance; tracking task; driving; multi- tasking There is a rich literature on language use while driving a car, largely showing that speaking on the telephone has a negative effect on driving performance (Just et al., 2008; Kubose et al., 2006). Further studies found that this is specific to conversa- tions with remote speakers (independent of whether one uses a hand-held device or free speaking), but that conversations with an in-car passenger are less problematic (Strayer, Drews, & Johnston, 2003; Drews, Pasupathi, & Strayer, 2004). It appears that passengers adapt their conversation to the traf- fic situation, leaving the driver more resources to deal with demands of the driving task when driving becomes difficult (Drews, Pasupathi, & Strayer, 2008; Crundall, Bains, Chap- man, & Underwood, 2005; Villing, 2009). By contrast, re- mote conversational partners cannot adapt their speech, so that the driver may reach the point of cognitive overload more easily and thus commit driving errors. However, these lines of research have not taken into account how the fine-grained details of linguistic complexity affect cognitive load and driv- ing task performance. On the other hand, there is a very rich literature on linguis- tic processing difficulty in single tasks using brain imaging, ERPs, and reading time studies, as well as a number of dual task experiments generally showing that performance on the linguistic task deteriorates with increased complexity of the other task, see for example King and Just (1991). Finally, multiple models explain the effect of cognitive load in one task on performance in another (Baddeley, 2003; Wickens, 2008; Just, Carpenter, & Miyake, 2003). We see, however, unbroken ground in relating the effect of linguistic complexity on a realistic task (e.g., driving) and the size of the interference of linguistic processing with driving performance. This study takes a step in this direction in test- ing different methods for assessing cognitive load and the ef- fect of one particular linguistic structure—incrementally am- biguous relative clauses—on driving performance in a sim- plified but controllable and continuous driving task. Introduction Is there a relationship between psycholinguistic measures of language complexity and quantified cognitive workload in dual-task environments? To answer this question, we exper- imentally evaluate these measures of language processing in an environment where one task is language-related and the other not. Such language complexity measures have been shown in single-task studies to account for processing diffi- culty. This work represents a first step in which we inves- tigate the effect of a grammatical structure (German locally ambiguous subject vs. object relative clauses) on a simplified, well-controlled non-linguistic task, a driving task. Dual tasks are ubiquitous in everyday life, often in situa- tions where attention and performance in the primary task is critical. An example is driving while engaging in dialogue, be it with a passenger, a dialogue-controlled interface, or re- motely via mobile phone. Engaging in dialogue generally af- fects driving performance and safety (Just, Keller, & Cynkar, 2008; Young, Regan, & Hammer, 2007). We manipulated the driving task difficulty and the struc- tural complexity of the linguistic items. We also collected measurements of performance in both tasks and fine-grained physiological indicators of cognitive load, namely skin con- ductance levels and pupil sizes. We computed values from pupil size for the recent Index of Cognitive Activity (ICA). To our knowledge, this is the first study using the ICA mea- sure in a setting with a language task. The dual-task experiment The ConTRe task Our primary task was a tracking task (Jagacinski & Flach, 2003) presented as a car driving scenario and called the “Con- tinuous Tracking and Reaction” (ConTRe) task (Mahr, Feld,