Modality switch effects emerge early and increase throughout conceptual processing: Evidence from ERPs
Creators: name: Bernabeu, Pablo
name: Willems, Roel
name: Louwerse, Max
Publisher: name: figshare
Year: 2017
Subjects: subject: Neuroscience and Physiological Psychology
subject: 179999 Psychology and Cognitive Sciences not elsewhere classified
subject: FOS: Psychology
subject: FOS: Psychology
subject: 170299 Cognitive Science not elsewhere classified
subject: 170199 Psychology not elsewhere classified
subject: 170110 Psychological Methodology, Design and Analysis
subject: 170204 Linguistic Processes (incl. Speech Production and Comprehension)
subject: Language

description: Original conference paper in <i>Proceedings of the 39th Annual Conference of the Cognitive Science Society.</i>Files: paper, poster, design overview, stimuli, EEG montage used, waveforms, window 1 close-ups, subtraction topographies, detailed waveforms, critical results, fixed effects of final models, entire modeling, raw data. For best resolution, download image(s). Waveforms per participant and electrode (link). <b>OSF archive with all data and code (link).</b><b>Abstract </b>We tested whether conceptual processing is modality-specific by tracking the time course of the Conceptual Modality Switch effect. Forty-six participants verified the relation between property words and concept words. The conceptual modality of consecutive trials was manipulated in order to produce an Auditory-to-visual switch condition, a Haptic-to-visual switch condition, and a Visual-to-visual, no-switch condition. Event-Related Potentials (ERPs) were time-locked to the onset of the first word (property) in the target trials so as to measure the effect online and to avoid a within-trial confound. A switch effect was found, characterized by more negative ERP amplitudes for modality switches than no-switches. It proved significant in four typical time windows from 160 to 750 milliseconds post word onset, with greater strength in posterior brain regions, and after 350 milliseconds. These results suggest that conceptual processing may be modality-specific in certain tasks, but also that the early stage of processing is relatively amodal.<b></b><b></b><b></b><b></b><b></b><b></b>ReferencesCollins, J., Pecher, D., Zeelenberg, R., &amp; Coulson, S. (2011). Modality switching in a property verification task: an ERP study of what happens when candles flicker after high heels click. <i>Frontiers in Psychology, 2.</i>Hald, L. A., Marshall, J.-A., Janssen, D. P., &amp; Garnham, A. (2011). Switching modalities in a sentence verification task: ERP evidence for embodied language processing. <i>Frontiers in Psychology, 2.</i> Hauk, O. (2016). Only time will tell—Why temporal information is essential for our neuroscientific understanding of semantics. <i>Psychonomic Bulletin &amp; Review, 23</i>.<i></i>Louwerse, M., &amp; Connell, L. (2011). A taste of words: linguistic context and perceptual simulation predict the modality of words. <i>Cognitive Science, 35, </i>2, 381-98.Mahon, B.Z., &amp; Hickok, G. (2016). Arguments about the nature of concepts: Symbols, embodiment, and beyond. <em>Psychonomic Bulletin &amp; Review, 23, </em>941-958.