A Multimodal Investigation of Recognition Performance for Target-Aligned but Irrelevant Stimuli

A Multimodal Investigation of Recognition Performance for Target-Aligned but Irrelevant Stimuli Andrew D. Dewald (adewald@hawaii.edu) Department of Psychology, University of Hawaii at Manoa 2530 Dole Street, Honolulu, HI 96822 Scott Sinnett (ssinnett@hawaii.edu) Department of Psychology, University of Hawaii at Manoa 2530 Dole Street, Honolulu, HI 96822 Abstract Overtly presented, but ignored visual and auditory stimuli presented within the same sensory modality are inhibited in a later recognition task if previously presented synchronously with an attended visual target (Tsushima, Sasaki & Watanabe, 2006; Tsushima, Seitz & Watanabe, 2008; Dewald, Doumas & Sinnett, 2010; Dewald & Sinnett, 2011). We extend these findings to conditions in which task irrelevant stimuli (written or spoken words) were presented in a separate sensory modality than task-relevant targets (picture or sound repetitions). A subsequent recognition task was given for the previously presented irrelevant stimuli (words). Words that had been simultaneously presented with a target in the previous repetition detection task were later recognized at chance levels, demonstrating a bolstered recognition of task- irrelevant items (e.g. target-aligned words) when compared with performance under unimodal presentation. Key words: Attention, Multimodal Presentation, Recognition Introduction It has been demonstrated throughout cognitive psychology’s history that attention is a limited resource (Broadbent, 1954; Cherry, 1953; James, 1890; Mack & Rock, 1998; Rees, Russell, Frith, & Driver, 1999; Sinnett, Costa & Soto- Faraco, 2006; Triesman, 1960). Interestingly, the capacity of the attentional system seems to be modulated if a difficult unisensory task is divided across multiple sensory modalities (i.e., a multiple resources theory, see Wickens, 1984). For instance, Sinnett et al. (2006) showed that under multimodal presentations, inattentional blindness for words was ameliorated (i.e., perception improved) when compared with unimodal conditions, regardless of the modality of word presentation (see also Toro, Soto-Faraco, & Sinnett, 2005 for a similar example involving statistical learning). These findings seem to provide support for an attentional system that is segregated, such that each sensory modality has access to individualized attentional resources (see Wickens, 1984). Providing support for a multiple resource view, Duncan, Martens, and Ward, (1997) demonstrated that participants had difficulty identifying target items presented 300 ms or less after previous targets under unimodal presentations (auditorily or visually), due to the well-documented phenomenon termed the attentional blink (AB; see Shapiro, 1992). However, when targets were presented across sensory modalities (that is, when first identifying a visual target and then an auditory target, or vice versa), typical AB effects were not observed. That is, participants were able to detect an auditory or visual target even when it was immediately preceded by a target occurring in the other modality (i.e., visually or auditorily respectively), whereas they were unable to do so under unimodal presentations. This finding suggests that attentional costs are reduced if a difficult task is divided across sensory modalities. Further investigating this issue, Sinnett et al. (2006) explored a multiple resource view of attention by means of an inattentional blindness (IB) paradigm. IB is a well- studied phenomenon in attention research that illustrates a situation in which an individual fails to detect an explicitly presented event due to attention being directed elsewhere (Mack & Rock, 1998; Simons & Chabris, 1999). The overwhelming conclusion of IB research is that if attention is deployed to a difficult or demanding primary task, information not relevant to the primary task goes unprocessed, at least not to levels of explicit awareness. A curious finding by Sinnett et al. (2006), however, was that when irrelevant information was presented in a sensory modality separate from the target information, levels of IB was reduced for irrelevant items (i.e., performance improved when compared to unimodal presentations). Sinnett et al. (2006) utilized an IB paradigm (see also Rees et al., 1999) incorporating multisensory presentations in which participants detected immediate repetitions in a stream of rapid serially presented items in either visual, auditory, or bimodal conditions. In the unimodal conditions (visual only or auditory only streams) the primary task was to monitor either pictures (or sounds) presented simultaneously with written words (or spoken, respectively) and detect immediate repetitions in either the word or the distractor stream. Immediately following this task, a surprise word recognition test was administered for unattended words (after having attended to pictures or sounds). In the crossmodal condition words were presented either visually or auditorily, with a distracting stream of either overlaid/superimposed sounds or pictures in the opposite modality. Unimodal performance (both auditory