Interactivity during spoken word recognition: Evidence from bimodal bilinguals

Interactivity During Spoken Word Recognition: Evidence from Bimodal Bilinguals Anthony Shook (a-shook@northwestern.edu) Department of Communication Sciences and Disorders, 2240 Campus Drive Evanston, IL 60208 USA Viorica Marian (v-marian@northwestern.edu) Department of Communication Sciences and Disorders, 2240 Campus Drive Evanston, IL 60208 USA simultaneously activated. This effect appears to be bottom- up in nature – as auditory input enters the language system, it non-selectively activates lexical items in both languages based on structural overlap. Critically, a dual-language bottom-up pathway cannot exist in bimodal bilinguals, as their languages do not utilize the same structural input. The cross-modal nature of bimodal bilingualism therefore allows for the direct investigation of top-down mechanisms in isolation. If bimodal bilinguals co-activate their two languages in the absence of form overlap, it would suggest that language co-activation can be driven by top-down information, and would require a system capable of activating the non-target language via top-down or lateral connections. Models that consider exclusively bottom-up information for lexical activation or selection (such as the Shortlist Model, Norris, 1994) are unlikely to be able to explain this result, as they limit activation to items that exist in the same modality as the target. Therefore, a bimodal bilingual, when faced with single-modality input (e.g., spoken English), should not activate the signed language. However, recent research indicates that bimodal bilinguals do co-activate their languages. For example, hearing ASL-English bilinguals produce speech and signs simultaneously (Emmorey, Borinstein, Thompson, & Gollan, 2008), deaf ASL-English bilinguals show interference from sign-language while processing written- English (Villwock, Wilkinson, Bailey, Kroll, Morford, & Pinar, 2009), and late-learning Dutch-Sign Language of the Netherlands bilinguals show interference from English while processing SLN signs (Van Hell, Ormel, van der Loop, & Hermans, 2009). In addition to clarifying the role of top-down mechanisms in language processing, language co-activation in bimodal bilinguals would suggest that linguistic information is readily transmitted across modalities, such that two unrelated languages can be activated simultaneously, even when phonological information from one of the two languages is absent. The current study used an adapted visual world paradigm to examine parallel language processing in normal-hearing, ASL-English bilinguals. Investigating whether languages that do not share modality are co-activated in bimodal bilinguals can provide insight into the influence of top-down mechanisms on language processing and the architecture of the language system in general, as well as reveal the extent to which linguistic information is modality independent. Abstract We explore the role of top-down information in language processing by investigating parallel language activation in bimodal bilinguals, who are fluent users of a spoken and a signed language. In an eye-tracking study, bimodal bilinguals showed activation of their signed language while receiving input in English only. Since spoken and signed languages do not share structure, the results suggest that linguistic information can be readily transmitted across modalities, and that parallel language activation can be driven by top-down processes. Keywords: bilingualism; ASL; language co-activation; top- down processing; eye-tracking; visual world paradigm Introduction The architecture of the language system is determined by the way that information flows among levels of processing. Language processing may involve both bottom-up/feed- forward and top-down/feed-back mechanisms (Rapp & Goldrick, 2000; Navarette & Costa; 2009). However, exclusively feed-forward systems may also be capable of explaining language processing without the aid of feed-back mechanisms (Hagoort & Levelt, 2009; Levelt, Roelofs, & Meyers, 1999; McQueen, Jesse, & Norris, 2009; Norris, 1994). Proponents of language systems that recruit top- down mechanisms face the difficulty of separating the impact of top-down information from that of bottom-up information. When both forms of information are present, it is difficult to disentangle the unique contributions each may make to language processing. To understand the role of top- down mechanisms during language processing, the influence of top-down pathways must be measured in isolation. One possible way of limiting the impact of bottom-up information is by investigating language processing in bimodal bilinguals. Unlike unimodal bilinguals, who use two spoken languages, bimodal bilinguals are fluent in a spoken and a signed language. Research on unimodal bilinguals has revealed non-selective language effects, wherein unimodal bilinguals activate both of their languages in parallel (Blumenfeld & Marian, 2007; Marian & Spivey, 2003; Weber & Cutler, 2004). For example, when a Russian- English bilingual hears the English word “marker,” she will also make eye movements to items that are phonologically similar in the non-target language (e.g., Russian) such as “marka” (stamp), suggesting that her Russian is