Visual perceptual assimilation of non-native speech

ionist models (D. Norris, 1994) (Dennis Norris & McQueen, 2008) propose that listeners store one representation – a canonical form of each word. Any variation in speaking rate has to be abstracted pre-lexically to access word meaning. These models assume that any signal variability is resolved before lexical meaning can be accessed. Signal degradation, such as due to segmental deletion in fast speech for example would increase cognitive load due to more challenging mapping between signal and representation. The theories assume that naïve Visual perceptual assimilation of non-native speech 151 listeners, who do not have representations of non-native speech, will be less able to map nonnative signals into existing representations. Exemplar models (e.g., (Goldinger et al., 1991) assume that storage of a sound pattern including exemplars produced by different speakers, with different rates, etc., form fuzzy categories for perception. Acoustic input is compared to sound representations. Variability due to speaking rate, speaker idiosyncrasies is not abstracted. Rather, spoken input is directly mapped onto the exemplars of fast or slow speech, for example, and compared to the best matching exemplars. The underlying assumption is that linguistic categories include a highly detailed collection of instances/exemplars and listeners can draw on knowledge of how fast or slow speech occurs. Indeed, in the current study participants performed better than chance with time-compressed native stimuli. This class of model generates predictions about the processing of non-native speech because it allows higher-order assumptions to play a role. For instance, in the study by Bosker and Reinisch (2015) listeners used pre-existing knowledge that non-native speech is typically slower than native speech, therefore presenting native and non-native speech at the same duration makes non-native input sound faster. However, the question remains if higher order assumptions about visual aspects of non-native speech. The results of the study failed to support for this idea, as participants were at chance for Russian time-compressed speech. Another class of models probabilistic models of speech perception, attempt to explain how listeners adapt to variability of a speech signal. These models, such as the Belief Updating Model of perceptual adaptation (Kleinschmidt & Jaeger, 2015) assume that listeners keep up to date with the consistencies of the acoustic signal, such as idiosyncratic pronunciations of certain sounds, speaking rate. Listeners then form adapted models of cue distributions that can be reapplied when encountering the same speaker, thus facilitating perceptual adaptation. An extension of the model to visual aspects of speech, would postulate a parallel process (or an integrated bimodal process) for the visual aspects of speech. The model is appealing because of its flexibility but there is little evidence from the current study that listeners were able to adapt their models of visual cue distribution to non-native speech. The Reverse Hierarchy Theory (RHT) of perceptual learning (Hochstein & Ahissar, 2002); (Ahissar & Hochstein, 2004; Ahissar et al., 2009) makes the assumption that perception and learning occurs in a top-down manner by default. Since lexical and semantic context is rarely accessible in non-native speech (with the exception of cognates), the theory would predict that low-level (sub-lexical) acoustic properties would be accessed first then mapped backwards in a top-down (lexical) manner. Similarly, the theory predicts that if listeners are presented with Visual perceptual assimilation of non-native speech 152 time-compressed native speech, lexical cues would not be available and sub-lexical cues will be accessed in a top-down manner. This theory also predicts a cost for backwards search and can account for discrepancy between performance for native and non-native time-compressed speech. Therefore, the finding of such differences here can be accommodated in part by RHT. Theoretical models make different assumptions about the perception and production of timemodified speech occurs. Dupoux and Green (Dupoux & Green, 1997) for example argue that speech processing occurs at a relatively abstract level of perception. Therefore errors may be constrained by top-down cognitive processes in speech comprehension and biased by native speech categories. Others, such as Cutler and Mehler (1993) argue that adaptation to timecompressed speech can occur for non-native speech but will depend on whether a language pair shares rhythmic properties (among others). In support of the latter argument, Sebastian– Galles and colleagues (Sebastián-Gallés et al., 2000; Sebastian-Galles & Soto-Faraco, 1999) manipulated different language pairs, such as Catalan-Spanish (similar in sound, lexicon, morphological system, and syntax), French-English (dissimilar in sound, lexicon, morphological system, and syntax), and Dutch-English (mixed). They showed that adaptation to time-compressed speech was language dependent and argued that speech representation is pre-lexical (intermediate between lexical and acoustic/phonetic levels) and speech adaptation depends on the contrast between language pairs at a pre-lexical level, including regularities in syllable stress. The findings here support the view that processing of time-compressed speech is language dependent. Moreover, the results suggest that naïve English participants do not extract visual features of an unknown language at the pre-lexical level. Other findings from this study contribute to understanding of non-native speech perception. Specifically, the hypothesis derived from previous studies of second-language lip-reading that sentences would be processed more readily in the visual domain compared to less informative units (words or single vowel/syllables) was supported. However, the results also contrast with findings from other studies. For example, Soto-Faraco and colleagues (Soto-Faraco et al., 2007) and Ronquest and colleagues (Ronquest et al., 2010) reported that participants could discriminate languages based on visual signals alone and discrimination was better for long sentences than for shorter ones. Ronquest and colleagues (2010) also showed that discrimination of languages based on visual input was better for sentences than for words. The current study failed to identify the same effects and participants could not discriminate English and Russian sentences based on visual-only input. The present results suggest that whereas Soto-Faraco and colleagues (2007) and Ronquest and colleagues (2010) found that rhythmic information facilitates discrimination, their results may depend on language familiarity or similarity. We assume that Australian English speakers do not know the subVisual perceptual assimilation of non-native speech 153 lexical features (e.g. rhythmic, stress and vowel information) of Russian and so they rely on lexical information to make decisions about whether speech is English or not. Several studies, including Soto-Faraco and colleagues (2007) and Ronquest and colleagues (2010) showed that this information is scarce, with the former study reporting that only 3% of lexical information (words) could be recalled from sentences and the latter showed that only a quarter of lexical content was extracted from native VO speech stimuli. The difficulties with visual lip-reading of words and sentences displayed by the participants here are also in line with results from studies highlighting high levels of task difficulty in lip reading for hearing participants (Bernstein, Demorest, & Tucker, 2000; Kim & Davis, 2004). Given that the present study failed to support the primary hypothesis, what might facilitate the process of native and non-native speech discrimination using visual-only aspects of speech? The current study hypothesised that the provision of a written cue/prime would result in better performance. Indeed, this hypothesis received some support from the results of Experiment 6 showing that performance for words and sentences was above chance if written cues/primes (cognates) were provided prior to VO presentation. Importantly, provision of a native cognate cue followed by non-native visual input resulted in a bias whereby participants incorrectly labelled the VO stimulus as native. The fact that discrimination was also above chance when native and non-native sentences were presented with a written English sentence also suggests that such stimuli provided sufficient information for discrimination. What, therefore, are the processing differences underlying cued and non-cued conditions? One model designed to account for an effect of printed prime cues is the bi-modal interactiveactivation (BIAM) model (Diependaele, Ziegler, & Grainger, 2010; Grainger & Ferrand, 1994; Grainger & Holcomb, 2009; Kiyonaga, Grainger, Midgley, & Holcomb, 2007, cited in Okano, Grainger, Holcomb, 2016). In this model a printed prime activates representations of sound at different level of speech processing including the level of phonemes and syllables (sub-lexical) and whole-words (lexical and semantic levels). This activation is bidirectional insomuch as hearing a word auditorily may also activate an orthographic word representation. Evidence for the BIAM model comes from a variety of cross-modal priming experiments (e.g. Grainger, Diependaele, Spinelli, Ferrand, & Farioli, 2003; Kiyonaga et al., 2007, cited in Okano et al, 2016) showing that related primes (repetitions) result in better identification. The facilitatory effect varies according to type of stimuli and prime duration. Whilst the BIAM model does not address VO speech, we can conjecture that orthographic representations activate auditory representation and perhaps also articulatory and visual representations. Indeed, Lansing and Helgeson (1995) demonstrated that when a written word (semantically related or not) was followed by VO speech