Lexical information drives perceptual learning of distorted speech : evidence from the comprehension of noise-vocoded sentences

Speech comprehension is markedly resistant to acoustic distortion in the input, reflecting listeners’ ability to adjust their perceptual system to the properties of distorted speech. We tested native English speakers’ comprehension of noise-vocoded (NV) sentences; a manipulation that removes spectral detail from speech (Shannon, Zeng, Kamath, Wygonski & Ekelid, 1995). With simple presentation, listeners’ sentence-report scores improved from near 0 to 70% correct over 30 NV sentences (Experiment 1). Learning was enhanced if listeners heard distorted sentences while they knew the identity of the undistorted target (Experiments 2 and 3). Learning was absent when listeners were trained with nonword sentences (Experiment 4), even if written feedback was provided to reduce short-term memory load for nonword sentences (Experiment 5). These results suggest that learning to understand NV speech depends on the availability of higher-level information, consistent with a top-down, lexically driven learning process. Similar processes may facilitate comprehension of speech in an unfamiliar accent, or following cochlear implantation. Implications for cognitive models of spoken language comprehension are discussed. Davis Lexical information drives perceptual learning Page 3 We are able to understand speech in a variety of situations that dramatically affect the sounds that reach our ears. We can understand talkers with quite different (foreign or regional) accents, who speak at different speeds, in rooms that introduce different amounts of reverberation, or when the speech is conveyed over low-fidelity devices such as the telephone. The robustness of speech comprehension to many forms of variation and distortion is currently unmatched by computer speech recognition systems, and may therefore reflect a unique specialization of the human perceptual system. Experimental work has demonstrated that speech perception remains robust even when challenged with extreme forms of artificial distortion. For example, speech remains understandable when formants are resynthesised as sinusoids (Remez, Rubin, Berns, Pardo, & Lang, 1994; Remez, Rubin, Pisoni, & Carrell, 1981), a manipulation that removes most of the natural qualities of the human voice from a speech signal. More recent demonstrations have shown that dramatic alterations to both the temporal (Mehler et al., 1993; Saberi & Perrott, 1999) and spectral (Shannon, Zeng, Kamath, Wygonski, & Ekelid, 1995; Smith, Delgutte, & Oxenham, 2002) properties of speech do not substantially impair the intelligibility of spoken language when background noise is absent. It is unlikely that there is any single set of acoustic properties or cues that are preserved in all of these different forms of distorted yet still-intelligible speech (cf. Bailey & Summerfield, 1980). Therefore, robustness in speech perception must, in part, reflect the multiple acoustic means by which stable elements of speech (such as phonetic features or syllables) are coded in clear speech: this redundancy permits comprehension when any single cue is lost. Robustness in speech comprehension may also derive from the operation of compensatory mechanisms that are recruited when speech becomes difficult to understand: processes of adaptation and perceptual learning are two such mechanisms. Davis Lexical information drives perceptual learning Page 4 The human language system can dynamically adapt to variation in the acoustic realisation of speech: tuning the perceptual system so as to optimally process the current speech input. These adaptation processes can take place very rapidly. For example, adaptation to natural changes in speech rate (e.g. Miller, 1981; Miller & Lieberman, 1979; Summerfield, 1981) or to changes in the spectral characteristics of the communication channel (Ladefoged & Broadbent, 1957; Summerfield, Haggard, John, & Gray, 1984; Watkins, 1991) occur in less than a second. Such adjustments occur relatively automatically and largely without listeners being aware of the perceptual consequences of their operation. These rapid adaptation processes may be limited to forms of variation that can be compensated for by applying existing knowledge of the nature of the acoustic environment or speech signal. More extreme or unnatural forms of distortion can require longer periods of exposure for listeners to achieve full and effortless comprehension. For example, adaptation to artificial changes in speech rate that are outside the normal range (as in time-compressed speech) may require several minutes of compressed input (Mehler et al., 1993; Pallier, Sebastian-Galles, Dupoux, Christophe, & Mehler, 1998). We consider that a form of longterm learning is required for listeners to change their perception in such cases, as evidenced by the fact that trained listeners outperform naïve listeners when tested one year after their original exposure to time-compressed speech (Altmann & Young, 1993). These longer-term adaptation processes may not be limited to artificially degraded speech, but may also be invoked in any situation in which listeners are faced with speech sounds that are outside the range that they have previously experienced. For instance, when presented with heavily accented speech, listeners must adjust to a set of unfamiliar phonemic and prosodic properties. Experiments have demonstrated that effective perception of speech in an unfamiliar accent can require several minutes or more of exposure to allow full comprehension (Bent & Bradlow, 2003; Clarke, 2002; Weill, 2001). Experimental Davis Lexical information drives perceptual learning Page 5 investigations of the role that learning plays in the comprehension of accented speech require participants to be unfamiliar with a particular accent that they may, in fact, have encountered in their day-to-day life. The use of artificial distortions that simulate such real-world variability ensure that all participants are equally naïve. In a recent series of experiments, Norris and colleagues (Norris, McQueen, & Cutler, 2003) exposed listeners to artificially modified speech in which an ambiguous fricative replaced all occurrences of either /f/ or /s/. This manipulation provides an effective simulation of a speaker producing speech that contains phonological boundaries placed in an unfamiliar location, such as might occur in accented speech. Following exposure to 20 words that contained this ambiguous fricative, listeners showed a marked difference in their subsequent perception of /s/ and /f/, depending on which phoneme had been altered. Interestingly, changes to phoneme boundaries were only observed for listeners who were exposed to the ambiguous fricative in the context of real words, but not in nonwords. Lexical knowledge seems to play a crucial role in learning the correct interpretation of ambiguous fricatives. Norris and colleagues concluded that their result illustrates an important role for top-down connectivity in models of speech perception, in enabling lower-level processes to adapt to unusual forms of speech input. However, it is not clear that all forms of adaptation are dependent on lexical information. For instance, listeners’ perception of time-compressed speech improves even when trained with “jabberwocky” sentences in which content words are replaced with nonwords (Altmann & Young, 1993; Mehler et al., 1993). Furthermore, listeners show improved comprehension of time-compressed sentences in their own language after first hearing timecompressed sentences in a language that is phonologically similar to the target language but that they do not understand (Pallier et al., 1998; Sebastian-Galles, Dupoux, Costa, & Mehler, 2000). In both of these cases, lexical information does not appear to be required for Davis Lexical information drives perceptual learning Page 6 adaptation to occur. Thus, adaptation to degraded speech can occur in different ways, perhaps via different mechanisms. Models of speech comprehension typically incorporate several processing levels mediating between an incoming speech signal and the computation of its meaning (Gaskell & Marslen-Wilson, 1997; Luce & Pisoni, 1998; McClelland & Elman, 1986; Norris, 1994). Although the precise details of the representations that mediate between speech and meaning vary in different accounts, these models share a common assumption that processing is hierarchically organized, with greater abstraction from the surface (acoustic) properties of speech at higher processing levels. However, although all of these models include computational mechanisms that assist in dealing with noise and variability in speech (for instance, by incorporating lexical competition, McClelland & Elman, 1986, Norris, 1994, or mechanisms for phonological inference, Gaskell & Marslen-Wilson, 1997), these abstractionist accounts do not include clearly specified mechanisms by which the perceptual system can adjust its processing properties to the precise form of incoming speech. One means for incorporating adaptation within these accounts is to propose that lowlevel feature or phoneme detectors are retuned to the properties of incoming speech. The effect of lexical information on learning to recognise ambiguous phonemes might suggest a role for top-down influences in supervising this tuning process (cf. Norris et al., 2003). However, since it is not clear that all forms of adaptation show a similar dependence on lexical information, it is possible that other processes might also be involved. For instance, adaptation to artificially distorted speech might only require mechanisms that operate at lower levels of processing. At present, however, there are few, well-specified accounts of the mechanisms involved in perceptual tuning and few experimental techniques for establishing whether adaptation is dependent upon or independent of higher-level, lexical, information. Davis Lexical information drives perceptual learning Page 7 In this paper, w