Using the Distributional Statistics of Speech Sounds for Weighting and Integrating Acoustic Cues

Using the Distributional Statistics of Speech Sounds for Weighting and Integrating Acoustic Cues Joseph C. Toscano (joseph-toscano@uiowa.edu) University of Iowa, Department of Psychology, E11 SSH Iowa City, IA 52242 USA Bob McMurray (bob-mcmurray@uiowa.edu) University of Iowa, Department of Psychology and Iowa Center for Developmental and Learning Sciences, E11 SSH Iowa City, IA 52242 USA Abstract learns the number of categories in the input and their statis- tical distributions, reflecting the developmental trajectory of speech category formation in the process. This demonstrates that unsupervised statistical learning mechanisms are able to describe how infants can acquire these categories. However, many critical aspects of this process have yet to be addressed. In particular, acoustic analyses have revealed that most phonemic distinctions are marked by multiple cues. Lisker (1978) cites that there are at least 16 acoustic cues that distinguish voiced and voiceless sounds in word-medial po- sition in English. Existing models of statistical learning and speech sound category acquisition are insufficient to describe how listeners learn multiple cue distributions and integrate these cues when perceiving speech. These challenges are also not limited to understanding speech development and percep- tion. Similar difficulties would be encountered when trying to combine multiple features in other domains as well. Can statistical learning approaches be extended to capture listeners’ use of multiple acoustic cues? How could this prob- lem be instantiated in a model of speech sound categoriza- tion? One way would be to present each cue along a separate dimension, as in Figure 1B, leading to n-dimensional cate- gories, where n is the number of cues to be learned. This type of model has the advantage of allowing us to fully represent the acoustic space. However, it can also lead to computational complexity (representing a 16-dimensional category for the cues to voicing), and it would present the learner with a sparse space from which to extract the categories, since many of the possible combinations of cues would never be heard. An alternative approach would be to weight and combine sets of cues that lie along the same phonetic dimension, as shown in Figure 1C. Cues to voicing, for example, are not orthogonal; they each indicate the same category structure: voiced or voiceless for English. Indeed, they are similar to visual cues to depth that observers integrate during percep- tion to determine the three-dimensional structure of objects (Jacobs, 2002). Thus, it is possible to integrate them into a single phonological dimension upon which the relevant cat- egories can be learned. This approach reduces the computa- tional complexity of the problem, and it can be expanded to account for a large number of acoustic cues. This paper seeks to address several questions about how sets of multiple cues can be learned: A great deal of behavioral evidence suggests that infants can use distributional statistics to learn speech sound categories. Recently, a number of computational approaches have demon- strated the feasibility of statistical learning by showing that the distributional statistics of linguistically-relevant acoustic cues can be learned in an unsupervised way. However, speakers and listeners use a large number of acoustic cues to distinguish phonetic categories, and it is not clear how multiple cues are combined during perception. We propose a model of speech sound category acquisition that learns the distributions of mul- tiple cues that lie along the same dimension and combines them. We demonstrate that the model is able to account for trading relations between cues (an indicator of the size of the effect of each cue) for word-initial voicing contrasts in English. Keywords: speech perception; speech development; mixture of Gaussians; cue integration; statistical learning. Introduction The sound systems of human languages vary greatly. One of the first steps in language acquisition is for infants to deter- mine the sound structure of their native language. In a given language, phonetically-relevant acoustic cues are distributed such that they tend to cluster into categories. For example, voice-onset time (VOT; the delay between the opening of the vocal tract and the onset of vocal energy) values in English tend to cluster into voiced and voiceless categories, near 0 and 50 ms, respectively (Lisker & Abramson, 1964). Thus, the distributional statistics of this cue contains information about the voicing categories of English. Previous work has demonstrated that infants can track these distributional statistics and use this information to learn categories (Maye, Werker, & Gerken, 2002). Recently, re- searchers have begun to use computational models to under- stand this process more deeply. These models have been used to describe how learning unfolds over development and leads to stable speech sound categories. One way to model this process is to represent each phonetic category (such as voiced or voiceless) as a Gaussian distri- bution, providing us with a representation that corresponds to the frequency distribution of an acoustic cue. McMurray, Aslin, and Toscano (in press) present a model that uses this approach. Figure 1A shows how the model might represent the VOT distribution of English. It contains two categories, one centered at the mean of the voiced VOT values and the other centered at the mean of the voiceless values. The model