Templatic features for modeling phoneme acquisition

Templatic features for modeling phoneme acquisition Emmanuel Dupoux (emmanuel.dupoux@gmail.com) Ecole des Hautes Etudes en Sciences Sociales, Ecole Normale Superieure 29 rue d’Ulm, 75005 Paris, France Guillaume Beraud-Sudreau (guillaumeberaud@gmail.com) Laboratoire de Sciences Cognitives et Psycholinguistique, Centre National de la Recherche Scientifique 29 rue d’Ulm, 75005 Paris, France Shigeki Sagayama (sagayama@hil.t.u-tokyo.ac.jp) Departement of Information Physics and Computing University of Tokyo, 7-3-1, Hongo, Bunkyo-ku, Tokyo 113-8656 Japan Abstract These studies, however, did not use raw speech signals, but rather a small number of parameters extracted by hand: e.g., the frequency of the first and second formants, vowel duration, etc. (Valhabba et al 2007). This presupposes that infants are equipped with fairly speech-specific perceptual abilities and, crucially, that they know how to segment the continuous stream into discrete segments like consonants, or vowels. This latter assumption is problematic given that such segmentation is not universal, but depends on the phonology of the language (Dupoux et al, 1999). Varadarajan et al. (2008) is one of the few published paper that attempted to learn phonemes from raw speech. Using an optimized version of Successive State Splitting (SSS, Takami & Sagayama, 1992), they grew in an unsupervised fashion a large network of Hidden Markov Model (HMM) states. These states were shown to encode speech sounds with no loss of information compared to supervized HMMs, but there were two problems. First, the states of the HMM network did not correspond to phonemes, but rather to subphonemic units of the size of acoustic events (e.g. burst, closure, transition, etc). This is the oversegmentation problem. Second, even combining states into sequences did not yield phonemes, but rather, context dependant variants (contextual allophones). This is the contextual variability problem. Here, we address the first problem, the second problem being address in other work (Peperkamp et al, 2006; Martin et al. submitted; Boruta et al. 2011). The oversegmentation problem of SSS, although disappointing, is not entirely surprising. State-of-the art supervised HMMs have the same problem: segments are typically modeled using three states, not a single state. The reason is that HMMs represent speech as local spectral feature vectors (e.g. Mel-Frequency Cepstral Coeficients – MFCC, computed over a 15-20ms window), whereas phonemes are realized as a complex articulatory trajectory spanning between 50 and 150ms, sometimes involving a sequence of events (constriction, release, changes in the source, etc.). Since HMMs are modeling speech sounds through Gaussians distributions (which are local), the only way to model phonemes accurately is to segment them into subparts. This problem is not limited to MFCC features, but would also apply to any local feature, like for instance We describe a model for the coding of speech sounds into a high dimensional space. This code is obtained by computing the similarity between speech sounds and stored syllable- sized templates. We show that this code yields a better linear separation of phonemes than the standard MFCC code. Additional experiments show that the code is tuned to a particular language, and is able to use temporal cues for the purpose of phoneme recognition. Optimal templates seem to correspond to chunks of speech of around 120ms containing transitions between phonemes or syllables. Keywords: Early language acquisition, modeling, phonemes Introduction Infants spontaneously learn their ambient language at an amazing speed. During their first year of life, they construct abstract perceptual categories corresponding to the phonemes of their language. They lose the ability to distinguish fine phonetic variants that belong to the same phoneme category, and enhance their ability to distinguish between category contrasts (see a review in Kuhl, 2000). This is done without any supervision from the parents, before a substantial recognition lexicon has been built (12- month-olds are believed to recognize about 100 words), and before they can articulate correctly the phoneme categories they recognize. How do infants achieve this? One possibility is that they perform some kind of unsupervised statistical clustering of the ambient speech signals. Maye, Werker and Gerken (2002) showed that 6-month-old infants perform such computations, using artificial languages with either a monomodal statistical distribution or a bimodal distribution of phonetic cues. Only a limited number of studies have addressed the computational mechanisms that could underlie such acquisitions. Guenther & Gjaja (1996) showed that Self- Organizing Maps have the potential to construct phoneme categories in an unsupervised fashion (see also Valhabba et al., 2007; Gauthier, Shi & Xu, 2007). Valhabba et al (2007) implemented an incremental version of Expectation Maximization on a Gaussian Mixture model, and showed that both the number of vowels and their statistical distributions can be inferred from the signal in an unsupervised fashion.