"Less is More" in Bayesian Word Segmentation: When cognitively plausible learners outperform the ideal

“Less is More” in Bayesian word segmentation: When cognitively plausible learners outperform the ideal Lawrence Phillips (lawphill@uci.edu) Department of Cognitive Sciences, 2235 SBSG Irvine, CA 92697 USA Lisa Pearl (lpearl@uci.edu) Department of Cognitive Sciences, 2314 SBSG Irvine, CA 92697 USA addition, GGJ conducted an ideal learner analysis, which assumes unlimited processing and memory resources for the learner. PGS investigated the impact of this assumption, finding a limited ―Less is More‖ effect (Newport 1990) where cognitive resource limitations help, rather than hinder, some Bayesian learners. We examine the effect of the phoneme assumption in addition to these cognitive resource assumptions. We find not only that syllable-based Bayesian learners can do well at word segmentation but also a much more robust ―Less is More‖ effect in our constrained Bayesian learners. This suggests that the unit of representation for models of language acquisition plays a crucial role. Here, using more cognitively plausible assumptions showcases a surprising learning effect, the ―Less is More‖ effect that has been hypothesized to explain language acquisition success in children. Abstract Purely statistical models have accounted for infants’ early ability to segment words out of fluent speech, with Bayesian models performing best (Goldwater et al. 2009). Yet these models often incorporate unlikely assumptions, such as infants having unlimited processing and memory resources and knowing the full inventory of phonemes in their native language. Following Pearl, et al. (2011), we explore the impact of these assumptions on Bayesian learners by utilizing syllables as the basic unit of representation. We find a significant ―Less is More‖ effect (Pearl et al 2011; Newport 1990) where memory and processing constraints appear to help, rather than hinder, performance. Further, this effect is more robust than earlier results and we suggest this is due a relaxing of the assumption of phonemic knowledge, demonstrating the importance of basic assumptions such as unit of representation. We argue that more cognitively plausible assumptions improve our understanding of language acquisition. Keywords: language acquisition; Bayesian modeling; cognitively plausible learning; less is more; statistical learning; word segmentation The syllables as the representational unit The first evidence that infants possess categorical representations of syllabic units appears at 3 months: Eimas (1999) finds that infants have categorical representations of syllables whereas infants at this age have no categorical representation of phonemes. Since word segmentation first occurs around 7.5 months (Jusczyk & Aslin 1995), it is likely that infants have robust access to syllables at this age. In contrast, knowledge of phonemes does not occur until approximately 10 months (Werker & Tees 1984) making it unlikely the learner has adult knowledge of their native language phonemes during the initial stages of word segmentation. Although it is possible that word segmentation and phoneme learning bootstrap from one another, we consider a more conservative approach which assumes infants only have access to syllabic information. While the success of previous statistical word segmentation models is heartening, how dependent is their success on the assumption of the phoneme as a representational unit? With this question in mind, we modify existing phoneme-based statistical models of word segmentation that use Bayesian inference (GGJ, PGS) to operate over syllables. All of our modified Bayesian learners treat syllables as atomic units in the same way phonemes are thought of as atomic units. This mimics the performance of infants who are able to discriminate between syllables such as /ba/, /bu/, and /lu/, but who are unable to Introduction Knowledge of words plays a crucial role in language acquisition but requires a child to identify words out of fluent speech. Children seem to accomplish this word segmentation very early (~7.5 months (Jusczyk & Aslin 1995; Echols et al. 1997; Jusczyk et al., 1993a)), and therefore many strategies have been proposed for this early success. One popular explanation for initial language learning relies purely on distributional information, rather than language-specific biases. This idea is bolstered by findings that infants keep track of the statistical regularities in speech (Saffran et al. 1996), and because languages vary greatly in their cues to word boundaries which would weaken the use of language specific knowledge. One very successful, purely distributional, learning approach uses Bayesian inference (Goldwater, Griffith & Johnson 2009 (GGJ), Pearl, Goldwater & Steyvers 2011 (PGS)). However, these Bayesian models incorporate modeling assumptions that are unlikely to be true. Both have assumed that the basic unit of representation available to the infant is the phoneme. We will argue from experimental evidence that syllables (or syllable-like representations) are a more natural representation for infants at this stage of acquisition. In