A Statistical Model of Developmental Changes in Early Word Learning

A Statistical Model of Developmental Changes in Early Word Learning Chen Yu (chenyu@indiana.edu) Weixia Huang (whuang@xeroxlabs.com) Department of Psychology and Program in Cognitive Science Indiana University Bloomington, IN 47405 USA Abstract Young language learners are able to map a word onto its ref- erent from an infinite number of possible word-to-world map- pings at early stage and further this learning ability turns to be much more effective at the later stage. What mechanisms underlie behavioral changes during word learning? This pa- per presents a developmental model of statistical associations, suggesting that as far as human minds are equipped with gen- eral associative learning mechanisms, ambiguity in the word- learning situation can be significantly reduced by recruiting cu- mulative lexical knowledge in statistical computations. Conse- quently, this leads to increasingly fast subsequent learning and abrupt behavioral changes, such as the vocabulary spurt and fast mapping. We provide a formal account of this argument by developing a computational model fed with the data col- lected from a series of picture-book reading sessions to sim- ulate word acquisition in natural contexts. The results show that previously learned lexical knowledge can not only nar- row the search space but also bootstrap the subsequent learn- ing process, which greatly improves learning results without the change of the underlying learning mechanism. Hence, this work suggests that using lexical knowledge accumulated in subsequent statistical learning is a scaffold for vocabulary growth in early language acquisition. 1. Introduction From scratch, young children need to solve many complex learning problems in early word acquisition, such as speech segmentation to discover word units, conceptual learning to identify the meanings (prelinguisitic concepts) of words, and then word-to-world mapping to associate words with mean- ings. Gleitman, Cassidy, Nappa, Papafragou, and Trueswell (2005) argued that among these tasks, a considerable part of the bottleneck for young language learners resides in the tools for solving the word-to-world mapping problem – how to map a phonological form to a conceptual representation. A com- mon conjecture is that children map sounds to meanings by seeing an object while hearing an auditory word-form. The most popular mechanism of this word learning process is as- sociationism. Richards and Goldfarb (1986) proposed that children come to know the meaning of a word through re- peatedly associating the verbal label with their experience at the time that the label is used. Smith (2000) argued that word learning is initially a process in which children’s attention is captured by objects or actions that are the most salient in their environment, and then they associate it with some acoustic pattern spoken by an adult. However, the associative approach has been criticized on the grounds that it does not provide a clear explanation about how infants map a word to a potential infinity of referents when the word is heard, which is termed reference uncer- tainty by Quine (1960). Quine presented the following puz- zle to theorists of lexical learning: Imagine that you are a stranger in a strange land with no knowledge of the language or customs. A native says ”Gavagai” while pointing at a rab- Xerox Corp. 800 Philips Road Webster, NY 14580 USA bit in the distance. How can you determine the intended ref- erent? Quine offered this puzzle as an example of the inde- terminacy of translation. Given any word-event pairing, there are, in fact, an infinite number of possible intended meanings – ranging from the rabbit as a whole, to its color, fur, parts, or activity. One explanation termed “cross-situational learn- ing” has been proposed by many theorists, such as Pinker (1989) and Gleitman (1990). The idea is that when a child hears a word, she can hypothesize all the potential meanings for that word from the non-linguistic context of the utterance containing that word. Upon hearing that word in several dif- ferent utterances, each of which is in a different context, she can intersect the corresponding sets to find those meanings which are consistent across the different occurrences of that word. Presumably, hearing words in enough different situa- tions would enable the child to rule out all incorrect hypothe- ses and uniquely determine word meanings. In light of this and with recent empirical evidence demonstrating that chil- dren and even infants, possess powerful statistical learning capacities (Saffran, Newport, & Aslin, 1996), Yu and Bal- lard (2004a) (see also Yu, Ballard, & Aslin, in press) devel- oped a computational model of how young language learners perform statistical computations on cross-situational observa- tions, which is reviewed in Section 3. Based on our previous work, this paper attempts to apply the statistical learning mechanism to interpret developmen- tal changes in word learning. Specifically, we investigate whether the associative learning mechanism can account for discontinuities in behaviors during the second year of life, such as the vocabulary spurt and fast mapping (see Section 2). These phenomena has attracted much attention in the field of cognitive development since they are difficult to ex- plain in terms of simple and transparent causes, and pose theoretical challenges that require sophisticated and highly- constrained learning principles. Nonetheless, complementary studies in connectionist modeling (e.g. Elman et al., 1996; see a good review in Regier, 2003) suggest that abrupt internal changes may not be needed to produce discontinuous external changes in behaviors. These works show that some factors, such as limited memory at the starting point of learning (El- man et al., 1996), nonlinearity of neural networks (Plunkett, Sinha, Miller, & Strandsby, 1992), and gradual emergence of attention to some aspects of the world (Regier et al., 2001), may contribute to nonlinear behaviors of human learning. In light of this, the present paper suggests another possible ex- planation of discontinuous behaviors in early word learning – the performance of the same learning mechanism can signif- icantly improve by storing lexical knowledge previously ex- posed and then recruiting it in subsequent learning. This idea may seem obvious and it is certainly consistent with many formal theories of learning. However, the main contribution of this paper is to propose and implement such a learning