Word Learning as Bayesian Inference: Evidence from Preschoolers

Word Learning as Bayesian Inference: Evidence from Preschoolers Joshua B. Tenenbaum Fei Xu fei@psych.ubc. ca jbt@mit.edu Department of Brain and Cognitive Sciences Massachusetts Institute of Technology Department of Psychology University of British Columbia Abstract Most theories of word learning fall into one of two classes: hypothesis elimination or associationist. We propose a new approach to word learning within a Bayesian framework. Tenenbaum and Xu (2000) presented a Bayesian model of adults learning words for hierarchically structured categories. We report two experiments with 3- and 4-year-old children, providing evidence that the basic principles of Bayesian inference are employed when children acquire new words at different hierarchical levels. Implications for theories of word learning are discussed. Introduction The problem of word learning has been a well-cited example of the general problem of induction. With each referential act, e.g., “Look, a dog! ” there are an infinite number of hypotheses for the meaning of the word “dog” that is consistent with the data (Quine, 1960). The word could refer to all (and only) dogs, all mammals, all animals, all Dalmatians, this individual Max, all dogs plus the Lone Ranger’s horse, all dogs except Labradors, all spotted things, the front half of a dog, undetached dog parts, things which are dogs if first observed before next Monday but cats if first observed thereafter, and on and on. Yet despite this severe under-determination, even 2- or 3-year-old children seem to be remarkably successful at learning the meanings of words from just one or a few positive examples (Bloom, 2000; Carey, 1978; Markman, 1989; Regier, 1996; among others). How do they do it? Most theories of word learning fall under what we call the hypothesis elimination approach to learning. The learner effectively considers a hypothesis space of possible concepts onto which words will map, and assumes that each word maps onto exactly one of these concepts. The act of learning consists of eliminating incorrect hypotheses about word meaning, based on a combination of a priori knowledge and observations of how words are used to refer to aspects of experience, until the learner converges on a single consistent hypothesis. More precisely, some logically possible hypotheses may be ruled out a priori because they do not correspond to any psychologically natural concepts, e.g., the hypothesis that “dog” refers to things which are dogs if observed before Tuesday but cats if observed thereafter. Other hypotheses may be ruled out because they are inconsistent with examples of how the word is used, e.g., the hypotheses that “dog” refers to all and only terriers, can be ruled out upon seeing the example of Max the Dalmatian. Settling on one hypothesis by eliminating all others as incorrect amounts to taking a deductive approach to the logical problem of word learning. The learner essentially deduces the word’s meaning from a set of premises, which include the assumption that the word maps onto one of the learner’s hypotheses and the a priori knowledge and observational data that rule out all but one hypothesis. The success of word learning is then explained by the deductive validity of this inference. Thus we will sometimes refer to hypothesis elimination approaches as deductive approaches to word learning. To illustrate how word learning has traditionally been explained within a deductive framework, let us return to our opening question of how a child could possibly infer the meaning of the word “dog” from a typical labeling event. One influential proposal has been that children come to the task of word learning equipped with strong prior constraints on viable word meanings (e.g., Markman, 1989), allowing them to rule out a priori many logically possible but psychologically unnatural extensions of a word. Often it is most natural to view these constraints as giving structure to the learner’s hypothesis space, but they could also be seen as eliminating the implausible hypotheses from a larger space of all logically possible concepts. Two classic constraints on the meanings of common nouns include the whole object constraint and the taxonomic constraint (Markman, 1989). The whole object constraint requires words to refer to whole objects, as opposed to parts of objects or attributes of objects, thus ruling out word meanings such as the front half of a dog, or undetached dog parts. The taxonomic constraint requires words refer to taxonomic classes, typically in a tree-structured hierarchy of natural kind categories (Keil, 1979), thus ruling out word meanings such as all dogs plus the Lone Ranger’s horse, all spotted things, all running things or all dogs except Labradors. Whether these constraints are specific to the domain of word meaning or reflect more general restrictions on the structure of natural kind concepts is controversial (e.g., Tomasello, 2001), but their importance in guiding the process of word learning is fairly well accepted. In most cases, such as our example of a child learning the word “dog”, these constraints are useful but not sufficient to solve the inference problem. Even after ruling out all hypotheses that are inconsistent with a typical labeled example (e.g., Max the Dalmatian), a learner will still be left with many consistent hypotheses that also correspond to possible meanings of common nouns. How are we to infer whether a word that has been perceived to refer to Max applies to all and only Dalmatians, all and only dogs, all canines, all mammals, or all animals, and so on? This problem of inference in a hierarchy is interesting in its own