Babies, Variables, and Relational Correlations

Babies, Variables, and Relational Correlations Michael Gasser ( GASSER @ CS . INDIANA . EDU ) Eliana Colunga ( ECOLUNGA @ CS . INDIANA . EDU ) Computer Science Department, Cognitive Science Program Indiana University Bloomington, IN 47405 Abstract Recent studies have shown that infants have access to highly useful language acquisition skills. On the one hand, they can segment a stream of unmarked syllables into words, based only on the statistical regularities present in it. On the other, they can abstract beyond these input-specific regularities and gen- eralize to rules. It has been argued that these are two separate learning mechanisms, that the former is simply associationist whereas the latter requires variables. In this paper we present a correlational approach to the learning of sequential regular- ities, and its implementation in a connectionist model, which accommodates both types of learning. We show that when a network is made out of the right stuff, specifically, when it has the ability to represent sameness and the ability to represent relations, a simple correlational learning mechanism suffices to perform both of these tasks. Crucially the model makes dif- ferent predictions than the variable-based account. Background Two recent papers in Science have demonstrated the remark- able language learning abilities that are possessed by infants. In both cases the infants were presented with sequences of syllables embodying some sort of regularity and later tested with sequences that agreed or disagreed in certain ways with the training set. In the experiments of Saffran, Aslin, and Newport (1996), eight-month-olds heard strings of syllables consisting of randomly concatenated three-syllable “words,” sequences which never varied internally. Thus the transition probabilities within words were higher than between words. Later the infants were able to differentiate between these words and non-word three-syllable sequences which they had either heard with less frequency than the words or not heard at all. This is taken as evidence that they had picked up the statistics in the training set. Marcus, Vijayan, Bandi Rao, and Vishton (1999) presented seven-month-olds with series of three-syllable sequences separated by gaps. Each sequence consisted of two different syllables arranged in a fixed pat- tern, AAB, ABB, or ABA. For example, in the ABB condi- tion, the presented patterns included sequences such as le di di and ji je je. Later the infants responded differently to novel sequences of three syllables which matched the pattern they had been trained on than to novel sequences which did not. This is taken as evidence that they had in some sense picked up the rule implicit in the training patterns. Marcus et al. (1999) and Pinker (1999) argue that the two studies, taken together, point to at least two distinct learn- ing mechanisms which are behind language learning. One of these, revealed in the experiments of Saffran et al. (1996), can learn relationships such as the tendency for ti to imme- diately follow ga. It is sensitive to the content of the items, not caring about the similarity among different items. For Pinker (1999), this is just the associationism proposed in the eighteenth century by Hume and still proposed as the funda- mental mechanism of the mind by modern connectionists and others. The other mechanism, revealed in the experiments of Marcus et al. (1999), can learn relationships such as the fact that the first syllable in a sequence is the same as the second but different from the third. This mechanism ignores spe- cific content, caring only about sameness or difference. In this sense the second mechanism seems to require variables, placeholders which are ignorant of their specific content. For Pinker (1999), this mechanism is an instantiation of what was proposed by the early rationalists and what we think of today as “symbolic. ” Thus Marcus et al. (1999) and Pinker (1999) now believe that the mind, specifically the portion of it used in language learning, is both associationist and symbolic. The question, as Marcus et al. (1999) make clear, is not whether connectionist networks can learn to solve both kinds of tasks, but what sorts of mechanisms are required and whether these differ for the two tasks. In this paper, we present a model of the learning of regularities in patterns which accommodates both kinds of patterns in terms of cor- relations. We argue that a correlational account, to deal with the tasks in Marcus et al.’s experiment, needs two mecha- nisms in addition to those usually found in such accounts, neither of which amounts to explicit variables. We show how a connectionist network implementing this theory (the PLAYPEN architecture) can learn aspects of the Saffran et al. task, as well as the Marcus et al. task. What is crucial about this account is not that it handles variable-like behavior within a correlational framework but that it makes predictions that differ from the variable-based account. Pattern Regularity Learning Saffran et al.’s and Marcus et al.’s experiments are not di- rectly comparable. In Saffran et al.’s experiments, the bound- aries between the patterns must be extracted, while these are provided in Marcus et al.’s task. However, both are learning tasks in which the learner is presented repeatedly with pat- terns consisting of sequences of syllables and extracts some sort of regularity from the sequences. We agree with Marcus et al. and Pinker that there are other differences in what is going on in these two tasks, but we believe that both are fundamentally statistical, based on the extraction of correlations from input patterns. The main dif-