Reply to Christiansen and Curtin (1999)

Christiansen and Curtin (in this issue1Christiansen M.H. Curtin S.L. Transfer of learning: rule acquisition or statistical learning?.Trends Cognit. Sci. 1999; 3: 289-290Abstract Full Text Full Text PDF PubMed Google Scholar, described in more detail in2Christiansen, M.H. and Curtin, S.L. The power of statistical learning: no need for algebraic rules Proc. 21st Annu. Conf. Cognit. Sci. Soc. (Mahwah, NJ), Erlbaum (in press) (a Web-based version can be found at http://www.siu.edu/∼psycho/faculty/morten/statlearn.html)Google Scholar) offer two results that they take to be arguments against our arguments3Marcus G.F. Vijayan S. Rao S.B. Vishton P.M. Rule learning in seven-month-old infants.Science. 1999; 283: 77-80Crossref PubMed Scopus (790) Google Scholar, 4Marcus G.F. Rethinking eliminative connectionism.Cognit. Psychol. 1998; 37: 243-282Crossref PubMed Scopus (181) Google Scholar, 5Marcus G.F. Can connectionism save constructivism?.Cognition. 1998; 66: 153-182Crossref PubMed Scopus (75) Google Scholar, 6Marcus G.F. Exercises in Rethinking Innateness: A Handbook for Connectionist Simulations (Review).Trends Cognit. Sci. 1997; 1: 318-319Google Scholar, 7Marcus G.F. Do infants learn grammar with algebra or statistics? Response to Seidenberg & Elman, Negishi, and Eimas.Science. 1999; 284: 436-437Google Scholar, 8Marcus G.F. Connectionism: with or without rules? Response to J.L. McClelland and D.C. Plaut.Trends Cognit. Sci. 1999; 3: 168-170Abstract Full Text Full Text PDF PubMed Google Scholar, 9Marcus G.F. Rule learning by seven-month-old infants and neural networks. Response to Altmann and Dienes.Science. 1999; 284: 875Crossref Google Scholar about simple recurrent networks. The first result concerns a discriminant analysis which shows that the ‘internal representations [in a simple recurrent network] incorporated sufficient information to reliably distinguish between items that were either consistent or inconsistent with the habituation stimuli’. In effect, what Christiansen and Curtin did was to plot twelve sentences with either AAB or ABB syllable sequences in an 80-dimensional space, and then ask whether it was possible to draw a line anywhere in those 80 dimensions that would divide the AAB test items from the ABB test items. It is likely that in many dimensions, the AAB items and the ABB items overlap with one another; wherever there is overlap, one could not, by definition, separate the AAB items from the ABB items. What Christiansen and Curtin’s first result shows us is that not all the dimensions are like that; instead, there exists at least one dimension in which the AAB and ABB items do not overlap; in that dimension (or set of dimensions), one could draw a line between the AAB and ABB items. While we do not at all doubt that this is the case, the result is all but a statistical certainty – excluding repetitions of identical sentences, there are only four distinct data points (‘ba-ba-po’ and ‘ko-ko-ga’ versus ‘ba-po-po’ and ‘ko-ga-ga’) to be separated, and 80 chances (one per hidden unit) to draw such a line. So the result might well have happened even by chance. To protect against this possibility, Christiansen and Curtin ran a control in which they randomly reassigned three vectors from Group A to Group B, finding that one could no longer draw a line that perfectly splits the two groups. But this control stacked the deck in Christiansen and Curtin’s favor by asking the discriminant analysis to do the impossible: to succeed, it would have had to separate two instances of some sentence, say ‘ba-ba-po’ from the third instance of the very same sentence. As a consequence, the control confounds repetition of sentences with categorization of sentences, and hence shows nothing about whether the initial result is meaningful. (A more appropriate control to assess the meaningfulness of the discriminant analysis might have been to ask whether a line could be drawn between the six sentences starting with the word ba and the six sentences starting with the word ko.) Even if the discriminant analysis were real, what one would really want to do is to show that the internal representations play a causal role in behavior; Christiansen and Curtin’s second analysis, which attempts to do this, is therefore the crucial one. In particular, in this analysis they showed that if our materials are presented to a simple recurrent network as a single uninterrupted string with no pauses between words, the model was slightly better (80% versus 75%) at ‘segmenting out the words in the inconsistent items’. From this, they argued that, ‘This would make the inconsistent items more salient and therefore explain why the infants preferred these to the consistent items’1Christiansen M.H. Curtin S.L. Transfer of learning: rule acquisition or statistical learning?.Trends Cognit. Sci. 1999; 3: 289-290Abstract Full Text Full Text PDF PubMed Google Scholar. But even if we accepted the premise that infants would look longer at items that were easier to segment, Christiansen and Curtin‘s position runs into at least two serious problems. First, their result, if it is real, might rely on setting up our task as a segmentation problem in which there are no pauses between words – but, our materials included a 250-ms pause between words (see footnote 11, 3Marcus G.F. Vijayan S. Rao S.B. Vishton P.M. Rule learning in seven-month-old infants.Science. 1999; 283: 77-80Crossref PubMed Scopus (790) Google Scholar). Christiansen and Curtin’s attempt to explain the infants’ performance as a segmentation task thus does not fit the actual task, and we suspect that their effect – if it is real – would go away if these gaps between words were included. Second, an even more pressing issue is to see whether the result is real. Given that in the experimental stimuli grammatical consistency was not correlated with any sort of segmentation cue, it might turn out that Christiansen and Curtin’s result is simply a chance occurrence that has been wrongly interpreted as meaningful: the differences observed were small, statistically significance has not yet been reported, and the simulations have not yet been replicated. In the absence of assurances that their behavioural result is robust, it does not merit further discussion at present. I thank Morten Christiansen, Bob Hadley, Pat Shrout and Zsofia Zvolenszky for helpful discussion.