Addressing the Learnability of Verb Subcategorization with Bayesian Inference

Addressing the Learnability of Verb Subcategorizations with Bayesian Inference Mike Dowman ( Mike@cs.usyd.edu.au ) Basser Department of Computer Science, F09, University of Sydney, NSW2006, Australia Abstract Elman (1993) has shown that simple syntactic systems can be learned solely on the basis of distributions of words in text presentation. However Pinker (1989) has proposed that chil- dren must make use of verbs’ semantic representations in or- der to infer their syntactic subcategorizations (semantic boot- strapping). Results reported here demonstrate how Bayesian statistical inference can provide an alternative, and much simpler, account of how subcategorizations are learned. The acquisition mechanism described here suggests that syntactic acquisition may involve a much larger component of learn- ing, and less innate knowledge, than is presumed within mainstream generative theory. Introduction This paper investigates how children learn their first lan- guage, and in particular the syntactic system of that lan- guage. It conceives of the problem in the following way: when exposed to utterances in that language, how is it pos- sible to infer the grammatical system which produced those utterances. Further, the learner is assumed not to know the meanings of the words, have access to prosodic cues to structure, or to receive feedback about which sentences are not grammatical. Currently the major paradigm within which language ac- quisition is explained is the parameter setting framework (Chomsky, 1995). Within this framework it is proposed that knowledge of language is largely specified innately, and learning consists of identifying word tokens and setting a limited number of parameters according to the syntactic structures to which the child is exposed. Chomsky argues that this position is necessary because ‘even the most super- ficial look reveals the chasm that separates the knowledge of the language user from the data of experience. ’ (p. 5). Gold (1967) investigated this problem more formally, and proved that without negative evidence (explicit information about which sentences are ungrammatical) languages are not ‘learnable in the limit’ unless the class of languages which the learner may consider is restricted a p riori, for example by innate knowledge. Below I will discuss an alternative result by Feldman, Gips, Horning and Reder (1969) which suggests that Gold’s result is not relevant to the circum- stances under which children learn languages. Redington, Chater and Finch (1998) investigated to what extent syntactic categories could be inferred based on distri- butions alone, without knowing a p riori what syntactic categories existed in the language. They formed vectors by taking the two preceding and two following context words for each occurrence of each target word in a large corpus of transcribed speech, and recorded how often each context word occurred in each position. Only the 150 most frequent words were used as context, and so this resulted in 600 di- mensional vectors for each word (there being one entry for each of the 150 context words in each of four positions). Clustering those words whose vectors were most similar in terms of Spearman’s rank correlation resulted in clusters which corresponded to appropriate word classes for most of the 1,000 target words. While this system was good in that it could be applied to naturally occurring speech, it was neces- sary to decide at what level of dissimilarity to form separate classes, and so it doesn’t completely solve the problem of recovering the syntactic classes used by the original speak- ers. Elman (1993) demonstrated that not only word classes, but also syntactic patterns in which words belonging to those classes appeared, could be learned without much in- nate syntactic knowledge, at least for simple languages. He trained a recurrent neural network to predict the following word in artificially generated sentences conforming to a simple syntactic system containing 23 words, and syntactic features such as number agreement and recursion in relative clauses. Once trained on 50,000 sentences in this simple language, the network performed at near optimum accuracy at predicting the subsequent word at any stage in a sentence, showing that the network had internalized the structural constraints implicit in the data. While both Redington et al (1998) and Elman (1993) demonstrate that much of syntactic structure can be learned by making statistical inferences based on the distributions of words, Pinker (1989) suggests that some aspects of syntax cannot be learned in this way. He proposes that, in order to determine verbs’ subcategorizations in the absence of nega- tive evidence, children must rely on complex innate rules combined with knowledge of the verbs’ semantic represen- tations. Verbs such as give can appear in both the prepositional dative construction (1a), and the double object dative con- struction (1b), but there is a class of verbs such as donate which can only appear in the prepositional construction, (1c and 1d). However Gropen et al (1989) observe that, based on the alternation between (la) and (lb), children generalize this alternation to verbs such as donate, and so produce un- grammatical sentences such as (1d). They also demonstrated that when presented with novel, nonce, verbs in the preposi- tional construction, children will productively use them in the double object construction in appropriate contexts. However, ultimately children do learn which verbs cannot occur in the double object construction, and so we need a theory which can explain why children first make such gen-