Syllables, Morphemes and Bayesian Computational Models of Acquiring a Word Grammar

Syllables, Morphemes and Bayesian Computational Models of Acquiring a Word Grammar C ¸ a˘gri C ¸ o ¨ ltekin Cem Bozs¸ahin Cognitive Science Cognitive Science and Computer Engineering Middle East Technical University (METU), Ankara 06531 Turkey METU, Ankara 06531 Turkey cagri@xs4all.nl bozsahin@metu.edu. tr Abstract There are indeed phonological and prosodic cues for dis- cerning substrings smaller than words, namely syllables (rhythm), stress and pitch accents. In this work, we report a computational study which starts with the ability to iden- tify syllables, and learns the meaning and category of words and morphemes without the assumption that only words and morphemes have a meaning. The kind of meanings that the system starts with and learns more of is not lexical meanings, such as what it means to be a dog or to sleep (see Tenen- baum & Xu 2000 for a Bayesian way to tackle that prob- lem), but the combinatory meaning and its syntactic reflex in the form of a category, as a lexical hypothesis, for example how pisi-ler-e (kitty-PLU-DAT, Turkish), with the syllables pi·si·le·re, can come to be associated with a logical form such as to 0 (plu 0 cat 0 ) and the syntactic type N for nouns (and oth- ers, such as VP modifiers). We show that under a Bayesian scenario of hypothesis revision with the Universal Grammar as the provider of likelihoods and priors, starting with sylla- bles and the assumed ability to associate forms with mean- ings converges to a lexicalized grammar of words and mor- phemes, by showing a significant overlap with the lexical hy- potheses of a learning model which works with the assump- tion that only morphemes and words constructed from them have meanings. Crucially, morphology of words is a hidden variable in our model, and the input to the system are pairs of sequence of syllables (in lieu of phonological form, PF) and a logical form (LF), without any indication as to which syllable contributes to what part of the LF, or which part of the meaning of a mor- pheme is covered by a syllable. This is unlike the approach of Jack et al. (2006), another syllable-based acquisition model, in which a sequence of syllables is paired not with a possi- bly ambiguous LF but with a disambiguated representation of world meanings. We do not assume that the child knows pisi is kitty, and ler is plural; she might (wrongly) hypoth- esize pisi is plural and ler means kitty, or the first syllable of pisi (pi ) means kitty, etc. We also differ from Aronoff et al. (2006), whose model detects frequently-occurring sound sequences and hypothesizes that they are morphemes. Our model aims to learn the correct LF of the purported mor- pheme as well, not just its form. We report a computational study on the CHILDES database for learning a word grammar of Turkish nouns. The syllable-based model converges to a morpheme-based model in terms of over- laps in the set of lexical hypotheses. Morphology is a hidden variable in all models, and the search problem for hypotheses is narrowed down by a probabilistic conception of universal grammar a ` la Combinatory Categorial Grammar. The conver- gence of the syllable model suggests that morphemehood can be an emergent computational property. Keywords: Morphology, grammar, learning, Bayesian model. Introduction How can the meaning and category of words arise in the mind of a child? On one hand, we have the problem of identifying segments of speech as word-like units. On the other, we have the problem of identifying which meanings go with which substrings in speech. The assumption, common to both gener- ative and cognitive linguistics, is that the child has the innate capacity to associate forms with meaning, and it is a question of acquisition to tackle the problem of deciding which forms go with which meanings. A quick glance over the Turkish fragment of the child- directed speech in the CHILDES database reveals that 44% of the nouns are uninflected; the remaining 56% are inflected by means of affixes and clitics. The question then arises as to how the meaning and category of the inflected words, which constitute the majority, are acquired by the child. A common concept, influential at least since Bloomfield, is that morpheme is the minimal meaning-bearing element in natu- ral languages. Nevertheless, although there are clear phono- logical and prosodic cues for word boundaries (e.g. Jusczyk, 1999; Thiessen & Saffran, 2003), there are no apparent cues for morpheme boundaries, hence the task of learning mor- pheme meanings to come up with word meanings is not made easier by labeling some items as morphemes in the compe- tence grammar of adults. 1 Aksu-Koc & Slobin (1985); Peters & Menn (1993) report pro- duction data of respectively Turkish and English children of age 2;6 and younger, during which the child produces meaningless filler syl- lables. Peters & Menn data show this is not idiosyncratic to verbs. Contra the remarks of both work for Turkish without a statistic, mor- pheme and syllable boundaries do not generally coincide. Only 23% of the syllables in nouns (out of 20,433 syllables) are also mor- phemes in the CHILDES database. If we only match boundaries (the beginning and end of a morpheme align with a syllable boundary, ir- respective of whether the syllable and the morpheme are the same), e.g. araba-lar (car-PLU, Turkish) versus the syllables a·ra·ba·lar providing two matches out of 4 syllables, the overlap is 57%. Universal Grammar What allows our system to learn with reasonable efficiency is that the search problem for lexical hypotheses is kept man- ageable by a Universal Grammar (UG) and the current lexi-