Acquisition and Evolution of Natural Language Acquisition and Evolution of quasiregular languages : Two puzzles for the price of one

The quasi-productivity of natural languages appears to pose two difficult problems for language research. Firstly, why do irregularities in natural language not disappear over time, leaving languages completely regular (a transmission problem), and secondly, how did such irregularity arise in the first place (an emergence problem)? To address the transmission problem, we present an artificial, simplicity-based learner capable of acquiring quasi-regular structures. In doing so, we present an explicitly psychological model of a famously problematic aspect of language acquisition known as Baker’s Paradox. We present several simulations of an Iterated Learning Model (ILM) illustrating the emergence and stability of quasi-regular irregularities using a rudimentary language. These simulations offer a possible resolution to the emergence problem. Other possible resolutions are discussed. Introduction Natural languages are most often characterized as a combination of rule-based generalization and lexical idiosyncrasy. The English past tense is a familiar case, in which the irregular form went replaces the expected +ed construction *goed. Baker (1979) notes that this is a relatively benign example for learners, since irregular forms are frequently encountered in the course of their linguistic experience. The experience of the form went may block *goed, if the learner assumes that verbs typically have a single past tense form---thus, an observed alternative form can serve as evidence that an absent regular form is not allowed in the language (e.g. the Competition model, MacWhinney, 1989). Much more troubling are cases where an apparently legal construction is idiosyncratically absent, without any alternative. The dative shift in English is a welldocumented example: Acquisition and Evolution of Natural Language (1) John gave/donated a book to the library (2) John gave/*donated the library a book In such cases we can think of linguistic rules as being quasi-regular: they license the combination of some members of syntactic categories, but not others. The difficulty of learning such idiosyncratic absences from partial input and without negative evidence (as is the case with natural language) has become notorious in the language acquisition literature. In particular, given that only a finite set of sentences is ever heard, out of the infinite set of possible sentences in a natural language, it is clear that mere absence of a linguistic form cannot be directly used as evidence that the form is not allowed. Yet, such ‘holes’ are clearly specific to particular natural languages, and hence cannot be explained by adversion to innate linguistic principles. This problem has been viewed as so severe that it has been labeled Baker’s paradox; and viewed as raising logical problems for the theory of language acquisition (e.g., Baker & McCarthy, 1981)1. The approach we adopt here is to apply a general principle of learning to explain how linguistic idiosyncracies can be acquired. Note that the mechanism must be that is sufficiently flexible to capture the huge range of idiosyncrasies across a huge range of linguistic contexts. Moreover, the existence of such a mechanism is required, we contend, to explain the existence of idiosyncracies in language: idiosyncrasies could not have emerged or survived in its absence, as they would have been winnowed out by learning failures by successive linguistic generations. In this respect, Baker’s paradox raises a secondary paradox for language evolution. The puzzle of how language acquisition processes can capture what appear to be idiosyncratic ‘holes’ in the language also raises the puzzle of how difficult-to-acquire linguistic patterns are emerge and are transmitted in the development of languages. Note that, on pain of circularity, whatever learning mechanisms are responsible for learning such idiosyncracies must pre-date the emergence of such idiosyncracies. That is, we cannot view the idiosyncratic nature of language as a stable environment to which biological basis for language acquisition adapted---because without relevant prior learning mechanisms already established, language could not have developed with such idiosyncracies in the first place. In this chapter we consider two questions for language evolution raised by the existence of these idiosyncrasies. The first is a problem of transmission: what kind of learning mechanism could ensure the stability of idiosyncratic absences across generations and be sufficiently flexible and general to pre-date their emergence? The second is one of emergence: even assuming that such a mechanism exists, what conditions might give rise to these irregularities? Acquisition and Evolution of Natural Language The chapter falls into five distinct sections. In the first we discuss the ubiquity of quasi-regular constructions. In the second section we outline why they constitute such an apparently difficult learning problem. We then discuss the relationship between acquisition and evolution, in particular the idea that any hard learning problem of culturally transmitted information entails evolutionary puzzles. In the third section we present a model that is able to learn quasi-regular structures in a rudimentary language from positive evidence alone, using a very general learning principle: simplicity. The model learns by creating competing hypothetical grammars to fit the language to which it has been exposed, and choosing the simplest. As an explicit metric for simplicity we use Minimum Description Length (MDL), a mathematical idea grounded in Kolmogorov complexity theory (Li & Vitányi, 1997). In acquiring quasi-regular language structures, our model addresses the transmission/acquisition problem. In the fourth section, we detail several simulations based on an Iterated Learning Model (ILM, e.g. Kirby, 2001) in which a probabilistically generated artificial language is transmitted over 1,000 generations of simplicity-based learners. The results of these simulations chart not only the stability but also the emergence of quasi-productivities in the language. In particular we show that: a) Exceptions are stable across successive generations of simplicity driven learners. b) Under certain conditions, statistical learning using simplicity can account for the emergence of quasiproductivity in a language. In the final section we discuss the results of the ILM simulations, in particular the conditions in which quasiregular structures might emerge. Baker’s Paradox and linguistic quasi-productivity. A mainstay of linguistic analysis has been that human languages are composed of a limited number of basic units (features, segments, syllables, morphemes, words, phrases, clauses, etc.) that can be combined by a small number of generative rules to create larger units. Postulating the existence of recursive rules allows for an infinite number of sentences to be created. This generativity goes well beyond theoretical linguistic description, as it is typically taken to be embodied in the psychological mechanisms responsible for acquiring and representing linguistic rules and units. Although the capacity to generalize from a limited set of examples to novel instances is an uncontroversial aspect of the human cognition, a puzzle that has attracted linguists is that natural languages, although productive, are never fully regular. There appear to be finely-tuned lexical and syntactic selectional Acquisition and Evolution of Natural Language constraints that native speakers are aware of. Expected regular structures may either be replaced (e.g. went for *goed) or they may be disallowed completely. These semi-productive structures may be seen as a special case of irregularity where the irregular form is absent, i.e. there seems to be an unfilled slot that constrains open-ended productivity. Consider, for instance, a transformational rule such as to be Deletion (after Baker, 1979): (1) X – to be – Y