Frequency Effects in the Processing of Inflectional Morphology : a Distributed Connectionist Account

One source of evidence concerning the representation and processing of inflected words comes from frequency effects. Experiments in Dutch reported by Baayen, Dijkstra and Schreuder (1997) are interpreted as evidence for a dual-route race model in which inflected words are both stored as whole forms and decomposed into stems and affixes. In this paper we report two sets of neural network simulations demonstrating that a singlemechanism, distributed, connectionist model, trained to map from orthographic input to a representation of the meaning and inflectional status of words, can account for the Baayen et al. results without requiring two computational mechanisms. The pattern of frequency effects produced by this single-mechanism model depends critically on including homonymous affixes in the training set, such as the Dutch plural affix –en which also marks verb infinitives. This finding suggests that frequency effects for morphologically complex words can be accounted for in a distributed system, without postulating distinct processing mechanisms of whole-word storage and morphological decomposition. Davis Frequency Effects in the Page 3 Frequency effects in the processing of inflectional morphology: a distributed connectionist account Words that occur more frequently in language are processed more quickly. This is the longest-standing, and most clearly established result in experimental psycholinguistics (for early summaries of research on word frequency, see Howes & Soloman, 1951; Broadbent, 1967; Morton, 1969). This basic result has been replicated in a range of tasks, such as reading aloud, picture naming, semantic or lexical decisions, in a range of languages. The word frequency effect is taken as evidence that the systems involved in language processing respond to basic statistical properties of an individual’s linguistic experience. Furthermore, through careful manipulations of different word properties, it has been possible to observe effects of the frequency of individual morphemes in complex words (Taft, 1979; Sereno & Jongman, 1997, Baayen, Dijkstra & Schreuder, 1997) or effects of the frequency of different meanings of homonymous words (Rodd, Gaskell & Marslen-Wilson, 2002; Borowsky & Masson, 1996) in order to infer which aspects of word frequency are crucial for a particular task. However, despite widespread agreement that word frequency plays an important role in language processing, a variety of accounts have been proposed of the mechanism by which frequency effects arise. It has been proposed that word frequency effects arise since access to the mental lexicon involves a search through a frequency-ordered word list (in the search model of Forster, 1976). Alternatively, it is proposed that changes to the processing properties of the units that represent individual lexical items (such as lowering the threshold for recognition in the logogen model of Morton, 1969 or raising the resting activation level in the interactive-activation model of word recognition proposed by McClelland and Rumelhart, 1981) allows high frequency words to be recognised more quickly. Davis Frequency Effects in the Page 4 Both serial search and logogen accounts share a single common assumption; namely that there is a single lexical unit that uniquely represents each word – that is, they are localist models. Thus effects of the frequency of occurrence can be localised to changes occurring to a lexical unit. In contrast, a recent class of computational account, distributed connectionist models, propose that individual words are represented as a pattern of activation over many active units, with different banks of units representing domains of word knowledge such as orthography, phonology and semantics. Linguistic knowledge in distributed models is represented by the strength or weights of connections that link these banks of units. These connection weights are not pre-determined but are gradually learnt by training the network to translate from one representation to another (e.g. reading aloud involves a translation from orthography to phonology, auditory comprehension involves mapping from phonology to semantics, etc.). During training, a learning algorithm (typically back-propagation – Rumelhart, Hinton & Williams, 1986) adjusts connection strengths to reduce the discrepancy between the network’s output and a target representation. One crucial aspect of distributed connectionist accounts is that they change the interpretation of a distinction that is common to both linguistics and psycholinguistics; the distinction between rule-governed forms and exceptions. In distributed models, a single set of connections is not only able to acquire knowledge of individual lexical items or exceptions, but is also able to extract and apply regularities to new items, showing how these systems can generalise in a seemingly rule-governed way. For example, in generating the past-tense forms of English verbs from their stems, a single system can learn to phonologically translate both regular verbs (jump-jumped, play-played) and irregular verbs (leap-leapt, give-gave) as well as generalising to novel forms (wug-wugged) (Rumelhart & McClelland, 1986; Plunkett & Marchman, 1991; 1993). Thus the distinction between rule-governed forms and exceptions to Davis Frequency Effects in the Page 5 these rules may be an accurate description of the structure of linguistic knowledge but need not imply that there are two underlying mechanisms by which these forms are processed. In the current paper, we explore whether these distributed connectionist models are able to account for the recognition (rather than the phonological transformation) of regularly inflected words. In describing these simulations we will not only consider the capabilities of distributed connectionist networks (that is, whether they can perform the task) but, crucially, we will evaluate the behaviour of these networks by comparison with experimental investigations of the processing of regularly inflected words. In order to conclude that a single-mechanism distribution connectionist model can account for the recognition of inflected words, we require the model to simulate the pattern of behavioural data produced by human participants. Before describing these network simulations, we will therefore review the target empirical phenomena, and their interpretation in more traditional, localist accounts of lexical representation and processing. Frequency effects in the recognition of regularly inflected words Tasks that involve accessing stored lexical information are sensitive to the frequency of occurrence of these units in the linguistic environment. For example, in the lexical decision task, where participants make a speeded response to indicate whether a letter string or word is a real word or a non-word, a typical response time of 600ms for a word that occurs once in a million words of text, will decrease by approximately 10% for an otherwise matched word that occurs with a frequency of 100 occurrences per million words. In logogen-style models (c.f. Morton, 1969), in which words are identified when the activation of the lexical unit exceeds some threshold value, this effect is simulated by assuming that more frequently occurring words have a higher level of resting activation or a lower recognition theshold. In Davis Frequency Effects in the Page 6 either case, more frequent words have a head-start in the recognition process, and will be identified faster. One important theoretical question in this framework is whether the critical logogens that are activated during word identification represent whole-words or individual morphemes. Are inflected words like “tables” stored whole or decomposed into the smaller units “table” + “s”? This issue can be addressed using the frequency effect as a diagnostic. For example, if we consider the words “neck/necks” and “lip/lips”, these words are approximately matched on the frequency of occurrence of the lemma {neck} or {lip} at around 80 occurrences/million words in the CELEX database (Baayen, Pipenbrock & Guilikers, 1995). However, as we might expect (since individuals typically have two lips but only one neck), the frequency of occurrence of singular and plural forms of these two items is very different. The word “neck” occurs many times more frequently than “necks” (word-form frequency, neck = 72/million, necks = 7/million, these frequencies are singular-dominant) whereas the plural “lips” is used much more frequently than the singular “lip” (word-form frequency, lip = 17/million, lips = 61/million, showing that the noun {lip} is plural-dominant). By an account in which there are individual logogens for the singular and plural form of these words (a whole-word storage account), we would expect substantial differences in response times to “lip” and “neck” or to “lips” and “necks”, with the singular of the singular dominant form being responded to more quickly than the singular of the plural dominant form, and vice-versa for plural forms. This pattern of results is illustrated graphically in Figure 1a. Conversely, if both of these words were decomposed into smaller units such that a single lemma unit (for {neck} or {lip}) was activated for both singular and plural forms, we would not expect a difference in response time between singularand plural-dominant nouns, so long as those items are matched on lemma frequency. To build a model that functions in Davis Frequency Effects in the Page 7 this way, we need to invoke an additional process that allows the inflected words “necks” and “lips” to gain access to the appropriate lemma representation. This decomposition process may add to the time required to respond to plural nouns so that although there is no difference between response times to singularand plural-dominant nouns, responses are generally slowed to plural forms. This predicted pattern is illustrated in Figure 1b. <INSERT FIGURE 1 ABOUT HERE> Ex