Slaapte or Sliep? Extending Neural-Network Simulations of E nglish Past Tense Learning to D utch and G erman (2023)

Abstract: This work studies the plausibility of sequence-to-sequence neural networks as models of morphological acquisition by humans. We replicate the findings of Kirov and Cotterell (2018) on the well-known challenge of the English past tense and examine their generalizability to two related but morphologically richer languages, namely Dutch and German. Using a new dataset of English/Dutch/German (ir)regular verb forms, we show that the major findings of Kirov and Cotterell (2018) hold for all three languages, including the observation of over-regularization errors and micro U-shape learning trajectories. At the same time, we observe troublesome cases of non human-like errors similar to those reported by recent follow-up studies with different languages or neural architectures. Finally, we study the possibility of switching to orthographic input in the absence of pronunciation information and show this can have a non-negligible impact on the simulation results, with possibly misleading findings.

## introduction the plausibility of neural network-based or connectionist models in simulating psycholinguistic behaviours has been attracting considerable attention since @xcite first modeled the past-tense acquisition with an early example of sequence-to-sequence network. their experiment received harsh criticism (e.g., @xcite but also inspired cognitive scientists with alternatives (e.g., @xcite @xcite . much more recently, @xcite replicated @xcite 's simulations using a modern encoder-decoder neural architecture developed for the task of morphological paradigm completion. their improved results resolved much of the original criticisms by @xcite . the main purpose of this paper is to study the generalizability of @xcite 's findings beyond the case of english. specifically, we consider two languages that are genetically related to english, but morphologically richernamely, dutch and german. in these languages too, past tense inflection is divided into regular and irregular verbs, but with different proportions and different inflectional patterns than english. moreover, german and dutch are characterized by a much more transparent orthography than english @xcite , which allows us to study the usability of grapheme-based input for simulating past tense acquisition patterns when pronunciation information may not available. concretely, we aim to answer the following research questions: to answer these two questions, we build and release a new past-tense inflection dataset of english, dutch, and german, covering both grapheme and phoneme features (section 3). 1 we then replicate the single-task learning experiments of @xcite (section 4) and extend them to our multilingual dataset, using both phoneme-and grapheme-based input for comparison (section 5). our findings reconfirm the potential and limitations of using neural networks for the simulation of human language learning patterns. our model shows human-like behavior in learning past tenses of verbs, such as the micro u-shape coined by @xcite and over-regularization errors in all the examined languages; however non human-like errors are also reported. we also find that learning irregular past tense forms is considerably easier in dutch and german than in english. finally, we observe that higher orthographic transparency indeed leads to more consistent learning results when a model is trained with grapheme vs. phoneme input. ## background past tense debate the acquisition of verbal past tense in english, particularly the overregularization of the irregular verbs in the process of learning @xcite , has been serving as a testing ground for different hypotheses in language modelling for decades. a much debated question is whether the past tense of (ir)regular verbs is learnt by rules and memories (e.g., @xcite @xcite @xcite , by analogy (e.g., @xcite or by a dual mechanism @xcite . @xcite posited the necessity of mental rules in learning german irregular verbs. by contrast, @xcite and @xcite studies on dutch and german respectively provided evidence in favour of connectionist and analogical approaches: they showed that humans tend to choose wrong past tense suffixes for regular verbs whose phonological structure is similar to that of irregular ones. recent connectionist revival the recent development of deep learning methods in computational linguistics has led to a renewed interest in connectionist approaches to modelling language acquisition and processing by humans (e.g., @xcite @xcite @xcite . last year, modelling morphological acquisition trajectories was adopted as one of the shared tasks of sigmorphon-unimorph @xcite . the three submitted neural systems @xcite @xcite exhibited overregularization and developmental regression, but non-human-like behaviours were also observed. some recent studies have revealed a poor alignment between the way humans and neural encoder-decoder models generalize to new words (wug test) in the case of english verb past tense @xcite and german plural nouns @xcite . @xcite observed cognitively plausible representations in a recurrent neural network (rnn) trained to inflect german plural nouns but also found evidence of problematic 'shortcut' learning. @xcite observed that transformers resemble humans in learning the morphological inflection of english and german in the wug tests but they also pointed out the divergence of the model in german production. however, computational simulations have succeeded in replicating the u-shaped learning curve during the acquisition of past tense @xcite . additionally, further probing experiments have suggested that neural models do learn linguistic representations @xcite @xcite . our research continues on exploring the cognitive plausibility of neural networks in modeling language inflection learning. ## datasets to replicate the results published by k&c, we employ their dataset based on celex (baayen et al., 1993). 2 to extend the experiments to dutch and german and compare the results to english, we build a new dataset containing past tense forms in all three languages. ## replication of k&c before moving to the main multilingual experiments, we replicate the original k&c experiments (single-task only). ## multilingual experiments this section presents the results of our main experiments aimed at comparing dutch and german past learning patterns to the english ones. it also presents the results of grapheme vs phoneme sequence learning in all three languages. because dutch and german pronunciation is more predictable than the english one, we expect that the difference between grapheme and phoneme learning will be smaller in these languages. for comparability, all experiments in this section use the newly introduced unimorph-based dataset, which includes a similar amount of training forms in all languages (cf. we use our newly-created data for multilingual experiments without resampling tokens by their frequency. this decision is informed by research suggesting that human learners generalize over type frequency, rather than token frequency @xcite and is consistent with the experimental design of k&c. other studies have suggested that word frequency is important for children's past tense acquisition @xcite @xcite , but we do not examine this hypothesis in this work. result overview for the forms seen in training, the model is able to learn both regular and irregular past tense inflection with more than 95% accuracy ( comparing in the following sections, we discuss these results in more detail. ## conclusions in this work, we study the plausibility of using sequence-to-sequence neural networks for simulating human patterns of past tense acquisition. more specifically, we replicate findings by @xcite and examine their generalizability beyond the specific case of english, using a new dataset of english/dutch/german (ir)regular verb forms based on unimorph @xcite . we show that the main findings of k&c also largely hold for dutch and german, including over-regularization errors and the oscillating (or micro u-shape) learning trajectory of individual verb forms across training epochs. at the same time, we also observe cases of non human-like errors, for instance when the model just keeps the present form unchanged or randomly removes phonemes from it. a notable difference among our studied languages concern unseen english irregular verbs, which appear to be much harder to inflect than the dutch and german ones. we also observe that the orthographic transparency of a language influences and possibly confounds the model's learning performance: higher transparent orthography contributes to more reliable and consistent simulation results, but in general this aspect should be seriously considered when setting up new benchmarks of morphological acquisition. future work could include the construction of a nonce word benchmark in dutch and german to enable a multi-lingual evaluation of this task @xcite , as well as an in-depth investigation of the different level of irregular past inflection difficulty in our three languages. @xcite provided very promising evidence for the use of modern neural networks to model the human language acquisition patterns. our work confirms the potential of this research direction, but also raises important issues and joins recent follow-up studies @xcite @xcite that have warned against over-optimistic conclusions.