Frequency matters: Modeling irregular morphological patterns in S panish with Transformers (2025)

Abstract: Over the past decade, various studies have addressed how speakers solve the so-called ‘The Paradigm Cell Filling Problem’ (PCFP) (CITATION) across different languages. The PCFP addresses a fundamental question in morphological processing: how do speakers accurately generate inflected forms of words when presented with incomplete paradigms? This problem is particularly salient when modeling complex inflectional systems. We focus on Spanish verbal paradigms, where certain verbs follow an irregular L-shaped pattern, where the first-person singular present indicative stem matches the stem used throughout the present subjunctive mood. We formulate the problem as a morphological reinflection task. Specifically, we investigate the role of input frequency in the acquisition of regular versus irregular L-shaped patterns in transformer models. By systematically manipulating the input distributions and analyzing model behavior, we reveal four key findings: 1) Models perform better on L-shaped verbs compared to regular verbs, especially in uneven frequency conditions; 2) Robust primacy effects are observed, but no consistent recency effects; 3) Memorization becomes more prominent as the proportion of L-shaped verbs increases; 4) There is a tendency to regularize L-shaped verbs when their consonant alternation pairs are rare or absent in the training data.

## introduction a common generation task in morphology is morphological inflection, where a target form has to be generated from its corresponding lemma and feature tag, e.g., (lemma:decir, target tag:<v;ind;prs;1;@xmath0. a central challenge in understanding how speakers handle morphological inflection is the paradigm cell filling problem (pcfp) @xcite , which asks how speakers can reliably produce inflected forms of words when they are presented with incomplete paradigms. to address the pcfp, encoder-decoder based neural networks have been used to simulate the learning and generation of inflected forms @xcite . our study extends this line of research by applying the pcfp to a morphomic pattern using encoderdecoder transformers. the morphomic pattern, as introduced by @xcite , is a morphological pattern that exists independently of semantics or syntax. it is purely based on the form and structure of words. a key characteristic of morphomic patterns is their predictability within the verbal paradigm. the verb forms that are part of the pattern share morphological features, despite a lack of apparent semantic or syntactic motivation @xcite . @xcite @xcite identified morphomic patterns across romance languages. we focus on spanish for data availability reasons @xcite . to illustrate an example, the spanish verb forms "digo" (1st person singular, indicative) and "digan" (3rd person plural, subjunctive) of the verb decir 'to say' (see spanish exhibits several morphomic patterns, namely l-, n-, p-and f-shaped patterns @xcite . of these, only the l-shaped pattern has been explored for human learnability @xcite . we choose the l-shaped pattern for our study because it allows us to assess the cognitive plausibility of our neural network models. the l-shaped pattern is characterized by the use of a distinct stem form in the first person singu-lar present indicative and all cells of the present subjunctive mood. for example, the irregular verb decir exhibits the l-shaped morphome pattern, as shown in in our study, we investigate the impact of type frequency on the learnability of morphomic patterns, specifically the l-shaped pattern, by implementing a morphological reinflection task framed as a pcfp using transformer models. the morphological reinflection task aligns with the morphological framework of abstraction based on data directly available to speakers (i.e., inflection forms) @xcite , providing a realistic setting. we imple-ment a multi-source setup of the morphological reinflection task @xcite , which uses multiple source form-tag pairs instead of one formtag pair. the task here is to generate an inflected form from two source form-tag pairs and the target feature tag to predict the target inflected form, e.g., (source form 1:digo, source tag 1:<v;ind;prs;1;sg>, source form 2:diga, source tag 2:<v;sbjv;prs;1;sg>, target tag:<v;sbjv;prs;2;@xmath1. the choice of two-source setup is motivated by two key considerations: 1) identifying l-shaped verbs requires knowledge of at least two paradigm cells (one cell within the l-shaped pattern, one cell outside; see the reinflection task is particularly challenging due to the variability of the starting point (the source), which can be any other inflected form of the same lemma. this variability makes the task more cognitively plausible, reflecting the data sparsity encountered by human speakers, who never encounter all of the inflected forms of their language @xcite . the main aims of the study are: 1) to model the learning of the l-shaped morphome in spanish using transformers for morphological reinflection; 2) to analyze the models' performance across varying input frequency distributions of regular and l-shaped verbs; 3) to conduct post-hoc analyses investigating: a) the impact of paradigm cell combinations on l-shaped learning, b) models' ability to memorize and generalize morphomic patterns, and c) sensitivity to the input frequency of consonant alternations. we publish the dataset and code used in our study at https://github.com/hhuslamlab/ modeling_spanish_acl 2 related work @xcite introduced the encoder-decoder approach to pcfp by formulating the problem as a morphological reinflection task. the following paragraphs will provide an overview of the methodologies used to address this problem over time. initial non-neural models focused on learning string edit rules from data using sequence-to-sequence models @xcite and string transductions @xcite . subsequently, @xcite proposed a finite state construction and used a classifier to select the correct inflection. similarly, alegria and etxeberria (2016) used a single weighted finite-state transducer (wfst) @xcite to model the mapping between lemmas and inflected forms. @xcite developed a morphological analyzer that learns from the training data and applies the learned patterns to reinflect test data. @xcite implemented a discriminative transducer based on @xcite that searches for a series of character transformation rules to perform the inflection. @xcite and @xcite applied affixing rules to generate inflected forms, which involved appending or altering affixes (prefixes, suffixes, infixes, etc.) according to morphological rules. more recently, @xcite used a non-neural system to predict inflected forms based on string patterns observed in training samples. @xcite introduced an improved affixing system that incorporated additional linguistic information to better capture the complexities in morphological generation. these methods, while interpretable, often struggled with irregular forms and low-resource scenarios. the introduction of neural-based sequence-tosequence models marked a significant milestone in modeling morphological reinflection by learning complex morphological patterns without explicit rules @xcite @xcite . subsequent studies built upon this approach, including the hard monotonic attention for strict alignment between input and output sequences @xcite , multisource setups with bi-directional lstms @xcite , character-level lstms (silfverberg and hulden, 2018a), and encoder-decoder based transformers @xcite . recent developments include phonologically-aware embeddings @xcite to capture both orthographic and phonetic information of words. other approaches include treating morphological reinflection as a classification problem @xcite , and the application of imitation learning @xcite . lastly, large language models have also been explored for morphological reinflection, including analyzing chatgpt's capability in morphological generation across multiple languages @xcite . our study implements a model that closely follows the formulation of the encoder-decoder transformer for character-level transduction proposed by @xcite , due to its high performance on inflectional tasks across various languages @xcite @xcite . ## analysis and results we begin by presenting the sequence accuracies of the models across the three frequency conditions. subsequently, we conduct a series of post-hoc analyses to understand the factors influencing learning of morphomic patterns under varying frequency conditions. in section 4.1, we examine potential position biases in models. section 4.2 investigates the models' sensitivity to memorization versus generalization effects @xcite . lastly, in section 4.3, we examine the models' behavior for specific phonological alternations. we analyze the models' performance across varying input frequency distributions of regular and l-shaped verbs, addressing our research aim 2. we use two evaluation metrics: sequence accuracy and stem-only accuracy, as the errors can only occur in the suffixes and the stem of the predicted form @xcite . we first evaluate sequence accuracies (refer to figure 2 ) across frequency conditions. in the 10%l-90%nl condition, nl-shaped verbs have a mean accuracy of 61.8%, which is 25.05% higher than the 36.75% mean accuracy of l-shaped verbs. conversely, in the 90%l-10%nl condition, lshaped verbs have a mean accuracy of 88.75%, outperforming nl-shaped verbs by 64.58%, which have a mean accuracy of 24.17%. in the 50%l-50%nl condition, l-shaped verbs have a mean accuracy of 72.31%, while nl-shaped verbs have a mean accuracy of 55.19%, with l-shaped verbs performing 17.12% better. subsequently, we analyze stem accuracies to isolate the models' performance on stem alternations across the frequency conditions (appendix a.1). the results show a clear trend: the more frequent verb type in each condition consistently has better stem accuracy. in the 10%l-90%nl condition, nl-shaped verbs achieve a mean stem accuracy of 68.59%, which is 11.89% higher than the 56.7% mean stem accuracy of l-shaped verbs. in the 50%l-50%nl condition, l-shaped verbs have a mean stem accuracy of 77.24%, slightly better than nl-shaped verbs at 70.25%, with a difference of 6.99%. in the 90%l-10%nl condition, l-shaped verbs have a higher mean stem accuracy of 89.51%, outperforming nl-shaped verbs which have a mean stem accuracy of 31.09%, showing a difference of 58.2%. these results show a clear performance difference between the models based on the distribu-tion of verb types in the training data. in the 10%l-90%nl condition, which most closely approximates the natural distribution in spanish, the models perform better on nl-shaped verbs than on l-shaped verbs. however, we find a learning advantage for l-shaped verbs in other conditions, suggesting the models might be learning specific characteristics of l-shaped verbs. to further understand the factors influencing the acquisition of morphomic patterns in the models', we conduct a series of post-hoc analyses. ## conclusion in this paper, we examine the learning capabilities of transformer models with respect to morphomic pattern, specifically the l-shaped pattern in spanish. we conduct a series of post-hoc analyses to understand the factors influencing the learning of morphomic patterns under varying frequency con-ditions. transformer models have shown remarkable ability to learn irregular patterns in related language tasks, such as english past tense inflection, german noun plurals and arabic noun plurals @xcite . however, morphomic patterns are complex linguistic patterns that are hard to acquire @xcite . the models' ability to capture morphomic patterns was not guaranteed because these patterns operate independently of semantics, syntax, and phonology. in our study, the models' performance on l-shaped verbs, especially in conditions with uneven frequency, indicates that transformers have developed some level of competence in recognizing and applying morphomic patterns from the training data. we first look at the learning strategies of the transformer with respect to input ordering. we observe a clear primacy effect in models' processing of l-shaped verbs, suggesting that the models are more influenced by the first source cell in making predictions. we also observe that models have higher accuracy in predicting target forms that are part of l-shaped pattern compared to those outside the l-shaped pattern within a morphological paradigm. this bias suggests that the models are capturing some aspects of the overall paradigm structure, particularly the distribution of irregular forms within the l-shaped pattern. in our investigation of models' strategies for balancing memorization and generalization, we show that the type frequency of l-shaped verbs impacts the models' ability to both memorize and generalize stem-final consonant alternations. for memorization, we observe a positive correlation between the frequency of l-shaped verbs in the training data and the model's ability to correctly reproduce seen stem-final consonants. this suggests that increased exposure to l-shaped verbs enhances the model's ability to retain and apply stem-final consonant alternations. this finding aligns with studies which show morphological patterns with higher type frequency to be more productive @xcite ; baer-henney and van de vijver, 2012, inter alia). however, when it comes to generalization, we observe a non-linear relationship between the frequency of l-shaped verbs in the training data and the model's ability to produce unseen stem-final consonants. we find that generalization performance peaks in the balanced condition (50%l-50%nl), but decreases in skewed conditions (90%l-10%nl and 10%l-90%nl conditions). while increased exposure to l-shaped verbs enhances memorization, it may not necessarily lead to better generalization. while the models demonstrate some ability to learn and apply the l-shaped pattern, they exhibit a clear preference for regular patterns when encountering unfamiliar consonant alternations. this suggests that the models simply rely on frequencybased pattern and have not fully acquired abstract morphological rules. furthermore, the models' superior performance on regular verbs in the 10%l-90%nl condition (a close approximation to natural language distribution) validates the results from @xcite 's study. in their study with spanish speakers on a wug-test-like inflection task, 71.9% of the participants showed a preference to nl-shaped responses over l-shaped responses. this similarity in behavior between transformers and human participants motivates a deeper comparison. in the future, we aim to compare our model's performance with the experimental results from human speakers, such as those from @xcite and @xcite . through this comparison, we could assess how effectively our computational approach captures the linguistic and cognitive phenomena observed in human morphological processing in the context of morphomic patterns.