Linear Discriminative Learning: a competitive non-neural baseline for morphological inflection
Year: 2023

Abstract:
This paper presents our submission to the SIGMORPHON 2023 task 2 of Cognitively Plausible Morphophonological Generalization in Korean. We implemented both Linear Discriminative Learning and Transformer models and found that the Linear Discriminative Learning model trained on a combination of corpus and experimental data showed the best performance with the overall accuracy of around 83%. We found that the best model must be trained on both corpus data and the experimental data of one particular participant. Our examination of speaker-variability and speaker-specific information did not explain why a particular participant combined well with the corpus data. We recommend Linear Discriminative Learning models as a future non-neural baseline system, owning to its training speed, accuracy, model interpretability and cognitive plausibility. In order to improve the model performance, we suggest using bigger data and/or performing data augmentation and incorporating speakerand item-specifics considerably.

heading: 1 Introduction
text: There has been a heated debate on whether human language users generate language in a generative manner (e.g., Chomsky and Halle (1968)) or an output-oriented manner (e.g., Prince and Smolensky (2004)). In accordance with the theoretical stance, computational models have been proposed. The generative approach is essentially rule-based at an abstract level. The Minimal Generalisation Learner (MGL) by Albright and Hayes (2003) is
† These authors contributed equally to this work.
Senior and corresponding author: Kevin Tang (kevin.tang@hhu.de)
a traditional, symbolic rule learner. More recent rule-based computational approaches include Allen and Becker (2015) and Belth et al. (2021).
With the availability of large corpus data, outputoriented models have become widely popular. Output-oriented models can be rule-based or endto-end. The former includes Prince and Smolensky (2004) and Lignos et al. (2009); in the former, a search procedure is implemented on a set of candidates, or outputs, in order to find the surface form that is most compatible with the underlying representation. On the other hand, output-oriented models can be rule-free. For instance, Malouf (2017) showcased a recurrent deep learning model to predict paradigm forms. Similarly, Kirov and Cotterell (2018) proposed a encoder-decoder network architecture to model linguistic phenomena.
Both approaches have their own advantages and disadvantages. However, in terms of model performance, output-oriented models surpass generative ones presumably due to difficulty of incorporating many other variations in conversation. Nevertheless, output-oriented models are not panacea. They are not as cognitively motivated, thus making them less appealing for cognitive research. Deep learning-based models, in particular, show great performance, but they lack interpretability. These shortcomings necessitate a hybrid model, which is i) cognitively motivated, ii) more agnostic than generative models, and iii) more transparent than deep learning-based models.
Recently, Baayen et al. (2019) proposed Linear Discriminative Learning (LDL), part of the discriminative lexicon (Chuang and Baayen, 2021). As the model follows the Rescorla-Wagner Rule and Widrow-Hoff Rules, some insight into human cognition can be obtained through model imple-
138
mentation. Moreover, as it implements a linear mapping between form and meaning in simple two layers without any hidden layer, LDL features higher interpretability and embraces linguistic engineering. Considering these advantages, an LDL model was chosen as the main model for the SIGMORPHON 2023 shared task 2, which aims to generalize morphological inflections in Korean. A transformer model, which has been state-of-the-art models for various NLP tasks, was also implemented for comparison. The code and data are available here: https://github.com/ hhuslamlab/sigmorphon2023

heading: 2 Related Work
text: While neural-based systems typically dominate SIGMORPHON challenges, perhaps because they generally perform well in morphological inflection tasks, their limitations can be better examined using wug testing. For instance, McCurdy et al. (2020) examined the ability of modern EncoderDecoder (ED) architectures to inflect German plurals and concluded that ED does not show humanlike variability as shown in wug data. In fact, recent SIGMORPHON challenges involve learning from corpora that better represent the actual linguistic input of children (such as child-directed speech) and evaluating on phonetically-transcribed spoken production by children or adults in corpora or in experiments. For instance, the SIGMORPHON 2021 shared Task 0 Part 2 was to predict the judgement ratings of wug words (Calderone et al., 2021) as opposed to using real words held-out from the training data as test data. Similarly, the SIGMORPHON 2022 challenge involved computational modeling of the data drawn from corpora of child-directed speech and evaluation on children’s learning trajectories and erroneous productions (Kodner and Khalifa, 2022; Kakolu Ramarao et al., 2022).
Turning to studies from the field of laboratory phonology, there is a long history of training models on corpora to learn specific aspects of morphophonological grammar and evaluating their productivity with experimental data (wug-test and acceptability judgement). For instance, Jun (2010)’s study on stem-final obstruent variation in Korean trained a model with multiple stocastic rules (Albright and Hayes (2002)’s Paradigm Learning Model) in which the acquisition of morphology is based on the distributional pattern of the learning data, using the Sejong corpus, and evaluated on acceptabil-
ity judgement data. Related to the linguistic phenomenon in this study, Albright and Kang (2009) conducted a computational modeling of inflected forms of Korean verbs using the Minimal Generalization Learner algorithm (Albright and Hayes, 2002) and evaluated the model’s performance with attested child errors and historical changes.
Finally, there is a growing number of morphological inflection studies that use the Linear Discriminative Learning model (which will be introduced later) to train on corpus data and evaluate on experimental data (Nieder et al., 2021; Heitmeier et al., 2021; Chuang et al., 2020; Heitmeier and Baayen, 2020; Baayen et al., 2018) and they typical yielded relatively high performance (compared to traditional, symbolic rule learner) while being easy to interpret and cognitively motivated.

heading: 3 Task and Evaluation Details
text: We challenge the shared task 2 Cognitively Plausible Morphophonological Generalization in Korean. The aim of this task is to predict human responses to a generalization task (wug-test), considering high-frequency, low-frequency, and pseudoword items. This implies that human responses may vary depending on the word frequency and familiarity.
The phonological phenomenon to be tested through this task is Korean Post-Obstruent Tensification (henceforth, POT). In Korean, when a lenis consonant in the coda is followed by another obstruent, it can be tensified. However, when a consonant cluster occurs in the coda position, it undergoes Consonant Simplification (henceforth, CS) before POT if the following segment is either an obstruent or a sonorant. On the other hand, neither CS nor POT does occur when the following segment is a vowel.
Depending on the type of the deleted consonant, variation can occur, which can be affected by such speaker- and item-specific features as language familiarity and frequency. In this regard, the main task can be rephrased as predicting features as completely as possible in accordance with those in the answers. For this task, both corpus and experimental data are provided, which include variation patterns. Models are to be evaluated on the accuracy of the prediction of the feature vectors given the corresponding features in the answers from unseen participants.

heading: 4 Data
text: Both corpus and experimental data are provided for this task. The National institute of Korean Language (NIKL) Korean Dialogue corpus (NIKL, 2021) is provided as the main corpus data. All the word tokens affixed with -lC verbs, except for -lh final stems, are provided after manual annotation by the organizers. They further are categorized as lC+Obstruent, lC+Sonorant, and lC+Vowel, depending on the type of the consequent segment. lC+Obstruent and lC+Sonorant data each include target words with morphological boundaries and produced words with syllable breaks both in a Romanized form and Korean orthography. Whether target words undergo POT is also provided, as well as such features as obstruent deletion and lateral deletion pertinent to CS and POT.
On the other hand, lC+Vowel data only provide target words with morphological boundaries both in a Romanized form and Korean orthography. They do not include produced word information as this condition is not subject to POT and CS; thus, all the feature values of obstruent deletion and lateral deletion are labeled as 0 with the POT value being labeled as 0. Whether the target is lateralizated or nasalized is marked only in the lC+Sonorant data while labeled as NA in the others. The number of tokens in lC+Obstruent data is 876, that of tokens in lC+Sonorant data is 95, and that of tokens in lC+Vowel data is 2,525 with 514 types–1,485 if frequency information in lC+Vowel data is ignored. Thus, the total number of tokens in the NIKL data is 3,496.
In addition to the adult corpus, some part of a child spontaneous speech corpus, the Ko corpus (Ko et al., 2020), is provided. As with the NIKL data, the Ko corpus provides target words and their production but solely in a Romanized form. The POT value with the lateralization and nasalization feature values are labeled. As it does not include any lC verbs followed by a sonorant, both nasalization and lateralization are not applicable. A total of 336 tokens are provided.
In the case of experimental data, the experimental responses of 12 participants, in addition to 4 participants for the development data, are provided. They include what are included in the corpus data with the experimental specifics: the trial number, the trial ID, the subject ID, option, language familiarity, and word frequency. A total of 2,843 tokens
are provided.

heading: 5 System Description
text: The main task is to accommodate as many variation patterns as possible. We navigated through all the corpus and experimental data, in which process we found inconsistencies in transcription in the data. In particular, the Ko corpus includes items transcribed in a very detailed manner, including phonological processes, such as deletion and insertion, other than those pertinent to the task. We also found that the target item does not always feature one-to-one mapping, but more-than-one mapping.
Based on the observations above, data preprocessing was of primary importance and was the most time-consuming component. We manually corrected the Ko corpus and automatically unified the transcription style. We then selected models adequate to this task. Considering the nature of the shared task that investigates morphological variations with both corpus and experimental data and the time constraint, LDL was chosen as the main model, along with a Transformer model that has demonstrated great performance in NLP. For each of the two modelling approaches we conducted two studies: Study 1 experimented with systems that train only on the corpus data and/or only on the experimental data; Study 2 used the insights from Study 1, and experimented with systems that train on both the corpus and the experimental data.

heading: 5.1 Linear Discriminative Model
text: LDL generates a system of form-meaning relations by discriminating between different forms and meanings, with forms and meanings being represented by numerical vectors. Form vectors are based either on segmental representations of various lengths, or on representations of acoustic transitions gleaned directly from the speech signal (Arnold et al., 2017; Shafaei-Bajestan et al., 2021). Meaning itself is taken to be a dynamic concept, being emergent from the context in which words are being used, and is represented by semantic vectors, similar to approaches in distributional semantics (Boleda, 2020). The idea is that if both forms and meanings can be expressed numerically, we can mathematically connect form and meaning, i.e. map meaning onto form and vice versa. In this system of mappings, the two sets of vectors are combined into matrices – a form matrix and a semantic matrix. The form vectors are mapped onto
semantic vectors to model comprehension, and semantic vectors are mapped onto form vectors to model production. The mapping between them at the theoretical end-state of learning is predicted using multivariate multiple linear regression (hence the term ‘Linear Discriminative Learning’). The network is simple and interpretable, because, in contrast to deep learning networks, it features just two layers (i.e. the form and meaning matrices), both of which are linguistically transparent.

heading: 5.1.1 Data Preprocessing
text: Due to the time limitation, we decided to winnow out the data that are only pertinent to POT and CS from the Ko corpus. To be specific, the Ko corpus includes tokens involving other phonological processes, like insertion, as well. For instance, there are 6 instances of ipko and the produced forms are ikgo, lipgo, tipgo, ikgu, linkgo, and nipgo. Considering POS and CS rules, the ideal outputs are ipgo and ikgo. Moreover, based on the tendency of negative vowelization of the mid rounded vowel in conversation, ikgu is another candidate. The others are also producible, especially considering that the Ko corpus consists of children speech, but they are definitely not canonical outputs from the pertinent rules. Thus, if the input and the output are hugely different from each other because of other phonological processes, the tokens were discarded. All the duplicated tokens were also removed. Thus, 286 tokens were left from 336 tokens. Lastly, morphological boundary and feature representation were manually incorporated following the style of the other data.
We also observed that there are inconsistencies in transcription style between the two corpus data and the experimental data. The following data transformations were performed on the corpus data. In the Production_R, the tense forms of the plosives p*,t*,k* are replaced with b,d,g, and those of the alveolar fricative and the alveolo-palatal affricate S,c* are replaced with s*, J, respectively. Moreover, there are several inconsistencies in transcription style between the input (Morphology_R) and the output (Production_R). First, the middle yin diphthong yv or jv in Morphology_R is replaced with jv or yv in Production_R. Second, the alveolar fricative S in Morphology_R is replaced with s*, but the reverse transformation is conducted in Production_R. Lastly, the tense stops in Korean orthography P, T, K in Morphology_R are replaced with p, t, k, except when they occur in
the word-initial position. As a result, the pre-processed data contained s* as phone representation. That is, a single phone is represented by two symbols. For triphones, this would lead to unwanted consequences: Triphones which contain information only on two phones, i.e. s* and any other phone, and triphones which contain only one of the two parts of s*. Therefore, s* was replaced with S for the implementations of LDL presented in the subsequent sections.

heading: 5.1.2 General Model Architecture
text: The form matrices C used for the present implementations of LDL consisted of triphones, i.e. sequences of three phones within a word form. Triphones overlap and can be understood as proxies for phonological transitions. In each word’s individual form vector c, the presence of a triphone is marked with 1, while the absence is marked with 0. The form vectors of all words of a set of words constitute its C matrix and each row in such a C matrix represents a word form, while the columns of the C matrix represent all triphones of its underlying word set. Triphones were used as previous studies found overall good performance for triphones (e.g. Chuang et al. 2021; Schmitz et al. 2021).
The semantic matrices S used for the present implementations of LDL deviate from those usually found in studies using LDL. Commonly, semantics are introduced via semantic vectors obtained by methods of distributional semantics, e.g. via fastText (Bojanowski et al., 2016) or naive discriminative learning (Baayen et al., 2011). However, with the small amount of language data provided, the computation of such semantic vectors is barely feasible. While creating semantic vectors based on a larger corpus of Korean may be one option to solve this issue, we decided against this solution as it would mean using data that is not part of the current challenge. Instead, semantic vectors were created based on morphemes and in a binary fashion. That is, similar to the form vectors, in each word’s individual semantic vector s, the presence of a morpheme is marked with 1, while the absence is marked with 0. The semantic vectors of all words of a set of words constitute its S matrix.
With C and S, one can straightforwardly map forms onto meanings and meanings onto forms:
CF = S
SG = C
If one wants to predict the forms or semantics for words that are not yet part of the implementation, additional steps are required. Predicting semantics for newly introduced forms, one computes with C ′ denoting the Moore-Penrose generalised inverse. Using the transformation matrix F and a combined form matrix for previously and newly introduced forms Ccombined, then
S = CcombinedF
Using this method, previous studies have analysed the semantics of pseudowords (Cassani et al., 2020; Chuang et al., 2021; Schmitz et al., 2021). Adapting this method for the prediction of forms, as for the present study, one computes
G = S′C
Then, using the transformation matrix G and a combined semantic matrix for previously and newly introduced words, the following is solved:
C = ScombinedG
Note that this method comes with an important caveat: Newly introduced words must not contain any triphones that are not part of the original C matrix when predicting their meaning, and, in the present case, they must not contain any morphemes that are not part of the original S matrix.

heading: 5.1.3 Study 1
text: For a first implementation of LDL, the following rationale was adopted. First, the combined data of the NIKL and Ko corpora were taken to represent the mental lexicon of a speaker of Korean. That is, we assumed that this knowledge is shared by all participants. Second, based on this shared prior knowledge, participants individually produced word forms during the experiment. Predicting these forms, and in turn the phonological processes underlying them, via prior knowledge was the aim of this implementation.
The combined NIKL and Ko corpus data were used as initial word set (n = 632 after duplicate removal). Based on the corpus data, C and S matrices were created following the specifications in Section 5.1.2. After obtaining the required transformation matrix G via G = S′C, G was based on the triphone to morpheme relations found in the corpus data it was trained on. In a next step, one would then use G to compute C = ScombinedG. However,
the experimental data contained 111 triphones (out of 247) that were not part of the corpus data. As G was not trained to predict these triphones, any further computations were rendered meaningless.

heading: 5.1.4 Study 2
text: Instead, a second LDL network was implemented. The rationale of this implementation was to first create individual networks for all sixteen train and dev participants. Each participant’s network was trained on the combined corpus data and on their experimental data. In a second step, each of the sixteen participants and their networks were then used to predict all other participants’ produced word forms. This provided insight in how far pertinent participants were able to predict other participant’s productions, allowing the selection of a ‘best’ participant to then predict the test participants’ produced word forms.
First, for each of the sixteen train and dev participants a data set containing the combined NIKL and Ko corpus data (n = 632 after duplicate removal) as well as their experimental data was created (n = 175 to n = 180). Based on this data set, C and S matrices were created and comprehension as well as production were modeled following the specifications outlined in Section 5.1.2.
Second, each of the sixteen participants’ G matrices was used to predict the forms produced by all other train and dev participants in the experiment. In contrast to Section 5.1.3, this computation did not pose a problem as experiment items and their triphones were already introduced during the first step. As a result, we obtained prediction accuracies for all sixteen participants by all sixteen participants. Accuracy here refers to whether a word form was predicted correctly. The overall and individual accuracies for low-, high-frequency, and pseudoword items are available on GitHub.
Across all sixteen train and dev participants, it was found that participant 597515 clearly outperformed the other fifteen participants in terms of prediction accuracy across all experimental items. Their mean prediction accuracy across all experimental items was 73%, with 76% for low frequency, 71% for high frequency, and 73% for pseudoword items. Their overall Precision, Recall, and F1 scores for the training data are given in Table 1.
In an attempt to understand why this particular participant showed the best prediction results for the other fifteen train and dev participants and to find out whether we could determine differ-
ent ‘best’ participants for different participants to be predicted, we implemented three multiple regression models for each of the sixteen train and dev participants, i.e. one for high frequency, one for low frequency, and one for pseudoword items. For a given participant’s multiple regression models, the dependent variable was the set of prediction accuracies reached by the other participants for that participant. As predictors, the biographical background information, LANGUAGEPREFERENCE and AGESTARTEDSPEAKING, were introduced. Across the sixteen low frequency item models, we found that one participant with a LANGUAGEPREFERENCE of 3 showed an effect for LANGUAGEPREFERENCE (p = 0.02). This presumably indicated that the other participant with a LANGUAGEPREFERENCE of 3 was the ‘best’ prediction candidate for this participant. Across the sixteen high frequency item models, we found that both participants with a LANGUAGEPREFERENCE of 3 showed an effect for LANGUAGEPREFERENCE (p = 0.02; p = 0.0002), indicating that they were each other’s best prediction candidates. Another participant showed a barely significant effect of LANGUAGEPREFERENCE (p = 0.046), and yet another participant showed a significant effect of AGESTARTEDSPEAKING (p = 0.03) . Across the pseudoword item models, no effects were found. As these results were inconclusive, we decided to drop this attempt and to use participant 597515’s G matrix to predict the forms, and hence the underlying phonological processes, for the seven test participants.
The predicted forms and their underlying representations were used to derive information on which of the five phonological processes of interest were predicted for a pertinent word form.

heading: 5.2 Neural Network

heading: 5.2.1 Data Preprocessing
text: See the data preprocessing steps in Section 5.1.1.

heading: 5.2.2 Model Architecture
text: Our model closely follows the formulation of the encoder-decoder Transformer for characterlevel transduction model proposed by Wu et al. (2021). We use multi-headed Transformers with self-attention and implement them with Fairseq (Ott et al., 2019) tool, a PyTorch-based sequence modeling toolkit. Both Encoder and Decoder have four layers with four attention heads, an embedding size of 256 and hidden layer size of 1,024. We use Adam Optimizer (Kingma and Ba, 2015), with an initial learning rate of 0.001, a batch size of 400, 0.1 label smoothing and 1.0 gradient clip threshold. Models are trained for a maximum of 3,000 optimizer updates. Checkpoints are saved every 10 epochs. Beam search is used at the decoding time with a beam width of 5.
The checkpoint with the smallest loss on the development data is chosen as the best model.
For the evaluation, we consider the models’ sequence accuracy (henceforth, accuracy), where only instances for which the entire output sequence equals the target are considered correct.

heading: 5.2.3 Study 1
text: The inputs to each model are the individual characters of the romanized. For example, for the model trained against the raw NIKL dataset, the input is J a l p - k o and the output is J a l . g o
Model Training Three models were trained on i) the raw NIKL dataset (with a total of 1485 tokens), ii) the raw Ko corpus (with a total of, and iii) the combined datasets (NIKL and Ko). The data in each model were split into train (70%), dev (10%), test (20%) sets. The sequence accuracies of the three models are 71.7% (raw NIKL)1, 35.3% (raw Ko) and 65.5% (the combined dataset). Furthermore, we trained a model on all experimental items following the same train-test split as stated above and the accuracy was found to be 69.4%.
While these models were evaluated on a different set of test data, their accuracies can nonetheless suggest how the different datasets should be used in Study 2 (Section 5.2.4). Training on the Ko corpus alone is unlikely to be sufficient as it yielded the lowest accuracy. While combining NIKL with Ko yielded a poorer model compared to just using
1We experimented with a model using the NIKL dataset but without syllable boundaries, and it yielded an accuracy of 71.6% – a negligible difference compared to the model with syllable boundaries 71.7%
NIKL alone, the Ko corpus should not be excluded given that it is arguably more ecologically valid than NIKL and the amount of training data is already small in this challenge. Finally, training only on experimental items resulted in a comparable performance as the combined dataset. This model was not used as it was explicitly discouraged by the challenge.
To determine how well a model trained only on corpus data would perform on the experimental data, we evaluated the best model (trained on raw NIKL) against the experimental data (and removed the syllable boundaries in the predictions to match the transcription style of the experimental data) and it yielded a much lower accuracy of 29.9%, suggesting that we should incorporate the experimental data as part of training.

heading: 5.2.4 Study 2
text: In this study, we primarily used the pre-processed dataset (using methods described in section 5.2.1) that consists of both NIKL and child spontaneous speech dataset. We then incorporate parts of experimental data along with the combined dataset during both training and development phase. The test data provided by the organizers is used during the testing phase.
Model Training We first incorporated models with training the combined dataset using i) productions from best participant and ii) productions from worst participant, as development data , that yielded accuracy scores of 43.8% and 39.4%.
Next, we trained a model on the combined dataset (NIKL, and the Ko corpus) with the productions from 4 best participants and using responses from a random participant as development data which produced an accuracy score of 68.1%.
Finally, a model was trained on all participants’ except the best participant’s responses with the combined dataset (NIKL, and the Ko corpus) and using the productions from best participant as development data that yielded an accuracy of 69.2%. The accuracies of this model for: i) low-frequency ii) high-frequency and iii) pseudoword items are 64.4%, 77.7% and 65.38% respectively. The overall Precision, Recall and F1 scores for the five phonological processes in the test data are given in Appendix A.

heading: 5.3 Results
text: Predicting the seven test participants’ productions using the ‘best’ participant’s LDL network as detailed in Section 5.1.4, an overall accuracy of 83.32% was reached. The accuracies of this model for: i) low-frequency ii) high-frequency and iii) pseudoword items are 83.56%, 83.58% and 82.84%. The overall Precision, Recall, and F1 scores for the test data are given in Table 2. The model performed best on tensification (F1: 0.98), and worst on lateralisation (F1: 0.41). The mean perplexity scores for the train and dev as well as for the test data are 2.11 and 1.97 respectively. The performance of the model on the test data is similar to that on the training data (Table 1) with the exception of simplify delete obstruent being better predicted than lateralization in the test data. Comparing to the best Transformer model, LDL performed better in terms of the overall accuracies of the model; however, the relative performances of the five phonological processes (Precision, Recall and F1 scores) are largely the same (Appendix A).

heading: 6 Variability: Items and Participants
text: To examine the variability of the phenomenon, Shannon entropy (base 2) (Shannon, 1948) was used to quantify how variable are the items in the experimental data and how variable are the participants. In this study, we considered sixteen possible combinations of the five phonological processes (therefore sixteen events in entropy’s term) (See Appendix C). With sixteen combinations, the highest possible entropy value is 4 which means each combination has a probability of 1/16 indicating a high level of variability, and the lowest possible entropy is 0 which means there is only one attested combination indicating no variability. For detailed analyses, see Appendix B.
First, we computed the by-item entropy values by computing the proportion of the sixteen response combinations using the sixteen participants (training and development). The 180 ex-
perimental items have a mean entropy of 0.584. Pseudoword items have the highest mean entropy (0.612), followed by high-frequency items (0.596) and low-frequency items (0.544). These entropy values suggest that the experimental items in general have low variability and unsurprisingly the pseudoword items were particularly variable compared to the real words. However, these differences in entropy values were not statistically significant (ps > 0.3842).
Second, we computed by-participant entropy values by computing the proportion of the sixteen response combinations. Across all the experimental items, the sixteen participants have a mean entropy of 2.143. Participants show the lowest mean entropy with the high-frequency items (2.049) , followed by low-frequency items (2.111) and pseudoword items (2.112). However, these differences in entropy values were not statistically significant (ps > 0.2542). Our ‘best’ participant 597515 has an entropy of 2.192 across all items, 2.176 for highfrequency, 2.154 for low-frequency and 2.140 for pseudoword items, with all values similar to their means. Therefore, the participant’s superiority is not purely due to their responses being more variable.

heading: 7 Discussion and Conclusion
text: We demonstrated that LDL is capable of modelling morphological inflection trained on limited corpus and experimental data. Its performance is competitive to that of the Transformer model that we experimented with. Past SIGMORPHON shared tasks (2017–2022) with a focus on morphological inflection have generally received more neural-based systems than non-neural ones and found that neuralbased ones tend to be superior (Kodner and Khalifa, 2022; Kodner et al., 2022; Pimentel et al., 2021; Vylomova et al., 2020; McCarthy et al., 2019; Cotterell et al., 2018, 2017, 2016). Amongst the submitted non-neural systems, LDL has never been utilized. Our study cannot conclude that LDL is superior to the transformer architecture as the latter was not fully optimized. However, it has great potential to serve as a non-neural baseline system for future shared tasks as well as allowing researchers to conduct rapid experiments, because of its architecture simplicity, performance (with accuracies from 59% to 99% in a range of languages, e.g. Heitmeier et al. 2021; Schmitz et al. 2021; Stein and Plag 2021; Chuang et al. 2020; Baayen et al.
2019) and speed (in our study one model required on average 35 seconds of CPU processing on an i7-9750H 2.60 GHz system with 32 GB memory).
Our study found that training on the corpus data alone was insufficient and that our models require at least one participant’s experimental data in order to inflect the experimental items well. However, from an ecological perspective, a model should only be trained on the corpus data (NIKL and Ko), excluding the experimental data, as the corpus data serve to represent the participants’ actual linguistic input. The corpus data we have are likely unrepresentative of the actual linguistic input of our participants. Firstly, the verbs were not embedded in an utterance, and even if the full utterances were used the overall amount of data would still be small with only 53,000 words from the Ko corpus, and 900,000 phrases from the NIKL corpus. Based on spoken speech input alone, Brysbaert et al. (2016) estimated that for American English, the total input from social interactions (in a dialogue) would be equal to 11.688 million word tokens per year and a 20-year-old would have been exposed to about 234 million word tokens. Using a much larger speechlike or transcribed corpus such as SUBTLEX-KR (Tang and de Chene, 2014) (90 million eojeols) is a promising approach for examining morphological inflection patterns (see de Chene (2014) on regularisation in Korean noun inflection).
Our item variability analyses suggest that the three item types (high-, low-frequency and pseudowords) are not particularly different in their variability. This might be reflecting how the LDL model reported in Section 5.3 performed similarly with them (high: 83.58%, low: 83.56%, and pseudowords: 82.84%). However, the best Transfomer model was sensitive to item types yielding a higher accuracy for the high-frequency items (77.7%), than the low-frequency (64.4%) and pseudoword (65.38%) items.
Our attempts in understanding why the ‘best’ participant was the best in predicting individual participants’ productions were not successful. Response variability was unable to explain why our ‘best’ participant was the best, as it has neither high nor low in variability compared to the other 15 participants. Our regression analyses predicting individual participant accuracies using the participants’ demographics was inconclusive. While one may assume that the LDL prediction results should improve when one predicts speakers of similar back-
grounds, the nonetheless satisfying LDL prediction results suggest that demographics-matching was not needed. Overall, our results suggest that LDL is suitable for tasks such as the one at hand.
Limitations
The small amount of training data provided in this shared task poses a challenge for models that need large amounts of data to reliably learn linguistic patterns. While we did not employ any data augmentation techniques, we suggest future work to train the models on all the possible feature combinations (weighted with the probabilities as the experimental data) for the stems in the two corpora.
Owning to the lack of time and computing resources, we did not fully optimize our transformer models and we did not fully utilize and explore i) speaker-specific information, especially for the transformer models, ii) token frequency information in the corpora, as we assumed extension of morphological patterns is based on type, not token, frequency (Bybee, 2001; Pierrehumbert, 2001). Furthermore, we did not experiment with training models with either high-frequency, low-frequency and pseudoword items. It is possible that some speakers’ high/low/pseudoword items would be better served as part of the training set.
The LDL model in Section 5.1.3 was not able to evaluate the experimental items due to the unattested triphones. This shortcoming can be mitigated by using phonological features (Tang and Baer-Henney, 2023).
Ethics Statement
In the beginning of the challenge, we discovered the answers of the experimental test data were accidentally released early by the organisers. We immediately informed the organisers and as requested, we deleted the test data and committed to not use it until it was officially released. Our work was trained on speech corpora of adults which were recorded with ethics approval. The broader impact of the work includes i) improving how morphological inflection models can be trained with lowresource languages or phenomena, ii) developing speaker-specific morphological inflection models, iii) establishing a new baseline model architecture (LDL) that has a low carbon footprint.
CRediT authorship contribution statement
CJ, DS, and AKR contributed equally to this work. KT served as the senior and corresponding author on this paper.
We follow the CRediT taxonomy2. Conceptualization: KT; Data curation: CJ, AKR; Formal Analysis: AS, DS; Investigation: KT, DS, CJ, AKR; Methodology: KT; Supervision: KT; Visualization: AS; and Writing – original draft: KT and Writing – review & editing: KT, AS, CJ, DS, AKR.

heading: Acknowledgements
text: We gratefully acknowledge the support of the central HPC system “HILBERT” at Heinrich-HeineUniversity, Düsseldorf.

heading: B Appendix: Variability analyses

heading: A Appendix: Evaluation metrics for the Neural Network model

heading: C Appendix: Feature combinations

heading: Future Work
text: However, it has great potential to serve as a non-neural baseline system for future shared tasks as well as allowing researchers to conduct rapid experiments, because of its architecture simplicity, performance (with accuracies from 59% to 99% in a range of languages, e. g. Heitmeier et al. 2021; Schmitz et al. 2021; Stein and Plag 2021; Chuang et al. 2020; Baayen et al. 2019) and speed (in our study one model required on average 35 seconds of CPU processing on an i7-9750H 2. 60 GHz system with 32 GB memory). Our study found that training on the corpus data alone was insufficient and that our models require at least one participant’s experimental data in order to inflect the experimental items well. However, from an ecological perspective, a model should only be trained on the corpus data (NIKL and Ko), excluding the experimental data, as the corpus data serve to represent the participants’ actual linguistic input. The corpus data we have are likely unrepresentative of the actual linguistic input of our participants. Firstly, the verbs were not embedded in an utterance, and even if the full utterances were used the overall amount of data would still be small with only 53,000 words from the Ko corpus, and 900,000 phrases from the NIKL corpus. Based on spoken speech input alone, Brysbaert et al. (2016) estimated that for American English, the total input from social interactions (in a dialogue) would be equal to 11. 688 million word tokens per year and a 20-year-old would have been exposed to about 234 million word tokens. Using a much larger speechlike or transcribed corpus such as SUBTLEX-KR (Tang and de Chene, 2014) (90 million eojeols) is a promising approach for examining morphological inflection patterns (see de Chene (2014) on regularisation in Korean noun inflection). Our item variability analyses suggest that the three item types (high-, low-frequency and pseudowords) are not particularly different in their variability. This might be reflecting how the LDL model reported in Section 5. 3 performed similarly with them (high: 83. 58%, low: 83. 56%, and pseudowords: 82. 84%). However, the best Transfomer model was sensitive to item types yielding a higher accuracy for the high-frequency items (77. 7%), than the low-frequency (64. 4%) and pseudoword (65. 38%) items. Our attempts in understanding why the ‘best’ participant was the best in predicting individual participants’ productions were not successful. Response variability was unable to explain why our ‘best’ participant was the best, as it has neither high nor low in variability compared to the other 15 participants. Our regression analyses predicting individual participant accuracies using the participants’ demographics was inconclusive. While one may assume that the LDL prediction results should improve when one predicts speakers of similar back-
grounds, the nonetheless satisfying LDL prediction results suggest that demographics-matching was not needed. Overall, our results suggest that LDL is suitable for tasks such as the one at hand. Limitations
The small amount of training data provided in this shared task poses a challenge for models that need large amounts of data to reliably learn linguistic patterns. While we did not employ any data augmentation techniques, we suggest future work to train the models on all the possible feature combinations (weighted with the probabilities as the experimental data) for the stems in the two corpora. Owning to the lack of time and computing resources, we did not fully optimize our transformer models and we did not fully utilize and explore i) speaker-specific information, especially for the transformer models, ii) token frequency information in the corpora, as we assumed extension of morphological patterns is based on type, not token, frequency (Bybee, 2001; Pierrehumbert, 2001). Furthermore, we did not experiment with training models with either high-frequency, low-frequency and pseudoword items. It is possible that some speakers’ high/low/pseudoword items would be better served as part of the training set. The LDL model in Section 5. 1. 3 was not able to evaluate the experimental items due to the unattested triphones. This shortcoming can be mitigated by using phonological features (Tang and Baer-Henney, 2023)