StemCorrupt induces compositional structure
In this section, we analyze the ramifications of training on the synthetic training dataset generated by StemCorrupt ( D^{Syn}_{train} ) and the human-annotated training dataset (D_{train}) . Our analysis focuses on asymptotic behaviour, specifically, how the conditional distribution P(\mathbf{Y}|\mathbf{X},\mathbf{T}) evolves as we add more augmented data ( \lvert D^{Syn}_{train}\rvert\to\infty ).

For all \langle\mathbf{X},\mathbf{Y},\mathbf{T}\rangle datapoints in D_{train} , assume that \mathbf{X} and \mathbf{Y} share a stem \mathbf{Y}_{stem} that is non-empty, and let \mathbf{Y}_{affix} be the remaining characters of \mathbf{Y} . Let \mathbf{X}_{stem} and \mathbf{X}_{affix} be analogously defined. Further, let D^{Syn}_{train} be generated with StemCorrupt using \theta=1 . 5 5 5 That is, we substitute all characters in the stem. Next, consider the data-generating probability distribution P(\mathbf{Y}|\mathbf{X},\mathbf{T}) over D^{Syn}_{train}\cup D_{train} . Then, as \lvert D^{Syn}_{train}\rvert\to\infty , we have that P(\mathbf{Y}|\mathbf{X},\mathbf{T})\equiv P(\mathbf{Y}_{stem},\mathbf{Y}_{affix}|\mathbf{X},\mathbf{T})=P(\mathbf{Y}_{affix}|\mathbf{X}_{affix},\mathbf{T})P(\mathbf{Y}_{stem}|\mathbf{X}_{stem}) .

The augmentation method thus makes the model more effective at capturing concatenative morphological patterns, as the conditional probability distribution becomes factorized into a root generation component ( P(\mathbf{Y}_{stem}|\cdot) ) and an affix generation component ( P(\mathbf{Y}_{affix}|\cdot) ). Crucially, this removes the potential for any spurious correlation between these two substructures. 6 6 6 Additionally, this factorization is likely to reducing overfitting: The model effectively learns to minimize the negative log-likelihood of this simplified and factorized data distribution P(\mathbf{Y}_{stem}|\cdot)P(\mathbf{Y}_{affix}|\cdot) , rather than the complex joint distribution P(\mathbf{Y}_{stem},\mathbf{Y}_{affix}|\cdot) .

While concatenation is a cross-linguistically prevalent strategy for inflection (Haspelmath and Sims, 2013 ) , stems and affixes are rarely entirely independent. Therefore, enforcing complete independence between these structures is an overly strong constraint that StemCorrupt places on the training data. The constraint is consistently violated in Turkish, for example, where front-back vowel harmony constraints dictate that the vowels in the suffix share the same front/back feature as the initial vowel in the stem. This leads to forms like ``daların'' and prevents forms like ``dalerin'', ``dalerın'', or ``dalarin'' (Kabak, 2011 ) . In Section 5 , we show that StemCorrupt regularly generates examples violating vowel harmony, and that this can undermine its effectiveness. Nevertheless, the empirical success of StemCorrupt demonstrates that the benefits of its concatenative bias outweigh its limitations.

Our analysis provides a simple yet the most accurate characterization of StemCorrupt . Previous works have called StemCorrupt a beneficial ``copying bias '' (Liu and Hulden, 2022 ; Jaidi et al., 2022 ) or a strategy for mitigating overgeneration of common character n-grams (Anastasopoulos and Neubig, 2019 ) .
However, our analysis demonstrates that neither of these characterizations are entirely accurate. First, the denotation of a ``copying bias'' is only suggestive of the second factor in our statement P(\mathbf{Y}_{stem}|\mathbf{X}_{stem}) , and does not address the impact on affix generation. In contrast, our analysis shows that both stem and affix generation are affected. Furthermore, alleviating overfitting to common character sequences is also misleading, as it would suggest that StemCorrupt serves the same purpose as standard regularization techniques like label smoothing (Müller et al., 2019 ) . 7 7 7 Our preliminary experiments did not support this position; compositional generalization performance was not sensitive to label smoothing, yet was significantly improved by StemCorrupt .