Morphological Inflection with Phonological Features (2023)

Abstract: Recent years have brought great advances into solving morphological tasks, mostly due to powerful neural models applied to various tasks as (re)inflection and analysis. Yet, such morphological tasks cannot be considered solved, especially when little training data is available or when generalizing to previously unseen lemmas. This work explores effects on performance obtained through various ways in which morphological models get access to sub-character phonological features that are often the targets of morphological processes. We design two methods to achieve this goal: one that leaves models as is but manipulates the data to include features instead of characters, and another that manipulates models to take phonological features into account when building representations for phonemes. We elicit phonemic data from standard graphemic data using language-specific grammars for languages with shallow grapheme-to-phoneme mapping, and we experiment with two reinflection models over eight languages. Our results show that our methods yield comparable results to the grapheme-based baseline overall, with minor improvements in some of the languages. All in all, we conclude that patterns in character distributions are likely to allow models to infer the underlying phonological characteristics, even when phonemes are not explicitly represented.

## introduction in recent years, morphological tasks received much attention in nlp through various tasks such as (re)inflection, lemmatization and others, specifically through the sigmorphon shared tasks @xcite @xcite @xcite . state-of-the-art models seem to achieve quite high results in such cross-lingual evaluation campaigns, although recent works showed that there is still room for improvements @xcite . most studies aiming at morphological tasks design models that operate at the character level, without reference to the phonological components that compose the phonemes represented by the characters. 1 this is despite the fact that many morphological processes have distinct phonological features, rather than phonemes, as either the trigger or target of morphological processes. for example, in vowel harmony, a single feature of a vowel in the stem determines the vowels that appear in the affixes added to that stem. without direct evidence of the phonological features composing every phoneme, models must resort to memorizing groups of phonemes that pattern together for an unobserved reason. in this work we hypothesize that explicitly inputting models with phonological features will lead to better modelling of morphological tasks. we set out to equip models with two alternative methods for incorporating that information. one method replaces the character-level tokens with phonological feature tokens; and another one equips the model with a self-attention mechanism that learns representation of phonemes from their features. we implement these methods on the task of morphological reinflection, where forms of the same lemma are inflected from one another. we experiment with 8 languages and 2 models: an lstm encoder-decoder with global attention by @xcite ; and a transducer by @xcite that predicts edit actions between source and target word-forms and is suitable for lower amounts of training data. our experiments show that the proposed methods yield results comparable to the grapheme-based baseline setting for the transducer model. on average across languages, the best phonologicallyaware method suffered from a drop of 2.8 accuracy points, although the performance on some individual languages marginally improved. we thus conjecture that the phonological characteristics are already encoded in the graphemic representations elicited by this model. the results of this work are in line with other works, performed in different settings, investigating the role of phonology in morphological models (see section 6). we further note that the lstm model, unlike the transducer, did not perform well on graphemic data and suffered from a severe drop when applied on phonological data in all tested languages. we attribute this to the transducer's attested ability to perform well particularly in low-resource setting. we subsequently conjecture that, for the phonologically-aware variant of the reinflection task, standard amounts of reinflection data should be effectively considered low-resourced. ## morpho-phonological processes utterances in natural language -sentences and words -are composed of phonemes. yet, one can further decompose phonemes to their very atomic elements: phonological distinctive features. a phonological feature is the minimal unit within a phoneme that distinguishes it from other phonemes. every phoneme can be described as a unique combination of such features. vowels, for example, are said to take the features: backness of the tongue, height of the lower jaw, and roundness of the lips; the sound /a/ then has the values front, open and unrounded. consonants usually take the features: place of articulation, manner of articulation and voiceness, e.g. /g/ has the values velar, plosive and voiced. 2 many languages exhibit morphological processes whose target or trigger are phonological features. for instance, turkish has vowel harmony at the backness feature: the stem's last vowel controls (harmonizes) the backness of other vowels in morphemes added to that stem. when a character-level inflection model learns this process, it has to memorize the relation beels) instead of aligning vowels explicitly by their backness feature. in general, describing such processes at the grapheme level is often intricate and requires models trained on morphological tasks to put unnecessary effort in learning patterns that are more complicated than their original cause. because the underlying phonological information is not explicitly shown to them, instead of learning simple rules of phonological features, they memorize groups of characters that pattern together for no observable reason. a model that is aware of phonological features would be able to easily learn these relations and treat morpho-phonological processes straightforwardly. in order to construct such a model there is a need for phonologically annotated data or for a tool that converts words to their corresponding sequences of phonemes (their verbal pronunciation) and decomposes the phonemes into their phonological distinctive features. a simple option would be to employ a component that performs grapheme-tophoneme (g2p) and phoneme-to-grapheme (p2g) conversions for every language, as well as decomposes the phonemes to their corresponding distinctive features. thus, every character-level model would be able to process phonological data. in the next section we present two ways to incorporate such signals into the data and models for morphological tasks. ## modeling reinflection with phonology we set out to re-model morphological tasks by integrating phonological information, in order to make phonological processes explicitly learnable for models. we propose two generic methods that are applicable to any morphological model. formally, we denote 3 alphabets, for graphemes σ g , phonemes σ p and phonological features σ f . the first one is language-dependent while the others are universally defined in the ipa list of symbols and features (association, 1999). 3 we treat a word as a tuple of its composing graphemes @xmath0. correspondingly, the sequence of phonemes that is the result of applying the g2p component to g is denoted by @xmath1, and the phonemes' decomposition to a sequence of features is denoted by @xmath2. suppose we have a morphological task t , in which the input is g src and the output ground truth is g trg . that is we denote m t as a model that solves t , i.e. it takes g src and s, and generates ĝtrg , a prediction of the target word: in order to incorporate the phonological information to m t , its inputs should obviously be changed to include this information -either phonemes or phonological features. however, changes can also be done to m t itself to treat better the new inputs. we thus propose two methods for inducing phonological information to morphological models: one manipulates only the source and target data to include phonological features, and one adds a learnable layer to the model in order to facilitate better processing of the new input. both methods leave s untouched, the model processes s in the exact same way as in the graphemic setting. ## results and analysis to provide insights into the sufficiency of the data and the richness of the signal, we plot on figure 1 (in appendix a) learning curves for the transduce model per language. we trained each model over an increasing number of train samples from 1,000 to 8,000 and evaluated them on the same test sets for each language. the general trends show that the amount of data is indeed sufficient for the model and the signal is not richer, as in most cases the test accuracy with 8,000 samples is similar to the one with 3,000 samples. moreover, the graphs show that our methods have no clear advantage over the baseline even in as few as 1,000 training examples. ## discussion and conclusion in this work we incorporated phonological information into morphological tasks. we proposed two methods: one that modifies the data, and one that also manipulates the model. we exemplified them on reinflection for two models and found out that, on average, our methods are comparable with the baseline and do not surpass it. we conclude that the embeddings obtained for the graphemic representations in such tasks may already encode the underlying phonological information in the data. this conclusion is in line with the work of @xcite , who similarly aimed, with no success, to use phonological data in morphological inflection. unlike our work, they used a weaker inflection model as a baseline for modification and they had a different method in constructing the phonologically-aware embeddings. more crucially, they experimented with a form-split setting, which means that there was significant overlap between the sampled lemmas in the train-test split. our results also corroborate the findings of @xcite , who examined phoneme embeddings from various sources, including from a morphological inflection model, and showed that they implicitly encode phonological features, thus supporting our main conclusion.