Decomposing Fusional Morphemes with Vector Embeddings (2024)

Abstract: Distributional approaches have proven effective in modeling semantics and phonology through vector embeddings. We explore whether distributional representations can also effectively model morphological information. We train static vector embeddings over morphological sequences. Then, we explore morpheme categories for fusional morphemes, which encode multiple linguistic dimensions, and often have close relationships to other morphemes. We study whether the learned vector embeddings align with these linguistic dimensions, finding strong evidence that this is the case. Our work uses two low-resource languages, Uspanteko and Tsez, demonstrating that distributional morphological representations are effective even with limited data.

## introduction distributional semantics, which models the meanings of words according to the contexts in which they appear @xcite , has proven highly successful for language modeling. generally, this has been achieved through word embeddings, which represent words with many-dimensional vectors @xcite @xcite , and capture many linguistic patterns and regularities @xcite . linguistic research has suggested that this distributional approach can be effective across all units of language @xcite . prior work @xcite has explored a distributional approach to phonology, finding that embeddings for phonological units can capture predictable linguistic features and natural classes. we explore whether this approach is also useful for morphology, hypothesizing that many grammatical morphemes can be described primarily by the contexts in which they appear. for example, a first person verbal affix might typically co-occur with first person pronouns, depending on the properties of the language being modeled. we focus on groups of highly related morphemes, in particular instances of fusional morphology. languages with fusional morphology include single morphemes that encode multiple grammatical features (as opposed to agglutinating morphology, where each morpheme corresponds to a single grammatical function). it is disputed whether languages exist with solely agglutinating or fusional morphological systems; rather, evidence suggests that many languages incorporate both processes @xcite . we compute morphological embeddings using standard vector embedding algorithms on morphological sequences from two low-resource languages, uspanteko and tsez (section 3). we compare these embeddings to handcrafted feature vectors based on the linguistic dimensions that make up the morphemes (see figure 1 ). we find that there is a consistent correlation between the vector embedding space and this linguistic feature space. we use data from @xcite , which we have formatted in huggingface datasets, available online. 2 we use the train splits from @xcite , with 9,774 uspanteko sentences and 7,116 tsez sentences. languages uspanteko (usp), or uspantek, is an endangered mayan language of guatamala with around 6,000 speakers @xcite . the language uses a system of absolutive and ergative affixes which generally attach to verbal stems @xcite . these affixes are fusional, encoding case (absolutive or ergative), number (singular or plural), and person (first, second, or third-person). tsez (ddo), or dido, is a language in the nakh-daghestanian family, with around 14,000 speakers in daghestan, russia. tsez utilizes a highly agglutinating and fusional morphological system, with morphemes often encoding two to five distinct linguistic dimensions. our data is originally from the tsez annotated corpus project @xcite . ## static morphological embeddings we first investigate whether distributional representations are applicable to morphological sequences-that is, do the contexts that morphemes occur in reflect any meaningful linguistic relationships, and can we capture those relationships with distributional methods? to do this, we train embeddings over sequences of morphological categories from the gloss lines of the igt from the corpora described in section 2. we might also have trained embeddings over the morphemes themselves, rather than their glosses/categories. however, our corpora are rather small, and the majority of morphemes occur very rarely, making it difficult to induce meaningful representations. by studying sequences of morpheme categories, we can gain insight into broader morphological patterns, despite limited data. ## conclusion we find evidence that distributional vector representations of morpheme categories capture linguistic regularities, even with limited data. broadly, morphological features such as number, case, and person seem to correlate with the contexts those morphemes appear in. we suggest that distributional morpheme representations are a viable model for morphology, particularly in languages with highlyproductive, fusional morphemes. this research is motivated primarily by linguistic understanding; that is, we are interested in determining whether morpheme contexts have predictable relationships. however, we suggest these findings could be applied in future research to more practical ends. for example, a linguist might use this approach to investigate a hypothesis about the relatedness of certain morphemes, providing for data-driven, large-scale evidence. alternately, an nlp practioner could use these findings in a task such as morpheme glossing @xcite to design models that utilize shared features to make predictions.