Lightweight morpheme labeling in context: Using structured linguistic representations to support linguistic analysis for the language documentation context (2023)

Abstract: Linguistic analysis is a core task in the process of documenting, analyzing, and describing endangered and less-studied languages. In addition to providing insight into the properties of the language being studied, having tools to automatically label words in a language for grammatical category and morphological features can support a range of applications useful for language pedagogy and revitalization. At the same time, most modern NLP methods for these tasks require both large amounts of data in the language and compute costs well beyond the capacity of most research groups and language communities. In this paper, we present a gloss-to-gloss (g2g) model for linguistic analysis (specifically, morphological analysis and part-of-speech tagging) that is lightweight in terms of both data requirements and computational expense. The model is designed for the interlinear glossed text (IGT) format, in which we expect the source text of a sentence in a low-resource language, a translation of that sentence into a language of wider communication, and a detailed glossing of the morphological properties of each word in the sentence. We first produce silver standard parallel glossed data by automatically labeling the high-resource translation. The model then learns to transform source language morphological labels into output labels for the target language, mediated by a structured linguistic representation layer. We test the model on both low-resource and high-resource languages, and find that our simple CNN-based model achieves comparable performance to a state-of-the-art transformer-based model, at a fraction of the computational cost.

## introduction linguistic analysis is a core task in the documentation, analysis, and description of endangered and less-studied languages. one frequent goal of language documentation projects is to produce a corpus of interlinear glossed texts, or igt (figure 1 shows an example from the mayan language uspanteko). igt can take many different forms, but canonically consists of the target language sentence, morphological segmentation of each word, glossing of each word with its stem translation and any relevant morphosyntactic features, and a translation into a language of wider communication. in addition to providing insight into the properties of the language being studied, the linguistic information in igt can support a range of applications useful for language teaching and revitalization. modern nlp methods typically require both large amounts of annotated data in the target language and compute resources beyond the capacity of most research groups and language communities. in this paper we address the task of lightweight morpheme labeling in context @xcite , developing a model which achieves reasonable accuracy with minimal requirements for both labeled data and computational expense. following previous work @xcite @xcite @xcite , among others), we aim to predict the parts of speech (pos) and morphosyntactic features for each word in the target sentence, producing the third line of figure 1 , with stem translations replaced by pos labels. this model can produce a first-pass labeling for correction by human experts working on the documentation project, saving large amounts of time (as shown by @xcite and freeing experts to work on more complex aspects of linguistic analysis. to match the language documentation context, where we often have a transcription and translation of the text before any other labeled data, we model morpheme labeling as a translation task. specifically, the model should learn to transform labels for the high-resource translation into labels for the target language; hence the name gloss-to-gloss (g2g). for initial model development, we use data labeled in the unimorph foot_0 format, so that we can test the model's performance on a range of languages. next, we test the same model on uspanteko data from a language documentation project @xcite , which involves the steps of: for step (a), the expected unimorph representation for the uspanteko sentence above might be: step (c) creates pseudo-parallel english data for the texts. for figure 1 , this step yields the following (noisy) morphological analysis: in step (d), we concatenate a vector of the english morpheme labels with static word embeddings for the english lexical items; this combined representation serves as input to the final classification layers, whose task is to produce the appropriate labels for the target language. to keep computational demands low, we use a rather simple cnn-based architecture, and compare to a fine-tuned bert @xcite model. on standard evaluations, the cnn model achieves performance comparable to the bert model, at a fraction of the computational expense. the contributions of this work are: we described related work in section 2, our approach to data representation in section 3, and the model architecture in section 4. section 5 describes results for the high-resource language development experiments, and section 6 presents our core results on igt for uspanteko. we wrap up with discussion and conclusions. one goal of this work is to develop time-saving tools for use in the language documentation context. specifically, we aim to support the production of interlinear glossed text (igt) with a lightweight model that can be run on a standard laptop, using whatever previously-produced igt might be available for the target language. ## wurde sie mitglied des lägerdorfer adgb -aktionsausschusses. ## data and its representation for model development, we start with data from the 2019 sigmorphon shared task on morphological analysis in context. 3 once the model has been developed and tested, we apply it to a true low-resource language (section 6.) the shared task data is a collection of datasets of varying sizes, from 68 different languages and/or varieties, with sentence level morphological analysis in the unimorph @xcite style. here we report results for 9 languages, selected for diversity of morphological systems. for each language, we select the first 10,000 sentences from the corpus and use a train/dev/test split of 60/20/20. ## experiments, results, and discussion to evaluate performance of the model, we test it on nine different language pairs (see as a baseline, we fine-tune a pre-trained bert model for the morpheme labeling task, using the same data and splits, but in a standard supervised learning set-up (i.e. no parallel data and no lds). the cnn model experiments were run on a 2.6 ghz intel(r) core(tm) cpu, taking an average of 8.5 minutes to train. the bert baseline experiments were run on a multi-gpu cluster, taking an average of 3.5 hours to train. evaluation. results. some patterns hold across most language pairs. for the most part, the cnn does not quite match the performance of the transformer. crucially, though, the cnn trains on a single laptop in under 10 minutes, where the transformer needs a compute cluster and multiple hours to train. the cnn performance is generally within 5 percentage points of the bert model, and this may be an acceptable performance in most documentation contexts -an empirical question for future work. the pos score represents the proportion of the part of speech predictions that were correct. because there are latent associations between pos category and morpheme labels (for example, it would be highly unusual to see aspectual features marked on nouns), the pos score should be directly proportional to the final f1 scores that we obtain for each language. this is reflected across our results. while both models struggle with turkish and russian, the cnn also performs poorly on basque. at the ld level, the model's performance is somewhat similar across the three romance languages we considered (spanish, french and italian). while the cnn fails to perform better than the transformer in most scenarios, it is interesting to note that the cnn performs marginally better than the transformer on french. however, both models show a significant performance dip when it comes to spanish. the cnn's f1 dips to 0.66 for turkish and 0.73 for russian. this may be due to their high morphological complexity. ## applying the model to language documentation data finally, we apply our model to data from the mayan language uspanteko @xcite , using the train/dev/test splits defined for the sigmor-phon 2023 shared task: training on 21 texts (9774 sentences), and using one text each for dev and test (around 200 sentences each). the model's performance on language documentation data parallels the results for high-resource languages. ## conclusion we have presented the g2g model, a new architecture for morphological analysis that dramatically reduces compute time by modeling the task as, essentially, an image-to-image translation task. the model incorporates knowledge of linguistic categories by mapping labels to their linguistic dimensions, with the effect of narrowing the space of possible outputs. these strategies result in an enormous reduction of compute time and a system more suitable for use in low-resource scenarios than the large language models currently achieving top performance for this and similar tasks. model variants and future work. working with language documentation data adds several layers of complexity. in this work, our model's output lacks the ordered association with individual morphemes typical of most igt. we use an unordered set of labels to describe the morphological features of a word, as shown in in addition, there is wide variability in both the label sets and the glossing scheme across language documentation projects. one widely-used scheme is encoded in the leipzig glossing rules @xcite ; the sigmorphon 2023 shared task on interlinear glossing foot_8 hews closer to this goal. in this shared unser-n väter-n our-dat.pl father dat.pl "to our fathers." limitations and ethical considerations. first, the system's performance is constrained by the use of automated systems to produce pseudo-parallel data. errors in translation and morpheme labeling on the high-resource side propagate to the output and cause mistakes in target side labeling. we have not yet performed the extensive error analyses needed to understand how much error propagation might be affecting the system. second, we have not yet tested the system in an actual documentation project. when working on nlp with endangered and/or indigenous languages in mind, there is a clear risk of perpetuating existing oppression @xcite . we hope to avoid some of these harms by using data from a wide range of non-threatened languages first, waiting to involve language community members and documentary linguists until we have a system with good enough results that we expect it could actually be helpful in real world contexts. we have already developed collaborations with several speakers of endangered languages and linguists working on documentation projects, and we look forward to continuing this work with their guidance and involvement. guillaume wisniewski, séverine guillaume, and alexis michaud. 2020. phonemic transcription of lowresource languages: to what extent can preprocessing be automated? in proceedings of the 1st joint workshop on spoken language technologies for under-resourced languages (sltu) and collaboration and computing for under-resourced languages (ccurl), pages 306-315, marseille, france. european language resources association. xingyuan zhao, satoru ozaki, antonios anastasopoulos, graham neubig, and lori levin. 2020. automatic interlinear glossing for under-resourced languages leveraging translations. in proceedings of the 28th international conference on computational linguistics, pages 5397-5408, barcelona, spain (online). international committee on computational linguistics.