Towards Multilingual Interlinear Morphological Glossing (2023)

Abstract: Interlinear Morphological Glosses are annotations produced in the context of language documentation. Their goal is to identify morphs occurring in an L1 sentence and to explicit their function and meaning, with the further support of an associated translation in L2. We study here the task of automatic glossing, aiming to provide linguists with adequate tools to facilitate this process. Our formalisation of glossing uses a latent variable Conditional Random Field (CRF), which labels the L1 morphs while simultaneously aligning them to L2 words. In experiments with several under-resourced languages, we show that this approach is both effective and data-efficient and mitigates the problem of annotating unknown morphs. We also discuss various design choices regarding the alignment process and the selection of features. We finally demonstrate that it can benefit from multilingual (pre-)training, achieving results which outperform very strong baselines.

## introduction interlinear morphological gloss (img) @xcite is an annotation layer aimed to explicit the meaning and function of each morpheme in some documentation ('object') language l1, using a (meta)-language l2. in computational language documentation scenarios, l1 is typically a low-resource language under study, and l2 is a well-resourced language such as english. figure 1 displays an example img: the source sentence t in l1 is overtly segmented into a sequence of morphemes (x), each of which is in one-to-one correspondence the corresponding gloss sequence y. each unit in the gloss tier is either a grammatical description (obl for the oblique marker in figure 1 ) or a semantic tag (son in figure 1 ), expressed by a lexeme in l2. an idiomatic free translation z in l2 is usually also provided. y and z help linguists unfamiliar with l1 to understand the morphological analysis in x. figure 1: a sample entry in tsez: l1 sentence (t), and its morpheme-segmented version (x), its gloss (y), and a l2 translation (z). grammatical glosses are in small capital, lexical glosses in straight orthography. in this paper, we study the task of automatically computing the gloss tier, assuming that the morphological analysis x and the free l2 translation z are available. as each morpheme has exactly one associated gloss, 1 an obvious formalisation of the task that we mostly adopt views glossing as a sequence labelling task performed at the morpheme level. yet, while grammatical glosses effectively constitute a finite set of labels, the diversity of lexical glosses is unbounded, meaning that our tagging model must accommodate an open vocabulary of labels. this issue proves to be the main challenge of this task, especially in small training data regimes. to handle such cases, we assume that lexical glosses can be directly inferred from the translation tier, an assumption we share with @xcite . in our model, we thus consider that the set of possible morpheme labels in any given sentence is the union of (i) all grammatical glosses, (ii) lemmas occurring in the target translation, and (iii) frequently-associated labels from the training data. this makes our model a hybrid between sequence tagging (because of (i) and (iii)) and unsupervised sequence alignment (because of (ii)), as illustrated in figure 2 . our implementation relies on a variant of conditional random fields (crfs) @xcite , which handles latent variables and offers the ability to locally restrict the set of possible labels. the choice of a crf-based approach is motivated by its notable data-efficiency, while methods based on neural networks have difficulties handling very low resource settings-this is again confirmed by results in §5.1. in this work, we generalise previous attempts to tackle this task with sequence tagging systems based on crfs such as @xcite @xcite and makes the following contributions: (a) we introduce ( §2) a principled and effective endto-end solution to the open vocabulary problem; (b) we design, implement and evaluate several variants of this solution ( §3), which obtain results that match that of the best-performing systems in the 2023 shared task on automatic glossing ( §5.1); (c) in experiments with several low-resource languages ( §4), we evaluate the benefits of an additional multilingual pre-training step, leveraging features that are useful cross-linguistically ( §5.5). owing to the transparency of crf features, we also provide an analysis of the most useful features ( §5.6) and discuss prospects for improving these techniques. y g {lat gen1 ... iii pst.unw} x nesi s ł q ono uži zow n z he had three sons y l (x) (or y) ... he.obl son be.nprs ... 2 a hybrid tagging / alignment model ## related work language documentation with the everpressing need to collect and annotate linguistic resources for endangered languages, the field of computational language documentation is quickly developing, as acknowledged by the computel workshop. 17 regarding annotation tools, recent research has focused on all the steps of language documentation, from speech segmentation and transcription to automatic word and morpheme splitting to automatic interlinear glossing. automatic glossing automatic glossing was first studied in @xcite , where active learning was used to incrementally update an underlying tagging system focusing mainly on grammatical morphemes (lexical items are tagged with their pos). @xcite added to this an extra layer aimed to annotate the missing lexical tags, yielding a system that resembles our crf+maj baseline. both @xcite and (mcmillan-major, 2020) rely on crfs, the latter study being closer to our approach as it tries to combine post-hoc the output of two crf models operating respectively on the l1 and l2 tier, where our system introduces an integrated end-to-end architecture. zhao et al. ( 2020 ) develop an architecture inspired by multi-source neural translation models, where one source is the l1 sequence, and the other the l2 translation. they experiment with arapaho, lezgi, and tsez, while also applying some sort of cross-lingual transfer learning. the recent sigmorphon exercise @xcite is a first attempt to standardise benchmarks and task settings and shows that morpheme-level accuracy in the high 80s can be obtained for most languages considered. latent variable crfs models extended crf models were proposed and used in many studies, including latent variables to represent, e.g. hidden segmentation as in @xcite or hidden syntactic labels @xcite . closer to our work, @xcite use latent structures to train discriminative statistical machine translation systems. other relevant work on unsupervised discriminative alignment are in @xcite , while @xcite use a supervised symmetric bi-dimensional word alignment model. ## conclusion this paper presented a hybrid crf model for the automatic interlinear glossing task, which was specifically designed and tailored to work well in low data conditions and to effectively address issues due to out-of-vocabulary morphemes. we presented our main approach, which relies on analysing the translation tier, and discussed our main implementation choices. in experiments with five low-resource languages, we obtained accuracy scores that match or even outperform those of very strong baselines, confirming that accuracy values in the 80s or above could be obtained with a few hundred training examples. using a large multilingual gloss database, we finally studied the possibility of performing cross-lingual transfer for this task. there are various ways to continue this work and improve these results, such as removing the noise introduced via erroneous alignments linkseither by marginalising over the 'origin' variable, by filtering unlikely alignments based on link poste-rior values, or by also trying to generate alignment links for function words @xcite . our initial experiments along these lines suggest that this may not be the most promising direction. we may also introduce powerful neural representations for l1 languages; while these were usually available for a restricted number of languages, recent works have shown that even lowresource languages could benefit from these techniques @xcite .