Unsupervised Stem-based Cross-lingual Part-of-Speech Tagging for Morphologically Rich Low-Resource Languages (2022)

Abstract: Unsupervised cross-lingual projection for part-of-speech (POS) tagging relies on the use of parallel data to project POS tags from a source language for which a POS tagger is available onto a target language across word-level alignments. The projected tags then form the basis for learning a POS model for the target language. However, languages with rich morphology often yield sparse word alignments because words corresponding to the same citation form do not align well. We hypothesize that for morphologically complex languages, it is more efficient to use the stem rather than the word as the core unit of abstraction. Our contributions are: 1) we propose an unsupervised stem-based cross-lingual approach for POS tagging for low-resource languages of rich morphology; 2) we further investigate morpheme-level alignment and projection; and 3) we examine whether the use of linguistic priors for morphological segmentation improves POS tagging. We conduct experiments using six source languages and eight morphologically complex target languages of diverse typologies. Our results show that the stem-based approach improves the POS models for all the target languages, with an average relative error reduction of 10.3% in accuracy per target language, and outperforms the word-based approach that operates on three-times more data for about two thirds of the language pairs we consider. Moreover, we show that morpheme-level alignment and projection and the use of linguistic priors for morphological segmentation further improve POS tagging.

## introduction low-resource languages lack annotated data even for basic syntactic information such as parts of speech (pos). to address this problem, two main unsupervised approaches have been adopted: zeroshot model transfer @xcite and crosslingual pos tagging via alignment and projection @xcite @xcite @xcite @xcite @xcite . @xcite show that the alignment and projection approach is less sensitive to the morphological dissimilarities between the source and target languages than zero-shot model transfer. in annotation projection, the word structure in the source and target languages impacts the quality of the alignment and projection phases, and hence affects the overall performance of the ultimate pos model. this becomes a concern for languages with rich word structure where affixation is common as they usually suffer from sparse alignment models that often fail to align words corresponding to the same citation form due to the extensive paradigms and translation inconsistencies. sparse alignment hinders the ability of a system to project the tags properly and results in null tags on the target side. these null tags then reduce the number of qualifying training examples and impact the pos model by introducing non-continuous labeled sequences. adding to these practical issues, the concept of word as a unit of structure has long been questioned in language sciences @xcite . we hypothesize that using the stem as the core unit of abstraction results in better pos models for low-resource languages of rich morphology. our contribution is three-fold. unsupervised stem-based cross-lingual approach for pos tagging for morphologically complex low-resource languages, where we use the stem as the core unit of abstraction. in order to adapt a fully-unsupervised approach, we use a stateof-the-art unsupervised morphological segmenter, morphagram @xcite , to derive the stems and morphemes. we follow the setup of @xcite using the bible as the only source of parallel data in order to emulate a low-resource scenario. we experiment with the same six source languages, namely english, spanish, french, german, russian and arabic, but choose six morphologically complex target languages, namely amharic, basque, finnish, indonesian, telugu and turkish and add two new target languages, namely georgian and kazakh, where we contribute a small pos-annotated dataset for the former. we show that the stem-based approach outperforms the word-based one in 43 language pairs out of 48, with an average relative error reduction of 10.3% in accuracy per target language, up to 21.0% in the case of kazakh. we also show that the stem-based approach outperforms the word-based approach which operates on three-times more data for about two thirds of the experimental pairs. morpheme-level alignment and projection, which allows for abstracting away from how the morphemes are combined in the source and target languages. we test the setup with arabic as the source language and show improvements for seven out of the eight target languages. using linguistic priors in morphological segmentation, which results in better segmentation models towards better alignment and projection. using georgian as a case study, we show that the use of linguistic priors, in the form of a set of affixes provided by an expert in the target language, improves the ultimate pos models. finally, we make our code publicly available to encourage further research 1 . ## approach we perform fully unsupervised cross-lingual pos tagging via alignment and projection. we follow the main architecture presented by @xcite (section 2.1). a primary difference is that we harness unsupervised morphological segmentation to use the stems as the core unit of abstraction for both alignment and projection (section 2.2). in addition, we experiment with morpheme-level alignment and projection (section2.3) and examine the use of linguistic priors towards better morphological segmentation and pos tagging (section 2.4). this allows for less sparse alignment models and denser projections, which in turn produces larger pos training data of a better quality. ## conclusion and future work we presented a fully unsupervised stem-based approach for cross-lingual pos tagging via alignment and projection, where we use the stem as the core unit of abstraction to abstract away from complex affixation. our experiments using six source languages and eight morphologically rich target lan-figure 3 : absolute performance increases (accuracy) when applying the stem-based approach using the new testament as the source of parallel data as compared to the word-based approach using the entire bible as the source of parallel data guages in low-resource setups show improvements over the word-based approach in 43 language pairs out of 48, with an average relative error reduction of 10.3% in accuracy per target language. in addition, we examined morpheme-based alignment and projection and the use of linguistic priors in morphological segmentation, which further improve pos tagging. in the future, we plan to study the role of morphological typology in cross-lingual learning. this allows for deriving disciplined guidelines for the selection of an appropriate source language that transfers well to the target language of interest.