BPE vs. Morphological Segmentation: A Case Study on Machine Translation of Four Polysynthetic Languages (2022)

Abstract: Morphologically-rich polysynthetic languages present a challenge for NLP systems due to data sparsity, and a common strategy to handle this issue is to apply subword segmentation. We investigate a wide variety of supervised and unsupervised morphological segmentation methods for four polysynthetic languages: Nahuatl, Raramuri, Shipibo-Konibo, and Wixarika. Then, we compare the morphologically inspired segmentation methods against Byte-Pair Encodings (BPEs) as inputs for machine translation (MT) when translating to and from Spanish. We show that for all language pairs except for Nahuatl, an unsupervised morphological segmentation algorithm outperforms BPEs consistently and that, although supervised methods achieve better segmentation scores, they under-perform in MT challenges. Finally, we contribute two new morphological segmentation datasets for Raramuri and Shipibo-Konibo, and a parallel corpus for Raramuri–Spanish.

## introduction polysynthetic languages are known because of their rich morphology, that encodes most parts of the semantics into verbs, leading to a high morphemeper-word rate. the resulting combinations of morphemes and roots result in extreme type sparsity. thus, polysynthetic languages represent a challenging environment for nlp methods @xcite . subword segmentation has been a common method to reduce sparsity @xcite . moreover, as these languages are mostly extremely lowresource (elr), the challenge is even harder. some of the reasons behind this is that most of them are endangered and spoken by minority groups @xcite . but what impact does morphological segmentation have on downstream tasks like machine translation (mt), when translating from or into fusional languages? linguistically inspired segmentation was considered to be the best option to handle rich morphology @xcite until the appearance of byte-pair encodings (bpes; @xcite and has been adopted as the default segmentation technique. bpes earned this status for its good results, unsupervised training and language independence. @xcite show that there is no significant gain when using an unsupervised morphological segmentation for the input over bpes when evaluating those methods in moderate lr scenarios for nepali-english and kazakh-english, contradicting initial findings of @xcite . however, how would bpes perform for polysynthetic languages in elr scenarios? @xcite compare bpe, with morfessor @xcite and rule-based morphological analyzers for medium resourced inuktitut-english, and for the elr yupik-english and guarani-spanish. their results show that bpes outperform morfessor and the morphological analyzer in all mt cases (but with better language modeling capabilities of morphological models over bpes). however, most of these studies only rely on the usage of a limited set of segmentation methods and do not consider the quality of the used morphological segmentation methods. this study aims to answer the following research questions: i) is morphological segmentation beneficial for mt where one language is polysynthetic and elr?; and ii) is higher morphological segmentation quality correlated with higher mt scores? to answer these questions, we perform segmentation experiments on four polysynthetic languages: foot_0 nahuatl (nah), raramuri (tar), shipibo-konibo (shp) and wixarika (hch) and apply those segmentations to mt paired with spanish (spa). first, we revisit a wide set of supervised and unsupervised methods and apply them to the input of mt transformer models. this study is the first to show that strong unsupervised morphological approaches outperform bpes consistently on elr polysynthetic languages, except for nah. these results are related to @xcite , that found that a morphologically guided bpe can improve the mt performance for guarani-spanish. on the other hand, even when supervised morphological segmentation methods achieve better results for the segmentation task, when it comes to mt systems they under-perform all other approaches. we hypothesize that this might be due to overfitting the clean and out-of-domain morphological training set. to make all these experiments possible we introduce additionally two new morphologically annotated datasets for tar and shp; and one parallel dataset for spa-tar foot_1 . ## conclusion in this paper, we compared a wide set of morphological segmentation models with bpes when applied to the input of neural machine translation systems for extreme low-resource polysynthetic languages. we found that unsupervised morphological segmentation outperformed bpes significantly on 5 out of 8 language pairs, setting a consistent overall performance. surprisingly sota supervised morphological segmentation achieved the lowest performance of all systems. in future, we will explore adaptor-grammars @xcite @xcite for segmentation, and also the way to make unsupervised segmentation more robust and suitable for mt including the reduction of produced unk symbols.