Advancing NMT for Indigenous Languages: A Case Study on Y ucatec M ayan and C hol (2024)

Abstract: This study leverages Spanish-trained large language models (LLMs) to develop neural machine translation (NMT) systems for Mayan languages. For this, we first compile and process a low-resource dataset of 28,135 translation pairs of Chol and Yucatec Mayan extracted from documents in the CPLM Corpus (Martínez et al.). Then, we implement a prompt-based approach to train one-to-many and many-to-many models. By comparing several training strategies for two LLMs, we found that, on average, training multilingual models is better, as shown by the ChrF++ reaching 50 on the test set in the best case. This study reinforces the viability of using LLMs to improve accessibility and preservation for languages with limited digital resources. We share our code, datasets, and models to promote collaboration and progress in this field: https://github.com/RIKEN-DKO/iikim_translator.

## introduction in recent times, there has been a push towards creating nlp tools for the native languages of the americas @xcite . within this context, mayan languages have not received attention in machine translation (nmt) studies, despite their deep linguistic roots and large speaker populations. our study aims to bridge this gap by specifically developing and refining nmt systems for mayan languages. by leveraging advancements in large language models (llms) pre-trained in spanish, we aim to overcome the scarcity of a comprehensive parallel corpus for mayan languages. as a result, building nmt systems for languages could greatly benefit these language communities by enabling them to access services and information related to law, healthcare, and finance in their mother tongues. the mayan languages form a family spoken by the maya peoples, is primarily spoken across various regions in central america. this family stands as among the most thoroughly researched and documented in the americas @xcite . it is believed that the contemporary mayan languages originated from the proto-mayan language, which was likely spoken over 5,000 years ago. this ancient language eventually branched out into at least six distinct lineages: huastecan, quichean, yucatecan, qanjobalan, mamean, and ch'olan-tzeltalan 2 2.1 yucatec mayan yucatec mayan, commonly referred to as maya is a language spoken in the yucatán peninsula and the northern regions of belize. being one of the mayan languages yucatec mayan plays a vital role, in connecting us to the diverse cultural and historical legacy of the mayan civilization. unlike indigenous languages yucatec mayan boasts a substantial number of speakers with an estimated count of approximately 800,000 individuals foot_1 . ## models our selection criteria focused on recent models with extensive pretraining in spanish, as evidence suggests this significantly aids in translating to native languages @xcite . accordingly, we selected the t5s (t5 spanish) and m2m100 (480m version) models for our translation tasks. while both models adopt the encoder-decoder architecture foundational to transformer models, they are distinguished by their underlying philosophies and optimizations. this research aims to compare two approaches to language models (llms): t5s, which is versatile for various nlp tasks, and m2m100, which is specialized for translation purposes. this comparison intends to evaluate how well a general model like t5s can handle low resource translation scenarios and determine the performance of m2m100 in translating between less commonly spoken languages. through this method, we aim to identify which model design and training approach are most effective in creating mayan nmt systems. ## experiments and results this section outlines our training approach, experimental setup, and the results obtained from deploying various strategies on the t5s and m2m100 models. ## conclusion and future work this study has successfully demonstrated that the t5s and m2m100 models can be adapted for translation tasks between spanish and mayan languages, showcasing the potential of neural machine translation (nmt) in enhancing language preservation and accessibility. the m2m100 model, with its translation-focused architecture, excels in one-tomany translation scenarios. conversely, the t5s model shows versatility in managing bidirectional translations, benefiting from its flexible design. the incorporation of the "amenlp + uniform" strategy has emerged as a promising method to broaden the models' capabilities across multiple languages, though it introduces challenges that necessitate further exploration. initial experiments have validated the potential of nmt for yucatec mayan and chol, with both models performing effectively in low-resource settings. despite the variation in translation quality, the results affirm the capacity of these models to acquire meaningful 140 translations from scant data. this study marks the beginning of exploring nmt systems designed specifically for mayan languages, highlighting both the possibilities and challenges of using nmt for languages, with resources. moving forward, our future efforts will concentrate on expanding datasets and investigating active learning and few shot learning approaches. furthermore, we plan to customize the m2m100 model for other native languages and delve into the nuances of tag selection to enhance translation accuracy. by progressing in these areas, we aim not only to improve the effectiveness of nmt systems for languages but also to contribute to the broader field of language technology and digital transformation (dx). additionally, we plan to apply the techniques developed here to other low-resource scenarios, such as natural language to sparql translation.