Machine Translation for Highly Low-Resource Language: A Case Study of A inu, a Critically Endangered Indigenous Language in N orthern J apan (2023)

Abstract: This paper explores the potential of Machine Translation (MT) in preserving and revitalizing Ainu, an indigenous language of Japan classified as critically endangered by UNESCO. Through leveraging Marian MT, an open-source Neural Machine Translation framework, this study addresses the challenging linguistic features of Ainu and the limitations of available resources. The research implemented a meticulous methodology involving rigorous preprocessing of data, prudent training of the model, and robust evaluation using the SacreBLEU metric. The findings underscore the system’s efficacy, achieving a SacreBLEU score of 32.90 for Japanese to Ainu translation. This promising result highlights the capacity of MT systems to support language preservation and aligns with recent research emphasizing the potential of computational techniques for low-resource languages. The paper concludes by affirming the significant role of MT in the broader context of language preservation, serving as a crucial tool in the fight against language extinction. The study paves the way for future research to harness advanced MT techniques and develop more sophisticated models for endangered languages.

## introduction the ainu language, a polysynthetic and culturally rich language, has been traditionally spoken by the ainu people in the northern regions of japan, such as hokkaido, southern sakhalin, and the kuril islands. despite its intricate structure, the ainu language faces significant endangerment. in 2009, unesco classified ainu as a "critically endangered" language @xcite , underscoring the critical need for efforts towards its preservation. the language's vulnerability is further highlighted by the dwindling number of native ainu speak-ers and the loss of many ainu dialects, including sakhalin ainu and kuril ainu. the language itself has linguistic uniqueness such as polysynthesis and noun incorporation, which are characteristics of many indigenous languages in north america. example 1 exemplifies its polysynthesis and noun incorporation. the ultimate objective of this study is to develop an ai-assisted educational program and a teaching robot to facilitate the learning and preservation of the ainu language. the proposed program intends to incorporate several components like speech recognition, speech generation, part-ofspeech tagging, and universal dependencies tagging, among other linguistic technologies, based on recent studies on the ainu language. previous studies, such as the work of @xcite on the mingmatch-an n-gram model for ainu word segmentation, and the creation of an ainu folklore speech corpus by @xcite , the works mentioned in the next section, have laid the groundwork for this research. building on these pivotal studies, this research aims to develop a robust nlp model that leverages the marian mt as the primary translating model for ainu to japanese and japanese to ainu. the insights gained from this endeavor will inform the design of ai-assisted educational tools, thereby fostering the preservation and understanding of the ainu language and culture. ## previous literature the potential for leveraging advanced computational techniques such as nlp and mt for language revitalization is gradually being explored. previous work by @xcite showcased a fast n-gram model for word segmentation of the ainu language. this work signaled the potential of computational approaches for improving the accessibility and study of ainu. further efforts in this direction were made by @xcite , who developed a digital corpus and core language technologies for ainu. in another study, @xcite proposed better text-processing tools for the ainu language. these seminal works laid the groundwork for applying nlp techniques to ainu, facilitating its digitalization. nowakowski's later work @xcite adapted a multilingual speech representation model for under-resourced languages through multilingual fine-tuning and continued pretraining. this showcased how techniques in nlp could be adjusted for low-resource languages like ainu. efforts have also been made to apply speech recognition technology to the ainu language. matsuura et al. (2020b) developed a speech corpus of ainu folklore and end-to-end speech recognition for the ainu language. these authors also successfully utilized generative adversarial training data adaptation for very low-resource automatic speech recognition @xcite . these studies significantly contribute to the field and provide a solid foundation for further exploration of nlp applications in low-resource languages. in terms of the linguistic study of ainu, the work of @xcite in developing universal dependencies for ainu and @xcite in improving part-of-speech tagging of the ainu language have contributed significantly to the understanding of ainu syntax and morphology, which is essential in developing accurate nlp tools. the broader challenges of mt are aptly high-lighted in the works of @xcite , which underscored the need for advanced techniques to address these challenges effectively. in language education, an innovative application of nlp tools was demonstrated by @xcite through developing an ainu languagespeaking pepper robot, indicating the potential of such technologies in promoting and preserving endangered languages. the insights and methodologies proposed in these studies pave the way for further exploration into using mt and other nlp technologies for language preservation, particularly for endangered languages such as ainu. ## methodology in this study, we utilized marian mt. this efficient and adaptable open-source mt framework has demonstrated excellent performance in numerous research projects, particularly in scenarios involving low-resource languages @xcite . our choice for marian mt was also informed by its inherent capacity to handle different language structures, an essential feature for polysynthetic languages such as ainu @xcite . our methodology commenced with data preprocessing obtained from multiple digital ainu text sources. these included the ainugo archive from the national ainu museum foot_1 , the glossed audio corpus of ainu folklore from the national institute for japanese language and linguistics foot_2 , and the ilcaa ainu language resource from the tokyo university of foreign studies foot_3 . we converted all the katakana transcription into the roman alphabet. the collected corpus was subsequently cleansed to eliminate redundancies and inconsistencies. following this, we tokenized the data and segmented it into sentence pairs. given the polysynthetic structure of ainu (see example 1), we took extra caution during the tokenization process to correctly separate individual morphemes. the number of sentence pairs of the ainu original text and the japanese translation is around 100,000. we trained the marian mt model with our prepared corpus, translating ainu to japanese and japanese to ainu directions. the model parameters were optimized through a learning rate schedule combined with early stopping. the learning rate schedule gradually reduced the learning rate during the training process, thereby preventing the overfitting of the model to the training data. the early stopping technique mitigated overfitting by terminating the training when the model's performance on a validation set stopped improving (almansor and al-ani, 2018). we utilized the sacrebleu metric to evaluate the performance of our mt system @xcite . sacrebleu provides a reliable and uniform method for comparing different mt systems or versions, implementing identical tokenization and detokenization procedures across all systems evaluated. it also accounts for multiple reference translations, thereby offering a more comprehensive evaluation of the translation quality @xcite ). this feature is especially beneficial for languages like ainu, where the availability of parallel corpora is limited, and a given sentence could have multiple valid translations. our methodology thus encapsulated a combination of the marian mt framework, rigorous preprocessing of the ainu corpus, meticulous model training in ainu to japanese and japanese to ainu directions, and robust evaluation using the sacre-bleu metric. with this method, we developed a robust mt system capable of translating between ainu and japanese with significant accuracy. ## results this chapter elucidates the outcomes of our mt experiments and provides an extensive discussion of their implications. the core of our study centered around two translation tasks: from japanese to ainu, from ainu to japanese, and both directions foot_4 ( the model was trained on an extensive dataset gathered from various ainu digital text sources for the japanese-to-ainu translation task. this resulted in a sacrebleu score of 32.90, which implies a significant level of translation quality. this achievement showcases the model's ability to translate between these two disparate languages precisely. notably, these results were obtained despite the inherent challenges posed by developing an mt system for a low-resource language like ainu. the ainu-to-japanese translation task brought additional challenges, mainly due to the limited resources available for the ainu language. regardless, the mt system achieved a sacrebleu score of 10.45. we also trained marian mt for bidirectional translations, namely japanese-ainu and ainu-japanese, with a doubled corpus but reversed in the order of two languages in the latter half. the sacrebleu score of this bi-directional experiment was 29.91, and the input can be both japanese and ainu, but the output is in the other language, which was not typed in the input. our research's relatively high sacrebleu scores underline the feasibility of utilizing mt to aid language preservation and revitalization efforts. the results demonstrate that, even with limited resources, mt models can achieve a level of proficiency that renders them practical tools for ainu learners and researchers @xcite . additionally, our study supports the successes of previous attempts to apply computational techniques to the ainu language. a notable instance is the ainu speech recognition project by the kawahara lab at kyoto university, whose results were documented in @xcite . together, these studies underscore the potential contributions of nlp and mt technologies to preserve and revitalize endangered languages. the outcomes of our study should inspire further exploration of mt applications in low-resource language contexts. future endeavors could focus on refining the model's performance, expanding the dataset, and investigating how this technology can be integrated into interactive language learning platforms. such efforts would further contribute to the revitalization of the ainu language and culture. in the neural mt framework, marian mt, and powered by a comprehensive dataset sourced from various ainu digital text corpora, our study has made significant strides in demonstrating the feasibility and efficacy of mt in language preservation efforts. the outcomes of our study are promising. with a sacrebleu score of 32.90 for the japanese to ainu translation task, the quality of translation produced is commendable, particularly considering the challenging polysynthetic nature of the ainu language @xcite . even more impressively, the model achieved a respectable sacrebleu score of 29.91 for the japanese-ainu and ainu-japanese bi-directional translation task, underlining the robustness of the neural mt framework when dealing with complex, low-resource languages @xcite . these findings contribute to the expanding body of research that explores the potential of mt in bridging linguistic gaps and aiding in the preservation of endangered languages. our results align with studies such as those conducted by @xcite , which emphasized the potential of neural mt for low-resource languages, and @xcite , which explored mt in low-resource language varieties. our research underscores the necessity for further work to leverage advanced mt techniques for low-resource languages, particularly where traditional linguistic databases may be limited or non-existent. by contributing to the intersection of natural language processing (nlp), mt, and language preservation, our study offers a replicable methodology and highlights the importance of continuous innovation in these areas @xcite . revitalizing endangered languages is complex and multifaceted, necessitating collaborative efforts across linguists, educators, technologists, and communities. our research reiterates that mt and other language technologies are crucial in this process. while further refinement of models and expansion of datasets can enhance translation quality, our current findings underscore the significance of mt in the broader context of language preservation. in conclusion, our study suggests that mt can make even the most resource-limited languages, like ainu, more accessible. by facilitating communication, preserving cultural heritage, and fostering a deeper understanding of diverse human experiences, our research reaffirms the profound value of language preservation and the transformative power of technology in these endeavors. in light of unesco's classification of ainu as "critically endangered," we believe our research can add a crucial layer of defense in the fight against language extinction and contribute to celebrating our shared linguistic heritage @xcite .