C hr E n: C herokee- E nglish Machine Translation for Endangered Language Revitalization (2020)

Abstract: Cherokee is a highly endangered Native American language spoken by the Cherokee people. The Cherokee culture is deeply embedded in its language. However, there are approximately only 2,000 fluent first language Cherokee speakers remaining in the world and the number is declining every year. To help save this endangered language, we introduce ChrEn, a Cherokee-English parallel dataset, to facilitate machine translation research between Cherokee and English. Compared to some popular machine translation language pairs, ChrEn is extremely low-resource, only containing 14k sentence pairs in total. We split our parallel data in ways that facilitate both in-domain and out-of-domain evaluation. We also collect 5k Cherokee monolingual data to enable semi-supervised learning. Besides these datasets, we propose several Cherokee-English and English-Cherokee machine translation systems. We compare SMT (phrase-based) versus NMT (RNN-based and Transformer-based) systems; supervised versus semi-supervised (via language model, back-translation, and BERT/Multilingual-BERT) methods; as well as transfer learning versus multilingual joint training with 4 other languages. Our best results are 15.8/12.7 BLEU for in-domain and 6.5/5.0 BLEU for out-of-domain Chr-En/EnChr translations, respectively; and we hope that our dataset and systems will encourage future work by the community for Cherokee language revitalization.

## introduction the cherokee people are one of the indigenous peoples of the united states. before the 1600s, they lived in what is now the southeastern united states @xcite . today, there are three federally recognized nations of cherokee src. ꭵꮭ ꭱꮆꭿ ꭰꮑꭿ ᏹꭹ, ꮎꮝꭹꮿ ꭰᏼ ꭱꮆꭿ ꭸꭲ ꮒꭸꮢꮎ ꮵꭹ. ref. they are not of the world, even as i am not of the world. smt it was not the things upon the earth, even as i am not of the world. nmt i am not the world, even as i am not of the world. to revitalize cherokee, language immersion programs are provided in elementary schools, and second language programs are offered in universities. however, students have difficulty finding exposure to this language beyond school hours @xcite . this motivates us to build up english (en) to cherokee (chr) machine translation systems so that we could automatically translate or aid human translators to translate english materials to cherokee. chr-to-en is also highly meaningful in helping spread cherokee history and culture. therefore, in this paper, we contribute our effort to cherokee revitalization by constructing a clean cherokee-english parallel dataset, chren, which results in 14,151 pairs of sentences with around 313k english tokens and 206k cherokee tokens. we also collect 5,210 cherokee monolingual sentences with 93k cherokee tokens. both datasets are derived from bilingual or monolingual materials that are translated or written by first-language cherokee speakers, then we manually aligned and cleaned the raw data. 2 our datasets contain texts of two cherokee dialects (oklahoma and north carolina), and diverse text types (e.g., sacred text, news). to facilitate the development of machine translation systems, we split our parallel data into five subsets: train/dev/test/out-dev/out-test, in which dev/test and out-dev/out-test are for indomain and out-of-domain evaluation respectively. see an example from chren in the translation between cherokee and english is not easy because the two languages are genealogically disparate. as shown in figure 1 , cherokee is the sole member of the southern branch of the iroquoian language family and is unintelligible to other iroquoian languages, while english is from the west germanic branch of the indo-european language family. cherokee uses a unique 85character syllabary invented by sequoyah in the early 1820s, which is highly different from english's alphabetic writing system. cherokee is a polysynthetic language, meaning that words are composed of many morphemes that each have independent meanings. a single cherokee word can express the meaning of several english words, e.g., ꮻꮣꮵꮑꭹꮟ (widatsinegisi), or i am going off at a distance to get a liquid object. since the semantics are often conveyed by the rich morphology, the word orders of cherokee sentences are vari-able. there is no "basic word order" in cherokee, and most word orders are possible @xcite , while english generally follows the subject-verb-object (svo) word order. plus, verbs comprise 75% of cherokee, which is only 25% for english @xcite . hence, to develop translation systems for this low-resource and distant language pair, we investigate various machine translation paradigms and propose phrase-based @xcite statistical machine translation (smt) and rnnbased @xcite or transformer-based @xcite neural machine translation (nmt) systems for both chr-en and en-chr translations, as important starting points for future works. we apply three semi-supervised methods: using additional monolingual data to train the language model for smt @xcite ; incorporating bert (or multilingual-bert) @xcite representations for nmt @xcite , where we introduce four different ways to use bert; and the back-translation method for both smt and nmt @xcite @xcite . moreover, we explore the use of existing x-en parallel datasets of 4 other languages (@xmath0. empirically, nmt is better than smt for in-domain evaluation, while smt is significantly better under the out-of-domain condition. rnn-nmt consistently performs better than transformer-nmt. semi-supervised learning improves supervised baselines in some cases (e.g., back-translation improves out-of-domain chr-en nmt by 0.9 bleu). even though cherokee is not related to any of the 4 languages (czech/german/russian/chinese) in terms of their language family trees, surprisingly, we find that both transfer learning and multilingual joint training can improve chr-en/en-chr performance in most cases. especially, transferring from chinese-english achieves the best in-domain chr-en performance, and joint learning with english-german obtains the best in-domain en-chr performance. the best results are 15.8/12.7 bleu for in-domain chr-en/en-chr translations; and 6.5/5.0 bleu for outof-domain chr-en/en-chr translations. finally, we conduct a 50-example human (expert) evalua-tion; however, the human judgment does not correlate with bleu for the en-chr translation, indicating that bleu is possibly not very suitable for cherokee evaluation. overall, we hope that our datasets and strong initial baselines will encourage future works to contribute to the revitalization of this endangered language. ## data description it is not easy to collect substantial data for endangered cherokee. we obtain our data from bilingual or monolingual books and newspaper articles that are translated or written by first-language cherokee speakers. in the following, we will introduce the data sources and the cleaning procedure and give detailed descriptions of our data statistics. ## models in this section, we will introduce our cherokee-english and english-cherokee translation systems. adopting best practices from low-resource machine translation works, we propose both statistical machine translation (smt) and neural machine translation (nmt) systems, and for nmt, we test both rnn-based and transformer-based models. we apply three semi-supervised methods: training language model with additional monolingual data for smt @xcite , incorporating bert or multilingual-bert representations into nmt @xcite , and back-translation for both smt and nmt @xcite . further, we explore transfer learning @xcite from and multilingual joint training @xcite with 4 other languages (czech/german/russian/chinese) for nmt. ## conclusion and future work in this paper, we make our effort to revitalize the cherokee language by introducing a clean cherokee-english parallel dataset, chren, with 14k sentence pairs; and 5k cherokee monolingual sentences. it not only can be another resource for low-resource machine translation research but also will help to attract attention from the nlp community to save this dying language. besides, we propose our chr-en and en-chr baselines, including both smt and nmt models, using both supervised and semi-supervised methods, and exploring both transfer learning and multilingual joint training methods with 4 other languages. experiments show that smt is significantly better and nmt under out-of-domain condition while nmt is better for in-domain evaluation; and the semi-supervised learning, transfer learning, and multilingual joint training can improve simply supervised baselines. overall, our best models achieve 15.8/12.7 bleu for in-domain chr-en/en-chr translations and 6.5/5.0 bleu for outof-domain chr-en/en-chr translations. we hope these diverse baselines will serve as useful strong starting points for future work by the community. our future work involves converting the monolingual data to parallel and collecting more data from the news domain. cherokee: the overhill dialect, most widely spoken in oklahoma (ok), and the middle dialect, most widely used in north carolina (nc).