From Nile Sands to Digital Hands: Machine Translation of C optic Texts (2024)

Abstract: The Coptic language, rooted in the historical landscapes of Egypt, continues to serve as a vital liturgical medium for the Coptic Orthodox and Catholic Churches across Egypt, North Sudan, Libya, and the United States, with approximately ten million speakers worldwide. However, the scarcity of digital resources in Coptic has resulted in its exclusion from digital systems, thereby limiting its accessibility and preservation in modern technological contexts. Our research addresses this issue by developing the most extensive parallel Coptic-centered corpus to date. This corpus comprises over 8,000 parallel sentences between Arabic and Coptic, and more than 24,000 parallel sentences between English and Coptic. We have also developed the first neural machine translation system between Coptic, English, and Arabic. Lastly, we evaluate the capability of leading proprietary Large Language Models (LLMs) to translate to and from Coptic using a few-shot learning approach (in-context learning). Our code and data are available at https://github.com/UBC-NLP/copticmt .

## introduction with written records dating back to 3,400 b.c., egyptians trace their roots to one of the world's oldest civilizations. while modern egyptians primarily speak the egyptian dialect of arabic, millions in northeastern and eastern africa continue to use coptic as their everyday language. this unique language combines elements of the ancient egyptian languages demotic, hieroglyphic, and hieratic. it was the language of the ptolemaic rulers following the spread of greek culture throughout much * these authors contributed equally. large language models (llms) are known for their exceptional performance with languages extensively represented in their pre-training data @xcite . however, their performance significantly declines for languages with lower representation. several methods have been proposed to address this issue. these methods include continual training incorporating new data representing the low-resource languages @xcite @xcite , instruction fine-tuning @xcite , and vocabulary extension to cover new languages @xcite . the primary challenge with these methods is their reliance on substantial amounts of new data. in the case of coptic, as far as we know, no specialized dataset exists for pre-training or fine-tuning models. furthermore, an examination of the vocabulary of several llms @xcite @xcite reveals that coptic characters are entirely absent. we hypothesize that the absence of coptic language symbols will prevent these models from generating or understanding the coptic language. to address these challenges for coptic, this study begins by compiling the available coptic text from the internet into a unified parallel dataset. we then develop a compact translation model between arabic/coptic and english/coptic language pairs. to develop the model, we initially establish a baseline using a simple seq2seq architecture @xcite . we train this model with english-coptic and arabic-coptic pairs and then experiment with a many-to-many approach where the same model is trained to translate between all three languages. in the second phase, we perform full fine-tuning on more specialized models such as mt5 and m2m100 @xcite , using both pairwise and many-tomany translation methods. moreover, as we have a limited dataset and unseen scripts, we hypothesize that the t5-small version will generalize better to the dataset compared to the t5-base since the latter is larger and can overfit the training data. furthermore, we assess the capabilities of the current top proprietary and open-source llms on coptic-centered translation. we experiment with closed models such as gpt-4-turbo-2024-04-09 (openai, 2024), claude 3 opus @xcite , and gemini 1.5 pro-api-0409-preview @xcite . for open-source models, we investigate llama-3-70b-instruct @xcite . we focus on translating between the various pairs without any training, utilizing an in-context (zero-shot and few-shot) learning approach. the rest of this paper is organized as follows: section 2 is about related work. section 3 offers a background about the coptic language. in section 4, we introduce our datasets. we present our methods in section 5 and our results in section 6. we conclude and provide an outlook about future work in section in section 7. consequently, we have endeavored to compile available parallel corpora and then assess the best methods to fine-tune llms for translating between these language pairs. previous research, such as dipmt @xcite , has explored translation in low-resource languages through prompting, providing translations for uncommon words using existing dictionaries. this approach suits languages where llms exhibit basic syntactic understanding, potentially learned through n-gram language modeling. however, traditional prompting methods are often inadequate when faced with a completely new language, from which llms must learn vocabulary and grammar from scratch. this led to the development of dipmt++ and has inspired further research into machine translation for low-resource languages. and the coptic not only most of the llms, hasn't been seen during the training moreover the coptic characters are not present in the dictionary of most of the llms such as llama2, llama3 @xcite , mt5 @xcite with this in mind we have decided to handle the tasks of the multilingual machine translation between the arabic, english and coptic in the 5 setups the first one we are going to build the baseline using seq2seq @xcite we are going to train in two fashions the first one in which we have pairs b (english ↔ coptic) and (arabic ↔ coptic) alongside use of the many to many langauge in which the same model is trained to translate between three languages (arabic ↔ coptic ↔) the goal is to test which approaches can best results as the available parallel corpus is limited and we aim to use whatever resource we have to the fullest, the second setup is which in we are going to perform full finetuning to @xcite ) models and we are going to before also pairs and many translation, the third setups is using zeroshot learning in which we are going to use open source models such as llama3 and other closed source models such as gpt3.5, gpt4 and gemini models, and as the closed source models will require us to access them via api for those closed model we are going to use portion of the test set that is sampled randomly . the fourth setup is fewshot learning, the fifth is the in-context learning for the gemini model, and finally, we are going to discuss how we can tune @xcite model to handle the case of the coptic arabic and english translation to handle the machine translation between the different languages. in this paper, we are taking our first steps to create such a machine translation system. it is like we are building a bridge between languages. to help our system learn, we are introducing it to 24,000 parallel coptic and english sentences and 16,800 arabic to coptic parallel data. we aim to build a multi-language machine translation system capable of translating between arabic and coptic, coptic and arabic, english and coptic, and coptic and english. we have collected a corpus of parallel sentences and proposed transformers @xcite pipeline for coptic to english translation. additionally, through transfer learning, we found that the mt5 @xcite model can generalize and adapt to languages with unseen scripts, such as coptic, even without prior exposure to coptic data during training. our data collection efforts and adaptation of large language models like mt5 @xcite for unseen scripts also hold promise for other languages. ## related work this section reviews research on fine-tuning large language models (llms) for machine translation in low-resource languages. most studies focus on languages encountered during llm pre-training, excluding coptic. we examine specific llms and pre-trained language models (plms) used in our study to determine their exposure to coptic. we also review methods for fine-tuning llms for lowresource languages, establishing a baseline, and identifying challenges. our goal is to adapt and enhance these techniques for coptic, which is significantly underrepresented in machine translation research. ## data collection during the early modern era, the coptic language, once a medium of daily communication, became obsolete and was largely supplanted by egyptian arabic. nevertheless, its significance persisted, primarily as the liturgical medium for coptic christians. consequently, the predominant body of extant coptic literature is constituted by ecclesiastical writings penned by illustrious figures of the coptic church and some seminal translations of the bible into coptic dialects. notably, the coptic rendition of the bible spans both the old and new testaments and is available in multiple cop-tic dialects, such as sahidic or bohairic. 4.1 english corpus selection to compile our dataset, we sourced translated iterations of biblical texts and hagiographical narratives of coptic saints from the coptic scriptorium. the scriptorium, denoting sahidic corpus research: internet platform for interdisciplinary multilayer methods, grants open access to a vast repository of coptic manuscripts. these are available in multiple digital formats, among which are xml (.xml) and treetagger (.tt) foot_0 . given its enriched metadata, we opted for the treetagger format, which was later transformed into html to streamline the process of accessing and parsing file details during the preparatory stage. our selected corpus comprises more than 120 biblical texts, aggregating over 950 chapters. in contrast, the hagiographical narratives detailing the lives of saints encapsulate 36 books attributed to 12 distinct authors. contrary to the modular structure of the biblical texts, the hagiographical data is more aggregated, with each file amalgamating several chapters. ## methods this section outlines the models used for machine translation between coptic, arabic, and english. it includes three subsections: the first introduces the baseline model, the second covers the adaptation strategies, and the third discusses zero-shot and few-shot translation methods. ## results in this section, we present the outcomes of our experiments. first, we offer the results of our seq2seq transformer-based models compared to mt5 (see despite the fact that mt5 small and large models were not exposed to coptic text during the pretraining stage, and their dictionaries do not include any coptic text, the results demonstrate that the mt5 model can successfully adapt and generate coptic text. however, it was observed that the m2m100 model struggles with encoding and decoding in this context, leading to frequent generation of "<@xmath0. consequently, we have chosen not to report the results for the m2m100 model as the results highlight that m2m100 cannot adapt to unseen language with unseen scripts in the dictionary of the m2m100. ## conclusion we presented our efforts to develop a machine translation system tailored for the coptic language, a language characterized by its limited resources. our endeavors led to the assembly of the most extensive parallel corpus involving arabic, english, and coptic to date. building on this foundation, we introduced a novel framework emphasizing crosslingual adaptive training for the mt5 model, particularly beneficial for languages with rare scripts. the empirical results from our study underscore the potential of cross-lingual adaptive training as an instrumental approach for enhancing language processing with unseen scripts. as we look ahead, our research is geared towards refining adaptation techniques, with a particular focus on harnessing the capabilities of english-centric as well as non-english-centeric pretrained llms. in addition, we plan to amass an even more extensive coptic dataset. with this enriched dataset, we plan to continue pretraining several models such as mt5 as well as larger models such as llama3.