AGE : A mharic, G e’ez and E nglish Parallel Dataset (2024)

Abstract: African languages are not well-represented in Natural Language Processing (NLP). The main reason is a lack of resources for training models. Low-resource languages, such as Amharic and Ge’ez, cannot benefit from modern NLP methods because of the lack of high-quality datasets. This paper presents AGE, an open-source tripartite alignment of Amharic, Ge’ez, and English parallel dataset. Additionally, we introduced a novel, 1,000 Ge’ez-centered sentences sourced from areas such as news and novels. Furthermore, we developed a model from a multilingual pre-trained language model, which brings 12.29 and 30.66 for English-Ge’ez and Ge’ez to English, respectively, and 9.39 and 12.29 for Amharic-Ge’ez and Ge’ez-Amharic respectively.

## introduction language is the foundation on which communication rests, allowing us to share ideas and interact with one another @xcite . one of the nlp applications is machine translation (mt), which helps facilitate human-machine and human-human communications @xcite . data availability is one of the criteria to categorize one language as a high or low-resource language @xcite . recently, interest in low-resource mt has been increasing both within the mt research community @xcite , as well as in native speaker communities @xcite . modern nlp technologies, however, have primarily been developed in western societies @xcite . the current state-of-the-art(sota) mt models were trained on enormous datasets, including sentences in a source language and their corresponding target language translations, which is the most effective of these systems @xcite . to date, there is no publicly available mt system for ge'ez language and it's not represented in the commercial mt systems such as lesan foot_0 , google translate foot_1 , microsoft translator 3 , and yandex translate foot_2 . it is also not included in large-scale pre-trained multilingual models like nllb @xcite , mt5 @xcite , byt5 @xcite , and m2m-100 @xcite . this makes it harder for people to learn and use the language. so, by focusing on ge'ez, an ancient language with profound cultural and religious significance in ethiopia, alongside amharic, the country's official working language, and english, a global lingua franca, this dataset aims to bridge the gap between historical linguistic treasures and modern technological advancements. through this work, we aim to provide a dataset for researchers and technologists aiming to advance machine translation capabilities, linguistic studies, and cultural preservation efforts. furthermore, by enriching the available resources for ge'ez, we contribute to the broader goal of advancing low-resource languages. ## related work one of the major challenges in developing mt models for ge'ez is the lack of public data. there were attempts to compile parallel corpora for ge'ez to english and ge'ez to amharic mt tasks, but the development was unsatisfactory. @xcite researched ge'ez-amharic mt using smt. he used irstlm for language modeling. the research was conducted on a dataset comprising 12,840 parallel amharic-ge'ez sentences, achieving an average translation accuracy with a bleu score of 8.26 based on 10-fold cross-validation. @xcite is the only publicly available dataset that was part of an effort to train statistical machine translation(smt) for english-ethiopian languages and made 11,663 ge'ez-english parallel sentences. they achieved english-ge'ez and ge'ez english translations with a bleu score of 6.67 and 18.01, respectively. using deep learning approaches, @xcite have explored bidirectional nmt from ge'ez to english. the experiment was conducted by leveraging 16,569 parallel sentences from the holy bible and battle of saints and manually preparing daily conversational sentences. the results indicated that the transformer (?) model achieved bleu scores of 27.19 for english to ge'ez translation and 29.39 for ge'ez to english translation. another work by @xcite used nmt and transformers and attempted three experiments that used a pretrained masked language model (mlm) utilizing a monolingual dataset of 33,004 sentences for each language. the experiments involved a parallel corpus for supervised learning without a pre-trained model and fine-tuning a pre-trained mlm with a bilingual dataset. the outcomes were evaluated using the bleu score, achieving 31.65 in the second and 33.02 in the third experiment. another recent work by (wassie, 2023) improved translation by 4 bleu using a new model but faced challenges with nllb-200 for ge'ez due to insufficient data. they also experimented with gpt-3.5's trial, which resulted in a 9.2 bleu score, underperforming compared to their model's 15.2. they also highlighted the difficulties of training ge'ez mt models. a recurring issue noted in these experiments is the absence of data sharing with the public domain. as shown in ## ge'ez language ge'ez (ethiopge'ez), which is also known as ethiopic, is one of the oldest semitic languages @xcite and its alphabets is among the oldest alphabets still in use in the world of today. furthermore, the ge'ez language is among the four languages (sabaean, greek, and arabic) that have been and continue to be used for ancient inscriptional arts. ge'ez is currently not an actively spoken language nor a native tongue of any people. its use is limited to the liturgical language of the ethiopian orthodox tewahedo, eritrean orthodox tewahedo, ethiopian catholic, and eritrean catholic christians(molla and tabor, 2018). it is also used during prayer and at regularly scheduled public religious feast celebrations. the bible dominates the literature, and it comprises the deuterocanonical books. according to @xcite , this language also has many medieval and early modern original texts. the majority of the essential works are correspondingly the literature of the ethiopian orthodox tewahedo church. these works include christian orthodox liturgy (service books, prayers, hymns), hagiographies, and a range of patristic literature. around 200 texts were written about home-grown ethiopian saints from the fourteenth to the nineteenth century. the religious alignment of ge'ez literature was due to traditional education being the obligation of priests and monks. more info about the alphabet on ge'ez can be found in the appendix section. ## creation of the dataset we introduce our newly ge'ez-centered parallel dataset; age -amharic, ge'ez , english for machine translation. ## baseline experiments as shown in to extend these studies, we incorporated an approach by leveraging the nllb-200 (team et al., 2022), a pre-trained language model. accessing the large nllb-200 model requires a minimum of four 32gb gpus just for inference, showcasing the need for significant computational resources. so, we used the nllb-200 600m parameter variant, a dense transformer model distilled from nllb-200 due to its much lower resource requirements, making it a more practical option for our computational constraints. the work by @xcite to effectively adapt to large-scale pre-trained models and get improved performance suggests that these models have better capability for relatively smaller datasets. so, we split our dataset into train (80%), dev (10%), and test split (10%). we fine-tune the model using the huggingface transformer tool @xcite with a learning rate 5e-5, a batch size of 4 per device, a maximum source length, and a maximum target length of 128, and a beam size of 10. all the experiments were performed on google colab pro. then, the quality of translation is assessed using the bleu score @xcite , a standard in the field for its objectivity and correlation with human judgment. our baseline experiments focused on bidirectional translation tasks, amharic-ge'ez language pair bleu amharic-ge'ez 9.39 ge'ez-amharic 12.29 english-ge'ez 12.87 ge'ez-english 30.66 and english-ge'ez translations, aiming to establish a foundational understanding of the nllb-200 600m model's capabilities within the context of our dataset. since our primary focus was developing machine translation for ge'ez, we skipped training the model on bidirectional english-amharic translation. ## conclusion and future works this paper presents an attempt to prepare a standard parallel corpora for ge'ez. one thousand newly translated sentences were gathered from nonreligious domains, and the rest text data was gathered from religious domains on the internet. then, the data are further pre-processed and normalized to prepare a parallel dataset for the model training task. using our dataset, we fintuned nllb-200 model. the experimental results show that translating to and from english resulted in a better bleu score than english to ge'ez and amharic to ge'ez and vice versa. the abundance of english data in the pre-trained model and the morphological richness of ethiopian languages significantly impact the model's performance during bidirectional training involving ge'ez and amharic and when these languages are target languages. to the best of our knowledge, this is the first ready-to-use amharic, ge'ez, english tripartite dataset. our initiative to make the dataset and models open source will open doors for many researchers and developers. future works include increasing both the quantity and diversity of the dataset. we also intend to incorporate the several ge'ez data sources that are now absent from this dataset.