S irius_ T ranslators at OSACT 6 2024 Shared Task: Fin-tuning Ara-T5 Models for Translating A rabic Dialectal Text to M odern S tandard A rabic (2024)

Abstract: This paper presents the findings from our participation in the 6th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT6) in 2024. Our specific focus was on the second task (Task 2), which involved translating text at the sentence level from five distinct Dialectal Arabic (DA) (Gulf, Egyptian, Levantine, Iraqi, and Maghrebi) into Modern Standard Arabic (MSA). Our team, Sirius_Translators, fine-tuned four AraT5 models namely; AraT5 base, AraT5v2-base-1024, AraT5-MSA-Small, and AraT5-MSA-Base for the Arabic machine translation (MT) task. These models were fine-tuned using a variety of parallel corpora containing Dialectal Arabic and Modern Standard Arabic. Based on the evaluation results of OSACT6 2024 Shared Task2, our fine-tuned AraT5v2-base-1024 model achieved an overall BLEU score of 21.0 on the development (Dev) set and 9.57 on the test set, respectively.

## introduction to emphasize the significance of addressing arabic dialects, it's noteworthy that ethnologue foot_0 ranks arabic as the language with the 5th highest number of native speakers, totalling approximately 420 million individuals across 22 countries in the middle east and north africa region. arabic is characterized by diglossia, a linguistic phenomenon where msa is used in formal contexts, while da is prevalent in informal settings @xcite . dialects are broadly categorized by region, such as egyptian or gulf dialects, but they also exhibit nuanced variations even within individual countries. the linguistic variation presents substantial challenges for mt models trained on msa. employing these models, designed specifically for msa, on da can be problematic, resulting in subpar translation outcomes when applied to da. one potential solution to overcome this challenge involves creating parallel corpora, including msa translations of text written in da. recently, considerable efforts have been devoted to translating dialects into msa. however, the prevalent approach across most studies involves treating each dialect independently. as a result, it is crucial to formulate models with the capability to collectively manage and process at least the most common arabic dialects. in this paper, we detail the experiments conducted to develop da mt model. more precisely, we evaluate the results of fine-tuning different architectures (versions) of the arat5 transformer model @xcite , employing various datasets for the training phase. the structure of the paper is as follows: section 2 provides background information about arabic dialects. section 3 outlines related works. section 4 describes the dataset used. the research methodology, including the fine-tuning of arat5 models and training configuration, is presented in section 5. in section 6, we discuss the obtained results. finally, section 7 offers a conclusive summary and discusses potential future work. ## arabic dialects overview provided here is contextual background on the variation found in arabic dialects. msa represents the formal variant of arabic, taught in educational institutions and utilized for formal texts and news presentations. msa has its roots in the classical arabic of the qur'an, albeit experiencing changes in vocabulary and specific aspects of grammar over time. nevertheless, the majority of arabs speak their regional dialect as their natural language which is notably different from msa form of arabic. while the precise categorization of regional dialects may not be entirely consistent, here are a few main groups: ## related work in the realm of neural machine translation (nmt) for da, the predominant emphasis has revolved around translating these dialects into msa. nonetheless, a majority of these studies often centre on a singular dialect, as discussed earlier in this document, leading to a deficiency in models that cover a wide range of arabic dialects. as an example, al-ibrahim and duwairi (2020), conducted research focusing on translating the jordanian arabic dialect into msa through deep learning techniques, employing an rnn encoderdecoder model. the progress of their work was, however, constrained by the limited size of the corpus. likewise, @xcite addressed the task of translating levantine dialects, encompassing jordanian, syrian, and palestinian, into msa. they utilized a comparatively small dataset of parallel sentences sourced from madar padic corpora. in their approach, they adopted a multitask learning model, where the decoder was shared across various language pairs, while each source language had its dedicated encoder. in a similar fashion, kchaou et al. (2022) adopted a hybrid approach in constructing a translation model for the tunisian dialect. they proposed various augmentation methods to generate a large corpus and subsequently tested different nmt models using this corpus. in the domain of low-resource nmt for the algerian arabic dialect, @xcite , introduced a transductive transfer learning approach. in this approach, the knowledge is conveyed from parent to child models. the evaluation was conducted employing two datasets; madar and padic. the implementation of the transductive transfer learning approach done by using two types of nmt models namely: seq2seq and attentional-seq2seq. furthermore, nagoudi et al. (2022b) developed turjuman foot_1 , a comprehensive neural toolbox with the capability of translating 20 different languages into msa. the turjuman toolbox leverages the strengths of arat5 model and explores its proficiency in arabic decoding. turjuman was developed to utilize semantic similarity for collecting parallel data samples that are openly accessible, ensuring the quality of the collected data. most recently, researchers in the nmt field, have come to the fact that transfer learning through straightforward fine-tuning is an effective method, particularly when applied between closely related high-resource and low-resource languages @xcite . ## datasets in task 2 of osact6 workshop, organizers shared the dev and test set in codalab foot_2 , for developing and testing purposes respectively. in addition, participants were free to use any of the available linguistics resources and corpora for training their models, called the training set (train). our methodology starts with the training of the chosen arat5 models, employing five distinct datasets, and fine-tuning their hyper-parameters as a result. after the training phase, we evaluated the performance of our models on the dev set provided by the organizers of the osact6 2024 shared task. ultimately, predictions were generated using the optimal model configuration on the test set. the following sections in this paper will provide details about the selected datasets, arat5 models, and the training configuration. shared task 2 of the osact6 workshop allowed the participants to use any available resources and tools for training and fine-tuning their models. this section details the datasets employed in training our models. while exploring potentially valuable publicly available datasets, we considered those encompassing various arabic dialects, specifically regional variations pertinent to the five dialects of interest in shared task 2 of osact6. we identified and made use of five datasets: 1)madar, 2)padic, 3)dial2msa, 4) arabic semantic textual similarity (sts) and 5)sadid datasets. while padic @xcite is a parallel corpus comprising texts that belong to two primary arabic dialects alongside the msa form. it comprises three sub-dialects from maghrebi: algerian, anab, and tunisian. additionally, it incorporates two sub-dialects from levantine: syrian and palestinian. dial2msa the dial2msa dataset, as outlined by @xcite , encompasses tweets written in four distinct arabic dialects: egyptian, gulf, lev-antine, and maghrebi, along with their respective msa translations. it's important to note that the validation process for the translations was carried out manually only for the egyptian and maghrebi dialects. in this research, the entire padic dataset, which includes translations that have not undergone validation, was employed during the training phase of our models. arabic sts dataset collected by, al sulaiman et al. (2022), focuses on determining semantic similarity between two given arabic sentences. each english phrase was translated into three target languages namely: msa, egyptian and saudi dialect sadid (abid, 2020) is a parallel corpus for english, egyptian, levantine and msa. the dialectal texts were collected from three distinct sources:1) wikipedia for its diverse domains and clear language, 2) aesop's fables for its narrative style, and 3) specific dialogues from movie subtitles. english was chosen as the source language for sentences rather than msa to avoid introducing bias into the translations @xcite . various translators offer translations with varying degrees of dialectal influence. ## methodology within this section, we present the arat5 models that serve as our foundation, illustrate the finetuning process, and delve into the optimization of hyper-parameters. ## results and discussion all models utilized in our research underwent evaluation using the bilingual evaluation understudy (bleu) metric @xcite , which measures the matching between text generated by the machine (model) and the reference translation based on overlapping words.