The H elsinki- NLP Submissions at NADI 2023 Shared Task: Walking the Baseline (2023)

Abstract: The Helsinki-NLP team participated in the NADI 2023 shared tasks on Arabic dialect translation with seven submissions. We used statistical (SMT) and neural machine translation (NMT) methods and explored character- and subword-based data preprocessing. Our submissions placed second in both tracks. In the open track, our winning submission is a character-level SMT system with additional Modern Standard Arabic language models. In the closed track, our best BLEU scores were obtained with the leave-as-is baseline, a simple copy of the input, and narrowly followed by SMT systems. In both tracks, fine-tuning existing multilingual models such as AraT5 or ByT5 did not yield superior performance compared to SMT.

## introduction this paper presents the helsinki-nlp submissions to the nadi 2023 shared tasks. we participated in subtasks 2 and 3, which consisted in translating dialectal data into modern standard arabic (msa) @xcite . this was the first time the nadi shared task involved translation, following past tasks on dialect identification and sentiment analysis @xcite @xcite . the arabic dialectal continuum stretches from morocco in the west to oman in the east. various classifications of the dialects have been proposed, ranging from large regions to country-level or even city-level divisions @xcite . the arabic language area is also well known for its diglossic situation. while modern standard arabic is used in education, media and culture across the continuum, it is not native to any of the dialectal regions. the translation subtasks focused on four arabic dialects: egyptian, emirati, jordanian, and palestinian. the shared task organizers provided the madar corpus @xcite as the training material for the closed track (subtask 2), which did not allow for the use of additional training data. the subtask 3 was described as open track where any additional training material was allowed. since our initial experiments showed that neural models were particularly affected by the small size of the madar training data, a large part of our efforts went into creating additional parallel data for the in subtask 3 models. in particular, we focused on freely available monolingual msa corpora, which we then back-translated to three target dialects, grouping jordanian and palestinian together. adding the back-translated data to the original training corpus allowed our neural models to perform on par with less data-hungry statistical models. we participated in subtask 2 with three submissions and in subtask 3 with four submissions. our submissions can be divided into four different approaches: section 2 describes the data collection and preparation whereas section 3 outlines the proposed models in more detail. our results are presented in section 4 and further discussed in section 5. section 6 offers conclusions of our work. ## conclusions in this paper, we described our participation in the nadi shared task, where we submitted seven systems to two tracks. our submissions placed second on both tracks. our strongest translation method was smt in both tracks, but given the difficulty of the task, it was outperformed by an lai baseline in the closed track. neural models closed the gap to the smt models only with large amounts of additional parallel data obtained through backtranslation. we would like to note again that our open track submissions do not use any human-translated parallel training data besides madar, and that the total amount of training data is smaller than what was used for arat5 pre-training. this makes our models, in particular the smt ones, more data efficient than large pretrained models such as arat5 or byt5. we also showed that the participating systems could have been ranked differently with a characterbased evaluation metric, which underlines the importance of the selected metrics.