Parallel resources for T unisian A rabic Dialect Translation (2020)

Abstract: The difficulty of processing dialects is clearly observed in the high cost of building representative corpus, in particular for machine translation. Indeed, all machine translation systems require a huge amount and good management of training data, which represents a challenge in a low-resource setting such as the Tunisian Arabic dialect. In this paper, we present a data augmentation technique to create a parallel corpus for Tunisian Arabic dialect written in social media and standard Arabic in order to build a Machine Translation (MT) model. The created corpus was used to build a sentence-based translation model. This model reached a BLEU score of 15.03% on a test set, while it was limited to 13.27% utilizing the corpus without augmentation.

## introduction nowadays, the use of the dialect in social networks is becoming more and more important. besides dialects emerge as the language of informal communication on the web (forums, blogs, emails and social media). thus, there is a shift from a purely oral language to a written language, without established standardization or standard spelling. it is also obvious that the dialect becomes intensively employed by internet users in social networks to share their thoughts and opinions. this kind of writing is sometimes incompressible even by persons speaking the same dialect, especially arabic written using the latin script. consequently, it is necessary to transform the text from its informal type to a formal type in order to be understandable and suitable for natural language processing (nlp) tools. in this work, we create parallel resources in order to build a mt model able to transform the tunisian dialect (td), written in social networks, into the modern standard arabic (msa). this represents a motivating task for both academic and industrial fields. indeed, switching to a standard language facilitates communication between people overall the word. it also makes automatic opinion analysis easier using well-equipped language tools. for instance, businesses operating on the global market rely on social media campaigns to promote their brand and products, creating engaging and catchy content that can be exploited by users. however, to our knowledge, until now there is no works that have dealt with the translation of td, especially the one that exists in social networks. we introduce, in this study, an augmentation technique in order to build useful resources to train an efficient td-msa mt model. for this, we collected a significant number of parallel written resources to start with a total word count of about 75k. then, we segmented the msa sentences and we generated partial parallel sentences using back translation based on a bilingual dictionary created from the collected corpus. finally, in order to trained the efficiency of the proposed resources, we test a statistical mt model on different configurations of the created corpus. obviously, the translation performance is improved by augmenting the size of td-msa parallel data sets. the remainder of this paper is structured as follows: in section 2, we review related work on data augmentation. in section 3, we describe the process of building parallel resources. section 4 depict the different steps of building a statistical translation model. we discuss in section 5 the performance of the resulting model. ## related work machine translation for low-resource language is a well-known task in the nlp community. several works focused on it to form efficient mt models. in order to train these models, different configurations of parallel resources have been proposed. there are those who have proven that sentence segmentation associated with the technique of back translation is a very suitable method for constructing parallel resources for low resourced languages. for example, @xcite applied this technique to generate pseudo-parallel pairs of japanese-chinese sentences by dividing the long parallel sentence pair of the corpus into parallel partial sentences pairs and using back-translation from the target partial sentence. the pair of long parallel sentences was divided into segments at the level of punctuation marks such as ",", ";", ":". this method improved the performance of translation. @xcite proposed also a back translation method for low resource languages. the authors used an iterative back-translation of monolingual low resource data with the models trained on the transliterated high-resource data and utilized the resulting parallel corpus to train the final models. similarly, another method was suggested by @xcite to generate an english-german and turkish-english parallel corpus using the back-translating of monolingual target data into the source language in order to obtain pseudo-source sentences. the back-translation was based on a statistical translation model. the size of the corpus was increased by the pairs of pseudo-source sentences and the original target sentences. otherwise, @xcite proposed two models : a neural model and a sentence-based model in order to generate automatically a parallel data using the iterative back-translation and denoising the effect of the language models formed on the target side. @xcite generated new pairs of sentences containing rare words in new synthetically-created contexts. researchers used language models trained on a large amount of german monolingual data to produce new english-german and german-english sentence pairs. @xcite presented a soft contextual data augmentation method in order to build neural machine translation. the method consists in replacing a randomly-chosen word in a sentence with a soft distributional representation provided by a language model. the translation results proved the effectiveness of this method. a new efficient data augmentation approach to improve german-english translation performance was presented in @xcite . authors proposed a diversified data augmentation strategy that does not use extra monolingual data. they trained a forward and backward model to translate the training data. ## td-msa parallel resources parallel corpus aligned at the sentence level are essential resources to build mt systems. one of our goals is to collect the largest possible set of parallel td-msa sentences. we collected in the first step, the existing free parallel corpus used in the state of the art. secondly, we translated manually a corpus scraped from social networks. ## statistical machine translation model in order to test the efficiency of the resource-created method in the context of machine translation, we adapted a simple traditional sentence-based statistical machine translation (smt) model. indeed, deep learning-based methods have made significant progress in recent years in the context of machine translation. we opted for the same methodology of @xcite . the latter has shown its effectiveness in several similar works as it is the case in @xcite . train set. we carried out three experiments to analyze the utilized data. we increased the train corpus gradually in order to test the effect of the data augmentation on the applied model, as shown in experiment 1 : we started by training the model with the available parallel data. we employed firstly td-msa data from the madar and padic corpus. we got a bleu score of 13.07% on the test set. then, we added the td-const data set collected from the tunisian constitution. this enrichment by the constitution data degraded the result of translation to 12%. this score can be explained by the nature of vocabulary utilized in the tunisian constitution corpus which is different from that of social networks. afterwards, we integrated the data into 400 sentences from the td-com corpus. the bleu score was improved to 13.27% on the same test set. following this extension, we trained a model on the following configuration of corpus : madar, padic and td-com. the resulting model achieved an improvement in the bleu score that rose to 13.67%. we can conclude, from this experiment, that the increase in corpus is preferably done with a corpus of the same domain or close domain. experiment 2 : based on the results of experiment 1, we considered the corpus composed by the sub-corpora of madar, padic and td-com. we segmented all long sentences of this corpus. the aim of this experiment is to test the influence of the sentence size on the quality of translation. on the segmented corpus version, resulting from the back translation, we have learned again the smt model. by this textual representation, the performance of the model has degraded. the model achieved a blue score of 11.58%. the dictionary-based translation led to the loss of meaning of some sentences and, consequently, reduced the score obtained in this experiment. experiment 3 : in this experiment, we combined, in the training set, the pairs of the segmented sentences with the pairs of the original sentences in the corpus. this improved the performance of the model which achieved a blue score of 15.03%. ## conclusion we presented, in this paper, a process of creating parallel td-msa resources by exploiting firstly the available resources. then, we enriched the data by building a parallel corpus scraped from social media. afterwards, we proposed a corpus augmentation technique based on segmenting the well-endowed language and back translation. in order to test the effectiveness of the constructed resources, we trained a smt model using different configurations of collected corpus and we tested it on a test corpus taken from social media. experimental results show that the "augmented" configuration achieved better result of translation with a bleu score of 15.03%. we aim in future work, to further increase the size of the corpus and include other types of comments like those that contain code switching. we also aim to test the efficiency of the neuronal approach to translate the written comments in social networks.