Neural Machine Translation for Similar Languages: The Case of I ndo- A ryan Languages (2020)

Abstract: In this paper we present the WIPRO-RIT systems submitted to the Similar Language Translation shared task at WMT 2020. The second edition of this shared task featured parallel data from pairs/groups of similar languages from three different language families: Indo-Aryan languages (Hindi and Marathi), Romance languages (Catalan, Portuguese, and Spanish), and South Slavic Languages (Croatian, Serbian, and Slovene). We report the results obtained by our systems in translating from Hindi to Marathi and from Marathi to Hindi. WIPRO-RIT achieved competitive performance ranking 1st in Marathi to Hindi and 2nd in Hindi to Marathi translation among 22 systems.

## introduction wmt 2020 is the fifth edition of wmt as a conference following a series of well-attended workshops that date back to 2006. wmt became a well-established conference due to its blend of research papers and popular shared tasks on different topics such as translation in various domains (e.g. biomedical, news), translation quality estimation, and automatic post-editing. the competitions co-organized with wmt provide important datasets and benchmarks widely used in the mt community. the vast majority of these tasks so far, however, involved training systems to translate to and from english @xcite while only a few of them addressed the problem of translating between pairs of languages with less resources. to address this issue, in 2019, the similar language translation (slt) shared task was introduced at wmt. slt's purpose was to evaluate the performance of state-of-the-art mt systems on translating between pairs of similar languages without english as a pivot language @xcite . the organizers provided participants with training, development, and testing parallel data from three pairs of languages from three different language families: spanish -portuguese (romance languages), czech -polish (slavic languages), and hindi -nepali (indo-aryan languages). systems were evaluated using automatic metrics, namely bleu @xcite and ter @xcite . in slt 2020, the task organizes once again included an indo-aryan language track with hindi and marathi. indo-aryan languages are a sub-family of the indo-european language family which includes bengali, bohjpuri, hindi, marathi, and nepali. these languages are mainly spoken in north and central india, and some neighbouring countries such as nepal, bangladesh, and pakistan etc. the script used in most of these languages are derived from the ancient brahmi script and enriched with high grapheme to phoneme correspondence leading to many orthographic similarities across these languages. in addition to hindi and marathi, slt 2020 features two other tracks with similar languages from the following language families: romance languages (catalan, portuguese, and spanish) and south slavic languages (croatian, serbian, and slovene). in this paper we describe the wipro-rit submission to the slt 2020 indo-aryan track. our wipro-rit system is based on the model described in @xcite . wipro-rit achieved competitive performance ranking 1 st in marathi to hindi and 2 nd in hindi to marathi translation among 22 systems. ## related work with the substantial performance improvements brought to mt by neural approaches, a growing interest in translating between pairs of similar languages, language varieties, and dialects has been observed. recent studies have addressed mt between arabic dialects @xcite catalan and spanish, croatian and serbian @xcite , @xcite , brazilian and european portuguese @xcite , and several pairs of languages and language varieties such as brazilian and european portuguese, canadian and european french, and similar languages such as croatian and serbian, and indonesian and malay @xcite . the interest on diatopic language variation is evidenced by the recent iterations of the var-dial workshop in which papers on mt applied to similar languages varieties, and dialects @xcite @xcite have been presented along with evaluation campaigns featuring multiple shared tasks on a number of related topics such as cross-lingual morphological analysis, cross-lingual parsing, dialect identification, and morphosyntactic tagging @xcite @xcite . ## data for our experiments, we use the hindi-marathi and marathi-hindi wmt 2020 slt data. the released parallel dataset was collected from news @xcite , pmindia @xcite and indic wordnet @xcite datasets. to augment our dataset, we use english-hindi parallel data released in wmt 2014 @xcite , consisting of more than 2 million parallel sentences, which is available as an additional resource. we use a subset of 5 million segments of hindi monolingual news crawled from ca. 32 million data. we also use a subset 5 million marathi monolingual data. we performed similar cleaning and pre-processing methods as we described in case of parallel data. the five million hindi monolingual sentences were first back-translated to english using a hindi-english nmt system. the hindi-english nmt system was trained on english-hindi parallel data released in wmt 2014 @xcite , iitb parallel corpus @xcite , the parallel dataset was collected from news @xcite and the pmindia (haddow and kirefu, 2020) parallel corpus (see we performed the following two steps: (i) we use the cleaning process described in @xcite , and (ii) we execute the moses @xcite corpus cleaning scripts with minimum and maximum number of tokens set to 1 and 100, respectively. after cleaning and removing duplicates, we have 1m en-hi parallel sentences. next, we perform punctuation normalization, and then we use the moses tokenizer to tokenize the english side of the parallel corpus with 'no-escape' option. finally, we apply true-casing. for the case of hindi and marathi, we use indic nlp library foot_0 (kunchukuttan, 2020b) for tokenizaton. ## model architecture our model is based on a transformer architecture @xcite built solely upon such attention mechanisms completely replacing recurrence and convolutions. the transformer uses positional encoding to encode the input and output sequences, and computes both self-and cross-attention through so-called multi-head attentions, which are facilitated by parallelization. we use multi-head attention to jointly attend to information at different positions from different representation subspaces. we present a single multilingual nmt system based on the transformer architecture that can translate between multiple languages. to make use of multilingual data within a single nmt model, we perform one simple modification to the source side of the multilingual data, we use an additional token at the beginning of the each source sentence to indicate the target language by the nmt model would be translated as shown in we train the model with all the processed multilingual data consisting of sen-tence aligned multiple language pairs at once, during inference, we also need to add the aforementioned additional token to each input source sentence of the source data to specify the desired target language. ## experiments in the next sub-sections we describe the experiments we carried out for translating from hindi to marathi and from marathi to hindi for wipro-rit's wmt 2020 slt shared task submission. ## results we present the results obtained by our systems for hindi-marathi in ## conclusion and future work this paper presented the wipro-rit system submitted to the similar language translation shared task at wmt 2020. we presented the results obtained by our system in translating from hindi to marathi and marathi to hindi. our primary system achieved competitive performance ranking first in marathi to hindi and second in hindi to marathi among 22 teams in terms of bleu score. in future work, we would like to further explore the similarity between these two languages in translating to other indo-aryan languages (e.g. bengali, bhojpuri, and nepali). we expect the models presented in this paper to perform well for other indo-aryan language provided that suitable training data is available. furthermore, we would like to apply and evaluate our method on the two other groups of languages in the wmt slt 2020 shared task, romance languages: catalan, portuguese, and spanish, and south slavic languages: croatian, serbian, and slovene. finally, we will be incorporating the translation models presented in this paper to cata-log, an open-source online cat tool that provides users with both mt and tm outputs @xcite .