E nglish- M arathi Neural Machine Translation for L o R es MT 2021 (2021)

Abstract: In this paper, we (team - oneNLP-IIITH) describe our Neural Machine Translation approaches for English-Marathi (both direction) for LoResMT-20211 . We experimented with transformer based Neural Machine Translation and explored the use of different linguistic features like POS and Morph on subword unit for both English-Marathi and Marathi-English. In addition, we have also explored forward and backward translation using web-crawled monolingual data. We obtained 22.2 (overall 2 nd) and 31.3 (overall 1 st) BLEU scores for English-Marathi and Marathi-English on respectively

## introduction machine translation (mt) is a field of natural language processing which aims to translate a text from one natural language (i.e english) to another (i.e marathi). the meaning of the source text must be fully preserved in the resulting translated text in the target language. recent years have seen significant quality advancements in machine translation with the advent of neural machine translation. for the translation task, different types of machine translation systems have been developed and they are mainly categorized into rule based machine translation (rbmt) @xcite , statistical machine translation (smt) @xcite and neural machine translation (nmt) @xcite . rule based machine translation (rbmt) translates on the basis of grammatical rules. it involves a grammatical analysis of the source language and the target language. based on the analysis, it generates the translated sentence @xcite . statistical machine translation (smt) is based on statistical models, which analyse large parallel and monolingual text and tries to determine the correspondence between a source language word and a target language word. nmt @xcite is an end to end approach for automatic machine translation without heavy hand crafted feature engineering. due to recent advances, nmt has been receiving heavy attention and achieved state of the art performance in the task of language translation. with this work, we intend to check how nmt systems could be developed for low resource machine translation. this paper describes our experiments for loresmt-2021 foot_1 @xcite . the third edition of loresmt-2021 aims at building mt systems for low-resource language pairs on covid-related texts. for our work, we focused only on english-marathi language pair (both directions) and participated for categories where in first, we only used given parallel training data (constrained) and in second, we utilized available parallel corpora from different sources for english-marathi and english-hindi (unconstrained). in this work, we experimented only with transformer @xcite based neural machine translation throughout. along with it, we also explored the morph @xcite induced sub-word segmentation with byte pair encoding (bpe) @xcite to enable open vocabulary translation. we used pos tags as linguistic feature for english-marathi direction along with forward and back translation to leverage synthetic data for machine translation. we also explored the use of english-hindi parallel data for english-marathi as origin of these two languages are the same and they are indo-aryan languages (wikipedia, 2021). hindi is said to have evolved from sauraseni prakrit (wikipedia @xcite whereas marathi is said to have evolved from maharashtri prakrit (wikipedia marathi, 2021) and they both use the same writing script -devanagari foot_2 . in loresmt-2021, we participated as team "onenlp-iiith". we utilized provided parallel and monolingual corpora for the machine translation task on @xmath0. table-1 describes the training (parallel and monolingual) and validation data (parallel) after cleaning (i.e removed parallel data from training which are also in validation). we carried out constrained experiments on this data. for unconstrained experiments we use additional parallel dataset from samanantar @xcite . for back and forward translation, we web-crawled monolingual data for both english and marathi to aid relatively new nlp domain covid. table-2 describes this additional dataset in terms of number of sentences, token and type. ## data pre-processing for data pre-precessing, we used indicnlp tool foot_3 with in-house tokenizer to tokenize and clean both english and marathi corpora (train, test, valid and monolingual) as a first step. following subsections explain other pre-processing steps for our mt experiments. ## training configuration throughout all experiments, we used transformer sequence to sequence architecture with the following configuration for constrained and unconstrained experiments. for these experiments, we used shared vocab across trainings. we used opennmt-py @xcite toolkit with above configuration for our experiments. using the above described configuration, we performed experiments based on different parameter (feature) configurations. we trained and tested our models on word level, bpe level and morph + bpe level for input and output. we also used pos tagger and experimented with shared vocabulary across the translation task. the results are discussed in following result section. ## result table-3 and table-4 show performance of our systems with different configurations in terms of bleu score @xcite for english-marathi and marathi-english respectively on the validation and test data. we achieved highest 17.9 development and 22.2 test bleu scores for english-marathi and highest 32.88 development and 31.6 test bleu scores for the results show that for low resource settings, transformer network based mt models can be improved with linguistic information like morph and pos features. the results also indicate that morph based segmentation along with byte pair encoding improves bleu score and can be used for morph rich languages. the results also suggest that performance drastically improves when model is exposed to more parallel data (for unconstrained setting). our experiments suggest that use of english-hindi parallel data gives performance boost by 3.0+ bleu points for english-marathi and almost 10.0+ bleu points for marathi-english. also, forward and back translated synthetic data obtained from same covid domain improves quality of nmt models marginally, as they could be helping models to do better generalization. from the test results (table -3 and table-4 ), we stand at overall 2 nd and 1 st for english-marathi and marathi-english respectively. ## conclusion from our experiments, we conclude that linguistic feature driven nmt for low resource languages is a promising approach and use of similar language training data gives a significant boost in performance to the low resource language.