NMT based Similar Language Translation for H indi - M arathi (2020)

Abstract: This paper describes the participation of team F1toF6 (LTRC, IIIT-Hyderabad) for the WMT 2020 task, similar language translation. We experimented with attention based recurrent neural network architecture (seq2seq) for this task. We explored the use of different linguistic features like POS and Morph along with back translation for Hindi-Marathi and Marathi-Hindi machine translation.

## introduction machine translation (mt) is the field of natural language processing which aims to translate a text from one natural language (i.e hindi) to another (i.e marathi). the meaning of the resulting translated text must be fully preserved as the source text in the target language. for the translation task, different types of machine translation systems have been developed and they are mainly rule based machine translation (rbmt) @xcite , statistical machine translation (smt) @xcite and neural machine translation (nmt) @xcite . statistical machine translation (smt) aims to learn a statistical model to determine the correspondence between a word from the source language and a word from the target language. neural machine translation is an end to end approach for automatic machine translation without heavily hand crafted feature engineering. due to recent advances, nmt has been receiving heavy attention and achieved state of the art performance in the task of language translation. with this work, we intend to check how nmt systems could be used for low resource and similar language machine this paper describes our experiments for the task of similar language translation of wmt-2020. we focused only on hindi-marathi language pair for the translation task (both directions). the origin of these two languages are the same as they are indo-aryan languages(wikipedia, 2020). hindi is said to have evolved from sauraseni prakrit (wikipedia hindi, 2020) whereas marathi is said to have evolved from maharashtri prakrit (wikipedia marathi, 2020). they also have evolved as two major languages in different regions of india. in this work, we focused only on recurrent neural network with attention based sequence to sequence architecture throughout all experiments. along with it, we also explored the morph @xcite induced sub-word segmentation with byte pair encoding (bpe) @xcite to enable open vocabulary translation. we used pos tags as linguistic feature and back translation to leverage synthetic data for machine translation task in both directions. in the similar language translation task of wmt-2020, we participated as team named "f1plusf6". ## data we utilised parallel and monolingual corpora provided for the task on @xmath0. table-1 describes the training data (parallel and monolingual) on which we carried out all experiments. we deliberately excluded indic wordnet data from the training after doing manual quality check. as this is a constrained task, our experiments do not utilise any other available data. ## pre-processing as a first pre-precessing step we use indicnlp toolkit 1 along with an in-house tokenizer to tokenize and clean both hindi and marathi corpora (train, test, dev and monolingual). ## training configuration recurrent neural network (rnn) based machine translation models work on encoder-decoder based architecture. here, the encoder takes the input (source sentence) and encodes it into a single vector (called as a context vector). then the decoder takes this context vector to generate an output sequence (target sentence) by generating a word at a time @xcite . attention mechanism is an extension to this sequence to sequence architecture to avoid attempting to learn a single vector. instead, based on learnt attention weights, it focuses more on specific words at the source end and generates a word at a time. more details can be found here @xcite , @xcite . for our experiments, we utilize sequence to sequence nmt model with attention for all of our experiments with following configuration. • copy attention @xcite on dynamically generated dictionary as these are two similar languages, share writing scripts and large sets of named entities, we used shared vocab across training. we used opennmt-py @xcite toolkit with above configuration for our experiments. ## back translation back translation is a widely used data augmentation method for low resource neural machine translation @xcite . we utilised monolingual data (i.e of marathi) and a nmt model trained on given training data for a direction (i.e, marathi to hindi) to enrich training data of the opposite directional nmt training (i.e, hindi -marathi) by populating synthetic data. we used around 5m back translated pairs (after perplexity based pruning with respect to sentence length) for both translation directions. using above described configuration, we performed experiments based on different parameter (feature) configurations. we trained and tested our models on word level, bpe level and morph + bpe level for input and output. we also used pos tagger and experimented with shared vocabulary across the translation task. the results are discussed in following result section. ## result table-2 and table-3 show performance of systems with different configuration in terms of bleu score @xcite for hindi-marathi and marathi-hindi respectively on the validation data. we achieved 20.62 and 25.55 development and 5.94 and 18.14 test bleu scores for hindi-marathi and marathi-hindi systems respectively. the results show that for low resource similar language settings, mt models based on sequence to sequence neural network can be improved with linguistic information like morph based segmentation and pos features. the results also show that morph based segmentation along with byte pair encoding improves bleu score for both directions. but marathi-hindi directed translation shows considerable improvement. therefore our method shows improvement while translating from morphologically richer language (marathi) to comparatively less morphologically richer language @xcite . the results also suggest that the use of back translated synthetic data for low resource language pairs reduces the overall performance marginally. the reason for this could be, due to low quantity of training data for nmt models, they could be over learning and back translation could be helping to do better generalization. ## conclusion we conclude from our experiments that linguistic feature driven nmt for similar low resource languages is a promising approach. we also believe that morph+bpe based segmentation is a potential segmentation method for morphologically richer languages.