WT : Wipro AI Submissions to the WAT 2020 (2020)

Abstract: In this paper we present an English–Hindi and Hindi–English neural machine translation (NMT) system, submitted to the Translation shared Task organized at WAT 2020. We trained a multilingual NMT system based on transformer architecture. In this paper we show: (i) how effective pre-processing helps to improve performance, (ii) how synthetic data through back-translation from available monolingual data can help in overall translation performance, (iii) how language similarity can aid more onto it. Our submissions ranked 1st in both English to Hindi and Hindi to English translation achieving BLEU 20.80 and 29.59 respectively.

## introduction now-a-days neural machine translation (nmt) has evolved promising machine translation (mt) paradigm to an established state-of-the-art technology. the sota mt models are following the popular encoderdecoder framework, which encodes the source sentences and decodes the target sentences with a transformer network @xcite . transformer networks are trained in a supervised manner, relying on paired source-target datasets. high quality translation from nmt can be achieved by using large amounts of sentence aligned parallel corpora and an efficient modeling of an nmt architecture. however, the upstream precess i.e., data scarcity can be a challenge for low resource language pairs. therefore, existing sota nmt architecture like transformer fails to produce quality translation output for low resource scenario. however, nmt systems have constantly ranked in the top positions in wmt @xcite and wat @xcite . given the youth of the paradigm and while the main structure of encoder-decoder is still maintained. the research in nmt goes in many directions, including subword unit @xcite for translation of rare words, back translation @xcite or transfer learning @xcite for low resource settings and recently unsupervised training for less or no resources @xcite . in this paper we describe the wipro-nmt submission to the wat 2020 translation track @xcite . our wipro-nmt system is inspired from the model described in @xcite and trained using transformer network. the system achieved best performance ranking 1 st in english to hindi and hindi to english translation among all participants. the paper poses the following contributions: ## data for our experiments, we use the hindi-english and english-hindi workshop of asian translation (wat) 2020 translation data. we used a subset of the released parallel dataset, was collected from news @xcite , pmindia @xcite and indic wordnet @xcite datasets. to augment our dataset, we use english-hindi parallel data released in wmt 2014 @xcite , consisting of more than 2m parallel sentences, is available as an additional resource. all dataset used to train our system are detailed in ## model architecture our model is based on transformer architecture. the transformer architecture (vaswani 1 http://www.statmt.org/wmt20/similar.html et al., 2017) is built solely upon such attention mechanisms completely replacing recurrence and convolutions. the transformer uses positional encoding to encode the input and output sequences, and computes both self-and cross-attention through so-called multi-head attentions, which are facilitated by parallelization. we use multi-head attention to jointly attend to information at different positions from different representation subspaces. we present a single multilingual nmt system based on transformer architecture that can able to translate between multiple languages. to make use of multilingual data within a single nmt model, we perform one simple modification to the source side on the multilingual data, we use an additional token at the beginning of the each source sentence to indicate the target language by the nmt model would be translated. examples are shown in we train the model with all the processed multilingual data consisting of sentence aligned multiple language pairs at once, during inference, we also need to add the aforementioned additional token to each input source sentence of the source data to specify the desired target language. ## experiments in the next sub-sections we describe the experiments we carried out for translating from hindi to english and from english to hindi for wipro's wat 2020 shared task submission. ## results we present the released results obtained by our systems for hindi-english and english to hindi in ## conclusion and future work this paper presented the wipro-nmt system submitted to the translation shared task at wat 2020. we presented the results obtained by our system in translating from hindi to english and english to hindi. our system ranked first among all participated teams in terms of bleu score. this paper also shows how effective pre-processing, back-translation and language similarity help in improving performance. in future work, we would like to further explore the similarity between languages in translating to other indo-aryan languages (e.g. bengali, magadhi, and nepali) and expect that the method presented here to perform well for other languages provided that sufficient training data is available. furthermore, we would like to apply and evaluate our method on other language families. finally, we will be incorporating the translation models to catalog, an open-source online cat tool @xcite .