ANVITA Machine Translation System for WAT 2021 M ulti I ndic MT Shared Task (2021)

Abstract: This paper describes ANVITA-1.0 MT system, architected for submission to WAT2021 MultiIndicMT shared task by mcairt team, where the team participated in 20 translation directions: English→Indic and Indic→English; Indic set comprised of 10 Indian languages. ANVITA-1.0 MT system comprised of two multi-lingual NMT models one for the English→Indic directions and other for the Indic→English directions with shared encoder-decoder, catering 10 language pairs and twenty translation directions. The base models were built based on Transformer architecture and trained over MultiIndicMT WAT 2021 corpora and further employed back translation and transliteration for selective data augmentation, and model ensemble for better generalization. Additionally, MultiIndicMT WAT 2021 corpora was distilled using a series of filtering operations before putting up for training. ANVITA-1.0 achieved highest AM-FM score for English→Bengali, 2nd for English→Tamil and 3rd for English→Hindi, Bengali→English directions on official test set. In general, performance achieved by ANVITA for the Indic→English directions are relatively better than that of English→Indic directions for all the 10 language pairs when evaluated using BLEU and RIBES, although the same trend is not observed consistently when AM-FM based evaluation was carried out. As compared to BLEU, RIBES and AM-FM based scoring placed ANVITA relatively better among all the task participants.

## introduction this paper presents anvita-1.0 (a neural version of indic translation assistance) mt system, architected for submission to wat 2021 mul-tiindicmt shared task by mcairt team. wat 2021 multiindicmt shared task @xcite comprised of translation of 10 indian lan-guages bengali(bn), gujarati(gu), hindi(hi), kannada(kn), marathi(mr), malayalam(ml), oriya(or), punjabi(pa), tamil(ta), telugu(te) and engish(en) in 20 translation directions (@xmath0. developing quality machine translation system for the indian languages still remains a major challenge, as large number of indian languages are individually resource poor which greatly impacts translation quality. however some of the recent developments do show that careful utilization of multilingualism and/or monolingual corpora, translation quality can be boosted @xcite .the purpose of wat 2021 multiindicmt shared task is to validate the utility of mt techniques that focus on multilingualism and/or monolingual data in the context of indian languages. our anvita-1.0 is realized as a multilingual neural machine translation(mnmt) system based on transformer architecture @xcite . as transformer is sensitive to training noise @xcite , we have rigorously cleaned up the training corpus by applying set of heuristics. for better transfer of translation knowledge among the language pairs, anvita-1.0 used multilingual nmt approach and trained two models, one for the @xmath1.al @xcite . additionally, we employed back-translation @xcite and transliteration techniques between related languages @xcite for selective data augmentation followed by model ensemble for better generalization. as indian languages are morphologically rich, instead of word level tokenization, anvita-1.0 employed subword level tokenization, sentence piece @xcite before putting up for training. details are mentioned in the subsequent sections. anvita-1.0 achieved highest am-fm score for @xmath2, 2nd for @xmath3, @xmath4. overall, as compared to bleu, ribes and adequacy-fluency based scoring relatively placed us better in the ranking chart. ## related work a comprehensive survey covering challenges, design choices and other aspects related to multilingual neural machine translation(mnmt) was presented by dabre et.al @xcite . siripragada et al. (2020) published a low resource indian language dataset and trained a multilingual nmt model on it. @xcite presented a massive multilingual neural translation model with 102 languages. @xcite has done rigorous filtering of parallel corpora. @xcite and @xcite have proposed some heuristics for rigorous filtering of noise from parallel corpora. @xcite have proposed combining parallel corpora by transliteration of related languages(grammar similarity) which improves performance. back translation @xcite is considered by many as one of the effective mechanism for enhancing mt performance. multiindicmt wat 2021 corpora @xcite as shared by the organizer comprises of approximately 10 million parallel sentences covering 10 language pairs (indic, english) and sourced from the following multiple datasets. cvit-pib, pmindia, iitb 3.0, jw, nlpc, ufal entam, uka tarsadia, wikititles, alt, open-subtitles, bible-uedin, mtenglish2odia, odiaen-corp2.0, ted, wikimatrix. multiindicmt wat 2021 training corpora is summarised in ## system overview this section describes anvita-1.0 mt system and its subsystems with reasonable details. ## experimental details anvita-1.0 used opennmt-py 2.0 @xcite toolkit for training. training configuration are 600000 steps for @xmath5, 440000 steps for @xmath6, with batch size of 4096, dropout 0.1, batch type tokens, adam optimizer, warmup steps 8000, word embedding size 512, encoder layers 6, decoder layers 6, heads 8,feed forward dimension of 2048, rnn size 512 and noam as learning rate decay method. anvita-1.0 was trained on nvidia dgx machine having 4 v100 gpu cards, each having 32gb of gpu memory. training of @xmath7. ## evaluation and results translation quality of anvita-1.0 was assessed by the organizer @xcite on the official wat 2021 multiindicmt test set using bleu, ribes @xcite and adequacy-frequency @xcite based metrics. the official evaluation results as declared by the organizer for all the 20 translation directions are shown in performance of @xmath8.29 to 40.05 bleu points, where @xmath9. for @xmath10.85 to 6.17 bleu points, in which @xmath11. we believe that, because of the relatively high resource nature of hindi↔english language pair, this particular pair outperformed all other pairs. ## conclusion and future directions the overall translation performance achieved by anvita-1.0 for the @xmath12. data augmentation largely aided the relatively lower resource languages well. transfer of translation knowledge through shared encoderdecoder seems to be aided the related language better and data filtering improved the overall performance. ribes and am-fm based scoring placed us relatively better than bleu. translation performance figures for the @xmath13.0 are relatively better than that of @xmath14, when evaluated using bleu and ribes, though the same trend is not observed consistently when am-fm based evaluation was carried out. potential reasons could be one to many mapping is relatively harder to learn as compared to many to one mapping with shared decoder. one of the future direction would be to closely investigate whether having shared encoder but separate decoders helps for one-to-many models in the indic context. though we have applied a large number of data filtering heuristics, we noticed that training data was still not free from noises. so another potential future direction would be to explore more effective data filtering techniques and its impacts on mt performance. exploration of additional data augmentation strategies and effective transfer of