Language Relatedness and Lexical Closeness can help Improve Multilingual NMT : IITB ombay@ M ulti I ndic NMT WAT 2021 (2021)

Abstract: Multilingual Neural Machine Translation has achieved remarkable performance by training a single translation model for multiple languages. This paper describes our submission (Team ID: CFILT-IITB) for the MultiIndicMT: An Indic Language Multilingual Task at WAT 2021. We train multilingual NMT systems by sharing encoder and decoder parameters with language embedding associated with each token in both encoder and decoder. Furthermore, we demonstrate the use of transliteration (script conversion) for Indic languages in reducing the lexical gap for training a multilingual NMT system. Further, we show improvement in performance by training a multilingual NMT system using languages of the same family, i.e., related languages.

## introduction neural machine translation @xcite @xcite has become a de-facto for automatic translation of language pairs. nmt systems with transformer @xcite based architectures have achieved competitive accuracy on data-rich language pairs like english-french. however, nmt systems are datahungry, and only a few pairs of languages have abundant parallel data. for low resource setting, techniques like transfer learning @xcite and utilization of monolingual data in an unsupervised setting @xcite @xcite have shown support for increasing the translation accuracy. multilingual neural machine translation is an ideal setting for low resource mt @xcite since it allows sharing of encoder-decoder parameters, word embeddings, and joint or separate vocabularies. it also enables zero-shot translations, i.e., translating between language pairs that were not seen during training @xcite . in this paper, we present our system for multi-indicmt: an indic language multilingual task at wat 2021 @xcite . the task covers 10 indic languages (bengali, gujarati, hindi, kannada, malayalam, marathi, oriya, punjabi, tamil, and telugu) and english. to summarize our approach and contributions, we (i) present a multilingual nmt system with shared encoder-decoder framework, (ii) show results on many-to-one translation, (iii) use transliteration to a common script to handle the lexical gap between languages, (iv) show how grouping of languages in regard to their language family helps multilingual nmt and (v) use language embeddings with each token in both encoder and decoder. ## system overview in this section, we describe the details of the submitted systems to multiindicmt task at wat2021. we report results for four types of models: to train our multilingual models, we use shared encoder-decoder transformer architecture. to handle the lexical gap between indic languages in multilingual models, we convert the data of all indic languages to a common script. we choose the common script as devanagari (arbitrary choice). we also perform a comparative study of systems when the encoder and decoder are shared only between related languages. to perform this comparative study, we group the provided set of languages in two parts based on the language families they belong to, i.e, the system is trained from indo-aryan (group) to english, and dravidian (group) to english. indo-aryan-to-english contains bengali, gujarati, hindi, marathi, oriya, punjabi to english, and dravidian-to-english contains kannada, malayalam, tamil, telugu to english. we use shared subword vocabulary of the languages involved while training multilingual models, and a common vocabulary of source and target languages to train bilingual models. ## conclusion in this paper, we study the influence of sharing encoder-decoder parameters between related languages in multilingual nmt by performing experiments with the grouping of languages based on language family. furthermore, we also perform experiments of multilingual nmt with all indic language data converted to the same script, which helps the model in learning better translation by utilizing the benefit of better shared vocabulary. in the future, we plan to utilize monolingual data from @xcite to improve multilingual nmt further.