Multilingual Machine Translation Systems at WAT 2021: One-to-Many and Many-to-One Transformer based NMT (2021)

Abstract: In this paper, we present the details of the systems that we have submitted for the WAT 2021 MultiIndicMT: An Indic Language Multilingual Task. We have submitted two separate multilingual NMT models: one for English to 10 Indic languages and another for 10 Indic languages to English. We discuss the implementation details of two separate multilingual NMT approaches, namely one-to-many and many-to-one, that makes use of a shared decoder and a shared encoder, respectively. From our experiments, we observe that the multilingual NMT systems outperforms the bilingual baseline MT systems for each of the language pairs under consideration.

## introduction in recent years, the neural machine translation (nmt) systems @xcite @xcite have consistently outperformed the statistical machine translation (smt) @xcite systems. one of the major problems with nmt systems is that they are data hungry, which means that they require a large amount of parallel data to give better performance. this becomes a very challenging task while working with low-resource language pairs for which a very less amount of parallel corpora is available. multilingual nmt (mnmt) systems @xcite alleviate this issue by using the phenomenon of transfer learning among related languages, which are the languages that are related by genetic and contact relationships. @xcite have shown that the lexical and orthographic similarity among languages can be utilized to improve translation quality between indic languages when limited parallel corpora is available. another advantage of using mnmt systems is that they support zero-shot translation, that is, translation among two languages for which no parallel corpora is available during training. a mnmt system can also drastically reduce the total number of models required for a large scale translation system by making use of a single many-to-many mnmt model instead of having to train a separate translation system for each of the language pairs. this reduces the amount of computation and time required for training. among various mnmt approaches, using a single shared encoder and decoder will further reduce the number of parameters and allow related languages to share vocabulary. in this paper, we describe the two mnmt systems that we have submitted for the wat 2021 multiindicmt: an indic language multilingual task @xcite as team 'cfilt', namely one-to-many for english to indic languages and many-to-one for indic languages to english. this task covers 10 indic languages which are bengali, gujarati, hindi, kannada, malayalam, marathi, oriya, punjabi, tamil and telugu. a second line of work on mnmt uses a single shared encoder and decoder @xcite irrespective of the number of languages on the source or the target side. an en-bn en-gu en-hi en-kn en-ml en-mr en-or en-pa en-ta en-te 2017 )'s approach where in one-to-many and many-to-many models a language specific token is prepended to the input sentence to indicate the target language that the model should translate to. we use transformer @xcite architecture which has proven to give superior performance over the rnn based models @xcite @xcite . ## our approach the various types of multilingual models that we have implemented are one-to-many and many-toone, each of which are discussed below. ## experiments in this section, we discuss the details of the system architecture, dataset, preprocessing, models and the training setup. ## results and analysis the bilingual evaluation understudy (bleu) @xcite metric, the rank-based intuitive bilingual evaluation score (ribes) @xcite metric and adequacy-fluency metrics (amfm) @xcite are used to report the results. the baseline results are obtained by training bilingual models and then we have used automatic evaluation procedures same as those performed in wat 2021. the one-to-many and many-to-one results are those reported by wat 2021 on our submitted translation files. we observe that for all language pairs in both the translation directions, the mnmt models give superior performance as compared to the bilingual nmt models. for relatively high resource language pairs like english-hindi and english-bengali the increase in bleu score is less while for relatively low resource language pairs like english-kannada and english-oriya the increase in bleu score is substantial. from the above observation it follows that low resource language pairs benefit much more from multilingual training than high resource language pairs. an increase of up to 8.93 bleu scores (for kannada to english) is observed using mnmt systems over the bilingual baseline nmt systems. ## conclusion in this paper, we have discussed our submission to the wat 2021 multiindicmt: an indic language multilingual task. we have submitted two separate mnmt models: a one-to-many (english to 10 indic languages) model and a many-to-one (10 indic languages to english) model. we evaluated our models using bleu and ribes scores and observed that the mnmt models outperform the separately trained bilingual nmt models across all the language pairs. we also observe that for the lower resource language pairs the improvement in performance is much more as compared to that for the higher resource language pairs.