PICT @ WAT 2022: Neural Machine Translation Systems for Indic Languages (2022)

Abstract: Translation entails more than simply translating words from one language to another. It is vitally essential for effective cross-cultural communication, thus making good translation systems an important requirement. We describe our systems in this paper, which were submitted to the WAT 2022 translation shared tasks. As part of the Multi-modal translation tasks’ text-only translation sub-tasks, we submitted three Neural Machine Translation systems based on Transformer models for English to Malayalam, English to Bengali, and English to Hindi text translation. We found significant results on the leaderboard for English-Indic (en-xx) systems utilizing BLEU and RIBES scores as comparative metrics in our studies. For the respective translations of English to Malayalam, Bengali, and Hindi, we obtained BLEU scores of 19.50, 32.90, and 41.80 for the challenge subset and 30.60, 39.80, and 42.90 on the benchmark evaluation subset data.

## introduction the initial approach used in machine translation was rule-based. rbmt (rule-based machine translation) models use linguistic information about both the source and the target language to generate the translation. platforms such as apertium 1 use this approach. eventually, smt (statistical machine translation) models came about, which did not use a predefined set of rules but inferred the rules by analyzing the given text @xcite . while smt-based models provide more natural translations, rbmt systems provide translations that are truer to the original text @xcite . machine translation (mt) systems have struggled with ambiguity in the source language while 1 https://github.com/apertium translating text, among other challenges. with the advent of deep learning techniques, neural networks are being used for machine translation tasks. neural machine translation (nmt) models use massive amounts of training data and computational power to correctly identify the importance of the portion of the text data to generate the output text @xcite . recent advances in neural machine translation have focused on translating a source language into a specific target language. for this job, several approaches have been offered. early nmt architectures used a fixed length approach to generate variable length outputs. the source text's length was fixed, irrespective of the length of the text. models such as rctm @xcite and rnnencdec @xcite use this approach. eventually, newer architectures began using a variable length representation for the input text. gnmt @xcite and bytenet @xcite are architectures that use layered neural networks for translation @xcite . in this paper, we describe our nmt systems, which were submitted to the translation shared tasks at wat 2022 @xcite . ## related work the majority of nmt research has focused on using monolingual data or parallel data that includes other language pairs. nmt systems have consistently outperformed conventional machine translation methods such as rule-based and statisticalbased approaches. nmt models typically operate with a fixed vocabulary; however, the translation is an open-vocabulary problem. several approaches have been proposed to resolve this issue. byte-pair-encoding (bpe) @xcite enables nmt model translation on open vocabulary by en- nmt models typically employ the conventional sequence-to-sequence learning architecture, made up of an encoder and a decoder. in encoder-decoder mechanisms, words are translated into word embeddings in the encoder and then transferred to the decoder, which generates the following word in the translation using an attention mechanism, encoder representations, and preceding words. several methodologies based on deep neural networks have been proposed, such as recurrent neural networks @xcite , lstm @xcite , convolutional neural networks @xcite , and transformers @xcite , which can serve as encoders and decoders. several approaches have been explored for machine translation in malayalam, bengali and hindi. ## results we have provided a comparative analysis between the effects of using fine-tuned pre-trained models and models trained from scratch. the optimal results on the leaderboard were obtained using the fine-tuned models. ## conclusion in this paper, we discuss the submissions made to three tasks at wat 2022: neural machine translation systems for indic languages. we participated in the text-only subtask of the multimodal translation tasks of english to malayalam, english to bengali, and english to hindi translations. in the future, we would like to experiment with multimodal mt models and incorporate multimodal aspects for the facilitation of better translation systems.