ODIANLP ’s Participation in WAT 2020 (2020)

Abstract: This paper describes the ODIANLP submission to WAT 2020. We have participated in the English-Hindi Multimodal task and Indic task. We have used the state-of-the-art Transformer model for the translation task and InceptionResNetV2 for the Hindi Image Captioning task. Our submission tops in English->Hindi Multimodal task in its track and Odia<->English translation tasks. Also, our submissions performed well in the Indic Multilingual tasks.

## introduction although machine translation (mt) has proven very successful for many high resource languages, it is still challenging for low resource languages and translation effectively utilizing other modalities (e.g. image; @xcite . workshop on asian translation (wat) 1 is a recurring event focusing on the asian languages @xcite since 2013. this year, wat introduced the translation task for one of the low resource indian languages, odia. 2 odia is 1 http://lotus.kuee.kyoto-u.ac.jp/wat/ 2 https://www.britannica.com/topic/ odia-language nowadays spoken by 50 million speakers. it is heavily influenced by the dravidian languages as well as arabic, persian, and english. odia's inflectional morphology is rich with a three-tier tense system. the prototypical word order is subjectobject-verb (sov) @xcite . in this system description paper, we explain our approach for the participated tasks. section 2 describes the datasets used in our experiment. section 3 presents the model and experimental setups used in our approach. section 5 provides the official evaluation results of wat2020 foot_0 followed by the conclusion in section 6. ## dataset we have used the official datasets provided by the wat2020 organizers for the tasks and also used additional datasets recommended by the organizers. task 1: @xmath0, the organizers provided hindi-visualgenome 1.1 @xcite foot_1 dataset (hvg for short). the training part consists of tokens set sentences english hindi hvg train 28930 143164 145448 iitb train 1.5 m 20.6 m 22.1 m d-test 998 4922 4978 e-test 1595 7853 7852 c-test 1400 8186 8639 task 2: indic odia↔english translation for this task, the organizers provided odiencorp 2.0 @xcite . 6 to train the model, we used an additional dataset (odienmonocorp foot_4 ) suggested by the organizers @xcite . the statistics of the datasets are shown in task 3: indic multilingual translation for this task, the organizers provided filtered data of the pmindia dataset @xcite ). 8 we have not used any additional resources in this task. the statistics of the dataset are shown in ## experiment we focussed only on the text translation and image captioning task. in the @xmath2, for the 'text-only' subtask, we used the transformer model @xcite which is popular for machine translation and other text-processing tasks, such as low resource text summarization @xcite . we have used the transformer model as implemented in opennmtpy @xcite . 9 we used inceptionres-netv2 foot_7 for 'hindi-only' image captioning subtask. in the odia↔english translation task, we used the transformer model as implemented in opennmt-tf. 11 in the indic multilingual translation task, we used the transformer (big) model with relative position representations @xcite as implemented in opennmt-tf @xcite . ## official results we report the official automatic and human evaluation results of our models for all the participated tasks here in ## discussion we have analyzed the result and report some observations based on the automatic evaluation scores. for the @xmath3, our text-only submission (with an additional resource for training) obtains the best result compared to utilizing both text and image (by competitors in the multimodal track). a sample output generated from our model is shown in figure 3 . although our nmt model able to translate many ambiguous words (e.g. cross) in many instances, still it fails in some instances which could be resolved using an image as shown in figure 3 . our submission to the image captioning task ('hindi only', denoted with 'mm*hi20en-hi' in bleu scores for the @xmath4, but this does not necessarily indicate that the translation into english would be better, because crosslanguage comparison of bleu scores is generally not possible. the situation is different in the indic multilingual task ( ## conclusion and future scope in this system description paper, we presented our systems for three tasks in wat 2020 in which we participated: i) @xmath5, ii) indic odia↔english, and iii) indic multilingual translation task. as the next steps, i) we plan to explore more on the indic multilingual task utilizing additional resources for training, ii) analyze the image captioning task which didn't work, and iii) utilize image features for improving the translation quality.