O dia G en AI ’s Participation at WAT 2023 (2023)

Abstract: This paper offers an in-depth overview of the team “ODIAGEN’s” translation system submitted to the Workshop on Asian Translation (WAT2023). Our focus lies in the domain of Indic Multimodal tasks, specifically targeting English to Hindi, English to Malayalam, and English to Bengali translations. The system uses a state-of-the-art Transformer-based architecture, specifically the NLLB-200 model, fine-tuned with language-specific Visual Genome Datasets. With this robust system, we were able to manage both text-to-text and multimodal translations, demonstrating versatility in handling different translation modes. Our results showcase strong performance across the board, with particularly promising results in the Hindi and Bengali translation tasks. A noteworthy achievement of our system lies in its stellar performance across all text-to-text translation tasks. In the categories of English to Hindi, English to Bengali, and English to Malayalam translations, our system claimed the top positions for both the evaluation and challenge sets. This system not only advances our understanding of the challenges and nuances of Indic language translation but also opens avenues for future research to enhance translation accuracy and performance.

## introduction machine translation (mt) is a well-established field within natural language processing (nlp) that focuses on developing computer software to automatically translate text or speech between different languages. while significant progress has been made in achieving human-level translation for high-resource languages, challenges still remain, especially for low-resource languages @xcite . additionally, recent research has explored the effective integration of other modalities, such as images, into the machine translation process. the wat is an open evaluation campaign focusing on asian languages since 2013 @xcite . the multimodal translation tasks in wat2023 consist of image caption translation, in which the input is a descriptive source language caption together with the image it describes, while the output is a target language caption. the multimodal input enables the use of image context to disambiguate source words with multiple senses. in this system description paper, we (team "odiagen") explains our approach for the tasks (including the sub-tasks) we participated in: we used the datasets specified by the organizer for the related tasks without any additional synthetic data. task 1: @xmath0, the organizers provided hindivisualgenome 1.1 @xcite foot_0 dataset (hvg for short). the training part consists of 29k english and hindi short captions of rectangular areas in photos of various scenes and it is complemented by three test sets: development (d-test), evaluation (e-test) and challenge test set (c-test). our wat submissions were for e-test (denoted "ev" in wat official tables) and c-test (denoted "ch" in wat tables). the statistics of the datasets are shown in task 2: @xmath1, the organizers provided malayalamvisualgenome 1.0 dataset foot_1 (mvg for short). mvg is an extension of the hvg dataset for supporting malayalam, which belongs to the dravidian language family @xcite . the dataset size and images are the same as hvg. while hvg contains bilingual english-hindi segments, mvg contains bilingual english-malayalam segments, with the english, shared across hvg and mvg, see task 3: @xmath2, the organizers provided bengalivisualgenome 1.0 dataset foot_2 (bvg for short). bvg is an extension of the hvg dataset for supporting bengali. the dataset size and images are the same as hvg, and mvg, see ## experimental details this section describes the experimental details of the tasks we participated in. . we leverage the hugging face's transformers library, specifically using the automodelforseq2seqlm class for the model architecture as shown in figure 1 . the pipeline shown in figure 2 includes several distinct steps: ## conclusion in this system description paper, we presented our system for three tasks in wat2023: (a) @xmath3, (b) @xmath4, and (c) @xmath5. we released the code through github for research foot_3 . these empirical results underscore the effectiveness of the methodology adopted for these machine translation models. leveraging a fine-tuned nllb-200 model with language-specific visual genome datasets provides a robust solution to the machine translation task for the languages under study: hindi, bengali, and malayalam. the results also pave the way for further enhancements and investigations in the realm of machine translation.