Overview of the 10th Workshop on A sian Translation (2023)

Abstract: This paper presents the results of the shared tasks from the 10th workshop on Asian translation (WAT2023). For the WAT2023, 2 teams submitted their translation results for the human evaluation. We also accepted 1 research paper. About 40 translation results were submitted to the automatic evaluation server, and selected submissions were manually evaluated.

## introduction the workshop on asian translation (wat) is an open evaluation campaign focusing on asian languages. following the success of the previous workshops wat2014- @xcite , wat2023 brings together machine translation researchers and users to try, evaluate, share and discuss brand-new ideas for machine translation. we have been working toward practical use of machine translation among all asian countries. for the 10th wat, we included the following new tasks/languages: all the tasks are explained in section 2. wat is a unique workshop on asian language translation with the following characteristics: ## participants ## baseline systems human evaluations of most of wat tasks were conducted as pairwise comparisons between the translation results for a specific baseline system and translation results for each participant's system. that is, the specific baseline system served as the standard for human evaluation. at wat 2023, we adopted some of neural machine translation (nmt) as baseline systems. the details of the nmt baseline systems are described in this section. the nmt baseline systems consisted of publicly available software, and the procedures for building the systems and for translating using the systems were published on the wat web page. we also have smt baseline systems for the tasks that started at wat 2017 or before 2017. smt baseline systems are described in the wat 2017 overview paper @xcite . the commercial rbmt systems and the online translation systems were operated by the organizers. we note that these rbmt companies and online translation companies did not submit their systems. because our objective is not to compare commercial rbmt systems or online translation systems from companies that did not themselves participate, the system ids of these systems are anonymous in this paper. ## evaluation results in this section, the evaluation results for wat2023 are reported from several perspectives. some of the results for both automatic and human evaluations are also accessible at the wat2023 website. 20 8 findings 8.1 @xmath0, mm) of the @xmath1. the wat2023 automatic evaluation scores for the participating teams are shown in tables 11 , 14 , 17 and 20. for the text-only sub-task (text), one team "odiagen" participated in the evaluation (e-test) and challenge (c-test) set by fine-tuning the transformer model using nllb-200 from facebook. their scores were outperformed in comparison to all previous years' submissions. it is worth mentioning, they did not use any additional resources. for the multimodal sub-task (mm), we received two submissions from the teams "odia-gen", and "bits-p", respectively. the team "bits-p" obtained a bleu score of 45.00 for the evaluation (e-test) by nllb model finetuning on captions with object tags of original and synthetic images using detr model. the team "bits-p" used additional resources for their model building. the team "odiagen" obtained bleu score of 41.60 by using image features (extracting object tags) appended with text and mbart finetuning. for the challenge (c-test) set, both teams ("bits-p", and "odiagen") obtained bleu scores of 52.10 and 42.80 respectively following the same approaches as in e-test. human evaluation was done for the challenge test set multimodal translation (mm) as shown in figure 6 . ## conclusion and future perspective this paper summarizes the shared tasks of wat2023. this year, we had 2 participants who submitted their translation results. both teams participated to the indic multimodal translation tasks. this year we had smaller number of participants compared to the previous years. for the next wat workshop, we want attract much more people to join our shared tasks.