Overview of the 8th Workshop on A sian Translation (2021)

Abstract: This paper presents the results of the shared tasks from the 8th workshop on Asian translation (WAT2021). For the WAT2021, 28 teams participated in the shared tasks and 24 teams submitted their translation results for the human evaluation. We also accepted 5 research papers. About 2,100 translation results were submitted to the automatic evaluation server, and selected submissions were manually evaluated.

## introduction the workshop on asian translation (wat) is an open evaluation campaign focusing on asian languages. following the success of the previous workshops wat2014-wat2020 (nakazawa et al., 2020), wat2021 brings together machine translation researchers and users to try, evaluate, share and discuss brand-new ideas for machine translation. we have been working toward practical use of machine translation among all asian countries. for the 8th wat, we included the following new tasks: all the tasks are explained in section 2. wat is a unique workshop on asian language translation with the following characteristics: ## participants there were no participants in the newswire (jiji) task, bsd task and jarunc task. ## baseline systems human evaluations of most of wat tasks were conducted as pairwise comparisons between the translation results for a specific baseline system and translation results for each participant's system. that is, the specific baseline system served as the standard for human evaluation. at wat 2021, we adopted some of neural machine translation (nmt) as baseline systems. the details of the nmt baseline systems are described in this section. the nmt baseline systems consisted of publicly available software, and the procedures for building the systems and for translating using the systems were published on the wat web page. @xcite we also have smt baseline systems for the tasks that started at wat 2017 or before 2017. the baseline systems are shown in tables 15 , 16 , and 17. smt baseline systems are described in the wat 2017 overview paper (nakazawa et al., 2017). the commercial rbmt systems and the online translation systems were operated by the organizers. we note that these rbmt companies and online translation companies did not submit their systems. because our objective is not to compare commercial rbmt systems or online translation systems from companies that did not themselves participate, the system ids of these systems are anonymous in this paper. ## human evaluation in wat2021, we conducted jpo adequacy evaluation (other than en-hi and en-ml multi-modal task, section 6.1). ## evaluation results in this section, the evaluation results for wat2021 are reported from several perspectives. some of the results for both automatic and human evaluations are also accessible at the wat2021 web-site. foot_35 ## conclusion and future perspective this paper summarizes the shared tasks of wat2021. we had 24 participants worldwide who submitted their translation results for the human evaluation, and collected a large number of useful submissions for improving the current machine translation systems by analyzing the submissions and identifying the issues. for the next wat workshop, we will try to add more indic languages to our multiindicmt task along with newer evaluation sets. also, we will add a new @xmath0.