Findings of the Word-Level A uto C ompletion Shared Task in WMT 2023 (2023)

Abstract: This paper presents the overview of the second Word-Level autocompletion (WLAC) shared task for computer-aided translation, which aims to automatically complete a target word given a translation context including a human typed character sequence. We largely adhere to the settings of the previous round of the shared task, but with two main differences: 1) The typed character sequence is obtained from the typing process of human translators to demonstrate system performance under real-world scenarios when preparing some type of testing examples; 2) We conduct a thorough analysis on the results of the submitted systems from three perspectives. From the experimental results, we observe that translation tasks are helpful to improve the performance of WLAC models. Additionally, our further analysis shows that the semantic error accounts for a significant portion of all errors, and thus it would be promising to take this type of errors into account in future.

## introduction computer-aided translation (cat) helps human translators produce high-quality translations with the assistance of machine translation systems @xcite , and it has witnessed a lot of attention during the past decades @xcite @xcite @xcite @xcite @xcite . among all the tasks in cat, word-level autocompletion (wlac) is one of the most fundamental tasks and its goal is to autocomplete a word when a human translator types a sequence of characters @xcite , in order to accelerate the editing process for human translators under cat settings. to facilitate the research in wlac, the first wlac shared task was held in wmt 2022 @xcite @xcite @xcite . this year, we continue holding the second edition of wlac shared task in wmt 2023. in this paper, we summarize the overview of the wlac shared task in wmt 2023, which is named by wlac 2023 for brevity, including data preparation process, submitted systems and their evaluation results. specifically, wlac 2023 involves two language pairs, i.e. chinese-english and german-english, and contains four directional sub-tasks in total, similar to wlac 2022 shared task. for training data preparation, we follow the common practice of leveraging a bilingual corpus for simulation. for test data preparation, however, there is one important difference in this year to make the test data more similar to realistic scenarios: for some testing examples (see §2.2), their typed character sequences are obtained from the typing process of human translators. we have received twenty-one submissions in total from four teams in wlac 2023. we evaluate all these submissions and present their overall evaluation results. in particular, we conduct a thorough analysis of submitted systems to better understand the challenges and difficulties emerged in wlac tasks. the analysis of these systems is investigated according to three perspectives which include the frequency of target words, the size of context, as well as the human defined error types. from all the perspectives, we observe some insights which might be useful for further improvement on wlac in future. in summary, our main findings are highlighted as follows: ## conclusion this paper presents the overview for the shared task of word-level auto-completion, which is the key component of computer-aided translation. we describe the task definition, data preparation process, the submitted systems, evaluation metric and evaluation results of the systems. we have received twenty-one submissions from four participants this year. we report the evaluation results of all systems, conduct a thorough analysis on the prediction results of these systems and obtain some insightful findings. we hope that our findings can encourage the emerge of more powerful models and attract more researchers to participate the study of computer-aided translation.