C ross QE : HW - TSC 2022 Submission for the Quality Estimation Shared Task (2022)

Abstract: Quality estimation (QE) is a crucial method to investigate automatic methods for estimating the quality of machine translation results without reference translations. This paper presents Huawei Translation Services Center’s (HW-TSC’s) work called CrossQE in WMT 2022 QE shared tasks 1 and 2, namely sentence- and word- level quality prediction and explainable QE.CrossQE employes the framework of predictor-estimator for task 1, concretely with a pre-trained cross-lingual XLM-RoBERTa large as predictor and task-specific classifier or regressor as estimator. An extensive set of experimental results show that after adding bottleneck adapter layer, mean teacher loss, masked language modeling task loss and MC dropout methods in CrossQE, the performance has improved to a certain extent. For task 2, CrossQE calculated the cosine similarity between each word feature in the target and each word feature in the source by task 1 sentence-level QE system’s predictor, and used the inverse value of maximum similarity between each word in the target and the source as the word translation error risk value. Moreover, CrossQE has outstanding performance on QE test sets of WMT 2022.

## introduction quality estimation (qe) is the task of evaluating a translation system's quality without access to reference translations @xcite . in wmt 2022 qe shared task 1 , there are three tasks -quality prediction, explainable qe and critical error detection. each task involves several language pairs. our team -huawei translation services center (hw-tsc) -participated in quality prediction and explainable qe tasks over all language pairs. this paper describes the hw-tsc's systems called crossqe submitted for these tasks. some key steps are summarized as follow: our methods achieve impressive performance on both sentence-and word-level tasks. specifically, we peak the top-1 on quality prediction sentencelevel sub-task over chinese-english language pair and word-level sub-task over english-japanese language pair. we also win the first place in explainable qe task in khmer-english and pashto-english language pairs. we will describe the datasets and our methods for those tasks in section 2 and section 3. section 4 presents details of our experimental setup and results. in section 5, a brief discussion and conclusion are presented.