HW - TSC 2023 Submission for the Quality Estimation Shared Task (2023)

Abstract: Quality estimation (QE) is an essential technique to assess machine translation quality without reference translations. In this paper, we focus on Huawei Translation Services Center’s (HW-TSC’s) submission to the sentence-level QE shared task, named Ensemble-CrossQE. Our system uses CrossQE, the same model architecture as our last year’s submission, which consists of a multilingual base model and a task-specific downstream layer. The input is the concatenation of the source and the translated sentences. To enhance the performance, we finetuned and ensembled multiple base models such as XLM-R, InfoXLM, RemBERT and CometKiwi. Moreover, we introduce a new corruption-based data augmentation method, which generates deletion, substitution and insertion errors in the original translation and uses a reference-based QE model to obtain pseudo scores. Results show that our system achieves impressive performance on sentence-level QE test sets and ranked the first place for three language pairs: English-Hindi, English-Tamil and English-Telegu. In addition, we participated in the error span detection task. The submitted model outperforms the baseline on Chinese-English and Hebrew-English language pairs.

## introduction quality estimation (qe) involves automatically scoring machine translation outputs without depending on reference translations @xcite . in the wmt 2023 qe shared task, there are two subtasks -quality estimation and finegrained error span detection and each task involves several language pairs. our team -huawei translation services center (hw-tsc) -participated in the sentence-level quality prediction and the finegrained error span detection tasks over all language pairs except the zero-shot language pair. finetuning pre-trained language models, which offers abundant semantic information, has become the paradigm for qe tasks @xcite . in this paper, we describe ensemble-crossqe, hw-tsc's system for sentence-level qe task, which leverages multiple pre-trained language models and data augmentation technique. our system designs can be summarized as follow: our system achieves remarkable results and outperforms the baseline given by the competition organizer by a large margin. additionally, we provide detailed results of each model with and without data augmentation in ## conclusion this paper mainly presents hw-tsc's sentencelevel qe system called ensemble-crossqe. using our previous year's model crossqe as the foundation, we carried out comprehensive experiments with different pre-trained models. to further improve the robustness for low-resource language pairs and provide various checkpoints for model