NJU ’s submission to the WMT 20 QE Shared Task (2020)

Abstract: This paper describes our system of the sentence-level and word-level Quality Estimation Shared Task of WMT20. Our system is based on the QE Brain, and we simply enhance it by injecting noise at the target side. And to obtain the deep bi-directional information, we use a masked language model at the target side instead of two single directional decoders. Meanwhile, we try to use the extra QE data from the WMT17 and WMT19 to improve our system’s performance. Finally, we ensemble the features or the results from different models to get our best results. Our system finished fifth in the end at sentence-level on both EN-ZH and EN-DE language pairs.

## introduction quality estimation (qe) is a task to predict the quality of translations without relying on any references. qe plays a critical role in machine translation to reduce human efforts, such as deciding whether a translation is good enough for postediting and indicating what edits are needed. this paper describes our system of the shared task on word and sentence-level (qe tasks 2) at wmt20. with the post-edited translations, all the quality scores can be computed automatically by ter-com @xcite . traditional qe models @xcite use some time-consuming and expensive hand-craft features to represent the translation pairs. with the great success of deep neural networks in natural language processing (nlp), some researches have begun to apply automatic neural features to do qe tasks @xcite . however, the rare qe data can't fully release the power of deep neural networks. to address this problem, researchers try to transfer bilingual knowledge from parallel data to qe tasks @xcite . these works usually follow a predictor-estimator framework @xcite . this framework first trains the predictor to predict each token of the target sentence given the source and the context of the target sentence on parallel data. then, the estimator is trained using the features of qe data produced by the predictor. however, existing predictor-estimator frameworks cannot fully use the information from parallel data because of the discrepancy of data quality between the predictor and the estimator. the predictor is trained on parallel data, which are nearly no errors in translations. while the translations in qe data is generated by a real machine translation system and may have some errors. when the estimator is training on the qe data, the predictor needs to extract the features of translations with some errors, which is quite different from the parallel data. thus, the predictor can't extract features well. to fix this problem, we present two different approaches in this paper. the first model masks some tokens at the target side but still need to predict every token correctly, and it enhances the ability of the model to deal with translations with errors. and to obtain the deep bi-directional information, we use a masked language model at the target side instead of two single directional decoders. meanwhile, we try to use the extra qe data, which are from the wmt17 and wmt19 to improve our system's performance. finally, we ensemble the features or the results from different models to get our best results. our system finished fifth in the end at sentence-level on both en-zh and en-de language pairs of the wmt20 qe shared tasks @xcite . ## methods as we all know, using different sub-models for ensemble will have better results @xcite . we ensemble different methods in our system, some of them are existing methods, and the others are proposed by us. next, we will describe these methods. ## conclusion this paper describes our system of the wmt20 qe shared task. our work mainly follows the qe brain. to bridge the gap between parallel data and qe data, we use a simple way to bring noise into target sentences of parallel data. and to achieve deep bidirectional information, we use a masked language model at the target side. experiments show that our two-step approaches achieve improvements. meanwhile, we try to train our models on more qe data with the same language pair and ensemble different methods through different ways to get our final results.