Test-time Adaptation for Machine Translation Evaluation by Uncertainty Minimization (2023)

Abstract: The neural metrics recently received considerable attention from the research community in the automatic evaluation of machine translation. Unlike text-based metrics that have interpretable and consistent evaluation mechanisms for various data sources, the reliability of neural metrics in assessing out-of-distribution data remains a concern due to the disparity between training data and real-world data. This paper aims to address the inference bias of neural metrics through uncertainty minimization during test time, without requiring additional data. Our proposed method comprises three steps: uncertainty estimation, test-time adaptation, and inference. Specifically, the model employs the prediction uncertainty of the current data as a signal to update a small fraction of parameters during test time and subsequently refine the prediction through optimization. To validate our approach, we apply the proposed method to three representative models and conduct experiments on the WMT21 benchmarks. The results obtained from both in-domain and out-of-distribution evaluations consistently demonstrate improvements in correlation performance across different models. Furthermore, we provide evidence that the proposed method effectively reduces model uncertainty. The code is publicly available at https://github.com/NLP2CT/TaU .

## introduction the evaluation of machine translation (mt) systems aims to quantitatively assess their performance using either automatic metrics or human evaluators. when developing cutting-edge mt systems, selecting the optimal model using automatic metrics is highly significant to save human labor, given a large number of candidate models. over the last decade, the researchers have primarily relied on traditional metrics based on text overlap (papineni et @xcite @xcite to evaluate system performance. however, these metrics fall short in capturing semantic-level information and exhibit poor correlation with human ratings when assessing the latest neural mt systems because of increased model capacity @xcite . consequently, several neural metrics @xcite @xcite @xcite and test sets @xcite @xcite have been proposed to provide broader evaluation perspectives and show outstanding performance in evaluating state-of-the-art systems. despite the superiority of neural metrics, the adoption of these metrics over traditional overlap-based measures has witnessed a gradual pace. the people engaged in mt research and industry remain cautious due to concerns surrounding potential robustness issues, thereby hindering the progress of popularizing neural metrics. the source of robustness problem can be attributed to data shift. the fine-tuning data used when developing neural metrics is composed of labels derived from human ratings obtained when evaluating strong mt systems in the news domain, which largely limits the generalization capability of the obtained model. in real-world scenarios, the evaluation metric must be capable of assessing text originating from diverse domains with varying levels of quality. however, neural metrics, trained on limited data, may exhibit biases when dealing with out-of-distribution data. these factors present challenges in establishing neural metrics as reliable evaluation measures across a wide range of applications. @xcite proposed employing uncertainty quantification @xcite to assess the risk associated with utilizing neural metrics in evaluation and discovered a correlation between model uncertainty and model prediction errors, as depicted in figure 1 . @xcite have explored the uncertainty of neural metrics, the quest for a solution to mitigate uncertainty in mt evaluation remains an under-explored research area. one intuitive approach is fine-tuning the model using diverse and multi-domain data. unfortunately, there is currently no publicly available dataset that satisfies this requirement. in this paper, we propose an unsupervised approach for neural metrics aimed at minimizing uncertainty during test time and mitigating the challenges posed by out-of-distribution data. our proposed method involves two additional stages integrated before the normal inference process: uncertainty estimation and test-time adaptation. firstly, our model leverages the monte carlo approach @xcite to estimate the uncertainty of the current input data. subsequently, the estimated uncertainty serves as a guiding signal to optimize a small fraction of model parameters using gradient descent. finally, the model proceeds with the regular inference procedure, utilizing the adapted parameters to make predictions. in this way, the model can adjust its parameters dynamically to better cope with diverse data, which is flexible and does not require any labeled data. we use the representative metric family comet @xcite as our testbed and conduct experiments on wmt21 benchmark @xcite , which accounts for evaluating outof-distribution data. the experimental results show that our method can improve the system-level correlation performance as well as the ranking accuracy of partial comet baselines. furthermore, our analysis highlights the applicability of our method and confirms its efficacy in reducing uncertainty. there are three types of metrics based on their utilization of reference information: reference-based metric m(⟨h, •, r⟩) (which solely utilize the target translation or jointly consider both the source and target information), and reference-free metric m(⟨h, s⟩) (which solely rely on the source input). among these, reference-based metrics are widely employed, and reference-free metrics are often categorized as quality estimation metrics @xcite . the neural metrics build a regression scoring model by leveraging pre-trained representation, which have achieved remarkable performance in mt evaluation. in this way, the metric m is parameterized by model θ: @xmath0, s, r⟩; θ) as an example, the comet @xcite ) framework employs two distinct downstream architectures to leverage a pre-trained xlm (conneau et al., 2020) model. it fine-tunes the additional regression and ranking models using human rating data obtained from the wmt metrics task, ensuring that the tuned parameters can evaluate the translation quality. uncertainty as deep neural networks are widely used in real-world applications, uncertainty is a critical measurement that indicates how a model is confident in the predictions in order to prevent causing serious consequences such as gender bias @xcite . there are two kinds of uncertainty proposed by previous research: aleatoric uncertainty and epistemic uncertainty @xcite . while aleatoric uncertainty pertains to data noise in observations and cannot be easily eliminated, epistemic uncertainty stems from the insufficient knowledge of a model. given that the training data for neural metrics primarily revolves around the news domain, this paper focuses on reducing epistemic uncertainty, particularly for out-of-distribution data. test-time adaptation domain adaptation (pan and yang, 2010) offers a deterministic target and can be trained with additional data through supervised or unsupervised methods, providing an intuitive approach to reduce epistemic uncertainty. however, there is a dearth of research exploring domain adaptation in mt evaluation due to the scarcity of multi-domain human ratings. another limitation of using domain adaptation methods to mitigate epistemic uncertainty is the unknown domain of input data in real-world scenarios. this becomes particularly crucial for neural metrics, as they need to score diverse inputs without introducing domain bias. test-time adaptation paradigm handles this challenge as a viable solution and can be categorized into test-time training @xcite and source-free test-time adaptation @xcite @xcite . it generalizes the model to out-of-distribution data during the testing phase without necessitating additional fine-tuning operations. notably, a con-current work @xcite in the image classification tasks has also proposed minimizing uncertainty during test time. however, there are notable distinctions between our approach and theirs in terms of learning objectives and the specific type of uncertainty being targeted. in the context of mt evaluation, we present the first application of this paradigm and contribute a novel method that minimizes epistemic uncertainty at test time. ## method the proposed method, as illustrated in figure 2 , is comprised of three distinct stages. these stages will be thoroughly discussed in the following section. since both reference-based regression model and quality estimation (qe) model are used in comet framework, we use ⟨h, s, •⟩ to denote input data in order to take two major types of metrics mentioned in section 2. ## analysis in this section, we will discuss the effectiveness of our method by answering three questions: 1) how do different optimization settings impact performance? 2) when does test-time adaptation work? 3) can the proposed method effectively reduce epistemic uncertainty? among these questions, the last one serves to justify our research objective and entails a segment-level analysis to understand why ## conclusion the uncertainty of neural metrics is proven to be associated with prediction error and limits generalizing them for a wider range of applications. in this paper, we propose a novel method, tau, to minimize the uncertainty of neural metrics at test time in unsupervised settings without learning extra data. our experimental results showcase the efficacy of tau in reducing test-time uncertainty while simultaneously improving the performance of widely used metrics. in addition, our findings indicate that the proposed method exhibits enhanced effectiveness when applied to out-of-distribution data in comparison to in-domain data, which lays a solid foundation for its potential application to other models. however, the segment-level performance does not significantly outperform the baselines. in the future, we will polish the methods for better segment-level correlation performance and explore the test-time adaptation on large language models across various tasks.