Faithful Model Evaluation for Model-Based Metrics (2023)

Abstract: Statistical significance testing is used in natural language processing (NLP) to determine whether the results of a study or experiment are likely to be due to chance or if they reflect a genuine relationship. A key step in significance testing is the estimation of confidence interval which is a function of sample variance. Sample variance calculation is straightforward when evaluating against ground truth. However, in many cases, a metric model is often used for evaluation. For example, to compare toxicity of two large language models, a toxicity classifier is used for evaluation. Existing works usually do not consider the variance change due to metric model errors, which can lead to wrong conclusions. In this work, we establish the mathematical foundation of significance testing for model-based metrics. With experiments on public benchmark datasets and a production system, we show that considering metric model errors to calculate sample variances for model-based metrics changes the conclusions in certain experiments.

## introduction in the field of natural language processing (nlp), continuous progress hinges upon the development of novel techniques that outperform existing ones. however, accurately assessing the effectiveness of these new techniques requires a comprehensive evaluation framework. model evaluation serves as the foundation for assessing the performance and impact of nlp advancements. significance testing is a crucial tool in the evaluation process, enabling us to derive accurate conclusions. it allows us to determine whether the obtained evaluation results hold significance or are merely coincidental. as the cost of human annotation for evaluating models using deterministic metrics is substantial, there is a growing trend towards utilizing modelbased metrics for evaluation purposes. model-* these authors contributed equally to this work. we conduct several experiments on using model-based metrics including hate speech detection @xcite and user perceived defects detection. the experimental results show that considering prediction errors in significance testing changes the conclusions in certain experiments. thus, we propose that the research community utilizes our framework for performing statistical testing with model-based metrics. in the following sections, we derive the mathematical equations for significance testing for model-based metrics and conduct experiments on several public benchmark datasets and a production system. finally, we discuss related works and draw final conclusion. ## model evaluation with model-based metrics in this section, we provide background on significance testing and then derive how we can modify it to incorporate model's prediction errors when using model-based metrics. ## experiments we perform several experiments on public benchmark datasets and a production system to validate the proposed framework. in this section, we first introduce the experimental details on public benchmark datasets and then describe the experiments on the production system. finally, we report the experimental results and analysis. ## related work in their paper, dror et al. @xcite established fundamental concepts of significance testing in nlp research and proposed a simple practical protocol for statistical significance test selection in nlp setups. berg-kirkpatrick et al. (berg-kirkpatrick et al., 2012) investigate the relation between significance level and the magnitude of the gain. they also studied how the standard i.i.d. notion of significance hold up when there is domain shift. with the increasing of evaluation datasets, it is becoming common to evaluate nlp models on multiple datasets. such multiple comparison poses challenges for significance testing in nlp. dror et al. @xcite proposes replicability analysis framework for the analysis of multiple comparisons between nlp algorithms. in their book, @xcite discuss the opportunities and challenges of using significance testing in nlp. the book covers multiple topics including choosing the appropriate significance test for a nlp task, dealing with the uniqueness of significance testing for non-convex deep neural networks, etc. the current works on significance testing in nlp do not cover the area of model-based metrics. in this work, we propose to consider model prediction errors for more faithful evaluation and reaching the right conclusion when conducting significance testing for model-based metrics. ## conclusion significance testing is an important tool for us to draw accurate conclusions for evaluating nlp models. existing evaluation works using model-based metrics do not consider model variance for significance testing, which can lead to wrong conclusions. in this work, we lay the mathematical foundation of significance testing for model-based metrics. we conduct experiments on public benchmarks and a production system. the significance testing results on these experiments show that model based errors need to be considered and incorporated for accurate evaluation. in this work, we focus primarily on computing confidence interval with model-based metrics which use binary classification. in the future, we plan to extend our work to more general types of model-based metrics. further, we assumed that the samples are independent and identically distributed. in practice, we often have a score associated with the metric model which can be used to relax this assumption. we leave this as future work.