Characterizing the Confidence of Large Language Model-Based Automatic Evaluation Metrics (2024)

Abstract: There has recently been a growing interest in using Large Language Models (LLMs) to evaluate NLP tasks automatically. Considerable research effort has been put into improving such systems towards achieving high correlations with human judgement. However, it is still unclear what level of correlation is good enough for practical applications of LLM-based automatic evaluation systems. This paper characterizes these LLM evaluators’ confidence in ranking candidate NLP models and develops a configurable Monte Carlo simulation method. We show that even automatic metrics with low correlation with human judgement can reach high-confidence rankings of candidate models with reasonable evaluation set sizes (100s of examples). Further, we describe tradeoff curves between the LLM evaluator performance (i.e., correlation with humans) and evaluation set size; loss in correlation can be compensated with modest increases in the evaluation set size. We validate our results on RoSE, a text summarization dataset, and find our estimates of confidence align with empirical observations.Code available at https://github.com/rickardstureborg/llm-eval-confidence

## introduction automatic evaluation is a staple of natural language processing (nlp) tasks, from the popular rouge score in text summarization to bleu score in machine translation. these metrics often rely on human-written references, increasing the cost and effort of evaluation. recently, large language models (llms) have become commonly used evaluators because of their zero-shot capability in understanding the quality of texts @xcite . these methods, which we refer to as llm-based automatic evaluation metrics (or here, candidate models produce responses for the same evaluation set t of input prompts, and the mean scores by an llm-based automatic evaluation metric are compared. suppose that s a is mv-bart and s b is pegasus foot_1 , and the evaluation set is each of their responses across 100 prompts. our results indicate llm evaluators would require approximately @xmath0.42 correlation with human judgement to reach a 95% confidence in selecting the correct ranking. llm evaluators, in short), do not require humanwritten reference outputs and can be reconfigured to new tasks quickly. the goal of these automatic metrics is to replace human annotators in evaluating systems since human annotation is often expensive, slow, and difficult to manage @xcite . as such, well-performing auto-evaluators are the ones that correlate highly with human judgements. however, relying on llm evaluators as a replacement for human judgement comes with familiar challenges @xcite . since the metrics are not perfect, we may want to further assess not just the correlation they have with human judgements (and our confidence in that value), but also the confidence one can have in downstream decisions we make using the metrics. a common use of llm evaluators are to compare new, specialized natural language gener-ation (nlg) models for a task. how confident can we be that the ranking given by an llm evaluator is the same as the ranking human evaluators would give? estimating this quantity is difficult, since it depends on many factors (evaluation set size, the evaluator's correlation with human labels, the magnitude of candidate models' performance gap, etc.), and available datasets often only compare a handful of candidate models at a time. the research question we tackle in this paper is to characterize how likely (how confident) llm evaluators are to predict correct pairwise rankings among candidate models. to estimate confidence, our work suggest a configurable monte carlo simulation developed based on empirical observations about llm evaluators. we explore possible tradeoffs between factors affecting this confidence which could save on inference costs. ## methodology the task we are interested in solving is to quantify the confidence in an llm evaluator's decision when ranking two candidate models against each other. figure 1 shows a useful diagram of how we compare two models or systems against one another. appendix a includes a formal description of this task. ## results and discussion the adapted g-eval-3.5 system performs much worse on acu score than on the summeval labels it was evaluated on, indicating the potential dif-ficulty of adapting llm evaluators to new tasks. the correlation with human judgements is given in in figure 4 and appendix c.2 we show the relationship between n and r with confidence. as expected, larger evaluation set sizes and higher correlation with human judgement both lead to greater confidence. our simulation results indicate that llm evaluators are able to reach fairly high agreement despite low correlation. with an evaluation set of only 100 examples, models with just 0.2 correlation are able to correctly rank a 0.10 acu difference with ≈ 80% confidence. however, it should be noted that a difference of 0.10 acu is substantial (10% of the entire range for the metric). therefore there remains much room for improvement by llm-based automatic evaluation metrics to discern nuanced performance differences with efficient evaluation set sizes. next we characterize the tradeoff between evaluation set size and correlation in figure 3 . we note that the tradeoff is steep with respect to correlation around the current performance of sota llm evaluators (0.40-0.50). for relatively small increases in n one can trade away substantial correlation performance (which is much harder to come by than extra evaluation examples). 0.0 0.1 0.2 0.3 0.4 0.5 difference in expected model score (normalized acu) 0.60 0.65 0.70 0.75 0.80 0.85 0.90 0.95 1.00 probability of correct ranking @xmath1.19 @xmath2: effect on confidence of increasing the evaluation set size n . the plot shows confidence of an llm-based automatic metric in ranking two summarization candidate models given their true expected quality differences. as an evaluation of the simulations estimates of confidence, we compare our results to empirical observations from bootstrap sampling g-eval-3.5 predicted scores in figure 5 . samsum and xsum serve as out-of-domain test sets given that we trained our simulation on the validation set of cnndm. each point shows the probability of correctly ranking two candidate models using an llmbased automatic evaluation metric. generally, the empirical observations seem to match the simulated results well, although there is larger variance in the empirical results than the simulated, again highlighting the difficulty of the task. samsum serves as an out-of-domain test set since the monte carlo simulation was configured with data from the cnndm validation set only, including the correlation value. predictions by the g-eval were sampled with replacement and final datapoints were averaged into buckets of true acu differences between the models being ranked. ## related work automatic evaluation metrics are well established entities of nlp have been the subject of substantial research efforts. while many historical automatic metrics have been based on reference texts, some work @xcite investigates metrics that score generations without any human-written references. @xcite investigates the use of neural frameworks for automatic evaluation. @xcite investigates the accuracy of rouge 1 and 2 scores in comparing summarization systems. their work tries to identify the best metrics by ignoring system comparisons which have insignificant differences in performance. their work does not consider more modern metrics such as llm-based evaluators, nor quantifying the confidence in a given model ranking. @xcite investigate the statistical power of their dataset as a function of sample size, and analyze how metrics like rouge score's power compares to their proposed metric. this is a helpful analysis of their dataset's utility and the value of acu as a metric, which we extend by explicitly investigating how likely an llm-based evaluation system is to correctly rank models according to this metric. @xcite look at automatic metrics for machine translation, examining how reliable such metrics are as compared with human judgements when ranking machine translation systems in pairs. @xcite explore how precise estimates of correlation with human judgement are and find that confidence intervals of these reported correlations are wide. their work focuses on many classic automatic evaluation metrics such as rouge and qaeval, but does not include newer llm-based automatic metrics nor investigations of using the metrics to rank candidate models. similarly, @xcite build a bootstrapping method for estimating confidence intervals of bleu/nist scores, and describe the effect of evaluation set size and number of reference translations on the confidence intervals of system-wide bleu scores. these works do not investigate the tradeoffs between factors influencing confidence and cost. ## conclusion our work investigates the confidence of llm evaluators in making downstream decisions by proposing a configurable monte carlo simulation. we show that even automatic metrics with low correlation to human judgement can reach highconfidence rankings of candidate models with modest evaluation set sizes (100s of examples). we also describe the exact tradeoff curves between this correlation and evaluation set size, so that cost of running inference can be minimized without sacrificing confidence. our methods are validated by empirical observations on rose.