Likelihood-based Mitigation of Evaluation Bias in Large Language Models (2024)

Abstract: Large Language Models (LLMs) are widely used to evaluate natural language generation tasks as automated metrics.However, the likelihood, a measure of LLM’s plausibility for a sentence, can vary due to superficial differences in sentences, such as word order and sentence structure.It is therefore possible that there might be a likelihood bias if LLMs are used for evaluation: they might overrate sentences with higher likelihoods while underrating those with lower likelihoods.In this paper, we investigate the presence and impact of likelihood bias in LLM-based evaluators.We also propose a method to mitigate the likelihood bias.Our method utilizes highly biased instances as few-shot examples for in-context learning.Our experiments in evaluating the data-to-text and grammatical error correction tasks reveal that several LLMs we test display a likelihood bias.Furthermore, our proposed method successfully mitigates this bias, also improving evaluation performance (in terms of correlation of models with human scores) significantly.

## introduction large language models (llms) exhibit robust language comprehension and text generation capabilities @xcite . relying on this ability, recent studies @xcite @xcite have employed llms as evaluators for natural language generation tasks, surpassing the performance of existing automatic evaluation methods such as bleu @xcite and rouge @xcite . to assess the quality of a text, llms output a comprehensive evaluation score based on criteria such as fluency and meaning. llms generate a text based on the likelihood estimations derived from the training process that aims to maximize the likelihood of their large-scale training data. consequently, it is intuitively possible that the likelihood of a text influences the generation of its evaluation score. however, the likelihood estimations by llms may not necessarily align with the quality of the text. for instance, the likelihood calculated by the llm fluctuates due to superficial differences in sentences, such as word order and sentence structure, even for sentences with identical meaning @xcite . such fluctuation of likelihood could negatively impact the evaluation score based on meaning criteria. in this paper, we introduce likelihood bias, where llm-based evaluators overrate highlikelihood sentences and underrate low-likelihood ones compared to human scores. figure 1 shows one example of likelihood bias. here, a biased evaluator gives a lower score of 3/5 to a correct but low-likelihood output (top) while giving a higher score of 5/5 to a high-likelihood output (bottom), based on the criteria of data coverage. addressing this issue, we propose a method that a) quantifies and b) mitigates likelihood bias. we quantify the bias by using the correlation between the disparity in evaluation scores generated by llms and those provided by human evaluators, and the likelihood of a target text. our bias reduction method identifies and utilizes highly biased instances as few-shot examples for in-context learning. the extent of likelihood bias may vary with the evaluation criteria used. for instance, likelihood bias is anticipated to be more pronounced in criteria like data coverage, which is less directly related to likelihood. conversely, the bias is expected to be less significant in criteria like fluency, which is closely related to likelihood. to verify this characteristic of likelihood bias, we adopt two tasks: datato-text and grammatical error correction (gec). we use these tasks because, unlike most existing data @xcite @xcite , the evaluation data for these tasks include multiple criteria such as fluency, grammar, and data coverage. our experimental results show that both evaluators based on gpt-3.5 and llama2-13b @xcite indeed suffer from likelihood bias. moreover, our bias reduction method mitigates likelihood bias, and improves evaluation performance in many cases. ## method following a common methodology in llm-based evaluation @xcite , we calculate the llm's evaluation score score m based on the models' response to a prompt. specifically, we calculate score m as the expected value over candidate scores (e.g. {1, 2, 3, 4, 5}) based on the probability that models output these scores, following the setting of @xcite . our prompt includes a task description and the evaluation criteria, and several few-shot example instances for in-context learning foot_0 . the reason we use in-context learning is that it is known to stabilize the model. this puts us in a position to quantify the strength of likelihood bias. ## conclusion this paper identifies likelihood bias in llms as the phenomenon of llms overrating high-likelihood texts and underrating low-likelihood ones. we introduce a method for quantifying bias and propose a solution to the bias problem: using highly biased instances as few-shot examples for in-context learning. experiments with two tasks (data-to-text and gec) show that llms exhibit strong likelihood bias, and that our method successfully mitigates it, improving evaluation performance. in future work, we aim to investigate the relation-ship between other biases in llm-based evaluators, such as self-enhancement bias @xcite and verbosity bias @xcite , and likelihood bias. for instance, we may be able to explain self-enhancement bias through the concept of likelihood bias, as the answers generated by llms might have high likelihoods calculated by the models themselves. additionally, we plan to examine the impact of instruction-tuning and model size on likelihood bias.