Self-Preference Bias in LLM-as-a-Judge

## Abstract

Abstract Automated evaluation leveraging large language models (LLMs), commonly referred to as LLM evaluators or LLM-as-a-judge, has been widely used in measuring the performance of dialogue systems.
However, the self-preference bias in LLMs has posed significant risks, including promoting specific styles or policies intrinsic to the LLMs.
Despite the importance of this issue, there is a lack of established methods to measure the self-preference bias quantitatively, and its underlying causes are poorly understood.
In this paper, we introduce a novel quantitative metric to measure the self-preference bias.
Our experimental results demonstrate that GPT-4 exhibits a significant degree of self-preference bias.
To explore the causes, we hypothesize that LLMs may favor outputs that are more familiar to them, as indicated by lower perplexity.
We analyze the relationship between LLM evaluations and the perplexities of outputs.
Our findings reveal that LLMs assign significantly higher evaluations to outputs with lower perplexity than human evaluators, regardless of whether the outputs were self-generated.
This suggests that the essence of the bias lies in perplexity and that the self-preference bias exists because LLMs prefer texts more familiar to them.

## 1 Introduction

Measuring the quality of responses in dialogue systems presents unique challenges due to the diverse range of response strategies.
Nowadays, thanks to the strong text understanding of large language models (LLMs), various automatic evaluations can be easily implemented by employing LLMs as evaluators (often referred to as LLM evaluators or LLM-as-a-judge).
MT-Bench (Zheng et al., ) is an example of a benchmark utilizing this approach to score dialogue systems.

However, as Deutsch et al. () reported, LLM evaluators are inherently biased.
The inherent bias might cause inappropriate testing and inject biased preferences into the target dialogue systems (Zheng et al., ).
One of the most significant biases is self-preference bias, which refers to the tendency of LLMs to overestimate the quality of their own outputs, as demonstrated in Figure 1(a).
The self-preference bias poses potential risks, including the promotion of specific ideologies or response styles intrinsic to the LLM evaluator.

Several studies have addressed the issue of self-preference bias; however, there is a lack of reliable metrics to quantify the extent of self-preference bias, and the fundamental causes of this phenomenon remain unclear.
Panickssery et al. () reported on the relationship between self-preference bias and self-recognition ability, but their work lacked a comparison with human evaluators.
Xu et al. () and  Stureborg et al. () addressed quantifying self-preference bias within an evaluation approach where LLMs assign an absolute score to a single generated text.
In this approach, gold-standard evaluators are required to assign scores based on abstract criteria while ensuring consistency with prior evaluations, making it challenging to obtain accurate assessments.
As a result, these studies often restricted their scope to specific tasks, such as text summarization or machine translation, and relied on reference-based metrics like BLEURT (Sellam et al., ), which does not reflect the diversity of real-world use cases.

By contrast, a pairwise evaluation approach that involves direct comparison between two texts enables evaluators to recognize specific differences more readily, resulting in more consistent human judgments.
Consequently, such pairwise evaluation methods are particularly suitable for analyzing biases related to discrepancies with human evaluations.

In this paper, we measure the self-preference biases of LLMs in the pairwise evaluation.
To accomplish this, we propose a new metric to quantify self-preference bias on the basis of algorithmic fairness concepts, thereby enabling discussions within the existing frameworks of fairness.
In our experiment, we measured self-preference bias in eight LLMs.
The results indicated that GPT-4 exhibited a significant self-preference bias (Figure 1(b)).
This finding suggests a potential concern: using GPT-4 as a judge may lead to excessive influence from GPT-4’s unique styles and policies.

Furthermore, we investigated the underlying causes of self-preference bias.
Although LLM evaluators are not explicitly informed whether a given text is their own, they still exhibit self-preference bias.
We hypothesized that LLM evaluators might be affected by the perplexity of the text, which tends to be lower perplexity when it is generated by themselves.
To test this hypothesis, we analyzed the relationship between the perplexities of the texts to be evaluated and their corresponding evaluations.
Our analysis revealed that LLMs assign significantly higher evaluations to texts with lower perplexity than human evaluators, regardless of whether the texts were self-generated.
This suggests that the fundamental cause of self-preference bias may be the familiarity of the texts to the LLM evaluators, specifically how likely they are to generate the same response.

Figure: (a)
Refer to caption: extracted/5930576/images/example_self_preference_bias.png

#### Contributions

The contributions of this paper are threefold: (1) We propose a new metric to quantify self-preference bias in LLMs; (2) Using this metric, we evaluate the extent of self-preference bias across eight different LLMs; and (3) We identify a tendency for LLMs to assign higher ratings to texts with lower perplexity while exploring potential causes for this self-preference bias.

## 2 Related Work

LLM-as-a-judge has been studied in various directions.
The use of large language models (LLMs) for benchmarking has been advancing steadily.
LLMs are increasingly employed in dialogue evaluation due to their ability to provide flexible assessments from perspectives such as utility and safety (Zheng et al., ; Sottana et al., ; Wang et al., ; Schick et al., ).
Furthermore, numerous studies have explored the use of feedback from LLMs to enhance the learning of the LLMs themselves (Madaan et al., ; Chen et al., ).
In particular, pairs of texts annotated with their relative quality have been proven to be useful training signals (Sun et al., ; Yuan et al., ), combined with various learning methods (Ouyang et al., ; Rafailov et al., ).

The self-preference bias is only one of several limitations inherent in the LLM-as-a-judge paradigm.
Zheng et al. () identified other biases such as position bias, where specific positions within a prompt are preferentially selected, and verbosity bias, which favors longer responses.
Position bias can be addressed relatively easily by rearranging the order of comparative options within the prompt, and alignment-based methods to mitigate the bias have also been proposed (Li et al., ).
Additionally, Saito et al. () investigated the presence of verbosity bias in GPT-4 and GPT-3.5-Turbo, focusing on the quantification of bias by examining the error rates associated with longer versus shorter text options.

## 3 Preliminaries: Fairness and Bias

The measurement of bias in classifiers has been widely discussed within the framework of fairness (Calders et al., ; Hardt et al., ).
In this section, we focus on a representative definition of fairness, Equal Opportunity, as the preparation for quantifying the self-preference bias in LLM evaluators.

Equal Opportunity (Hardt et al., ) is a fairness definition that requires the classifier to achieve equal recall across groups with different sensitive attributes (e.g. gender or race).
Specifically, let the sensitive attribute be $S\in\{0,1\}$, the prediction of the classifier be $Y^{\prime}\in\{0,1\}$, and the ground truth label be $Y\in\{0,1\}$.
The classifier satisfies Equal Opportunity if the following condition holds:

$$ $P(Y^{\prime}=1|S=1,Y=1)=P(Y^{\prime}=1|S=0,Y=1)$ (1) $$

When quantifying bias in the classifier, the difference or ratio between both sides of the equation are often used.
The amount of bias derived from this definition is the difference in how well the classifier matches the ground truth between groups with different sensitive attributes.
Therefore, if there is already bias towards sensitive attributes during the creation of the ground truth, Equation 1 cannot be considered an ideal definition of fairness.

To address this drawback, the fairness definition known as Demographic Parity (Calders et al., ), which does not rely on ground truth, is also widely used.
Demographic Parity requires that the predictive distribution of the classifier be consistent across groups with different sensitive attributes.
It is based on the assumption that there is no inherent causal relationship between the sensitive attributes and the predicted labels.

LLM-as-a-judge is a technique aimed at replacing human evaluators with LLMs, and more specifically, it assumes that dialogue system are aligned based on human preferences.
Therefore, in this study, we employ the concept of Equal Opportunity to quantify self-preference bias by treating LLM evaluators as classifiers.

## 4 Quantifying Self-Preference Bias

In this section, we propose a new metric to quantify self-preference bias.
In particular, we focus on a setting where evaluators compare two texts and select the one higher quality, allowing comparison with reliable human evaluations.

### 4.1 Self-preference Bias Metric

To measure the extent to which an LLM’s evaluation deviates from human evaluations across its own responses and those generated by others, we employ the concept of Equal Opportunity (Hardt et al., ).
The bias is quantified by calculating the difference between both sides of Equation 1.
A rigorous definition is provided below.

###### Definition 4.1 .

(self-preference bias of the evaluator $f$.)
Let $f$ be the evaluator assessing the quality of dialogue responses, capable of generating its own responses.
For a pair of responses $y_{0}$ and $y_{1}$ in a dialogue, define:
$Y\in{0,1}$ as the index of the human-preferred response,
$Y^{\prime}\in{0,1}$ as the index of the $f$-preferred response, and
$S\in{0,1}$ as the index of the $f$-generated response.
We define the self-preference bias of the evaluator $f$ in dialogue comparison evaluation as follows:

$$ $\text{Bias}=P(Y^{\prime}=1|S=1,Y=1)-P(Y^{\prime}=1|S=0,Y=1).$ (2) $$

In this definition, the amount of bias is represented by the difference between the conditional probability of the evaluator $f$ rating itself favorably given that the human evaluator has rated it favorably, and the conditional probability of the evaluator $f$ rating itself unfavorably given that the human evaluator has rated it unfavorably.
A value of 0 0 indicates the absence of bias, while a value close to $1$ suggests a high degree of bias.
Conversely, a value of $-1$ would indicate the presence of a reverse bias, where the evaluator $f$ tends to undervalue its own responses.
In this definition, the case where $y_{1}$ is preferred is explicitly considered, but this does not result in a loss of generality by treating the preferred response as $y_{1}$.

###### Definition 4.1 .

### 4.2 Experimental Setting

The aim of our experiment is to quantify the self-preference bias in various LLMs.
To address a wide range of tasks and topics, we have LLMs evaluate responses in open-ended dialogues and measure the bias using the Definition 4.1.

First, we provide the LLM evaluator with a user query written by a human and two responses generated by different LLMs.
We then ask the LLM evaluator to determine which of the two responses demonstrates higher quality.
For clarity and ease of reference during evaluation, we designate the two responses as “response A" and “response B".
Finally, using the probabilities that the LLM evaluator outputs for “A" and “B", denoted as $p(\text{A}|\text{context})$ and $p(\text{B}|\text{context})$, we calculate the score for response A using the following equation.

$$ $\text{score}_{\text{A}}=\frac{p(\text{A}|\text{context})}{\sum_{w\in\{\text{A, B}\}}p(w|\text{context})}$ (3) $$

The score for response B is calculated in the same manner.
This post-processing, as shown in Equation 3 is designed to enhance the interpretability of analyses across LLMs with different probability distributions, following the approach of  Schick et al. ().
Additionally, according to Zheng et al. (), LLMs may exhibit position bias, which refers to the tendency to prefer responses located in specific positions within the prompt.
To mitigate this bias, we swap the positions of responses A and B, and the evaluation scores are averaged over two iterations.
Using the scores obtained through the above process, we examine the extent to which there is a difference in evaluation scores between the LLM’s own generated response and responses generated by other LLMs.

Empirical evaluation uses Chatbot Arena dataset (Zheng et al., ), which contains 33,000 dialogues and each consisting of a user query and a pair of responses generated by two different LLMs.
We calculate evaluation scores for the pre-existing response pairs stored in the dataset.
In this dataset, every response pair is labeled with either "model_a", "model_b", or "tie" as a result of a human evaluation comparing the quality of the responses.

As the LLM evaluators, we employed the following eight LLMs, which also used in Chatbot Arena dataset: the closed-source GPT-3.5-Turbo and GPT-4 (Josh et al., ), and the open-source Vicuna-7b, Vicuna-13b (Chiang et al., ), oasst-pythia-12b (Biderman et al., ), dolly-v2-12b (Conover et al., ), Koala-13b (Geng et al., ), and stablelm-tuned-alpha-7b (Jonathan Tow, ).
We used the same prompt as Zheng et al. () to calculate the evaluation scores of responses A and B for all LLM evaluators.
This prompt was created based on the work of  Zheng et al. (), and it instructs the LLM to output either “[[A]]", “[[B]]", or “[[C]]" after providing an explanation.
In the implementation, we used the output probability distribution following the token corresponding to “[[" in the generated text to compute the score using Equation 3.
Additionally, if the LLM evaluators failed to comply with the prompt instructions and did not provide a response in the format of “[[A]]" or a similar structure, these cases were excluded from the analysis.

### 4.3 Result

Figure: Figure 2: Confusion matrix for each LLM evaluator’s assessment of pairs including its own output. It suggests that some LLMs, including GPT-4, have relatively high true positive rates than true negative rate, which means that they have self-preference bias.
Refer to caption: extracted/5930576/images/confusion_matrix.png

**Table 1: An example of self-preference bias in GPT-4. The user query involves a simple request to list blue items. GPT-4 responds by first acknowledging its lack of physical cognition, and then proceeds to list blue items. In contrast, GPT-3.5-Turbo directly lists blue items without any preliminary explanation which human evaluation prefers.**
| User: Please list the blue items |  |
| --- | --- |
| GPT-4: As a language model AI, I can’t see the<br>physical objects around you. However, here are<br>some common blue items that people might have:<br>1. Blue shirt<br>2. Blue jeans<br>3. Blue pen<br>4. Blue coffee mug<br>5. Blue pillow<br>6. Blue blanket<br>7. Blue towel<br>8. Blue notebook<br>9. Blue umbrella<br>10. Blue vase<br>Please specify the context or the category of items<br>you are interested in if you need a more precise list. | GPT-3.5-Turbo: Sure, here are some blue items:<br>1. Blue jeans<br>2. Blueberries<br>3. Blue shirt<br>4. Blue pen<br>5. Blue car<br>6. Blue backpack<br>7. Blue notebook<br>8. Blue umbrella<br>9. Blue sneakers<br>10. Blue curtains |
| GPT-4 preferred | Human and GPT-3.5-Turbo preferred |
|  |  |

The results of the bias measurement using Definition 4.1 are presented in Figure 1(b).
It was confirmed that GPT-4 exhibits the highest self-preference bias.
Definition 4.1 focuses on the recall of the LLM evaluator concerning both high and low ratings by the human evaluator.
Thus, it can be concluded that GPT-4 showed lower recall in cases where humans evaluated unfavorably compared to when higher evaluating.
When examining the recall values in the confusion matrix shown in Figure 2, they are calculated as $0.945\approx\frac{1852}{108+1852}$ and $0.425\approx\frac{118}{160+118}$.
The difference between these values is $0.520$, which corresponds to the value reported in Figure 1(b).
Following GPT-4, Vicuna-13b and Koala-13b also exhibited significant bias.
In contrast, other LLMs displayed values relatively close to zero.
Notably, oasst-pythia-12, dolly-v2-12b, and stablelm-tuned-alpha-7b showed negative values, indicating a reverse bias where the LLMs tend to underestimate their own outputs.

Table 1 presents a randomly selected example of self-preference bias in GPT-4, where humans favored the alternative response.
In this example, the user query is a straightforward request to list blue items.
While GPT-4 states that it lacks physical recognition before listing, GPT-3.5-Turbo directly lists the blue items without any such explanation.
Both responses are of high quality, and the final evaluation reflects the evaluator’s policy and stylistic preferences.
Although humans and GPT-3.5-Turbo preferred the response from GPT-3.5-Turbo, GPT-4 favored its own response, illustrating a typical case of self-preference bias.

## 5 How Do LLMs Overestimate Their Own Outputs?

We have shown that certain LLM evaluators tend to assign disproportionately high scores based on whether the output being generated from themselves, even though LLM evaluators are not explicitly provided with labels indicating whether a given output is their own.
We hypothesized that LLM evaluators might be affected by the similarity of the response to their own output.
To investigate this further, we focus on how LLM evaluators change their evaluation depending on the perplexities of responses.

Figure: Figure 3: LLMs vs human conditioned on perplexity. Winning judgment rates by LLMs conditioned on perplexity with human winning judgment rates are plotted. All models except dolly-v2-12b and stablelm-tuned-alpha-7b demonstrated a clear tendency to assign higher evaluations to responses with lower perplexity.
Refer to caption: extracted/5930576/images/win_rate_each_perplexities.png

Figure: Figure 4: LLM vs other LLMs conditioned on perplexity. Winning judgment rates by LLMs on their own texts and texts generated by other models conditioned on perplexity are plotted. Across all models, except for dolly-v2-12b and stablelm-tuned-alpha-7b, no significant difference was observed between the judgment rates for their own texts and those generated by other models. This suggests that LLM evaluators assign higher ratings to texts with lower perplexity, regardless of whether the text was self-generated or produced by other models.
Refer to caption: extracted/5930576/images/win_rate_each_perplexities_self_other_wo_human.png

Figure: Figure 5: Average of log-perplexity of responses for each LLM evaluator. The red bars represent the perplexity for responses generated by the LLM evaluator itself, while the blue bars represent the perplexity for responses generated by other LLMs. Across all models, the average perplexity is lower for responses generated by the evaluators themselves.
Refer to caption: extracted/5930576/images/mean_perplexities_self_other.png

As in previous experiments, we employed the pairwise evaluation framework where evaluators compare two responses, response A and B.
First, we computed the perplexities of responses conditioned on the prompt, and took the difference between response A and B for all samples.
Next, we divided the perplexity differences into bins and calculated the probabilities that each LLM evaluator judged response A as the winner in each subset.
Additionally, we computed the winning judgment rate of the human for response A within each bin.
In this experiment, we excluded GPT-4 and GPT-3.5-Turbo, as perplexity values could not be obtained for these models.

Figure 3 shows the comparison of winning judgment rates within each perplexity bin for six models.
All models except stablelm-tuned-alpha-7b demonstrated a clear tendency to assign higher evaluations to responses with lower perplexity.
Furthermore, it was confirmed that this tendency was stronger in vicuna-13b, vicuna-7b, koala-13b, and oasst-pythia-12b than in humans.
This result indicates that LLM evaluators overly change their evaluation depending on perplexities of responses.

To further investigate the effect of differences in whether the output is self-generated or not, we segmented the evaluation results by the LLM evaluators into two distinct groups: one where the LLM evaluators’ own output is included in the pair and another where it is not.
We present the results with the output of the LLM evaluator as response A, without loss of generality.
As shown in Figure 4, the winning judgment rates between two groups were similar for all models except dolly-v2-12b and stable-tuned-alpha-7b.
This suggests that the factor influencing the LLM evaluators’ judgments is not whether the response is their own but rather the perplexity of the responses.
As confirmed by Figure 5, LLMs tend to exhibit lower perplexity for their own outputs.
In other words, these findings imply that self-preference bias may be a phenomenon where the model’s own output inherently exhibits lower perplexity.

In these experiments, we were unable to obtain perplexity values from GPT-4 and GPT-3.5-Turbo, resulting in a lack of analysis on the competitive LLMs.
To address this gap, we conducted additional experiments using Llama2 (Touvron et al., ) and Llama3 (Dubey et al., ).
However, the Chatbot Arena dataset does not contain human annotations for these LLMs’ responses.
Therefore, we obtained evaluations for existing responses from these Llama models and only compared them with human evaluations.

The results are presented in Figure 6.
We found that all models changed their evaluations more than humans, depending on the difference in perplexity.
This indicates that even in the competitive models, including Llama 3.1, perplexity may be causing unfair bias in the evaluation.

Figure: Figure 6: Llama family vs human conditioned on perplexity. Winning judgment rates with condition on perplexity are plotted. Llama-2 and later models models exhibited significantly higher winning judgment rate with lower perplexity compared to human evaluators.
Refer to caption: extracted/5930576/images/win_rate_each_perplexities_llamas.png

## 6 Discussion

To reduce self-preference bias, one possible approach is ensemble evaluation using multiple models.
This method is expected to provide a more equitable evaluation by avoiding reliance on a single model.
Specifically, when a model exhibits low perplexity on a sample, decreasing the weight assigned to that model’s evaluation for that sample may contribute to bias mitigation.
To evaluate the effectiveness of bias reduction strategies, our proposed new metric can be utilized.
Therefore, we believe that our research makes a significant contribution to the understanding of self-preference bias and will greatly facilitate the development of future research in this area.

Our experimental results reveal that LLM evaluators tend to assign higher scores to texts with lower perplexity.
We further discuss the reasons behind this phenomenon.
First, LLMs are trained during the pretraining phase to reduce perplexity on large-scale text corpora.
Moreover, when aligning with human preferences, the models are also trained to minimize perplexity on the given dialogue data.
Therefore, high-perplexity texts are likely those that the LLM has not frequently encountered during training, suggesting that such texts may be related to domains that the LLM evaluators do not fully comprehend.

This observation may seem contradicted by the fact that GPT-4, which is well-versed across various domains due to a wide range of benchmarks, exhibits a high degree of self-preference bias.
However, by investigating specific cases of self-preference bias, as shown in Figure 1(a) and Table 1, we found that the bias was often not related to clear factual errors but rather to differences in response styles, such as the handling of specialized domains or the description of premises before answering.
This suggests that, advanced models like GPT-4, which thoroughly understand and adhere to their predefined policies, may use the degree of alignment with these policies as a deciding factor when evaluating responses of comparable quality.

The proposed metric in Definition 4.1 is based on the fairness definition known as Equal Opportunity.
Demographic Parity Calders et al. () is also a prominent definition of fairness.
Demographic Parity requires that the predictive distribution of a classifier be consistent across sensitive groups, regardless of the ground truth.
In the context of this study, the focus is on whether the evaluations by LLMs align with human preferences, which is why Demographic Parity was not the focal point.
While the bias metric based on Demographic Parity cannot demonstrate the unfairness of an evaluation, it is useful for analyzing how highly each LLM evaluator rates its own outputs within the given experimental setup.
Similar to the Definition 4.1, the self-preference bias based on Demographic Parity also can be quantified as follows:

$$ $Bias=P(Y^{\prime}=1|S=1)-P(Y^{\prime}=1|S=0).$ (4) $$

The scores derived from this metric of eight LLMs are presented in Table 2.
The results indicate that GPT-4 exhibited significant bias, followed by Vicuna-13b, which aligns closely with the results obtained using Definition 4.1.

**Table 2: Self-preference bias scores based on Demographic Parity. GPT-4 assigns the highest scores to its own outputs, followed by Vicuna-13b. However, it is important to note that these scores do not take intrinsic quality into account. Therefore, this analysis reflects how highly each LLM evaluator rates its own outputs within the given experimental setup and should not be interpreted as an indication of unjust evaluation.**
| LLM | Bias |
| --- | --- |
| GPT-4 | 0.749 |
| GPT-3.5-turbo | 0.191 |
| Vicuna-13b | 0.382 |
| Vicuna-7b | 0.052 |
| Koala-13b | 0.175 |
| oasst-pythia-12b | 0.006 |
| dolly-v2-12b | -0.069 |
| stablelm-tuned-alpha-7b | -0.032 |
|  |  |

## 7 Conclusion

In this study, we propose a metric to quantify the self-preference bias in LLM-as-a-judge and measured the self-preference bias of eight LLMs.
Experimental results confirmed that GPT-4, in particular, exhibits a high self-preference bias.
This finding suggests a risk that GPT-4 as a judge may inadvertently reinforce its own style and policies.

Furthermore, we hypothesized that the self-preference bias is related to the perplexity of the texts, and showed that, compared to human evaluators, LLM evaluators assigned higher evaluations to texts with lower perplexity, and this tendency was observed regardless of whether the text was generated by themselves or not.
This suggests that the essence of the bias lies in perplexity and that the self-preference bias exists because LLMs prefer texts more familiar to them.

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 3
presentation: 3
questions: 1. Considering that LLMs may exhibit different biases depending on the context, how does your method account for contextual variations? Can the bias calculation be adapted to reflect dynamic changes in model behavior?

2. How do you handle potential inconsistencies in human evaluations that serve as the benchmark for calculating bias? What processes are in place to mitigate subjectivity in these evaluations?

3. Given your hypothesis that self-preference bias is related to text perplexity, how do you isolate the effect of perplexity from other factors when interpreting the results? What methodology do you use to analyze this relationship?

4. Have you considered the impact of model capabilities and the data distribution of self-preference bias results obtained for each model on the bias?
soundness: 2
strengths: 1. The research introduces a new metric for quantifying self-preference bias in LLMs, providing a tool for evaluating the performance of language models in a systematic manner.

2. The study assesses the self-preference bias across eight different LLMs, offering a broad perspective on the prevalence of this bias within various models, particularly highlighting the significant findings related to GPT-4.

3. The paper explores the relationship between LLM evaluations and output perplexity, revealing a tendency for LLMs to favor outputs with lower perplexity. This finding explores the nature of self-preference bias and encourages further research on how confusion affects model evaluation.
summary: This work introduces a pairwise evaluation approach to assess self-preference bias in LLM evaluators, enhancing the ability to identify specific differences between responses. Unlike previous studies that relied on absolute scoring and limited task scopes, this method allows for more consistent human judgments and broader applicability across diverse dialogue scenarios. It addresses the need for reliable metrics to quantify self-preference bias while directly comparing LLM evaluations with human assessments. The main contributions are:
1. The authors propose a new quantitative metric specifically designed to measure self-preference bias in LLMs. The metric is applied to assess the extent of self-preference bias across eight different LLMs, providing insights into the biases present in these models.
2. The paper explores the relationship between LLM evaluations and output perplexity, revealing a tendency for LLMs to favor outputs with lower perplexity. This analysis suggests that the self-preference bias may stem from LLMs' familiarity with certain text styles, highlighting the importance of perplexity in understanding LLM behavior.
weaknesses: 1. The formula used to quantify this bias is unreasonable. Using a static probabilistic model may fail to capture the dynamic characteristics of model behavior, affecting the applicability and utility of the findings. Additionally, as shown in Figure 2, GPT-4 correctly identifies a significant number of cases in both True and Predicted values. This leads to a large value in the first term of the formula. GPT-4’s considerably stronger performance compared to other models impacts the bias result. In the relatively small set of negative samples, GPT-4 predicts it is correct 160 times compared to 118, a difference of only a few dozen. However, due to the formula, GPT-4’s bias score appears large simply because it performs well in many cases. This formula for measuring bias is too simplistic and does not appropriately account for the capabilities of large models, being influenced by their performance.

2. This method primarily focuses on specific conditions for bias, potentially overlooking other factors such as the content, context, and other semantic features of the input text. This narrow focus may lead to a partial understanding of the model's self-preference bias. Additionally, the scale of the analyzed data is insufficient and lacks comprehensiveness in broader contexts. The choice of specific input features in calculating the bias can influence the results.

3. In the writing, Equation 3 lacks an explanation of ω.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: - As your investigation has shown the main reason for self-preference bias is actually coming from favoring low ppl response, does this suggest there is actually no self-preference bias but only low-ppl bias?
soundness: 3
strengths: - The writing good in general, although the content is a bit short, but overall positive.
- Although the idea of self-preference bias is not new, this paper proposed an approach to quantify this bias.
- This paper did disentangle the positional bias from the self-preference bias during quantification.
- The observations on LLM evaluators favor candidates with lower perplexities is insightful and interesting.
summary: This paper addresses the issue of self-preference bias of LLM evaluator when doing pairwise comparisons. The paper proposed a method to quantify the self-preference bias and evaluate the self-preference bias on 8 recent LLMs using ChatBot Arena annotations. The paper further investigate the potential reason for self-preference bias might be LLMs tend to favor candidates with lower perplexities.
weaknesses: - The formula for the metric is valid, but there are some concerns when calculating it:
    - The main concern for the metric is the class balancing issue. For example, in the Fig.2, there are 108+1852 = 1960 comparisons where gpt-4 wins, but only much less 118+160=278 comparisons where gpt-4 fails. Comparing percentages calculated on them might lead to concern that the bias might be less accurate when true label class is more imbalanced. This means if a LLMs is more preferred compared to others, the self-preference bias is less accurate.
    - Although the paper has disentangled positional bias from self-preference bias, but what about other bias? e.g. verbosity bias? As shown in your table 1, is the gpt-4 really preferred its own answer or just any longer answer?

- As the content is a bit short, I suggest a bit further investigation on the 'tails-up' phenomena as shown in Fig3 and 4. When the diff. log ppl is close to the right side, the winning rate of A has a sudden jump.

---

confidence: 4
contribution: 2
initial_rating: 6
presentation: 3
questions: Questions
You mentioned in line 208 that you excluded responses that did not follow the prompt you specified. What percentage of the data was that? Also is there any significance testing? Is a bias of 0.25 significantly worse than let's say 0.2?

Suggestions
One suggestion I have is to better explain in text how to read the plots in Figures 3 and 4.
soundness: 3
strengths: The continued study of LLMs as evaluation metrics is critical and it is important to continue to study the pros/cons of these metrics. We already know that these LLMs are good at a variety of tasks but if we continue to rely on them as metrics then we are self-reinforcing that the responses they select are good and are then injecting the metric's bias into our system. Therefore the authors providing an explanation as to where the bias is coming from (perplexity) can help the community then think of ways to mitigate this bias. Overall this conclusion is the biggest contribution of this work.
summary: The authors propose a metric to quantify the self-preference bias in LLMs or in other words measure how biased an LLM is towards it's own response. Their metric is inspired by the concept of Equal Opportunity (Hardtet al., 2016) where a classifier needs to achieve equal recall across classes. From this they derive their bias metric as the difference between the probability of the LLM rating itself high given that the human rated it high and the probability of the LLM rating itself low given that the human rated it low.

They leverage the Chatbot Arena dataset which has dialogs consisting of a prompt and a pair of responses generated by two different LLMs. They then measure the bias of 8 different LLMs as evaluators on this dataset and find that a majority of these models such as GPT-4 is biased towards selecting a response generated by itself. 

The authors then try to answer why this self-preference bias may exist and look at the perplexity scores of responses. They find that on average models assign higher evaluations to responses with lower perplexity. 

They then offer some suggestions to mitigate these biases such as using multiple models.
weaknesses: One concern I have is around how novel are the conclusions and how different is this from previous work. 

1. Deutsch et al. (2022) gets cited as work that looked at bias within these LLMs(https://aclanthology.org/2022.emnlp-main.753/). In that work it is mentioned "Not only do they favor the underlying models’ outputs, but they are also biased toward outputs from models which are similar to their own". What sets this paper apart if Deutsch et al. (2022) is already looking at this same type of bias?

2. It has been well documented that perplexity is not a good automatic metric for dialog systems (https://arxiv.org/pdf/1603.08023, https://arxiv.org/pdf/1906.09308). Since one of the conclusions from this paper is that the use of perplexity leads to bias should we then conclude perplexity shouldn't be used as a metric? And if so what then sets this paper part from the work I listed. 

Overall if the authors can make it more clear how their work is different from the previous ones I mentioned that would help alot in putting it's novelty into perspective.

---

confidence: 3
contribution: 2
initial_rating: 3
presentation: 2
questions: - Why does the caption of Figure 3 state that all models except dolly-v2-12b and stablelm-tuned-alpha-7b demonstrated a clear tendency to assign higher evaluations to responses with lower perplexity? According to the results in Figure 3, dolly-v2-12b shows a trend of decreasing winning judgment rates as perplexity decreases. This caption is also contrary to your conclusion in the text (Lines 312-316).

- How were human evaluations obtained in the experimental results shown in Figure 6?

- How was the temperature set in all experiments? Was there any consideration of the impact of temperature on the experimental conclusions?

- Have you tested the effectiveness of your proposed bias elimination methods (Lines 407-413)? Have you considered some commonly used bias elimination methods, such as special prompts or SFT?
soundness: 2
strengths: - The writing is clear.
- The focus of this paper is crucial.
- This paper provides some interesting conclusions.
summary: This paper introduces a quantitative metric to measure self-preference bias in LLM-as-a-Judge. The authors analyze the relationship between LLM evaluations and the perplexities of outputs. Their findings reveal that LLMs assign significantly higher evaluations to outputs with lower perplexity than human evaluators, regardless of whether the outputs were self-generated.
weaknesses: - The discussion of related work is insufficient, lacking discussion of several relevant works, such as "Benchmarking Cognitive Biases in Large Language Models as Evaluators," and doesn't introduce mainstream LLM-as-a-Judge methods, including single-answer grading and pairwise evaluator.

- A more detailed and complete definition and explanation of the proposed metric are needed.

- The authors did not account for the impact of response length on the LLM evaluator during experiments, weakening the validity of the conclusions. Bias in LLM-as-Judge is a complex issue where position bias, verbosity bias, and self-preference bias interact and affect each other. In the Chatbot Arena dataset used by the authors, significant differences in response lengths are common. The authors need to discuss how to ensure this discrepancy does not affect their experimental conclusions.

- There is ambiguity in the setting of Figure 6. The authors did not clearly explain how the experimental results shown in Figure 6 were obtained. Please refer to the related questions in the question section.

- The authors did not conduct experiments to explore how to eliminate self-preference bias, only briefly discussing possible measures in the discussion section, which reduces the completeness of the work.

Meta-review: The authors investigate the issue of self-preference bias when the same model is used to generate output and to judge it. They focus their analysis on perplexity, claiming that it is the fact that the lower perplexity of the model's own outputs is the cause of self-preference bias. The reviewers appreciate the authors' efforts in introducing a new metric for measuring self-preference bias in LLMs, their analysis across 8 LLMs, clarity of writing, and insights provided. However the reviewers also raise several concerns such as the simple nature of the formula to compute the metric that may fail to capture the dynamics of text generation or the fact that it doesn't properly take class imbalance and response length into account, and the overall novelty of the work. The authors attempt to address these concerns but overall their responses do not seem convincing; while promising, this work is not mature yet for publication.

Decision: Reject