Unbiased Evaluation of Large Language Models from a Causal Perspective
Authors: Meilin Chen
Jian Tian
Liang Ma
Di Xie
Weijie Chen
Jiang Zhu
Venue: ICML 2025 poster (2025)
Keywords: Agents-as-an-Evaluator
LLM Evaluation
Evaluation Bias
Primary area: deep_learning->large_language_models
TL;DR: This paper is the first to identify the biases in Agents-as-an-Evaluator and propose the Unbiased Evaluator, an evaluation protocol that delivers a more comprehensive, unbiased, and interpretable assessment of LLMs.

Abstract: Benchmark contamination has become a significant concern in the LLM evaluation community. Previous Agents-as-an-Evaluator address this issue by involving agents in the generation of questions. Despite their success, the biases in Agents-as-an-Evaluator methods remain largely unexplored. In this paper, we present a theoretical formulation of evaluation bias, providing valuable insights into designing unbiased evaluation protocols. Furthermore, we identify two type of bias in Agents-as-an-Evaluator through carefully designed probing tasks on a minimal Agents-as-an-Evaluator setup. To address these issues, we propose the Unbiased Evaluator, an evaluation protocol that delivers a more comprehensive, unbiased, and interpretable assessment of LLMs. Extensive experiments reveal significant room for improvement in current LLMs. Additionally, we demonstrate that the Unbiased Evaluator not only offers strong evidence of benchmark contamination but also provides interpretable evaluation results.

Unbiased Evaluation of Large Language Models from a Causal Perspective
Meilin Chen Jian Tian"
Liang Ma
Di Xie
Weijie Chen
Jiang Zhu'
Abstract
generation
evaluation
Benchmark contamination has become signif- icant concern in the L.M evaluation commu- nity. Previous Agents-as-an-Evaluaa this issue by involving agents the generation of questions. Despite their success, the biases Agents-as-an-Evee Evvaluurt methods remain largely unexplored. In this paper, we present theoretical formulation of evaluation bias, providing valu- able insights into designing unbiased evaluation protocols. Furthermore, we identify two type of through carefully designed probing tasks on a minimal Agents-as- an-Evaluator setup. To address these issues, we propose the Unbiased Evaluator, evaluation protocol that delivers biased, and interpretable assessment of LLMs. Extensive experiments reveal significant room for improvement in current LLMs.. Additionally, we demonstrate that the Unbiased Evaluator not only offers strong evidence of benchmark contamina- tion but also provides interpretable evaluation re-
address
familiar!
tough!
data blas
model bias
tough!
bias in Agents-as-an-Evallln
tough!
BOAT
comprehensive, un-
(b) our proposed anblased
Figure 1. (a) Agents-as-an --aauuuut suffers from data and model bias. (b) Our proposed Unbiased Evaluator dynamically evaluate the LLMs with designed Bags Of Atomic InPerventions (BOAT).
2022; White et al., 2024), or by introducing more complex and challenging tasks to push the boundaries model capa- bilities et al., 2024; He al., 2024b; Lightman et al., 2023; AI-MO, 2024; 2023; He al., 2024a).
sults.
1. Introduction
Despite their success, public benchmarks, most widely used to assess and compare model performance, are particularly vulnerable to contamination issues(L.ovin, 2023; Bender et al., 2021; Kocoń et al., 2023; Li, 2023; Zhou et al., 2023; Ni al., 2024), which are increasingly inevitable due to the scale of training data used in modern models. Recently, agent- based evaluation have been proposed to address con- tamination issue(Zhu et al., 2024; Liu et al.). Among them, MPA (Zhu et al., 2024) proposes to involve probing and judging agents to automatically transform existing prob- lems benchmarks into new ones. CogMath(Liu et al.) decouple questions into several evaluation dimensions via an multi-agent system for evaluating LLM's mathematical abilities.
Recently, proprietary models such as GPT-4 (Achiam et al., 2023), Claude(Anthropic 2024), Gemini(Team et al., 2023), and open-source ones, such Llama Mistral(Jiang et al., 2023), Qwen(Bai et al., 2023), Yi(AI et al., 2024) have demonstrated remarkable capabilities in natural language processing tasks and beyond. As the ca- pabilities of community models continue to grow, the im- portance of robust and fair model evaluation becomes in- creasingly critical. The community has made great efforts evaluate model performance by expanding the compre- hensiveness of benchmarks( WWagg 2018; Wang al., 2019; Srivastava al., 2022; Hendrycks et al., 2021; Liang al.,
al., 2023),
Equal contribution 'Hikvision Research Institute. Correspon- dence to: Meilin Chen merlinnarrr@ Emerinnrrr@ggail.comm Liang Ma <maliangó@ www.@eiiiiiiiiimmm......
We termed this paradigm Agents-as-an--yylucae For- mally, Seents-as-an-Evallue refers to an LLM-based eval- uation paradigm which LLMs (or Agents) not only assess responses but also actively contribute to generating evalua- tion criteria and questions. Different from previous L.M-
Proceedings of the nnnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Unbiased Evaluation of Large Language Models from Causal Perspective
1.4
Agents-as-
2.25
an Evaluator
0.25
2.00
1.2
Owen
Uama
1.75
0.20
0.4
1.0
Yi
1.50
Mistral
Roc
1.25
0.15
0.6
1.00
0.2
0.4
0.75
0.50
0.2
0.05
0.25
0.0
1.0
0.0
1.0
1.0
0.0
1.0
ARC/Strength
MMLU/Strength
ARC/Strength
MMLU/Strength
Figure Model visualizations. Left: Roc vs. Strength on ARC-C and MMLU datasets. Right: Rve vs. Strength on ARC-C and MMLU datasets. Strength refers to the probability defined in Equation 3. A higher strength value indicates greater proportion of "processed" samples within dataset ("process" denotes rephrasing and BOAT in Agents-as-an-Fvall anaauuuttr and Unhiased Evaluator, respectively). The point where strength=0 represents the original datasets. Mistral, Yi, Ilama, Qwen represents Mistral-L 2rrge-241 Yi1.5-34B Chatt Llama3. 1--70--Inssttut and Qwen2.5-72B InIstruct respectively, For Agents-ass as as-an-Eveer -n-auuluaoor, we observe a significant increase in Roc with growing strength, while Rue remains relatively stable, indicating the existence of model bias. Compared with Agents-as-an-Evenn E-valuuaoor,, our Unbiased Evaluator remains relatively stable on both Roc and Ruc-
as-a-Judge (Zheng et al., 2023), which operates on the eval- uation side by solely determining whether something falls within the scope of a given rule, Agents-as-an an-Evaluator extends beyond assessment to the generation side, where LLMs actively contribute to the design of the very questions involved in the task. Considering that prior works have re- vealed that LLM-as-a-Judae posses certain biases(Blodge et al., 2020; Ahn et al., 2022; Ferrara, 2023; Gallegos et al., 2024), compared to LLM-as-a-Judes the critical step of question generation Agents-as- arents-as-an--ae -vvaluorrl may intro- duce greater potential for bias. Therefore, an important question rises:
that Agents-as Beeett-as-an-Evaane an-Evaluator exhibit two biases: data bias and model bias. As shown in Fig. (a), data bias emerges from accuracy imbalance across different domains during generation, while model bias stems from inherent unfairness during evaluation.
To mitigate the bias in existing methods, we propose the Unbiased Evaluator, as shown in Fig. 1 (b), an evalua- tion protocol grounded in a causal perspective. Drawing inspiration from interventions causal inference, where Al systems are tasked with responding manipulated pairs to understand how altering one factor ("intervention") im- pacts other variables in a complex system. Specifically, the evaluation process is formulated as causal diagram, where LLMs reason over input variables to arrive at the final an- swer. Taking advantage this, we design Bags Of Atomic InTerventions (BOAT) to dynamically evaluate the LLMs. By combining these interventions, the Unbiased Evaluator provides more comprehensive, unbiased and interpretable assessment.
To what extent are Agents-as-ana -ai-------a-Evuuatorr biased and is there a simple, unbiased alternative for benchmark con- tamination?
In this paper, we begin with a theoretical formulation of evaluation bias. followed by an in-depth analysis of the potential bias in designing evaluation protocols. Our results demonstrate that evaluation bias can be decomposed into three components: original, independent and related terms. Therefore, the key takeaway for future research in designing evaluation protocols that any newly introduced biases should ideally mitigate and, more importantly, counteract existing biases in the original benchmark.
In summary, our contributions are three-fold: (1) A the- oretical formulation of evaluation bias, offering valuable findings for the importance of minimizing the relative term when designing evaluation protocols. (2) The first compre- hensive bias analysis for srents-as-an-Eyaluaa data and model bias which undermine the reliability and trustworthiness of aantt--asaan--vauuaee (3) An unbiased evaluation protocol, Unbiased Evaluator, provides a more comprehensive, unbiased and interpretable assessment for benchmark contamination.
revealing
To conduct bias analysis, design minimum Agents-as- an-Evaluator, i.c LLM is tasked to rephrase the questions without changing their meaning. Build upon this, together with our designed bias probing task, we empirically reveal
2

---

Unbiased Evaluation of Large Language Models from Causal Perspective
Table 1. Data bias. The Pearson and Kendall correlation between domain acc. of MMI.U and ReE across four L.Mss.
Agents-as-ann-Eallaato
Ours
Models
Kendall
Pearson
Kendall
Pearson
p-value
p-value
T
p-value
c
p-value
Mistral-Large-2411
-0.38
2.73x10-0
-0.47
2.18x10-1
0.02
0.76
0.07
0.59
Yi1.5-34B-Chat
-0.17
5.86x10-3
-0.38
3.52 x10
0.10
0.24
0.16
0.20
Llama3.1-70B- 1-..------st-tcce
-0.39
181x10
-0.45
4.86x
0.14
0.10
0.15
0.05
Qwen2.5-72B-Instee
-0.25
5,51x10-3
-0.41
1.44x10-3
0.07
0.41
0.10
0.12
2. Related Works
evaluating LLM's mathematical abilities.
2.1. LLMs Evaluation
Our proposed Unbiased Evaluator stands in contrast to these, as it is designed to be generalized for a wide range of tasks and ensures an unbiased evaluation.
The rapid growth of large language models (LLMs) un- derscores the need for increasingly robust and fair evalu- ation methods. Benchmarks offer effective alternative for model evaluation. The research community has made significant strides in expanding the comprehensiveness of benchmarks (Wang, 2018; Wang et al., 2019; Srivastava et al., 2022: Hendrycks et al., 2021; Liang et al., 2022: White al., 2024), while also introducing more complex and challenging tasks push the boundaries of model capa- bilities (Wei al., 2024; He al., 2024b; Lightman al., 2023; AI-MO, 2024; 2023; He et al., 2024a).
2.3. Evaluation Bias
Recent studies have highlighted that LLM-as-a-Judge ex- hibit various types of biases across various tasks(Dai et al., 2024; Gallegos et al., 2024; Chen et al.; Ye et al., 2024), such as position bias, length bias, self-enhancemmnt bias etc. These internal biases of LLMs may also affect LLM-as- a-judge, leading to unfair evaluation outcomes and subse- quently impacting the development of LLMs.
Complementing these evaluation benchmarks, our pro- posed Unbiased Evaluator introduces an evaluation protocol grounded in a causal perspective, offering a more compre- hensive and unbiased assessment.
Unlike prior research that mainly focuses on biases in LLM- as-a-Judge, this paper addresses the biases inherent in the generation of Agents- aaaaututssaa-Eaaluatoo, an area largely unex- plored but critical to understanding the fairness and impact of LLMss in evaluative roles.
2.2. Benchmark Contamination
3. Bias Analysis
Recent research has attached great importance to contami- nation in LLMs. particular, (Lee et al., 2021; Sainz al., 2023; McIntosh al., 2024; Riddell et al., 2024; Jiang et al., 2024) knowledged that contamination poses significant chal- lenges to the reliability and validity of LLM evaluations. Several research studies (Ni et al., 2024) developed various methods to detect data contamination.
3.1. Theoretical Analysis
Formulation of Evaluation Bias. Consider in an LLM evaluation, where the true capability of a model parame- terized by é, which a fixed but unknown quantity we aim to estimate. Given evaluation data X' {-1, Ø2, ... (nn) the estimator á = ( is function of the data and pro- vides an approximation of ø. Then, the difference between expectation of ④ and & can be defined evaluation bias:
Several works (Fan et al., 2023; Lei al., 2023; Zhu et al., 2023; 2024: Liu et al.) have been proposed to address the contamination issue. Among these, protocols like (Fan et al., 2023; Zhu et al., 2023; Liu et al.) specifically designed for mathematical tasks, while (Lei et al., 2023) focuses on long-context evaluation. Among the research of Agents-as- un-EvaluatorrZhu et al., 2024; Liu et al.), MPA (Zhu et al., 2024) proposes to involve paraphrasing and judging agents to automatically transform existing problems in benchmarks into new ones. CogMath(Liu et al.) decouple questions into several evaluation dimensions via an multi-agent system for
E[Φ]
(1)
where [B(d) is the expected value of the estimator over X. bias greater than 0 indicates that the estimator overestimates Φ, often due to contaminated data. A bias of O means the estimator is unbiased, providing a perfect estimate. A bias less than 0 suggests the estimator underestimates Φ, which can occur when a model is evaluated on limited dataset that fails to capture the task's complexity.
3

---

Unbiased Evaluation of Large Language Models from Causal Perspective
Proposition 3.1. (Proof in Appendix D.I) Given a new designed evaluation protocol, which transforms original benchmark D into D'. Then, the strength of bias with the new evaluation protocol can be decomposed into three terms: original, related and independent term.
3.3. Probing Task
In this section, we design probing task to detect the poten- tial bias in Agents- -n----------------------- -n-Evaluator.
Minimum Sgents-as-an-Evallato To perform bias anal- ysis, following (Zhu et al., 2024), we design a minimal Agents-as-an-Bvaluae
framework. Specifically, given an original benchmark D, an LLM is tasked with rephrasing the questions in D without altering their meaning. Formally, this process generates new benchmark, D' (L).
Ble( = + 2Cov(E(éD).
original
related
+ 20] J + EE[
(2)
Task Design. Given original benchmark D with evaluation problems { ..... Zm. and cutting edged LLMs M {81. e,....,,,, }, ). D', denotes the rephrased benchmark using i-th LLM A, rephrased j-th evaluation problem d'y in D' is rephrased from dj with probability p. i.e.
independent
where, A is the delta bias which arises from the introduction of new evaluation protocol.
)] is an original term that is associated with the original bias existing in the original benchmark D.
2Cov(e(@D), A) is a related term which pertains to biases that newly introduced biases are correlated with the pre-existing biases in the original benchmark.
D',
dm random S p Vely E D',
(1()))
z/1 Iz
(3)
random > p
( ][[ + E is an independent term, stem- ming from biases inherent to the methodology itself.
Note that during probing task, the ground truth label of each problem remains unchanged. With D',. we perform LLM- as-a-Judge using all models in M. Specifically, we ask each model to assess its confidence that it thinks the ground truth label is still the right answer for the rephrased question, i.e.
Proposition 3.1 offers pivotal perspective for future re- search in designing evaluation protocols that any newly introduced biases should ideally mitigate and, more impor- tantly, counteract existing biases in the original benchmark.
3.2. Bias in Leents-as-an-Evaluee
Sisk = 0x(gg) Vi [1, [1, ml: [1,
In addition to our theoretical analysis, we conduct extensive experiments in this section to demonstrate that Agents-as- an-Evaluator exhibit two types of bias: data bias and model bias.
(4)
where Siyk k is the confidence score assessed by 0s for ques- tion a'y- The score S,ju lies within the range [1, 10), where higher score indicates that the model has greater confi- dence in the ground truth label being the correct answer, while lower score suggests stronger confidence that the ground truth label is not the correct answer. Detailed prompt is presented in Appendix B.
Data bias arises from an imbalance in accuracy across different domains during generation. For example, in tasks involving diverse domains, such various sub- jects in MMLU, LLMs tend to excel in domains where they already perform well while struggling signifi- cantly in domains where their performance is weaker.
Metric design. To quantify bias during LLMs evaluation, we draw inspiration from group consensus in human society, where collective opinions are often regarded more reli- able and fair compared to individual perspectives. Building this idca, we propose three metries: Re Error Rate), (ov(OOee--onfideee Rate) and Runde Confidence Rate), incorporating both self and collective judgment.
Model bias originates from inherent unfairness during evaluation. an LLM tends to generate content that aligns more with its implicit strengths, giving it an unfair advantage.
Discussion. Previous works(Chen et al.; Ye et al., 2024) have revealed the self-enhanced bias in LL--ass--Juddee i.c. LLM judges may favor the answers generated by them-
1
RoB =
(10 110-SpM S.jt) * (10- 1.00-Spp S.y) * Sgn(Stip < tu)
selves. Model bias, however, focus more on the bias of generation side in Agents-as as-an- -at---as-------aaattte Evaluator, which still re- main unexplored.
1=1
consensus intensity
all consensus

---

Unbiased Evaluation of Large Language Models from Causal Perspective
This is familiar to me!
Let me think!
Input variables
B
B
F( (T)
latent variables
BOAT
final variables
traditional evaluation
(b) unbiased evaluator
Here is multiple-choics question.
Distractor
Here is a multiple-ches question. there is no correct answer in the options, please reply with N.
Distractor Question
Which geologic proceas most likely caused the formatior of the Mount St. Holens volcano?
Answer Removal
Which geclogic process nost likely caused the formation ion of the Mount St. Heler: Volcano?
Option Shuffling
converging boundaries diverging boundaries C: transform faults rift zones
Label Replacement
rift zones
II
transform faults
Binary Transformation
III Recycle the old ammonia bottle IV
diverqing boundaries
(c) an example of BOAT
Label: A
Label: N
Figure 3. (a) Traditional evaluation methods rely static and fixed variables, suffering from contamination issues. (b) Unbiased Evaluator enhance the evaluation process by augmenting these variables through carefully designed Bags Of Atomic InTerventions (BOAT). (c) An example of BOAT. Underlined contents derived from multiple interventions.
where sgn is sign function. lu upper threshold of "NO". In this paper, following prompt in probing task, tu is set to 5. Siş is the averaged score among the other models:
hand, the Over-Confidence Rate Roe measures the nario where LLM exhibits high self-confidence in predict- ing "Yes", while the collective consensus predicts "NO", In contrast, the Under- Mader-Confidence Rate Ruc quantífies the scenario where a L.LM exhibits high confidence in predict- ing "NO", while the collective consensus predicts "Yes".
Sty - Ek=t (S
1 Roc m
Sijs (10 - S.))
Data Bias. We treat each subject in MMLU as a domain and calculate the Pearson and Kendall correlation between domain accuracy and REE across four LLMs. The results, presented in Table 1, reveal relatively strong negative cor- relation with small p-values. This indicates that LLMs tend to perform better domains with lower RcE values while facing greater challenges in domains where their perfor- mance is weaker.
conflict intensity
n II
sgn Sigj d
Sgn(Sijk tu)
k= k!=s colleosive consensus
self conCidence
1 m
RuC m
(10 Siji) Sty
Model Bias. In Fig. 2, visualize how Roo and Ryo evolve the strength parameter p in Eq. (4) increases. We observe a significant increase in Roc with growing strength, particularly for models like Llama3. --7bb--nntsn- and Yi- 34B-Chat contrast, Ruc remains relatively stable. This suggests that transitioning from the original evaluation pro- tocol (p 0) to the Agents-as-Evvan Evaluator (p 1) causes LLMs generate content that aligns closer with their im- plicit strengths (Roc), rather than diverging from them (Ruc), ultimately providing themselves with an unfair ad- vantage.
conflict intensity
sgn [Sije < tu *
sgn(Sijk > tl)
Mimi
self confidence
colleasive
where // is lower threshold of "Yes", and il is set to 6.
The Consensus-Erra Rate ReE quantifies the overall con- sensus predicting "NO", with a weight that increases as the confidence values Sijs and S.y decrease, reflect- ing higher confidence in predicting "NO". On the other
5

---

Unbiased Evaluation of Large Language Models from Causal Perspective
4. Unbiased Evaluator
4.2. Bags of Atomic Interventions
4.1. General formulation
Based on the general formulation described above, we pro- vide demo in Table 6 for a clear demonstration of our designed Bags of Atomic Interventions. Specifically, we focus on two widely used tasks: Multiple Choice Questions (MCQ) and Mathematics, For Multiple Choice Questions, we have designed six atomic interventions targeting differ- ent intervention positions within the task.
Consider evaluating an LLM 0, where the model is tasked with generating answers for a set of evaluation problems X' (2i, 22, ,.n). The ground truth labels y' fyr, 1/2....,, vn) are then used calculate performance met- rics. For each problem 2; X' and its corresponding label 9: E V/ the evaluation process can be framed as a causal analysis, where the input X' and the output Y' represent the cause and effect, respectively.
Distractor Hint. A hint is introduced in the form of an additional option indicating that no correct answer exists for the question.
The evaluation process for a question r can generally be rep- resented as a causal diagram, specifically a causal directed acyclic graph (DAG) 9 (E, V). In this representation, the nodes or vertices V = innp,, Viatent, Rt) correspond to the observed input variables Vinput, unobserved latent variables Viscont. and the ground truth label R. The edges E capture the direct causal relationships between these variables.
Distractor Question. To ensure the model thoroughly understands the question, a distractor question is intro- duced, randomly selected from the same dataset.
Answer Removal. Building upon the first strategy, some answers (including correct or not) removed and replaced with an unrelated option from other questions.
Specifically, Vinput {A, C, D} represents the input variables, which are composed of several atomic elements derived from the input samples. In widely used multiple- choice questions, atomic variables may include elements such as the instruction, answer labels, and other context- specific components. For instance, consider the question: "Is 9.8 bigger than 9.11? A: True, B: False". Here, the atomic variables include "9.8", "bigger", "9.11", "A", "B", "True", and "False". A large language model (LLM) reasons over these atomic variables to arrive at the final answer.
Option Shuffting. The order of the options is ran- domly shuffled to investigate whether the model's per- formance is influenced by their positional arrangement.
Label Replacement. The conventional option labels (A, B, C, D) are replaced with numerical labels (1, 2. 3, or I, II, III, IV) to explore whether the labeling format affects the model's output tendencies.
Binary Transformation. The stem of a multiple- choice question is combined with each of the four options to create four true/false questions.
Traditional evaluation methods, as depicted in Figure 3 (a), rely on static and fixed inputs, i.e., input variables, to inves- tigate causal effects. However, this rigid framework may in- advertently introduce contamination issues, limiting the va- lidity and robustness of the causal analysis. In contrast, our proposed approach, termed the Unbiased Evaluator, adopts more dynamic and adaptive strategy, as shown in Figure 3 (b). Specifically, given input variables and ground truth label pairs { B, D), D),RR R} for a task T, we enhance the evaluation by augmenting variables through carefully de- signed Bags Of Atomic InTerventions (BOAT), i.e. new input variable configurations, such as {{Å, B, C, C), and {[ C, LE,RR, are intervened with F(-IT). More intervened examples are presented in Appendix E.
For mathematics, we design two interventions to transform questions into True/False question and MCQ, and then com- bined with aforementioned MCQ interventions.
Question Jitter. Slightly altering the numbers a question allows an open-ended math problem to be reframed as a True/False question.
Answer Jitter. To transform an open-ended math prob- lem into multiple-choiee question, the numerical an- swer can be adjusted with minor variations.
Our Unbiased Evaluator aims to assess whether models can genuinely answer a question correctly by employing causal interventions that align with human recognition. Unbiased Evaluator not only mitigates potential contamination by incorporating diverse interventions but also offers an inter- pretable framework for evaluating LLMs. By examining counterfactual scenarios, such comparing the effects of A versus A, our method fosters deeper and more transparent understanding of model performance. We demonstrate this in the following experiments procedures.
Note that these atomic interventions can be organically com- bined to form complex interventions, providing a more com- prehensive evaluation of large language models. For in- stance, Distractor Hint can be seamlessly integrated with Answer Removal, Option Shuffling, and Label Replacement to create more sophisticated and targeted intervention.
Discussion. The design of BOAT is guided by the findings from Proposition 3.1. Specifically, the mechanisms of An- swer Removal and Binary Transformation are introduced to
6

---

Unbiased Evaluation of Large Language Models from Causal Perspective
Table 2. The performance of different L.MSs on vanilla benchmarks and Unbiased Evaluator.
ARC-C
MMLU
GSM8K
Model
Vanilla
Ours
A
Vanilla
Ours
A
Vanilla
Ours
A
GPT-4o
94.51
89.36 = 0.26
5.15
83,51
68.82 + 0.42
14.69
96.21
86.01 t 0.64
10.20
GPT-4-Turbo
96.48
89.74 = 0.13
6,74
84.10
71.81 0.28
12.29
97.73
90.18 + 0.34
7.55
Gemini-2.0
95.71
88.15 + 0.26
7.56
86.20
73.57 1 0.09
12.63
97.88
89.35 + 0.04
8.53
Qwen2.5-72B-Insttee
94.33
85.69 + 0.86
8.64
84.07
69.56 + 0.39
14.51
98.41
88.86 + 0.89
9.55
Llama3,1-700
93.99
81.23 + 0.74
12.76
80.70
61.93 0.12
18.77
95.98
82.97 + 0.66
13.01
Yi1.5-34B-Chur
93.91
71.79 + 0.45
22.12
77.72
56.70 + 0.18
21.02
91.96
69.60 + 0.92
22.36
Mistral-Large-24111
94.85
81.89 + 0.31
12.96
82.37
61.63 + 0.09
20.74
97.73
90.04 + 0.68
7.69
Ours
True
0.65
0.09
True
0.81
0.13
100
90
100
90
ORG
ong
Accuracyt9
80
880
70
60
50
40
40-
30
30-
20
7b
146
32b
726
1100
76
14b
320
72b
1100
Models
Models
Dwenn..5-72B-Instee
(b) Llama3. Instructt
(a) ARC-C(Qwenl.5) 1.5)
(b) MMLU(wenl..5
Orig
Ours
Orig
True
Truc
0.81
100
100
90
oni
70
False
0.01
0.05
False
0.01
0.04
50
40
30
True
Ours
False
True
Ours
False
70
130
70b
20
76
1:l
706
(c) Yil.5-34B-Chat
(d)
Lastral--Lage-2411
(c) ARC-C(L)
(d) MMLU(Llama2)
Figure 4. The confusion matrix original benchmarks and Unbi- ased Evaluator ARC-C.
Figure 5. The performance of Qwen1.5 series and Llama2 series models on ARC-C and MMLU, considering both original bench- marks and Unbiased Evaluator.
address biases inherent the original benchmark, thereby mitigating related term. For instance, public benchmarks like MMLU are known exhibit ambiguities, which will be alleviated with these interventions. Additionally, have made effort to minimize independent term introduced by our method (refer to the implementation details).
(Team et al., 2023). Additionally, we assess several open- source models, including Llama (Touvron et al., 2023), Mis- tral (Jiang et al., 2023), Qwen (Bai et al., 2023), and Yi (AI et al., 2024).
Implementation Details. We build our method on the widely-used OpenCompass (Contributors, 2023) evaluation framework. Following the approach in (Zhu et al., 2024), we set the generation temperature to 0 for all models and cap the output length at a maximum 1000 tokens. All evaluations are conducted in a 5-shot setting, with results averaged over 5 runs. Moreover, the interventions in BOAT are not randomly combined but follow specific constraints. We regulate the probability of each intervention to ensure balance. When applying a binary transformation to ques- tions, modifications involving phrases such as "which" or "following" were excluded. Furthermore, during the Answer Removal process, we ensured that the answers extracted
5. Experiments
5.1. Experimental Setup
Evaluated Datasets and LLMs. Following (Zhu et al., 2024), we evaluate on two widely used benchmarks for multiple-choice questions: ARC-Challenge (ARC-C) (Clark et al., 2018 8) and MMLU (Hendrycks et al., 2021). For math- ematical problem-solving we utilize the GSMSK dataset (Cobbe et al., 2021). The evaluation includes three pro- prietary large language models (LLMs): GPT-4o (OpenAI, 2024), GPT-4-Turbo (Achiam al., 2023), and Gemini 2.0

---

Unbiased Evaluation of Large Language Models from Causal Perspective
from different questions were not identical. For additional details, please refer Appendix C.
Table 3. Contamination analysis. We fine-tune I Llama2-13B on the original samples from the MMLU test set and evaluate it on MMLU test set under two conditions: with and without our Unbi- ased Evaluator.
5.2. Main Results
Table 2 presents the evaluation results of different LLMs on the original protocol and our Unbiased Evaluator, showing all LLMs experienced performance degradation the Un- biased Evaluator. Specifically, GPT-4-Turbo and Gemini 2.0 demonstrated the smallest relative performance drop, main- taining their position as the strongest models. In contrast, GPT-4o performed similarly to GPT-4-Turbo on the original protocol but exhibited a larger average performance decline, particularly on MMLU and GSM8K. We hypothesize that the omni design of GPT-4o may hinder its performance source
train set
w/o
w/
Llama2-13B
55.6
33.7
Llama2- 13B + original test set
96.6
37.1
6.3. Evaluation Reliability
Our Unbiased Evaluator demonstrates significantly stronger alignment with human expert judgments. Since obtaining comprchensive expert evaluations across multiple models is both costly and impractical, we instead benchmark our method against LiveBench (White et al., 2025), a contin- uously updated and widely recognized evaluation platform. Specifically, we compute both Pearson and Kendall correla- tion coefficients between our averaged results of Table 2 and the global average scores reported in the latest L LiveBench release (2024-11-25). To ensure fair comparison, we ex- clude two models GPT-4-TTubbl and Vi1.5-34B-CHB as they are not covered in the corresponding LiveBench update.
NLP tasks. Compared to proprietary models, open-source ones showed a more significant average drop, with Yi1.5- 34B-Chat standing out as particularly affected, suggesting a substantial potential for improvement.
6. Ablations
6.1. Bias Analysis
Following Sec. 3.3, we conduct an analysis of data and model bias for our proposed method, with the results pre- sented in Table and Fig. 2. Regarding data bias, our method exhibits a smaller coefficient and a large p-value, in- dicating reduced data bias. In terms of model bias, compared to Agents-as asEvvaluaoo,, our Unbiased Evaluator remains relatively stable in both Roc and Ruc the strength in- creases, suggesting lower model bias.
Table 4. Evaluation Reliability. We compute both Pearson and Kendall correlation coefficients between our averaged results of Ta- ble 2 and the global average scores reported in the latest LiveBench release.
Pearson
Kendall
6.2. Contamination Analysis
Vanilla
0.918
0.600
Unbiased Evaluator
0.949
1.000
Our Unbiased Evaluator aims to assess whether models can genuinely answer question correctly by employing causal interventions that align with human recognition. Therefore, Unbiased Evaluator provides a more accurate measure of a model's true and robust performance on a given bench- mark by eliminating performance inflation caused by data contamination. The decline of accuracy in Table 2 actu- ally reflects the decrease of contamination. To validate this, following (Yang et al., 2023), we further provide addi- tional fine-tuning ablation study. Specifically, we fine-tune Llama2- 13B on the original samples from the MMLU test set and evaluate on MMLU test set under two conditions: with and without our Unbiased Evaluator.
Results in Table 4 confirm that our method aligns more closely with LiveBench. Notably, it achieves perfect rank- ing correlation with LiveBench (as measured by Kendall), a significant improvement over baseline. Unlike LiveBench, which covers diverse tasks and requires substantial resources to update questions regularly, ours leverages existing bench- marks and requires almost no additional resources.
6.4. Confusion Matrix Analysis
We present a comprehensive confusion matrix Fig. 4, which evaluates performance across four dimensions: "True/False" labels from the original protocol and our Unbi- ased Evaluator. For instance, the category labeled "Original True" and "Unbiased Evaluator False" highlights scenarios where the model performs correctly under the original pro- tocol but fails when assessed with the Unbiased Evaluator. A striking observation across all models is the high fre- quency of "Original True" and "Unbiased Evaluator False"
Results in 3 highlight that our Unbiased Evaluator provides more rigorous assessment of benchmark contamination. Even when trained directly the original test set, the model struggles to perform well under the Unbiased Evaluator, suggesting that it effectively mitigates data contamination and ensures a more robust evaluation.
8

---

Unbiased Evaluation of Large Language Models from Causal Perspective
6.7. Human Verification
90
To ensure the correctness of proposed method, we randomly selected 300 samples from the MMLU and 100 samples from the ARC-C and GSM8K, 500 questions in total. human experts (with bachelor or higher degree) are divided into 3 groups, each with 3 person. They were asked to judge the following question: Whether the answers to the intervened questions are correct. As shown in Table 6.7, the human verification demonstrates an overall accuracy rate of 99.3%, 99.9% and 99.7% for ARC-C, MMLU and GSM8K, respectively, indicating the effectiveness of our methodology.
80
Accuracy
70
60
50
40
orig
DQ
DH
AE
05
LR
BT
All
Interventions
Figure 6. Accuracy of Qwen2.5-7B 10e-7B-Innttrn Instruct ARC-C when each single atomic interventions are applied. Orig denotes the original benchmark. DQ DH, AE, OS, LR and BT represents Distractor Hint, Distractor Question, Answer Removal, Option Shuffling, Label Replacement and Binary Transformation, respectively.
Table 5, Results of human verification ARC-C, MMLU and GSMSK datasets.
ARC-C
MMLU
GSM8K
Average
Group
1.000
0.997
0,990
0.996
Group 2
0.990
1.000
1.000
0.997
instances. This trend suggests issues with data contami- nation and inherent capability limitations within the mod- els. Notably, the "Original True" and "Unbiased Evaluator False" of Yi1.5-34B- 3httChat are significantly higher compared to other larger models. This indicates that it may be more susceptible to data contamination and capability constraints.
Group 3
0.990
1.000
1.000
0.997
Average 0.993
0.999
0.997
/
7. Future Work
Our proposed method can be easily extended towards open- ended tasks. Most tasks (e.g., multiple-choice, math), as demonstrated in this paper, inherently follow natural rules in either the questions or answers, and rule-based interventions can be automatically applied. The other small percentage of tasks, can use debiased Agents-as-Eae as-Evaluator version. Concretely, our study has revealed the data and model bi- ases of previous version, inspiring two designs to mitigate them: (1) Cross-generation: to reduce model bias, we can break down question generation into multiple chunks, using different models for each. (2) Cross-checking: multiple advanced models can be used to cross-check the output to mitigate data bias and enhance quality. Overall, our method is easily scaled most tasks, and our insights will provide valuable inspiration for future advancements in evaluation methodologies. We leave these directions to future work.
Effeec of Interventions & Evaluation Interpretability
Figure 6 illustrates the accuracy of Srren..5-7-IIntrrue when each single atomic interventions are applied. The re- sults show that the model's performance varies significantly depending on the type of intervention. Notably, compared to the original protocol, the Binary Transformation interven- tion poses the greatest challenge, leading to substantial decline in accuracy. This suggests that the model may not fully comprehend every option in MCQ but instead might rely on certain heuristic shortcuts to arrive at an answer. Moreover, even with relatively simple interventions, such as Option Shuffling and Label Replacement, the model's s performance exhibits a slight degradation. This provides strong evidence of potential data contamination.
6.6. Effect of the Scaling
8. Conclusions
As shown in Figure 5, we evaluate the Qwen ..5 and Llama2 series models with Unbiased Evaluator on MMLU and ARC- C. The results indicate that as the model parameters increase, performance gradually improves. Notably, under the origi- nal evaluation protocol, the larger Qwen 1.5 models achieved over 90% performance, nearing saturation and limiting the ability to evaluate larger models, such as 32/72/1 10B. In contrast, Unbiased Evaluator demonstrate a consistent per- formance improvement from 32B to 10B.
This paper present a theoretical analysis of evaluation bias, offering valuable findings for designing protocols. More- over, two types of bias in Agents-as-an-Evaluual are iden- tified with probing task. To mitigate the bias, guided with previous findings, an new evaluation protocol, Unbiased Evaluator, is proposed to offer unbiased and interpretable assessment for benchmark contamination. We look forward that our method may bring inspirations for future design for L.LMs evaluation.

---

Unbiased Evaluation of Large Language Models from Causal Perspective
Impact Statement
Chen, G. H., Chen, S., Liu, Z., Jiang, F., and Wang, B. Humans Ilms the judge? study on judgement biases, 2024, URL https://arxiv. org/abs/2402, 10669.
Bias in the evaluation LLMs is a crucial issue for ensuring responsible AI development in society. This work conduct detailed bias analysis on previous protocol, followed by novel evaluation protocol designed to fairly measure the true capabilities of LLMs. By providing a more precise assessment framework, this protocol aims to enhance our understanding of these models, ultimately contributing to more transparent, fair, and reliable Al systems.
Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., and Tafjord, O. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint /803. 2018
Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al. Training verifiers to solve math word problems. arXiv preprint (..Xx:21100144000 2021.
References
Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, L., Aleman, F. I.,, Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., al. Gpt-4 technical report. arXiv preprint arXiv:2303.00 087744 2023.
Contributors, O. Opencompass: A universal evaluation platform for foundation models. https: https://githinn /githhb. com/ hee//ppn-ccccc compass/openconeeeic
2023.
Dai, S., Xu, C., Xu, S., Pang. L., Dong, Z., and Xu, J. Unify- ing bias and unfairness in information retrieval: A survey of challenges and opportunities with large language mod- els. arXiv preprint arXiv:2404. 1457, 2024.
Ahn, J., Lee, H., Kim, J., and Oh, A. Why knowledge distillation amplifies gender bias and how to mitigate from the perspective of distilbert. In Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP), pp. 266-272,2022
Fan, L., Hua, W., Li, L., Ling, H., and Zhang, Y. Nphard- eval: Dynamic benchmark on reasoning ability of large language models via complexity classes. arXiv preprint arXiv:2312. 0at:2232..44880 148900 2023.
i, Young, A., Chen, B., Li, C., Huang, C., Zhang, G., Zhang, G., Li, H., Zhu, J., Chen, J.. Chang, J., Yu, K., Liu, P., Liu, Q., Yue, S., Yang, S., Yang, S., Yu, T., Xie, W., Huang, W., Hu, X., Ren, X., Niu, X., Nie, P. Xu, Y., Liu, Y., Wang, Y., Cai, Y., Gu, Z., Liu, Z., and Dai, Z. Yi: Open foundation models by 01.ai. 2024.
Ferrara, E.
Should chatgpt be biased? challenges and risks of bias in large language models. arXiv preprint arXiv:2304.03736 2023.
AI-MO. ame 2023. https: Datssets/AL--0//anooovllllaa 2023.
co/
Gallegos, O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., and Ahmed, N. K. Bias and faimess in large language models: A survey. Computational Linguistics, pp. 1-79, 2024.
AI-MO. Aime 2024. https::/hugginggfat co/ MAAA-M//aio----a -ime, 2024.
He, C., Luo, R., Bai, Y., Hu, S., Thai, Z. L.. Shen, J.. Hu, J., Han, X., Huang, Y., Zhang, Y., et al. Olympiadhench: A challenging benchmark for promoting agi with olympiad- level bilingual multimodal scientific problems. arXiv preprint arXiv:2402. /4008, 2024a.
Anthropic.
The claude model family: Opus, sonnet, haiku. 2024. URL http:t//ppp...emmaaicicoo.. www.sticscoonoooaaaoooooooccc http://oppusI:::222222222
He, Y., Li, S., Liu, J.. Tan, Y., Wang, W., Huang. H., Bu, X., Guo, H., Hu, C., Zheng, B., et al. Chinese simpleqa: A chinese factuality evaluation for large language models. arXiv preprint arXiv:241 1.07111.114040 2024b.
Bai, J., Bai, S., Chu, Y., Cui, 7., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., Huang, F., et al. Qwen technical report. arXiv preprint arXiv:2309. 16609, 2023.
Bender, E. M., Gebru, T., McMillan- Major. A., and Shmitchell, S. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM conference on fairness, accountability, and trans- parency, pp. 610-623, 2021.
Hendryeks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song. D., and Steinhardt, Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR), 2021.
Jiang, A. Q. Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., Casas, D. d. L., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., et al. Mistral 7b. arXiv preprint arXiv:2310 06826.0 2023.
Blodgett, S. L., Barocas,
S., Daumé III, H., and Wallach, H. Language (technology) is power: A critical survey of" bias" in nlp. arXiv preprint arxiv22005. /4050, 2020.
10

---

Unbiased Evaluation of Large Language Models from Causal Perspective
Jiang M., Liu, K., Zhong, M., Schaeffer, R., Ouyang, S., Han, J., and Koyejo, S. Does data contamination make difference? insights from intentionally contaminating pre-training data for language models. In ICLR 2024 Workshop on Navigating and Addressing Data Problems for Foundation Models, 2024.
Sainz, O., Campos, J. A., Garcia-Ferrero, I., Etxaniz, J., de Lacalle, O. L.,, and Agime, E. NIp evaluation in trouble: On the need to measure Ilm data contamination for each benchmark. arXiv preprint arXiv:2310. 18018, 2023.
Srivastava, A., Rastogi, A., Rao, A., Shoeb, A. A. M., Abid, A., Fisch, A., Brown, A. R., Santoro, A., Gupta, A., Garriga-Alonso, A., et al. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv preprint arXiv:2206.cr 22..2200000045.5 04615, 2022.
Kocoń, J., Cichecki, I., Kaszyca, O., Kochanek, M., Szydlo, D., Baran, J., Bielaniewicz, J., Gruza, M., Janz, A., Kan- clerz, K., et al. Chatgpt: Jack of all trades, master of none. Information Fusion, 99:101 2023.
Team, G., Anil, R., Borgeaud, S., Alayrae, J.- Yu, J., Sori- cut, R., Schalkwyk, J., Dai, A. M., Hauth, A., Millican, K., et al. Gemini: family of highly capable multimodal models. arXiv preprint arXiv:2312.
Lee, K. Ippolito, D., Nystrom, A.. Zhang, C., Eck, D., Kallison-Burchhe C., and Carlini, N. Deduplicating train- ing data makes language models better. arXiv preprint arXiv:2107. 00..06499 2021.
2023.
Touvron, H., Martin, L.,, Stone, K., Albert, P., Almahairi, A., Babaci, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama Open foundation and fine- tuned chat models, arXiv preprint arXiv:2307.09288. 2023.
Lei, F. Liu, Q. Huang, Y. He, S., Zhao, J., and Liu, K. S3eval: synthetic, scalable, systematic evalua- tion suite for large language models. arXiv preprint 1rXiv:2310.15147, 2023.
Wang, A. Glue: A multi-task benchmark and analysis plat- form for natural language understanding. arXiv preprint 1al1::8804.77455 2018.
Li, Y. An open source data contamination report for Ilama series models. arXiv preprint arXiv:2310. 17589, 2023.
Liang, P., Bommasani, R., Lee, T., Tsipras, D., Soylu, D., Yasunaga, M., Zhang, Y., Narayanan, D., Wu, Y., Kumar, A., et al. Holistic evaluation of language models. arXiv preprint arXiv:221 1.091 10, 2022.
Wang, A., Pruksachatkun, Y. Nangia, N., Singh, A.. Michael, J., Hill, F., Levy, O., and Bowman, S. Super- glue: stickier benchmark for general-purposs language understanding systems. Advances in neural information processing systems, 32, 2019.
Lightman, H., Kosaraju, V., Burda, Y., Edwards, H., Baker, B., Lee, T., Leike, J., Schulman, J., Sutskever, I., and Cobbe, K. Let's verify step by step. arXiv preprint arXiv:2305.20055: 2023.
Wei, J., Karina, N., Chung, H. W., Jiao, Y. J., Papay, S., Glaese, A., Schulman, J., and Fedus, W. Measuring short- form factuality in large language models. arXiv preprint arXi::2411.04366 2024.
Liu, J., Huang, Z. Dai, W., Cheng, C., Wu, J., Sha, J., Liu, Q., Wang, S., and Chen, E. Cogmath: Evaluating llms' authentic mathematical ability from a cognitive perspective.
White, C., Dooley, S., Roberts, M., Pal, A., Feuer, B., Jain, S., Shwartz-Ziv, R., Jain, N., Saifullah, K., Naidu, S., et al. Livebench: A challenging, contamination free Ilm benchmark. arXiv preprini arXiv:2406. 19314 2024.
Lovin, B. Gpt-4 performs significantly worse on coding problems not in its training data, 2023.
White, C., Dooley, S., Roberts, M., Pal, A., Feuer, B., Jain, S., Shwartz-Ziv, R., Jain, N., Saifullah, K., Dey. S., Shubh-Agrawal, Sandha, S. S., Naidu, S. V., Hegde, C., LeCun, Y., Goldstein, T., Neiswanger, W., and Goldblum, M. Livebench: challenging, contamination-fe LLM benchmark. In The Thirteenth International Conference on Learning Representations, 2025.
McIntosh, T. R., Susnjak, T., Arachchilage, N., Liu, T., Wat- ters, P., and Halgamuge, M. N. Inadequacies of large language model benchmarks in the era of generative ar- tificial intelligence. arXiv preprint arXiv:2402.09888) 2024.
Ni, S., Kong, X., L. C., Hu, X., Xu, R., Zhu, J., and Yang, M. Training on the benchmark is not all you need. arXiv preprint 4rXiv:2409, 00090,, 2024.
Yang. S., Chiang, W.-L., Zheng, L., Gonzalez, J. E., and Stoica, I. Rethinking benchmark and contamination for language models with rephrased samples. arXiv preprint arXiv:2311.0483 2023.
OpenAL. Chatgpt, 2024. URL https: /opp:ai./ppp/.////ee com/ index/hello pypt-40/.
Ye, J., Wang. Y., Huang, Y., Chen, D., Zhang, Q. Moniz, N., Gao, T., Geyer, W., Huang, C., Chen, P.-Y,, et al. Justice or prejudice? quantifying biases in Ilm-as-a-judee arXiv preprint arXiv:2410.027344 2024.
Riddell, M., Ni, A., and Cohan, A. Quantifying contamina- tion in evaluating code generation capabilities of language models. arXiv preprint arXiv:2403.044 048//, 2024.

---

Unbiased Evaluation of Large Language Models from Causal Perspective
Zheng, L.,, Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z-, Li, Z.,, Li, D., Xing, E.,, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Ad- vances in Neural Information Processing Systems, 36: 46595-46623, 2023.
Zhou, K., Zhu, Y., Chen, Z.,, Chen, W., Zhao, W. X., Chen, X., Lin, Y., Wen, J.-R., and Han, J. Don't make your Ilm an evaluation benchmark cheater. arXiv preprint 1rXiv:2311.01960 2023.
Zhu, K., Chen, J., Wang. J., Gong, N. Z., Yang, D., and Xie, X. Dyval: Graph-informed dynamic evaluation of large language models. arXiv preprint arXiv:2309.17161 2023.
Zhu, K., Wang, Zhao, Q. Xu, R., and Xie, X. Dynamic evaluation of large language models by meta probing agents. In Forty-first International Conference on Ma- chine Learning, 2024.
12

---

Unbiased Evaluation of Large Language Models from Causal Perspective
A. Demo Case of BOAT
Table Bags of Atomic Interventions (BOAT). This demo case clear demonstration. For further details, please refer to Sec. 5.1. For each intervention, we present simple question to show its effect. The "intervened" column only displays the part that has changed, highlighted the same color as the original part it replaces. The other parts of the content remain unchanged.
examples
T
positions
interventions
original
intervened
label
instruction
Distractor Hint
Here is a multiple choice question, answer A or B, if there no answer, reply
Here is multiple choice question, answer
MCQ
question
Distractor Question
Is 9.8 bigger than 9.11?
B. Is 9.8 bigger
1+1-?
Answer Removal Option Shuffling
A: False
answer
A: False B: True
Label Replacement
than 9.117
I: True II: False
overall
Binary Transformation
A: True B: False
Here is a multiple choice question, judge True (T) or False (2) for each choice. Judge True (T) or False (F) for given possible answer. 1+2=22
TF
overall
Question Jitter
Math
Answer this question. 1+1=?
overall
Answer Jitter
Here is multiple choice question. 1+1=? A: 0 B: 1 C: 2 D: 3
C
B. Prompt in Probing Task
have question with possible answer.
### Question: [question)
### Possible Answer: (answer)
You need to rate the given possible answer on scale of to 10 based on the confidence of its correctness for the question, using the following rating rules:
Score to 2: You are very confident that the given possible answer completely
incorrect for the question.
Score 3 to 4: You are fairly confident that the given possible answer is incorrect, but there is small chance it could be partially correct. Score to 6: You
uncertain about the correctness of the given possible anawer; it could be right or wrong.
Score 7 to 8: You are fairly confident that the given possible answer is correct, but is completely correct Finally, you must rate the answer strictly on scale of to 10 with in the format of 'eec >>>'', for example, "<<<5>>>,"'
there is small chance it could be partially incorrect.
Score 9 to 10: You are very confident that the given possible
for the question.
13

---

Unbiased Evaluation of Large Language Models from Causal Perspective
Table The BOAT constraints represented in matrix where each cell indicates whether the second intervention (column) can he applied when the first intervention (row) is introduced. A check mark (v) denotes that the second intervention can be combined with the first one, while a black triangle indicates the second intervention a required addition. DQ. DH, AE, OS, LR BT represents Distractor Hint, Distractor Question, Answer Removal, Option Shuffling, Label Replacement and Binary Transformation, respectively.
First/Second Intervention
DH
DQ
AR
OS
LR
BT
DH
DQ
V
AR
V
OS
d
LR
d
BT
C. Implementation Details
C.1. Hyperparameter
To ensure balance of interventions in BOAT, the probability for each intervention was set to 0.5, except for Binary Transformation, which was assigned probability of 0.1. Under the 5-shot evaluation setting, we introduced the same combinations of atomic interventions to the few-shot samples as those in the final question. An exception was made for questions affected by Answer Removal: in such cases, we randomly selected half of the few-shot samples for intervention to prevent the model from repeating the output corresponding option N.
C.2. BOAT constraints
As shown in 7, there are constraints among different interventions, and not all interventions can combined arbitrarily, For example, the Answer Removal can only introduced when the Distractor Hint intervention exists. Moreover, the binary transformation intervention cannot be combined with other interventions.
Please note that the instructions will change depending the interventions. The detailed instructions are presented in 8.
Table 8. The detailed instructions under different interventions. DQ, DH, AE, OS, LR and BT represents Distractor Hint, Distractor Question, Answer Removal, Option Shuffling, Label Replacement and Binary Transformation, respectively.
Intervention
Instruction
DH
If there is no correct answer in the options, please reply with N. Here are two questions and only one of them corresponds to the options. Please select the correct answer. Here is a multiple choice question.
DQ
AR
OS
Here multiple choice question. Here is multiple choice question. The following are true/false questions. If the answer is correct, please reply with T, otherwise reply with F.
LR
BT
D. Proof of Proposition
D.I. Proof of Proposition 3.1
Given the definition in 3.1, the evaluation bias on benchmark D is given by:
<(Ōp)
E[δn]
14

---

Unbiased Evaluation of Large Language Models from Causal Perspective
Similarly, the evaluation bias rephased one D' from D defined as ep', then
<lδD') El Φb Elòd'| - (BlâD) - c(δD))
(Blån/] Bløn!) <(OD)
= A + ((0p)
where the delta bias which arises from the introduction of new evaluation protocol. Consider that <(δdx) could be positive, negative, and ideally, zero. Therefore, we seek to analyze the mean squared term:
Ele((dn))) = E[(A + {())))
R[A?] +
(10)
Because,
Cov(e(eD), A) E[(e(ŌD) B(((é)))) B[A]]
E[((dD) c(òb)E[A) AB(((OD) + E(c(A
[]((OD) E[((ó)) [[A]
[[A] - [[A]E(((dD]] E(e(d)
(11)
Therefore,
Blc(ôn) = Cov(e( A) + B((()))]E[A)
(12)
Substitute the expression into Equation (10).
Ble(@n)) B[(( + 2Cov(e(ên). +) E[A?]
(13)
Here,
Eje(òd)?) is an original term that is associated with the original bias existing in the original benchmark D.
2Cov(e(δD), A) is related term which pertains to biases that newly introduced biases correlated with the pre-existing biases in the original benchmark.
[B]{()))][[A] + [[ is an independent term, stemming from biases inherent the methodology itself.
E. Intervened Examples
Here is multiple choice question. IF there is correct in the options, please reply with N.
Question:
The end result in the process of phetosynthesis is the production of sugar and oxygen. Which step signals the beginning of photosynthesis? I. Chemical energy is absorbed through the roots.
II. Light energy is converted to chemical energy. III. Chlorophyl1 in the leaf captures light energy.
IV. Sunlight is converted into chlerophyll.
Answer:
III
Example #1
Here are two questions and only of them corresponds to the options. Please select the correct answer.
Question:
15

---

Unbiased Evaluation of Large Language Models from Causal Perspective
group of engineers wanted to know how different building designs would respond during an earthquake. They made several models of buildings and tested each for its ability to withstand earthquake conditions. which will most likely result from testing different building designs?
The voltage is held constant in electric circuit. What will happen to the current in this circuit if the resistance doubled?
A. buildings will be built faster
B. buildings will be made safer
C. building designs will look nicer
D. building materials will be cheaper
Answer:
8
Example #2
Here are two questions and only one of them corresponds to the options. Please select the correct answer. If there is no correct answer in the options, please reply with N. Question:
which statement best explains why what happens to a wooden log when it burned?
branch floats on water?
I. Wood is light.
II. parasitism
III. Wood is magnetic.
IV. Wood is porcus. Answer: N
Example #3
The following are true/false questions. If the answer is correct, please reply with T, otherwise reply with F.
Question:
statement A polished metal ball looks very shiny and bright on a sunny day. what makes the ball look shiny? The ball makes light.
Statement 2: A polished metal ball looks very shiny and bright on a sunny day. What makes the ball look shiny? The ball reflects light. statement A polished metal ball looks very shiny and bright
sunny day. what makes the ball look shiny? The ball absorbs light and then releases it. statement 4: A polished metal ball looks very shiny and bright on a sunny day. What makes the ball 1ook shiny? The ball absorbs light and keeps it inside.
Answer:
Example #4
16


Reviews:
[{"review_id": "kdrEuOSiAT", "reviewer": "Reviewer_D15D", "summary": "This paper presents Bags of atomic interventions (BOAT) to address the data contamination problem in LLM evaluation. It first develops a theoretical formulation of evaluation bias, and identity the data and model bias in agents-as-an-evaluator paradigm. It then proposes the unbiased evaluator to help evaluate LLMs with less bias.\n\n## update after rebuttal\nIn my initial comment, I mainly question the justification of BOAT. During the rebuttal, the author has thoroughly addressed this concern, so I updated my score to support the work.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Response to reviewer D15D\n> **Q1**: The unbiased evaluator heavily depends on the BOAT, which is hand-designed, and how these principles are hand-designed to fully follow the theoretical framework.\n\n**A1**: **The design of the Unbiased Evaluator is grounded in our theoretical findings** (see the detailed discussion in L366). In particular, Proposition 3.1 shows that the bias in the new evaluation protocol can be decomposed into original, related, and independent terms. For the related term, BOAT’s interventions help mitigate the biases present in the original benchmark (such as ambiguities), thus reducing the impact of the related term in Proposition 3.1. Additionally, the independent term is minimized by our rule-based design.\n\nOverall, this paper provides the first comprehensive bias analysis for Agents-as-an-Evaluator and designs a simple unbiased alternative guided by our theoretical insights. Our theoretical insights, as well as bias analysis, will inspire future design for LLM evaluation.\n\n\n> **Q2**: Discussion with previous paper [1]\n\n**A2**: We argue that our paper differs from paper [1] in the following aspects:\n\n- **Different focus, findings, and methodologies**: First, paper [1] mainly addresses **contamination**, specifically the inclusion of rephrased test samples in training data. In contrast, our work focuses on **evaluation bias** in the Agents-as-an-Evaluator paradigm (rephrasing is a special case of Agents-as-an-Evaluator). Second, while paper [1] utilizes an LLM-based decontaminator to identify rephrased samples, we take a fundamentally different approach by mitigating evaluation bias through causal interventions.\n\n- **Our method naturally extends and advances the contributions of paper [1]:** Paper [1] highlighted the challenges in contamination formulation for future works in the end (see Section 6.1), such as mathematical cases where a training and test example differ only in numerical values and background details. As outlined in our general evaluation formulation (L253), we formulate the evaluation process into a causal analysis, and it is crucial to assess whether the model is genuinely capable of these causal connections. Based on this, a robust and contamination-free evaluation protocol should determine whether the model truly possesses the ability to answer the questions correctly. Our proposed Unbiased Evaluator achieves this by assessing the model’s responses under various causal combinations of Bags of Atomic Interventions (BOAT).\n\nFor a more comprehensive understanding of the Unbiased Evaluator, following the contamination detection methodology in [1], we perform an evaluation of the fine-tuned model using our approach. Specifically, we fine-tune Llama2-13B on both rephrased and original samples from the MMLU test set and evaluate it on MMLU test set under two conditions: with and without our Unbiased Evaluator. In particular, the results in parentheses are the results from Table 2 of paper [1].\n\n|train set|w/o Unbiased Evaluator|w/ Unbiased Evaluator|\n|----------------------|---------|---------|\n|Llama2-13B|55.6 (54.8)|33.7|\n|Llama2-13B + rephrased test set|85.7 (85.9)|32.8|\n|Llama2-13B + original test set|96.6 (100)|37.1|\n\nThese results highlight that our Unbiased Evaluator provides a more rigorous assessment of benchmark contamination. Even when trained directly on the original test set, the model struggles to perform well under the Unbiased Evaluator, suggesting that it effectively mitigates data contamination and ensures a more robust evaluation.\n\nOverall, grounded in our theoretical findings and the first bias analysis for Agents-as-an-Evaluator, Unbiased Evaluator is designed to provide a more robust and unbiased assessment for benchmark contamination. We sincerely appreciate your valuable suggestions and will cite paper [1] and include the discussions above into the related works section. \n\nIf our rebuttal successfully addresses your concerns, we kindly ask you to consider raising our score.\n\n[1] Rethinking Benchmark and Contamination for Language Models with Rephrased Samples", "claims_and_evidence": "Yes or no.\n\nOne of the major claim in the paper is supported by Table 3 which indicates the contamination problem in the current benchmark. The unbiased evaluator heavily depends on the BOAT, which is hand-designed (Section 4.2). The reviewer is not fully aware of how these principles are hand-designed to fully follow the theoretical framework.", "methods_and_evaluation_criteria": "Yes, the paper uses ARC-C and MMLU, GSM8K for evaluation, gpt-4 and gemini, llama, mistral, qwen and yi for models. The reviewer is convinced these choices are reasonable.", "theoretical_claims": "Yes, the reviewer checks the theoretical analysis in Section 3.", "experimental_designs_or_analyses": "The experimental designs largely make sense, but the reviewer is not convinced of the derivation of BOAT.", "relation_to_broader_scientific_literature": "The paper largely cited proper papers. However, the reviewer believes there is a popular and similar work that the paper does not discuss [1]. Can the author includes the discussion against this paper?\n\n[1] Rethinking Benchmark and Contamination for Language Models with Rephrased Samples.", "essential_references_not_discussed": "Please see above comments.", "other_strengths_and_weaknesses": "Other strength:\n The paper is addressing an important problem and attempt from a theoretical perspective. The major weakness is the justification of BOAT and does not distinguish against the above paper. The reviewer is willing to raise the score they can be addressed adequately.", "other_comments_or_suggestions": "Please see the above comments.", "questions_for_authors": "Please see above questions.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jxR1PieAIX", "reviewer": "Reviewer_3rKe", "summary": "The paper introduces ‘Agent as an evaluator’ paradigm with the goal to increase the robustness of LLM-as-a-Judge based evaluations. The evaluation protocol introduces the ability to test model and data bias by taking an active/intervening (agentic) process of evaluating the benchmarks. The query breakdown focuses on problem rephrasing to assess stability of responses. The authors design probing tasks to identify various contamination effect biases. These tasks are designed to reveal data and model biases, informing the development of the Unbiased Evaluator. The work is somewhat inspired from similar research like CogMath where CogMath formalizes the reasoning process into three stages: problem comprehension, problem solving, and solution summarization - however this research generalizes to all other of evaluations as well as introduces error breakdown and analysis with theoretical underpinning and shows strong correlations with performance metrics using statistical metrics.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Response to reviewer 3rKe\n> **Q1**: The paper writing and organization can be improved a lot. The paper starts and shows the cryptic Fig. 1 and non-standard Fig 2 (and talks about Fig 2 much later) - the definitions are vague (still confused how strength parameter is varies during evaluations)\n\n**A1-part1 (for Figure 1):**\nFigure 1 illustrates the overall pipeline of Agents-as-an-Evaluator and our proposed Unbiased Evaluator.\nThe Agents-as-an-Evaluator process (Figure 1a) consists of generation and evaluation stages. The generation phase is affected by data bias (Table 1), where LLMs tend to perform generation task (such as rephrasing) significantly worse in domains where their evaluation performance are weaker. Furthermore, the evaluation phase involves model bias — LLMs generate content that aligns more closely with their strengths, giving themselves an unfair advantage (represented by term \"familiar\"). In contrast, our proposed Unbiased Evaluator (Figure 1b) evaluates the LLMs with designed BOAT. \nConsidering that Figure 1 may lead to potential misinterpretation, **we provide a simplified version (refer to https://i.imgur.com/XpnwLsk.jpeg)**. We will update Figure 1 with this version and provide a more detailed caption to enhance clarity.\n\n\n**A1-part2 (for Figure 2 and strength parameter):**\nFigure 2 visualizes the variation of two proposed metrics ($R_{OC}$ on the left and $R_{UC}$ on the right) as strength parameter changes, using two datasets (MMLU and ARC-C). As stated in L267, **strength refers to the probability defined in Equation 3. A higher strength value indicates a greater proportion of \"processed\" samples within the dataset** (\"process\" denotes rephrasing and BOAT in Agents-as-an-Evaluator and Unbiased Evaluator, respectively).\nAs the strength increases, for Agents-as-an-Evaluator, we observe a significant rise in $R_{OC}$, while $R_{UC}$ remains relatively stable, suggesting the existence of model bias. In contrast, our Unbiased Evaluator remains relatively stable on both metric.\n\nWe will incorporate this clarification of the strength parameter into the caption and relocate Figure 2 closer to Section 3.3 for better alignment.\n\n> **Q2**: There is very little (almost no) comparison to other work in the field. For example, authors refer to Ye et al. 2024 work ( https://arxiv.org/pdf/2410.02736) where they refer to various biases coming from LLM-as-a-Judge and how the work has defined Robustness rate and consistency rate metrics to assess some of these biases - how do these compare with this work or other relevant work?\n\n**A2**: As demonstrated in A1, we **DO** compare our method with previous relevant works on two widely-used benchmarks (MMLU and ARC-C), considering both types of bias. Building upon the theoretical findings and the first comprehensive bias analysis of Agents-as-an-Evaluator, our method is designed as an unbiased LLM evaluation protocol. Therefore, we compare Unbiased Evaluator with previous Agents-as-an-Evaluator on both data (Table 1) and model bias (Figure 2).\n\nAs for previous evaluation bias works, such as [Ye et al. 2024]( https://arxiv.org/pdf/2410.02736), we have presented detailed discussion on Section 2.3 and L197. Prior works mainly focus on biases in LLM-as-a-Judge, which operates on the judge side by solely determining whether input falls within the scope of a given rule (e.g. score range 0~5). In contrast, our paper is the first to address the biases inherent in the generation side of Agents-as-an-Evaluator, where LLMs actively contribute to the generation of the very questions.\n\n> **Q3**: \"Agents-as-an-Evaluator\" seems like a new and slightly confusing term and needs some definition\n\n**A3**: Integrating agents into the evaluation process is a very recent research direction. Building on the concept of LLM-as-a-Judge, we introduce the term Agents-as-an-Evaluator and have clarified its distinction from LLM-as-a-Judge in L48-L53. Formally, Agents-as-an-Evaluator refers to an LLM-based evaluation paradigm in which LLMs (or Agents) not only assess responses but also actively contribute to generating evaluation criteria and questions. We will incorporate this formal definition into the introduction for better clarity.", "claims_and_evidence": "The contributions (according to authors) are as follows: \n- A theoretical formulation of evaluation bias, offering valuable findings for the importance of minimizing the relative term when designing evaluation protocols. \n- The first comprehensive bias analysis for Agents-as-an-Evaluator, revealing data and model bias which undermine the reliability and trustworthiness of Agents-as-an-Evaluator. \n- An unbiased evaluation protocol, Unbiased Evaluator, provides a more comprehensive, unbiased and interpretable assessment for benchmark contamination. \n\nThe claims are backed with proofs, design of experiments and various results and analysis to validate the claims. There are some similar work that the authors have attributed to in this paper.", "methods_and_evaluation_criteria": "The Unbiased Evaluator employs 'BOAT' based probing method to dynamically assess LLMs, aiming to reduce evaluation biases (unlike the baseline evaluation that the authors say give an unfair advantage to larger LLMs that show higher over-confidence, for example). This method seeks to provide a more accurate representation of an LLM’s capabilities by minimizing the influence of data and model biases.", "theoretical_claims": "The unbiased evaluation protocol systematically applies statistical principles to decompose the evaluation bias. The decomposition into original, related, and independent terms provides valuable insights into how new biases (using probes) interact with existing ones, guiding the design of more unbiased evaluation protocols.", "experimental_designs_or_analyses": "The experiments make a lot of sense validating the experimental design and analysis.", "relation_to_broader_scientific_literature": "This work generalizes evaluations using LLM-as-a-Judge paradigm which is one of the few scalable methods today for LLM evaluations (without humans in the loop). It decomposes the evaluation by breaking up the generation, coming up with various probes and then defining theoretical framework for measuring various metrics (consensus, OC, UC).\nThis field is emerging where currently there is a lack of rigor in evaluation for most benchmarks.", "essential_references_not_discussed": "Seems pretty good. However, I may have missed some theoretical references related to studying various error types.", "other_strengths_and_weaknesses": "- The paper writing and organization can be improved a lot. The paper starts and shows the cryptic Fig. 1 and non-standard Fig 2 (and talks about Fig 2 much later) - the definitions are vague (still confused how strength parameter is varies during evaluations) \n- There is very little (almost no) comparison to other work in the field. For example, authors refer to Ye et al. 2024 work ( https://arxiv.org/pdf/2410.0273) where they refer to various biases coming from LLM-as-a-Judge and how the work has defined Robustness rate and consistency rate metrics to assess some of these biases - how do these compare with this work or other relevant work?", "other_comments_or_suggestions": "Agents-as-an-Evaluator seems like a new and slightly confusing term - needs some definition and clarification imo.", "questions_for_authors": "- please explain the strength parameter and it is varied in the experiments", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2e4KxNheJx", "reviewer": "Reviewer_A1m7", "summary": "This paper studies potential biases in LLM-based evaluators (“Agents-as-an-Evaluator”) and proposes a new protocol, called the “Unbiased Evaluator,” which systematically introduces small interventions (“Bags Of Atomic Interventions”) into evaluation tasks to mitigate data and model biases. The authors present both theoretical and empirical analyses suggesting their protocol reduces correlation based artifacts and helps reveal model weaknesses that standard benchmarks may overlook.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Response to reviewer A1m7\n> **Q1**: Bias mitigation approaches in the LLM-as-a-Judge are related to the paper, such as Length-controlled AlpacaEval and Arena-Hard Style Control.\n\n**A1:** Thank you for your suggestion. Unlike previous bias mitigation approaches in the LLM-as-a-Judge, which primarily focus on the judge side, our paper is the first to analyze bias from the generation side, i.e., Agents-as-an-Evaluator. We will incorporate these works into our related work section for clarification.\n\n> **Q2**: Some aspects of the theoretical framework remain high-level, and more rigorous proofs or formal constraints on interventions might strengthen the argument.\n\n**A2**: We argue that **our theoretical framework is intuitive and sufficient to support our method’s design**. Inspired by findings in Proposition 3.1, BOAT is designed to mitigate the impact of the related and independent terms (detailed in L366). Further refinements and formal extensions of our framework are important directions that we plan to explore in future work.\n\n> **Q3**: How would the proposed atomic interventions scale to more complex structured tasks beyond multiple-choice towards open-ended prompt?\n\n**A3:** As shown in Table 3, our method, with designed Question and Answer Jitter, has been successfully scaled to the mathematics benchmark GSM8K, which does not rely on multiple-choice style. Based on our causal formulation of evaluation, we can categorize tasks into two types. \n- **Most tasks** (e.g., multiple-choice, math), inherently follow natural rules in either the questions or answers, and rule-based interventions can be automatically applied.\n- **The other small percentage of tasks** (e.g., CivilComments), can use a **debiased Agents-as-Evaluator version**. Concretely, our study has revealed the data and model biases of previous version, inspiring two designs to mitigate them: (1) cross-generation: to reduce model bias, we can break down question generation into multiple chunks, using different models for each. (2) cross-checking: multiple advanced models can be used to cross-check the output to mitigate data bias and enhance quality.\n\nOverall, our method is easily scaled to most tasks, and our insights will provide valuable inspiration for future advancements in evaluation methodologies. We will add a section of **Future Work** to include these discussions.\n\n> **Q4**: Did you observe any qualitative differences in model performance across different types of math or reasoning questions when interventions stack up?\n\n**A4:** Yes. In addition to GSM8K in Table 3, we conduct further experiments on a more challenging benchmark, MATH500. We also evaluate a recently open-sourced reasoning model, QWQ-32B, with 16k context. Our experiments revealed two key observations.\n\n- **the performance gap between models becomes more pronounced from GSM8K to MATH500**. Notably, Qwen2.5-72B remains the strongest, on par with Mistral-Large-2411 (123B). Meanwhile, the gap between Qwen2.5-72B and models like Llama3.1-70B widens considerably—rising from 5.89 on GSM8K to 19.56 on MATH500, highlighting the superior capabilities of Qwen2.5-72B and Mistral-Large-2411 in handling complex mathematical reasoning.\n- **the reasoning model exhibits stronger generalization on mathematical benchmarks**, experiencing a significantly smaller performance drop compared to others.\n\n|Model|GSM8K Vanilla|GSM8K Ours|Δ|MATH500 Vanilla|MATH500 Ours|Δ|\n|-|-|-|-|-|-|-|\n|Qwen2.5-72B|98.41|88.86|9.55|92.23|77.57|14.66|\n|Llama3.1-70B|95.98|82.97|13.01|75.87|58.01|17.86|\n|Yi1.5-34B|91.96|69.60|22.36|65.44|56.37|9.07|\n|Mistral-Large-2411|97.73|90.04|7.69|86.71|77.51|9.20|\n|QWQ-32B(16k)|99.32|95.32|4.00|89.78|88.18|1.60|\n\n> **Q5**: How changes in model performance under your method correlate with human expert judgments overall? \n\n**A5:** **Our Unbiased Evaluator provides a much more correlative assessment with human expert judgments**. \nSince collecting overall expert judgments across multiple model is costly and impractical, we instead compare our method with LiveBench, a continuously updated benchmark. Specifically, we compute the Pearson and Kendall correlations between our averaged results (Table 3) and the global average results in latest LiveBench-2024-11-25 (https://livebench.ai). Notably, we exclude two models (GPT-4-Turbo and Yi1.5-34B-Chat) that are not evaluated in LiveBench-2024-11-25 for a fair comparison.\n\n||Pearson|Kendall|\n|-|-|-|\n|Vanilla|0.918|0.600|\n|Unbiased Evaluator|0.949|1.000|\n\nThese results confirm that our method aligns more closely with LiveBench. Notably, it achieves a perfect ranking correlation with LiveBench (as measured by Kendall), a significant improvement over baseline. Unlike LiveBench, which covers diverse tasks and requires substantial resources to update questions regularly, ours leverages existing benchmarks and requires almost no additional resources. We sincerely appreciate your valuable suggestions and will add these results to our ablations.", "claims_and_evidence": "The main claim is that existing multi agent evaluators introduce bias during question generation, and that the proposed BOAT based evaluator offers a more “unbiased” alternative. While the experiments (notably the confusion matrices) do highlight differences in evaluation outcomes, the evidence for truly mitigating all bias remains somewhat limited and relies on a relatively small set of carefully selected interventions. It is unclear whether these interventions comprehensively address the broad range of biases in LLM based evaluations.", "methods_and_evaluation_criteria": "The authors conduct a systematic causal analysis, framing QA as a DAG with interventions on specific “atomic” components.\n\nThey define several carefully controlled transformations, such as adding distractor questions, to stress test LLM understanding.\n\nAccuracy across these perturbed scenarios is aggregated and compared with standard benchmarks.", "theoretical_claims": "The decomposition of evaluation bias appears logically consistent, and the provided proofs in the appendix are straightforward, though the argument is more conceptual than heavily formal.", "experimental_designs_or_analyses": "The experiments cover multiple model sizes (both open-source and proprietary), several well-known benchmarks (ARC, MMLU, GSM8K), and detailed ablations of single versus combined interventions.\n\nHuman verification of a subset of transformed samples is used to confirm correctness of the approach (high agreement rate).\n\nThe methods are transparent, and the sample sizes are standard for these benchmarks, though additional clarity on how random interventions might differ across runs could improve reproducibility.", "relation_to_broader_scientific_literature": "The paper builds on existing work on LLM-as-a-Judge by extending to “Agents-as-an-Evaluator\" by dissecting biases in both question generation and model self-assessment. Drawing on causal-inference ideas (e.g., interventions on input variables), it aligns with literature on benchmark contamination and fairness in NLP.", "essential_references_not_discussed": "I think mentioning how common bias mitigation approaches proposed within the LLM-as-a-Judge framework is related to the paper, such as Length-controlled AlpacaEval (Dubois et al) and Arena-Hard Style Control (Li et al).", "other_strengths_and_weaknesses": "**Strengths**:\n1. Presents a fresh causal perspective on LLM evaluations.\n2. Detailed metrics for identifying overconfidence and underconfidence biases.\n3. Scalability: the method can be adapted to various choice-based tasks.\n\n**Weaknesses**:\n1. Some aspects of the theoretical framework remain high-level; more rigorous proofs or formal constraints on interventions might strengthen the argument.\n2. The paper focuses primarily on multiple-choice formats; it would be insightful to see how the method generalizes to more open-ended tasks.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "1. How would the proposed atomic interventions scale to more complex structured tasks beyond multiple-choice towards open-ended prompt?\n2. Did you observe any qualitative differences in model performance across different types of math or reasoning questions when interventions stack up?\n3. Can you provide more details on how changes in model performance under your method correlate with human expert judgments overall?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "W4Hcn08Ppg", "reviewer": "Reviewer_zS8E", "summary": "The paper explores bias in agents as evaluators (LLMs generating new tasks for evaluating another agent), and detects different kinds of biases. They introduce an unbiased evaluator using causal inference.\n\n\"## update after rebuttal\"\nI thank the authors for answering my questions. After seeing the other reviews and discussion, I still have doubts about the significance of the paper, but some other issues have disappeared so I increase the score.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Response to reviewer zS8E\n> **Q1**: Reference AI-MO 2024 should be AIME 2024\n\n**A1:** AI-MO (refer to https://aimoprize.com), the AI Mathematical Olympiad, adapts data from AIME 2024 as its competition benchmark. This widely-used version is publicly available as [aimo-validation-aime](https://huggingface.co/datasets/AI-MO/aimo-validation-aime). Therefore, as referenced in L474, we cite it (in-text citation appears as AI-MO, 2024.) as:\n```\nAI-MO. AIME 2024. https://huggingface.co/datasets/AI-MO/aimo-validation-aime, 2024.\n```\n\n\n> **Q2**: Table 2 is hard to understand and going to appendix D didn't help in my case. \n\n**A2:** Table 2 illustrates a demo case of Bags of Atomic Interventions (BOAT) to demonstrate how each intervention works. For each intervention, we present a simple question to show its effect. To save space, the \"intervened\" column only displays the part that has changed, highlighted in the same color as the original part it replaces. The other parts of the content remain unchanged. For example, with the Distractor Hint intervention, the full modified content would look like this:\n```\n## original question\nQuestion: Here is a multiple choice question, answer A or B. Is 9.8 bigger than 9.11?\nOption: A: True B: False\nLabel: A\n\n## intervened question\nQuestion: Here is a multiple choice question, answer A or B, if there is no answer, reply N. Is 9.8 bigger than 9.11?\nOption: A: True B: False\nLabel: A\n```\nWe will add detailed demonstrations into the caption of Table 2 for next version.\n\n> **Q3**: In Figure 3 the options and the option of \"no correct\" is introduced, but what's the problem with the original questions, other than contamination?\n\n**A3:** As detailed in our general evaluation formulation (L253), we argue that the evaluation process can be seen as a causal analysis. It is essential to assess whether the model truly understands and can make these causal connections. In this context, the Distractor Hint/Answer Removal is designed to evaluate not only whether the model selects the correct answer, but also whether it effectively rejects the incorrect options.\n\n> **Q4**: The transformation makes them more difficult, but perhaps not only because of removing contamination. Can these two things being separated?\n\n**A4:** We argue that our Unbiased Evaluator does **NOT** inherently make questions more difficult. The adversarial benchmark in paper [1] mentioned by the reviewer is fundamentally different from our approach. Specifically, adversarial benchmarks are designed to exploit a model’s weaknesses, often **using supervised optimization algorithms to find desired perturbations** that lead to incorrect predictions. In contrast, our Unbiased Evaluator aims to assess **whether models can genuinely answer a question correctly by employing causal interventions that align with human recognition**. Therefore, rather than increasing difficulty, our Unbiased Evaluator provides a more accurate measure of a model’s true and robust performance on a given benchmark by eliminating performance inflation caused by data contamination.\n\nTo clarify the functioning of the Unbiased Evaluator, we have included a separation ablation in Figure 6, which demonstrates the impact of each individual intervention. Even simple manipulations, such as Option Shuffling, Label Replacement, and Binary Transformation, lead to noticeable degradation in the model’s performance. This provides strong evidence that data contamination plays a significant role in inflating evaluation outcomes.\n\n> **Q5**: The baseline not changing the meaning is an interesting baseline, but where is this used?\n\n**A5:** The rephasing baseline, without changing the question's meaning, is referred to as a minimal Agents-as-an-Evaluator in this paper (see L208), and it serves as a comparative baseline in Figure 2 and Table 1.\n\n> **Q6**: Figure 5 shows the accuracy goes down, but I'm not sure that means that contamination goes down as well. How can we know?\n\n**A6:** As presented in A4, Unbiased Evaluator evaluates a model's true and robust performance on a given benchmark. Therefore, **the decline of accuracy actually reflects the decrease of contamination**. To validate this, we further provide an additional fine-tuning ablation study. Specifically, we fine-tune Llama2-13B on the original samples from the MMLU test set and evaluate it on MMLU test set under two conditions: with and without our Unbiased Evaluator.\n\n|train set|w/o Unbiased Evaluator|w/ Unbiased Evaluator|\n|----------------------|---------|---------|\n|Llama2-13B|55.6|33.7|\n|Llama2-13B + original test set|96.6|37.1|\n\nEven when trained directly on the original test set, the model struggles to perform well under the Unbiased Evaluator, suggesting that it effectively mitigates data contamination and ensures a more robust evaluation.\n\n[1] Adversarial benchmark evaluation rectified by controlling for difficulty", "claims_and_evidence": "- The problem of bias is very important when using LLMs evaluating other LLMs.\n- The causal interventions are not based on the causal graph, even if the causal graph was introduced to understand the bias.\n- The interventions may overcompensate? It's not clear the reduction of performance is actually (only) compensating for the bias.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "The theoretical decomposition. I haven’t checked it and find it a bit too abstract, and perhaps not that relevant.", "experimental_designs_or_analyses": "Yes, I checked the design as written in the paper.", "relation_to_broader_scientific_literature": "I miss an independent assessment of question difficulty by humans, to understand if the interventions are changing some other things. The reformulation should give exactly the same results for humans, and some others that make the questions more difficult should create a similar effect as in humans.\nFor the role of difficulty: Adversarial Benchmark Evaluation Rectified by Controlling for Difficulty\nhttps://www.researchgate.net/publication/374304817_Adversarial_Benchmark_Evaluation_Rectified_by_Controlling_for_Difficulty", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "No", "other_comments_or_suggestions": "Reference AI-MO 2024 should be AIME 2024?", "questions_for_authors": "Table 2 is hard to understand and going to appendix D didn't help in my case. In Figure 3 the options and the option of \"no correct\" is introduced, but what's the problem with the original questions, other than contamination?\n\nAnd the transformation makes them more difficult, but perhaps not only because of removing contamination. I don't see a clear causal graph justifying the intervention. Can these two things being separated?\n\n\nThe baseline not changing the meaning is an interesting baseline, but where is this used?\n\n\nFigure 5 shows the accuracy goes down, but I'm not sure that means that contamination goes down as well. How can we know?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper provides a causal framework to evaluate agents as an evaluator LLMs. The paper provides an evaluation protocol by designing a handful of rule‑based edits (BOAT) such as option shuffling, distractor insertion and label replacement to mitigate different sources of bias. It attempts to formally motivate these interventions via a decomposition of the evaluator bias. Empirically, they apply the protocol to different standard LLM benchmarks and show that it can mitigate inflated scores and finds close agreement of the model rankings with livebench. Another ablation shows that finetuning on the test set directly would yield substantially lower gains for models under the proposed evaluation protocol providing further evidence to support the key claims. 

There are limitations to the proposed approach: the formalism is high level and the proposed interventions are hand crafted and are not naturally generalizable say to more open ended QA benchmarks. Furthermore the proposed interventions could be modelled as augmentations to the training dataset limiting the generalizability of the specific interventions.

Nevertheless, all reviewers agree that the paper addresses a timely issue: trustworthy evaluation of LLMs and the proposed protocol gives a pragmatic, well‑validated solution. Initial concerns about the hand‑crafted nature of BOAT and the clarity of figures seemed to have been somewhat clarified in the rebuttal. After clarifications and new experiments, all reviewers lean to accept; the method’s practicality and breadth outweigh the aforementioned limitations so I recommend acceptance.
Decision: Accept (poster)