Preference Leakage: A Contamination Problem in LLM-as-a-judge
Authors: Dawei Li
Renliang Sun
Yue Huang
Ming Zhong
Bohan Jiang
Jiawei Han
Xiangliang Zhang
Wei Wang
huan liu
Venue: ICLR 2026 Poster (2026)
Keywords: LLM-as-a-judge
Preference Leakage
Data Contamination
Primary area: alignment, fairness, safety, privacy, and societal considerations

Abstract: Large Language Models (LLMs) as judges and LLM-based data synthesis have emerged as two fundamental LLM-driven data annotation methods in model development. While their combination significantly enhances the efficiency of model training and evaluation, little attention has been given to the potential contamination brought by this new model development paradigm. In this work, we expose preference leakage, a contamination problem in LLM-as-a-judge caused by the relatedness between the synthetic data generators and LLM-based evaluators. To study this issue, we first define three common relatednesses between the data generator LLM and the judge LLM: being the same model, having an inheritance relationship, and belonging to the same model family. Through extensive experiments, we empirically confirm the bias of judges towards their related student models caused by preference leakage across multiple LLM baselines and benchmarks. Further analysis suggests that preference leakage is a pervasive and real-world problem that is harder to detect compared to previously identified biases in LLM-as-a-judge scenarios. All of these findings imply that preference leakage is a widespread and challenging problem in the area of LLM-as-a-judge. We release all codes and data at: \url{https://github.com/David-Li0406/Preference-Leakage}.

Published as a conference paper at ICLR 2026
Preference Leakage: A CONTAMINATION PROBLEM IN LLM-AS-A-JUDGE
Dawei Li Renliang Sun*?, Yue Huang', Ming Zhong'. Bohan Jiang', Jiawei Han", Xiangliang Zhang', Wei Wang", Huan Liu'
1 'Arizona State University, University of California, Los Angeles, University of Notre Dame, UUniversity of Illinois Urbana- Urbana-Cham Champaign da daweili i 58 asu. edu
ABSTRACT
Large Language Models (LLMs) judges and LLM-based data synthesis have emerged as two fundamental LLM-driven data annotation methods in model devel- opment. While their combination significantly enhances the efficiency of model training and evaluation, little attention has been given to the potential contamination brought by this new model development paradigm. In this work, expose prefer- ence leakage, a contamination problem in ----s-a-jjude caused by the related- ness between the synthetic data generators and -.Le- evaluators. To study this issue, we first define three common relatednesses between the data generator LLM and the judge LLM: being the same model, having an inheritance relation- ship, and belonging to the same model family. Through extensive experiments, we empirically confirm the bias of judges towards their related student models caused by preference leakage across multiple LLM baselines and benchmarks. Further analysis suggests that preference leakage is a pervasive and real-world problem that is harder to detect compared to previously identified biases in LLM- L.L--as-a-judge scenarios. All of these findings imply that preference leakage is a widespread and challenging problem in the area of LLM-as-a-iudes We release all codes and data at: https:: /ttttpb thub. wtt.c/mm//aa// www.:/twww.........oootoo
Leakage
INTRODUCTION
Recent advancements in Large Language Models (LLMs) Achiam et al. (2023) ; Jaech et al. (2024);
Tong et al. (2024); Zhang et al. (2024a); Tan al. (2024a] 2025); Huang al. (2026) have empowered various downstream tasks and applications. However, this also poses substantial challenges to the automatic evaluation these models. Representatively. LLMd AI agents' focus transfer from traditional natural language processing tasks Yang et al. (2023); Zhang et al. (2023) to real-world Liu et al. (2023b); ; Huang et al. (2023), open-ended response generation Wu et al. (2024), which greatly limits the applicability of traditional n-gram matching methods (e.g., BLEU Papineni et al. (2002) and ROUGELim (2004) Liu et al. (2016); Reiter (2018) or model- eva evaluators Zhang al. 2020);; Zhong al. (2022) for evaluation.
To address these challenges, the paradigm of L.LM-as-a-judge Zheng et al. (2023a) Li et al. (2024a) ;
Tiang al. (2024a): /Zhog al. (2024); Li et al. (2025a); Qu et al. (2025) has been proposed, designed to leverage LLM as evaluators to assess response quality. By combining powerful LLMs with well-designed prompting strategies, LLM-as-a-judge enables human-likk evaluation of long- form and open-ended generation in more cost-efficient and scalable manner. However, recent studies point out some weaknesses of such an assessment. For instance, Ye et al. al.] (2024) explores various biases and vulnerabilities of LLM-as-a-judee highlighting the importance of developing reliable and fair
evaluation system.
In this work, we aim to highlight subtle yet critical bias in LL--as--- Preference Leakage. This issue arises when the L.LMS used for data generation and evaluation are closely related, causing
Equal contribution.
Moor resources I.LM--s-a- the website: https://11mm judge.githu..ii.......

---

Published as a conference paper at ICLR 2026
Data lealagg
Taln Lnaloge!
modrl
Train
PU9
Evalate
P
Training Corpus
Trained Model
Evaluation Tesfet
Trained Model
Traking
Evaleatian
Overlap
Spalket
Preference lealaage
Proforence I Lnalaree
Synthesize
W the model family
Dale morrato
Synthotic
Judge Model
Data
11M for Data Syethens
s=]iddge
Relotedhews
Figure 1: Overview of preference leakage. We make a comparison between data leakage and preference leakage and present three types of relatedness: being the same model, having an inheritance relationship and belonging to the same model family.
the preference of the LLM evaluators to leak to the student models through synthetic data and thus inflating the evaluation score (as illustrated in Figure Synthetic data generated by LLMs Gan
et al. (023); Tan et al. (2024b); Li et al. 2202bb c) has become a cornerstone of model training Lee al. 2025). When combined with LLM au-ge- they offer significant efficiency gains in model development. However, limited attention has been given to the potential contamination that occurs when the generator and evaluator L.LMs share a close relationship. During our preliminary study, we find this issue is particularly pervasive in popular LLM-as-a assa--udgee benchmarks (e.g., AlpacaEval 2.bDuss et al. (2024) and Arena-Hard Li et al. (2024ch) and LLM-relevant studies (more details can be found in Appendix B), due to the common reliance
the most advanced LLMs, such as GPT-4 Achiam et al. 2023), for both data synthesis and evaluation to ensure the highest quality outputs. In work. reveal this relatedness akin to the overlap between training data and evaluation sets in traditional data contamination woould introduce a systematic bias of judge LLMs towards their related student models (i.e., the model distilled by the data generator which is related to the judge). Compared to other biases -.L---s-a-Judge such as length bias or egocentric bias Ye al. (2024); Panicksssrrr et al.) (2024), preference leakage is subtler and more challenging detect, especially given that most LLMs do not disclose their training data.
To investigate and reveal the preference leakage problem, we first define three relatednesses between data generator LLM and judge LLM: being the same model, having an inheritance relationship, and belonging to the same model family. Each of these scenarios commonly encountered in real- world applications. Then, we pose and answer three core research questions about preference leakage:
Does preference leakage introduce systematic biases LLM-based evaluation? To answer it, we conduct experiments with various LLM baselines in two widely recognized LLM-as- a-judge benchmarks, also introduce the preference leakage score to quantify the bias caused by preference leakage. The analysis results suggest an obvious bias of judging LLMs toward their related student models due to preference leakage.
RQ1:
RQ2: What is the severity of preference leakage under various scenarios? We conduct experiments under various data mixing strategies, relatedness settings, tuning techniques and real-world applications to address it, finding that preference leakage consistently affects judge LLMs. Moreover, the severity of preference leakage correlates with the degree of relatedness between the data generator and LLM judges, as well as the proportion of synthetic data.
RQ3: What are the underlying mechanisms causing preference leakage? For this question, we analyze I.MSs' recognition capabilities on their related student models" generation well as the distribution of bias across different question types and judgment dimensions. The analysis reveals that preference leakage is subtle, hard-to-detect issue for the LLM evaluators, particularly affecting subjective questions and judgment dimensions.
To summarize, our contributions in this work are as follows:
For the first time, we introduce preference leakage, a contamination issue arising from the related- ness between the data generator and judge LLMs.
We conduct extensive experiments across various LLMs and benchmarks to study how and to what extent the potential bias brought by preference leakage influences judgment.
2

---

Published as a conference paper at ICLR 2026
Our further analysis reveals that preference leakage is prevalent in diverse scenarios and difficult for judge LLMs to detect, providing valuable insights for future research on this challenging issue.
2 RELATED WORK
-aad------ LLM-as-a--Jude,, introduced by Zheng et al. (023a). leverages L.Ms to auto- matically evaluate responses and assign rewards. This approach has gained widespread adoption in
areas such as model alignment Zhang et al. (202d)) and benchmarking Liu et al. (202a): Zhang et al. (2024b); Gao et al. (2023) y Zhong et al. (2024); Beigi al.
(2024): ei et (026). driving significant progress in the field. Building on this concept, Zhuge et al. (2024) proposed agent-as-a-Judge, where
agentic systems are employed to evaluate other agentic systems. Additionally, Prometheus, series
of open-source LLMs tailored for LLM-as-a-Judge Kim et al. (2023) (024), addresses the prohibitive costs associated with proprietary models, further democratizing the technology.
Despite its promising potential, recent studies have highlighted the vulnerabilities and biases of L.MM-as-aJJudge Zheng et al. (2023a); Ye et al. (2024): Koo et al. (2023); Chen et al. (2024); Zheng et al. (2023a) Huang et al. (2024): Thakur al. (2024); Shi et al. (2024). Among these, egocentric bias, where LLM evaluators tend to favor their generations Koo et al. (2024); Liu et al. (2024b);
Wataoka et al. (2024); Xuu et al. (2024b): Rando al. (2025); Panickssery al. (2024); Chen et al. (0225), is most closely related to the preference leakage proposed in this work.
However, in contrast the relatively straightforward setting of egocentric bias, preference leakage presents a more complex and dynamic challenge. It can arise from various types of relatedness between data-generating and evaluating LLMs, as well as the intricate flow of synthetic data among modern LLMs Tan et al. (2024b). Moreover, detecting preference leakage is also more challenging, given LLMs often do not disclose their training data and the difficulty in distillation quantifica- tion Wadhwa et al. (2025): Lee et al. (025)..
Data Leakage. The possible overlap between training data and evaluation benchmarks has become a central issue, since LLMs are usually trained on extensive web corpora Dodge et al. (2021). This phenomenon, known as data leakage, can artificially improve the performance of L.MMs and
undermine the reliability of the assessment Deng et al. (2024a); Diang et al. (2024b): Li et al. (025bb).
Several researchers have proposed methods to detect and mitigate data contamination. [eng et al. (2024b) proposed retrieval-based approach to assess the degree of overlap between pre-training text and benchmark data. Golchin & Surdeanu (023) have developed "guided instruction" to flag contaminated instances. Dong et al. (2024b) proposed the CDD method to identify peaks in the output distribution to detect data contamination. Several studies analyze data leakage for specific LLMS Balloccu al. (2024); Xu al. (2024a) and report contamination such cross-language contamination Yao et al. (2024) and task contamination| [. & Flanigan (2024) that can evade traditional detection methods. To address data contamination issues, Ni et (2024) have used web user query detection and benchmark mixture, White et al. (2024) use the most recent information to update the problem.
PREFERENCE LEAKAGE
3.1 LLMS AS ORACLES: A NEW AVENUE FOR CONTAMINATION
With the advent of LLMs, these models are increasingly employed as "oracles" in various scenarios: for both synthetic data generation (Ma) and employed as evaluators (M,) to automate the assess- ment. While these approaches enhance scalability and efficiency, they also introduce potential risks. Specifically, if the LLM used for data generation (Mc) and the LLM used for evaluation (M,) are not independent, new contamination prrffrrnncee leakage- can emerge, biasing evaluation outcomes.
3.2 DEFINING PREFERENCE LEAKAGE IN LLM-BASED EVALUATION
Formally, to define preference leakage, consider the following entities in models development:
Data Generator LLM, Mg. defining conditional distribution PMO (y|z) for generating an output y given prompt v, forming the synthetic dataset Dagn for student L.LMs training.
3

---

Published as a conference paper at ICLR 2026
Student LLM, Ms, trained on data generated by Ma. producing output distribution PM4 (3)2z).
Judge LLM, M,, providing scoring function SM, (y]z) that assesses output y for prompt æ.
Preference leakage occurs when the evaluation score assigned by M, to Ms's 's outputs is inflated due underlying relatedness between Mg and M,. This implies that M, may favor outputs from Ms not solely based on their intrinsic quality, but because they exhibit spurious features (e.g. style, format, wording) inherited from Ma. which My is predisposed due to this relatedness:
(I) where ys are outputs from Ms. The relation Mg real M, denotes that judge Mr is related to Mc. while Mc ret yr denotes that alternative judge M, is not related to MG and possess comparable intrinsic quality assessment capabilities M, The expectation is taken over the input distribution X and the trained Student LLM's output distribution PMs.
Pay (SM, (ys|z) reel M,] > P , [G Pret Myl,
3.3 TYPE OF LLM "RELATEDNESS"
The condition MG rrel M, in Equation Tencassulates several ways the Data Generator LLM and Judge LLM can be interconnected. We identify three common types in the real world:
Being the Same Model: The most direct form of relatedness occurs when the Data Generator LLM and the Judge LLM are the exact same model instance:
MG M,.
In this scenario, the inherent preferences in the model that shape its generative distribution PMo (y|z)) are precisely the same as those guiding its evaluation via the scoring function SMa (s/zz)
Inheritance Relationship: One model's development directly based on another, either by fine-tuning the existing model or by training new model on the other's outputs, for instance:
Mj + Fine TaneeTuneMMae Dtrain) or M, + Fine Maase, D Dayna)
(3)
where Dirain represents general training data used to adapt Mg into M,, Maose is a base model, and Dayne denotes synthetic data generated by Mc. This type of relationship is bidirectional; Mg can similarly inherit from My through analogous processes. In such cases, the descendant model is likely internalize and thus favor the preferences, styles, biases of its progenitor.
Within the Same Model Family: The Data Generator LLM Mg and Judge LLM M, belong to the same model family (e.g., different versions or sizes of GPT). Models within such family typically share common architectural blueprint (Ax) and are often developed from foundational
models pre-trained on substantially overlapping datasets (Dx). This shared foundation (Ax, Dx) would lead to correlated preferences and systemic biases characteristic of the common origin:
Mr E Family Dx) for k [G.
MAIN EXPERIMENT
4.1 EXPERIMENT SETUP
Models. We choose three powerful LLMs as data generator/ judge models. They are GPT-40-2024- 1 1--2 Achiam al. (202)), Gemini- 1.5-flash Team al. (2024), and 3... -.nstuctt turbo Dubey et al. 2024). For the student model, we choose Mistral-7B-vO. Jiang et al. (2023) and Qwen-2.5-1 2.5-14B Yang et al. (2024). To avoid potential preference leakage due distilling data from other LLMs during the instruction-tuning process, choose to use the -PRE-TRAINED version rather than the -NNTRRCC version of these student models.
Evaluation Datasets. We choose twO representative pairwise evaluation datasets, Arena-Hard Li et al. (024c) and AlpacaEval 2.0 Dubois et al.] (2024), to evaluate the trained student models. Arena-Hard includes 500 challenging questions in English. Additionally, the evaluation agreement between Arena-Has Hard and LMArena Zheng et al. (2023 's hard prompts achieved a 96.7% Spearman correlation, demonstrating the consistency of Arena-Hard with human preferences L et al. (002e)).
4

---

Published as a conference paper at ICLR 2026
AlpacaEval 2.0 is improved evaluation method based AlpacaEval Li et al. (2023) and contains 805 questions. Compared to version 1.0, AlpacaEval 2.0 significantly reduces the effect of text length on the evaluation results.
Implementation Details. our main experiment, we examine the preference leakage introduced
by using the same data generator and evaluator in supervised fine-tuning (SFT). We will diseuss other relatedness and learning methods Section [ To obtain synthetic datasets, We first randomly sample 30,000 prompts from the Ultrafedback dataser(Cun al. (2024). The Ultrafeedback dataset includes instructions from several publicly available high-quality datasets such as TruthfulQA Lin al. (2022), FalseQA Hu et (2023). and Svol-Instruct/ Xu al. (2023). For each data generator model, we provide these prompts for them to produce synthetic responses, resulting in three synthetic instruction datasets. We then use each dataset to supervised fine-tune the student model, obtaining three different versions for each baseline: Mistral/ Owen-GPP--4e Mistral/ Swen-GGmini-1.. and Mistral/ Owen-LLaMA-3... After that, we pair each two student models and obtain three model pairs. For each model pair, perform the pairwise comparison using the three judge models respectively.
Metries Based on our hypothesis, preference leakage would lead to bias judge LLMs towards their related student models. Following this principle, we design the preference leakage score PLS(i,.) to measure the bias in model pair (i, caused by preference leakage:
WR(G.)) -VGG(.) + AVG((.)
(RRG..)--VGG
AVG(
PLS(I.))
(5)
2
AVG(L.)) WR(1, WR(i.))
(6)
Here WR(1.)) represents the win-rate score from judge model j to student model Intuitively, large preference leakage score indicates that the two judge models demonstrate strong bias toward their related student models, suggesting a significant preference leakage phenomenon.
The main experiments in Section 4 and the mitigation analyses in Sections 5.4 and 5.7 are designed for complementary purposes. The main experiments focus on controlled and interpretable measurement of preference leakage itself-quane quant--uani the phenomenon across models and conditions while minimizing confounding factors such as human labeling noise. In contrast, the mitigation analyses prioritize realism and external validity, using human-labeled benchmarks and metrics. Since these setups involve labeled data rather than automatically computed PLS, they serve as realistic extensions that test mitigation feasibility in practical LLM-as-a-judge scenarios. More details about model training and metric explanation can be found in Appendix
Table 1: Preference leakage score result on Arena-Hare and AlpacaEval 2.0. The blue background indicates negative preference leakage score value and the purple background indicates positive value. The deeper the color, the larger the absolute value.
Model
Data Generator/ Judge Pair GPT-4o Gemini-1.5
Arena-Hard
AlpacaEval 2.0
Avg.
28.7%
18.4%
23.6%
Mistral-7B
GPT-4o & LLaMA-333
-1.5%
1.4%
-0.1%
LLaMA-3.3 & Gemini-1.5
13.1%
19.8%
16.4%
GPT-4o Gemini-1.5
37.1%
18.6%
27.9%
Qwen-2.5-14B
GPT-4o & LLaMA-3.3
1.0%
2.3%
1.7%
LLaMA-3.3 & Gemini-1.5
25.4%
18.4%
21.9%
4.2
MAIN RESULTS
In our main experiment, we aim to provide insights into RQI.
Preference leakage exists in most model pairs. The original judgment results from Arena-Hard and AlpacaEval 2.0, along with the calculated preference leakage scores, are shown in Table [ As the results demonstrate, in most model pairs (except MPstral-GPT-4o Mistral-LLaMAA-3, and Qwen-GPT-4o vs Owen-LLaMA-3333) the judge LLMs exhibit a strong preference toward their related student models, leading to large positive values in the preference leakage scores. This finding
5

---

Published as a conference paper at ICLR 2026
suggests that preference leakage, along with the resulting bias, is widespread in SFT when the data generator and evaluator are the same.
more bias from judge LLMs. To investigate the impact of student model size on the degree of preference leakage, we conduct additional experiments using various sizes of the LLaMA-3, Qwen-2.5 and Qwen-3 models. As shown in Figure 4.2 (a), notable finding is that the smallest models (LLaMA-3-IB, Qwen-2.5-3B and Qwen-3- 1.7BB exhibit the highest PL scores than their larger counterparts, indicating greater bias from preference leakage. This trend contrasts with the influence of model size in data contamination, where larger models are typically more susceptible Bordt et (2024). We assume that this gap arises from the differing learning capabilities and behaviors of large and small LLMs: while larger models are more prone to memorizing [uann et al. (024)) information that exacerbates data contamination. Compared with them, smaller models may only be able to learn those spurious features that repeatedly occurs (e.g..
Smaller student models cause
format), leading to more serious preference leakage.
Different benchmarks result in varying degrees of bias under preference leakage. Another observation from Table [ and Figure 4.2 (a) that the PL scores ArenaHard are generally higher than those in AlpacaEval 2.0. One possible explanation is the difference in question difficulty between the two benchmarks, as ArenaHard contains more challenging questions. Additionally, it may also stem from differences in the distribution of question types, the impact of which preference leakage will be further analyzed in Section 5.6
Model Family
30
36
LLaMA-3
Preference Leakage Score (%6)
34-
3
Average PLS (96
322
30
15
28
10
AlpacaEval2 Manual
ArenaHard Manual
AlpacaEva/20 Synthetic
Arenatlard Synthetic
12
14
100
Model Size (B)
Contamination Ratio (%)
(a) on models with various sizes. conduct the experiment with GPT-4o and Gemini as data generators and judges.
(b) Experiment results on data mixing. 'Manual' and "Synthetic represent mixing with manually-written data and other synthetic data, respectively.
Figure 2: Experiment results additional models and data mixing settings.
FURTHER ANALYSIS
In this section, conduct data mixing analysis, relatedness analysis, learning method analysis, and real-world impact analysis (Section 5. 5.4) to answer RQ2. Due to the cost consideration, we conduct these analyses analysis and category analysis to answer RQ3. Additionally, we also benchmark and explore various calibration methods to address preference leakage in Section 5.8
Histral-GPT--
Mistral-Gemini- 1.5. Morcover, perform recognition
5.1 DATA MIXING ANALYSIS
In real-world applications, synthetic data from a single LLM is often mixed with manually-written data or other multi-source synthetic data to train student models. To mimic these scenarios and explore how much synthetic data could lead to preference leakage, we conduct data mixing analysis. Specifically, we randomly sample 10%, 30%, 50%, and 70% from the original synthetic dataset and mix it with canually-writter data and multi-source synthetic data, respectively, in order to maintain consistent total volume of training data (30,000). For the manually-written data, we sample from the data pool collected in Section 5.3 For the multi-source synthetic data, data from Ultrafeedback, which includes responses generated by various LLMs (c.g.. WizardLM, Flcon, etc.). After obtaining the mixing training data, we train the student models using SFT and
the original synthetic

---

Published as a conference paper at ICLR 2026
calculate their preference leakage scores based the judgment results. Figure 4.2 (b) presents the results with two mixing strategies across two benchmarks.
The degree of preference leakage is directly proportional to the amount of synthetic data. We observe a strong correlation between the proportion of synthetic data in the mixture and the preference leakage score, with clear threshold separating cases with preference leakage from those without. This suggests that preference leakage can occur even with a small amount leaked synthetic data, posing significant challenges for its detection.
5.2 RELATEDNESS ANALYSIS
We demonstrate the impact of different relatedness conditions between the data generator and the judge LLM on the preference leakage problem, as shown in Table
Preference leakage under inheritance settings causes obvious bias of judges towards their related students. For the inheritance relationship. we consider the situation where the data generator inherited from the judge model. We conducted the following two experiments: (1). we give the same instructions again as in the SFT stage (Inheritance w/ same ins.), or (2). we sample the same number of different instructions from the Ultrafeedback (Inherence w/ different ins.). Then, we let the fine-tuned Mistral model generate the answers and use these generated data to fine-tune a new Mistral student model. From the results, with same instructions, the average preference leakage score 19.3%. In comparison, the score with different instructions is 22.3% %. Firstly, in inheritance setting, data generators can inherit judges' preferences, which are then passed on to new student models, thereby compromising the fairness of their evaluation. Second, even when different instructions used, judges' preferences leaked to data generators can still be transferred to the new student model through synthetic data, leading to a high preference leakage score.
Models within the same series tend to cause more significant bias. For two models within the same family, we consider two settings: (1) Same series, where training data is generated by GPT-4o and Demmnni-155-flar by GPT-4-turbo and Semmini-.5-ppmt (2) Differ- ent series, where training data is still generated by GPT-4o and Gemini- 1.5-flash, but judged by
Table 2: Preference leakage score in different relat- edness between the data generator and the judging LLM.
and judged
Arena-Hard
AlpacaEval 2.0
Avg.
Same Model
28.7%
18.4%
23.0%
Inherifance
17.8%
20.7%
19.3%
w/ same ins.
Inheritance
18.3%
26.3%
22.3%
w/ different ins.
GPT-3.5-turbo and Gemini-1.0-pm In the same
Same Family
series setting, the average preference leakage score is 8.9,, indicating that despite using dif- ferent models for data generation and judgment, their relatedness in terms of model family leads to some preference leakage. In contrast, the dif- ferent series setting yields a significantly lower leakage score of 2.8%, likely due to differences in architecture, training data, and other factors, reducing the influence of model-related biases in evaluation.
w/ sume series
10.1%
7.6%
8.9%
Same Family
2.2%
2.8%
w/ different series
5.3 LEARNING METHOD ANALYSIS
We also compare three learning methods, super- vised fine-tuning (SFT), direct preference op- timization (DPO) Rafailov et al. (2024), and in-context learning (ICL) D et al. (2024a). to explore the different influences to them under preference leakage. We first build data pool based on human-written instruction-tumm -utnnnnggtttgggggnnn data from OASST Köpf et al. (2024). LIMA Zhou et al. (2024), and MOSS Sun et al. (2024b) to super- vised fine-tune the pre-trained model. For DPO, sample 2 responses for each instruction from sampled UltraFeedback instruction and prompt each data generator to produce the pairwise feedback. Then use the DPO loss to further train the fine-tuned policy on each synthetic pairwise dataset. Appendix shows the prompt we use to craft synthetic pairwise feedback. For ICL, we sample responss pairs from each LLMSs synthetic dataset as the demonstration during inference.
Table 3: Preference leakage score in different learn- ing methods.
Arena-Hard
AlpacaEval
Avg.
28.7%
18.4%
23.6%
DPO
7.7%
2.7%
5.2%
ICL
-4.2%
-1.1%
-2.7%
7

---

Published as a conference paper at ICLR 2026
Tuning approaches would leak judges' preference to the student models. Various learning methods show significant differences in preference leakage scores across learning methods. SFT exhibits the highest average leakage score at 23.6%. In contrast, DPO achieves much lower score of 5.2%, which is consistent with previous studies in data contamination that pairwise optimization can contaminating sensitive training data compared straightforward
reduce the risk of memorizing supervised fine-tuning Hayes et al.) Meanwhile, ICL, which relies on contextual examples without model tuning, least affected by the data generator's preferences, resulting in the lowest leakage scores.
5.4 REAL-WORLL IMPACT ANALYSIS
Table 4:
Impact analysis of preference leakage real-world L.Le-as-a-Judge leaderboards. For each bias type, assess its impact by calculating the ranking difference of the corresponding model in LMArena and AlpacaEval 2.0, obtained by subtracting the ranking in AlpacaEval 2.0 from that in LMArena. A larger positive ranking difference indicates AlpacaEval 2.0 ranks the target models in higher positions, denoting greater impact of the corresponding bias.
Bias Type
Evaluator
Target Models
Ranking Difference
Egocentric Bias
GPT-4 Preview
1.00
GPT-4 Preview
Preference Leakage
Vieuna 7B/ 13B/ 33B
1.33
In this section, we investigate the impact of preference leakage in real-world LLM-as-a-Judge leader- boards. While broader leaderboard coverage would enhance external validity, few student-teacher (distillation) pairs are publicly documented, and most leaderboards lack the metadata needed for controlled cross cross-mode model comparisons. Moreover, re-evaluating all leaderboard entries with alternate judges would be computationally prohibitive at the current scale. Therefore, focus AlpacaEval and AMrrea as interpretable case studies and leave large-scale multi-judge re-evaluations for future work. To quantify the effect of each bias type, we calculate the ranking difference of each target model in LMArena and AlpacaEval 2.0.
As shown in Table 4. both egocentric bias and preference leakage result in a positive ranking difference, indicating that both lead to evaluator bias favoring the target models. Notably, the ranking difference associated with preference leakage is even higher than that of egocentrie bias, highlighting the substantial impact of preference leakage on real-world LLM-as-a-Jude- leaderboards.
5.5
CAN JUDGES RECOGNIZE STUDENT MODELS?
Table 5: Student recognition (binary classifica- tion) and response classification results (three-class classification). SR: Student Recognition, RC: Re- sponse Classification.
Previous studies demonstrate the LLM judges can recognize and thus prefer their own gener- ation Panickssery et al. (2024). In this work, pose a similar question: Does preference leak- age also source from the LLM judges' recog- nition of their related student models' genera- tion? To study this, we follow Panickssery et al.) (0024) to prompt the three judge LLMs and test whether they could recognize their related stu- dent models' generation. Additionally, we split three student models' generation into training and testing sets, and train BERT classifier to perform three-class classification inspired by the previous study on detecting human- AI text Zhang et al. (2024c). For student recognition, follow Panickssery et al. 2024) to both pointwise and pairwise settings. Due to the space limitation, more detailed prompting and training settings can be found in Appendix G
Accuracy
Task
Model
Pointwise
Pairwise
GPT-4o
41.0%
52.0%
SR
Gemini-1.5
53.2%
44.2%
LLaMA-3.3
41.8%
29.8%
RC
BERT
82.4%
Judge LLMs do not show good performance in recognizing the generation of their student models. As the result presented in T we find that the recognition performance of each judge LLM in the content of related students poor, with accuracy around the performance of random
8

---

Published as a conference paper at ICLR 2026
guesses. This suggests that preference leakage is subtler and harder-to-deted for judge LLMs, in contrast to the more obvious egocentric bias.
Certain features embedded in student models through synthetic data. Although judge LLMs do not perform well in related student recognition, we notice the fine-tuned BERT classification demonstrates high accuracy suggests that certain characteristics- such as style and format-and are embedded the student models through the synthetic responses. This finding further supports the existence of preference leakage and lays the groundwork for future research aimed at detecting and preventing it. For example, an external detector estimating model-relatedness could provide auxiliary confidence signal calibrate or penalize biased judgments.
in classifier responses generated by each student model. This
5.6
IMPACT ON QUESTION TYPE & JUDGMENT DIMENSION
Preforunce Leakaage Score 1%)
50
31.4
Preference Leakage Score (%6
32.4
21.0
23,8
28.6
28.8
24.2
30.2
30.7
20
28
27.9
16.5
17.2
17.3
10-
24
Mathematics
Buainess
Daily Life
Science
Writing
others
Programming
Completanads
Clarity
Richnesse
Satiefaction
Factralit
Logizal
others
Creativity
Faimess
(a) Question Type
(b) Judgment dimension
Figure3 3: Category analysis results on question type and judgment dimension.
In this section, we explore the impact of preference leakage across various question types and judgment dimensions. For the question type analysis, we first propose several general question types based on the question clusters introduced by Arena-Hard. Then, we prompt GPT-4o to map each question in Arena-Hard and AlpacaEval one of the question types and calculate the preference leakage score for each question category. For the judgment dimension analysis, judgment dimensions introduced by Liu et al. (2023a) and also utilize GPT-4o to map the rationale generated by judge LLMs to one or multiple judgment dimensions, More detailed prompt can be found in Appendix The analysis results are presented in Figure
follow the
Subjective question and judgment dimension tend to lead to more bias. For question type analysis, we find objective questions with a definitive answer, like mathematical ones, demonstrate the least preference leakage. By contrast, subjective questions that have as programming and writing, usually lead to a more obvious preference leakage. This observation also applied judgment dimension analysis, as objective dimensions (like completeness) have an overall lower leakage degree compared with subjective ones (like fairness). This suggests that preference leakage tends be more significant in objective questions and dimensions, where the contaminated model is more likely to receive biased preference.
than standard answer, such
5.7 EFFECT OF SPURIOUS FEATURES ON PREFERENCE LEAKAGE
Table Effect removing spurious features on the PLS. We consider style, format and wording as potential spurious features in this analysis.
Setting
GPT & Gemini
GPT & LLaMA
LLaMA & Gemini
Baseline
17.5%
2.3%
18.8%
w/o style
9.0%
3.3%
14.6%
w/o format
9.8%
1.9%
14.5%
w/o wording
11.2%
2.4%
18.2%
To further validate that spurious stylistic or formatting cues contribute to preference leakage, we conduct an additional ablation experiment focusing on three major feature categories: style, format,

---

Published as a conference paper at ICLR 2026
and wording. Using the Qwen-3-8B 3-BB model the rewriting model, we apply paraphrasing pipeline to selectively remove each type of spurious feature from model responses before evaluation. We employ Gemini-2.0 as the judge model since Gemini-1.5 is no longer available at the time did this experiment. The rewriting process ensures that the semantic content of each response remains intact while selectively neutralizing surface-level artifacts such as syntactic rhythm, punctuation patterns, and lexical framing cues. By isolating these variables, the experiment provides a more direct lens into how superficial similarity between generator and judge responses shapes preference leakage.
The resulting PLS are reported in Table 6 We observe notable reductions in PLS for the two model pairs that originally exhibited the strongest leakage (GPT & Gemini, LLaMA & Gemini), confirming that removing spurious stylistic alignment substantially mitigates bias. Among the three feature types, eliminating style and format yields the largest decrease in leakage suggesting that judges tend to rely heavily on stylistic egularities-such as tone consistency, sentence cadence, and punctuation density- whhnn forming preference judgments. In contrast, removing wording-level features (e.g.. synonym substitution or phrase order changes) produces only minor improvements, implying that lexical similarity alone is not the dominant driver. Interestingly, the magnitude of reduction varies across judge families: GPT-based judges appear especially responsive stylistic coherence, while LLaMA-based judges are more influenced by formatting regularity. This diversity in sensitivity indicates that each model family has distinct perceptual priors about linguistic structure, which can amplify different forms of spurious correlation. Overall, these findings empirically substantiate our mechanistic explanation that stylistic and formatting artifacts embedded in student models act as hidden conduits for preference leakage, shaping judge behavior through subtle surface-level mimicry rather than semantic alignment.
5.8 EXPLORING MITIGATION METHOD FOR PREFERENCE LEAKAGE
To benchmark and explore mitigation methods for preference Icakage, we collected human-labeled pair- wise judgments from several reward benchmarks, in- cluding PPE [Perez et al.] (002)), MTBench ZZeng al. (2023b). and Human Preference Chiang et al. (2024). Using GPT-4 as the target model, we selected samples in which one of the responses was gener- ated by GPT-4's related student (e.g., Vicuna, Al- paca). We then tested several mitigation methods on this dataset, including prompting, chain-of -hfthugghtt (CoT), paraphrasing, auto-calibration, and contextual calibration.
Table 7: Error Bias with various mitigation methods (lower is better).
Method
Error Bias
Base
17.8
+ Prompting
18.3
+ Chain-of-Thought
15.6
Paraphrase
18.7
+ Auto Calibration
20.7
+ Contextual Calibration
7.3
The explored mitigation strategies can be grouped into two complementary layers: (i) Input- or reasoning- debiasing (prompting, CoT, paraphrasing) that modifies inputs or reasoning chains; and (ii) Output-level calibration (auto- or contextual calibration) that adjusts scores post-hoc. We further propose new metric, Error Bias,
Nasgmaatn Intuitively. this other- preferred errors; a value close to 0 indicates that preference leakage is mitigated. Our preliminary results show that contextual calibration with an additional held-out set for bias adjustment is the most effective, reducing Error Bias from 17.8 to 7.3. We provide a more detailed explanation about each method in Appendix C.4
based on human-labeled judgments: ErrorBias Nappseelissttewn
Natmenn
metric quantifies the difference between target-preferret errors and other-
CONCLUSION
In this work, formally highlight the preference leakage problem in LLMe-s-a-judde systems. The results of our main experiment, measured using the proposed preference leakage score, reveal clear bias in each judge toward their respective student model. We also observe that this bias is more pronounced in certain question types and smaller student models. Furthermore, we conduct additional analysis on various factors, including the relationship between the data generator and judge LLMs, model tuning techniques, data mixing strategies, and real-world applications. Our findings suggest that preference leakage can cause significant bias across diverse scenarios. Finally, we investigate the underlying mechanisms of preference leakage, demonstrating that it is a challenging and hard-to-detect issue, especially in subjective questions and judgment dimensions.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
We adhere to the ICLR Code of Ethics. No private, sensitive, personally identifiable data involved. Our work does not raise foreseeable ethical concerns or produce harmful societal outcomes.
REPRODUCIBILITY STATEMENT
Reproducibility is central work. All datasets used in experiments standard benchmarks that are publicly available. We provide full details of the training setup, model architectures, and evaluation metrics in the main paper and appendix. Upon acceptance, we will release our codebase, including scripts for preprocessing. training, and evaluation, along with configuration files and documentation to facilitate exact reproduction of our results. Random seeds and hyperparameters will also be included to further ensure reproducibility.
ACKNOWLEDGEMENT
This work was supported by the U.S. Department of Defense, Army Research Office (Award No. W91 INF-24-2-0175) The views and conclusions contained in this document those of the authors and should not be interpreted as representing official policies, either expressed or implied, of DOD or ARO.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, lige Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et Gpt-4 technical report. ArXiv preprint, abs/2303.08774 2023. URL http::t/tas/tttitt org hatp///a///// /tabb//2333.. 08774
Simone Balloccu, Patrícia Schmidtová, Mateusz Lango, and Ondřej Dušek. Leak, cheat, repeat: Data contamination and evaluation malpractices in closed-source llms. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 67-93, 2024.
Alimohammad Beigi, Bohan Jiang, Dawei Li, Zhen Tan, Pouya Shaeri, Tharindu Kumarage, Amrita Bhattacharjee, and Huan Liu. Can Ilms improve multimodal fact-checking by asking relevant questions? arXiv preprint arXiv:2410006616 2024.
Sebastian Bordt, Harsha Nori, and Rich Caruana. Elephants never forget: Testing language models for memorization of tabular data. In NeurlPS 2023 Second Table Representation Learning Workshop.
2024.
Guiming Hardy Chen, Shunian Chen, Ziche Liu, Feng Jiang, and Benyou Wang. Humans or llms as the judge? study on judgement biases. arXiv preprint arXiv:2402. 10669, 2024.
Wei-Lin Chen, Zhepei Wei, Xinyu Zhu, Shi Feng, and Yu Meng. Do Ilm evaluators prefer themselves for a reason? arXiv preprint arXiv:2504.03846, 2025.
Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Hao Zhang, Banghua Zhu, Michael Jordan, Joseph E. Gonzalez, and lon Stoica. Chatbot arena: An open platform for evaluating Ilms by human preference, 2024.
Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Bingxiang He, Wei Zhu, Yuan Ni, Guotong Xie, Ruobing Xie, Yankai Lin, et al. Ultrafeedback: Boosting language models with sealed ai feedback. In Forty-first International Conference on Machine Learning, 2024.
Chunyuan Deng, Yilun Zhao, Yuzhao Heng, Yitong Li, Jiannan Cao, Xiangru Tang, and Arman Cohan. Unveiling the spectrum of data contamination in language models: A survey from detection to remediation. arXiv preprint arXiv:2406. 14644, 2024a.
11

---

Published as a conference paper at ICLR 2026
Chunyuan Deng, Yilun Zhao, Xiangru Tang, Mark Gerstein, and Arman Cohan. Investigating data contamination in modern benchmarks for large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume I: Long Pupers), pp. 8698-8711, 2024b.
Jesse Dodge, Maarten Sap, Ana Marasović, William Agnew, Gabriel Ilharco, Dirk Groeneveld. Margaret Mitchell, and Matt Gardner. Documenting large webtext corpora: A case study on the colossal clean crawled corpus. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 1286-1305, 2021.
Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Baobao Chang, et al. A survey in-context learning. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 1107-1128, 2024a.
Yihong Dong, Xue Jiang, Huanyu Liu, Zhi Jin, Bin Gu, Mengfei Yang, and Ge Li. Generalization or memorization: Data contamination and trustworthy evaluation for large language models. arXiv preprint arXiv:2400. 15938, 2024b.
Sunny Duan, Mikail Khona, Abhiram Iyer, Rylan Schaeffer, and Ila R Ficte. Uncovering latent memories: Assessing data leakage and memorization patterns in large language models. arXiv preprint arXiv:2406. (454:2996..4549, 2024.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, al. The llama herd of models. arXiv preprint arXiv:2407.2 1783, 2024.
Yann Dubois, Balázs Galambosi, Percy Liang, and Tatsunori B Hashimoto. Length-controoled alpacaeval: A simple way debias automatic evaluators. arXiv preprint arXiv:2404.00 044454444444445 04475, 2024.
Ruyi Gan, Ziwei Wu, Renliang Sun, Junyu Lu, Xiaojun Wu, Dixiang Zhang, Kunhao Pan, Ping Yang, Qi Yang. Jaaxing Zhang, et Ziya2: Data-centric learning is all llms need. arXiv preprint arXiv:2311.03301, 2023.
Mingqi Gao, Jie Ruan, Renliang Sun, Xunjian Yin, Shiping Yang. and Xiaojun Wan. Human-like summarization evaluation with chatgpt. arXiv preprint arXiv:2304.0255:5 2023.
Shahriar Golchin and Mihai Surdeanu. Time travel in 1lms: Tracing data contamination in large language models. arXiv preprint wrXiv:2308.084933 2023.
Jamie Hayes, Ilia Shumailov, William P Porter, and Aneesh Pappu. Measuring memorization in rlhf for code completion. In The Thirteenth International Conference on Learning Representations.
Shengding Hu, Yifan Luo, Huadong Wang, Xingyi Cheng, Zhiyuan Liu, and Maosong Sun. Won't get fooled again: Answering questions with false premises. In Proceedings of the 61 Anmual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.
2023.
Wei-Chieh Huang, Weizhi Zhang, Yueqing Liang, Yuanchen Bei, Yankai Chen, Tao Feng, Xinyu Pan, Zhen Tan, Yu Wang, Tianxin Wei, et al. Rethinking memory mechanisms of foundation agents in the second half. arXiv preprint arXiv:2602. 06052, 2026.
Yue Huang, Jiawen Shi, Yuan Li, Chenrui Fan, Siyuan Wu, Qihui Zhang, Yixin Liu, Pan Zhou, Yao Wan, Neil Zhenqiang Gong, al. Metatool benchmark for large language models: Deciding 2023.
whether to use tools and which to use. arXiv preprint arXiv:2310.0312
Yue Huang, Lichao Sun, Haoran Wang. Siyuan Wu, Qihui Zhang, Yuan Li, Chujie Gao, Yixin Huang. Wenhan Lyu, Yixuan Zhang, et al. Position: Trustllm: Trustworthiness in large language models. In International Conference on Machine Learning, pp. 20166-20270. PMLR, 2024.
Aaron Jacch, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El- Ei-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai ol system card. arXiv preprint arXiv:2412. 16720, 2024.
12

---

Published as a conference paper at ICLR 2026
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.06825 2023.
Bohan Jiang, Dawei Li, Zhen Tan, Xinyi Zhou, Ashwin Rao, Kristina Lerman, H Russell Bernard, and Huan Liu. Assessing the impact of conspiracy theories using large language models. arXiv preprint arXiv:24/2.07012 2024a.
Minhao Jiang, Ken Ziyu Liu, Ming Zhong, Rylan Schaeffer, Siru Ouyang, Jiawei Han, and Sanmi Koyejo. Investigating data contamination for pre-training language models. arXiv preprint arXiv:2401.0660 2024b.
Seungone Kim, Jamin Shin, Yejin Cho, Joel Jang, Shayne Longpre, Hwaran Lee, Sangdoo Yun, Seongjin Shin, Sungdong Kim, James Thome, et al. Prometheus: Inducing fine-gratd ane-grained eval- uation capability in language models. In The Twelfth International Conference Learning Representations, 2023.
Seungone Kim, Juyoung Suk, Shayne Longpre, Bill Yuchen Lin, Jamin Shin, Sean Welleck, Graham Neubig, Moontae Lee, Kyungjae Lee, and Minjoon Seo. Prometheus 2: An open source language model specialized in evaluating other language models. arXiv preprint arXiv:2405.01
2024.
Ryan Koo, Minhwa Lee, Vipul Raheja, Jong Inn Park, Zae Myung Kim, and Dongyeop Kang. Bench- marking cognitive biases in large language models as evaluators. arXiv preprint arXiv:2309.17011 2023.
Ryan Koo, Minhwa Lee, Vipul Raheja, Jong Inn Park, Zae Myung Kim, and Dongyeop Kang. Benchmarking cognitive biases in large language models as evaluators. In ACL (Findings), 2024.
Andreas Köpf, Yannic Kilcher, Dimitri von Rütte, Sotiris Anagnostidis, Zhi Rui Tam, Keith Stevens, Abdullah Barhoum, Due Nguyen, Oliver Stanley, Richard Nagyfi, et al. Openassistant conversation-ddemocaaiiingg large language model alignment. Advances in Neural Information Processing Systems, 36, 2024.
Harrison Lee, Samrat Phatale, Hassan Mansoor, Thomas Mesnard, Johan Ferret, Kellie Ren Lu, Colton Bishop, Ethan Hall, Victor Carbune, Abhinav Rastogi, et al. Rlaif vs. rlhf: Scaling rein- forcement learning from human feedback with ai feedback. In Forty-first International Conference on Machine Learning. 2024.
Sunbowen Lee, Junting Zhou, Chang Ao, Kaige Li, Xinrun Du, Sirui He, Jiaheng Liu, Min Yang, Zhoufutu Wen, and Shiwen Ni. Distillation quantification for large language models. arXiv preprint arXiv:2501. 2025.
Changmao Li and Jeffrey Flanigan. Task contamination: Language models may not be few-shot anymore. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp- 18471-18480, 2024.
Dawei Li, Bohan Jiang, Liangjie Huang, Alimohammad Beigi, Chengshuai Zhao, Zhen Tan, Amrita Bhattacharjee, Yuxuan Jiang, Canyu Chen, Tianhao Wu, et al. From generation to judgment: Opportunities and challenges of llm-as-a-uudge. arXiv preprint arXiv:2411.16594, 2024a.
Dawei Li, Zhen Tan, Tianlong Chen, and Huan Liu. Contextualization distillation from large language model for knowledge graph completion. arXiv preprint arXiv:2402.01122, 2024b.
Dawei Li. Shu Yang. Zhen Tan, Jae Young Baik, Sunkwon Yun, Joseph Lee, Aaron Chacko, Bojian Hou, Duy Duong-Tran, Ying Ding, et al. Dalk: Dynamic co-augmentation of llms and kg to answer alzheimer's disease questions with scientific literature. arXiv preprint arX5v:2405.049
2024c.
Dawei Li, Zhen Tan, and Huan Liu. Exploring large language models for feature selection: A data-centric perspective. ACM SIGKDD Explorations Newsletter, 26(2):44-53, 2025a.
Dawei Li, Zhen Tan, Chengshuai Zhao, Bohan Jiang, Baixiang Huang. Pingchuan Ma. Abdullah Alnaibari, Kai Shu, and Huan Liu. Who's your judge? on the detectability of Imm- -m-eenerated judgments. arXiv preprint arXiv:2509.25 25099.555444 2025b.
13

---

Published as a conference paper at ICLR 2026
Dawei Li, Yuguang Yao, Zhen Tan, Huan Liu, and Ruocheng Guo. Toolprmbench: Evaluating and advancing process reward models for tool-using agents. arXiv preprint arXiv:260.112994, 2026.
Ming Li, Lichang Chen, Jiuhai Chen, Shwai He, Jiuxiang Gu, and Tianyi Zhou. Selective reflection- tuning: Student-selected data recycling for Ilm coruuciion-tuingg arXiv preprint arXiv:2402.10110, 2024d.
Tianle Li, Wei-Lin Chiang, Evan Frick, Lisa Dunlap, Tianhao Wu, Banghua Zhu, Joseph Gonzalez, and lon Stoica. From crowdsourced data high-quality benchmarks: Arena- hard and benchbuilder pipeline. arXiv preprint arXiv:2406.11939 2024e.
Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. Alpacaeval: An automatic evaluator of instruction folloooin models, 2023.
Chin- Lin. Rouge: A package for automatic evaluation of summaries. Text summarization branches out, pp. 74-81, 2004.
Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falschoods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3214-3252, 2022.
Chia-Wes Wei Liu, Ryan Lowe, lulian Serban, Mike Noseworthy, Laurent Charlin, and Joelle Pineau. How NOT to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation. In Jian Su, Kevin Duh, and Xavier Carreras (eds.). Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 2122-2132. Austin, Texas, 2016. Association for Computational Linguistics. doi: 10. 10.18653/v1/D16-1223 URL
12300
Wei Liu, Weihao Zeng, Keqing He, Yong Jiang, and Junxian He. What makes good data for alignment? a comprehensive study of automatic data selection in instruction tuning. The Twelfth International Conference on Learning Representations, 2024a.
Xiao Liu, Xuanyu Lei, Shengyuan Wang, Yue Huang, Zhuoer Feng. Bosi Wen, Jiale Cheng, Pei Ke, Yifan Xu, Weng Lam Tam, et al. Alignbench: Benchmarking chinese alignment of large language 2023a.
models. arXiv preprint
Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, et al. Agentbench: Evaluating Ilms as agents. arXiv preprint arXiv:2308. 03688, 2023b.
Yiqi Liu, Nafise Sadat Moosavi, and Chenghua Lin. Lims as narcissistic evaluators: When ego inflates evaluation scores. In Findings of the Association for Computational Linguisties ACL 2024, pp. 12688-12701, 2024b.
Jinjie Ni, Fuzhao Xue, Xiang Yue, Yuntian Deng, Mahir Shah, Kabir Jain, Graham Neubig, and Yang You. Mixeval: Deriving wisdom of the crowd from 1l benchmark mixtures. arXiv preprini arXiv:2406.06566 06566, 2024.
Arjun Panickssery, Samuel R Bowman, and Shi Feng. Llm evaluators recognize and favor their own generations. arXiv preprint arXiv:2404. 13076, 2024.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp. 311311333 311-318, 2002.
Ethan Perez et al. Ppe: Pairwise preference evaluation benchmark for large language models, 2022.
URL wttphttppp// httpp::////.tub.cccoccpp thub. o ippe
Huaizhi Qu, Inyoung Choi, Zhen Tan, Song Wang, Sukwon Yun, Qi Long, Faizan Siddiqui, Kwonjoon Lee, and Tianlong Chen. Efficient map estimation of Ilm judgment performance with prior transfer. arXiv preprint 25t.:2500.22889 2025.
14

---

Published as a conference paper at ICLR 2026
Rafael Rafailov, Archit Sharma, Erie Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024.
Javier Rando, Jie Zhang, Nicholas Carlini, and Florian Tramèr. Adversarial ml problems are getting harder to solve and evaluate. arXiv preprint arXiv:2502.02260, 2025.
Ehud Reiter. A structured review of the validity of BLEU. Computational Linguistics, 44(3):393-40,, 2018. doi: 10.1162/coli_a_a aatp:colll_a_000330 a_03222.
--002
Jiawen Shi, Zenghui Yuan, Yinuo Liu, Yue Huang, Pan Zhou, Lichao Sun, and Neil Zhenqiang Gong. Optinization-base prompt injection attack to lim-as-a-judge- In Proceedings of the 2024 on
ACM SIGSAC Conference Computer and Communications Security, CCS 24, pp. 660-674, New York. NY, USA, 2024. Association for Computing Machinery, ISBN 9798400706363. doi: 24.144//66584443300222 URLhttps: / org/ 1145 3t..//355664444 3690291
Renliang Sun, Mengyuan Liu, Shiping Yang, Rui Wang, Junqing He, and Jiaxing Zhang. Fostering natural conversation large language models with nico: a natural interactive conversation dataset. arXiv preprint urXiv:2408.093300 2024a.
Taaniiang Sun, Xiaotian Zhang, Zhengfu He, Peng Li, Qinyuan Cheng, Xiangyang Liu, Hang Yan, Yunfan Shao, Qiong Tang, Shiduo Zhang, Xingjian Zhao, Ke Chen, Yining Zheng, Zhejian Zhou, Ruixiao Li, Jun Zhan, Yunhua Zhou, Linyang Li, Xiaogui Yang, Lingling Wu, Zhangyue Yin, Xuanjing Huang, Yu-Gang Jiang, and Xipeng Qiu. Moss: An open conversational large language model. Machine Intelligence Research, 2024b. ISSN 2731-5398
URL https
//github. wwt.com///ooo/. wwwpw//wwww.//////a// MOSS
Zhen Tan, Lu Cheng, Song Wang, Bo Yuan, Jundong Li, and Huan Liu. Interpreting pretrained language models via concept bottlenecks. In Pacific-Asia conference on knowledge discovery and data mining, pp. 56-74. Springer, 2024a.
Zhen Tan, Dawei Li, Song Wang, Alimohammad Beigi, Bohan Jiang, Amrita Bhattacharjee, Man- sooreh Karami, Jundong Li, Lu Cheng, and Huan Liu. Large language models for data annotation and synthesis: A survey. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 930-957, 2024b.
Zhen Tan, Chengshuai Zhao, Song Wang, Jundong Li, Tianlong Chen, et al. Probing to refine: Rein- forcement distillation of IIm reasoners via explanatory inversion. In The Fourleenth International Conference on Learning Representations, 2025.
Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXhv:2403.055300 2024.
Aman Singh Thakur, Kartik Choudhary, Venkat Srinik Ramayapally, Sankaran Vaidyanathan, and Dieuwke Hupkes. Judging the judges: Evaluating alignment and vulnerabilities in alms-as-uudes. arXiv preprint 2r06v:2406. 26244 2024.
Yongqi Tong, Dawei Li. Sizhe Wang, Yujia Wang, Fei Teng, and Jingbo Shang. Can 1lms learn from previous mistakes? investigating Ilms' errors to boost for reasoning. arXiv preprint arXiv:2403.20046, 2024.
Somin Wadhwa, Chantal Shaib, Silvio Amir, and Byron C Wallace. Who taught you that? tracing teachers in model distillation. arXiv preprint (xXv:2502.00659) 2025.
Sizhe Wang, Yongqi Tong, Hengyuan Zhang, Dawei Li, Xin Zhang, and Tianlong Chen. Bpo: Towards balanced preference optimization between knowledge breadth and depth in alignment. arXiv preprint 2rXiv:2411..09111 2024.
Koki Wataoka, Tsubasa Takahashi, and Ryokan Ri. Self Self-preference bias in llm-as-a-judge. arXiv preprint arXiv:2410.218/9 2024.
15

---

Published as a conference paper at ICLR 2026
Colin White, Samuel Dooley, Manley Roberts, Arka Pal, Ben Feuer, Siddhartha Jain, Ravid Shwartz- Ziv, Neel Jain, Khalid Saifullah, Siddartha Naidu, et al. Livebench: A challenging, contamination- free Ilm benchmark. arXiv preprint arXiv:2406. 19314, 2024.
Siyuan Wu, Yue Huang, Chujie Gao, Dongping Chen, Qihui Zhang, Yao Wan, Tianyi Zhou, Xian- gliang Zhang, Jianfeng Gao, Chaowei Xiao, et al. Unigen: A unified framework for textual dataset generation using large language models. arXiv preprint arXiv:2406. 18966, 2024.
Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. Wizardlm: Empowering large language models to follow complex instructions. arXiv preprint arXiv:2304.12244 2023.
Cheng Xu, Shuhao Guan, Derek Greene, M Kechadi, et al. Benchmark data contamination of large language models: A survey, arXiv preprini arXiv:2406.04220 04244,, 2024a.
Wenda Xu, Guanglei Zhu, Xuandong Zhao, Liangming Pan, Lei Li, and William Wang. Pride and prejudice: Llm amplifies self-bias in self-refinement. In Proceedings of the 62nd Anmual Meeting of the Association for Computational Linguistics (Volume Long Papers), pp. 15474-15492, 2024b.
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.151155 2024.
Shiping Yang, Renliang Sun, and Xiaojun Wan. A new dataset and empirical study for sentence simplification in chinese. In Proceedings of the 6lst Anmual Meeting of the Association for Computational Linguistics (Volume Long Papers), pp. 8306 0321, 2023.
Feng Yao, Yufan Zhuang, Zihao Sun, Sunan Xu, Animesh Kumar, and Jingho Shang. Data contami- nation can cross language barriers. arXiv preprint 240.::2000..2336, 2024.
Jiayi Ye, Yanbo Wang, Yue Huang, Dongping Chen, Qihui Zhang. Nuno Moniz, Tian Gao, Werner Geyer, Chao Huang, Pin- Yu Chen, et al. Justice or prejudice? quantifying biases in llm-as-a-judge. arXiv preprint arXiv:2410.027:: 2024.
Hengyuan Zhang, Dawei Li, Yanran Li, Chenming Shang. Chufan Shi, and Yong Jiang. Assisting language learners: Automated trans-lingual definition generation via contrastive prompt learning. 2023.
arXiv preprint arXiv:2306.000
Hengyuan Zhang, Chenming Shang. Sizhe Wang, Dongdong Zhang, Feng Yao, Renliang Sun, Yiyao Yu, Yujiu Yang, and Furu Wei. Shifcon: Enhancing non-dominant language capabilities with thift-based contrastive framework. arXiv preprint 2rXiv22410.19453. 2024a.
Hengyuan Zhang, Yanru Wu, Dawei Li, Zacc Yang, Rui Zhao, Yong Jiang, and Fei Tan. Balancing speciality and versatility: a coarse to fine framework for supervised fine-tuning large language model. arXiv preprint arXiv:2404. 100.0006000 2024b.
Qihui Zhang, Chujie Gao, Dongping Chen, Yue Huang, Yixin Huang, Zhenyang Sun, Shilin Zhang, Weiye Li, Zhengyan Fu, Yao Wan, and Lichao Sun. a.M-assa-coauthhr: Can mixed human-written and machine generated text be detected? In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Findings of the Association for Computational Linguistics: NAACL 2024, pp. 409 436, Mexico City, Mexico, June 2024c. Association for Computational Linguisties. doi: 10.18653/w1/2024 findings-naacl... 29. URL https: a ha wwg.0/4... findi ngs naacl. .29/
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger. and Yoav Artzi. Bertscore: Evaluating text generation with bert. International Conference on Learning Representations, 2020.
Xiaoying Zhang, Baolin Peng. Ye Tian, Jingyan Zhou, Lifeng Jin, Linfeng Song, Haitao Mi, and Helen Meng. Self-alignment for factuality: Mitigating hallucinations in LLMs via self-evaluation. In Lun- Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Anmual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1946-1963 Bangkok, Thailand, August 2024d. Association for Computational Linguistics. doi: 10.18653/v1/2072 acl-long. 107. TRRlhttps: //ac htt.t::ss///aaaaaoloooooooooooo org. acl 107/
16

---

Published as a conference paper at ICLR 2026
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, with mt-bench and
Zi Lin, Zhuohan Li, Dacheng Li. Eric Xing, et al. Judging chatbot arena. Advances in Neural Information Processing Systems, 06:46595-466223 2023a.
Lianmin Zheng, Jiayi Wu, Shaohan Zhuang, al. Mt-bench: Evaluating large language models with multi-turn benchmarks. In arXiv preprint arXiv:2306. 05685, 2023b.
Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. arXiv preprint arXiv:2403. 13372, 2024.
Ming Zhong, Yang Liu, Da Yin, Yuning Mao, Yizhu Jiao, Pengfei Liu, Chenguang Zhu, Heng Ji, and Jiawei Han. Towards a unified multi-dimensional evaluator for text generation. In Yoav Goldberg. Zornitsa Kozareva, and Yue Zhang (eds.). Proceedings of the 2022 Conference Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pp. 2023-2038. Association for Computational Linguisties, 2022. doi: EMNLL-MIN..- URL https: wwtpoo...1../////./.... em:.p-mainn.13
10.
Ming Zhong, Aston Zhang, Xuewei Wang, Rui Hou, Wenhan Xiong, Chenguang Zhu, Zhengxing Chen, Liang Tan, Chloe Bi, Mike Lewis, et al. Law of the weakest link: Cross capabilities of large language models. arXiv preprint arXiv:2499.19951, 2024.
Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, al. Lima: Less is more for alignment. Advances in Neural Information Processing Systems, 36, 2024.
Mingehen Zhuge, Changsheng Zhao, Dylan Ashley, Wenyi Wang, Dmitrii Khizbullin, Yunyang Xiong, Zechun Liu, Ernie Chang, Raghuraman Krishnamoorthi, Yuandong Tian, al. Evaluate agents with agents. arXiv preprint 10934, 2024.
17

---

Published as a conference paper at ICLR 2026
A THE USE OF LLMS FOR WRITING
We employed Google's Gemini 2.5 Pro and OpenAT's GPT-5 as writing assistance tools during the preparation of this manuscript. Their role was exclusively for language refinement, such as improving readability and rephrasing for clarity in an academic writing style. This usage aligns with standard academic practices for language polishing.
B PRELIMINARY STUDY OF PREFERENCE LEAKAGE IN REAL WORLD
In our preliminary study, we investigate whether preference leakage is real- woald issue in mainstream leaderboards and benchmarks. To this end, we examine two widely used LLM-as-a-judge leaderboards (AlpacaEval 2.0 and Arena-Hard) and well-known benchmark (MTBench). All three rely GPT-4 as the judge model and report pairwise judgment results for various LLMs. Our analysis reveals that several candidate models distilled from GPT-4 or other GPT-series models (e.g.. Vieuna and Alpaca) appear across all these leaderboards and benchmarks, suggesting that preference leakage pervasive issue in these datasets. Besides, also examine if preference leakage exists in LLM-relevant research studies and also find bunch of work utilizing the same related model(s) to do distillation/ data synthesis and evaluation Yang al. (2023): Liu et al.) (2024a); Lee al. (2024); Li et al. (2024d); Wang et al. (2024); Sun et al. (2024a). All of these suggest preference Icakage to be a widespread problem in both a-dduuee datasets and
research.
C EXPERIMENT DETAILS
C.I TRAINING DETAILS
We use LLaMoyFactory Zheng et al. (2024). an efficient LLM tuning library for our experiment. The maximum sequence length set to 1024 tokens, and cutoff length of 1024 tokens is enforced to prevent excessive tokenization. The data preprocessing will be done in parallel with 16 workers to speed up the preparation process. The training use a per-device batch size of with gradient accumulation over 2 steps to simulate larger batch size for SFT and a per-device batch size of 1, with gradient accumulation over 4 steps simulate larger batch size for DPO. The learning rate set to 1.0e-5 and each model will be trained for 3 epochs. A cosine learning rate scheduler is used with warmup ratio of 0.1 to gradually increase the learning rate during the initial steps. All of the experiments use BF16 precision to speed up training while maintaining numerical stability. All the experiments are conducted in Nvidia A A100 GPU cluster with CUDA version 11.8.
Table 8: A case on AlpacaEval 2.0 with the model pair Mistral-GPT--4 vs Mistral-Gemini-Il 1,5 to demonstrate how the preference leakage score calculated.
-Pstral-GPT-4o Geminii-
Judge Model
Mistral-GPPT-4 Wins
Mistral-Gemini-1.5 Wins
GPT-4o
55.1 %
44.9%
Gemini-1.5
36.8%
63.2%
Preference Leakage Score
18.4%
C.2 DETAILED EXPLANATION FOR PREFERENCE LEAKAGE SCORE
We present in Table & to show how calculate the preference leakage score for the Mistral- GPT-4o vs I.5 pair on AlpacaEval 2.0. Based on the definition of preference leakage score, we first calculate:
55.1 + 36.8
AVG(Mistral-GPT-44. Mistriai------ooooirrararaGe Mistral-Gemini-I.. =
45.95%
(7)
2
63.2
44.9
AVGiMisstra-Gemmin 1.5, Mistral-GPP -Pttaal-GPT-oo)
54.05%
(8)
2
18

---

Published as a conference paper at ICLR 2026
After that, calculate the preference leakage score:
55.1 -45.55
63.2-54.00
PLS(MistralGPPT-4o Mattaallll-PPP-.oMiiialleeiiiiii
45,95
54.05
18.4%
(9)
2
C.3 MANUAL ANNOTATION DETAILS RESULTS
While have concluded that student model pairs with similar performance models tend to exhibit greater preference leakage, we also examine whether different data generator and judge LLMs contribute varying degrees of preference leakage. We randomly sample 100 questions from AlpacaEval 2.0 and ask three well-trained annotators to conduct pairwise comparisons of the responses from each model pair for these questions. For annotation efficiency, we also develop an annotation tool that involves the function of uploading multiple model responses, jumping to specific problems, and downloading annotation results (Figure 7 After annotation, we adopt the majority voting to get the final label for each response pair. We also calculate the average agreement of three annotators and find be 78.6, indicating a relatively consistent annotation result.
powerful student
Analyzing the manual annotation results presented in Figure we observe that Gemini 1.5 shows a strong bias toward students, followed by GPT-4o, with LLaMA-3.3 displaying the least bias. This variation in preference leakage may stem from differences in the level of leaked preference in the synthetic responses generated by the data generator LLMs. For instance, an LLM with a distinctive style format in its responses offers more opportunities for student models to learn these characteristics, potentially leading to more pronounced preference leakage during evaluation. Future work could further quantify the extent of leaked preference for each data generator model.
C.4 MITIGATION METHODS DETAILS
Dataset Construction. To systematically benchmark preference leakage, we curate pairwise judgment corpus by consolidating three widely used human-labeled reward datasets: PPE Perez et al. (2022), MTBench Zheng et al. (2023b), and the Human Preference dataset (hhian et al. (2024). Each dataset contains prompts and paired model outputs annotated with human preferences. We treat GPT-4 as the larget model and identify instances where one response originates from GPT-4 and the other from related open-source "student" model (e.g., Vicuna, Alpaca).
Mitigation Methods. We evaluate five representative strategies designed to counteract preference leakage:
Prompting- A straightforward baseline that refines evaluation instructions to explicitly warn against seff-preference, encouraging the evaluator to remain impartial and judge outputs solely on content quality and relevance.
Chain-of-Thought (CoT). Augments the evaluation prompt by encouraging the model articulate an explicit step-by-step reasoning process prior to producing its final decision, thereby reducing unconscious style matching.
Paraphrasing, Reduces lexical and stylistic overlap between the evaluator and candidate outputs by paraphrasing prompts or responses before evaluation, mitigating familiarity-driver bias.
Auto-Carn Aatto-alibbraion. Estimates a global bias term from held-out calibration set by analyzing the evaluator's bog-probabilities of choosing the target versus the student, then shifts future predictions to offset this bias.
Contextual Calibration. Extends calibbaabba by learning context-dependent bias adjustments. For each evaluation scenario, bias is estimated from similar held-out set and applied dynamically at inference time, offering finer-grained debiasing and achieving the strongest reduction in preference leakage.
D ADDITIONAL EXPERIMENTS
Due to the space limitation, put further experiments and analysis in the Appendix.
19

---

Published as a conference paper at ICLR 2026
MistralGPPIoo vs Mistral-Geminé-i
Nistral-dPTdo vs Mistal-LaMM-3.33
MistrakLLaM-3. vs Mistral- Gemini-k.
6PT-4--
52.4%
43.2%
$1.8%
32.2%
24.2%
Juegge Model
U.IMA333
12.0%
27.5%
Gem
H6.44
2145
1719
80.96
Human
64.7%
2.0%
20.0%
100.0%
100.00
6%
62.0%
Model Wina
Model B Win
Figure 4: Manual annotation result on 100 randomly selected samples from AlpacaEval 2.0.
D.I ORIGINAL EXPERIMENT RESULTS FOR PLS CALCULATION
Gentri-1.5
77.4%
0.04
20 0%
40.0%
60 0%
20.2%
100.0%
0.0%
DP
45.0%
02/0%
10.%%
100.%
2.0%
60.0%
$5.04
80.0%
100
(b). Mistra-78
Judggn
Owen GPT4o Owen Cemini-1.5
Qwen GPT4o Qwan-LLaMA-3J
Owan-LLaMA-3. Qwan- Genn---mini-5
GPT-4o
9m
0.25
6.555
24.5%
$1.00
12.4%
102%
ades
(b), Owen 5-144
Wine
Model
Figure 5: Judgment results with GPT-4o, LLaMA-3.3 and Gemini-1.5 on AlpacaEval 2.0. Different from Hard, there is no tie in AlpacaEval 2.0.
Hietral-GPTAa Mistrai Gemini-a.5
Mistral- GPTde Mittral--LaMM-2.3
Misttal-LLaMMA33 Mistral-Gemin-1..
27.2%
22.0%
222hh
3.3
24:25
$2.1b
11,05
26.24
Geminkl.5
2%
8%
13.0%
46.2%
42.7
11.2%
2%2%
31.4%
50.4%
0.0%
20 0%
42.2%
00.0%
16.2%
120.0%
0.6%
20 0%
42.0%
62.9%
50.2%
2010%
10%
20.3%
40.0%
$5.0%
100.0%
Mistril-88
Judge
Gemirk1.
0.0g
200
40
60.0%
10.0%
200 0%
0.0%
20.0%
45.0%
100.%%
3.%
20.04
60.0%
$5.04
80.0%
100.0%
(D). 2.cn-2.5:148
Medel A Wins
Tie
Model Wins
Figure 6: Judgment results with GPT-4o, LLaMA-3.3 and Gemini-1.5 on Arena-Hare
D.2 STABILITY ASSESSMENT OF EXPERIMENTAL RESULTS
Based on the results from three repeated experiments (Table 9 we observe consistently low variance across different comparisons, indicating high stability in performance measurements. This suggests that the conclusions drawn from these experiments are reliable and not significantly affected by random fluctuations, thereby strengthening the validity of findings.
D.3 PROMPT SENSITIVITY ANALYSIS
We examined the robustness of the Preference Leakage Score (PLS) under different evaluation prompts. Two LLM-as-a- LLM-as-a-judge protocols used: ARENAHARD and ALPACAEVAL 2.0, each with distinct prompts and question sets. We rewrote the prompts for both protocols and re-ran the evaluations.
20

---

Published as a conference paper at ICLR 2026
Table 9: Mean and variance of experimental results across two benchmarks in Mistral-7B-vO..l
Model Pairs
Mean
Variance
ArenaHard
mistral-GPTAA vs mistral-Geminis
28.67
0.063
mistral-GPT4o mistral-LLAMA-333
0.50
0.910
mistral-LLLAMA vs mistral-Gemini
12.93
0.583
AlpacaEval 2.0
miPoo mistral-Gemini-3 333
19.20
0.490
mistral-GPT4o mistral-LLAMA-
0.20
1,240
mistral-LLAMA vs mistral-Gemini
19.87
0.013
Table 10: PLS under different evaluation prompts.
Judge Pair
Prompt
Prompt 2
Dataset
18.4%
16.5%
AlpacaEval 2.0
GPT-4o vs Gemini-1.5
28.7%
38.7%
ArenaHard
1.4%
-1.2%
AlpacaEval 2.0
GPT-4o LLaMA-3.3
-1.5%
4.5%
ArenaHard
19.8%
17.9%
AlpacaEval 2.0
LLaMA-3.3 vs Gemini-1.5
13.1%
15.8%
ArenaHard
PLS remained consistently > O for key model pairs; ALPACAEVAL 2.0 was more stable to prompt changes than ARENAHARD.
D.4 STATISTICAL SIGNIFICANCE TESTS
We tested the hypothesis PLS >> 0 using non-parametric bootstrap with 10,000 resamples over 500 prompts in ARENAHARD.
Table 11: Bootstrap significance results for PLS > 0. ***; 0.001, " 0.01.
Judge Pair
Student
PLS (%)
Significance
GPT-4o vs Gemini-1.5 1.5
Mistral-7B
28.5
GPT-4o vs L.L.MA3.3
Mistral-7B
-1.1
I.L.MA-3.3 vs Gemini-1.5
Mistral-7B
GPT-4o vs Gemini-1.5
Qwen-2.5-1444
37.9
www
GPT-4o vs 3LaMA-3.3
Owen-225-14B
1.2
LLaMA-3.3 vs Gemini-1.5
Qwen-2.5-14B
26.3
D.5 LANGUAGE GENERALIZATION
To test cross- lross-lingual generalization, we synthesized Chinese SFT data (using Moss-3 instructions) and evaluated with Chinese versions of ARENAHARD Am-ARENAHARD) and XALPACAEVAL. Judges were GPT-4o and Gemini- the student model was Qwen-3-SB.
Significant preference leakage also appears in the Chinese setting.
D.6 EXPANDED
PAIRS
We added the judge model Claude-3.5. 3.5-ee3..Snoeeeee form three new judge pairs: GPT-4o & Claude-3.5, Gemini & Claude-3.5, and LLaMA-3.3 & Claude-3.5. Student models: Mistral-7B and Qwen-2.5- 14B.
21

---

Published as a conference paper at ICLR 2026
Table 12: PL.S in English vs. Chinese.
Language
AlpacaEval 2.0
ArenaHard
Avg
English Chinese
17.4% 12.3%
33.9%
25.7%
51.8%
32.1%
Table 13: PLS of new judge pairs (negative values indicate no leakage).
Judge Pair
ArenaHard
AlpacaEval 2.0
Avg
Mistral-7B
GPT-4o & Claude-3.5
12.2%
8.6%
10.4%
Gemini & Claude-3.5
16.5%
7.1%
11.8%
LLaMA-3.3 & Claude-3.5
-4.4%
-2.6%
-3.5%
Qwen-2.5-14B
GPT-4o Claude-3.5
13.0%
10.4%
11.7%
Gemini & Claude-3.5
18.5%
11.1%
14.8%
LLLMAA3.3 & Claude-3.5
0.0%
1.7%
0.9%
D.7 STUDENT MODEL SCALING
We tested PLS on wider range of student sizes within the Qwen and LLaMA families.
Table 14: PLS (%) for different student sizes.
Student
ArenaHard
AlpacaEval 2.0
Avg
LLaMA-3-IB
35.4
18.2
26.8
LLaMA-3-3B
32.5
16.4
24.5
LLaMA-3-8B
30.9
15.8
23.4
Qwen-2.5-0.3
40.9
21.2
31.1
Qwen-2.5-1.5B
38.0
23.2
30.6
Qwen-2.5-3B
50.7
20.1
35.4
Qwen-2.5-7B
32.2
22.1
27.2
Qwen-2.5-144
37.1
18.6
27.9
Qwen-3-Oke 3-B6-B
39,8
23.8
31.8
Qwen-3-1.7B
40.0
20.1
30.2
Qwen-3-4B
30.9
17.2
24.1
Qwen-3-8B
33.9
17.4
25.7
Qwen-3-14B
31.7
19.4
25.6
Within each family, smaller models generally exhibit higher PLS.
D.8 MITIGATION METHODS AND ERROR BIAS METRIC
We explored mitigation methods on human- labeled reward dataset, including: prompting, chain-of- thought (CoT), paraphrasing, cuto-calibration, and contextual calibration. We introduced the Error Bias metric:
Nurget prefer-other-win
N Nother- wwhher-rrfeer-trggee..oo
BrrorBias
(10)
N Nother-win
Winget-win
Contextual calibration with an additional held-out bias-adjustment set yielded the largest reduction.
E LEARNING METHOD ANALYSIS DETAILS
The table below presents the prompt generate synthetic pairwise feedback from each model.
22

---

Published as a conference paper at ICLR 2026
Pairwise Feedback Prompt
Please act impartial judge and evaluate the quality of the responses provided by two AI assistants to the user questien displayed below. Your evaluation should consider correctness and helpfulness. You will be given assistant A's answer, and assistant B's answer. Your job is to evaluate which assistant's is better. You should independently solve the st- first. Then compare beth assistants' answers with your answer. Identify and correct any mistakes. Avoid any
question
position biases and ensure that the order in which the respenses presented does not influence your decision. Do not allow the length of the responses to influence your evaluation. Do not favor certain names of the assistants. Be as objective as possible. After previding your explanation, output your final verdict by strictly fol following this format: "[[A]]" if assistant A is better, "[[B]]" if assistant B is better.
# Instruction:
[The Start of Assistant A/s Answer] IRESPONSE Al [Te End of Assistant A's Answerl
[Tee [RESPONSK 8]
Assistant Answer]
[The End of Assistant B's Answer]
Please output the generated content in a json format, for example: { "reason": // string, reasons behind the chosen preferred answer "prefered answer": // string, the prefered answer you selected, [[A]] or [[B]]
Formatted the abovementioned schema and produce the and preferred answer:
REAL-WORLD IMPACT ANALYSIS DETAILS
In the real-world impact analysis section, we the models that appear in both LMArena and AlpacaEval 2.0 leaderboard, including: GPT--0-2024-05-13, GP7-44-mini-2024-07--88 Meta- Nama-3.114055 --00S--Inttuutt-bl16 GPT-4-Turo-20224-0444
GPT-441106-prevvieer Meta-Llama-3.1 70B-Instruct, Claude 3 Opus, Llama-3-70B-Instrue Claude 3 Sonnet, Qwen2-72B-Instruee, GPT-4-
0314, Meta-Llama-3, Insttuutt GPT-4-0613, Mistral-Large-2402,, mand R (04-2024), Mistral Medium, Mixtral 1x222--nstruuttoo...1 0ww...5-2-..l1........... Gemini Pro, Yi-34B-Chat, Mixtral--x77--Intteeee 1.w.1..5111e.11514...4... 15-114B-hht,, GPT-355- DBRX-Instruct-
Com-
Preview, Tul-2--0O-00000, Llama-2-70B-chat Vicuna-33B, Gemma-1.1-7B8 OpenHermes-2.55 Mistral-7B,
Qwen1 1--7-Chha
GPT-3.5-Turbo- 1aana-2-13b-che
WizardLM-13b-v1.22. Vicuna- Llama-2-7B-che Guanaco-33B, Vicuna-7B, Gemma-2B
OpenAAsiitamm--Pthhiaa--
G RECOGNITON ANALYSIS DETAILS
The table below presents the pointwise and pairwise prompts for the recognition analysis.
Pointwise Recognition Prompt
Given instruction and response, your task is to judge whether this response generated by model that is trained on a synthetic dataset you produced. (your student model).
23

---

Published as a conference paper at ICLR 2026
## Instruction:
INSTRUCTION]
## Response:
[Response]
Please output the generated content in a json format, for example: "reason": // string, reasons behind the judgment "judgment": // string, whether the choose from yes or no
is generated by your student medel,
Formatted abovementioned schema and produce reasch judgment:
Pairwise Recognition Prompt
Given instruction and responses, your task is judge which response is generated by a model that is trained on synthetic dataset you produced (your student model).
## Instruction:
INSTRUCTION]
## Responsel:
[Response
1]
## Response2:
[Response
Please output the generated content in json format, for example: "reason": // string, reasons behind the judgment "judgment""; // int, 1 or 2, means responsel or response2 is from your student model
Formatted the ibovementioned schema and produce the reasch and judgment:
For response classification, we split all the response from three student models into training (80%) and testing (20%) subsets. Then, finetune bERT-base-uncased model in the training set. The model trained for 3 epochs with a learning rate 2c-5, a batch size of 16 for both training and evaluation, and weight decay of 0.01, with evaluations conducted the end of each epoch.
H CATEGORY ANALYSIS DETAILS
The tables below present the prompt use for question type and judgment dimension catcogory analysis.
Question Type Categorization Prompt
Given a question, please categorize to one of the follewing categories:
1. Computer Science Programming
2. Mathematics & Statistics
3. Science & Engineering
4. Business Finance
5. Writing & Communication
6. Social Daily Life
7. Others
# Question:
[QUESTION]
24

---

Published as a conference paper at ICLR 2026
Please output the generated content in json format, for example: "question categozy": // string, specific category name, such "Computer Science Programming"
Formatted abovementioone schema and categorize the given question:
Judgment Dimension Categorization Prompt
Given a pairwise comparison judgment made by an AI, please categorize each considered aspect in the rationale to one of the following categories:
"Whether the information provided in the response is acourate, based reliable facts and data.",
"Factuality":
"User Satisfaction"; "Whether the response meets the user's question and needs, and provides comprehensive and appropriate answer to the question.",
"Logical Coherence"; "Mhether the response maintains overall consistency and logical coherence between different secticns, avoiding self-contraadetioon.,
"Richness"; "Whether the response includes rich into, depth, context, diversity, detailed explanatiens and examples to meet user needs and provide a comprehensive understanding.""
"Creativity": "Whether the response is innovative or unique, providing novel insights soluticns.",
"Fairness and Responsibilityy: "Whether the advice information provided in the response is feasible, carries acertain degree of responsibility, and considers potential risks and consequences."
"Completeness"; "Whether the respense provides sufficient. information and details to meet the user's s needs, and whether it avoids omitting important aspects.",
"Clarity": "Whether the response is clear and understandable, and whether it uses concise language and structure that the user casily understand it.",
"Others": "Other aspects which is not listed above."
## Judgment:
[JUDGMENT]
Please output the generated content in a json format, for example: { "Factuality"; // list, all aspects that belong to this category, such as ("correctness", "mistakes")
Formatted the abovementioned schema and categorize aspects in the judgment:
I BROADER IMPACT
By revealing preference leakage, this work could help build more trustworthy and ethically grounded Al systems. The relatedness between data generators and evaluators can systematically bias evalu- ations, potentially compromising the fairesss and reliability of the automatic evaluation paradigm. These biased evaluations may indirectly affect downstream tasks such as Al alignment and decision- making systems, leading unintended ethical risks. To mitigate preference leakage, hope that
25

---

Published as a conference paper at ICLR 2026
Annotater Tool
wem pouhh mug las the daficl modeirg be reveectio bolkee letee dhe bund fos ly de lacu The fonk by teap cemoume to Reveote Fanccie Pumpar
codireen ca enian Light uut shiing Reiike lnn focor Pecdle licuut mg deiiig gienet colende octieve
ppourc at staq da coranatt.
Fenusets
kom be snmatts acatisu now cakioal pee ceport todeo aution
more demerr from akk chf The call en prooee revil ne Pawkint atorr provides comee
inteenee
revll iepriebase
inferntte Roctle heard
herter proceitatio iertude
stangg Labeer Jusuve papae by high gliig uy siuse
reusant acterve gilican
Figure 7: The annotation tool we develop for annotation efficiency.
researchers will propose more reliable evaluation methods, diversifyy training data sources, and develop contamination-resittee benchmarks in the future.
26


Reviews:
[{"review_id": "SeHBNLxNMB", "reviewer": "Reviewer_F5m9", "summary": "This paper introduces and defines \"preference leakage\", a novel and subtle contamination problem in the LLM-as-a-judge paradigm. Preference leakage occurs when the data-generating LLM ($M_G$) and the evaluating LLM ($M_J$) are \"related\", for example, being the same model, having an inheritance relationship (e.g., fine-tuning), or belonging to the same model family (e.g., GPT-3.5 and GPT-4).", "questions": "1. Could you apply your most promising mitigation method, Contextual Calibration, to your main experimental setup from Section 4? Specifically, how much does this method reduce the $PLS$ score (e.g., in the \"Mistral-7B, GPT-4o & Gemini-1.5\" case from Table 1, which was 28.7%)? Showing a direct reduction in your primary metric would make the contribution feel much more complete.\n\n2. You cleverly show that while LLM judges cannot recognize their students, a fine-tuned BERT classifier can. This is a great finding. Have you considered using this classifier as a detection or mitigation tool itself? For example, could the classifier's confidence score for \"relatedness\" be used as a penalty term in the evaluation, or to flag high-risk comparisons for human review?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The primary strength is the paper's originality. The formalization of \"preference leakage\" as a distinct contamination vector, separate from both traditional data leakage and simple egocentric bias, is a novel and important conceptual contribution. Also, the empirical work is high-quality. The authors conduct a thorough and \"full-factorial\" investigation across multiple generators, students, and benchmarks . The further analyses (Section 5) are comprehensive and anticipate many of the reader's questions, such as the impact of data mixing ratios , relatedness types , and learning methods (SFT vs. DPO vs. ICL) .", "weaknesses": "1. The paper's main weakness is the preliminary and somewhat disconnected nature of the mitigation analysis (Section 5.7). While the paper excels at diagnosing the problem, the treatment section feels like an add-on. The mitigation experiments use a different setup (new datasets like PPE/MTBench, new \"Error Bias\" metric) than the main experiments (Arena-Hard/AlpacaEval 42, $PLS$ metric). This makes it hard to connect the findings. For example, how does the best mitigation (Contextual Calibration) affect the $PLS$ score on the main paper's task? This is not answered.\n\n2. The paper explicitly defers a full solution: \"We leave the exploration of detection, mitigation and calibration methods of preference leakage for future works\". While this is fair, it does make the paper feel somewhat incomplete. The community is now aware of a major problem but is not given a validated, well-integrated solution within this same work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "X0HhMTtaJk", "reviewer": "Reviewer_nucm", "summary": "The paper focuses on \"preference leakage,\" a contamination issue in the LLM-as-a-Judge paradigm. It occurs when related data-generator LLMs (M_G) and evaluator LLMs (M_J) (same model, inheritance, same family) lead M_J to favor student models (M_S) trained on M_G’s synthetic data, inflating scores.\nUsing 3 generation/evaluation LLMs (GPT-4o, Gemini-1.5, LLaMA-3.3), 2 student models (Mistral-7B, Qwen-2.5-14B), and 2 benchmarks (Arena-Hard, AlpacaEval 2.0), the authors propose Preference Leakage Score (PLS) to quantify bias. Experiments confirm widespread leakage,worse in smaller M_S, subjective tasks, and SFT. They also analyze factors like data mixing and test mitigation.", "questions": "Regarding quantitative indicators：\n1. The formula defaults to a 50% fair win-rate for evaluators between two student models without leakage. However, inherent quality differences (architecture, training data) may cause deviations (a superior model naturally has a 60% win-rate). Lack of baseline calibration (e.g., human evaluations as the true benchmark) leads to misjudgment, attributing inherent quality gaps to preference leakage.\n2. The formula assumes unrelated evaluators (J_unrelated) are unbiased. In reality, all LLMs may have common biases (length bias, style preference) that skew judgments. Failure to exclude these biases undermines the accuracy of the \"normal preference\" baseline in PLS calculation.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The paper fills a gap in prior research on LLM-as-a-Judge biases (e.g., egocentric bias) by focusing on the subtle, synthetic data-mediated bias between related models, rather than simpler self-favoritism.\n\n* The paper maintains good clarity in structure and expression. It logically organizes content from problem definition to experimental design, results, and mechanism analysis.", "weaknesses": "* Mechanistic Analysis Insufficiency: While the paper attributes leakage to \"spurious features (style/format)\" inherited by student models, it does not empirically validate these features.  It could use feature attribution methods (e.g., SHAP, LIME) to identify which specific stylistic/formatting elements (e.g., sentence structure, terminology) drive M_J’s bias, or conduct ablation studies (e.g., paraphrasing synthetic data to remove style) to test if leakage diminishes.\n\n* Real-World Impact Evidence Weakness: The claim that preference leakage impacts leaderboards relies on a single case study (AlpacaEval 2.0) with limited model coverage (only Vicuna/GPT-4-related models). It lacks: 1) analysis of other major leaderboards (Chatbot Arena, MT-BENCH); 2) longitudinal data (e.g., whether model rankings shift when using unrelated M_J); 3) human validation (e.g., comparing biased LLM rankings to human rankings for leaked vs. non-leaked model pairs).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dLDgsybjDL", "reviewer": "Reviewer_HBz8", "summary": "The paper investigates preference leakage, a newly defined contamination phenomenon in LLM-as-a-Judge frameworks, where bias arises when data-generation and evaluation models are related (same model, inheritance, or same model family). Through large-scale experiments across GPT-4o, Gemini, and LLaMA-3 series, the authors show that related generator–judge pairs inflate evaluation scores, compromising benchmark reliability. They design a Preference Leakage Score (PLS) and analyze the issue under multiple conditions (data mixing, model size, learning method, real-world leaderboards, etc.), concluding that preference leakage is widespread and subtle compared to data leakage or egocentric bias.", "questions": "1 In Section 3.3, how do the authors define “same model family” when architectures diverge but pre-training corpora overlap (e.g., Mistral vs Gemma)?\n\n2 Can the authors provide significance testing or confidence intervals for the Preference Leakage Score (PLS) results in Table 1 and Figure 2?\n\n3 In the learning method analysis (Section 5.3), how do the authors distinguish the leakage reduction effects of DPO from general overfitting control—could this be further verified through ablation?\n\n4 What specific linguistic or structural features appear to be “embedded” in student models through synthetic data (Section 5.5)? Are they stylistic, lexical, or formatting-based?\n\n5 Could contextual calibration (Section 5.7) be combined with adversarial or paraphrased prompting to further mitigate preference leakage?\n\n6 Given that preference leakage may relate to model “affective preferences,” have the authors considered testing the phenomenon on affective or sentiment-analysis benchmarks?\n\n7 To what extent might cross-family transfer (e.g., DeepSeek ↔ Grok ↔ Gemini) reveal broader generalization or hidden correlations beyond single-family leakage?", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1 Novel and important concept: The notion of preference leakage extends beyond existing bias categories and raises an underexplored but crucial reliability issue in LLM evaluation.\n\n2 Systematic empirical validation: Comprehensive experiments using diverse generator-judge relations, datasets (Arena-Hard, AlpacaEval 2.0), and mitigation trials.\n\n3 Strong methodological framework: Clear definitions of “relatedness” and formalization of leakage conditions make this paper theoretically grounded.\n\n4 Practical relevance: Direct implications for the credibility of public leaderboards and synthetic-data pipelines. It will inspire industry focus on this area of preference leakage. \n\n5: The Preference Leakage Score (PLS)  is innovative and worth to further explore.", "weaknesses": "1 The discussion of features embedded in student models (Sec 5.5) is promising but under-analyzed; deeper probing or visualization (e.g., stylistic feature attribution) would enrich understanding.\n\n2 Some family coverage is narrow—adding DeepSeek and Grok series could strengthen generality and industry relevance.\n\n3 If “preference” is positioned as a form of affective bias, connections to affective-analysis or sentiment-evaluation benchmarks would contextualize it better.\n\n4 While mitigation methods (Sec 5.7) are valuable, quantitative comparisons among them are brief; more ablation or human-alignment results would improve credibility.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8KxlMvQFeq", "reviewer": "Reviewer_fJqb", "summary": "The paper proposes preference leakage, a contamination problem in LLM-as-a-judge. The paper identifies three common types of relatedness between data generator LLM and the judge LLM, and confirms that preference leakage causes LLM judge to prefer related student models via comprehensive experiments. In addition, LLM judges are not good at identifying responses from student models, indicating that preference leakage is hard to detect.", "questions": "Is there any effective method to detect preference leakage, especially the inheritance relationship? For instance, would it help to measure the correlation coefficient of the last hidden state between judge and student on the same prompt?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper proposes a novel concept of contamination in the domain of LLM-as-a-judge, and identifies the preference bias caused by such contamination. The paper conducts sufficient experiments to support its claim, raising new questions to LLM-as-a-judge research community on how to prevent or identify preference leakage.", "weaknesses": "The paper does not study in detail how to detect preference leakage, or how to mitigate preference leakage. Although they provide preliminary analyses in Section 5.7, the methods explored are general and not designed specifically for preference leakage mitigation. Considering that preference leakage is a novel concept, these weaknesses are acceptable.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)