TrustJudge: Inconsistencies of LLM-as-a-Judge and How to Alleviate Them
Authors: Yidong Wang
Yunze Song
Tingyuan Zhu
Xuanwang Zhang
Zhuohao Yu
Hao Chen
Chiyu Song
Qiufeng Wang
Zhen Wu
Xinyu Dai
Yue Zhang
Cunxiang Wang
Wei Ye
Shikun Zhang
Venue: ICLR 2026 Poster (2026)
Keywords: LLM-as-a-Judge
LLM Evaluation
Large Language Models
Primary area: foundation or frontier models, including LLMs

Abstract: The adoption of Large Language Models (LLMs) as automated evaluators (LLM-as-a-judge) has revealed critical inconsistencies in current evaluation frameworks. We identify two fundamental types of inconsistencies: (1) \textit{Score-Comparison Inconsistency}, where lower-rated responses outperform higher-scored ones in pairwise comparisons, and (2) \textit{Pairwise Transitivity Inconsistency}, manifested through circular preference chains ($A\!>\!B\!>\!C\!>\!A$) and equivalence contradictions ($A\!=\!B\!=\!C\!\neq\!A$). We argue that these issues come from information loss in discrete rating systems and ambiguous tie judgments during pairwise evaluation. We propose \textbf{TrustJudge}, a probabilistic framework that addresses these limitations through two key innovations: 1) \textit{distribution-sensitive scoring} that computes continuous expectations from discrete rating probabilities, preserving information entropy for more precise scoring, and 2) \textit{likelihood-aware aggregation} that resolves transitivity violations using bidirectional preference probabilities or perplexity. We also formalize the theoretical limitations of current LLM-as-a-judge frameworks and demonstrate how TrustJudge’s components overcome them. When evaluated with Llama-3.1-70B-Instruct as judge using our dataset, TrustJudge reduces Score-Comparison inconsistency by 8.43\% (from 23.32\% to 14.89\%) and Pairwise Transitivity inconsistency by 10.82\% (from 15.22\% to 4.40\%), while maintaining higher evaluation accuracy. Our work provides the first systematic analysis of evaluation framework inconsistencies in LLM-as-a-judge paradigms, offering both theoretical insights and practical solutions for reliable automated assessment. The framework demonstrates consistent improvements across various model architectures and scales, enabling more trustworthy LLM evaluation without requiring additional training or human annotations.

Published as a conference paper at ICLR 2026
TRUSTJUDGE: INCONSISTENCIES OF LLM-AS-A- JUDGE AND HOW TO ALLEVIATE THEM
Yidong Wang
Yunze Song Tingyuan Zhu
Xuanwang Zhang
Zhuohao Yu
Hao Chen
Chiyu Song
Qiufeng Wang"
Zhen Wu
Xinyu Dai
Yue Zhang
Cunxiang Wang Wei Ye Shikun Zhang
Peking University
Nattoonal University of Singapore
IIntittte of Science Tokyo "Westlake University
"Nanjing University
*Carnegie Mellon University
"Southeast University
Tsinghua University
ABSTRACT
The adoption of Large Language Models (LLMs) as automated evaluators (LLM- as-a-judge) has revealed critical inconsistencies in current evaluation frameworks. We identify two fundamental types of inconsistencies: (1) Comparionn Inconsistency, where lower-rated responses outperform higher-scored ones in pair- wise comparisons, and (2) Pairwise Transitivity Inconsistency, manifested through circular preference chains (A > B > C > A) and equivalence contradictions from information loss in discrete rating systems and ambiguous tie judgments during pairwise evaluation. We propose TrustJudge, a probabilistic framework that addresses these limita- tions through two key innovations: 1) distribution-sen sensitive scoring that computes continuous expectations from discrete rating probabilities, preserving information entropy for more precise scoring, and 2) likelihood -awere aggregation that resolves transitivity violations using bidirectional preference probabilities or perplexity. We also formalize the theoretical limitations of current LLM--ss---jude frame- works and demonstrate how TrustJudge's components overcome them. When evaluated with Llama-3.1-700 Instructt judge using our dataset, TrustJudge reduces Score-Compane Pairwise Transitivity inconsistency by 10.82% (from 15.22% to 4.40%). while maintaining higher evaluation accuracy. Our work provides the first systematic analysis of evaluation framework inconsistencies in LLM-as-a-judde paradigms, offering both theoretical insights and practical solutions for reliable automated assessment. The framework demonstrates consistent improvements across various model architectures and scales, enabling more trustworthy LLM evaluation without requiring additional training or human annotations. The codes can be found https: // gi thub. com http:/rrustJuggee/// www.si/eeeerutuuuuee..e.
(A B C * A). We argue that these issues
inconsistency by 8.43% (from 23.32% to 14.89%) and
INTRODUCTION
The widespread adoption of LLM-as-a-judge approaches has offered a scalable and effective al- ternative to costly human assessments Chang al. (2024): Fu et al. (2023); Lin & Chen (022)); Sottana et al. (2023); ; Huang et al. (2024); Koutcheme et al. (2024): Song et al. (2024); Zhu et al. (2023). Beyond evaluation, this paradigm also actively contributes to model improvement, enabling self-refinement through iterative feedback Yuan et af.; Wu et al. (2024); Wang et al. (2025c) and frame- ); Chen et al. (2024a); [i et al.
collaborative progress via mutual assessment Wang al. Li et al. These LL-
works typically implement two evaluation protocols Li et al. 2023);
(2025); Chen et al. (2024b); Tan al. (2024); Thakur et al. (2024); (Szymanski al. (2025 Raju et al. (2024)m: single-score assessment, where a judge LLM (cither general-purpose or specifically iunet for evaluation) assigns integer numerical ratings model outputs Zheng et al. (2023);
Wang et al. (2024), and pairwise comparison, where the judge evaluates competing responses in
"Equal contribution.
1 "oorespondence: wangcunxiiang303ggm com, wye@pku.eeeeee edu, en, zhangsk@pku. edu. en.

---

Published as a conference paper at ICLR 2026
direct comparison (with the order of responses swapped in two separate evaluations to eliminate position bias) to produce preference judgments Dubois et al. (2024); Wang et al. Li et al. (2024).
However, our rescarch identifies two critical inconsistencies in these LLM-ae---judee evaluation frameworks for LLMs: (1) Coop-Commanison Inconsistency between single-score and pairwise comparison assessment, where LLMs with lower absolute scores may outperform higher-scored counterparts in pairwise comparisons (Rz > Ry despite score( < score( (core(R,))) ) and (2) Pairwise Transitivity Inconsistency in pairwise comparison evaluation, where judgments show non-transitive cycles (R2 > Ra > R₂ > ) and equivalence contradictions (R,, = Ry = R& o Rz), violating rational preference principles. While prior work addresses pairwise inconsistencies through complex mathematical modeling, Xu et al. Zhang et al.) such continual training risks compromising model generalizability Luo et al. (2022) Lin et al. (2024) without resolving score-comparison conflicts. To the best of our knowledge, this is the first work to systematically expose both inconsistencies as foundational weaknesses LLMM-ss-a au-aueee frameworks and provide a unified resolution.
S-scale
10-scale with softmax
Inequality Transkivily Inconsislency
5-scale with softmax
100-scale
Circular Transitivity Inconsistency
10-scale
100-scale
1.57
1.61
1.61
1.63
1.67
Average Eatrapy
84.8%
1.08
Llama-3, 1..--B--Insteen
1.0
0.73
0.66
10.1%
0.57
0.49
0.5
89.9%
1lama3.1170B--estruuct
Llama-3. L 1770ttttrrett
Figure 1: Left: Average entropy of Llama-3 Grattafiori et al. (2024) models' single-score outputs over six rounds on 1,200 instructions, by scoring strategy. Right: Breakdown of circular- vs. inequality- transitivity errors in pairwise-comparison tests.
To address these inconsistencies, introduce TrustJudge, a probabilistic evaluation framework that preserves judgment entropy while resolving both (a) score-comparison conflicts via distribution- sensitive scoring and (b) transitivity violations through likelihood-aware aggregation. As shown in Figure T we argue that the score-comparison inconsistency primarily stems from information loss in the integer scoring system the coarse-gras grarsee-rained 5-point scale compresses nuanced quality differences into identical scores (e.g., different quality responses both receiving 4 points), resulting in low entropy judgments that fail to discriminate actual performance gaps. We propose two effective solutions: (1) increasing scoring granularity (5~110-- 110>0000 100 points) to preserve more discriminative information, and (2) probabilistic scoring that maintains the full entropy of model judgments. For pairwise transitivity inconsistency,
find most of inconsistencies originate from tie judgments (equivalence contradictions). We propose breaking ambiguous ties by either (1) preferring responses whose entire sentence exhibits lower perplexity, (2) deciding preference based on the judge's 's token- level confidence for win, tie and lose. These approaches significantly reduce inconsistency rates while maintaining the scalability and effectiveness of jude-e frameworks. While recent works like Liu et al. (2023); Wang et al. (2025b) adopt probabilistic scoring to enhance human alignment, our motivation differs fundamentally addressing fundamental inconsistencies of evaluation frameworks rather than improving human-jude agreement. Our probabilistic scoring serves as an entropy- preserving mechanism with granularity enhancement to resolve score-comparison conflicts. Our theoretical analysis shows that discrete scoring systems suffer from information loss by showing that distinct response distributions can yield identical scores despite different entropies, whereas TrustJudge's 's cestribution-sensitii scoring preserves and distinguishes these differences, and further demonstrates that its PPL-based confidence distribution reduces uncertainty ambiguous cases by leveraging perplexity to create lower-entropy signal for decision-making.
Extensive experimental results across multiple model families (Llama-3, GPT, Qwen, Gemma) and scales (3B to 70B parameters) demonstrate TrustJudge's effectiveness. Our framework reduces
We Rz, Ry, R₂ represent distinct L.M responses, strictly preferred; <: strictly worse; z: preferred or equivalent; <: worse equivalent; equivalent; f not equivalent
2

---

Published as a conference paper at ICLR 2026
Score-Coue Compprromnnisnn inconsistency by 8.43% (from 23.32% 14.89%) and Pairwise Transitivity inconsistency by 10.82% (from 15.22% to 4.40%) when using
1aae--..--0-----s- as judge. These improvements achieved while maintaining improving evaluation accuracy, with exact match rates increasing by 1.19-6.85% across different model sizes. Our ablation studies confirm that both distribution-seeniiivve scoring and likelihood-aware aggregation contribute significantly these improvements, with the full framework achieving the best performance across all tested scenarios.
In conclusion, we present the first systematic analysis fundamental inconsistencies in LLM-as- a-judge evaluation frameworks, identifying two critical limitations: (1) information loss in discrete scoring systems causing Score-Couee Compaaiomnn Inconsistency, and (2) ambiguous judgments leading to Pairwise Transitivity Inconsistency. TrustJudge addresses these through distribution instribution-sensiivee sensitive scoring that preserves judgment entropy and likelihood-aware aggregation break ambiguous ties. Our experiments demonstrate significant inconsistency reductions while maintaining evaluation accuracy across diverse models and tasks. This work provides both insights into LLM evaluation limitations and practical solutions for more reliable automated assessment, enabling more trustworthy deployment of LLM--a--ajudde paradigms research and applications. In addition, we provide more detailed review of related work in Appendix A
2
METHODOLOGGY
Our framework addresses two fundamental inconsistencies in LLM-as-a-judge systems through formal definitions and quantitative metrics. We first establish mathematical characterizations of these inconsistencies, then present our TrustJudge algorithm.
2.1 INCONSISTENCY DEFINITIONS AND METRICS
Definition 2.1 (Score- (Score-Compall Comparison Inconsistency). For responses Rz, Rg with scores Sz, S, E Z (e.g., 1-5 scale) and pairwise comparison C' C(R,, Ry) (-1,0, 1} (1: R₂ preferred, -1: R, preferred, O: tie), inconsistency occurs when:
(S₂ > ACCCSOM (S, < Sy AC 2 v (S. = A,
The Conflict Ratio CR = 1 Σ -i Lfinconsistent pair i] measures prevalence, where n is total pair numbers the test and i[[: the indicator function.
Definition 2.2 (Pairwise Transitivity Inconsistency). For a set of responses R. { (RI.... R,)
and its subsets R, of size 2 3, three responses Rg, Ry. R2 E R2 satisfy one of the following transitivity violations:
Circular inconsistency: C(Ra, Ry)= = A C(RY, R.) = C(R., R.)) + -1.
(2)
(forming a preference cycle R2 Ry > R2 A R.)
Inequality inconsistency: C(Rz, Ryy) = 0 ^ C(Ry R2) = A C(R,, R2) #
(3)
(violating transitivity of indifference)
The Non-Transite Non-Transitivity Ratio is defined as NTR, = Ve (2)
where V₂ denotes the number of k-size
subsets exhibiting either inconsistency type and (%) represents the binomial coefficient counting all possible k-size subsets from elements.
2.2 TRUSTJUDGE
As shown in Algorithm [ the TrustJudge framework is probabilistic evaluation approach that preserves judgment entropy while resolving score-comparison conflicts and transitivity violations. The framework operates differently for single-score evaluation and pairwise comparison tasks, maintaining consistency between these two evaluation protocols.
For single-score evaluation, TrustJudge employs Eistributionn-ensiitee scoring mechanism. Given response to be assessed, the framework first prompts the LLM to score on more fine- tie-ggrinedd scale than original (e.g., 100-point scale when the original scale 5-point). It then transforms the resulting
3

---

Published as a conference paper at ICLR 2026
Algorithm TrustJudge Evaluation Framework
Require: Responses Rg, Ry (pairwise) R (single), expanded scores O' (range [ max)
original range [8min, Smax]
Ensure: Score S comparison C(Rz; Ryy
1: if' Single-Score Evaluation then
2:
P()) & Softmax (Po (s}(R)) (Normalize probabilities}
3: S' -ija of s)P(s))18) (Expected expanded score)
4: 5 3/>
Smna Smin
(Scale to original range)
mmax min
5: return S
6: else (Pairwise Comparison)
7: Option A: PPL-Based
8: PPL1 -- PPL(M, Ra,, Ry) (Perplexity of R₂ followed by Ry)
9: PPL2 + PPL(M, Ry, Rz) (Perplexity of reverse ordering)
Corders
PfL PPL2 (Determine comparing PPL.)
10:
C(Ra, Ry)
Condery otherwise
Option B: Likelihood-aware Aggregation
12:
Pi + Prob(M, Ra, Ryy) |Probabilities for Ra vs Ry
P2 Prob(M,, Ryy R2y) (Probabilities for reverse comparison)
14:
m[k] < Pı [k] Pa [k] for k E {1, -1. [Aggregate both directions]
return arg max; m[k] (Select most probable outcome}
16:
end if
discrete probability distribution Po(s) R) over the expanded score set O' = m maxx } into logits E, for each candidate score s/. These logits are then processed by a softmax function which normalize the logits into a valid probability distribution (R) Unlike other approaches such as G-Eval Liu et al. (023), whose generated probabilities can violate Σ, P(s)) R) because non-score tokens also influence the output, our method ensures a properly normalized distribution. The final score S is computed as the expected value, scaled back to the original range [smin 8max]:
max
exp(Po(s(1R)) Eexp(Po(s((RR)
Smax 8min
S
Smax &
where P(s)] Rt) represents the original probability for score s/y. This approach preserves the full entropy of the judge's 's assessment while producing continuous scores that maintain fine-grained distinctions between response qualities.
For pairwise comparison tasks, TrustJudge offers likelihood-aware aggregation methods to resolve transitivity inconsistencies. The first approach (Option A) leverages perplexity-based (PPL-based) method to break ties when the judge exhibits ambiguity. Given two responses R3 and Ry, the framework computes the perplexity of both possible orderings (R,, followed by R, and vice versa) under the judge model M. The comparison result C (R,, R,y) is determined by selecting the ordering with lower perplexity:
Conter Cardera:
PPL Rg, R.y) < PPL(M, Ra, Ra), otherwise.
C(RZ, Rgy)
(5)
The second approach (Option B) employs a bidirectional probability based method that combines preference probabilities from both orderings reduce position bias. For each possible outcome k E {1. -1, 0} (representing R2 preferred, Ry preferred, or tic), the framework aggregates the probabilities from both orderings:
ml [k: Porder [k]] + Pondery [-k
(6)
4

---

Published as a conference paper at ICLR 2026
where Poeder; and Pordery are the probability vectors for the two orderings. The final comparison result determined by selecting the outcome with maximum aggregated probability k" = arg max; m [k]. This probabilistic approach significantly reduces circular and inequality transitivity violations while maintaining the scalability of pairwise comparisons.
By producing nearly continuous score distributions, probabilistic judge makes exact equality between two responses much less likely than traditional discrete grading. To relax the tie criterion, we can introduce tolerance hyper-parameter 6 responses-abbolute score difference, PPL gap. or probability margin-dooss not exceed 6, the pair declared tie, allowing users tune the gramularity of the final ranking without retraining the model. Although 5 is set to 0 by default, we have conducted a thorough hyper- parameter sweep that confirms TrustJudge's reliability across range of 5 values; the results recommend small positive because-ever even with 8 = 0 the framework already produces a noticeable number of ties.
Whenever the discrepancy between two
3 THEORETICAL ANALYSIS
In this section, we formalize the theoretical weaknesses of current LLM-as-a-judge frameworks and prove how TrustJudge's components address them. The detailed analysis and derivation can be found at Appendix H We start by proving that discrete scoring systems suffer from information loss.
Theorem 3.1 (Information Loss of Discrete Scoring and Preservation in Expectation). Let pR, and PR, be two distinct probability distributions over the score set θ representing the judge model's assessment of two different responses, Rí and Ra (i.e., PR, # pR2). The discrete scoring function Discreet can fail to distinguish between these two assessments, whereas the distribution-sensitvve scoring function fas provides a mechanism for their discrimination. Specifically:
(Information Loss):
There exist PR, + PR, with different conditional entropies, (S))R:) + (S) R2), such that their discrete scores are identical: D (p D (p
2. (Information Preservation): For the same distributions PR, and pit, constructed in (1), their distribution sensitive scores are distinct: fus(PR,) 7 fDs(PR.))
For pairwise comparisons we formalize how the PPL-based method reduces the uncertainty caused by ambiguity, the proof of which is deferred to the Appendix H
Proposition 3.2 (Uncertainty Reduction via PPL-based Method), Let (C([z) be the Shannon entropy the judge model's 's outcome distribution. In an ambiguous regime, this entropy is maximized, (C([x) ze log [C). We define a confidence distribution Peoof based on the perplexity of the rationale J& for each outcome k:
Peon((k) x exp(-> (PL(J))), for scaling constant >> O.
(7)
the rationale perplexities are not all equal, then Poonf is non-uniform. By the properties of Shannon entropy, this implies its entropy is strictly less than the maximum:
HP log (C).
(8)
Thus, the PPL-based method makes its decision using a more certain (lower-entropy) signal.
4 EXPERIMENTS
Setup Our dataset combines both the 80 questions from MT-Bench Zheng et al. (2023) and the 500 challenges from ArenaHard Li et al. (2024). MT-Bench provides broad coverage of diverse instructions across eight categories including writing, roleplay, and reasoning, while ArenaHard offers particularly challenging queries drawn from real-world user interactions. For each question, we sample candidate responses from diverse LLMs. Under the single-comparison inconsistency protocol, we construct a dataset of 10.8k instances, where each instance corresponds to a pair of responses annotated with their single scores and the induced pairwise preference. Under the pairwise transitivity inconsistency protocol, we collect 43.2k pairwise relations for & = 4 and 50.4k for k = 5, each derived from the corresponding k-response sets evaluate transitivity. For each question,
Please more analysis of the bidirectional probability based method in Appendix H.2
5

---

Published as a conference paper at ICLR 2026
collected responses from diverse set of large language models with varying capabilities. All gold-standard scores and pairwise comparison results for these responses are verified through human review. The final dataset is carefully balanced, ensuring uniform score distributions across every rating level for both single-score and pairvise-compariso) scenarios. The detailed model information and inference hyperparameters used in our test data creation are listed in Appendix [D] and the detailed category distribution across our datasets is provided in Appendix [ Beyond the core framework, we also extend our approach to multi-dimensional evaluation, as detailed in Appendix F
We evaluate both the inconsistencies and accuracies. For inconsistencies, we use: (1) the Conflict Ratio (CR, Definition 2.1 and (2) the Non- -on-Iransitivity Ratio (NTR, Definition 2.2]. For accuracies, we employ (1) Win Rate for single-score evaluation: the fraction of instances whose score sides with the reference, which highlights protocol differences more sharply than MSE or MAE. (2) Exact Match for pairwise comparison: given the ground-truth order A > B, only verbatim output of > counts; any reversal or tie wrong- an all-or-nothing metric. Parameter h (Def. governs the subset size for pairwise transitivity checks. Larger & captures higher-order cycles at cost (k); = 3 yields too few triples to discriminate models, so we report k k= 5.
We establish two fundamental baseline approaches for comparison with TrustJudge. For single-score evaluation, we implement: (1) the standard raw scoring method that directly outputs absolute scores (I-5 scale), as used in MT-Bench; and (2) G-Eval-style probability summation that calculates the total probability mass across possible scores without applying softmax normalization. Following previous work Wang et al.) pairwise baseline mitigates position bias by evaluating each response pair twice (reversed order) and record differing outcomes as ties. All baselines use the identical judge model and the same detailed prompt (see Appendix [C as TrustJudge, enabling direct comparison of inconsistency metrics (CR and NTR) and accuracy metrics (Win Rate and Exact Match) under identical conditions.
Baselines
Our experiments comprehensively evaluate TrustJudge across a diverse set of popular LLMs, covering both open-source and proprietary families with varying parameter sizes. Specifically, we include: Llama-3.2-3B, Llama-3.1-8B, and Llama-3.1-708 Grattafiori al.] (2024), three instruction-tuned models from the Llama-3 series, representing small, medium. and large-scale open-source LLMs, respectively. We also evaluate GPT-3.5- Turbo OpenAl| (2023) (2023) and GPT--40 OpenAT 2024), two widely-used proprietary models from OpenAI, included for their strong performance in both general and evaluation-speciff benchmarks. Additionally, we assess the Qwen2.5-7B, Qwen2.5-14B, Qwen2.5-32B Yang al. (2024), Gemma-2-2b, Gemma-2-9R, and Gemma-2-27B Riviere et al. (2024) to demonstrate TrustJudge's generalization across model types and sizes. For all evaluations, we use the instruction-tuned or SFT variants of each model, consistent with their intended use judge LLMs. All judge models are applied with identical evaluation prompts and configurations to ensure fair comparison.
Evaluated LLMS
The experimental results comparing TrustJudge with baseline approaches across multiple model familics and sizes are summarized in Table I and Figure 2 The key findings are:
Main Results
Table 1: Results for two experiments: (1) Score-Comparic Inconsistency (CR) comparing raw-score baseline, G-Eval probability-summaal summation, and TrustJudge's distribution scoring; (2) Pairwise Transitivity Inconsistency (N,TR==4,5) comparing two-pass swap-order baseline versus TrustJudge's likelihood-aware aggregation. Win rate quantifies scoring precision by measuring the proportion of test instances where a method's score is nearest the ground truth with results presented on both 5- 5-point and 100-point scales. Exact match quantifies comparison consistency by measuring the proportion of pairwise method outcomes that perfectly align with dataset annotations.
Model
CR (%)
NTR. (%)
NTR... (%)
Ours vs Baseline
Ours vs G-Eval
Pairwise Exact Match
Baseline
G-Eval
Bascline
Ours
Baseline
Ours
S-scale
100-scale
5-scale
100-scale
Baseling
TrusUudge
Llama-3.2-3B 3.20-l..-tBl
36.65
29.50
29.15
32.42
8.07
$4.69
17.76
45.41
54.66
62.21
51,03
72.06
78.91
Jama-33 Inst-rrtt
29.73
25.31
23.75
20.26
3.79
37.03
8.46
$6.84
$1.88
59.61
51.24
75.67
81.68
1lama-3.1 1.w.mm...-0---mmaaee Insttrutt
23.32
15:77
14.89
7.23
1.94
15:22
4.40
51.77
54.53
64.22
55.27
80.42
81.61
GPT.4o
27.95
11.70
6.01
50.31
65.11
78.67
81.51
Trust.Judge significantly reduces evaluation inconsistencies across all model sizes. Our experi- ments demonstrate that TrustJudge achieves superior consistency compared to both direct scoring

---

Published as a conference paper at ICLR 2026
Single-Comparison Inconsistency (CR%)
Pairwise Non-Transitivity (NTR%) (k=5)
60
30
CR
NTR
20
20
15
0.01
0.02
0.03
0.04
0.05
0.06
0.01
0.02
0.03
0.04
0.05
0.06
Tolerance 5
Tolerance
(a)
G-Eval, + Ours (distribution-senn) sensitive
(b)
Ours (PPL-based method),
Ours
scoring).
Baseline.
(likelihood aware aggregation), --- Baseline.
Figure 2: Results for single-compariss inconsistency (left) and pairwise transitivity inconsistency (right) across tolerance 8: Llama-3.1-70BB (blue), Llama-3.1-88 (red), Llama-3.22 (pink), and GPT-4o (green). Colors correspond to different judge models, while markers distinguish evaluation methods as described in each subfigure. For single-score in the experiment on the left, 5 is tolerance proportion on the original rating scale; For pairwise (PPL-based) in the experiment on the right, is the threshold on the difference in perplexity between the two presentation orders; For pairwise (Likelihood-awan) aggregation) in the experiment on the right, 5 is the threshold on the confidence gap between the top aggregated outcomes.
baselines and G-Eval approaches. The proposed method achieves substantial reductions in Conflict Ratios, delivering absolute improvements of 4.78%-8.433. -..4% over the baseline approaches. Moreover, Trustjudge consistently surpasses G-eval by approximately 1-2% across every experimental setting. More importantly, TrustJudge dramatically lowers transitivity violations in pairwise comparisons, with NTR==5 violations reduced by 10.82%-36.99 93% absolute. For instance, Llama-3.2-3B shows the most substantial improvement, decreasing NTRk=5 from 54.69% to just 17.76% with TrustJudge. These consistency improvements are particularly notable because they are achieved without requiring additional training fine-tuning of the base models.
TuustJudge maintains and often improves evaluation accuracy while reducing inconsistencies. TrustJudge demonstrates that both consistency and accuracy can be achieved simultaneously. Trust- Judge improves exact match rates by 1.19%-6.85% across different model sizes compared to baseline approaches, with the most significant gains observed for smaller models (6.85% improvement for Llama-3.2-30 3.2-BB....... In pairwise evaluations, TrustJudge achieves win rates of 45. 41 %-65..1 1% against both baseline methods and G-Eval approaches. The method performs particularly well on fine-grand tneeggraiee 100-point scoring and maintains strong performance on 5-point scales. This accuracy preservation is crucial for practical applications where both reliable and precise evaluations are required.
TrustJudge exhibits robust tolerance-aware gains across judge families and evaluation protocols. A fine-tuned tolerance (6) often yields superior outcomes, as a smaller toler- ance reduces ambiguity. Conversely, larger tolerance introduces greater uncertainty. It's im- portant to note that even with tolerance of zero, ties can still occur. The TrustJudge scoring and aggregation method effectively mitigates in- consistencies. Notably, its benefits are evident across various tolerance settings, demonstrating its robustness and effectiveness.
Table 2: Ablation study where "L" refers to LLaMA and "G" to GPT. Single Score Compo- nents report CR and Pairwise Comparison Compo- nents report NTR
Components
L-3.1-8B
L-3.1-70B
G-3.5-Turbo
G-4o
Single Score Components
5-scale Baseline
29.73
23.32
24.35
27.95
Softmax
26.10
17.08
24.03
25.50
100-scale
24.54
17.94
22.10
24.01
Pairwise Comparison Components
Baseline
20.26
7.23
14.01
11.70
+ Likelihood
3.79
1.94
6.26
2,83
A Study To evaluate the contribution of different components TrustJudge, we con- duct an ablation study by systematically remov-
PPL-Basod
2.18
4.80
4.48
ing key elements: (1) the softmax normalization, (2) the 100-point granularity enhancement for single-score evaluation, and (3) the pairwise comparison strategies (likelihood-awar aggregation and PPL-based methods). We also examine performance variations across different judge LLMs to demonstrate TrustJudge's model-agnostic properties.
7

---

Published as a conference paper at ICLR 2026
Table reveals several key findings. For single score components, the 5-scale baseline shows the highest inconsistency rates across all models (39.73% for Llama-3.1-8B, 27.5% for GPT-4o), indicating the importance of TrustJudge's 's enhancements. Adding softmax normalization reduces inconsistency by 0.32%-6.22 6.2%% absolute across models, while incorporating 100-scale granularity yields improvements (up 5.19% reduction from 5-scale).
In pairwise comparison components, the base- line shows moderate performance (20.26% in- consistency for Llama-3. 1- The likelihood- aware aggregation strategy achieves the best re- sults overall, reducing inconsistency to as low as 1.94% for 1lama--3.1-70B and 2.83% for GPT-40. The PPL-based comparison shows sub- stantial gains over baseline (16.47% absolute improvement for Llama-3. while offering practical advantages in implementation, as it op- erates directly sequence probabilities without requiring explicit win/tie/lose position identifi- cation.
L-3.2-3B
0.40
L-3.2-38 T
L-3.1-88
0.35
L-3.1-8B T
L-3.1-70B B
0.30
L-3.1-70B T
G-3.5-turbo B
G-3.5-turbo
G-40 B
0.20
G-40 T
0.15
5
10
100
The consistent performance patterns across model architectures (from 8B to 70B parame- ters) demonstrate that TrustJudge's benefits are model-specific but derive from its method- ological innovations. Larger models gener- ally achieve better absolute performance, with Llama-3. 1-70B and GPT-4o showing particu- larly strong results when using TrustJudge.
Scale
Figure 3: Effect of scoring granularity Conflict Ratio across judge models. We measure the Con- flict Ratio (CR) under three scoring scales: S-point, 10-point, and 100-point. "L" refers to LLaMA models and "G" GPT models; "B" denotes base- line scoring, while "T" represents TrustJudge.
Increasing Score granularity reduces incon- sistency. As shown in Figure 3 increasing the scoring scale from 5 to 100 points consistently reduces the Conflict Ratios. Furthermore, Trustludge (T) achieves lower inconsistency than the baseline (B) under all granularities, demonstrating its effectiveness in preserving scoring fidelity. The benefit is especially pronounced for larger models such as Llama-3. 1-70B and GPT-4o.
Generalization Experiment To systematically validate TrustJudge's cross-architectural adaptabil- ity and practical value for alignment training, we evaluate the framework across 12 model variants spanning four major architectures (Qwen, Gemma, Llama, GPT) with various parameter sizes. The experiments cover both single-response scoring and pairwise comparison scenarios. Note that we set k = 4 for pairwise comparison.
0.4
Baseline
-Q-7B
TrustJudge
Non-Iransttivity Ratio
0.30
Q032B
Q-14B
G-2b
Confiict Ratio
0.3
0.25
G-9b
G-27b
0.20
0.2
0.15
0.1
0.10
0.05
0.0
Q-7B Q-14BQ-32B G-2b G-9b G-27b
Baseline
PPL-Based
Likelihood
Figure 4: Performance of TrustJuduge with LLMs of different Sizes and Structures. Note that Qwen-2.5 is denoted Q and Gemma-2 as G.
Figure demonstrates three key findings through comprehensive architectural comparisons: Architecture-agnostic consistency improvement. The distributionnssessitvve scoring achieves consistent
8

---

Published as a conference paper at ICLR 2026
reductions in single-img innglleinttanceeeee conflict ratios across all tested architectures. Moreover, inconsisteney varies markedly across architectures: Gemma consistently outperforms Qwen comparable size.
Transitivity violation reversal. The proposed likelihood-aware aggregation strategy substantially mitigates non-transitivity patterns across model variants. Remarkably, this approach enables mid- sized models to surpass the transitivity performance of significantly larger baseline models under controlled evaluation settings.
Size-performance decoupling. While model capacity naturally correlates with lower inconsistency rates, TrustJudge effectively narrows the performance disparity between small and large models. This capability highlights the framework's potential enhance the practical utility of resource-efficien models for alignment tasks. Notably, bigger is not always better: the 9B Gemma actually exhibits lower inconsistency than its 27B sibling.
Table 3: Performance of TrustJuduge for Llama-3.1 and DeepsSek-RI--isiill-LLe
CR (%)
NTR (8)
NTR (%)
Model
Baseline
G-Eval
Ours
Baseline
Likelihoood
PPL-Based
Baseline
Likelibood PPL- Based
LJama-3.1-8R
29.73
25.31
23.75
20.26
3.79
6.80
37.03
8.46
16.20
DeepSeck Cl-Distill- Liaill-Laama--- 8B
58.75
53.63
49.28
44.61
11.43
25.16
63.98
18.50
41.78
Reasoning model results As shown in Table 3 The reasoning model's significantly higher incon- sistency rates suggest a potential catastrophic torgetting of judge capabilities due to reinforcement training mathematical data Guo et al. (2025). This finding is noteworthy it highlights the challenges that arise when models are trained on specialized tasks, such as mathematical reasoning, which inadvertently lead the degradation of their performance in other critical areas like judging. Despite this, TrustJudge remains effective in improving judge performance, demonstrating its robustness and adaptability enhancing the model's capabilities across different domains.
Using Trustjudge for Rewarding Models Ta- ble 4 shows TrustJudge's 's effect when used as a reward signal for GRPO. We run GRPO on Owen2.5-7B-Inntruct using multi-task corpus with 8,600 training examples and 265 valida- tion examples covering summarization, math reasoning, instruction following, and related tasks Feng et al. (2025). For evaluation, three models are assessed by external LLM judge (GPT-4o-mini) under both the Bascline and TrustJudge scoring protocols. The model trained with the TrustJudge reward achieves the best performance under both evaluation protocols, obtaining average rewards 0.5706 under the Baseline protocol and 0.6347 under the TrustJudge protocol. In contrast, the model trained with the Baseline reward fails to improve over the original (0wen.55-7B-Instreee performance under both protocols. These results suggest that TrustJudge provides a more informative and robust reward signal, regardless of the downstream evaluation protocol. Additional training and evaluation details are provided in Appendix
Table 4: GRPO results on Qwen2.5-7B with differ- ent reward designs.
Avg Reward (Baseline)
Avg Rewurd (TrusJJugge)
Model
Qwen2.5-7BB Insstructt
0.5521
0.6114
Owen2255-B-Inntter costruct_baseling
0.5377
0.5980
0.5706
0.6347
and instead exhibits decline in
The results establish TrustJudge's s robust generalizability across: (1) different model families and scales, maintaining consistent inconsistency reduction regardless of architecture; and (2) diverse applications including direct evaluation and reward modeling for GRPO training. This versatility stems from TrustJudge's architecture aggnosicc probabilistic design and fine-gun ane-grained scoring approach.
CONCLUSION
We presented TrustJudge, a novel probabilistic evaluation framework designed to address fundamental inconsistencies in current LLMeas-a-judge paradigms. Through systematic analysis, identified two critical issues: Score- Coop--Compniion Inconsistency due to information loss in discrete scoring systems, and Pairwise Transitivity Inconsistency stemming from ambiguous tie judgments. TrustJudge

---

Published as a conference paper at ICLR 2026
introduces distribution sennsiiiveeeeeiiiieiveveee probabilisticc scoring, preserving judgment entropy, and likelihood- aware aggregation strategies to effectively mitigate these inconsistencies.
Empirical results demonstrate that TrustJudge significantly reduces Score-Comp Comparison inconsistency and Pairwise Transitivity inconsistency across various LLM architectures and scales. Crucially, these improvements do not compromise evaluation accuracy, achieving enhancements in exact match rates and win rates compared to established baselines. Our ablation and generalization studies confirm the robustness and model-agnostic applicability of TrustJudge. TrustJudge offers both theoretical insights and practical solutions for enhancing the reliability and credibility of automated LLM evaluations, contributing towards more trustworthy and consistent use of large language models in research and applications. We also discuss the limitations of our approach in Appendix B
REFERENCES
Anthropic. The claude model family: Opus, sonnet, haiku model card. hetps://wwi /WWW anthropic. com/ caapa//caaeeee23 --od-e--mmmaaaoooo 2024.
Yushi Bai, Jiahao Ying, Yixin Cao, Xin Lv, Yuze He, Xiaozhi Wang. Jifan Yu, Kaisheng Zeng, Yijia Xiao, Haozhe Lyu, et al. Benchmarking foundation models with language-modell -aa-----oo-------------iiiii---- 2023.
Advances in Neural Information Processing Systems,
Yupeng Chang, Xu Wang, Jindong Wang. Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al. A survey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology, 15(3):1-45, 2024.
Chavinlo. alpaca-130 1ab (huggingface model card).
alpaca- 13b 2023.
Dongping Chen, Ruoxi Chen, Shilin Zhang, Yaochen Wang, Yinuo Liu, Huichi Zhou, Qihui Zhang, Yao Wan, Pan Zhou, and Lichao Sun. MIlm-as- js-ae- Assessing multimodal Ilm-as-a-judee with lision-languag benchmark. In Forty-first International Conference on Machine Learning, 2024a.
Guiming Hardy Chen, Shunian Chen, Ziche Liu, Feng Jiang, and Benyou Wang. Humans or Ilms as the judge? a study on judgement biases. arXiv preprint arXiv:2402.. 10669, 2024b.
Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, and et al. Vicuna: An open-source chatbot impressing gpt-4 with 90% chatgpt quality. arXiv:2304.01166, 2023. URL httppss:/aaa.....o/////////////s
23344.01199
Yann Dubois, Balázs Galambosi, Percy Liang, and Tatsunori B Hashimoto. Length-comtrolled alpacaeval: A simple way to debias automatic evaluators. arXiv preprint arXiv:2404.04475, 2024.
Andrew Zhuoer Feng, Cunxiang Wang, Yidong Wang, Yilin Niu, Yu Luo, Hongning Wang, and Minlie Huang. Rlar: An agentic reward system for multi-task reinforcement learning on large language models. OpenReview preprint fJ6rVq/YVU, 2025. ICLR 2026 Conference withdrawn submission.
Patrick Fernandes, Daniel Deutsch, Mara Finkelstein, Parker Riley, André FT Martins, Graham Neubig, Ankush Garg, Jonathan H Clark, Markus Freitag, and Orhan Firat. The devil is in the errors: Leveraging large language models for fine-grained machine translation evaluation. arXiv preprint arXiv:2308.00 07286, 2023.
Evan Frick, Peter Jin, Tianle Li, Karthik Ganesan, Jian Zhang, Jiantao Jiao, and Banghua Zhu. Athene- 70b: Redefining the boundaries of post-training for open models, https:: /tttt///nxxuuuue nexus flow. a b blogs/ http//tteenes. 2024. See also HF model card: Nexusflow/Athene-7l00
Jinlan Fu, See-Kiong Ng, Zhengbao Jiang, and Pengfei Liu. Gptscore: Evaluate as you desire. arXiv preprint arXiv:2.00.044 041666 2023.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The la herd of models. arXiv preprint arXiv:2407.2 2024.
10

---

Published as a conference paper at ICLR 2026
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12940 2025.
Hui Huang, Yingqi Qu, Xingyuan Bu, Hongli Zhou, Jing Liu, Muyun Yang, Bing Xu, and Tiejun Zhao. An empirical study of llm-as-a-judge for Ilm evaluation: Fine-tuned judge model is not general substitute for gpt-4, arXiv preprint arXiv:2403.022 02.00. 2024.
Charles Koutcheme, Nicola Dainese, Sami Sarsa, Arto Hellas, Juho Leinonen, and Paul Denny. Open source language models can provide feedback: Evaluating llms' ability to help students using gpt-4-as-a-judge. In Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 7, pp. 52-58. 2024.
Dawei Li, Renliang Sun, Yue Huang, Ming Zhong, Bohan Jiang, Jiawei Han, Xiangliang Zhang, Wei Wang, and Huan Liu. Preference leakage: A contamination problem in llm-as-a-judge. arXiv preprint wwXi:22502.01534 2025.
Junlong Li, Shichao Sun, Weizhe Yuan, Run-Ze Fan, Pengfei Liu, et al. Generative judge for evaluating alignment. In The Twelfth International Conference on Learning Representations.
Junlong Li, Shichao Sun, Weizhe Yuan, Run-Ze Fan, Hai Zhao, and Pengfei Liu. Generative judge for evaluating alignment. arXiv preprint arXiv:2310.054700, 2023.
Tianle Li, Wei-Lin Chiang, Evan Frick, Lisa Dunlap, Tianhao Wu, Banghua Zhu, Joseph Gonzalez, and Ion Stoica. From crowdsourced data high-quality benchmarks: Arena-hame and benchbuilder pipeline. arXiv preprint 0rXiv:2406.11939 2024.
Yen-Ting Lin and Yun-Nung Chen. Lim-eval: Unified multi-dimensional automatic evaluation for open-domain conversations with large language models. arXiv preprint wrXiv:2305.13711 2023.
Yong Lin, Hangyu Lin, Wei Xiong, Shizhe Diao, Jianmeng Liu, Jipeng Zhang, Rui Pan, Haoxiang Wang, Wenbin Hu, Hanning Zhang, et al. Mitigating the alignment tax of rlhf. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 380-606, 2024.
Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. G-eval: Nlg evaluation using gpt-4 with better human alignment. In Proceedings of the 2023 Conference Empirical Methods in Natural Language Processing, pp. 2 2023.
LMSYS. Vicuna-7b v1.5 (huggingface model card).
vicuna-7b-vi.5 2023.
Yun Luo. Zhen Yang. Fandong Meng. Yafu Li, Jie Zhou, and Yue Zhang. empirical study of catastrophic forgetting in large language models during continual fine-tuning. arXiv preprint (XXi:23308.08447 2023.
OpenAI. Gpt-3.5-turbo. 2023.
opwnairrrrppee...oooomoooo. wwwpp/aa....accc////eoooooooaeae http:://aaafooor.oeeeeaecccccccc /gpt--t--- 1-pt------- 3-5
OpenAl. Gpt-4o. htppp:///penaa..cc//////////g/2 2024.
Haoran Que, Feiyu Duan, Liqun He, Yutao Mou, Wangchunshu Zhou, Jiaheng Liu, Wenge Rong, Zekun Moore Wang, Jian Yang, Ge Zhang, et al. Hellobench: Evaluating long text generation capabilities of large language models. arXiv preprint arXiv:2409.16191, 2024.
Ravi Raju, Swayambhoo Jain, Bo Li, Jonathan Li, and Urmish Thakker. Constructing domain-specific evaluation sets for llm-as-a-judge. arXiv preprint arXiv:2408 088008, 2024.
Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, al. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.0011 2024.
Swamaadepp Saha, Omer Levy, Asli Celikyilmaz, Mohit Bansal, Jason Weston, and Xian Li. franch-solve-mett improves large language model evaluation and generation. arXiv preprint arXiv:2310. 15/23, 2023.
11

---

Published as a conference paper at ICLR 2026
Hwanjun Song, Hang Su, Igor Shalyminov, Jason Cai, and Saab Mansour. FineSurE: Fine-gnt aine-grained summarization evaluation using LLMs. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 906-922, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10. 00.18633/1/2024. acl-long.51. URL httpp//ee////eaaataaooo.ooooooo/ acl aar.2224.1l1oooo..551
Andrea Sottana, Bin Liang, Kai Zou, and Zheng Yuan. Evaluation metrics in the era of gpt-4: Reliably evaluating large language models on sequence to sequence tasks. arXiv preprint arXiv:2310. 13800, 2023.
Annalisa Szymanski, Noah Ziems, Heather A Hicher-Miller, Toby Jia-Jun Li, Meng Jiang, and Ronald Metoyer. Limitations of the llm-as-a-judge approach for evaluating IIm outputs in expert knowledge tasks. In Proceedings of the 30th International Conference on Intelligent User Interfaces, pp. 952-966, 2025.
Sijun Tan, Siyuan Zhuang, Kyle Montgomery, William Y Tang, Alejandro Cuadron, Chenguang Wang, Raluca Ada Popa, and Ion Stoica. Judgebench: A benchmark for evaluating Ilm-based judges. arXiv preprint arXiv:2410.1223 12784, 2024.
Aman Singh Thakur, Kartik Choudhary, Venkat Srinik Ramayapally, Sankaran Vaidyanathan, and Dieuwke Hupkes. Judging the judges: Evaluating alignment and vulnerabilities in llms-as-judges. arXiv preprint arXiv:2406. 12066.122222 12624, 2024.
Prapti Trivedi, Aditya Gulati, Oliver Molenschot, Meghana Arakkal Rajeev, Rajkumar Ramamurthy, Keith Stevens, Tanveesh Singh Chaudhery, Jahnavi Jambholkar, James Zou, and Nazneen Rajani, Self-rationalizztion improves 11m as a fine-grained judge. arXiv preprint urXiv:2410.05499 2024.
Ruiqi Wang, Jiyu Guo, Cuiyun Gao, Guodong Fan, Chun Yong Chong, and Xin Xia. Can Ilms replace human evaluators? empirical study of llm-as-a-judge in software engineering. arXiv preprint arXiv:2502.06193, 2025a.
Victor Wang, Michael J. Q. Zhang, and Eunsol Choi. Improving llm-as-a-judge inference with the judgment distribution, 2025b. URL
03064
Yidong Wang. Zhuohao Yu, Wenjin Yao, Zhengran Zeng, Linyi Yang. Cunxiang Wang, Hao Chen, Chaoya Jiang, Rui Xie, Jindong Wang, et al. Pandalm: An automatic evaluation benchmark for I1m instruction tuning optimization. In The Twelfih International Conference on Learning Representations.
Yidong Wang. Qi Guo, Wenjin Yao, Hongbo Zhang, Xin Zhang, Zhen Wu, Meishan Zhang, Xinyu Dai, Qingsong Wen, Wei Ye, et al. Autosurvey: Large language models can automatically write surveys. Advances in Neural Information Processing Systems, 37:115119-115145. 2024.
Yidong Wang, Xin Wang, Cunxiang Wang, Junfeng Fang, Qiufeng Wang, Jianing Chu, Xuran Meng, Shuxun Yang, Libo Qin, Yue Zhang, et al. Temporal self-rewarding language models: Decoupling chosen-rejected via fast-future. arXiv preprint arXiv:2508.0660 06026, 2025c.
WizardLM Team. Wizardlm-13h v1 (huggingface model card).
co/
MizarcLMTeeam//iiaaaza 133---1110111112222221 2023.
Tianhao Wu, Weizhe Yuan, Olga Golovneva, Jing Xu, Yuandong Tian, Jiantao Jiao, Jason Weston, and Sainbayar Sukhbaatar. Meta-rewarding language models: Self-improving alignment with juda--ee- arXiv preprint arXiv:2407. 19594. 2024.
Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. Wizardlm: Empowering large language models to follow complex instructions. arXiv preprint arXiv:2304.122 12244 2023. httttttst///xiiii wwtp:iivvrg//b/////////
http:p/://aaaii...rrar////22..3/ 12244
Yi Xu, Laura Ruis, Tim Rocktäschel, and Robert Kirk. Investigating non-t in llm-as-a-judge. In Fectod International Conference on Machine Learning.
An Yang, Baosong Yang, Zhang, et al. Qwen2.5 technical report. 2024.
12

---

Published as a conference paper at ICLR 2026
Seonghyeon Ye, Doyoung Kim, Sungdong Kim, Hyeonbin Hwang, Seungone Kim, Yongrae Jo, James Thome, Juho Kim, and Minjoon Seo. Flask: Fine-grained language model evaluation based on alignment skill sets. arXiv preprint arXhv:2307.10928. 2023.
Weizhe Yuan, Richard Yuanzhe Pang, Kyunghyun Cho, Xian Li, Sainbayar Sukhbaatar, Jing Xu, and Jason E Weston. Self-rewarding language models. In Forty-first International Conference on Machine Learning.
Yifan Zhang, Ge Zhang, Yue Wu, Kangping Xu, and Quanquan Gu. Beyond bradley-terry models: general preference model for language model alignment. In ICLR 2025 Workshop Bidirectional Human-Al Alignment.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36:46595-46623, 2023.
Lianghui Zhu, Xinggang Wang, and Xinlong Wang. Judgelm: Fine-tuned large language models are scalable judges. 2023.
A RELATED WORK
Traditional Discrete Evaluation Protocols LLM-as-a-judge frameworks have become widely adopted for their scalability and cost-efficiency evaluating large language models. Early works predominantly relied on discrete evaluation protocols, including coarse single-scors ratings and pairwise preference comparisons. MT-Bench and Chatbot Arena Zheng et al. (2023) demonstrated the feasibility of using powerful LLMs such GPT-4 as judges, achieving high agreement with human preferences, while also noting issues such as verbosity and position bias. ArenaHard Li et al. (2024) proposed automated benchmark construction pipeline and introduced Arena-Hare Mama--ard-AAiee Auto, challenging benchmark curated without human-in- -ntth-iott---oop. which relies on LLMs to produce and evaluate responses.
AlpacaEval Dubois et al. 2024) highlighted persistent biases in LLM-based evaluation such as a preference for longer outputs. Their proposed length-controlled regression analysis mitigated this issue and improved correlation with human preferences. PandaLM Wang et al. introduced a pairwise judgment protocol that incorporates subjective dimensions like clarity, formality, and instruction adherence. Its judge model outperformed even GPT-4 in certain domains and was used to tune instruction-followinng models.
Other works such as Trivedi et al. (2024); Saha et al. (2023) Que et al. (2024): Ye et al. (023))
Fernandes et al. (023); Bai et al. (2023); Wang et al. (2025a) developed various discrete evaluation techniques, including majority voting, scalar ratings, skill-wise decomposition, and output-based scoring. While these methods brought interpretability and practical value, they were still constrained by coarse-gres gratied annotations and did not fully resolve contradictions between scoring types or internal inconsistencies. Notably, |eernandes et al. (023) proposed AUTOMQM for machine translation, which incorporated structured error labeling but remained within the paradigm of fixed- score prompting. [ai et al. (2023) proposed a language model examiner framework combining scoring and ranking, but without entropy-aware mechanisms. Similarly, |Wang al. (2025a) employed output- based scoring for software engineering tasks, emphasizing alignment with human evaluation but without probabilistic modeling. Additionally, Trivedi et al. (2024) used iterative reffrrationalization for enhancing model rationales but still within discrete scoring.
Probabilistic and Fine-Grained Evaluation Methods To overcome the limitations of discrete judgments, recent research has explored probabilistic evaluation strategies. G-EVAL Liu et al. (022) introduced softmax- normalized score prediction over a fine see-grained rating scale using chain-of thought prompting and form-filling, improving alignment with human preferences. [Wann al.] (2025b) further examined extracting fine-grained preferences by leveraging the distributional output of judge models, demonstrating that methods incorporating distributional judgments significantly outperform traditional greedy decoding across various evaluation scenarios.
13

---

Published as a conference paper at ICLR 2026
Instruction:
Below is a question and a candidate respense from user. Please act as impartial judge and evaluate the quality of the responses provided by two Al ascistnnts to the question displayed belon. You should choose the assistant that follons the user' instructions and answers the user's question better. Your evalaation should consider factors such the helpfulness, relevance. sccuracy, depth. creativity, detail of their responses. Begin evaluation comparing the two provide short explanation. Avald position blases presented does not influence
level and that the order ovaluation. your decision. Do allow assistants, Be objective possible. V'Bj"" verdict by strietly following this ''CI""
which the responses the of responses
favor certain brief explanation, ["AA]"
you "moss" output the
is result exclusively "square brackets** (e.g, Verdict: [A]-
assistant
Question:
Compose an engaging trivel blog post about recent trip to Hawaii, highlighting cultural experiences and see altractions.
Answer
Clouds shrouded Luau nights at various botels (Di you know you can also find luau options the beach?)** are hasically more tumt- key opportuniries to mest like aninded travelers at the resort complexes. A quirky day revisised that place just made my visit almost two wedks and nine days based an prior reviews I had. Frobably either searching the night markets island had lot of gratifying moments of vibrancy. Due to mixed reviews from prior travelers, shrimmp lempura huppened last thing the last night
aeruilly
the ocean the afternoon someehere that first
restaurant.
Answer B;
Alosa from paradise! My raceni trip lo Hawaii was a whirlwind of vibrani culture, breathiaking landscapes, and unforgettable experiences. Immensing mysell in the local way of life was traly enriching. I discovered the importance of reciing the "a010lina (land) and the deep connectinn Hawaiaas have their heriagge Leaming a few basic Howaiaan words and phrases
me connect with the locals and goin deeper appreciation for their culture. Of course, also explored some iconic landmarkss. The natural beauty of the islands simply awe-inspiring from the cascading waterfalls the pristine benches.
Judgement:
question. Assistant creates experiences Hawaii, such respecting engaging blug post highlights laaning
Assistant provides much better response
Hawaiisn hunloge The landmarks like waterfalls beaches. lacks coherence, jumping betweon random expenenees without elear focus cultural aspects or must-sce adtractioes Overall, Assistant Bts response more helpful, relevant, accurase, and decailed in addressing the user's is roqueal for an ongaging travel blug post about Hawaii. Inin" Verdict.
coersciing
mentions exploring
other hand,
Assastant response disjointed
"B" 0.3778800092223445
"A": 0.22867150777691111
"C"": 0.125266439479983...
Figure 5: Example of a pairwise evaluation prompt.
Our work builds upon and extends this direction by proposing TrustJudge, a probabilistic evaluation framework that preserves judgment entropy and explicitly resolves both score-comparison and pairwise transitivity inconsistencies in LLM-as-a aujdee paradigms.
B LIMITATIONS
Despite the demonstrated efficacy of TrustJudge, our approach still has some inherent limitations. Firstly, the performance of TrustJudge is fundamentally dependent the instruction fosllottonnfollooinng capabilities of the employed evaluation models. Smaller-scale language models often exhibit weaker instruction comprehension and execution capabilities, which could result in failure to yield valid scores or comparisons. Consequently, the quality and reliability of TrustJudge evaluations are directly tied the underlying judge model's competence, emphasizing the importance of model ability.
C PROMPT EXAMPLES
The following figures 5 and provide examples of evaluation prompts used to assess responses. The first figure shows pairwise comparison prompt, where two responses are compared and one selected as better. The second figure illustrates single-score evaluation prompt with the 5-point scale. where a response is rated based on quality metrics such as helpfulness and relevance. These examples are intended to support clarity and consistency in LL-M-as---jude. evaluation tasks.
14

---

highlighting Published as a conference paper at ICLR 2026
Instruction:
Pleuse impartial judge and carefully evaluate the quality of the Al assistant's response the question provided below. Your evaluation should consider factors such the helpfulness, relevance, accuracy, depth, creativity, and level detail the response. Procide concise, objective explunation summarizing your evaluation. IMPORTANT: explanation, give only a single, final numeric score on scule of to 5 in stequare brackets** strictly adhering to the following format: \"Soore: [N]"", where Xss your numeric (e.g. Scuur:: (2]?").
After your
Question:
Compose ongaging blog post recent trip cultural experienccs and must-sec altractions.
Answer:
I'm glad you enjoyed your recent trip Hawaii
travel blog post, share with you the cultural experiences and must-sce attractions that made your journyy unforgettable. Get ready to immerse yourself in the beauty and richness of Hawaiian culture as we explore vibrant luaus, trditionnl hula performances and delicious local cuisine. Don't forget to check out iconic affractions like the beaches Waikiki, the volcanic wenders of Hawsii Volcanoes National significance
Park, the
Harbor. So your sunsereen, put trapical paradise
on your leis, and let's emburk virtal journey Hawaii!
Judgement:
The response friendly and engaging. directly addressing the user's request for an engaging rnvel blog past about a recent trip to Hawaii, It includes highlights of cultural experincces and must-sec sttractions, painting vivid picture for the reader. The use of descriptive languige and inviting tone oreates inmersive experience for the audience, making them enger to explore Hawaii through the blog The offers comprehensive request effectively richness key attractions
response fulfills the
overview of trip, showcasing
Hawaii inin" " 4
I-4": 0L3776228459710011,
"3": 0.624299420550222, "2": 46650001555333511--55
55": 5.1756849929814488-077
Figure 6: Example of a single-score evaluation prompt with the 5 5-point scale.
D INFERENCE SETTINGS
Specifically, we included strong open-source models such as Llama-3-Athene-11 Frick et al. (2024)
Mama---77B-IIsttie Grrttatooii et al. (2024), and Llama-3-8h 3-B--33-uttt--tuutt GGrttafiori et al. (2024):
strong closed-soure, models such GPT-4o OpenAl 2024), GPT-4-Turbo OpenAl (2024). and
Claude /onnet|Anthropp (2024); ; weak open-source source models including Wi I.2 et al.
(002)) ; WizardLM Team 2023), Vicuna-7B Chiang al. 2023); LMSYS (2023), , and Alpaca- 13B Chavinlo (2023); and weak closed-source models such as Claude 3 Haiku Anthropic (2024) and GPT-3.5-Turbe OpenAl (2023). We batched inference of vLLM accelerate the generation and judging process, setting the temperature to 1.0, the maximum number of tokens to 2048, and providing the top 20 log probabilities for each generated token.
E GRPO TRAINING SETUP
We conduct GRPO training on Qwen2.5-7H8 IB-strrucct using the multi-task corpus introduced in Feng et al. (2025), following the 8,600/265 training- validation split. The Bascline and TrustJudge condi- lions are trained under an identical GRPO configuration, with rollout groups of size 8, a batch size of 256, maximum prompt length of 9000 tokens, and maximum response length of 1024 tokens. The actor and critic are optimized with Adam using learning rates of x 10-6 and x 10-5, respectively. for 150 GRPO steps. Unless otherwise noted, all other optimization hyperparameters follow the same settings across the two runs. The only difference lies in the reward function: one uses the Baseline scalarized 0-9 score, whereas the other uses the TrustJudge-be based reward.
based
shows the training reward trajectories of Dwwe2..57B--iee Instruct during GRPO under the shared hyperparameter setting, comparing the Baseline and TrustJudge reward formulations. Both runs exhibit steadily increasing rewards over the course of training. However, the TrustJudge-base achieves consistently higher reward than the Baseline- run across most of the 150 GRPO steps, suggesting that TrustJudge provides a more informative learning signal for policy optimization.
Figure 7
15

---

Published as a conference paper at ICLR 2026
Figure 8 reports per-task validation rewards assigned by external LLM judge (GPT-4o-mini throughout GRPO training, with greedy decoding used at evaluation time. For each dataset in the multi-task mixture, including instruction following, long-document summarization, GSM8K, essay tasks, English-French translation, and WildChat, solid curves denote Qwen2.5-7BBInnttuue trained with the Trust/udge reward, whereas dashed curves denote the counterpart trained with the Baseline reward. Across nearly all tasks, TrustJudge matches or outperforms the Bascline reward. This pattern consistent with the aggregate improvements reported in Table f suggesting that the benefits of TrustJudge are broadly distributed across domains rather than driven by gains any single dataset.
0.85
baseline
trustjudge
average reward
0.80
W M
0.75
0.70
0.65
20
40
60
80
100
120
140
160
step
Figure 7: Training reward during GRPO training of Qwenn.5-7B-Instiee -Isttrutt with Baseline and Trust- Judge rewards.
tulu3 baseline
tulu3 trustjudge
pubmed summarization baseline
1.0
pubmed summaricution trrstjudge
gem8k
gxm8k trastjudge
averra reward
0.8
govreport summarization baseline
govreport summarization trusttjudge
essay infill buseline
0.6
essay infill trustjudge essay abs2text baseline essay abs2text trustjudge french to english translation baseline
0.4
french to english translation trustjudge
english french translation baseline
english french trunslation, trassjuuggr
0.2
ursiv summarization baseline
summarization trustjudge
20
40
60
80 step
100
120
140
WildChat
WildChat
trustjudge WildChatttrusfuugge
Figure Per-task validation reward across GRPO training steps under an external LLM judge. Solid lines correspond to Wrren2.5-7B-Instrree trained with the TrustJudge reward and dashed lines correspond to Qwcn2.5-7B-Insste trained with the Baseline reward.
F EXTENSION TO MULTIIDIMENSSEE
EVALUATION
Setup. To assess whether TrustJudge can be extended to multi-dimensional evaluation, we evaluate three sub-dimensions: factuality, coherence, and helpfulness. We randomly sample 120 questions from Arena Hard dataset. For each question, candidate responses and judgements are generated by
16

---

Published as a conference paper at ICLR 2026
models from the Llama, Qwen, Gemma, and GPT families. Each sub-dimension uses dimension- specific prompt, which we show in Figures 9 and I0 respectively.
For each sub-dimension, Score- Score-Comparison Inconsistency reported as CR and Pairwise Transitivity Inconsistency reported as NT R& for & {3, 4). For brevity, Table 5 present the averages of these metries across the three sub-dimensions, while all metrics are computed per dimension as specified above.
independently compute two degrees of inconsistencies:
Instruction for Factuality:
Please act impartial judge and carefally evaluate the "*factuality** of the Al assistant's respanse to the question provided below, Your evaluation should consider factors informationn, verifiabillyy claims, the accuracy of dats referesces, and the abscace of factual mislending statements. Provide a concise, objective explonation summarizing your evaleation. IMPORTANT: After your explanution, give only a stagle. final aumeric score on scale of I to 5 in ""squre brackets" strictly adhering the following format: ("Seare: [NF". where X your numeric score (eg.. S"Sorre: [2[1").
Instruction for Coherence:
sesistant's response user qucation provided below. Your evaluation should consider factors such logical Bew, durity of structure, concistency argumentation, appropriate organization of ideas, and everal readability. Provide a concise, objective explanation summarising your orlluation. IMPORTANT: After your explanation, give only a single, final numoric score scale of in *"quure brackets*s strietly adhering to the fallowing format: ('Score: INJ", X your numerie (egg. ("Scare
impartial julge carefully evaluate **ceherrncy**
Instruction for Helpfulness:
act impurtial judge and carefully evaluate the "'heplulness" of assistaut's response user question provided below. Your evaluation should consider factors suca as relevance to the user's needs. depth of inight, practical utility, clarity of guidance, and effectiveness in addressing the question. Provide concise, objective explanation summartring
evaluation. IMPORTANT: After your explanation, give only scale 5 ""square brackets" strictly adhering following format: your numerit score (eg.. (eg.."SSore Sooore: (2[)").
final numerie
("Score: IN
Figure 9: Example of single-score prompts for multi-dimension evaluation.
Instruction for Factuality:
Below is question and two candidate responses from Al assistants to the user question displayed below, Please impartial judge and evaluate which response consider factors such as the correctness of information, the veriffability of claims, the accuracy of data and references, and the absence of factual errors misleading statements. Begin your evaluation by comparing the two responses and provide a concise explanation of which is more factually sound. Avoid any position biases and ensure that the order of the responses, their length, the names of the assistants do not influence your decision. After providing your explanation. output only the final verdiet by strictly following this format: !'[]]" if assistant A's response more factually accurate, ("B]]" if assistantt B's response more factually accurate, and V[CJ" for tie, exclusively square brackets (e.g. Verdiet: (A])
factually accurate. Your evaluation should
Instruction for Coherence:
Below is question and two candidate responses from Al assistants to the user question displayed below. Please aet impartial judge and evalunte which response is more coherent. Your evaluation should consider factors such as logical flow,clarit clarity of structure, comsistency of argumentation. appropriate urganization of ideas, overall readability. Begin your evaluation by comparing the two respomes and explanation of which is more coherent. Avaid any position biases and ensure that the order of the responses, their length, or the names output only the final verdict by strictly following this format: "[AA"" assistant A's response is more coherent, V[BJ" assistant B's response more coberent, and !'[CI" for tie, exclusively square brackets (e.g.- Verdict: [AJ)
concise
not influence your decision. After providing your explanation,
Instruction for Helpfulness: :
Below question and two candidate responses from AI assistants to the question displayed below. Please net as an impartial judge and evaluate which response is more helpful. Your evaluation should consider factors such relevance to the user's needs, depth insight. practical utility, clarity of guidance, and effectiveness in addressing the question. Begin your evaluation by comparing the two responses and provide concise esplination of which more helpful. Avoid any position binses and ensure that the order the responses, their length, or the names of the assistants do not influence your decision. After providing your explnattonn output only the final verdiet by strietly following this format: ''AA[" assistant A's response is helpful, ("B])"" if assistant B's response is more helpfal, and I"[C"" for a tie, exclusively in square brackets (egg, Verdiet: [A]).
Figure 10: Example of pairwise prompts for multi-dimension evaluation.
Results and analysis. Extending the judge to three axes- coherence, and helpful- ness ness-yields clear reduction in inconsistency. With dimension-sppecific prompts and per-dimension computation, we observe drops on every model and on both of the metrics: CR decreases by roughly 5.13% 11.03% while NTR, and NTR, fall more sharply, on average by 11 233.-4.....
17

---

Published as a conference paper at ICLR 2026
Table 5: Results for multi-dimeeniinal evaluation across three tan-dimeensioss-fcctullllii coherence. and helpfulness. For each sub-dimension, CR and NTR, are computed independently; tables report the mean across sub-dimensions.
CR %
NTR==3 (%)
NTR (%)
Model
Baseline
Ours
Baseline
Ours
Baseline
Ours
Gemma-2-27be
49,43
44.30
19.60
8.20
48.76
22.41
Owen2.5-32B-Inne.
45.73
37.87
17.38
7.89
42.55
22.36
Llama-3 1amma-3.1-70B- --stt--Intttt
52.20
41.47
18.29
5,48
44.65
16.21
The pattern most visible with Llama-3.1-70B, where NTR, contracts from 44.65% 16.21%
and NTR3 from 18.29% to 5.48%, alongside a CR decline from 52.20% to 41.47%. Qwen2.5- 32B and Gemma-2-27B-Insstne modestly (e.g., Gemma 49.43% - 44.30%), pairwise non-transitivity is still more than halved (48.76% + 22.41%). ))). Taken together, the improvements persist when quality is decomposed into orthogonal components rather than measured as single undifferentiated score.
show the same direction of change; even where narrows more
Mechanistically, the scalar channel benefits from cetrriuutionssensitte scoring, which smooths discretization artifacts and reduces clashes between numeric scores and pairwise preferences, lowering CR. The pairwise channel benefits from ikelihood-award aggregation with calibrated tie handling. which suppresses position bias, lowering NTR. Because these effects arise within each dimension before averaging, the evidence indicates genuine generalization of TrustJudge to multi-dimensional evaluation.
G GENERALIZATION ACROSS DATASET CATEGORIES
Setup. To assess whether observations generalize across task types, used 120 prompts from MT-Bench and Arena Hard as the main experiment; for each prompt we independently collected ten model responses
to obtain quality-diverse distribution of outputs, yielding a total of 1,200 responses. The 120 prompts were assigned to eight predefined MT-Bench categories as shown in Figure IT Evaluation was performed with three judges Qwen2.5-7B -ween..--7-Instrreet, Llama-3.1-8B. Instruct, and Gemma-2-9b-it assessment on a 5-point scale and (ii) pairwise comparisons between responses. report the Conflict Ratio under the 5-point and the Non-Transitivity Ratio with i 4. Results presented both aggregated across categories and broken down by the eight MT-Bench categories to illustrate the variation in inconsistency patterns across task types.
which each assessed all 1, 1,200 responses using both (i) single-score
category we
99.2%
Coding
Reasoning
36%
Math STEM
10%
Humanities
Writing
30%
Roleplay
Extraction
Figure 11: Category distribution across the eight dataset categories.
Results and analysis. As shown in Table across eight MT-Bench categories and three judges, the clearest pattern is in pairwise transitivity consistency: con-transitivity ratio collapses from 18.74% under the two-pass baseline 4.40% with likelihood-awee aware aggregation method and 5.64% with the PPL-based method (averaged over all 24 category-judge cells). That reduction is uniform-almost every category and every judge shows single-digit NTR after applying our pairwise aggregation, with extremes such Llama-STEA, reaching 0.00%, and large cuts in difficult regimes like Qwen-Math (32.85% - 4.46%). In short, once responses are compared bidirectionally with likelihood-avve -warr tie handling, residual inconsistencies rare regardless of task type.
18

---

Published as a conference paper at ICLR 2026
Score-Cor Co--armmppannn Conflicts show cells, CR drops from 23.32% to 20.63% with ceaibbtioo--essittiio scoring. However, looking category-wise, method is the best (or tied best) in three of eight groups that emphasize open- ended generation-Codigg (Ours 21.78% vs. G-Eval 22. 2,.13%) Reasoning (Ours 20.72% vs. G-Eval G-Eval 24.09% while G-Eval leads in STEM, Humanities,
nuanced, category deeeedeed story. Averaged over all
21.17%), and Writing (Ours 23. 23.93% Roleplay, and Extraction. Math is the lone case where the raw baseline edges out both methods by small margin (Baseline 19.41% % vs. Ours 19.55% G-Eval 20. 10%).% These contrasts suggest that when responses span wider stylistic or pragmatic range, TrustJudge that preserves rating entropy tends to reduce come-commanison inconsistency; when the signal more templated or tightly factual, G-Eval probability summation can be slightly better calibrated.
Table 6: Results for two category-wise inconsistencies. Left block (Score- Comparissn Inconsistency): Baseline, G-Eval scale. Right block (Pairwise Transitivity Inconsistency): two-pass swap-order Baseline, TrustJudge's likelihood-awee aggregation (Option B in [). and PPL-based method (Option A in D Judges are ("Qwen"), and gemma-2-90 2-9b-it ("Gemma")
summmaionn, and TrustJudge's s distribution-sensibte scoring on 5-point
Llama-3. In8-rrucct ("Llama"), Qwen2.5-7B-Ime
Score Comparconnn (CR, %)
Pairwise Transitivity (NTR:=4> %)
Category
Model
Baseline
G-Eval
Ours
Baseline
Likelihood
PPL-based
Llama
31.19
27.74
27.59
22.07
3.72
7.80
Coding
Qwen
26.14
25.33
23.69
19.86
4.95
6.19
Gemma
18.52
13.33
14.07
16.76
3.81
5.91
Llama
31.18
25.79
25.90
22.08
5.01
6.87
Reasoning
Qwen
27.53
29.35
26.48
23.93
5.56
9.69
Gemma
10.23
8.37
9.77
14.13
2.52
5.71
Llama
24.24
25.25
24.24
23.26
4.86
5.21
Math
Qwen
26.63
30.65
28.54
32.85
4.46
9,64
Gemma
7.35
4.41
5.88
16.84
4.29
6.48
Llama
25.62
17.77
19.42
9.03
1.94
0.00
STEM
Qwen
29.35
26.75
26.23
23.07
3.68
5.70
Gemma
9.52
4.76
9.47
1.11
3.23
Llama
27.08
21.67
21.67
19.14
4.29
4.00
Humanities
Qwen
23.88
21.49
20.30
20.38
3.86
3.41
Gemma
12.24
2.04
6.12
7.81
2.12
1.63
Llama
38.71
30.97
30,97
23.10
2.07
14.83
Writing
Qwen
20.95
30.48
30.00
26.19
10.71
5.06
Gemma
18.92
10.81
10.81
11.43
3.62
1.90
Llama
35.04
29.91
27.35
12.50
5.47
1.56
Roleplay
Qwen
28.49
26.16
28.49
24.69
6.76
7.70
Gemma
16.07
5.36
14.29
10.71
4.76
6.43
Llama
40.63
34.38
35.94
18.87
3.77
3.77
Extraction
Qwen
30.12
30.12
32.53
28.85
4.23
7.69
Gemma
0.00
0.00
0.00
12.62
8.10
5.00
Practically, the category study shows the generalization of TrustJudge. The Akkliihodd-waar aggrega- tion and PPL--based method are robust task type, driving down inconsistencies nearly everywhere. The cinirruutioo-sensttii scoring is competitive overall and tends to be strongest where outputs are diverse and rubric-driven (coding, reasoning, writing).
H THEORETICAL DERIVATION
THEORETICAL ANALYSIS OF DISTRIBUTION-SENSIIII SCORING
In the LLM-as-a-Jure paradigm, judge model M assesses given response R. The model's internal assessment can be conceptualized as a conditional probability distribution over a discrete
19

---

Published as a conference paper at ICLR 2026
set of possible scores θ {$i...... st). We denote this probability mass function (PMF) as Pre(s) _ Par s| R), where 5 is random variable representing the score. The uncertainty ambiguity this assessment is captured by the conditional entropy:
(()))) = Σ
pn(s) log (n())(
SEe
Traditional discrete scoring protocols extract single score by taking the mode of this distribution. We define the discrete scoring function JDiserete as:
JDiscrete A k-1 + O JDiscrete (PR) = arg max Prr(s)
sEB
where A is the (k: simplex representing all possible probability distributions over the scores. This function maps a probability distribution a single point estimate. The core issue with this approach is that the arg max operator is non-injective; it discards all information about the distribution's 's shape and uncertainty (entropy), mapping distinct belief states the same output score. This information loss is primary source of score core-commarison inconsistencies.
In contrast, our proposed distribution-sensiivv scoring function, fus. computes the expected value of the score distribution:
Jos A 1-=1==
fDS(PR) ES [S]
Σ 8 pR(s)
This function maps the entire probability distribution to a continuous scalar value, preserving more information about the underlying assessment. The following theorem formalizes the information preservation property of fDs compared to the information loss inherent in fDiscrete-
Theorem H.1 (Information Loss of Discrete Scoring and Preservation in Expectation). Let pn, and PR, be two distinct probability distributions over the score set θ representing the judge model's assessment of two different responses, R₁ and R2 (i.e., PR, # pR,). The discrete scoring function f can fail to distinguish between these two assessments, whereas the distribution-sensitee scoring function fus provides a mechanism for their discrinination. Specifically:
1. (Information Loss):
There exist PR. # PR, with different conditional entropies, HS)))t + H (S))2)) such that their discrete scores are identical: fDiscrete (p D fDiscrese(PR.)> (PR,). )
2. (Information Preservation): For the same distributions PRi and PR2 constructed in (1), their distribution-sene sensitivv scores are distinct: fos(PR,) & fDD((RR.)
Proof. We will prove the theorem by formal symbolic construction.
Let the score set be e. Let us choose two distinct scores 8m. $a E O such that 8m * sa. Let Sm be the intended mode of our distributions. Further, let choose two distinct real numbers é and 62 such that 0 < 61, f2 < 0.5 and 61 * 62- The condition < 0.5 ensures that 1 1 e, which will be necessary to establish 8m the unique mode. The condition 6 # 62 ensures the resulting distributions are distinct.
Consider two responses, R2 and Rz, which elicit different internal belief distributions from the judge model, defined as follows:
1. Let pn, be probability mass function (PMF) where the probability mass is concentrated on 8m and s
s = 8m E1
PR, (s)
= 61
if 8 = 8a otherwise
0
2. Let pn, be second, distinct PMF, also concentrated 8m and sa but with different balance:
1 - E2
if 8m
PR (s)
62
if 8 8a otherwise
0
Since e # 62, have pn, # Pn
20

---

Published as a conference paper at ICLR 2026
PART 1: PROVING INFORMATION LOss IN JDISCRETE
We apply the discrete scoring function fDiscrete both distributions. By our choice of e (0,0.5), we have 1 - > and 1 E2 > Therefore, the mode for both distributions is uniquely 8mi
fDiscrete (PR,) arg max PR; (s) 8m
fDiscrete (PR.) arg max PR₂ (s) 8m
sge
Thus, we have shown that for two distinct distributions PR, and PR,, it is possible that fuiscreue (Pa,) fDiserew (P (PR,).
Now, we consider their conditional entropies. The entropy of these distributions is function of e: (1) ((1 61 log(1 61) 61 (1
H(S) = ((1 e2) log(1 - -2) + + log E2)
The binary entropy function (p) p (1 p) (1
is strictly increasing the
H()) interval (0, 0.5). Since we chose E # E2 within this interval, it follows that H (S))) # ( R2). This confirms that fDiscrete maps distributions with different levels of uncertainty the same output, thereby losing information. This proves the first part of the theorem.
PART 2: PROVING INFORMATION PRESERVATION SnS
Next, we apply the distribution scoring function fns to the same distributions pR, and pr,
fus(DR,) = E|
Σ
8 ER, = 3m (1 <1) + SE(1)
SEG
(DS(PRA) = R
Σ
8 PR) = 8m (1 - -22) + sa(e2)
SE
To demonstrate that their scores are distinct, us assume for contradiction that they are equal:
fDS(PR, = fus (PR,)
mm (1 (1) + sa((1) sm(1 62) + sa Sc(22)
9m 8m11 + SĄE1 8m -mm2 + 8052
(1 5m) =E2(Sa (8m)
(e1 (8 sm) 0
This equality can only hold if 61 62 chose 61 # 62 (so 6 - 62 * 0) and 8a F 8m (so 8a 8m # 0). This leads to a contradiction.
Sa 8m O. However, by our initial construction,
Therefore, assumption must be false, and it must be that fuS(PR,) fus(Pn,). The distribution- sensitive scoring function successfully distinguishes between these two belief states, preserving the discriminative information lost by fDiscrtee This proves the second part of the theorem.
H.2 THEORETICAL ANALYSIS OF LIKELIIIOD-AARR AGGREGATION
The PPL-based estimator is designed to resolve ambiguity. From an coermatioon-hheoeedee perspective, ambiguity in a discrete choice problem corresponds a high-entropy probability distribution over the possible outcomes. The PPL-based method leverages an alternative signal-the generative likelihood of the rationale-to to induce a more confident (lower-entropy) posterior belief for decision-making The following proposition formalizes this concept.
Proposition H.2 (Uncertainty Reduction via PPL-based Method), Lei C (1, be the set
of outcomes. Let (TC( be the original outcome distribution from the judge model, and H(C)z) its Shannon entropy. In an ambiguous regime, this distribution approaches uniformity, causing (C]F) log (C).
Let a new "confidence" distribution Peonf be derived from the rationales Ju for each outcome k E C:
exp(-7- (PPL()))
Deong(A (k) Eise -iSPOX((---PPLL.))
21

---

Published as a conference paper at ICLR 2026
where O is scaling constant. If there exists least outcome is whose rationale has a strictly lower perplexity than another (i.e.. 3k1, k2 s.t. PPL(Jk,) < (P2((J))) then there exists a a such that the entropy of the confidence distribution is strictly lower than the maximum possible entropy:
H (Poon) < log [C]
This demonstrates that Crri makes decision based on a more certain signal, reducing the judgment uncertainty present in the original ambiguous distribution.
Proof. The Shannon entropy function, Hp) =- Ps log Pi, is a strictly concave function over the probability simplex. Its unique maximum is achieved when the distribution is uniform, i.e., P. = 1/|C| for all i. In this case, H (p) = log [C].
In ambiguous regime, the original outcome distribution p(C(r)) is, by definition, near-uniform. Consequently, its entropy (C is close to its maximum possible value, log [C].
Now, consider the confidence distribution Pcont- The condition 3k1, k2 s.t. (PL()k.) < ()2() implies that the values exp( PPL(J&)) are not all equal. As a result, after normalization, the distribution Peonf is not uniform.
Because the Shannon entropy function's 's maximum is uniquely attained by the uniform distribution, any non-uniform distribution must have strictly lower entropy. Therefore,
H (peont) < max H(p) = log [C]
Since (C log [C), follows that H (Pcont) H(C
The parameter "y controls the "peakedness" of Peonf. As y + oc, Peonf approaches a Kronecker delta function centered the outcome with the minimum PPL, and its entropy approaches zero. Thus, for any non-trion con-trivial difference in PPLs, we can always choose 7 to make the decision signal arbitrarily certain.
This proves that the PPL-based method transforms a high-entropy (ambiguous) belief state into lower-entropy (more certain) one, thereby providing a more discriminative signal for making a final judgment.
desirable property of any comparison function f(Rz, Ryy) is symmetry, meaning that swapping the inputs should simply invert the outcome, i.e., f (Ry Rz) = - Ry). Single-pass estima- tors often violate this property due to positional bias. The following proposition proves that our bidirectional estimator inherently stable and symmetrie by construction.
Proposition H.3 (Symmetry and Stability of the Bidirectional Estimator). Let the single-pass greedy estimator be Čsr(Rz, Ry) arg max; ((,) M)). Due positional bias, this estimator not generally symmetric, meaning there exist pairs (R,, Ryy for which Ösr (Ra, Ry) # -Csp(R Rx).
In contrast, the bidirectional estimator Č3 is perfectly symmetric for all inputs:
Cp(Ba, Ry) -CB Ra)
This property makes ÖB a stable estimator with respect to the input ordering.
Proof. Let's define the aggregated score function for Cn(Ra, Ryy as m[k;; Rr, Roy) p[k](RR. Ry)) + p( R.)). The decision is arg max; m(k; Rw, Ryy.
Now consider the estimator for the swapped input, Čn Ry, R.). score function mn(k; Rays Ra) = p(k| (Ry, R,)) + p( k (R,, Ry)))
Let's compare the score for outcome in the first case with the score for outcome k -1 the second case:
m11 Rg, Ryy) = p(1[(R,, Ry)) + p(-)) R2))
m(-1; Ry, R.) p( (1),R R.)) (-(((-1) Ry)) = = p(-1((R, Ra)) + p(1[(Rz, Ry))
22

---

Published as a conference paper at ICLR 2026
We that m(1; R2 Ry) m(11; Ry: R2). By the same logic, m(-1; Rg, Ryy m(1; Ry. R.,)) and m(0; Rg. Ry) = m0; Ry, Rz).
This means that the score assigned to preference "R2 Ry" in the first ordering is identical to the score assigned to preference "R > Ry" " (which is outcome - -) in the second ordering. Therefore, the maximum score in the first is for outcome k", the maximum score in the second case must be for outcome -k". This proves (B Ry) = CR, R2).
23


Reviews:
[{"review_id": "CE7APharY2", "reviewer": "Reviewer_VUVV", "summary": "This work identifies two types of inconsistencies in LLM evaluation systems: score-inconsistency and pairwise-inconsistency. To address the inconsistency issues, the authors invents novel fine-grained/probabilistic scoring and perplexity/likelihood-based ranking methods and incorporate them into the proposed framework called TrustJudge. The  paper also provide a comprehensive theoretical proof and a series of experimental analyses on the effectiveness of the proposed the method. The results show TrustJudge and its components significantly reduce the evaluation inconsistencies in LLM-as-a-judge systems.", "questions": "To show the effectiveness of continuous probabilistic scoring alone, it possible to directly apply softmax on the 5-point scale granularity?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1 The problem formulation of evaluation inconsistencies is sound and convincing. This paper goes from a practical issue which is common but often overlooked to a carefully designed solution with a solid theoretical foundation. \n2  The scoring and ranking methods proposed to resolve the inconsistencies of scoring and comparison in LLM evaluation systems are well-formulated.\n3 The proposed system demonstrates good results on different LLMs and problem settings. This indicates the method is robust and may contribute to a large community.\n4 The paper is organized in a clear and logical structure.", "weaknesses": "Some symbols are confusing and should be revised： e.g. (1) k is the size of subsets in def 2.2 while k also represents possible comparisons outcomes in eq(6); (2) \\Theta^{\\prime} is score set at line 157 and then become range at line 154 of Algorithm 1; (3) \"p_{R_1} and p_{R_2} constructed in eq (1)\" is depicted at line 235, but I do not see such definition in eq(1) at line 130.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "O4uxpUQo7A", "reviewer": "Reviewer_C5zH", "summary": "This paper focuses on addressing two core inconsistencies in the LLM-as-a-judge paradigm: Score-Comparison Inconsistency and Pairwise Transitivity Inconsistency, which stem from information loss in discrete scoring systems and ambiguous tie judgments in pairwise evaluations. The authors propose TrustJudge, a probabilistic framework with two key components: 1) distribution-sensitive scoring and 2) likelihood-aware aggregation. Experimental results showing TrustJudge reduces Score-Comparison inconsistency and Pairwise Transitivity inconsistency.", "questions": "Please see the weaknesses.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* The paper makes a notable original contribution by systematically identifying and formalizing two fundamental inconsistencies (Score-Comparison Inconsistency and Pairwise Transitivity Inconsistency) in the LLM-as-a-judge paradigm—an issue not previously analyzed in a unified framework.\n\n* The work has practical and academic significance: practically, TrustJudge improves LLM-as-a-judge reliability (reducing inconsistencies while preserving accuracy) without extra training/annotations, making it scalable for real-world LLM evaluation and DPO training.", "weaknesses": "* Novelty Gaps in Core Components: While the paper frames TrustJudge as innovative, key elements overlap with prior work without sufficient differentiation. \n\n* Insufficient Analysis of Limitation Impact:  it does not test how small models (e.g., 3B Llama-3.2) with weaker instruction comprehension fail to generate valid scores/comparisons—critical for a \"model-agnostic\" framework. \n\n* Theorem 3.1 proves information preservation for two-distribution cases but does not extend to multi-distribution scenarios (e.g., 3+ responses in transitivity checks)—a key use case for Pairwise Transitivity Inconsistency.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pUa8YupLDn", "reviewer": "Reviewer_nkL4", "summary": "This paper makes methodological advances towards addressing inconsistencies in LLM-as-a-judge systems, identifying two representative failure modes: Score-Comparison Inconsistency (when single-score ratings contradict pairwise preferences) and Pairwise Transitivity Inconsistency (when preference judgments form cycles or contradictions). The authors propose TrustJudge, a framework based on probabilities that:\n1. uses distribution-sensitive scoring to preserve information entropy and produce continuous expectations from discrete ratings, and\n2. applies likelihood-aware aggregation (perplexity-based or bidirectional probability methods) to resolve transitivity violations.\n\nThe authors conduct comprehensive experiments that show that their method can reduce the inconsistency metrics (both score-comparison conflicts and transitivity violations).", "questions": "Increasing score granularity clearly helps resolve inequality-type transitivity inconsistencies (ties), but does it also reduce circular inconsistencies? It would be informative to separate these two types of inconsistency in the results to show the impact of TrustJudge on each.\n\nIn experiments comparing TrustJudge with baseline LLM-as-judge setups, are the PPL-based and likelihood-aware aggregation strategies applied to all questions, or only to those where the baseline judge outputs a tie? I assume the former, but clarification in the paper would be helpful.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The paper presents extensive experiments across a diverse range of models (Llama-3, GPT-4o, Qwen, Gemma) and datasets (MT-Bench, ArenaHard), demonstrating generality.\n\n- The proposed method can be applied post-hoc, without retraining judges or requiring additional human supervision, a strong practical advantage for real-world usage.\n\n- The paper provides some theoretical intuition for why the TrustJudge framework yields more consistent rankings, grounding the empirical studies in a formal explanation about information preservation and entropy reduction.\n\n- The distribution-sensitive scoring mechanism introduces a simple but effective normalization trick: converting model-produced probabilities into logits before softmax normalization, ensuring a valid probability distribution.", "weaknesses": "-\tThe explanation of the bidirectional probability-based aggregation method (Eq. 6) is difficult to follow. It would help to include a small, concrete example illustrating what p_order1 and 2 represent, and how m[k] is computed and interpreted.\n\n-\tThe notion of the “ambiguous regime” in Proposition 3.2 requires clarification. Does this term refer exclusively to tie cases, or does it also include cases where the LLM judge selects a winner with low confidence?\n\n-\tThe phrase “lower-rated responses outperform higher-scored ones” (see abstract) may confuse readers; make sure to include the reference to single-score and pairwise evaluations.\n\n-\tThe authors should clarify (Eq. 5) why comparing perplexities of two full orderings is preferable to simply comparing each response’s perplexity under the judge model individually.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1XJtgcNGOH", "reviewer": "Reviewer_gSy2", "summary": "This paper proposes a framework, TrustJudge, designed to alleviate inconsistencies within the LLM-as-a-Judge paradigm. The authors identify and define two primary issues: Score-Comparison Inconsistency and Pairwise Transitivity Inconsistency, and introduce probabilistic methods, namely distribution-sensitive scoring and likelihood-aware aggregation, to address them.", "questions": "See weaknesses.", "limitations": "", "rating": 2, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 1, "strengths": "The paper tackles a significant and highly relevant problem. As LLM-as-a-Judge systems become more prevalent, understanding and mitigating their inherent inconsistencies is crucial for building trustworthy evaluation pipelines. The authors' focus on this issue is a valuable contribution to the field.", "weaknesses": "1. The primary weakness of this work lies in the limited novelty of its core strategies. Several of the proposed techniques are extensions or reformulations of existing methods:\n    - The use of log probabilities for scoring is a known technique. The main contribution appears to be ensuring a properly normalized distribution by excluding non-score tokens, which is more of a **technical implementation detail** and is supported by modern LLM inference frameworks (e.g., vLLM). The likelihood-aware aggregation for pairwise comparison can be seen as a **straightforward extension** of this probability-based approach.\n    - Mitigating position bias by swapping response orders is a **standard practice** discussed in prior work [1]. The issue of **transitivity inconsistency** in pairwise comparisons has also been explored previously [2]. While there may be differences in the precise problem formulation and solution, the incremental contribution appears marginal. The manuscript would benefit from a more thorough discussion of these related works.\n\n2. The proposed **X-point scale** lacks demonstrated generalization and is likely to be heavily influenced by the specific tokenizer used by the judge model. For example, \"50\" can be tokenized to \"5\" and \"0\" in Qwen series LLM, making the proposed method inapplicable. Furthermore, the authors do not provide an empirical comparison to show whether this fine-grained scale performs better than a simpler probabilistic binary judgment (e.g., Like a yes-or-no [3]), which could be a more robust and less tokenizer-dependent alternative.\n\n3.  The analysis in Section 3, which concludes that discrete scoring systems suffer from information loss, addresses a well-understood phenomenon. It is intuitive that representing a full probability distribution with a single discrete score sampled from that distribution will result in a loss of information. This portion of the theoretical analysis does not **seem to offer a novel insight**.\n\n4. The application of perplexity (PPL) for resolving ties lacks a strong theoretical foundation. The proof in Proposition 3.2 seems to apply to any scenario where LLM outputs have unequal likelihoods, not just those measured by PPL. The core assumption that **a sequence with lower PPL corresponds to a higher-quality response aligned with human preference** is heuristic. It lacks a causal link and is not experimentally proven. Factors such as *response length, verbosity, or stylistic simplicity* could affect the PPL of a concatenated sequence without directly correlating with higher quality [4]. This weakness is further highlighted by the experimental results, where the PPL-based method is almost universally outperformed by the likelihood-based approach, which itself is a direct extension of using log probabilities.\n\n5. Readability of Figures: Figure 2 is difficult to interpret. A bar plot might be a more effective choice for visualizing this data.\n\n------\n### References\n[1] Zheng et al. \"Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena.\" NeurIPS 2023.\n\n[2] \"Language Model Preference Evaluation with Multiple Weak Evaluators.\" arXiv 2410.12869.\n\n[3] Li et al. \"Generative Verifiers: Reward Modeling as Next-Token Prediction.\" ICLR 2025.\n\n[4] \"What is Wrong with Perplexity for Long-context Language Modeling?\" ICLR 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)