TrustJudge: Inconsistencies of LLM-as-a-Judge and How to Alleviate Them
Authors: name: Yidong Wang, name: Yunze Song, name: Tingyuan Zhu, name: Xuanwang Zhang, name: Zhuohao Yu, name: Hao Chen, name: Chiyu Song, name: Qiufeng Wang, name: Cunxiang Wang, name: Zhen Wu, name: Xinyu Dai, name: Yue Zhang, name: Wei Ye, name: Shikun Zhang

The adoption of Large Language Models (LLMs) as automated evaluators
(LLM-as-a-judge) has revealed critical inconsistencies in current evaluation
frameworks. We identify two fundamental types of inconsistencies: (1)
Score-Comparison Inconsistency, where lower-rated responses outperform
higher-scored ones in pairwise comparisons, and (2) Pairwise Transitivity
Inconsistency, manifested through circular preference chains (A>B>C>A) and
equivalence contradictions (A=B=C\neq A). We argue that these issues come from
information loss in discrete rating systems and ambiguous tie judgments during
pairwise evaluation. We propose TrustJudge, a probabilistic framework that
addresses these limitations through two key innovations: 1)
distribution-sensitive scoring that computes continuous expectations from
discrete rating probabilities, preserving information entropy for more precise
scoring, and 2) likelihood-aware aggregation that resolves transitivity
violations using bidirectional preference probabilities or perplexity. We also
formalize the theoretical limitations of current LLM-as-a-judge frameworks and
demonstrate how TrustJudge's components overcome them. When evaluated with
Llama-3.1-70B-Instruct as judge using our dataset, TrustJudge reduces
Score-Comparison inconsistency by 8.43% (from 23.32% to 14.89%) and Pairwise
Transitivity inconsistency by 10.82% (from 15.22% to 4.40%), while maintaining
higher evaluation accuracy. Our work provides the first systematic analysis of
evaluation framework inconsistencies in LLM-as-a-judge paradigms, offering both
theoretical insights and practical solutions for reliable automated assessment.
The framework demonstrates consistent improvements across various model
architectures and scales, enabling more trustworthy LLM evaluation without
requiring additional training or human annotations. The codes can be found at
https://github.com/TrustJudge/TrustJudge.