Improving LLM -as-a-Judge Inference with the Judgment Distribution (2025)

Abstract: Using language models to scalably approximate human preferences on text quality (LLM-as-a-judge) has become a standard practice applicable to many tasks. A judgment is often extracted from the judge’s textual output alone, typically with greedy decoding. However, LLM judges naturally provide distributions over judgment tokens, inviting a breadth of inference methods for extracting fine-grained preferences. We find that taking the mean of the judgment distribution consistently outperforms taking the mode (i.e. greedy decoding) in all evaluation settings (i.e. pointwise, pairwise, and listwise). We further explore novel methods of deriving preferences from judgment distributions, and find that methods incorporating risk aversion often improve performance. Lastly, we analyze LLM-as-a-judge paired with chain-of-thought (CoT) prompting, showing that CoT can collapse the spread of the judgment distribution, often harming performance. Our findings show that leveraging distributional output improves LLM-as-a-judge, as opposed to using the text interface alone.

## introduction llm-as-a-judge has emerged as a scalable framework for evaluating model outputs by approximating human annotation @xcite @xcite . typically, such systems prompt off-the-shelf llms to score a response or rank multiple responses to a given user prompt. llm-as-a-judge methods boast strong agreement with human judgments across a breadth of domains and criteria @xcite , despite current limitations @xcite . most prior work involving llm-as-a-judge elicits judgments through the llm's text interface @xcite @xcite , where the most likely token (i.e. the mode of the next token distribution) or a sampled token is taken to represent the llm's judgment. recent works @xcite @xcite have suggested that taking the mean of the score token distribution can better represent the llm's judgment. in this work, we comprehensively evaluate design choices for leveraging llm judges' distributional output. 1 we show that the mean consistently outperforms the mode in the pointwise, pairwise, and listwise settings (i.e. evaluating one, two, and many responses at a time). specifically, the mean achieves higher accuracy in 42 out of 48 cases on re-wardbench @xcite and mt-bench @xcite . we further explore novel methods of deriving preferences from score distributions (section 4). for example, incorporating risk aversion often improves performance. categorizing methods as discrete or continuous, where discrete methods (e.g. mode) are simple to interpret like rubric scores, we find that continuous methods outperform discrete methods, due to the latter often predicting ties and failing to capture slight preferences. in particular, the mode assigns ties more frequently than every other method, leading to the lowest accuracy even among discrete methods. we further study how chain-of-thought (cot) prompting @xcite impacts the performance of llm-as-a-judge. after the cot reasoning, llms often exhibit sharper score distributions, making the mean judgment similar to the mode. removing cot increases the spread of the judgment distribution, often improving performance, and more so for taking the mean than taking the mode (e.g. absolute +6.5% for mean vs. +1.4% for mode, on average with pointwise scoring on re-wardbench), demonstrating the synergy between eliciting and using distributional output. llm judge 1 2 3 4 5 1 2 3 4 5 user prompt could you write a captivating short story… response a the bustling marketplace was a sensory… llm judge could you write a captivating short story… the bustling marketplace teemed with… mode mean 5 5 3.84 3.31 response b a b a b tie user prompt sys. prompt please rate the response from 1 to 5. consider… please rate the response from 1 to 5. consider… sys. prompt a wins score (k) p(k) score (k) p(k) distributional output to maximize the effectiveness of llm-as-a-judge, as opposed to using the text interface alone. as llm-as-a-judge paradigms are widely adopted for complex tasks, improving best practices for using llm-as-a-judge can impact many end tasks' development and evaluation. ## distributional judgment in this section, we present our findings comparing mode vs. mean inference and cot vs. no-cot prompting for llm-as-a-judge systems. ## study on pointwise scoring beyond the mode and mean discussed in prior work and the previous section, we further explore the design space of utilizing distributional output from llm scorers. ## study on pairwise ranking the judgment styles in section 3's overview were scoring (section 4) and ranking. in this section, we analyze design decisions for pairwise ranking, and in section 6 listwise ranking. ## listwise judgment listwise judgment is not as prevalent as pointwise or pairwise judgment, but it offers efficiency @xcite while granting the judge the maximal context for comparison @xcite . ## conclusion and recommendations we comprehensively evaluated design choices for leveraging llm judges' distributional output. for pointwise scoring, we showed that continuous methods (e.g. mean) outperform discrete methods (especially the mode) due to ties. for pairwise ranking, we related the mean vs. mode comparison to pre-vs. post-aggregation of the two presentation orders' judgments. although smaller llm judges suffer heavily from inconsistent judgments due to position bias, pre-aggregation effectively leverages the relative magnitudes of preference. we showed that cot collapses the spread of the judgment distribution, often hurting performance. this applies even to the challenging setting of listwise ranking, where accuracy was maximized by directly predicting the list without an intermediate pairwise step. we hope that highlighting this limitation of cot encourages the development of reasoning mechanisms that preserve output diversity and calibration for judgment and other subjective or open-ended tasks. recommendations we summarize our findings into guidelines for choosing judgment settings. large judges like gpt-4o should use pairwise ranking no-cot, or direct listwise ranking as an efficient alternative. smaller judges like llama-3.1-8b should use pointwise scoring no-cot. the mean should be used instead of the mode, but these setting guidelines apply even if one uses the mode.