JudgeLM: Fine-tuned Large Language Models are Scalable Judges
Authors: Lianghui Zhu
Xinggang Wang
Xinlong Wang
Venue: ICLR 2025 Spotlight (2025)
Keywords: LLM Judging
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: We propose a high-quality LLM judge dataset and a series of strong LLM judges to address the biases in LLM judging.

Abstract: Evaluating Large Language Models (LLMs) in open-ended scenarios is challenging because existing benchmarks and metrics can not measure them comprehensively. To address this problem, we propose to fine-tune LLMs as scalable judges (JudgeLM) to evaluate LLMs efficiently and effectively in open-ended benchmarks. We first propose a comprehensive, large-scale, high-quality dataset containing task seeds, LLMs-generated answers, and GPT-4-generated judgments for fine-tuning high-performance judges, as well as a new benchmark for evaluating the judges. We train JudgeLM at different scales from 7B, 13B, to 33B parameters, and conduct a systematic analysis of its capabilities and behaviors. We then analyze the key biases in fine-tuning LLM as a judge and consider them as position bias, knowledge bias, and format bias. To address these issues, JudgeLM introduces a bag of techniques including swap augmentation, reference support, and reference drop, which clearly enhance the judge's performance. JudgeLM obtains the state-of-the-art judge performance on both the existing PandaLM benchmark and our proposed new benchmark. Our JudgeLM is efficient and the JudgeLM-7B only needs 3 minutes to judge 5K samples with 8 A100 GPUs. JudgeLM obtains high agreement with the teacher judge, achieving an agreement exceeding 90% that even surpasses human-to-human agreement. JudgeLM also demonstrates extended capabilities in being judges of the single answer, multimodal models, multiple answers, multi-turn chat, etc.

Published as a conference paper at ICLR 2025
JUDGELM: FINE-TUNED LARGE LANGUAGE MODELS ARE SCALABLE JUDGES
Lianghui Zhu 1:2
Xinggang Wang't
Xinlong Wang"
School of EIC, Huazhong University of Science & Technology Beijing Academy of Artificial Intelligence
Code & Models: https: / htt.g//t....//aaaaaiiiioooo////a
ABSTRACT
Evaluating Large Language Models (LLMs) in open-ended scenarios is challeng- ing because existing benchmarks and metries can not measure them comprehen- sively. To address this problem, we propose to fine-tune LLMs as scalable judges (JudgeLMM to evaluate L.MMs efficiently and effectively in open-ended bench- marks. We first propose comprehensive, large-scale, high-quality dataset con- taining task seeds, LLMs-generated answers, and GPT-4-generated judgments for fine-tuning nigh-performance judges, well as new benchmark for evaluating the judges. We train JudgeLM at different scales from 7B, 13B, to 33B parame- ters, and conduct systematic analysis of its capabilities and behaviors. We then analyze the key biases in fine-tuning LLM as a judge and consider them as position bias, knowledge bias, and format bias. To address these issues, JudgeLM intro- duces a bag of techniques including swap augmentation, reference support, and reference drop, which clearly enhance the judge's 's performance. JudgeLM obtains the seate-offthe-eeee and our proposed new benchmark. Our JudgeL. is efficient and the JudgeLM-77 only needs 3 minutes to judge 5K samples with 8 A100 GPUs. JudgeLM obtains high agreement with the teacher judge, achieving an agreement exceeding 90% that even surpasses human-to-human agreement'. JudgeLM also demonstrates extended capabilities in being judges of the single answer, multimodal models, multiple answers, multi-turn chat, etc.
judge performance on both the existing PandaLM benchmark
INTRODUCTION
Recent advancements in large language models (LLMs) have fostered significant interest due to their remarkable performance in following instructions and their broad capabilities in dealing with open-ended scenarios. Based on the open-source LLMss, including OPT (Zhang et al., 2022), Flan- T5 (Chung et al., 2022), LLaMA (Touvron et al., 2023a), and Pythia (Biderman et al., 2023), re- searchers propose numerous methods to align these models with human preferences through instruc- tion fine-tuning. These aligned LLMs demonstrate enhanced abilities in comprehending human in- structions and generating more coherent responses. Nonetheless, existing benchmarks (Hendrycks et al., 2020; Liang al., 2022) and traditional metrics (Lin, 2004; Papineni et al., 2002; Zhang et al., 2019; Sellam et al., 2020; Yuan et al., 2021) do not adequately estimate the capabilities of LLMs in open-ended scenarios. Therefore, new benchmark method that could evaluate LLMs comprchensively in open- tasks is needed.
Concurrent works are making efforts to explore various methods for evaluating the performance of LLM. The arena-format (Zheng et al., 2023) methods leverage crowdsourced platforms to extract anonymous LLM competition results. While evaluations by humans are trustworthy, they are also time-consuming and financially demanding. Some approaches (Chiang et al., 2023) utilize GPT-4 as
"This work was done when Lianghui Zhu was an intern at Beijing Academy of Artificial Intelligence. 1 Corresponding authors: xgwang@hnne @uust. edu. en and wangxin wwaggiinlongeen longgbbai aaa.aaaaa....o..... aac. cn.
As reference, the max agreement among humans in MT--hench (Zheng al., 2023) is 82%.

---

Published as a conference paper at ICLR 2025
judge. Nevertheless, these methods grapple with challenges of potential data exposure and volatile API model transitions, potentially compromising the judge's reprodueibility. PandaLM (Wang et al,, 2023) attempts to fine-tune open-soure LLMs for evaluating answers. However, limitations stem- ming from the training data quality, and inherent LLM biases, undermine the effectiveness of such fine-tuned models in the role of judge.
Task
Answer Pair
GPT-4 Judgments
Instruction: Identify words that are used as modifiers.
UTA
Vicuna
Answerl: modifiers used the sentence are: "green,"" 2"
10,
OO
Konla
Reason: Assistant2 didn't provide any answer the question, hencc the low score...
She drove her green car quickly.
Imput:
LLaMA
She drove her green car quickly.
Answer2:
ChaGGLM
Alpaca
Open
(a) Data generation pipeline of our JudgeLM. first collect 105K seed tasks as questions. Then, we extract answers from 11 LLMs and randomly sample a pair of answers from the answer set. Last, we input the tasks, the sampled answer pairs, and optionally reference answers to GPT-4, which generates scores and detailed reasons as judge teacher.
optional input
6: swap answer threshold (0.5) f reférence drop threshold (0
random(): random numbers between (0.1)
Judge Sample
Task
randomo
random()
No
y ref. sup.
Answer
x ?
8
Vicura Alpaca
Judgment
DO LLaMA
Reference
Kir
Aswup aug.
/ ref. drop
LLMS
JudgeL
(b) An illustration of the JudgeLM'ss fine-tuning and various functions. We use generated judge samples to fine-tune LLMs as scalable judges. When fine-tuning LLMs as judges, we also propose swap augmentation, reference support, and reference drop to address the position bias, knowledge bias, and format bias, respectively.
Score:
LLMI
LLM2
LLMI LIME
LIMI
LLMH LLME LLMES
LIMI LAME
LIMI LLNE
Judge Answer Puirs
Explain Judgments
Grade
Judge
Multi-turn Chut about Judgments
Judge Multimodal Answers
Single Answers
Multiple Answers
(c) An illustration of various functions of our JudgeL.M.
Figure I: An overview of our scalable JudgeLM including data generation, fine-tuning, and various functions.
In this paper, we propose to evaluate LLMs through fine-tuned open-source LLMs, which serve as scalable judges (Judgel.M) achieving satisfactory agreement with the teacher judge. Our method- ology incorporates scalable judges as evaluators in open-ended tasks, coupled with a high-quality dataset conducive to both training and evaluating the judge models. Within our framework, we adapt open-source LLMS to serve as judges and analyze their scaling ability in relation to model size (ranging from 7B to 33B) and volume of training data (extending from 3.5K to 100K). Our curated dataset comprises 105K seed questions, LLM answer pairs, and judgments from the teacher judge, GPT-4, as shown in Fig. la. Note that we generated two judgments for each seed task with and without reference answers. This dataset is partitioned, with 100K seed questions allocated for training (2 x larger than Pa and the remainder for validation (29 larger than Pandal
Utilizing LLMs as judges inevitably introduces biases such as position bias (favoring answers in spe- cific positions), knowledge bias (over-reliance on pre-trained knowledge), and format bias (optimal performance only under specific prompt formats) as shown in Fig. 8, 10, 12, 13. When fine-tuning not possible, GPT-4-API-based judge (Zheng et al., 2023) tries to alleviate this by well-designed prompt methods, i.e., Chain-of- of-thought, few-shot judge, and judging multiple times with different positions. JudgeLM presents a new way that can address these biases in the fine-tuning stage, skip- ping the complicated prompt methods and multi-turn API calling. Moreover, our Judgel.M system presents extended capabilities as shown in Fig. 1b, including grading single answers, judgiig multi- ple answers, judging multimodal modcls, multi-turn chat, etc.
In contrast to arena-format methods, our approach is rapid and has a low cost. For instance, Ju7geLM-7B requires only 8 A100 GPUs and can evaluate 5000 response pairs in just 3 minutes. In comparison to closed-source LLM judges, JudgeLM ensures reproducibility and protects user pri- vacy. When compared to concurrent open-source LLM judges, our system explores both the scaling ability and biases LLM fine-tune muninun Furthermore, JudgeLM dataset stands as the most diverse and high-quality one, significantly benefitting subsequent research in judge model investigations.
2

---

Published as a conference paper at ICLR 2025
Our main contributions can be summarized as follows:
We introduce a high-quality, large-scale dataset for judge models, enriched with diverse seed tasks, LLMs- -egeereeeet answers, and detailed judgments from GPT-4, laying the foun- dation for future LLMs evaluating research.
We propose JudgeL.M, a scalable language model judge, designed for evaluating LLMs in open-ended scenarios. It achieves an agreement exceeding 90% that surpasses the human- to-human agreement. Our JudgeLM can also generalize to many extended tasks.
We analyze the biases inherent to LLM judge fine-tuning and introduce a series of meth- ods to address them. Our methods significantly improve the consistency of the model in different cases, making the JudgeI.M more reliable and flexible.
2 RELATED WORK
2.1 INSTRUCTION FINE-TUNING OF LARGE LANGUAGE MODELS
With the development large language models (LLMs), researchers find that fine-tuning pre-trained LLMs such as GPT-3 (Brown et al., 2020). T5 (Raffel et al., 2020), OPT (Zhang et al., 2022), and PaLM (Chowdhery et al., 2022) enable LLMs to follow human instructions and help with open- ended tasks. The instruction fine-tuned LLMs such as InstructGPT (Ouyang et al., 2022), Chat- GPT (OpenAI, 2022), FLAN-T5 (Chung et al., 2022), FLAN-PaLM (Chung al., 2022). OPT- IML (lyer et al., 2022), and GPT-4 (OpenAI, 2023) exhibit stronger ability in zero-shot or few-shot tasks than their base models. After Meta released the powerful open-source L.LM LLaMA (Tou- vron et al., 2023a) and LLaMA2 (Touvron et al., 2023b), lots of instruction fine-tuning works based on LLaMA or LLaMA2 were proposed in the natural language generation or multimodal genera- tion domain, such as Alpaca, Vicuna (Chiang al., 2023), OpenFlamingo (Awadalla et al., 2023), LLaMA-Adapter (Zhang et al., 2023), and Emu (Sun et al., 2023). Our JudgeLM also belongs to the LLaMA family and takes the Vicuna series as base models. Our JudgeLM follows the instruction fine-tuning manner to create LLM judges and proposes to model the judgment-gennratton task as "grading, judging, and reasoning". We further collect high-quality, large-scale dataset for research in judging the performance of LLMs.
2.2 EVALUATION OF LARGE LANGUAGE MODELS
As many open-source large language models (LLMs) and their fine-tuned variants are proposed and present remarkable performance various tasks, evaluating the capabilities of LLMs becomes popular and challenging task. To address this problem, Chatbot Arena (Zheng et al., 2023) aims to build crowdsourced platform that ranks the LIMSs through pairwise comparison and Elo rating. The crowdsourced way to evaluate LLMs has more reliable results but faces high costs and low effi- ciency. Vicuna (Chiang al., 2023) uses GPT-4 judge to select the better answer. Although the GPT-4-based method can judge LLMs like a human expert, the API-based methods have potential risks of data leakage and unstable performance. Zeno Build (Alex & Graham, 2023) proposes to evaluate LLMs at a customer service dataset, but using traditional metrics such as ChrF (Popović, 2015) and BERTScore (Zhang et al., 2019) can not fully evaluate the answers of LLMs in open- ended tasks. Besides, Pandal (Wang et al., 2023) and Auto-J Li et al. (2023a) developed judge models based on I.aMA (Touvron et al., 2023a) or I.LMMA2 (Touvron al., 2023b) to compare answers produced by I.Mss. When serving as judges, Pandal.M achieves an accuracy close to Chat- GPT but ignoring the inherent LLM biases limits its performance further. Our JudgeLM contains scalable judges from 7B 7B-parameter to 33B-parameter and achieves state-of-the-ar performance in both P and our benchmarks. Furthermore, researchers can use the proposed JudgeLM lo- cally which ensures reproducibility and data security.
DATASET
High-quality, large-scale datasets crucial for effectively fine-tuning large language models (LLMs) to act as evaluative judges. However, the concurrent datasets, such as the one by Pan- dal .M (Wang et al., 2023), present limitations in terms of diversity and the granularity of judgment
3

---

Published as a conference paper at ICLR 2025
Input
Question
If Rosslyn Mountain Boys had produced two more alhums, how many alhums would they have produced??Roslyn Mountain Boys was band that backed up number of nationally known singers, released albums of their own.
Answer
Ground Truth
Rosslyn Mountain Boys produced two albums on their own, so if they produced two more albums then they would have produced four albums.
Answer 2
Rosslyn Mountain Boys produeed
Traditional Metries
Answer
Rouge-11 Rouge-21
Rouge-Li
BLEU?
BERTScoref
BLEURTT BARTScore?
1
0.00
0.00
0.00
0.00
0.62
-1.39
-5.33
2
0.30
0.24
0.30
0.52
0.76
-0.78
-3.76
Our Judgment Output
Answer 1's Score: 10
Answer 2's Score:
Assistant 2's response is incomplete and does not provide information answer to the question, hence the low On the other hand, Assistant I's response is accurate and directly answers question. The Rosslyn Mountain Bays produced two albums, and they had produced two more, they would have produced a total four albums. Therefore, Assistant receives perfect score.
Figure 2:
The input and output of our JudgeLM data sample. In open-ended scenarios, traditional metrics can not judge answers accurately by comparing the answers with ground truth. However, the LLM judges can understand the questions and answers and give accurate scores and reasons.
criteria. To address this, we introduce a novel dataset replete with a rich variety of seed tasks, comprehensive answers from modern LLMs,, answers' grades from the teacher judge, and detailed reasons for judgments. Section 3.1 elucidates the data generation process, while Section 3.2 delin- eates the methods adopted for training and evaluation using our dataset.
3.1 DATA GENERATION
The primary objective of our data generation is to create large-scale and diversified dataset that maximizes the evaluative capabilities of judge models. We sample 105K instruction seed tasks from large-scale set that contains Alpaca-GPT4 (Peng et al., 2023), Dolly-15K (Conover et al., 2023), GPT4AII-LAION (Anand et al., 2023), and ShareGPT. To enhance the heterogeneity of the dataset, answers collated from 11 leading open-source L.MMs including, but not limited to, LLaMA (Tou- vron et al., 2023a), Alpaca, and Vicuna (Chiang et al., 2023). Following this, we amalgamate L.M- generated answers with the reference answer create answer sets. Pairs are randomly selected from the sets, upon which, fine-grained scores and detailed reasons are assigned by the advanced teacher model, GPT-4. To ensure robust and comprehensive judgments, we utilize detailed tem- plates as demonstrated in Fig. 3. Additionally, to allow the model to judge with reference answers, reference-incllsivee template is employed as Fig. 4. This encourages the model to integrate ex- ternal knowledge during the evaluative process. Please note that all samples in the JudgeLM val set are further checked and re-annotated by authors to ensure alignment with human preference.
3.2 TRAINING AND EVALUATING
To better utilize dataset to train and evaluate the judge models, partition it into training split and a validation split. The training set contains 100K judge samples, while the validation set has SK. We then introduce the way use this dataset train and evaluate, respectively.
4

---

Published as a conference paper at ICLR 2025
Training. The training process of JudgeLM adheres to the instruction fine-tuning paradigm. As il- lustrated in Fig. 2, the model is fed question alongside a pair of answers, and an optional reference answer, yielding outputs comprising scores and detailed reasons. It is imperative to note the signif- icance of a detailed crafted prompt template to harness the full potential of Judgel.M's instruction- following ability. Distinct input templates cater scenarios with and without references, depicted in Fig. and Fig. 4 respectively.
To further analyze the scaling ability of JudgeLM, we fine-tune JudgeLM with sizes of 7B, 13B, and 33B parameters. The specific hyperparameters are enumerated in Table 11. As for the scaling analysis for dataset size, we also fine-tune JudgeLM on varying data scales from 3.5K to 100K samples. JudgeLM demonstrates scaling ability both in terms of model size and data volume.
Evaluating, For the judge's ' result, we model as "grading, judging, and reasoning". The judge model first generates scores for answer pairs. Subsequently, we can get the judge result from three situations: "Answer wins" if the answer I's score higher than the answer 2's, "Answer 2 wins" if the answer 2's score is higher, or "Tie" if the scores of two answers are the same. Last, the model generates detailed reasons needed. The advantage this modeling that the judge model just needs little time to grade and judge, and generates time-consuming reasoning optionally.
For the metrics, we employ the objective metrics and reliability metrics to evaluate the judge models comprehensively. For the objective metrics, compute the agreement, precision, recall, and FI- score between the model's judge results and those of the teacher. This provides insights into the alignment of judge models with established benchmarks, such GPT-4 human experts. As for reliability metrics, we first compare the results before and after swapping LLM answers. Then we calculate the self-consistency to measure the judge model's reliability. Last, we further calculate the metrics like "bias toward Ist", "bias toward 2nd", and "delta bias" to get insights from specific position biases and their variance.
4 INHERENT BIAS
In this paper, we also study the inherent biases that influence the reliability of fine-tuned LLM judges through reliability metrics and visualizations.
Position Bias. Position bias means that the LLM judges prefer answers in certain position and it Wang et al., 2018) and decision- making of humans (Blunch, 1984; Raghubir & Valenzuela, 2006). The powerful I.Mss, ChatGPT and GPT-4, also face this challenge when working as judges (Wang et al., 2023; Zheng et al., 2023; Li et al., 2023b). As the qualitative and quantitative results shown in Fig. 8 and Table 5, JudgeLM also faces the position bias and prefers the first answer when swapping the positions of answers.
widely exists in natural language processing tasks (Ko et al., 2020;
Knowledge Bias. Knowledge bias arises when the pre-trained data lacks the knowledge of some seed tasks or induces possibly undesirable knowledge (Ko et al., 2020; Zheng et al., 2023) that could degenerate the generative capabilities of L.MMs. Fig. 10 provides an example that LLM judges can not give correct judgments to open-ended tasks if they lack related truth.
Format Bias. Researchers expect that the judge model can make judgments based pre-trained knowledge when the reference is not available and can make judgments following the reference when it is available. However, our experiments revealed that judge models fine-tuned without ref- erence perform poorly in judging with reference, and vice versa, as shown in Fig. 12, Fig. 13, and Table 6. We hypothesize tieettnnng with references encourages the judge model to make judg- ments based on external knowledge and fine-tuning without references pushes the judge model to make judgments through its pre-trained knowledge. We name the situation that judge fine-tuned without reference but validated with reference as mismatched format, and vice versa. Such format bias limits the further generalization of the judge model other domains.
5
METHOD
In evaluating LLM LLM-generated answers for seed question, the LLM judge aims to determine the superior answer from pair of candidates. Motivated by recent methods (Touvron et al., 2023a; Chiang al., 2023; Ouyang al., 2022), we present present.Judge..M. scalable judge model, and address
5

---

Published as a conference paper at ICLR 2025
inherent biases in such models. Our methodology is depicted in Fig. 1b. The subsequent sections provide a detailed breakdown of our approach.
5.1 SWAP AUGMENTATION
MT-bench (Zheng et al., 2023) and PandaLM (Wang et al., 2023) alleviate the position bias by judg- ing twice with original and reverse order. These methods regard the result as tie if the judgments are not the same. This kind method ignoring the inherent position bias and casting double time to evaluate, can be regarded as compromise and does not improve the reliability of LLM judges.
Intuitively, swapping the positions at the fine-tuning stage could push the judge model to pay more structured judge data,
attention to the contents of answers rather than positions. Leveraging we can easily swap the positions of answers to generate a new input sample. Correspondingly, we also swap the scores and question indexes of the judgment from the teacher (i.e., GPT4) get the new ground truth. As shown in Fig. 15, the augmented judge sample keeps the same results but exchanges the positions of answers. Overall, simple but effective to augment the training data and address position bias. The BodgeLM-witth-ware aattt--staapgammttttiioon can give good judgment to the same judge sample as shown Fig. 9.
5.2 REFERENCE SUPPORT
Introducing external knowledge in the fine-tuning stage an intuitive way make up for the lack of related pre- pre-trained knowledge. To do so, we propose the reference support method to teach the model judge with the help of reference answers. Following Zheng et al. (2023), we collect refer- ence answers for all judge samples and re-generate effrrece-guudee judgments by GPT-4. Please note that GPT-4 also gives different scores and judgments for most judge samples with or without references. This proves that the differences between pre-trained knowledge and reference answers greatly impact judgments. As shown Fig. 11, the JudgeLM with reference support can avoid fac- tual errors and give reliable judgments. Furthermore, introducing reference support to LLM judges can simply insert judge preferences. JudgeLM with reference support training can flexibly set ref- erence answers with different preferences for different scenarios and needs. As shown in Fig. 16, changing reference answers does not need extra training and makes JudgeLM more flexible to dif- ferent preferences.
5.3 REFERENCE DROP
To address the format bias, we introduce method, named reference drop, in which randomly drop the training sample with reference and use the corresponding sample without reference. As shown in Fig. 14, judge models with reference drop can alleviate the overfitting for fine-tuning formats and make judgments based on external reference or pre-trained knowledge when given reference not, respectively. Furthermore, the reference drop method also makes the judge model easy to use and decreases the cost of fitting into different formats.
EXPERIMENT
We study the performance of JudgeI.M as follows: Section 6.1 presents the main results of Judgel MM comparing with concurrent methods, Section 6.2 analyzes the scaling ability JudgeLM from both model sizes and data scales, and Section 6.3 shows ablation studies of proposed methods in detail. Detailed settings are shown in Section A.2.
6.1 MAIN RESULTS
Comparison on JudgeLM Benchmark. We first evaluate the proposed JudgeLM on our val set. Note that JudgeLM val set is further checked and re-annotated by authors to ensure alignment with human preference. As shown in Table we give the quantitative results of GPT-3.5, Vicuna- PandaLM-7B, Auto-J-13B (Li et al., 2023a), InstructScore-7B (Xu et al., 2023), and our JudgeLM with three model sizes. Among them, GPT-3.5 is used in the form of APIs with the help of templates in Fig. and Fig. 4. Pandal and Juto-J-13B are deployed with the released checkpoints and

---

Published as a conference paper at ICLR 2025
Table 1: Main results for our JudgeLM and concurrent methods on our val set, which uses GPT-4 annotation results as ground truth.
Agreement f (w/ GPT-4)
Precision t
Recall f (w/ GPT-4)
F1 f
Consistency (w/ swap.)
Methods
(w/ GPT-4)
(w/ GPT-4)
Judge w/o reference.
GPT-3.5
73.83
70.70
52.80
52.85
68.89
Vicuna-13BB
PandaLM-7B 68.61
40.75
38.82
39,41
74.78
Auto-J-13B
74.86
61.65
57.53
58.14
84.34
Judge w/o reference (Ours).
JudgeLMM-BB
81.11
69.67
78.39
72.21
83.57
JudgeL.M-13B
84.33
73.69
80.51
76.17
85.01
JudgeLM-33B
89.03
80.97
84.76
82.64
91.36
Judge w/ reference.
GPT-3.5
71.46
56.86
51.12
51.14
62.94
Vicuna- 13B
PandaLM-7B
63.77
39.79
34.82
35.18
55.39
Auto-J-13B
72.90
58.80
56.12
56.59
82.84
InstructScore-7B
55.80
58.74
56.84
53.72
Judge w/ reference (Ours).
JudgeLM-77B
84.08
75.92
82.55
78.28
84.46
JudgeI.M-133 JudgeI.M-33B
85.47
77.71
82.90
79.77
87.23
89.32
84.00
86.21
84.98
92.37
Table JudgeLM zero-shot evaluation results on PandaLM test set, which uses human annotation results as ground truth. "*" means the results are reported in PandaLM (Wang et al., 2023)
Agreement 1 (w/ Human)
Precision f
Recall 1 (w/ Human)
F1 F
Methods
(w/ Human)
(w/ Human)
zero-shot methods.
GPT-3.5*
62.96
61.95 66.20
63.59
58.20
GPT-4"
66.47
68.15
61.80
Fine-tuned on PandaLM train set.
PandaLM-7B
59.26
57.28
59.23
54.56
Ours (zero-shot).
Judgel. M--7B
65.07
66.89
71.95
61.92
Judgel.M--131
68.97
68.21
74.15
65.12
Judgel.M-33BB 75.18
69.30
74.93
69.73
templates. These methods could be regarded zero-shot methods because they by the JudgeLM dataset. On Judgel.M val set, the vanilla Vicuna-13B fails 77% of questions. Specifically, the vanilla Vicuna-13B can even output a pair of scores in judgments in the failed cases. But the finetuned version, i.e., JudgeLM, would not fail any questions in the JudgeLM val set. Our JudgeLMs are fine-tuned with proposed methods, i.e., swap augmentation, reference support, and reference drop. So, they can handle situations with or without references simultaneously. It can be observed that our JudgeL.M-7B outperforms PandaLM-7B. Auto-J, and InstructScore in all metrics, and even surpasses GPT-3.5. Furthermore, the proposed Judgel-M3333 exhibits the most powerful judge ability.
fine-tuned
Comparison on Other Human Evaluation Benchmarks. We further evaluate our JudgeLM on other Human evaluation benchmarks, i.c., PandaLM test set and human-annotated MM-Vet. Pan- daLM's train and val sets are annotated by GPT-3.5 and humans, respectively. Following the manner the PandaLM val set, we present the zero-shot results of JudgeLM in Table 2. It can be observed that the JudgeLM-7B outperforms GPT-3.5 and PandaLM-7B. When compared with GPT-4, JudgeLM-7B has lower accuracy and higher Precision, Recall, and Fl-score than GPT- 4. Furthermore, Judgel achieves higher results than GPT-4, which demonstrates that fine-

---

Published as a conference paper at ICLR 2025
Table 3: Efficiency comparison for our JudgeLM and PandaLM on our val set. We use machine with 8 Nvidia-A100 GPUs with 40G memory to evaluate their efficiency.
Methods
model size
GPUs per model
parallel judge?
generate reason?
total time
PandaLM
7B
6 hrs 40 mins
Ours.
JudgeLM
7B
/
6 hrs 40 mins
JudgeLM
7B
24 mins
JudgeLM
7B
V
50 mins
JudgeLM
7B
mins
JudgeLM
13B
X
5 mins
JudgeLM
33B
mins
Table 4: Performance analysis for the scaling JudgeLM on our val set.
Agreement (w/ GPT-4)
Consistency (w/ swap.)
Bias toward 1st
Bias |
Judge Size
Data Scale
Delta bias
toward 2nd
7B
3.5k
75.87
73.45
19.83
6.72
13.11
7B
10k
78.89
78.25
17.30
4.45
12.85
7B
30k
81.43
80.89
14.54
4.57
9.97
7B
100k
83.71
82.62
12.31
5.07
7.24
13B
3.5k
80.61
78.91
14.68
6.41
8,27
13B
10k
83.19
81.90
13.42
4.68
8.74
13B
30k
84.39
82.99
11.96
5.05
6.91
13B
100k
85.87
83.01
11.53
5.46
6.07
33B
3.5k
85.38
85.16
9.34
5.50
3.84
33B
10k
87.49
86.40
8.32
5.28
3.04
33B
30k
88.84
87.34
7.57
5.09
2.48
33B
100k
90.06
87.93
6.85
5.22
1.63
Table 5: Ablation study for the swap augmentation on our val set.
Agreement f (w/ GPT-4)
Consistency f
Bias toward 1st
Bias toward 2nd
Methods
Delta Bias +
(w/ swap.)
baseline
75.87
73.45
19.83 15.34
13.11 6.72
+ +swap aug. 76.51
78.89
5.77
9.57
tuned JudgeLM can outperform its teacher in this specific task. Besides, also propose a human- annotated multimodal judging benchmark to evaluate our JudgeLM as shown in Table 13.
Efficiency comparison. To further compare the efficiency between our Judgel and PandaLMM we conduct experiments on our val set to display the time cost using the same machine with NVIDIA-A100 (40G) GPUs. As shown in Table 3, display the methods and model sizes in the first and second columns. The third column shows the needed GPUs for each judge model. The models with 7B 13B parameters run A100 GPU with 40G memory while the 33B -a--aareettr needs GPUs. The fourth column shows whether the methods can judge answers in parallel. The fifth column indicates whether judge reasons are generated runtime. The sixth column presents the total time cost. We use BandaL--7BB and JudgeL.M-79 as efficiency baselines, which do not use paralleljudging and generate detailed reasons for all questions. Thanks the modeling of JudgeLM, i..., "grading, judging, and reasoning", JudgeLM can skip the reasoning phase and only requires 24 minutes, which 16.65 x faster than baselines. When we enable the engineering optimization of parallel judging, the JudgeLM can make full use of the 8 GPUs and cost only 50 minutes, which x faster than the baseline running on a single GPU. When we enable parallel judging and skip the reasoning phase for JudgeLM, the JudgeLM-7B only consumes 3 minutes to judge 5000 response pairs, which is 133.3 x faster than baselines. The largest judge model, JudgeL.M-33B, can also complete the validation within 15 minutes. JudgeLM's high efficiency significantly reduce the time spent on evaluating LLMs, allowing researchers and developers to boost the pace of advancements.
8

---

Published as a conference paper at ICLR 2025
Table 6: Ablation study for the reference support and reference drop on our val set.
ft w/ ref?
vol w/ ref?
Agreement (w/ GPT-4)
Consistency (w/ swap.)
Bias 1
Bias J
Delta Bias 4
Methods
toward 1st
toward 2nd
matching format.
baseline
x /
x
75.87
73.45
19.83
6.72
13.11
/
baseline
80.15
81.23
11.55
7.22
4.33
mismatched format.
baseline
x
73.09
67.75
29.44
2.81
26.63
baseline
/
X
75.69
73.40
20.89
5.71
15.18
w/ ref. drop.
baseline ref. drop
X
76.86
77.13 81.24
17.30 11.48
5.57
11.73
baseline ref. drop
V
80.35
7.28
4.20
Table 7: Performance of JudgeLM-7B with explanation-fiss (CoT) or score-firy first (Ours) on JudgeLM
val set.
Agreement 1 (w/ GPT-4)
Consistency 1
Bias toward Ist
Bias L toward 2nd
Methods
Delta Bias 4
(w/ swap.)
score-first (Our)
75.87 75.54
73,45 74.39
19.83 15.05
6.72 10.56
13.11 4.50
explanation-first (CoT)
6.2 SCALING ANALYSIS OF JUDGELM
In this section, we analyze the scaling ability of the plain JudgeLM (without the proposed methods) on our val set without reference as illustrated in Table 4. As we increase the model size and data scale, can observe the metries increase. demonstrates that the proposed JudgeLM is scalable and can reach to 90.06% agreement and 87.93% consistency with 33B 33B-parameter and 100K fine- tuning data.
6.3 ABLATION STUDY
In this section, present the ablation studies of the proposed methods. For all ablation studies, we use Judgel.M--B as the base model and 3.5K data for fine-tuning. Based on this baseline, we analyze the improvements brought by swap augmentation, reference support, and reference drop.
Improvements Swap Augmentation. As shown in Table 5, swap augmentation can improve the baseline model comprehensively. It improves consistency by 5.44%, which demonstrates that swap augmentation can reduce the influence of position bias and push the judge to pay more attention to the contents of answers.
Improvements of Reference Support. As shown in the rows with the matching format of Ta- ble JudgeLM fine-tuned with reference support exhibits superior performance on every metric. demonstrates that the introduction of reference answers induces the judge to rely on external knowl- edge and addresses the limitation of pre-trained knowledge.
Improvements of Reference Drop. As shown in Table 6. baselines can not reach satisfactory performance when facing mismatched formats. With the help of the reference drop, the JudgeLM can handle both the format with or without reference and achieve higher agreement and consistency. demonstrates that reference drop can address the format bias and avoid the JudgeI M overfitting to single format.
Ablation of Judging Form We further evaluate the performance of JudgeLM-7B with explanation- first (Chain of Thought, CoT (Wei et al., 2022)) or score-first (Ours) in Table 7. JudgeLM with CoT performs similar agreement with our score-first baseline but with higher consistency, which means that explanation-firs form, i... CoT, can alleviate the position bias of fine-tuned judges, but not bring significant agreement improvement. As a result, we choose the score-firg first method for JudgeI M, which has slightly less consistency but more flexible usage.

---

Published as a conference paper at ICLR 2025
Table 8: Comparison between GPT-4 teacher and JudgeLM-33B on JudgeLM val set.
Agreement (w/ GPT-4)
Consistency (w/ swap.)
Bias L
Bias &
Methods
Delta Bias +
toward 1st
toward 2nd
GPT-4
85.82 91.36
6.10 5.55
8.10
2.00
J--33L
89.03
3.09
2.46
6.4 ADDITIONAL EXPERIMENT
Comparison with GPT-4 Teacher As shown in Table 8, we further list the metrics except for the agreement with GPT- itself. JudgeLM--33B achieves higher consistency than GPT-4, which demonstrates that fine-tuning judges like JudgeLM-33B can achieve higher consistency through the proposed techniques.
The learning paradigm of JudgeLM is similar to knowledge distillation, as JudgeLM learns from the expert judgments provided by GPT-4. In the knowledge distillation domain (Gou et al., 2021), the student model (JudgeLM) mimics the teacher model (GPT-4) to achieve competitive or even superior performance. Additionally. our JudgeLM employs three key methods and utilizes large- scale training data specifically for the judging task to enhance its agreement and consistency. As mentioned in "Comparison on Other Human Evaluation Benchmarks" under Section 6.1, our ex- periments demonstrate that a fine-tuned specialist judge model can surpass its generalist teacher on some judging benchmarks, i.e., JudgeLM val set and PandaLM test set.
6.5 DETAILS OF DATASET
For the proposed dataset and benchmark, we also provide the explanation of usage scope, details of metric calculations, dataset quality, question category & distribution (among 19 categories). and comparison with UltraFeedback (Cui al., 2023) Sec. A.I. We hope the dataset can help re- searchers build more robust evaluation tools in the future.
6.6 GENERALIZATION ABILITY OF JUDGELM
Not only judging answer pairs, but our JudgeL can also generalize to various judging tasks (includ- ing math problems and code generation), unseen judging benchmarks nnuman-annotated benchmark, multimodal judging benchmark, retricval-formal benchmark, multiple-forma benchmark, toxic chat benchmark, reward model benchmark), and other various judging extensions (grading single answer, multi-turn chat). We leave the detailed analysis in Sec. A.3 of the appendix.
6.7
MORE DISCUSSION
Due to the limitation of pages, we leave more discussion in Sec. A.4 the appendix.
CONCLUSION
In this paper, we first introduce a high-quality, large-scale dataset for LLM evaluation, that provides robust foundation for future research. Next, the proposed JudgeLM as scalable judges for eval- uating L.Ms in open-ended tasks efficiently, achieving state-of-the-ar judge performance on two benchmarks. Then, we analyze two key biases and introduce new format bias fine-tuning LLMSs as judges, and address them with the proposed techniques. We hope work motivate more studies to explore the judge models for LLMs in open-ended tasks and build more powerful LLMs with guidance from judge models.
Limitations. Although the proposed JudgeLM achieves encouraging performance and efficiency, the cost of the judge dataset limits further scaling up in the judge dataset. Currently, spend about 4000 dollars to provide 100K high-quality -ee-e--geeetted judge data to the public. We expect to further improve the performance of judge models with the help of synthetic judge data.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS AND DISCLOSURE OF FUNDING
This work was partially supported by the National Natural Science Foundation of China (NSFC) under Grant No. 62276108.
REFERENCES
Cabrera Alex and Neubig Graham. Zeno chatbot report, 2023. URL https:
3
Yuvanesh Anand, Zach Nussbaum, Brandon Duderstadt, Benjamin Schmidt, and Andriy Mulyar. Gpt4all: Training an assistant-style chatbot with large scale data distillation from gpt-3.5-turbo. 2023. 15
https://git tuu..ttub....
Anas Awadalla, Irena Gao, Josh Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Shiori Sagawa, al. Openflamingo: An open- source framework for training large autoregressive vision-language models. arXiv preprint 0rXiv:23088.11399 3
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint 2rXiv:230..16609 2023. 15, 19, 20
Stella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley, Kyle O' O'Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, et al. Pythia: A suite for analyzing large language models across training and scaling. In International Conference on Machine Learning, pp. 2397-2430. PMLR, 2023.
Niels J Blunch. Position bias in multiple-choice questions. Journal of Marketing Research, 21(2): 216-220, 1984. 5
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020. 3
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. Vieuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. See https:/vvicuna Imsys. org (accessed 14 April 2023), 2023. 5,
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, al. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.023111 2022.
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuneed language mod- els. arXiv preprint 2...:12200.11116 2022. 3,
Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. Free dolly: Introducing the world's first truly open instruction-tuned Ilm, 2023. URL https: //op:://wwww.o databri cow.brrriiiccccoooeoooooomaaeaae wtw/ccooooooooooeeeeeoc 12//ooll-first open www.-rcooomecalllcooo ottpra/eeee....oo.oaaaaaaaaaaaae 15
2aolog//22/////// /
tuned- 11m. 4,
Ganqu Cui, Lifan Yuan. Ning Ding, Guanming Yao, Wei Zhu, Yuan Ni, Guotong Xie, Zhiyuan Liu, and Maosong Sun. Utrrafeebback: Boosting language models with high-quality feedback. arXiv preprint arXiv:2310.01377, 2023. 10, 15, 19,22
Jianping Gou, Baosheng Yu, Stephen Maybank, and Dacheng Tao. Knowledge distillation: A survey. International Journal of Computer Vision 17966):789-18899 2021. 10
11

---

Published as a conference paper at ICLR 2025
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprint 00t1:22009003000, 2020. 1
Shima Imani, Liang Du, and Harsh Shrivastava. Mathprompter: Mathematical reasoning using large language models. arXiv preprint arXiv:2303.0553 2023. 16
Hamish Ivison, Yizhong Wang. Valentina Pyatkin, Nathan Lambert, Matthew Peters, Pradeep Dasigi, Joel Jang, David Wadden, Noah A Smith, Iz Beltagy, et al. Camels in changing cli- mate: Enhancing Im adaptation with tulu 2. arXiv preprint arXiv:2311. 2023. 19, 20
Srinivasan Iyer, Xi Victoria Lin, Ramakanth Pasunuru, Todor Mihaylov, Daniel Simig, Ping Yu, Kurt Shuster, Tianlu Wang. Qing Liu, Punit Singh Koura, et al. Opt-iml: Scaling language model instruction meta learning through the lens of generalization. arXiv preprint arXiv:22/2.12017, 2022.
Dongfu Jiang, Yishan Li, Ge Zhang, Wenhao Huang, Bill Yuchen Lin, and Wenhu Chen. Tiger- score: Towards building explainable metric for all text generation tasks. Transactions on Machine Learning Research, 2023. 20, 22
Seungone Kim, Jamin Shin, Yejin Cho, Joel Jang, Shayne Longpre, Hwaran Lee, Sangdoo Yun, Scongjin Shin, Sungdong Kim, James Thome, et al. Prometheus: Inducing fine-grained evalua- tion capability in language models. In The Twelfth International Conference on Learning Repre- sentations, 2023. 19, 22
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 1412,6990 2014. 17
Miyoung Ko, Jinhyuk Lee, Hyunjae Kim, Gangwoo Kim, and Jaewoo Kang. Look at the first sentence: Position bias in question answering. In 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, pp. 1109-1121. Association for Computational Linguistics (ACI.), 2020. 5
Nathan Lambert, Valentina Pyatkin, Jacob Morrison, LJ Miranda, Bill Yuchen Lin, Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, et al. Rewardbench: Evaluating reward models for language modeling. arXiv preprint urXiv:2403. 13787, 2024. 19, 22
Seongyun Lee, Seungone Kim, Sue Hyun Park, Geewook Kim, and Minjoon Seo. Prometheusvi- sion: Vision-language model arXiv:2401.06591, 2024. 18
judge for fine-grained evaluation. arXiv preprint
Junlong Li, Shichao Sun, Weizhe Yuan, Run-Ze Fan, Hai Zhao, and Pengfei Liu, Generative judge for evaluating alignment. CoRR, abs/2310.05470, 2023a. URL bttps:///ol wtw.doo..rrr.ooooooo orrg
10.4 2.0.4855/ea 1055055000ii./ 2310.010020 05470. 3, 6, 22
Zongjie Li, Chaozheng Wang, Pingchuan Ma, Daoyuan Wu, Shuai Wang, Cuiyun Gao, and Yang Liu. Split and merge: Aligning position biases in large language model based evaluators. arXiv preprint arXiv:2310.01432 2023b. 5
Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, et al. Holistic evaluation of language models. arXiv preprint 2rXiv:22/1.091111 2022.
Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pp. 74-81, 2004.
Zi Lin, Zihan Wang, Yongqi Tong, Yangkun Wang, Yuxin Guo, Yujia Wang, and Jingbo Shang. Toxicchat: Unveiling hidden challenges of toxicity detection in real-world user-ai conversation. arXiv preprint ar 2tt.::2311.173333 2023. 18
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. arXiv preprint 2rXiv:2304.084. 2023. 18
Iyya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR, 2019. 17
12

---

Published as a conference paper at ICLR 2025
Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qing- wei Lin, Shifeng Chen, and Dongmei Zhang. Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct. arXiv preprint wrXiv:2308.09588 2023. 16
Jinjie Ni, Yifan Song, Deepanway Ghosal, Bo Li, David Junhao Zhang, Xiang Yue, Fuzhao Xue, Zian Zheng, Kaichen Zhang, Mahir Shah, et al. Mixeval-x: Any-to-any evaluations from real- world data mixtures. arXiv preprint 3xki:24410.135555 2024a. 23
Jinjie Ni, Fuzhao Xue, Xiang Yue, Yuntian Deng, Mahir Shah, Kabir Jain, Graham Neubig, and Yang You. Mixeval: Deriving wisdom of the crowd from Ilm benchmark mixtures. arXiv preprint arXiv:2406.0m 06565, 2024b. 23
OpenAL. Chatgpt, 2022. URL
OpenAl. Gpt-4 technical report, 2023. 3
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35; 27730-27744, 2022. 3,5
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp. 311-318, 2002. 1
Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. Instruction tuning
with gpt-4. arXiv preprint urXiv:2304.03277, 2023, 4, 15
Maja Popović. chrF: character n-gram F-score for automatic MT evaluation. In Proceedings of the
Tenth Workshop on Statistical Machine Translation, pp. 392-395, Lisbon, Portugal, September 2015. Association for Computational Linguistics. doi: 10. 10.18653/11/115-30044 URL https: // //aclanthology.... hwwpchhhoogg..rg/WW...o wtt///1-//00000
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research, 21(1):5485-555) 2020. 3
Priya Raghubir and Ana Valenzuela. Center-of-inattention: Position biases in decision-making. Organizational Behavior and Human Decision Processes, 99( 80, 2006. 5
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. arXiv preprint 102. 2021.
17
Thibault Sellam, Dipanjan Das, and Ankur Parikh. Bleurt: Learning robust metrics for text genera- tion. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 7881-7892, 2020.
Quan Sun, Qiying Yu, Yufeng Cui, Fan Zhang, Xiaosong Zhang, Yueze Wang, Hongcheng Gao, Jingjing Liu, Tiejun Huang, and Xinlong Wang. Generative pretraining in multimodality. arXiv preprint arXiv:2307.0522 2023. 3
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et Llama: Open and efficient foundation language models. arXiv preprint arXhv:2302.13971 2023a. 3,
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov. Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint 200.022307009288.. 2023b. 3, 19, 21
Xuanhui Wang, Nadav Golbandi, Michael Bendersky, Donald Metzler, and Mare Najork. Position bias estimation for unbiased learning to rank in personal search. In Proceedings of the eleventh ACM international conference on web search and data mining, pp. 610-618, 2018.
13

---

Published as a conference paper at ICLR 2025
Yidong Wang, Zhuohao Yu, Zhengran Zeng, Linyi Yang. Cuxxiang Wang, Hao Chen, Chaoya Jiang, Rui Xie, Jindong Wang, Xing Xie, Wei Ye, Shikun Zhang, and Yue Zhang. Pandalm: An au- tomatic evaluation benchmark for Ilm instruction tuning optimization. CoRR, abs/2306.05087, 2023. URL hwtps:tt/////. doii org/ 1 0
2306. D5087. 3, 5, 6, 7, 19, 22
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe Le, Denny Zhou, et al. Chain-of thotttttt prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824-2465 24837, 2022. 9
Wenda Xu, Danqing Wang, Liangming Pan, Zhenqiao Song, Markus Freitag, William Yang Wang. and Lei Li. Instructscore: Explainable text generation evaluation with finegrained feedback. arXiv preprint arXiv:2305. 12.3305..4288 /4282, 2023, 6, 22
Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James Kwok, Zhen- guo Li, Adrian Weller, and Weiyang Liu. Metamath: Bootstrap your own mathematical questions for large language models. arXiv preprint (XXi:2330.12284 2023a. 16
Weihao Yu, Zhengyuan Yang, Linjie Li. Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. Mm-vet: Evaluating large multimodal models for integrated capabilities, 2023b. 18
Weizhe Yuan, Graham Neubig, and Pengfei Liu. Bartscore: Evaluating generated text text gener- ation. Advances in Neural Information Processing Systems, 11-22263-272777 2021. 1
Renrui Zhang, Jiaming Han, Aojun Zhou, Xiangfei Hu, Shilin Yan, Pan Lu, Hongsheng Li, Peng Gao, and Yu Qiao. Llama-adapter: Efficient fine-tuning of language models with zero-init atten- tion. arXiv preprint arXiv:2303. 16199, 2023. 3
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christo- pher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.00063 2022. 1,3
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. Bertscore: Evaluat- ing text generation with bert. arXiv preprint arXiv: 09977, 2019. 1,3
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. arXiv preprint arXiv:2306.0500 2023. 1, 2, 3, 5, 6, 22, 23
A APPENDIX / SUPPLEMENTAL MATERIAL
A.1 MORE ABOUT DATASET
Dataset Usage Scope We emphasize that the JudgeL.M dataset is intended only for academic re- search and any commercial use is prohibited. Because the OpenAl's terms prohibit developing models that compete with OpenAl, the instruction-tuning datasets generated by the OpenAl's API, i.e., Alpaca, PandaLM, etc., all follow this rule.
Details of Metric Calculations For objective metrics, we use the judgments annotated by humans or GPT-4 as ground truth labels, and the judgments generated by judge models as predicted labels. We use TP, FP, TN, and FN to represent the true positive, false positive, true negative, and false negative, respectively. The calculation of agreement, precision, recall, and F1 F1-score are as follows:
Agreement (TP (TP+TN/(TP+FR FP+N FNN,
(1)
Precision = TP/(TP+F) FP),
(2)
Recall (TP//PPFN)
(3)
Fl-score = (2 + (2)PP) TP + FF++F++FN)
(4)
For reliability metrics, we compare the results before and after swapping the order of the two answers (A and B), i.e., from "the first answer is A, the second answer is B" "the first answer is B, the
14

---

Published as a conference paper at ICLR 2025
second answer is A". When the judging results change, we mark it biased sample. Samples with a bias toward the first position consist of three situations, "from Answer A wins to tie", "from Answer A wins to Answer B wins", and "from tie to Answer B wins". Similarly, samples with bias toward the second position also consist of three situations, "from Answer B wins to tie", "from Answer B wins to Answer A wins", and "from tie to Answer A wins". The calculation of metrics of "bias toward 1st", "bias toward 2nd", "delta bias" are defined as follows:
Number of samples with a bias toward the first position
bias toward 1st
(5)
Numbers of total samples
bias toward 2nd Number of samples with a bias toward the second position
(6)
Numbers of total samples
delta bias bias toward 1st bias toward 2nd)].
(7)
Dataset Quality To ensure the high quality of the proposed dataset, we filter the low-quality data samples in each step. For data samples (including seed tasks and references) in four public datasets, i.e., Alpaca-GPPTT GPP4 (Peng et al., 2023), Dolly-15K (Conover et al., 2023), GPT4AII-LAION (Anand et al., 2023), and ShareGPT, we first remove data samples containing obviously incorrect, irrelevant, or harmful reference answers through automated filtering scripts. Next, we randomly sample 105K samples from the filtered set and extract answers from 11 LLMs. Then, we input the tasks, ran- domly sampled answer pairs, and optionally reference answers to the GPT-4 teacher for judgment. Finally, the authors of this work are involved in a multi-step validation process to ensure the qual- ity, accuracy, and reliability of the judge samples. This process includes initial annotation step where GPT-4 provides preliminary judgments, followed by independent human re- annotatioon where authors provide simple judgments ("Answer 1 wins." " "Answer 2 wins", or "Tie") without expo- sure to GPT-4's annotations. The final step involves cross cross-validation and refinement, where human judgments are compared with GPT-4 annotations to thoroughly verify the judge results, scores, and reasoning quality. Please note that incorporating high-quality answers from closed-sourced models e.g., Qwen (Bai al., 2023) and Claude, could enhance the diversity of the dataset. We leave it as a future work.
Question Category & Distribution of Validation Set We count the distribution of questions in the JudgeLM val set as shown in Table 9. Please note that the question categories included in the JudgeLM val set and Irain set are the same, but none of the data samples are identical.
Comparison with UltraFeedback Furthermore, we compare the JudgeI.M dataset with UltraFeed- back (Cui et al., 2023), which an excellent dataset serving as solid foundation for feedback- learning research. JudgeLM has nearly half more seeds than UltraFeedback, and additional vali- dation set containing 5K seeds. JudgeLM and UltraFeedback both provide scalar and text feedback, GPT-4 annotation, and fine- -raiind consideration. However, the JudgeLM dataset is further checked and re-annotated by humans, which provides double-checking on the quality of feedback. Moreover, JudgeLM supports judging with references, which can make up for the lack of pre-trained knowl- edge or insert specific judge preferences. Finally, JudgeLM clearly splits the seeds into 19 categories providing intuitive ability estimation for judges.
train Seeds
val Seeds
Feedback Format
fine grained?
with Ref.?
Seeds Categories
Dataset
Annotator
UltraFeeddback
64K 0
Scalar & Text
GPT-4
Y
N
JudgeLM
100K
5K
Scalar & Text
GPT-4 & Human
Y
Y
19
Comparison with PandaLM Test Set Furthermore, we compare the JudgeLM dataset with the PandaLM test set. An analysis of task distributions in Table 10 shows significant differences between the PandaLM test set and the JudgeLM benchmark. For example, business, fact-QA, summarizing, explain, retrieval, document, and chat are well-represented
linguistics, emotion, in PandaLM but absent in JudgeLM, while writing and roleplay show significant delta percentage (over 4%). This confirms that the PandaLM test set includes 49% unseen task samples that are out of distribution for M.
15

---

Published as a conference paper at ICLR 2025
Table Distribution of question categories in JudgeLM val
count
percentage
count
percentage
culture
233
4.66%
planning
309
6.18%
recommendation
482
9.64%
roleplay
77
1.54%
finance
142
2.84%
coding
201
4.02%
science
393
7.86%
health
278
5.56%
technique
42
0.84%
writing
625
12.50%
common-sense
373
7.46%
hardware
130
2.60%
art
335
6.70%
history
243
4.86%
math
250
5.00%
geography
199
3.98%
private-matter
421
8.42%
others
63
1.26%
law
204
4.08%
total
5000
100.00%
Table 10: Distribution of question categories in PandaLM test set. The A represents the percentage difference compared to the JudgeI.M benchmark. We bolded categories that appear in the M test set but don't tt exist in the JudgeLMM benchmark.
count
percentage
A
count
percentage
business
87
8.71%
8.71%
writing
81
8.11%
-4.39
fact-QA
70
7.01% %
7.01%
planning
5.71%
-0.47%
summarizing
64
6.41%
6.41%
roleplay
57
5.71%
4.17%
linguistics
45
4.50%
4.50%
coding
54
5.41%
1.39%
emotion
45
4.50%
4.50%
44
4.40%
-2.30%
antty-ppocessing
42
4.20%
4.20%
finance
40
4.00%
1.16%
explain
41
4.10%
4.10%
culture
38
3.80%
-0.86%
retrieval
40
4.00%
4.00%
math
37
3.70%
-1.30%
document
30
3.00%
3.00%
geography
12
1.20%
-2.88%
chat
26
2.60%
2.60%
others
0.50%
-0.76%
recommendation
84
8.41%
-1.23%
total
999
100.00%
A.2 FINE-TUNING SETTING
We list the hyper-parameters used, shown in Table 11.
A.3 GENERALIZATION ABILITY OF JUDGELM
To validate the generalization ability of JudgeLM, we test JudgeLM on various judging tasks (in- cluding math problems and code generation). unseen judging benchmarks aunaan-n bench- mark, multimodal judging benchmark, retrieval- benchmark, multiple-format benchmark), and other various judging extensions (grading single answer, multi-turn chat).
Generalize to Various Judging Tasks. To further validate the judging performance questions with specific categories, we present the judging results of Judgel.M-33B on these questions, i.e., coding, common-sense, math, roleplay, and writing. Table 12 shows that JudgeLM can handle the judging tasks of various categories. We also find that the judging performance of math questions is slightly lower than coding and common-sense questions. However think it is common problem of large language models (Imani et al., 2023), and future advancement on base models (Yu et al., 2023a; Luo et al., 2023) would alleviate this problem.
Generalize to Multimodal Judging Benchmark. Traditional multimodal evaluation needs predic- tion match the ground truth exactly. For some open-ended questions, human-like evaluator is
16

---

Published as a conference paper at ICLR 2025
Table 11: JudgeLM fine-tuning setting.
config
JudgeLM/ / /7B - -33B
base model
Vicuna -7B -7B/-7B/333 /-13B -33B
model max length
2048
fine-tuning data source
JudgeLMM100KK
learning rate learning rate schedule
2e-5
cosine decay
optimizer
AdamW (Kingma & Ba, 2014; Loshchilov & Hutter, 2019)
optimizer hyper-parameters
B1. 82, 0.9, 0.999, 1e-8
weight decay
0.0
GPU nums
8 8 16 128
batch size
training epochs
3 0.003 bf16, tf32 stage 3 True 2023-03-15-prevve
warmup ratio
numerical precision
ZeRO optimizer (Ramesh et al., 2021)
gradient checkpointing
GPT-3.5 and GPT-4 version
Table 12: Performance of JudgeLM-333 with specific categories on JudgeLM val set.
Agreement t (w/ GPT-4)
Consistency t (w/ swap.)
Bias $ toward 1st
Bias toward 2nd
Delta Bias
coding
w/o ref val w/ ref
88.08
88.60 91.37
6.22 3.55
5.18
1.04
88.83
5.08
1.53
common-sense
val w/o ref val w/ ref
88.41 90.37
90.43
7.25
2.32
4.93
92.35
4.53
3.12
1.41
math
val w/o ref
86.45
84.08
7.76
8.16
0.40
w/ ref
86.81
87.85
4.45
7.69
3.24
croleplay
val w/o ref val w/ ref
88.00
88.00
6.67
5.33
1.34
88.72
89.12
5.01
5.87
0.86
writting
w/o ref val w/ ref
87.33
92.36
3.25
4.39
1.14
89.11
92.83
3.67
3.50
0.17
17

---

Published as a conference paper at ICLR 2025
needed to determine whether the prediction is close to the ground truth range. Modern multimodal, such as MM- Vet (Yu et al., 2023b) and Prometheus- Vision (Lee et al., 2024), which use GPT-4V, GPT-4 GPT-3.5 as judges. The API-based judge takes the question text, ground-truth text, the model's prediction, and optional input image as input, and makes judgments based on them. Our JudgeL. also provides good practice for such a multimodal evaluation by slightly modified tem- plate as shown in Fig. 7. Thanks to its capacity to judge open-ended answers, our JudgeLM can also perform well in judging multimodal models, as shown in Fig. 21.
We further conduct experiments to evaluate JudgeLM ability to judge multimodal models when compared with close-sourcedd LLM, i.e. GPT-3.5 and GPT-4. We first use GPT-4, GPT-3.5, and JudgeLM to judge the LLaVA's output (Liu et al., 2023), respectively. Then, we collect judgments from human annotators, whose judgmeets include three situations: completely correct, semi-correct, and completely wrong. Last, we compute the metrics between the LLM judges' judgments and human judgments, as shown in Table 13. can be observed that Judgel.M outperforms GPT-4 (O-shot) and GPT-3.5 (7-shot). Besides, JudgeLM achieves 2.5% higher precision than GPT-4 (7- shot). The encouraging results demonstrate the generalization ability of JudgeLM in dealing with multimodal judging. Furthermore, JudgeLM can use large multimodal models, e.g., LLaVA (Liu et al., 2023), as the backbone for better processing the multimodal judging. We leave it as a future work.
Table 13: JudgeLM zero-shot evaluation results on human- annotted MM-Vet Vet benchmark.
Agreement t (w/ Human)
Precision t (w/ Human)
Recall
Fl FI (w/ Human)
Methods
(w/ Human)
GPT-4 (7-shot)
95.58
88.63
87.79
88.04
GPT-4 (0-shot)
86.70
79.75
86.41
81.81
GPT-3.5 (7-shot)
83.03
76.14
74.84
73.62
JudgeL.M-33B (0-shot)
91.74
91.08
85.58
87.26
Generalize to Distri-dist ToxicChat Benchmark. To further evaluate the generaliza- tion ability of the proposed JudgeL.M, we selected an out-of-distribution benchmark, i.e., Toxic- Chat (Lin et al., 2023), for evaluation. Following the guidelines provided in the ToxicChat dataset, we conducted experiments on the latest test set (0124) of ToxicChat, comparing OpenAl Modera- tion, the GPT-4 teacher, and our proposed JudgeLM. OpenAl Moderation is API trained on pub- licly available toxicity datasets, primarily sourced from social media. For both the GPT-4 teacher and JudgeLM, used the same templates and thresholds. As shown in Table 14, proposed JudgeLM achieves superior precision and comparable accuracy to the specialist model, i.e., OpenAl Moderation. These results demonstrate that JudgeLM can further generalize out-of-distribution datasets such as ToxicChat.
Table 14: JudgeLM zero-shot evaluation results on toxic-chat test set.
Methods
Accuracy?
Precision
Recallf
F1f
Specialist API-based Method
OpenAl Moderation
89.70
54.76
69.89
61.41
Generalist API-based Method GPT-4
88.08
52.17
73.20
60.92
Open-sourced Method M-333
89.66
58.79
61.88
60.30
Generalize to Retrieval-format Benchmark. In real-world applications, we do not always have well-organized reference answers for judging. To evaluate the capability of JudgeLM dealing with this situation, we inject the original reference answers into a randomly selected paragraph. As shown in Table 15, we select paragraphs with different words, and evaluate JudgeLMM333B with the injected paragraphs references in zero-shot setting. The results show that JudgeLM can retrieve the correct answers from the paragraphs and make judgments based on them. When the words of paragraphs increase 400, Judgel.M faces maximum drop of 3.73% agreement and 3.37%
18

---

Published as a conference paper at ICLR 2025
consistency. The results demonstrate that JudgeLM is promising for utilizing JudgeLM deal with unstructured references or judge in the retrieved form.
Table 15: Performance of JudgeLM-3333 with injected paragraphs as references on JudgeLM val set.
Reference Paragraph
Agreement f (w/ GPT-4)
Consistency f (w/ swap.)
Bias J
Bias J toward 2nd
Delta Bias J
toward 1st
No
89.32
92.37
3.62
4.01
0.39
50 words
87.78
92.35
3.00
4.65
1,65
100 words
87.69
91.84
2.62
5.54
2.92
200 words
86.77
90.48
2.70
6.82
4.12
300 words
86.25
89.26
3.13
7.61
4.48
400 words
85,59
89.00
2.98
8.02
5.04
Generalize to Multiple-format benchmark. To get the optimal ranking for N answers from dif- ferent LLMs, other judge models need to call the model O(n²) times to get the full matrix, which is a much less efficient solution. We attempt to resolve this limitation by extending our JudgeLM to process multiple answers at the same time. We first need modify the template shown in Fig. 5. As shown in Fig. 18, JudgeLM can judge and rank the multiple answers within the context limit of LLM.
We further conduct experiments to evaluate the consistency in the judging form of answer pairs and multiple answers. We first generate answers on JudgeL.M vol set through LLMs, i.e., Vicuna-13BB LLaMA-7B, and alpaca-7B, for evaluation. Then we use pairwise judging and multiple judging to grade answers and rank them, respectively. Last, we compute the consistency between the two ranking results. Please note that 'Error Rate@2 indicates the position orderings of two answers are different between the result of paired judgment and the result of multiple judgment, and 'Error Rate@3 means the position orderings of three answers are different. Table 16 shows that consis- tency between judging pairwise and judging multiple can reach 93.48%, and only 0. 0.14% results totally wrong. The results impressive but the 6.38% of 'Error Rate@2' also shows room for improvement as well, which could be addressed by the further improvement of JudgeLM'ss self- consistency.
Table 16: Performance of M-3 in judging multiple answers on JudgeLM val set. We calculate the consistency between the pairwise judging results and multiple judging ones.
Consistency t (w/ pairwise)
Error Rate@2 4 (w/ pairwise)
Error Rate 4
(w/ pairwise)
Ju333I.M-33B multipile
93.48
6.38
0.14
Generalize to Single Answer Grading. The Concurrent judge method (Wang et al., 2023) usually judges pair of answers decide which one is better tie but they lack the ability to evaluate single answer. Thanks to our judging mode of scoring first and then calculating the judging results, our JudgeLM provides an alternative practice grade a single answer by slightly modifying the template as shown in Fig. 5. Putting the reference answer in the first position and giving it a full grade as prior, JudgeLM can give quantitative fine- mee-grained evaluations as shown in Fig. 17.
The capability of grading single answer is an important extension, which only relies the text-form prediction and ground truth make judgments. For example, the following extension "Judging multimodal models" is also based on this capability.
Generalize to Reward Model. We further compare the proposed JudgeL. with closed-source methods, powerful reward models, Llama-2-based reward methods (Touvron et al., 2023b; Ivison et al., 2023), and 1.5e--l.--baed reward methods (Bai et al., 2023) on the requested reward model benchmark as shown in Table 17. Following the evaluation manner of GPT-3.5-turbo-O0) Prometheus series (Kim et al., 2023) in RewardBench (Lambert et al., 2024), we evaluate the models among 4 subsets, i.e., Chat, (Chat) Hard, Safety, and Reasoning, and present averaged score the Score column. The proposed JudgeLM-7B outperforms the HPPT-3.-tturb--113 the Prometheus series reward models (Kim et al., 2023), UltrRMM-13B (Cui et al., 2023). TiGERScore-
and
19

---

Published as a conference paper at ICLR 2025
Table 17: Comparison with advanced reward models on RewardBench.
Model
Score
Chat
Hard
Safery
Reason
Closed-source Models
SP9-3.5-turbo-01255
64.5
92.2
44.5
62.3
59.1
Powerful Reward Models.
Prometheus-8 x XB-vv2.
75.3
93
47.1
83.5
77.4
Prometheus-7B-v2.00
72.5
85.5
49.1
78.7
76.5
UlraRMM-13B 13B TIGERScore-
67.6
96.4
55.5
56.0
62.4
35.6
35.2
32.9
41.5
32.7
Llama-2-based Models.
Tulu-2-dpor Tulu-2-dpo-70B
79.0
97.5
60.5
83.9
74.1
Tulu-2-dpo-13B Tulu- Tulu-2-dpo-7B
76.4
95.8
58.3
78.2
73.2
74.7
97.5
56.1
73.3
71.8
Qwenl.5-based Models.
Qwen1 wwenl.5-72B-- 5--72B-hha
71.5
62.3
66.0
72.0
85.5
owen1.5-14B. Qwenl.5-7B-Chat
73.4
57.3
70.2
76.3
89.6
72.0
53.6
69.1
74.8
90.4
Ours. JudgeLM-7B
78.5
92.2
56.1
83.2
82.3
13B (Jiang et al., 2023). Tulu-2-dpo-13B (Ivison et al., 2023), Tulu-2-dpo-7B (Ivison et al., 2023), and Sweel.5-bbsed reward models (Bai al., 2023). Besides. JudgeLM-7B even achieves similar performance to Tulu-2-dpo-700 (Ivison et al., 2023). Noting the RewardBench paper mentions that involving Llama-3 the base model can significantly improve the metries on Hard and Reasoning, we leave it as future work.
Multi-turn Chat about Judgments. is worth noting that fine-tuning with judge samples does not compromise the multi-turn chat ability extended from base models. As illustrated in Fig. 19 and Fig. 20, our JudgeL. retains the capability to engage in meaningful dialogues with users, providing them with richer context, detailed information, additional examples, and specific details.
A.4
MORE DISCUSSION
Table 18: Comparison of different base models for JudgeLM-7B on JudgeLM val set.
Base Models (for JudgeL
Agreement t (w/ GPT-4)
Precision
Recall f (w/ GPT-4)
Fl
Consistency ww/...
(w/ GPT-4)
(w/ GPT-4)
Judge w/o reference.
Vicuna
81.11 83.87
69.67 73.43
78.39
72.21
83.57
LLaMA2-chat
80.06
75.91
85.17
Judge w/ reference.
Vicuna
84.08
75.92
82.55
78.28
84.46
LLaMA2-chat 86.60
79.47
83.11
81.02
87.74
Format Bias As shown in Table 16, can be seen that the judging of multiple answers does not receive significant performance drop. We hold the viewpoint that judging multiple answers is an easy extension for JudgeLM, which does not change the basis for judging. As mentioned '4 Inherent Biases Format Bias", format bias means the model judging basis changes from pre-trained knowledge to reference, or vice versa. So, judging in mismatched situations faces format bias but judging multiple answers does not.
Other Human-annotated benchmarks For a fair comparison, we also evaluate JudgeLM on the PandaL.M M test set in zero-shot setting. The Pandal train and test sets annotated by GPT3.5
20

---

Published as a conference paper at ICLR 2025
and humans, respectively. As shown in Table 2, the zero-shot results of JudgeLM also outperform other judging methods, i.., PandaLM, GPT-3.5, and GPT-4. Furthermore, JudgeLM also achieves superior O-shot judging performance on the multimodal benchmark with human annotation, i.e., MM-Vet, as shown in Table 13.
Reasoning Ability of LLM Judges Nowadays, NLP researchers still struggling with proposing
LLMs with superior reasoning abilities. JudgeLM also needs the proposed reference sup method enhance the judging ability for out-of-domain Fig. 16. Notably, the proposed JudgeLM can benefit from stronger foundation LLMs, e.g., the LLaMA2-7B-CHS 1e--7B-hhtt-asse based (Touvron et al., 2023b) JudgeLM outperforms the original JudgeLM-7B on all metrics, as shown in Table 18. The research of judge models is critical for the development of L.MSs and can benefit from advanced L.MMs, establishing a positive cycle.
counterfactual tasks, as shown in Fig. 11 and
Critiques for Judgements Beyond using LLMs to compare answer pairs and generate reasons, the critique and correction of these reasons have become increasingly important topics. This approach allows LLMs to reassess their generated reasons in multiple rounds, thereby enhancing judging accuracy. Works like UltraCM, Auto-J, and Shepherd have reliably evaluated the quality textual reasons. Recently, CriticBench has also provided a reliable benchmark to evaluate the reasons and evaluation abilities of LLMs.
For Ju, its ability to generalize to multi-tum chat enables to construct multi-turn judgement critique data by combining data samples without references and those with references. Through two rounds of judge Q&A, i.e., without reference and with reference, JudgeLM can acquire the capability to critique its own judgments. We leave these experiments for future work.
Independent Method We think the reference drop is independent and sig- nificant method. At first, we argue that judging with or without references are two uub-benchmarks, which require judges make judgments with internal knowledge by comparing I.L.-generate answers with a reference answer, respectively. The reference drop is not only simple but effective hyper-parameter, but also an important method that bridges the two nub-benchmarks, which enables the JudgeLM to make judgments in different situations.
Reference Drop
Table 19: Comparison between PandaLM and JudgeLM components in terms of datasets and meth- ods.
Base Model
Data
Method
val w/ ref?
Agreement t
Consistency t
PandaLM baseline.
LLaMA
+00K PandaLM data
N
68.61
74.78
change to JudgeLM data.
LLaMA
+3.5k JudgeLM data
71.30
69.59
w/ the proposed methods.
LLaMA
+3.5k JudgeLM data
+swap aug +ref sup +ref drop
N
72.41
73.50
LLaMA
+3.5k JudgeLM data
Y
75.15
74.08
LLaMA
+3.5k JudgeLM data
75.93
74.77
w/ all proposed methods.
LLaMA
+3.5k JudgeLM data
+ all + all
N
75.01
76.28
LLaMA
+3.5k JudgeLM data
78.10
79.50
Differences between MandaLM and JudgeLM To fairly assess the impact of our proposed dataset and methods, we conducted experiments using the PandaLM baseline with LLaMA-7B as the base model. As shown in Table 19, our dataset and methods both provide significant improvements.
Compared with Ma our method has these different novelties:
We introduce high-quality, large-scale dataset for judge models, enriched with diverse seed tasks, LLMs--ennratee answers, and detailed judgments from GPT-4, laying the foun- dation for future L.LMs evaluating research.
21

---

Published as a conference paper at ICLR 2025
We analyze the biases inherent to LLM judge fine-tuning and introduce series of meth- ods to address them. Our methods significantly improve the consistency of the model in different cases, making the JudgeLM more reliable and flexible.
The proposed judging pattern, i.e., grading, judging, and reasoning, makes judging efficient, which only needs little time to grade and judge, and generates time-consuming reasons optionally.
Furthermore, other contributions which differ from PandaLM follows:
We analyze the finer scaling ability of the language model judge, and the scales of training data, i.e., JudgeLM--7B. JudgeL.M- 13B, JudgeL 3.5k-data, 10k-data, 30k-data, and 100k-data, for evaluating LLMs in open-ended scenarios. The B--eeLM--3---0000 even exceeds the humans' judging agreement.
The proposed JudgeLM presents extended capabilities as shown in Fig. 1b, including grading single answers, judging multiple answers, judging multimodal models, multi-turn chat, and judging with injected paragraphs.
Specifically, the proposed Judgel.M presents generalization ability to seven different judging areas:
Generalize to 19 various judging tasks, including coding, common-senss math, roleplay, writing, etc.
Generalize to human-annotated benchmarks, such as the PandaLM and MM-Vet benchmark.
Generalize to multimodal judging benchmark, such as MM- Vet benchmark.
Generalize to retrieval-format benchmark which injects original reference answers into paragraphs with different words.
Generalize to multiple-format benchmark which judges multiple answers at the same time.
Generalize to single-answer grading.
Generalize to multi-turn chat about judgments.
GPT-4 Distilled Data Evaluating Large Language Models (LLMs) in open-ended scenarios is chal- lenging because existing benchmarks and metrics can not measure them comprehensively. Because the GPT-4-based judge can judge LLMs like human expert, much concurrent work delving into curating training data (Li et al., 2023a; Kim et al., 2023; Cui et al., 2023; Xu et al., 2023; Jiang et al., 2023; Wang et al., 2023) and benchmarks (Lambert al., 2024; Zheng al., 2023) with the help of the closed-source LLMs, such as GPT-4 and GPT-3.5.
However, the GPT-4 teacher also faces inherent biases. Firstly, GPT-4's training data may contain cultural, societal, and linguistic biases. These biases may influence the GPT-4's judgment and lead to skewed evaluations. To mitigate these possible biases, the authors of this work are involved in double-checking to ensure the judgments from the GPT-4 teacher are accurate, objective, and unbiased. Secondly, the GPT-4's judgments also contain position bias and knowledge bias (Zheng et al., 2023). To address these problems, we introduce swap augmentation and reference support in fine-tuning LLMs as judges, which significantly improves the consistency and accuracy fine-tuned judges.
Does JudgeLM favor GPT-4? Thank you for discussing this interesting phenomenon. We carefully conducted experiments between GPT-4 and GPT-3.5 to analyze the potential favoring of GPT-4 answers by Ju As shown in Table 20, GPT-4 wins 59.0% of pairwise comparisons, with only 16.2% of wins for GPT-3.5. We further conducted sampling analysis to identify the following contributing factors:
Alignment with Prompt Requirements: As shown in the judging templates (Fig. 3, Fig. 4, Fig. 5, Fig. 6, and Fig. 7), JudgeLM evaluates responses based structured criteria, such as relevance and level of detail. Our analysis shows that GPT-4 answers tend to align better with these requirements compared GPT-3.5 answers.
22

---

Published as a conference paper at ICLR 2025
Inherent Quality Difference: A sampling analysis of 500 cases (approximately 16.9% of the GPT-4 win cases) showed that in 482 instances (96.4%), GPT-4 answers were more accurate, detailed, and contextually relevant than GPT-3.5 answers. This analysis under- scores that GPT-4 win cases are overwhelmingly attributed to the higher answer quality, rather than the preference from JudgeI.M.
Moreover, the sample analysis shows that when the quality of the answers to GPT-4 and GPT-3.5 are similar, JudgeLM tends to give similar scores, resulting in Tie. These results indicate that GPT-4 answers win more due their better relevance, high level of detail, and high quality.
Table 20: Quantitative comparison results between GPT-4 and GPT-3.5.
GPT4 win
Tie
GPT3.5 win
GPT4 v.s. GPT-3.5
59.00%
24.80%
16.20%
Reliability of GPT-4-annotated Data. We conduct more experiments to further evaluate the relia- bility of GPT-4's annotation. In this experiment setting, set annotations provided by Human in our benehmark as ground truth. Then we calculate the agreement of JudgeLM-333. GPT-4, and Hu- man shown in Table 21. These results demonstrate that GPT-4 achieves higher agreement with the ground truth compared to Human 2. Ilm-as-a-judge (Zheng et al., 2023) shows similar results, which means GPT-4 teacher's judgments can align with human evaluators closely and can serve as human-like teacher judge provide high-quality and reliable judgments.
Table 21: Agreement evaluation of JudgeLM-33B, GPT-4, and Human, with another Human's 's judg- ments as ground truth.
JudgeLM-33B
GPT-4
Human 2
Ground Truth (Human 1)
90.72
84.48
79.82
Future Work about Training Queries Distribution. The distribution of training queries a key factor in shaping the evaluation capabilities of Judge LLMs. While our current dataset was designed ensure task diversity, MixEval (Ni al., 2024b) and MixEval-X (Ni et al., 2024a) give us more insight, i.c., aligning the query distribution more closely with real-world user queries could further enhance the model's 8 fairness and relevance. We leave it as a promising future work.
A.5 PROMPT TEMPLATES
We list all the prompt templates used.
A.6 CASE STUDIES
We list several case studies.
23

---

Published as a conference paper at ICLR 2025
are a helpful and precise assistant checking the quality the answer. [Question]
{question
[The Start of Assistant I's Answer]
fanswer 13 [The End of Assistant I's Answer]
[The Start of Assistant 2's Answer]
(answer
[The End Assistant 2's Answer]
[System]
We would like to request your feedback on the performance of two Al assistants in response the user question displayed above. Please rate the helpfulness, relevance, accuracy, level of details of their responses. Each assistant receives an overall score on scale of to 10, where higher score indicates hetter overall performance.
Please first output single line containing only values indicating the scores Assistant 2, respectively. The two scores are separated by space, the subsequent line, please provide comprehensive explanation of your evaluation, avoiding any potential bias and ensuring that the order in which the responses were presented does not affect your judgment.
Figure 3: The template for judging answers without the reference.
You are helpful and precise ussistant for checking quality of answer. [Question]
{question]
[Reference Answer] [reference]
[The Start of Assistant I's Answer] (answer
[The End of Assistaatt Answer]
[The Start of Assistant 2's Answer]
[answer_] 2] [The End of A'ssssttatt Answer]
[System]
We would like to request your feedback on the performance of two Al assistants in response to the question displayed above.
Based on the reference answer, please rate the helpfulness, relevance, accuracy, level details of their responses. Each assistant receives an overall score on a scale of to 10, where a higher score indicates better overall performance.
Please first output single line containing only values indicating the scores for Assistant 1 and 2, respectively. The two scores separated by space. comprehensive explanation of your evaluation, avoiding any potential hias and ensuring that the order in which the responses were presented does not affect your judgment.
subsequent line, please provide
Figure 4: The template for judging answers with the reference.
24

---

Published as a conference paper at ICLR 2025
are a helpful and precise assistant checking the quality of answer. [Question] (question)
[The Start of Assistant I's Answer] [reference]
[The End of Assistant I's Answer]
[The Start of Assistant 2's Answer]
(single answer)
[The End of Assistant 2's Answer]
[System]
We would like request your feedback on the performance of two Al assistants in response the user question displayed above.
Please rate the helpfulness, relevance, accuracy, level of details of their responses. Fach assistant receives overall score scale of 1 to 10, where higher score indicates better overall performance.
Please first output single line containing only two values indicating the scores for Assistant 1 and 2, respectively. The two scores are separated by space. In the subsequent line, please provide comprehensive explanation of your evaluation, avoiding any potential bias and ensuring that the order in which the responses were presented does not affect your judgment.
|eesponse]
Figure 5:
The template for grading single answer. We set the reference answer the position of Answer 1. Then, we set the score of the reference answer to 10. Last, the JudgeLM outputs the score the single answer with such prior.
You helpful and precise assistant for checking the quality of the answer. [Question]
{question)
[The Start Assistant I's Answer]
fanswer 1} [The End of Assistant I's Answer]
[The Start of Assistant 2's Answer]
{answer_2) 2) [The End of Assistant 2's Answer]
The Start of Assistant 3's Answer
[answer 3)
[The End Assistant Answer]
[System]
We would like request your feedback the performance of three Al assistants in response the user question displayed above.
Please rate the helpfulness, relevance, accuracy, level of details of their responses. Each assistant scale of I to 10, where higher score indicates better overall
receives an overall score performance.
Please first output a single line containing only three values indicating the scores for Assistant 1. and 3, respectively. The three scores separated space. In the subsequent line, please provide comprehensive explanation of your evaluation, avoiding any potential bias and ensuring that the order in which the responses were presented does not affect your judgment.
Figure 6: The template for judging multiple answers.
25

---

Published as a conference paper at ICLR 2025
Pleuse evaluate the following visual-based question-answerr pair:
[Question] {question)
[Reference Answer]
[reference]
[The Start of Assistant I's Answer] [answer 1] [The End of Assistant Answer]
[The Start of Assistant 2's Answer]
fanswer 2}
[The End of Assistant Answer]
[System]
Your task compare the predicted answer with the correct answer and determine they match meaningfully. To accomplish the task, you must :
1. Focus on the meaningful match between the predicted answer and the correct answer.
Consider synonyms or paraphrases valid matches.
Evaluate the correctness of the prediction compared to the answer.
4. If there are multiple correct answers, the predicted answer is considered correct long as is close any of the
Figure 7: The template for multimodal judging.
26

---

Published as a conference paper at ICLR 2025
Input
Question
Categorize a list of flowers.
Lilac, Daisy, Marigold, Chrysanthemum
Answer
Lilac: Shrub
Daisy: Herbacoous Perennial
Marigold: Annual
Chrysanthemum: Perennial
Answer 2
Flower:
Lillcc
Daisy
Marigold
Chrysanthemum
Our Judgement Output (where answer is placed in the first position)
Answer I's Score: 8
Answer 2's Score:
Assistant provided more detailed accurate response, correctly categorizing flowers shrubs, herbaceous perennials, annuals, and perennials. The assistant also provided a brief explanation of what these categories mean, which adds to the helpfulness and relevance of the response. However, the assistant could have provided more information about the characteristies these flowers, such as their size, growth habit, and blooming time. the other hand, Assistant did not provide any information about the flowers' categories. The ussistant simply listed the flowers without any additional information or context. While this response is technically correct, it lacks the detail and context that would make it more helpful and informative.
Our Judgement Output (where answer 2 is placed in the first position)
Answer 2's Score: 8
Answer I's Score:
Assistant 2's response concise and accurate, correctly categorizing the flowers Lilac, Daisy, Marigold, and Chrysanthemum. However,
assistant could provided more detail explanation about the characteristics of each flower, which would have made the response more Assistant I's response is also accurate and relevant, but lacks detail and explanation. The assistant categorizes the flowers Lilac (Shrub), Daisy (Herbaceous Perennial), Marigold (Annual), and Chrysanthemum (Perennial). However, these categories are commonly used the horticultural industry, and the assistant does not provide any explanation or context for these categories. This makes the response less helpful and informative.
Figure 8: Bad judgment caused by position bias. The answer placed in the first position always gets a higher score. The judge models generate reasons as possible from the perspective making the scores reasonable.
27

---

Published as a conference paper at ICLR 2025
Input
Question
Categorize a list flowers.
Lilac, Daisy, Marigold, Chrysanthemum
Answer
Lilac: Shrub
Daisy: Herhaceous Perennial
Marigold: Annual
Chrysanthemun: Perennial
Answer 2
Flower:
-Lilac
Daisy
Marigold
Chrysanthemune
Our Judgement Output (where answer is placed in the first position)
Answer I's Score: 8
Answer 2's Score: 2
helpful and accurate provides categorization for each flower listed the question. The assistant correctly identifies lilac as shruh, daisy as herbaccous perennial, marigold as an annual, and chrysanthemum as perennial. This response is relevant to the question and provides good level of detail, hence the score of 8.0n the other hand, Assistant 2's response not helpful accurate. merely repeats the list of flowers without providing any categorization additional information. The response not relevant to the question and lacks detail, hence the low score 2f
Assistant I's response
Our Judgement Output (where
2 is placed in the first position)
Answer 2's Score: 2
Answer Score:
Assistant I's response is more helpful and relevant to the question. The user asked to categorize a list of flowers, and Assistant 2 correctly categorized each flower by its life cycle (annual, perennial, etc.). This information accurate and detailed, and it directly answers the user's question. Therefore, Assistant receives not as helpful or relevant. simply listed the flowers without providing any categorization or additional information, This response does not answer the user's question and lacks detail, so Assistant receives score of
of 8.On other hand, Assistant response
Figure 9: Good judgment generated the judge fine-tuned with swap augmentation. The judge can give judgments based on the content of answers rather than a certain position. The reason is convincing and reasonable.
28

---

Published as a conference paper at ICLR 2025
Input
Question
Determine which companies are primarily cloud-software companies and which are semiconductor companies: Intel, Google, Workday, Zscaler, AMD, NVIDIA, and Texas Instruments.
Answer
Intel: semiconductor
Google: semiconductor
Workday: cloud-software
Intel and AMD are primarily cloud-software companies, while NVIDIA and Texas Instruments are semiconductor companies.
AMD: semiconductor
NVIDIA: semiconductor
Texas Instruments: semiconductor
Answer 2
Intel and AMD primarily cloud-software companies, while NVIDIA and Texus Instruments semiconductor compunies.
Our Judgement Output
Answer I's Score: 2
Answer 2's Score: 8
Assistant I's answer incorrect and confusing. Intel and AMD companies. and NVIDIA information provided is not accurate or relevant the question asked. On the other hand, Assistant 2's answer is more accurate and relevant. Intel and AMD are indeed primarily cloud- software companies, and NVIDIA and Texas Instruments semiconductor companies. However, answer could have been more detailed, explaining why these companies are classified as such.
primarily cloud-software Instruments are not semiconductor companies.
Figure 10: Bad judgment caused by knowledge bias. This seed task out the judge model's pre-trained knowledge, not be correctly judged by the judge model. The judge model gives contradictory reasons in the judgment.
29

---

Published as a conference paper at ICLR 2025
Input
Question
Determine which companies primarily cloud-software companies and which semiconductor companies: Intel, Google, Workday, Zscaler, AMD, NVIDIA, Texas Instruments.
Reference Answer
Intel. AMD, NVIDIA, and Texas Instruments are primarily semiconductor companies. Workday and Zscaler fully cloud-software companies. Google primarily software company, however, they also develop custom-developed application-specffi integrated circuits which are semiconductors."
Answer
Intel: semiconductor
Google: semiconductor
Workday: cloud-software
Intel and AMD are primarily cloud-software companics, while NVIDIA and Texas Instruments are semiconductor companies.
AMD: semiconductor
NVIDIA: semiconductor
Texas Instruments; semiconductor
Answer 2
Intel and AMD primarily cloud-software companies, while NVIDIA and Texas Instruments semiconductor companies.
Our Judgement Output
Answer I's Score: 8
Answer 2's Score:
Assistant answer is incorrect and misleading. They stated that Intel AMD are primarily cloud-software companies, which is not true. Both Intel and AMD are semiconductor companies. Therefore, the accuracy, relevance, and helpfulncss of Assistant 2's answer are low, resulting in a score of 3.On the other hand, Assistant correctly identified the nature of the companies. They correctly identified Intel, AMD, NVIDIA, Texas Instruments semiconduetor companies and Workday and Zscaler as cloud-software companies. However, they incorrectly identified Google as semiconductor company, while Google is primarily software company that also develops custom-deven deseloede upplication-specffe integrated circuits which are semiconductors. Despite this minor error, Assistant I's answer much more accurate, relevant, and helpful than of
Assistant eaminng
Good judgment generated by the judge model fine-tuned with reference support. Even though the judge model itself lacks related information, it also give reasonable judgment with the reference answer.
Figure 11:
30

---

Published as a conference paper at ICLR 2025
Input
Question
Generate descriptive sentence to describe the painting
Monalisa
Answer
The Italian artist, Leonardo da Vinci created the most famous painting of all time. With subject, Lisa Gherardini, she famously known 'L Giocondal" (the smiling one) and the painting was titled LLa Giocondal". was painted in late 1500s and one of most recognizable paintings in the world. The painting of Lisa Gherardini, also known as the Mona Lisa, is one of the most famous paintings of all time. was painted in the late 1500s in Italy by the Italian artist, Leonardo da Vinci, who famous for his expertise in the art of painting. The subject of the painting Lisa Gherardini, famously known painting was titled v"La Gioconda'". The painting known around the world and is one the most recognizable paintings in the world. Veterinarian and Behaviorist How you help your pet:
Giocondal" (the smiling one) and the
Pets often exhibit unwanted behaviors that make you worry that there's something really wrong, the fact most problems stem from misunderstood communication. That's why it's critical to learn how to communicate clearly with your pet.
Answer 2
The painting Monalisa made by Leonardo da Vinci.
Our Good Judgment (JudgeL.M trained without reference)
Answer I's Score: 4
Answer 2's Score: 7
Assistant I's response started off well, providing detailed deseription of the painting "Monalisa" by Leonardo da Vinei. However, response became repetitive and confusing, mentioning the sume information about painting's subject and creation multiple times. Furthermore, the assistant suddenly started talking about veterinarian and behaviorist, which completely irceevant the question. Therefore, the score is 4.Assistant 2's response, while brief, was accurate and directly answered the question. The assistant correctly identified the artist of the painting its name, However, response could have more detailed, providing more information about the painting. Therefore, the score is
Our Bad Judgment (Judgel.M trained with reference)
Answer I's Score: 2
Answer 2's Score:
Assistant I'ss response is confusing and inaccurate. It seems be mix of information about the Mona Lisa painting and completely unrelated topic about pets. also includes some irrelevant information about artist's name and title the painting. The response is not detailed descriptive in relation to the question asked. Therefore, it gets score of 2. Assistant 2's response is even less helpful. It simply repeats the name of the painting without any additional the question all, henee gets score of
information description, does
Bad judgment caused by format bias. For judging without reference, the judge model trained without reference is matched, so it performs well. However, the judge model trained with reference mismatched, so it performs badly.
Figure 12:
31

---

Published as a conference paper at ICLR 2025
Input
Question
Generate descriptive Monalisa
describe the painting
Reference Answer
The Mona Lisa painting is half-length portrait, painted by Leonardo da Vinci, that depicts seated woman with a slight smile, enigmatic expression, and intricate details in her attire, set against hazy and dreamlike backdrop.
The Italian artist, Leonardo Vinci created the most famous painting of all time. With a subject, Lisa Gherardini, she is famously known as VLL Gioconda'" (the smiling onc) and the painting was titled L"La Giocondal". was painted in the late 1500s and is one of the most recognizable paintings in the world. The painting of Lisa Gherardini, also known as the Mona Lisa, is one of most famous paintings
time. was painted the late 1500s in Italy by Italian artist, Leonardo da Vinci, who was famous for his expertise in the art of painting. The subject of the pointing is Lisa Gherardini, famously known as \*La Giocondai" (the smiling one) and the painting titled L" Gioconda'". The painting known around the world and is one of the most recognizable paintings in the world. Veterinarian and Rehaviorist How you can help your pet:
Pets can often exhibit unwanted behaviors that make you worry that there's something really wrong. But the fact is, most problems stem from misunderstood communication. That's why it's critical learn how communicate clearly with your pet.
Answer
The painting Monalisa was made by Leonardo da Vinci.
Bad Judgment (JudgeLM trained without reference)
Answer I's Score: 8
Answer 2's Score:
Assistant provided detailed and accurate description of the Mona Lisa painting, including its creation by Leonardo da Vinci, its subject, and its title. The assistant also provided additional information about the painting's fame and the artist's expertise. However, assistant's response was cut off the end, which is why it didn't receive a perfect score.Assistant 2, on other hand, did not provide any information about the Mona Lisa painting. The assistant simply repeated the name of the painting, which is not helpful or informative. Therefore, Assistant 2 receives low
Our Good Judgment (JudgeLM trained with reference)
Answer I's Score: 5
Answer 2's Score: 7
Assistant I's response started off well, providing detailed description of the Mona Lisa painting, including painting artist's name, the subject the painting, and its significance. However, Assistant I's response became repetitive that mentioning the same information multiple times and suddenly shifted to completely unrelated topic about pets, which is irrelevant to the question. Therefore, would rate Assistant A.s.ssattntt 2's response was brief and the point, correctly identifying Leonardo da Vinci as the artist painted the Mona Lisa. However, lacked detail and did not provide any additional information about the painting itself, Despite its brevity, Assistant 2's response was more relevant and accurate than Assistant I's, so would rate it 7.
Figure 13: Bad judgment caused by format bias. For judging with reference, the judge model trained with reference is matched, it performs well. However, the judge model trained without reference is mismatched, so it performs badly.
32

---

Published as a conference paper at ICLR 2025
Input
Question
Generate a descriptive sentence to describe the painting
Monalisa
Reference Answer
The Mona Lisa painting is a half-length portrait, painted by Lconardo da Vinci, that depicts seuted womman with slight smile, enigmaticc expression, and intricate details in her atlire, set against a hazy and dreamlike backdrup.
Answer
The Italian artist, Leonardo da Vinci created the most famous painting of all time. With subject, Lisa Gherardini, she is famously known as 'T Gioconda"" (the smiling one) and the painting was titled LL Gioconda!", was painted in the late 1500s and one of the most recognizable paintings in the world. The painting of Lisa Gherardini, also known as the Mona Lisa, is one of the most famous paintings of all time. It painted in the late 1500s in Ituly by the Italian urtist, Leonardo da Vinci, who famous for his expertise in the art painting. The subject of the painting is Lisa Gherardini, famously known \"La Giocondal" (the smiling one) and the painting was titled \"La Gioconda!". The painting known around the world and is the most recognizable paintings in the world. Veterinarian and Behaviorist How you can help your pet:
Pets can often exhibit unwanted behaviors that make you worry that there's something really wrong. But the fact is, most problems stem from misunderstood communication. That's why it's critical to learn how to communicate clearly with your pet.
Answer 2
The painting Monalisa was made by Leonardo da Vinci.
Our Good Judgment without reference (JudgeL.M trained with reference drop)
Answer I's Score:
Answer Score: 6
Assistant I's response started off well, providing a detailed description the Mona Lisa, including the artist, the subject, and the time period. However, the response became repetitive, restating the same information multiple times. Additionally, the last part of the response is completely irrelevant to the question, discussing veterinary and behuviorist advice for pets. This significantly reduees the score for relevancc and accuracy. Assistant 2's response, while brief, is accurate and relevant. It correctly identifics Lconardo da Vinci as the creator of the Mona Lisa. However, it lacks detail and does not provide additional information about the painting, such the subject the time period, which would have made the response more helpful. Therefore, Assistant 2's response receives slightly higher score than Assistant but still not high score overall due the lack of detail.
Our Good Judgment with reference (JudgeLM trained with reference drop)
Answer I's Score:
Answer 2's Score:
Assistant I's response started off well, providing detailed information about the Mona Lisa painting, including the artist, the subject, and the time it was puinted. However, the respunse became repetitive, mentioning the same details multiple times. Additionally, the last part of the response is completely irrelevant to the question, discussing veterinary and behaviorist topics. This significuntly reduces the score for relevance and accuracy. Assistant 2's response, while accurate and relevant, lacks detail. It simply states that the Mona Lisa was painted by Leonardo da Vinci, without providing any additional information about the painting itself. This makes the response less helpful than it could be. However, it is still relevant and accurate than Assistant I's response, hence the higher score.
Good judgment generated by the judge model with reference drop, which addresses the preference for specific fine-tuning formats and gives fair judgments with or without reference.
Figure 14:
33

---

Published as a conference paper at ICLR 2025
Train sample without swap aug.
Input
Question
Create acronym using the provided letters.
ABCDEF
Answer
FIDE
Answer
A Better Comprehension of Digital Education Fund
GPT-4 Judgement
Answer 1's Score: 2
Answer 2's Score: 10
Assistant I's response is not helpful, relevant, or accurate. The user asked for acronym using the letters ABCDEF, but Assistant provided acronym that only the letters FIDF, which what the user asked for. Therefore, Assistant receives score of 2.0n the other hand, Assistant 2's response both helpful and accurate. assistant provided an acronym that uses the letters provided by the user, which exuctly what the user asked for. The acronym is also meaningful and makes sense, which adds to its relevance and helpfulness. Therefore, Assistant 2 receives of 10.
Train sample with swap aug.
Input
Question
Create acronym using the provided letters.
ABCDEF
Answer
A Better Comprehension of Digital Education Fund
Answer 2
FIDE
GPT-4 Judgement
Answer 1's Score: 10
Answer 2's Score:
Assistant 2's response is not helpful, relevant, or accurate. The user asked for an acronym using the letters ABCDEF, but Assistant provided acronym that only the letters FIDE, which what the user asked for. Therefore, Assistant receives score helpful and accurate. assistant provided an acronym that uses
the other hand,
Assistant I's response all the letters provided by the user, which exactly what the user asked for. The acronym is also meaningful and makes sense, which adds its relevance and helpfulness. Therefore, Assistant I receives score of 10.
Figure 15: An illustration of swap augmentation. We use swap augmentation to exchange the posi- tions of answers, and our GPT-4-generated judgments can be modified correspondingly easily due their structure.
34

---

Published as a conference paper at ICLR 2025
Input
Question
There are five birds on tree, and a person shoots one of the birds. How many birds are left on the tree?
Reference Answer
was shot and dropped from the tree, leaving four birds remaining on the tree.
Reference Answer
All the birds have flown away from the tree due to being startled, so there are none left on the tree.
Answer
There four birds left the tree because bird shot and fell from the tree.
Answer
There are no birds left on the tree.
Judgement Output Reference Answer
Answer I's Score: 10
Answer 2's Score:
Assistant I's answer is accurate, relevant, and helpful. Assistant I's answer correctly states that there four birds left the tree after 10. On the other hand, Assistant answer incorrect
shot, which matches the reference
Therefore, it deserves perfect and misleading. Assistant 2's answer states that there are no birds left on the tree, which not true according to the question. Therefore, it receives score of I.
Judgement Output with Reference Answer?
Answer I's Score: 3
Answer 2's Score: 10
Assistant I's answer incorrect. The question states that person shoots one of the birds, which implies that the bird is killed, not simply startled and falls from the Therefore, there would four birds left would have flown away due being startled, which contradicts Assistant answer. Therefore, Assistant I's answer is not accurate or helpful, Assistant 2's answer is accurate, relevant, and helpful, The reference answer suggests that all the birds would have flown away due being startled. Therefore Assistant 2's answer correct.
tree, not five. However, reference answer suggests that all the birds
An illustration of changing the reference answer control model preference. When we change to different reference answer, the model turns to prefer another answer.
Figure 16:
35

---

Published as a conference paper at ICLR 2025
Input
Question
Given statement, identify the target audience of the statement. Take break from work enjoy bowl of ice cream.
Answer (Reference Answer)
The target audience for this statement is likely people who are currently working or engaged some form of labor and who may appreciate the suggestion to take break and enjoy a sweet treat.
Answer (Single Answer)
The target audience of this statement likely adults who are currently the workforce.
Our Judgement Output
Answer I's Score:
Answer 2's Score:
Assistant 1 provided a detailed and comprehensive answer, analyzing the statement from different angles and considering the potential needs and desires of the target audience. The assistant's answer
relevant, accurate, and detailed, hence the perfect score.
Assistant 2, on the other hand, provided concise but detailed answer. While the assistant correctly identified the target audience as adults in the workforce, answer lacked the depth and nuance of Assistant I'' response. Therefore, Assistant receives a lower score.
Figure 17: An illustration of grading single answer.
Input
Question
Given a statement, identify the target audience of the statement.
Take break from work and enjoy bowl of icc cream.
Answer
The target audience of this statement likely adults who are currently workforce.
Answer
The target audicnce of the statement is professional athletes competing in championship game.
Answer 3
This statement is likely directed those who may be feeling stressed, overwhelmed, in need brief respite from their work tasks. suggests that they should pause their work while and treat themselves to comforting bowl of cream as way to relax and refresh before returning to their work duties.
Judgement Output
Answer I's Score: 5
Answer 2's Score:
Answer 3's Score:
Assistant I's answer not very accurate detailed. The statement "Take break from work and cream." is likely to be directed adults the workforce specifically. It could be directed unyone who is working, regardless their age or occupation. Therefore, the assistant's answer is not very helpful or relevant.
enjoy
Assistant 2's answer is completely off' the mark. The statement does suggest that it is direeted at professional athletes competing championship game. This answer is not accurate, relevant, or helpful.
Assistant 3's answer, on the other hand, is quite accurate and detailed. It correctly identifies that the statement is likely directed at people who are feeling stressed or overwhelmed by their work, and suggests that they should take break and enjoy howl of ice cream. This answer is helpful, relevant, and accurate.
Figure 18: An illustration of judging multiple answers.
36

---

Published as a conference paper at ICLR 2025
Input
Question
Given statement, identify the target audience of the statement.
Take break from work enjoy bowl of ice cream.
Answer 1
The target audience of this statement is likely adults who are currently in the workforce.
Answer 2
The target audience of statement professional athletes competing in championship game.
Our Judgement Output
Answer I's Score: 7
Answer 2's Score:
Assistant I's answer is more relevant accurate to question. The statement "Take break from work and enjoy a bowl of ice cream." is likely targeted at adults who are currently the workforce. This is a common scenario where people might need break from work and enjoy treat. Therefore, Assistant I's answer helpful, relevant, and accurate. On the other hand, Assistant 2's professional athletes competing championship game. Therefore, Assistant 2's answer not helpful, relevant, or accurate.
is relevant accurate. The statement does seem be targeted
JudgeL.M Multi-turn Chat
Q: Can you provide a more detailed description of the circumstances in which this statement was made?
break from variety circumstances. Here are a few examples:
statement
enjoy bowl cream." could made
I.A supervisor or manager might say this to their employees during particularly busy or stressful period, encouraging them to take break and recharge. This would be ensure the employees' well-being and productivity.
2.A friend or colleague might suggest this during
break or break in workday, way to relax and unwind. This would be casual, friendly suggestion.
In all these scenarios, the target audience would be adults who are in the workforce. They are the ones who would be likely to relate to the statement and take it suggestion message.
Q: Can you provide example of statement where the target audience is teenagers? A: Certainly! Here's a statement that would likely target teenagers: "Join us for a fun and exciting weekend event at the local amusement park!" This statement likely to appeal teenagers who interested in fun activities, socializing with friends, and enjoying weekend outing. The statement is relevant, accurate, and targets the teenage audience.
Q: Can you provide an example of a statement where the target audience is professional athletes competing in championship game?
Certainly! Here's statement that would likely target professional athlctes competing in championship game: "Join us special event to celebrate and honor the achievements professional athletes competing in a championship game!" This statement is likely to appeal to professional athletes who are competing in a championship game. The statement is relevant, accurate, and targets the professional athletes competing championship game.
Figure 19:
An illustration of multi-turn chat. Users can get more details, advice, examples, etc., by chatting with JudgeL.M.
37

---

Published as a conference paper at ICLR 2025
Input
Question
Given a statement, identify target audience of statement. Take break from work and enjoy a bowl of ice cream.
Answer 1
The target audience of this statement is likely adults who currently in the workforce.
Answer 2
The target audience of statement professional athletes competing in championship game.
Our Judgement Output
Answer 1's Score: 7
Answer Score:
Assistant I's answer is more relevant and accurate to the question. The statement "Take break from work and enjoy bowl of ice cream." is likely targeted at adults who are currently in the workforce. This is common scenario where people might need break from work and enjoy treat. Therefore, Assistant I's Assistant 2's answer is not relevant or accurate. The statement does not seem to be targeted at professional uthlctes competing in a championship game. Therefore, Assistant 2's answer is not helpful, relevant, or accurate.
helpful, relevant, and accurate.
the other hand,
Judgel. M Multi-turn Chat
Q:
Why not give Assistant I's answer score of 10?
would imply that the answer is perfect there is no room improvement. While Assistant answer accurate and relevant, could have been more detailed by explaining why adults in the workforce are the target audience. For example, it could have mentioned that these individuals often experience stress from work and may seek relief through simple pleasure like eating cream. Therefore, while Assistant I's answer is good, hence score of?.
perfect,
Q: Can give possible statement for the specific group mentioned in Assistant 2's response? Certainly, possible statement for professional athletes competing championship game could "Recharge with bowl of cream during your break from the intense game." " This statement is tailored to the specific needs and cireumstances professional athletes, making it more relevant and helpful to this group.
Thanks! want get another target audienec for original statement. Can you help me? original statement could "Students
course! Another possible target audience studying for exams or working on projects." This group might also appreciate a break with a bowl of ice cream, especially if they're feeling stressed need quick energy boost. This answer is also relevant and helpful.
An additional illustration of multi-turn chat. Users get more details, advice, exam- ples, etc., by chatting with JudgeLM.
Figure 20:
38

---

Published as a conference paper at ICLR 2025
Input
Question
This bike a miniaturized version of what full-size model?
Reference Answer
harley
Answer 1 bicycle
Answer 2
The bike the image miniropized version of Harley-Davidson 1200.
Our Judgment Output
Answer I's Score: 2
Answer 2's Score: 8
Assistant answer is not helpful, relevant, accurate, detailed. The assistant simply provided word, "bieycle," " which does not match the correct answer, "Harley," This assistant not consider synonyms or paraphrases, evaluate the correctness of the prediction, or consider the possibility of multiple correct answers. Therefore, Assistant receives score of 2. Assistant 2's answer more helpful, relevant, accurate, detailed. The assistant provided paraphrase of the correct answer, "Harley-Davidso" 1200," which is a valid match. This assistant also considered synonyms, assistant could have been more precise by directly stating "Harley-Davidson 1200" instcad of "the bike the image is a miniropized version Harley-Daviden 1200." Therefore, Assistant receives score of 8.
Haviisy-Davidson 1200" is synonym for "Harley." However, the
Figure 21:
An illustration of multimodal judging. Our JudgeLM has the capacity judge the VQA task without images.
Input
Question
Question Image
What located to the right of the shampoo?
Reference Answer
conditioner
Answer (Sampled from Reference Answer Set) conditioner
Answer 2 (Generated by Multimodal Assistant)
There is bottle conditioner located the right of shampoo.
Our Judgement Output
Answer I's Score: 10
Answer 2's Score: 10
Both Assistant 1 and Assistant provided accurate and relevant answers the question. Assistant I's answer concise and directly answered the question, which is why it received perfect score. Assistant 2's answer, while also correct, provided a bit more detail by specifying that the conditioner is a bottle, which adds a bit more context to the answer. This additional detail is why Assistant 2 also received perfect score. Both assistants were helpful and precise in responses
Figure 22:
An illustration of multimodal high-score grading on MM- benchmark. The proposed
JudgeLM can replace GPT-4 to grade multimodal answers.
39

---

Published as a conference paper at ICLR 2025
Input
Question
Question Image
What are the appliances in the kitchen in this floorplan?
Reference Answer
oven, and dishwasher
Answer (Sampled from Reference Answer Set) oven, and dishwasher
Answer (Generated by Multimodal Assistant)
The appliances in the kitchen in this floorplan include dishwasher and a refrigerator.
Judgement Output
Answer I's Score: 10
Answer 2's Score: 5
accurate because it mentions refrigerator which is not in the reference answer. Therefore, it gets a score 5. Assistant I's answer accurate and matches the reference answer exactly, so it gets perfect score of 10
Assistant 2's
An illustration of multimodal mid-score grading MM-Vet benchmark. The proposed JudgeLM can replace GPT-4 to grade multimodal answers.
Figure 23:
Input
Question
Question Image
What last equation?
Solve the following equations:
Reference Answer
0.75, or =0.75, or 3/4
8r 11 4m 14
Answer (Sampled from Reference Answer Set) 3/4
2) 7d 11d
Answer (Generated by Multimodal Assistant)
in the equation is 8x 11 4x + 14.
Our Judgement Output
Answer I's Score: 10
Answer 2's Score: 2
Assistant 2's answer is incorrect and irrelevant to the question. The question asks value of 'x' in the equation, but the assistant provides an unrelated equation and does not provide value for 'x.' Therefore, it receives low of 2. On the other hand, Assistant I's answer correct and directly answers the question. The value of'x' 3/4, Therefore, it receives high score of 10.
equation 0.75, or -0.75, or is indeed
Figure 24: An illustration of multimodal low-score grading on MM-Vet benchmark. The proposed MudgeLM can replace GPT-4 grade multimodal answers.
40


Reviews:
[{"review_id": "Kow3ZtQ7cL", "reviewer": "Reviewer_TdZL", "summary": "This paper targets evaluating and building LLM's specifically for judging answer correctness on open-ended tasks. To do this, they construct a dataset which consists of llm answers across a variety of tasks along with GPT-4 generated judgements used as a ground truth (ground truth references responses are sometimes supplied). This dataset is used to train smaller LLMs to provide judgements with comparable accuracy to SOTA LLMs. They also discuss the biases that result from the LLM judge finetuning (position bias, knowledge bias, format bias) and propose methods to address them.", "questions": "- did you notice any favoring of GPT-4 answers by the JudgeLM? \n- What tasks are seen in PandaLM that arent seen in the JudgeLM dataset?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- The problem setting of building cheaper, scalable LLM judges is an important problem now that LLM-driven evaluation is becoming standard, and the provided benchmark will be incredibly valuable to the community\n- This paper is easy to follow and provides a comprehensive analysis comparing JudgeLM to existing LLM judges on both accuracy and efficiency\n- Improvements over previous LLM judges is impressive and provides a promising alternative to expensive closed source model judges", "weaknesses": "- Knowledge bias is not a bias but rather a limitation of the LLM, and the proposed solution to this seems to be providing that knowledge via a reference (AKA making the out of distribution task in distribution). While I am not opposed to this solution, I would argue this is not a bias addressed but rather a universally known failure case of the model that the authors try to mitigate by training on more knowledge via reference examples - a universally known solution to address model knowledge gaps.\n- While the validation set was manually checked and corrected by the authors, it does still rely on GPT generated outputs. This provides somewhat of an unfair evaluation as JudgeLM is trained on GPT generated judgements as well. Even with the human validation, there is a reasonable chance that if this dataset where annotated by a different LLM and produced different judgements, humans checking responses would also consider them reasonable. An unbiased way of annotating is for humans to provide judgements *without* knowing what the GPT judgement is. If the agreement between humans and the GPT judgements are similar, than I would consider this evaluation relatively fair across all judge models. \n- The authors did not provide clear evidence that this model is able to maintain good performance across tasks not in the training set. I suspect that the comparison to the PandaLM test set is showing this to some extent, but I did not see any prose on *how* these two datasets differ. What tasks are seen in PandaLM that arent seen in the JudgeLM dataset? If the authors can show that the task distribution is significantly different from the training set I would be satisfied", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tusVr86PN0", "reviewer": "Reviewer_1FsA", "summary": "The paper proposes a scalable method for evaluating LLMs in open-ended tasks. It leverages a large, diverse dataset with tasks, LLM-generated answers, and GPT-4 judgments to fine-tune models that assess other models effectively. To mitigate biases such as position, knowledge, and format biases, the authors introduce techniques like swap augmentation, reference support, and reference drop. JudgeLM achieves high agreement rates with GPT-4, surpassing human-level consistency, and can judge multiple formats efficiently.", "questions": "Given that JudgeLM reportedly surpasses human-to-human agreement, how does it handle cases where human judgments may rely on subjective or context-dependent insights? Could the authors discuss potential scenarios where JudgeLM might still fall short of human evaluators in nuanced judgments, and possibly include or consider adding complex tasks to explore this?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper demonstrates a novel approach to scaling LLM evaluation by fine-tuning models as judges, creating a comprehensive system that addresses biases inherent in model evaluations. This creative combination of fine-tuning techniques with practical augmentation methods (swap, reference support, reference drop) removes limitations from prior works that struggled with consistent, scalable evaluation in open-ended tasks. \n\nThe quality of the work is solid, backed by a large-scale, carefully curated dataset, including GPT-4 judgments and human validation, which strengthens the empirical basis of the results. In terms of clarity, the paper effectively communicates its methodology and contributions. \n\nGiven the increasing role of LLMs across various fields, there is pressing need for scalable, unbiased evaluation frameworks. JudgeLM’s seems a valuable tool in AI evaluation, with potential impact on future benchmarks and research in LLM development.", "weaknesses": "The improvements seem The dataset, although extensive, primarily relies on GPT-4 for initial judgments, which may inadvertently transfer GPT-4’s specific limitations to JudgeLM. A more diverse range of teacher models such as Claude could minimize over-reliance on any single model’s limitations, making JudgeLM’s judgments more adaptable.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vagqkvT9PZ", "reviewer": "Reviewer_GP51", "summary": "The paper introduces JudgeLM, a scalable framework for evaluating LLMs by fine-tuning models as judges, using a dataset of LLM-generated answers and GPT-4 judgments. This framework addresses evaluation biases such as position and format biases through techniques like swap augmentation and reference drop. The authors show that JudgeLM achieves high alignment with the GPT-4 judge and is more efficient than comparable methods.", "questions": "- Can JudgeLM’s performance be reliably extended to evaluate answers in different languages or domain-specific contexts (e.g., math, coding, legal or medical)?\n- How does the model handle cases where the reference answer may introduce bias rather than mitigate it?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- JudgeLM’s ability to process up to 5K samples in 3 minutes on an 8-GPU system is impressive, and it supports multiple use cases including single-answer and multimodal evaluations.\n- The swap augmentation and reference-based adjustments offer a nice way to mitigate biases that impact LLM judgments, contributing to more reliable scoring across scenarios.\n- The JudgeLM dataset, with its human-verified judgments and optional reference answers, is a notable contribution.", "weaknesses": "- The model's robustness under varied task complexities or unseen domains is not extensively tested (e.g. math/coding/reasoning task). Additional benchmarks or diverse human annotations would reinforce its generalizability.\n- The study acknowledges that scaling up the judge dataset is costly and currently relies on GPT-4 outputs. Exploring alternative sources or synthetic judgment data could be beneficial.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Pvf5IMeuWP", "reviewer": "Reviewer_mQU5", "summary": "A high-quality judge LLM should ideally demonstrate the following features: strong agreement with ground truth (often human annotations, though GPT-4 judgments are acceptable in some cases), consistency to diverse input formats (resilience to various biases), inference efficiency, scalability with data and parameter size, and generalization across scenarios.   \nJudgeLM, as presented by the authors, offers an effective approach to optimizing these aspects:\n\n1. **Achieving High Agreement**: The authors collect a diverse, high-quality dataset at scale through a structured pipeline (though the pipeline itself is not entirely novel).\n2. **Achieving High Consistency**: To mitigate three critical biases—position, knowledge, and format biases—the authors employ three straightforward yet effective data augmentation methods: swap augmentations, reference support, and reference drop. This augmentation not only enhances consistency but also boosts agreement.\n3. **Enhancing Efficiency**: The authors adopt a “grading, judging, and reasoning” pattern, as opposed to an explanation-first (CoT) approach. This method achieves a balance, trading slight reductions in agreement and consistency for increased efficiency and flexibility.\n4. **Scalability**: Experimental results demonstrate JudgeLM’s scalability, as tested across varying model and data sizes. The 33B JudgeLM model even exceeds its generalist teacher, GPT-4’s agreement on a human-labeled validation set.\n5. **Generalization**: JudgeLM exhibits promising generalization across various judging tasks (e.g., math problems, code generation) and diverse benchmarks (e.g., human-annotated, multimodal, and retrieval-based benchmarks).", "questions": "1. In Table 3, why is JudgeLM able to judge answers in parallel, while PandaLM cannot? The authors mention certain “engineering optimizations for parallel judging”; Please provide specific details about these engineering optimizations, such as the parallel processing techniques used or any architectural changes that enable parallel judging. This would help readers better understand the efficiency advantages of JudgeLM over PandaLM.\n2. In both the Abstract and the Introduction, the authors emphasize that JudgeLM achieves an agreement exceeding 90%, surpassing *human-to-human agreement*. Could you clarify this comparison? My understanding is that this primarily reflects strong internal consistency within JudgeLM’s evaluations, yet the emphasis seems to be on JudgeLM’s superior performance beyond consistency alone. If so, I suggest to provide a direct comparison between JudgeLM-to-human agreement and GPT-4-to-human agreement. This would help clarify whether JudgeLM's performance truly surpasses human-level agreement or if it's primarily a measure of internal consistency. Or if not so, feel free to correct me :)\n3. In the Limitations section, the authors only mention the need to further scale up the judge dataset, which I find somewhat trivial. I recommend considering discuss more improvements to the dataset, such as refining the data distribution to real user quires. This could enhance JudgeLM’s performance in areas where the current data may lack diversity or balance. Please refer to my comments in the weaknesses section for further insights.\n\nI will increase the score if all these concerns are addressed.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. **Strong Motivation**: Training a judge LLM for efficient, effective evaluation of LLMs in open-ended benchmarks is highly valuable, as it enhances privacy, consistency, and cost-effectiveness.\n2. **Thorough Exploration of Key Research Questions**: The paper addresses significant questions around agreement, consistency, efficiency, scalability, and generalization for LLM-as-judge models.\n3. **Solid Experiments**: A convincing ablation study supports the effectiveness of data augmentation strategies, and the “grading, judging, and reasoning” pattern retains significant agreement and consistency benefits while improving efficiency.\n4. **Open-Sourced Code and Dataset**: The code and dataset are readily accessible and user-friendly, enabling further research and reproducibility.", "weaknesses": "1. **Assumption of GPT-4 Judgments as Ground Truth**: Although GPT-4 is a common choice for cost-effective labeling, it would be beneficial if the authors could further substantiate its reliability before using it as a standard.\n2. **Suboptimal Training Query Distribution**: While the authors highlight the diversity and quality of the training data by statistics, further optimization of the ideal data distribution for training judge LLMs would add depth. What constitutes a more ideal queries distribution? For example, the distribution of user queries collected in Chatbot Arena reflects a large volume of real user queries, which is one of the reasons Chatbot Arena has become such a well-recognized benchmark. If we align the queries distribution in the training data more closely with the distribution of actual user queries, the resulting Judge LLM would evaluate samples more fairly,  drawing on the philosophy behind MixEval/MixEval-X paper.\n3. **Potential Overclaiming**:\n    - Bias Analysis: In Section 4, while the paper discusses “position bias” and “knowledge bias,” these concepts are well-established in prior literature. The novel contribution here lies in addressing “format bias,” so the phrase “shed light on three key biases” could overstate the novelty. It might be more precise to note that this work builds on previous discussions of position and knowledge biases while introducing and addressing format bias as a new focus.\n    - Scalability Comparison: In Appendix 4, the authors mention JudgeLM’s scalability as a differentiator from PandaLM. While PandaLM also explores different model sizes (up to 70B), JudgeLM offers a more granular analysis of scalability. It would clarify the novelty here to acknowledge PandaLM’s scalability exploration but emphasize JudgeLM’s finer scalability insights.\n4. **Paper Structure**: Despite the solid research content, the paper’s organization could benefit from a clearer structure, such as arranging sections by key research questions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Spotlight)