RevisEval: Improving LLM-as-a-Judge via Response-Adapted References
Authors: Qiyuan Zhang
Yufei Wang
Tiezheng YU
Yuxin Jiang
Chuhan Wu
Liangyou Li
Yasheng Wang
Xin Jiang
Lifeng Shang
Ruiming Tang
Fuyuan Lyu
Chen Ma
Venue: ICLR 2025 Poster (2025)
Keywords: large language models
evaluation
revision
Primary area: generative models
TL;DR: we propose a novel evaluation paradigm, RevisEval,  to generate response-adaptive references for evaluation by revising responses to be evaluated.

Abstract: With significant efforts in recent studies, LLM-as-a-Judge has become a cost-effective alternative to human evaluation for assessing text generation quality in a wide range of tasks. However, there still remains a reliability gap between LLM-as-a-Judge and human evaluation. One important reason is the lack of guided oracles in the evaluation process. Motivated by the role of reference pervasively used in classic text evaluation, we introduce RevisEval, a novel text generation evaluation paradigm via the response-adapted references. RevisEval is driven by the key observation that an ideal reference should maintain the necessary relevance to the response to be evaluated. Specifically, RevisEval leverages the text revision capabilities of large language models (LLMs) to adaptively revise the response, then treat the revised text as the reference (response-adapted reference) for the subsequent evaluation. Extensive experiments demonstrate that RevisEval outperforms traditional reference-free and reference-based evaluation paradigms that use LLM-as-a-Judge across NLG tasks and open-ended instruction-following tasks. More importantly, our response-adapted references can further boost the classical text metrics, e.g., BLEU and BERTScore, compared to traditional references and even rival the LLM-as-a-Judge. A detailed analysis is also conducted to confirm RevisEval's effectiveness in bias reduction, the impact of inference cost, and reference relevance.

Published as a conference paper at ICLR 2025
REVISEVAL: IMPROVING LLM-AS-A-JUDOR VIA RESPONSE-ADAPTIS REFERENCES
Qiyuan Zhang', Yufei Wang', Tiezheng Yu², Yuxin Jiang? Chuhan Wu², Liangyou Li,, Yasheng Wang'. Xin Jiang", Lifeng Shang?. Ruiming Tang", Fuyuan Lyu", Chen Ma 'City University of Hong Kong. Huaaei Noah's 's Ark Lab, The Hong Kong University of Science and Technology (Guangzhou),
'MeGill University & MILA
qzhang7 -cmy ci cityu. edu, odu.hk, wang.
com
ABSTRACT
With significant efforts in recent studies, LLM-as-a-Judge has become a cost- effective alternative human evaluation for assessing text generation quality in a wide range of tasks. However, there still remains a reliability gap between LLMMas-- a-Judge and human evaluation. One important reason is the lack of guided oracles in the evaluation process. Motivated by the role of reference pervasively used in classic text evaluation, we introduce REVISEVAL, novel text generation evalua- tion paradigm via the response-adapted references. REVISEVAL is driven by the key observation that an ideal reference should maintain the necessary relevance to the response to be evaluated. Specifically, REVISEVAL leverages the text revision capabilities of large language models (LLMs) to adaptively revise the response, then treat the revised text as the reference sypponse-adapted reference) for the subsequent evaluation. Extensive experiments demonstrate that REVISEVAL out- performs traditional reference-free and reference-basee evaluation paradigms that use LLMM-a---Judde across NLG tasks and open-ended cnstruction-folowwwit tasks. More importantly, our response- esspons--ddapted references can further boost the classical text metrics, e.g.. BLEU and BERTScore, compared to traditional references and even rival the L..M--a-a-Judge REVIsEVAL's 's effectiveness in bias reduction, the impact of inference cost, and reference relevance.
detailed analysis is also conducted confirm
INTRODUCTION
As the large language model (LLM) already exhibits strong alignment with humans (Gilardi et al., 2023; OpenAl et al., 2024), LLM-as-a-Judge (Chang et al., 2024; et al., 2024b; Gao al., 2024b), aka. LLM-evaluator, has emerged as viable alternative human evaluation in assessing text generation quality. Given the task instruction and the corresponding model-generated responses, I.LMs are prompted to predict preferences scores for these responses. Despite considerable efforts have been made, such as chain-of-thought (Zheng et al., 2023), specialized rubrics (Liu et al., 2023), and extensive evaluation-speeific training datasets (Li et al., 2024a; Wang et al., 2024c:b), human evaluation remains the gold standard in text quality assessment (Zeng et al., 2024) and LLM-as-a- Judge struggles with particular biases (Huang et al., 2024) and being vulnerable to the misleading context (Dubois et al., 2024; Chen et al., 2024). One important reason is the lack of an oracle to direct the evaluation process. Fortunately, classical text evaluation metries, like BLEU (Papineni al., 2002) and ROUGE (Lin, 2004), offer valuable prompts in mitigating such gap: given an appropriate reference, i.e., the ground-truth answer the task, calculating the similarity between the references and the model-generated responses achieve satisfactory correlation with human evaluations. Furthermore, several studies highlight the reference could prevent being overly sensitive to semantic deficiency (Sheng al., 2024) and overcoming bias (Deutsch al., 2022) in certain cases.
However, straightforward leveraging references in LLM-as-a-Judee can also be challenging. In addition to the availability of high-quality references (Rei et al.. 2021), previous works (Mehri & Eskenazi, 2020; Gömez-Rodriguez & Williams, 2023; Guan & Huang, 2020) find that pre-existing references introduce noise across various text evaluation tasks due to the one-to-many problem,

---

Published as a conference paper at ICLR 2025
Figure I: Performance comparison of reference- free and reference-based evaluation paradigms across different similarity groups in MT-Bench, using GPT-4-as-a-Judge In the reference- based evaluation, the GPT-4 direct response is used as the reference, and the evaluated response with a higher BERTScore with the reference is regarded as the preferred one. As the similarity between the reference and the response increases, the human agreement accuracy of the snserence-basee evalu- ation significantly improves, while the reference- free evaluation maintains relatively consistent per- formance across all similarity levels.
Comparative Evalantion Performance across Sorted Similarity Groups MT-Bench
86
$4.07
83.48
Accurscy (%
822
81.75
80-
78-
12:12
26. 24.34
33.44
41.32
53.52
Similarity (BERTScoree 100)
where for given task input, there exist many diverse yet valid responses. In this case, particular pre-existing references could negatively penalize many appropriate but dissimilar responses in the evaluation process (Ji et al., 2022). Thus, we hypothesize that effective reference should be closely relevant to the response be evaluated. We further verify this on MT-Bench (Zheng et al., 2023), an open-ended instruction- followinnn dataset. As shown in Figure I, we use GPT-4 direct responses to the instructions as the references and quantify relevance by the similarity between the references and the responses using BERTScore (Zhang et al., 2020). We find that higher relevance simulates greater utility from the reference, resulting in more effective evaluations than a reference free evaluator.
Motivated by the above findings, deem that effective reference should maintain high quality while ensuring relevance to the response, which led us to consider that revising the response adaptively could be good candidate for this reference (Guo al., 2024). Therefore, we propose (REVISEVAT.). Specifically, given the
novel evaluation paradigm (instruction, response) pair, REVISEVAL first revise the response using the instruction and evaluation rubric, resulting in the response addated reference. REVISEVAL further leverages this generated aappons-addpptdd reference to guide final evaluation (e.g., scoring or pairwise comparison). By revising the original response, we ensure that the generated reference is both high-quality and closely relevant to the original content. The comparison between the original and revised responses offers valuable insights for evaluation. Orthogonal to previous work that only focuses on the discrimination of L.L.Ms, REViSEVAL stands out by fully utilizing the generative potential by revision.
We conduct comprehensive experiments to validate the effectiveness of our proposed REVISEVAL VAL. Using both proprictary and open-source LLMs, REVISEVAL consistently achieves better performance compared to reference-free and referrecc-bbsed evaluation paradigms in both NLG tasks and open- ended instruction-following tasks. Moreover, we seek to verify the effectiveness of the response- adapted references in the classic metrics, e.g.. BERT and BERTScore, showing that each metric exceeds itself by up to 3%-10% accuracy compared to using direct response as references. We then combine L.Le--e--rrevieerr with multiple classic metrics and find that it outperforms (reference-free) LLM-as-a-Judge by over 1.5% on average when using weak LLMs and is comparable when using GPT-4. Finally, we analyze how our paradigm achieves overall superiority. In reducing verbosity and positional bias, our approach offers clear advantages in adversarially designed LLMBar and swapping position testing. Merely increasing inference cost of reference-free evaluation still lags behind REVISEVAL, demonstrating our method's 's efficiency does not rely on naively accumulating cost. Meanwhile, we validate the relationship between reference relevance and efficiency using response-adapted references.
2 RELATED WORK
2.1 EVALUATION OF LARGE LANGUAGE MODELS
Instruction-tune -unded LLMs (Ouyang et al., 2022; Dubey et al., 2024; Team et al., 2023) have revo- lutionized the field of NLP due to their ability to handle a wide range of language-related tasks. Unlike traditional NL. tasks, such Machine Translation and Summarization, which can be eval-
2

---

Published as a conference paper at ICLR 2025
uated using N- N-gram-based metrics like BLEU (Papineni al., 2002), ROUGE (Lin, 2004), and METEOR (Banerjee & Lavie, 2005) by comparing responses with reference texts, LLMs excel at open-ended text generation tasks (e.g., story generation and instruction-followwe where no single reference response exists. Consequently, several studies embrace the potential of L.MM-as-a-Judge and shift toward reference-free metrics, advocating the abandonment of conventional reference-based evaluation methods (Sheng et al., 2024; Chen et al., 2023). In this paper, we revisit the significance of reference in LLM evaluation. Furthermore, to address the challenge of the absence of a single standard answer in certain evaluation tasks, we propose leveraging LLMs to generate aaponse-adappeed references, thereby improving the performance of both traditional metrics and LLM-as---Juude.
generation).
2.2 LLM-AS-A-JUDGE
Recent progress in NLP has introduced model-based evaluation metrics like BERTScore (Zhang et al., 2020) and BARTScore (Yuan al., 2021). However, these methods also depend on the availability of human-annotated references, which can be expensive, time-consuming. and labor-intensive (Zheng al., 2023). With the emergence of large language models (LLMs), several studies (Zheng et al., 2023; Dubois et al., 2024) have harnessed their robust evaluation capabilities for assessing natural language generation (NLG), particularly by employing proprictary models like GPT-4 (OpenAl et al.. 2024). To avoid information leakage caused by external API calls, some efforts advocate finctuming LLMs with evaluation data to obtain evaluator models (Vu et al., 2024; Li et al., 2024a; Wang et al., 2024c; Kim et al., 2024b). A wide variety of techniques are used to enhance the performance of LLLaaa--a-Judge, such as Chain-of-Thought (CoT; Wei et al. (2022)) to first generate concise reasoning and then the final decision, adding pre-defined rules (Zeng et al., 2024) in prompts to list some general rules for LLM-as-a-Jaaa a-ddge to follow explicitly, and swapping the two responses to avoid positional bias (Wang al., 2024a). Zheng et al. (2023) found that, even with the of a CoT prompt, LLM-as-aaJuude can still be misled by the surrounding context, particularly by erroneous response text. Therefore, they propose a setrrence-guided method where the --u--ss---udge's response is first generated independently based on the given instruction and then presented as a reference answer within the evaluation prompt. To the best of our knowledge, we are the first to generate response-adapted references from both the instruction and the response to be evaluated.
3 METHODOLOGY
In this section, we introduce our novel evaluation paradigm, REVISEVAL, which enhances the evaluation by generating response-adapted references. Illustrated in Figure 2, REVISEVAL consists of two components, response-adapted reference generation and reference-based evaluation, which we will discuss in Sec. 3.1 and 3.2, respectively.
Supposing is the response generated by a model for a given task instruction r, REVISEVAL assesses the quality of y on a specific rubric a. Firstly, in the generation phase, conditioned on (x, a), REVISÉVAL deploys a LLM reviser R to revise y to generate response- response-adapted reference r*. Secondly, in the evaluation phase, taking the (æ, a, y) and generated x* as input, REVISEVAL adopts LLM-aas---Jude Fr to assess y using r" as the reference. Besides, we further expand REVISEVAL to support traditional reference-based based metrics FM. The evaluation objective to ensure that the automated evaluations align closely with human evaluations, which is introduced in Appendix G.
3.1 RESPONSE- ADAPTED REFERENCE GENERATION
LLMs have already demonstrated their surprising revision capabilities in various tasks, including improving specific attribution (e.g., writing style and grammar) of passages (Gao et al., 2023). correcting hallucination (Akyurek et al.. 2023), post-editing the generated story (Yang et al., 2022) and generating higher-quality revised responses to complement preference pairs for DPO (Guo et al., 2024; Yoon et al., 2024; Jiang et al., 2024b; Xu et al., 2024). Thus, unlike previous works treating LLMs as discriminators (Hu al., 2024), we leverage the revision capabilities of LLMs to unlock the generative potential to offer richer and more valuable insights for evaluation. REVISEVAL deploys LLMs to revise the response y from the instruction x on a evaluation rubric o,
R(y (1(12,,0)) a),
(1)
3

---

Published as a conference paper at ICLR 2025
Instruction
Human Reference walked we
Similarity Soore
a ressenable exdig
METEOR
the filleninngg story.
evecyone at the party be started orughing and choking everyone pontcked
Sourro: my yoar old cousin was
ooting apple..
Human
Matric Far
BARTSorE....
Rating
Evaluation Prompt
Response
be brought apples from his class be drepped the apples he began firow them back up had hard line
Response-daapted
Jase and rallair
Reference x"
bnstrettinn saara
LLM-as-a- Reviser R
IIs had brought spoles from class, but actocenially dragood them, Struggling, ie attomppodd to pick tiem and in the process.
LM-ai-a- Judge Ff
eealattion staale fallode nérle: (arrards))
Pairwise Comparison
Revision Prompl
yeu nobuutin Eetpet wisa the feferarce sellowtice
gesrrted by
dicplaped restsdnn swold focas (e:narte)......
(i) Response-adapted Reference Generation
Reference-based Evaluation
Figure 2: Illustration of our proposed REVISEVAL. Given instance (z, a), we use REVISEVAL to assess y in rubric a. REVISEVAL, (i) reviser generates apponse-adaples reference by revising the y to enhance the (ii) following LLM-as-a-Judge, even classic text metrics. Here, represents retained segments during the generating response-adapted reference process.
where r" is the generated response-adapted reference for the subsequent evaluation. Notably, when REVISEVAL must give the preference on two responses, V1 and 1/, in pairwise comparison, we randomly select one for primary text to be revised while using the other as revision guidance. We remind reviser that this revision guidance may not be perfect and should be used with caution the revision prompt,
R(
if = ifC 2
(2)
(((22l11,
where C is random variable that decides which response is chosen for revision. C can be either 1 or 2, with each having an equal chance of occurring. Notably, we discuss other possible revision strategies in Appendix 1.2, which are less effective comparably. Introducing a qualified reference can reliably guide the evaluation process, for instance, acting as an "anchor" to reduce biases. Our strategy, which incorporates revision guidance and randomly sampling one as the primary text for revising, further reinforces faimess.. This will be validated in Sec. 4.2 and 4.5.
3.2 BAAEE EVALUATION
In the evaluation phase, REVISEVAL supports LLM-as-a-Judge Fø in a reference-based setting and remains compatible with previous metrics FM-
LLM-as-a-Judge. With powerful generalization capabilities, LLMs can serve as discriminators for evaluation, referred to as LLM-as-a-Judge, In this case, the evaluation can be operated using FF.
Fe(yla,a,)")
This can be easily accomplished by simply using a general prompt for inference, where we clarify the instruction, response, response-adapted reference, and evaluation rubric in the prompt, as shown in Appendix A. Moreover, we implement this for open-source LLMs through finetuning evaluation data in this task format, with the detailed process provided in Appendix C.
Metrics. Once we get the reference r", we can also implement classic metrics, regardless of statistical n-gram model-based metrics. The evaluation score is:
S = FM r r"I [l)..))).
(4)
More specifically, when FM are n-gram metrics, e.g., BLEU, ROUGE, and METEOR, can directly compute the similarity between and r*; when Fa are model-based metrics, e.g., BERT and BARTScore, optionally input x: and to the metrics. The effectiveness of metrics heavily relies on the reference. When the essponse-adaptee reference is appropriate, even simple metric can revive its evaluation functionality in open-ended tasks. We validate this point in Sec. 4.3 and 4.4.
4

---

Published as a conference paper at ICLR 2025
Table Kendall (7) and Spearman (p) correlation results comparing reference-free, reference-based, and REVISEVAL methods across natural language generation tasks. This table demonstrates that, without numan-annotarl references, our proposed REVISEVAL substantially outperforms reference- free and reference-based methods involving both open-source and proprietary LLM--as-aJuddee
SUMMARIZATION
TRANSLATION
DATA2TEXT
STORY GENERATION
Avg. T/p
Methods
7lp
7/p
T/p
rlp
N-gram Metries
BLEU
10.66/14.42
14.50/19.73
23.13/33.29
-1.93/-2.70
11.59/16. 19
ROUGE
10.81/14.85
13.19/17.83
24.74/35.49
-1.53/2.3
11.80/17.63
METEOR
12.37/16.72
16.52/18.80
25.58/36.27
-1.87/-2.65
13.15/17.29
Model-hased Metrics
BeRT
17.50/23.83
31.57/42.41
30.74/43.75
16.00/23.79
23.95/33.4
BARTScore
29.12/355.5 .50
7.01/12.83
22.32/34.33
14.15/33.44
18.15/29.04
UniEval
35.89/47.52
16.08/21.00 1,00
28. 58/556/88.8
31.22/44,46
27.94/38.07
GPTScoree
6.5008.90
16.16/23.91
17.72/24.76
InstructScore-7B
20.86/38.66
40.44/50.43
30.21/38.54
13.50/16.13
26,25/35.94
TIGERScore-7B
28.79/35.
33.65/4
32.4 32.44/42.39
29.72/39.
31.15/39.5
8B-Inst
19.59/23.54
28.46/36.24
26.13/29.99
25.42/30.19
Open-Source LLM-as-a-Judge
Ref-Free
30.84/38.66
38.75/49.32
25.74/31.77
30.79/37.0
Ref-Based
34,09/39.53
35.76/41.12
39,24/50.87
8.79/10.44
29.47/35.4 .49
REVISEVAL (Ours)
32.41/37.73
33.14/39.66
39.02/49.92
25.95/32.11
32.63/39.86
Proprictary L.M Judge
Ref-Free
31. 82/38.989 98
34, 04.62/43,,3 33
37 11.99999.550 .50
23.8 23.81/33.22
32.06/41.8 2.29
Ref-Based
32.56/40.01
41.47/45,29 40.72/45.32
37.35/49.0
17.58/24.86
32.24/39.88
REVISEVAL. (Ours)
33.63/41.15
37.90/50.93
25.1 2/351/3
34.34/43.17
EXPERIMENTS
In this section, we first present the comprehensive experimental settings Sec. 4. and evaluate the REVISEvAL in LLM-as-a-Juude across various tasks in Sec. 4.2; then verify the effectiveness of classic text evaluation metrics when using response-adapted references in Sec. 4.3; building on above findings, compare two evaluation paradigms, combining L.LM-as-a-revisee with multiple metrics and LLM--as-a-uddge, when using weak LLM in Sec. 4.4; finally, we conduct detailed comparative analysis of REVISÉVAL in Sec. 4.5, such bias reduction, inference and reference relevance.
4.1 EVALUATION SETTINGS
Evaluation benchmarks. We evaluate our approach on multiple classic NLG benchmarks by measuring the correlation between the evaluators/metrres and human annotations in a scoring rating task. We follow the experimental setting of Jiang et al. (2024a) and select four representative NLG tasks and corresponding benchmarks: Data-to- test-to-Texttee (WebNLG), Machine Translation (WMT-22 (zh- en)), Text Summarization (SummEval), and Story Generation (OpenMEVA), and Table 10 shows the details of these benchmarks. Additionally, we test our approach on the more challenging open- ended cnatructton-followwee benchmarks (MT-Bench, Alpacafarm, and LLMBar), which primarily rely on pairwise comparison task. Unlike the NLG benchmarks, these preference benchmarks contain general instructions covering broader range of tasks with more diverse responses and use accuracy to measure the evaluation performance. Remarkably, in NLG tasks, "ref-based" evaluation relies on numan-annotated references as foundation. contrast, open-ended instruction- following tasks lack pre-existing references, so "ref-based" evaluation uses machine generated responses as references conduct ablation studies verifying the effectiveness of ours references.
machine-generated
Base LLMs and metrics. Our proposed REVISEVAL aims to improve evaluation performance across both LLM-as-a-Judge and classic metrics, offering enhanced results. For proprietary LLMs, we adopt GPT-4 as the base model and focus implementing our paradigm during the inference stage. For open-source LLMs, we implement our method via finetuning the Llama 3.B model. Following the Jiang et al. (2024a)'s setting open-source models, distill the evaluation outputs generated by GPT-4 when inputting task instructions and corresponding evaluated responses, and tune them in models. Notably, training data has overlap with evaluation benchmarks. For
5

---

Published as a conference paper at ICLR 2025
Table 2: Accuracy of LLM-as-a-Judge on instruction-followimn preference tasks. Our proposed REVISEVAL considerably enhances the performance of both open-source and proprietary L---- Judge across various general evaluation tasks. Here, D.R. denotes Direct Response to instruction.
Methods
# of Training Samples
MT-BENCH
ALPACAFARM
LLMBAR
Arg.
Open-Source Luudee
Ju-geI.M--B (Zhu al., 2023)
100,000
64.1
53.9
36.3
51.4
Pu (Wang et al., 2024c)
300,000
75.0
54.9
31.7
53.9
Auto-J-13B (Li et 2024a)
4,396
75.2
64.6
36.0
58.6
Prometheus-7B (Kim et 2024a)
100,000
52.8
33.5
30.1
Prometheus-2-78 (Kim al., 2024b)
300,000
55.0
37.3
26.3
39.5
Llama 3.-e8B-Tnned
-Ref-Free
9,800
67.4
61.1
-Ref-Based (Llama-D.R.)
9,800
74.9
61.5
58.9
65.1
Reef-Baed (GPT-4-D.R.)
9,800
78.0
65.5
63.0
68.8
REVISEVAL (.lama-ns-a-Revieer
9,800
75.2
64.7
57.8
65,9
REVISEVAL (GPT-4-a---Revvaa
9,800
79.3
67.1
64.9
70.4
Proprictary Judge (GPT-4)
Ref-Free
81.2
70.9
72.6
74.9
Ref-Based (G..T--D.R.)
81.5
67.7
79.9
76.4
REVISEVAL. Revieesr))
83.0
72.9
79.0
78.1
classic metrics. cover various metrics, like n-gram based BLEU, ROUGE, METEOR and model based BERTScore, MOVERScore (Zhao et al., 2019), BARTScore, which rely heavily on references. We validate our method by assessing the utility of reference texts.
We ensure that our approach maintains versatility and fairness across models, with further details on prompts, inference, finetuning, and baselines in the Appendix A, B, C, E and F.
4.2 ENHANCING LLM-AS-A-JUDGE PERFORMANCE
We present the main results of our proposed REVISEVAL. across NLG evaluation tasks and instruction- following preference benchmarks Table and We summarize the conclusions below.
REVISEVAL achieves stronger performance across various NLG tasks. For the powerful pro- prietary LLMs, REVISEVAL outperforms reference-free and numan-annotater reference-based eval- uation across tasks with approximately 0.02 in Kendall correlation on average, as demonstrated in Table 1. Notably, in story generation, high-quality human references hinder evaluation for LLM-as-a Judge, which decreases by 0.06 compared to the reference-fres free method in the Kendall correlation. In contrast, REVISEVAL VAL shows that generated essponse-adapted reference can still greatly enhance evaluation by about 0.08 in Kendall than human reference- based evaluation. An exception is machine translation, where REVISEVAL aligns closely with reference-basee methods, and we analyze this result exists a consistent rationale about reference relevance in Sec 4.5.
For the open-source LLMs, REVISEVAL not only outperforms the reference-free method but also beats the reference- feesee methods by over 0.03 on average in Kendall correlation. Especially in story generation, the reference-bart based approach consistent with the above conclusion, trailing by approximately 0.17 in Kendall compared to our REVISEVAL. Furthermore, the LLM with specialized finetuning also performs better than the LLM with general instruction finetuning (i.e., Llama 1-8 Inst) NLG evaluation tasks, leading by about 0.02 in Kendall.
REVISEvAL excels on open-ended mnstruction-followinn preference benchmarks. As shown in Table 2, whether implemented on open-source or proprictary models, REVisEvAL consistently surpasses all baselines by at least 6.3% on average. The details of these baselines are listed in Appendix F, and they are tuned with tens of thousands of data for the LLM-as-a-Judge, On the same base LLM, REVISEVAL exceeds reference-free evaluation by 3%-6%. We compare REVISEVAL to reference-based evaluations followed by Zheng et al. (2023)'s setup, whose reference is the LLM's direct response to the instruction. Our approach performs better than reference- based evaluation on average when both references generated by the same base LLMs. Furthermore, using GPT-4 the reviser boosts Llama 3. 3.1-8B-as-a-Bee a--asee by over 4.5% compared to Llama Haama-as-a -s-e--a---iissss Re-eisser, highlighting the importance of reference quality.

---

Published as a conference paper at ICLR 2025
Sensearization
Story Gencralson
MI-Bonch
Alpacalurm
1010
Cerbitte
40000
40700
Figure 3: Comparative analysis of reference- metrics performance using references generated by HUMAN/GPT-4 and REVISEVAL on NLG and instruction following benchmarks. REVISEVAL greatly enhance traditional ref- based metrics, even achieving them comparable to GPT-4-as-a-Judee
LLM-as-a-Judee can be biased toward longer, verbose answers (Saito al., 2023; Dubois al., 2024) or answers that match similar format (Huang et al., 2024). LLMBar (Zeng et al., 2024) is challenging benchmark meta-evaluate such superficial quality biases. As demonstrated in Table these baselines, even after tuning > 100K samples, struggle to exceed 50% accuracy, exposing bias challenge. By using aeppose-adaatee references from REVISEVAL, the weak open-source LLM-as-a-Judge' performance improves substantially by about 6%, showing REVISEVAL can address superficial quality bias. On proprietary LLM, REVISEVAL achieves a 3.2% improvement in accuracy compared to reference- free evaluation. Our proposed response-adapter references perform slightly worse than direct responses from the same model. This is likely because LLMBar emphasizes mmnsrructionfoolowiin precision," where single golden response exists for the instruction.
In summary, REVISEVAL consistently outperforms traditional ref-free and res-based methods, and revision provides guidance for LLM evaluation by utilizing the generative advantages of LLM.
4.3 INVESTIGATION ABOUT RESPONSE-ADAPTIE REFERENCES
Revision Quality. As a post-editing mechanism to en- hance text quality, we expect the revision to correct errors within the text, including subtle errors (e-g., grammar), as well as higher-level issues (e.g., instruction following precision and factual/logical correctness). LLMBar is appropriate testbed for conducting detailed case studies and quality assessments. We directly use LLM to score the quality of the responses to be evaluated and the cor- responding response-adapted adapted references, focusing on cor- rectness. Table 3 demonstrates that our response-adapted references effectively improve response accuracies. Ad- ditionally, the case studies are presented in Appendix J.1; we test REVISEVAL on JudgeBench (Tan et al., 2024) in Appendix K, focusing on factual/logical correctness and on RewardBench (Lambert et al., 2024) Appendix N, for more challenging domains.
Table 3: Mean rating for responses and generated references in LLMBar. The detailed rating is in Table 14. Here, Re- sponses I and 2 correspond to the posi- tion of the response in each pair, and the rating range is in [1, 5].
Text
Mean Rating
Response Response
3.19
3.25
References (Ours)
4.75
References Effectiveness. Traditional reference-based metrics rely heavily on references, directly validating whether the references are effective. Therefore, we use response-adapted references with these metrics to observe evaluation performance across NLG and cestruction-oolowwiin benchmarks. We extend classic reference- based metrics to support pairwise comparisons by using an indicator function to determine preference between y1 and // based on their respective metric scores. We use benchmarks
human references in NLG tasks and GPT-4 direct responses in pnstruction-folloviii as the baseline references, comparing them with ours. As shown Figure 3, REVISEVAL enables each classic metric to substantially surpass its performance based on baseline references across all tasks, particularly in more complex open-ended tasks like story generation and AlpacaFarm. Additionally, using references generated by GPT-4 asva-Reviser combined with classic metrics can yield comparable evaluation performance to GPT-4's reference-Fie evaluation. Furthermore, we test response-adapted references in multi-referrnce setting, with results detailed in Appendix I. and M.
7

---

Published as a conference paper at ICLR 2025
Table 4: Comparative analysis of weak LLM-as-a-Judge and weak LLM--ssa-Revier-chassi metrics on instruction-following tasks. Under the same finetuning training resources, a weak LLM- Reviiee combined with classic metrics can produce better results.
Metries
MT-BENCH
ALPACAFARM
LLMBAR
Avg.
LLM-as-a-
BLEU
64.5
63.9
51.6
60.0
ROUGE
62.0
63.5
51.6
59.0
METEOR
66.4
67.7
46.3
60.1
SERT
62.3
62.3
54.4
59.7
MOVERScore
61.5
68.3
51.6
60.5
BARTScore
66.9
60.0
MAJORITY VOTING
63.4
68.5
52.5
61.4
LuM-ae-a-Judge
67.4
61.1
51.1
59.9
4.4 POTENTIAL EVALUATION PARADIGM FOR WEAK LARGE LANGUAGE MODELS
We observe that 1) in the Sec. 4.2, all weak LLMs still exhibit notable gap compared to GPT-4- as-a-Judge, even after extensive training with high-quality, evaluation -ppecii data, and 2) in the Sec. 4.3, classic metries combined with response-adapted references generated by GPT-4 can achieve performance close to the reference-free GPd---as-a-Judge Thus, should we consider a potential evaluation paradigm of "weak LLM-as-a- + metrics" instead of "weak LLMs-as-a-Judge "
To explore this, we compare lllma-as-a-Judges with "Llavi-as--RReviser + metrics," as shown in find that BLEU, METEOR.
Table When using references generated by "Llama-as-a-Revierr MOVERScore, and BARTScore can surpass "Llama-as-a-Judge" on average across tasks. Further- more, we apply a majority voting across multiple metrics, outperforming "Llama-as as----udgeee over 1.5% on average. This suggests that instead of continuously training weak LLMs to improve their evaluative discrimination capabilities, leveraging their generation abilities revision may more effective. Without extra inference costs, this approach can lead to better evaluation outcomes. We offer a general guideline: majority voting is recommended as it provides a stable superior evaluation approach, recognizing that no single metric outperforms others across all benchmarks.
Table 5: Positional bias analysis in pair comparison evaluations when applying different evaluation paradigms. This table presents the ratio changed evaluation results after swapping the response position. A lower proportion indicates less positional bias. REVISEALL stands out as the best, exhibiting the lowest bias among all paradigms.
Paradigms
MT-BENCH
ALPACAFARM
LLMBAR
LLAMA 3.B
GPT-4
LLAMA 3.1-8B
GPT-4
LLAMA 3.1-8B
GPT-4
Ref-Free
49.1
10.3
61.1
20.0
44.6
17.9
Ref-Based
22.8
6.5
34.1
22.2
32.5
11.2
REVISEVAL
20.5
5.9
30.1
19.9
30.3
7.9
4.5 COMPARATIVE ANALYSIS TO OTHER EVALUATION PARADIGMS
Positional bias analysis. Positional bias (Wang et al., 2024a: Zheng et al., 2023; Wu & Aji. 2023; Chen al., 2024) occurs when human LLM evaluators tend favor one side in pairwise comparison, regardless of answer quality. We investigate this bias by swapping answer positions and taking the LLM to re-evaluate, shown in Table 5, The results indicate that eeference-basee evaluation decisions have less variation than reference-free ones. REVISHVAL, is generally 2%-4% lower on the reference-basee evaluation, showing better consistency. This result probably because REVISEVAL. provides references more closely aligned with the answers, further minimizing bias.
The impact analysis of inference cost. Compared to the reference- free approach, REVISEVAL requires two cycles of inference (i.e., revision and evaluation). To show the impact of extra inference cost, we further conduct three cycles of reference-frr evaluations using extra different temperatures (0.3 and 0.7), followed by majority voting (see Table 6). For GPT-4, additional evaluation cycles
8

---

Published as a conference paper at ICLR 2025
Table 6: Ablation study on the impact of inference cost. Increasing evaluation cycles match or exceed REVISEVAL's inference cost in reference-free did not improve accuracy. This shows that REVISEVAL'S 's superior performance is not from twice inference.
MT-BENCH
ALPACAFARI
LLMBAR
Inference Cost
LLAMA 3.1-8B
GPT-4
LLAMA 3.1-8B
GPT-4
LLAMA 3.1-8B
GPT-4
1-Cycle Ref-Free
67.4
81.2
61.1
70.9
51.1
72.6
3-Cycle Ref-Free
64.1
81.2
54.9
71.9
53.2
74.9
REVISEVAL (2-Cycle)
71.3
83.0
64.7
72.9
54,9
79.0
Table 7: Comparative analysis of how reference- evaluation effectiveness varies with changes in the similarity between the response and reference texts across different constructing reference strategies. Here, Effectiveness, Preee refers to the performance ratio between reference-basee and reference-free free evaluation, where denotes the evaluation performance, e.g., Acc and Cor:; similarity is still measured by BERTScore.
Reference Source
WMI--2(EN-Z88)
WEBNLG
MT-BENCH
SUMMEVAL
ALPACAFARM
ROC
Similarity
Humun/GPT-4
RevisEival
63,03 (-3.3%)
65.12
76.41 (+293%)
59,09
30.29 (+21.2%)
25,00
35.63 (+51.6))
23.51
35.72 (+13.99)
13.04
27.57 (+11.48))
12:86
Effectiveness
HumanKiPPT-4
1.02
0.9
RevisRual
1.18 (-0:02)
(10.02)
1,02 (L0.0))
1,06 (40.04)
1,33 (10.00)
(-033)
slightly improve accuracy but still lag behind ours by 1%-4%. For weak LLMs, more cycles led to worse performance. The above results indicate that our approach provides more valuable guidance.
Evaluation performance improves with increased relevance between reference and responses. It's been observed that the less relevant a reference is the response, the less effective for evaluation in previous work and Figure 1. We further verify whether this trend holds true with our method. We define effectiveness describe whether reference- based evaluation more effective than reference-free evaluation. As shown in Table 7, the increasing similarity between reference and evaluated responses generally leads to better evaluation effectiveness. This explains why our method doesn't 't perform well in translation. where human references already highly similar to the response. For other tasks, human or GPT-4 direct-response references have lower similarity than references of REVISEVAL. leading to lower effectiveness. Additionally, for different tasks, the similarity between the human/GPTT4 reference and the evaluated response varies, reflecting the open-ended generative degree of this task. A lower similarity indicates greater diversity of potential valid responses. In this context, as the task becomes more open-ended, the effectiveness of REVISEVAL shows a greater improvement than the human/GPT-4 reference.
5 CASE STUDY: HOW DOES IT WORK?
We show representative examples in Table 8, one from the instruction-followinng task (Alpaca- Farm) and the other from the NLG evaluation task (story generation). The reference-free evaluation tends to be influenced by verbose text, often leading incorrect judgments, e.g., "provides clearer answer...". ". While the LLM's direct response human reference might not be erroneous and fulfill the instructions well, for instance, "Happy and Joyful belong to the wordgrropp of synonyms, in the first case, it has no relevance to responses to be evaluated and resulting "neither assistant directly addressed the classification," judgment. In contrast, the reference generated by REVISEVAL is more adaptive guidance, "Emottios", accurately helping the LLM to align the crucial elements, "the clarity and completeness". This is also evident in the second case, where differences between the response and the revised text, such as "had brought," "accidentally dropped." and others, directly highlight fiunncy issues with the response. In other words, this demonstrates transparent potential. Furthermore, we provide evaluation discrepancy statistics between REVISEVAL VAL and other evaluations to observe how different evaluation methods, demonstrated in Appendix I.3.

---

Published as a conference paper at ICLR 2025
Table 8: An mnstruction-following open-ended case of pairwise comparison and story-generation case of score rating evaluated by different paradigms. Score/Preference refers to accurate evaluation result and Score/Preference refers to wrong result. We emphasize the noisy evaluation elements in red, while highlighting the useful elements of the evaluation in green. Here, the base LLM is GPT-4.
Instruction: Classify following to the correct word group. lnkn Happy. Joyful
Response
Response
Huppy Emotions
Emotions
Reference-free Evaluation:
Both Assistant and Assistant ..... Both responses are correct, one might argue that including the words with the categorization, as Assistant A does, provides clearer answer and specifically indicates that hoth wonds have been clarsified...
Final Verdict: [[A]]
Referfence-hased Evaluation:
eeeeeeeeeee Direct Response-ay- aa-nsee----lae Reforerencee Hoggy and Joyful belong to the word gooe of synommm,, as they both exgress similar feelings of
pleasure and contentment.
Assistant A, while correct in stating that "Happy" belongs to "Emotions," did not mention "Joyfal" or classify them together as synonyms, Raiher, only assigned cutegory to one word.
Assistant ...... but fails to classify both words as instructed, missing the relationsip between "Happy" and "Joyfal."
Neither assistant directly addressed the classificution "synonymss." which was central the user's instruction.
Final Verdiet: [[A]]
REVISEVAL Evaluntion:
Response-adapted Reference: Emotions
However, it is clear from the classification provided that Assistant understands the context and correctly Gilegorizes "Happy" but overlooks "Joyful"
Assistant A
Assistant Bt simply responds with "Fmoriios"" effectively covering both "Happy" and "Joyful" This response uses more concise and generalized approach, which aligns perfectly with the reference answer.
Based on the reference answer and the clarity and completeness in addressing both terms, Assistant B's response is superior.
Final Verdict [B]]
Instruction: Generte reasomable ending for the following story. In Source Input: my year old cousin eating apple.
Response: he brought apples from his class. he dropped apples. he began throw them back up. he had hard time. Gold Score: 3.8
Reference-free Evaluation:
but it lacks oeherence and logical flow. The sentences disconnected, and the events descrihed huild upon each other in meaningful way. The output does not demonstrate a clear understanding of the story's context or the character's actions. Rating: [[]]]
Referfence-based Evaluation:
Human Reference: walked around between everyone at the partu, he started coughing and choking. peickeeee and started soceusetio spit out spple piece
him "eo
The model-geneerated output not fluent and lacks colirence in comparison to the reference outpoti.
Overall, model generated output meet the fluency criteria due to of coherence and grammatical emons.
Rating: [[]]]
REVISEVAL Exaluation:
Response-adaptel Reference; He bowggtt ggoles his class, accidentally drageed them. Singeling, ansmpted pick cove
be so wereon oo time mangging soples.
them up and the process bogao tossing them arowed, He was haviog
The model generated output somewhar fluent lacks coherence and completeness in compurison the reference output. The reference outpat provides a more detailed and logical sequence of events, enhancing the narrative's is cnherence and clarity.
model-generated Overall, the model- generated output fluent but could benefit from improvements in coberence and detail to match the quality of the reference outpat.
Rating: [3]
CONCLUSION
We introduce a novel evaluation paradigm that utilizes LLMs' revision capabilities to generate response-adapted references, enhancing LLM-as-a-Judge's reliability. The references produced by REVISEVAL significantly improve even simple n-gram metrics, achieving performance on par with LLMM-as-aJJudge. This is particularly beneficial for weaker LLMs, which often struggle to improve despite extensive training, offering an efficient way to enhance evaluation capabilities despite limited training. Our findings emphasize (1) the underestimated importance of references and (2) the potential of LLMs' generative strengths to improve evaluation by increasing reference relevance. Looking ahead, REVISEVAL enables promising extensions: (i) integrating a Reviser with classic metrics to better support smaller LLMs, (ii) applying revision mechanisms to mudai tasks like MLLM, and (iii) incorporating REVISEVAL. into multi-agent pipelines for robust evaluation frameworks.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENTS
This work was supported by the Early Career Scheme (No. CityU 21219323) and the General Research Fund (No. CityU 11220324) of the University Grants Committee (UGC), and the NSFC Young Scientists Fund (No. 9240127).
REFERENCES
Afra Feyza Akyurek, Ekin Akyurek, Ashwin Kalyan, Peter Clark, Derry Tanti Wijaya, and Niket Tandon. RL4F: Generating natural language feedback with reinforcement learning for repairing model outputs. In Annual Meeting of the Association for Computational Linguistics, pp. 7716- 7733, 2023.
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds Danny Hernandez, Tristan Hume, Scott Johnston, Shauna Kravec, Liane Lovitt, Neel Nanda, Catherine Olsson, Dario Amodei, Tom Brown, Jack Clark, Sam McCandlish, Chris Olah, Ben Mann, and Jared Kaplan. Training helpful and harmless assistant with reinforcement learning from human feedback. In arXiv, 2022.
Satanjeev Banerjee and Alon Lavie. METEOR: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pp. 65 15-72,25 2005.
Yupeng Chang, Xu Wang, Jindong Wang. Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al. A survey evaluation of large language models. ACM Traasactionss on Intelligent Systems and Technology, pp. 1-45, 2024.
Guiming Hardy Chen, Shunian Chen, Ziche Liu, Feng Jiang, and Benyou Wang. Humans or Ilms as the judge? study judgement biases. In arXiv, 2024.
Yi Chen, Rui Wang, Haiyun Jiang, Shuming Shi, and Ruifeng Xu. Exploring the use of large language models for reference-free text quality evaluation: An empirical study. In arXiv, 2023.
Daniel Deutsch, Rotem Dror, and Dan Roth. the limitations of reference-free evaluations of generated text. In Conference on Empirical Methods in Natural Language Processing, pp. 10960- 10977, 2022.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang. Angela Fan, et al. The Ilama 3 herd of models. In arXiv, 2024.
Yann Dubois, Balázs Galambosi, Percy Liang, and Tatsunori B. Hashimoto. Length-controlled alpacaeval: A simple way to debias automatic evaluators. In arXiv, 2024.
Alexander R. Fabbri, Wojciech Kryšciński, Bryan McCann, Caiming Xiong, Richard Socher, and Dragomir Radev. SummEval: Re-evaluating summarization evaluation. Transactions of the Association for Computational Linguisties, pp. 391-409, 2021.
Markus Freitag, David Grangier, and Isaac Caswell. BLEU might be guilty but references not innocent. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), Conference on Empirical Methods in Natural Language Processing, pp. 61-71, 2020.
Bofei Gao, Feifan Song, Zhe Yang. Zefan Cai, Yibo Miao, Qingxiu Dong, Lei Li, Chenghao Ma, Liang Chen, Runxin Xu, Zhengyang Tang, Benyou Wang, Daoguang Zan, Shanghaoran Quan, Ge Zhang, Lei Sha, Yichang Zhang, Xuancheng Ren, Tianyu Liu, and Baobao Chang. Omni- MATH: A universal olympiad level mathematic benchmark for large language models. In arXiv, 2024a.
11

---

Published as a conference paper at ICLR 2025
Luyu Gao. Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, and Kelvin Guu. RARR: Researching and revising what language models say, using language models. In Annual Meeting of the Association for Computational Linguistics, pp. 16477-16508, 2023.
Mingqi Gao, Xinyu Hu, Jie Ruan, Xiao Pu, and Xiaojun Wan. Llm-based nlg evaluation: Current status and challenges. In arXiv, 2024b.
Fabrizio Gilardi, Meysam Alizadeh, and Maël Kubli. Chatgpt outperforms crowd workers for text-annotation tasks. Proceedings of the National Academy of Sciences, 2023.
Carlos Gómez- Rodriguzz and Paul Williams. A confederacy of models: a comprehensive evaluation of LLMMs creative writing. In Findings of the Conference Empirical Methods in Natural Language Processing, pp. 14504-1452 14528, 2023.
Jian Guan and Minlie Huang. UNION: An Unreferenced Metric for Evaluating Open-ended Story Generation. In Conference Empirical Methods in Natural Language Processing, pp. 9157-9166, 2020.
Jian Guan, Zhexin Zhang, Zhuoer Feng, Zitao Liu, Wenbiao Ding, Xiaoxi Mao, Changjie Fan, and Minlie Huang. OpenMEVA: A benchmark for evaluating open-ended story generation metrics. In Annual Meeting of the Association for Computational Linguistics, pp. 6394-6407, 2021.
Geyang Guo, Ranchi Zhao, Tianyi Tang, Xin Zhao, and Ji-Rong Wen. Beyond imitation: Lever- aging fine-grained quality signals for alignment. In International Conference on Learning Representations, 2024. URL https::///...... //wpr//iiee/ereee.e./.ooo.. net -ow.te/oorumiii--II
Xinyu Hu, Mingqi Gao, Sen Hu, Yang Zhang, Yicheng Chen, Teng Xu, and Xiaojun Wan. Are LLM-based evaluators confusing NLG quality criteria? Annual Meeting of the Association for Computational Linguistics, pp. 9530-9570, 2024.
Hui Huang. Yingqi Qu, Hongli Zhou, Jing Liu, Muyun Yang, Bing Xu, and Tiejun Zhao. On the limitations of fine-tuned judge models for I1m evaluation. In arXiv, 2024.
Tianbo Ji, Yvette Graham, Gareth Jones, Chenyang Lyu, and Qun Liu. Achieving reliable hu- man assessment of open-domain dialogue systems. In Annual Meeting of the Association for Computational Linguistics. pp. 6416-6431 2022.
Dongfu Jiang, Yishan Li, Ge Zhang, Wenhao Huang, Bill Yuchen Lin, and Wenhu Chen. TIGERScore: Towards building explainable metric for all text generation tasks. Transactions on Machine Learning Research, 2024a. ISSN 2835 8856. URL https: https::/openeeeteeeeee /ttp///eeeeeee.e////// net/ net/forunt forum?
SIFEE1CBKCOS2
Yuxin Jiang, Bo Huang, Yufei Wang. Xingshan Zeng, Liangyou Li, Yasheng Wang, Xin Jiang, Lifeng Shang, Ruiming Tang, and Wei Wang. Bridging and modeling correlations in pairwise data for direct preference optimization. In arXiv, 2024b.
Seungone Kim, Jamin Shin, Yejin Cho, Joel Jang, Shayne Longpre, Hwaran Lee, Sangdoo Yun, Seongjin Shin, Sungdong Kim, James Thorne, and Minjoon Seo. Prometheus: Inducing fine- grained evaluation capability in language models. In International Conference on Learning Representations, 2024a. URL htttt::://operevviii nete ott.eerrrriiiieeeefoouooooooooeo
Seungone Kim, Juyoung Suk, Shayne Longpre, Bill Yuchen Lin, Jamin Shin, Sean Welleck, Graham Neubig, Moontae Lee, Kyungjac Lee, and Minjoon Sco. Prometheus 2: An open source language model specialized in evaluating other language models. arXiv, 2024b.
Tom Kocmi, Rachel Bawden, Ondřej Bojar, Anton Dvorkovich, Christian Federmann, and etal. Findings of the 2022 conference machine translation (WMT22). In Conference Machine Translation, pp. 1-45, 2022.
Nathan Lambert, Valentina Pyatkin, Jacob Daniel Morrison, Lester James Validad Miranda, Bill Yuchen Lin, Khyathi Raghavi Chandu, Nouha Dziri. Sachin Kumar, Tom Zick, Yejin Choi, Noah A. Smith, and Hanna Hajishirzi. Rewardbench: Evaluating reward models for language modeling. In arxiv, 2024.
12

---

Published as a conference paper at ICLR 2025
Junlong Li, Shichao Sun, Weizhe Yuan, Run-Ze Fan, hai zhao, and Pengfei Liu. Generative judge for evaluating aligmment. In International Conference on Learning Representations, 2024a. URL
httpp:://ppnreeview net/ htt.:/oorummiiiitfa5s522222
Zhen Li, Xiaohan Xu, Tao Shen, Can Xu, Jia-Chen Gu, Yuxuan Lai, Chongyang Tao, and Shuai Ma. Leveraging large language models for nig evaluation: Advances and challenges. In arXiv, 2024b.
Chin-Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pp. 74-81, 2004.
Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. G-eval: NLG evaluation using gpt-4 with better human alignment. In Conference on Empirical Methods in Natural Language Processing, pp. 2511-2522, 2023.
Shikib Mehri and Maxine Eskenazi. USR: An unsupervised and reference free evaluation metric for dialog generation. In Annual Meeting of the Association for Computational Linguistics, pp- 681-707, 2020.
OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, and etal. Gpt-4 technical report. In arXiv. 2024. URL https: /ttppss///axxvv.oo abs/2303. 08774.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang. Sandhini Agarwal, Katarina Slama, Alex Ray, al. Training language models to follow instructions with human feedback. In Advances in neural information processing systems, pp. 27730-27744, 2022.
Liangming Pan, Michael Saxon, Wenda Xu, Deepak Nathani, Xinyi Wang, and William Yang Wang. Automatically correcting large language models: Surveying the landscape diverse automated correction strategies. Transactions of the Association for Computational Linguistics, pp. 484-506, 2024.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Annual Meeting of the Association for Computational Linguistics, pp. 311-318, 2002.
Ricardo Rei, Ana C Farinha, Chrysoula Zerva, Daan van Stigt, Craig Stewart, Pedro Ramos, Taisiya Glushkova, André T. Martins, and Alon Lavie. Are references really needed? unbabel-IST 2021 submission for the metries shared task. In Conference Machine Translation, pp. 1030-1040 2021.
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA: A graduate-level google-proof q&a benchmark. In Conference on Language Modeling, 2024.
Keita Saito, Akifumi Wachi, Koki Wataoka, and Youhei Akimoto. Verbosity bias in preference labeling by large language models. In arXiv, 2023.
Shugian Sheng. Yi Xu, Luoyi Fu, Jiaxin Ding, Lei Zhou, Xinbing Wang, and Chenghu Zhou. Is reference necessary in the evaluation of NLG systems? when and where? In Conference of the North American Chapter of the Association for Computational Linguisties, pp. 8580-8596, 2024.
Sijun Tan, Siyuan Zhuang, Kyle Montgomery, William Y. Tang. Alejandro Cuadron, Chenguang Wang, Raluca Ada Popa, and Ion Stoica. JudgeBench: benchmark for evaluating llm-based judges. In arXiv, 2024.
Tianyi Tang, Hongyuan Lu, Yuchen Jiang, Haoyang Huang, Dongdong Zhang, Xin Zhao, Tom Koemi, and Furu Wei. Not all metrics are guilty: Improving NLG evaluation by diversifying references. In Conference of the North American Chapter of the Association for Computational Linguistics, pp.
6596-6610, 2024.
13

---

Published as a conference paper at ICLR 2025
Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean- -aat-Battste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. Gemini: a family highly capable multimodal models. In arXiv, 2023.
Tu Vu, Kalpesh Krishna, Salaheddin Alzubi, Chris Tar, Manaal Faruqui, and Kun-Hsuan Sung. Foundational autoraters: Taming large language models for better automatic evaluation. arXiv, 2024.
Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Lingpeng Kong. Qi Liu, Tianyu Liu, and Zhifang Sui. Large language models are not fair evaluators. Annual Meeting of the Association for Computational Linguistics, pp. 9440-9455 2024a.
Tianlu Wang, Ilia Kulikov, Olga Golovneva, Ping Yu, Weizhe Yuan, Jane Dwivedi- Yu, Richard Yuanzhe Pang, Maryam Fazel-Zarandi, Jason Weston, and Xian Li. Self-taught evaluators. In arviv, 2024b.
Yidong Wang. Zhuohao Yu, Wenjin Yao, Zhengran Zeng. Linyi Yang, Cunxiang Wang, Hao Chen, Chaoya Jiang. Rui Xie, Jindong Wang, Xing Xie, Wei Ye, Shikun Zhang, and Yue Zhang. PandaLM: An automatic evaluation benchmark for LLM instruction tuning optimization. In International Conference on Learning Representations, 2024c. URL https: /tpp////eee.....oe/mmmo
Jason Wei, Xuezhi Wang, Dale Schuurmans. Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et Chain-of-thoughe prompting elicits reasoning large language models. Advances in neural information processing systems, pp.
2022.
Minghao Wu and Alham Fikri Aji. Style over substance: Evaluation biases for large language models. In arXiv, 2023.
Dehong Xu, Liang Qiu, Minseok Kim, Faisal Ladhak, and Jaeyoung Do. Aligning large language models via fine-grained supervision. In Annual Meeting of the Association for Computational Linguisties, pp. 673 680, 2024.
Wenda Xu, Danqing Wang. Liangming Pan, Zhenqiao Song, Markus Freitag, William Wang, and Lei Li. INSTRUCTSCORE. Towards explainable text generation evaluation with automatic feedback. In Conference on Empirical Methods Natural Language Processing, pp. 5967-5994, 2023.
Kevin Yang, Yuandong Tian, Nanyun Peng. and Dan Klein. Re3: Generating longer stories with recursive reprompting and revision. In Conference on Empirical Methods in Natural Language Processing, pp. 4479, 2022.
Eunseop Yoon, Hee Suk Yoon, SooHwan Eom, Gunsoo Han, Daniel Nam, Daejin Jo, Kyoung- Woon On, Mark Hasegawa-Johnson, Sungwoong Kim, and Chang Yoo. TLCR: Token-level continuous reward for fine- tinegrrined reinforcement learning from human feedback. In Findings of the Association for Computational Linguistics, pp. 14969-149999 14981, 2024.
Weizhe Yuan, Graham Neubig, and Pengfei Liu. BARTScore: Evaluating generated text as text generation. In Advances Neural Information Processing Systems, pp. 27263-27277, 2021.
Zhiyuan Zeng, Jiatong Yu, Tianyu Gao, Yu Meng, Tanya Goyal, and Danqi Chen. Evaluating large language models at evaluating instruction following. In International Conference on Learning Representations, 2024. URL https: http:///ppee/iiieeeeeee
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations, 2020. URL fettps:/copenreet
net forum?id-sSeeuucvEr
Wei Zhao, Maxime Peyrard, Fei Liu, Yang Gao, Christian M. Meyer, and Steffen Eger. Mover- Score: Text generation evaluating with contextualized embeddings and earth mover distance. In Conference Empirical Methods in Natural Language Processing, pp. 563-578, 2019.
14

---

Published as a conference paper at ICLR 2025
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging LLM-as-a-judge with MT-bench and chatbot arena. In Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023. URI. met/fformm dducccHPGD
Giulio Zhou and Gerasimos Lampouras. WebNLG challenge 2020: Language agnostic delexicali- sation for multilingual RDF-to-text generation. In International Workshop on Natural Language Generation from the Semantic Web, pp. 186-191, 2020.
Lianghui Zhu, Xinggang Wang, and Xinlong Wang. Judgelm: Fine-tuned large language models are scalable judges. In arXiv, 2023.
15

---

Published as a conference paper at ICLR 2025
Please act as impartial judge evaluate the quality of the responses provided by two AI assistants
user question displayed below. You should choose the assistant that follows the user's instructions and answers the user's question better. Your evaluation should consider factors such as (rubric: the helpfulness, relevance, accuracy, depth, creativity, and level of detail of their responses). Begin your evaluation by comparing the two responses and provide short explanation. Avoid any position biases and ensure that the order in which the responses
presented does not influence your decision. Do not allow the length of responses to influence your evaluation. not favor certain the assistants. as objective as possible, After providing your explanation, output your final verdict by strictly following this format: "[[A]]" if assistant is better, "[[B]]" if assistant is better.
[Instruction]
(instruction)
[The Start of Assistant Answer] (response_output_1)
[The End of Assistant A's Answer]
[The Start of Assistant B's Answer]
(response gesponse_output_22 2]
[The End of Assistant B's Answer]
Figure 4: The prompt of reference-free pairwise comparison evaluation.
Please act as an impartial judge and evaluate the quality of the responses provided by two AI assistants to the user question displayed below. You should choose the assistant that follows the user's instructions and answers the user's question better. Your evaluation should consider factors such the {rubric: helpfulness, relevance, accuracy, depth, creativity, and level of detail of their responses).
also give a reliable reference answer, and begin your evaluation by comparing the two responses with the reference answer and provide short explanation. Avoid any position biases and ensure that the order in which the responses were presented does not influence your decision. Do not allow the length of the responses to influence your evaluation. Do not favor certain names of the assistants. Be as objective possible. After providing your explanation, output your final verdict by strictly following this format: "[[A]]" " if assistant is better, "[[B]]" if assistant is better.
[Instruction]
(instruction)
[Reference Answer]
[ref_answer]
[The Start of Assistant A's Answer]
(response_t) __1} [The End of Assistant A's Answer]
[The Start Assistant B's Answer]
(response 2} [The End of Assistant B's Answer]
Figure 5: The prompt of reference-based pairwise comparison evaluation.
A PROMPT TEMPLATE
We provide the prompt templates used for evaluation and revision. These prompts are either taken directly from MT-BBench adapted from it, ensuring the universality of our proposed paradigm.
16

---

Published as a conference paper at ICLR 2025
Please act as impartial judge and evaluate the quality of the model model-generated output provided by AI assistant to the instruction with source input displayed below. Your evaluation should consider the following aspect: (rubric). Begin your evaluation by providing short explanation. Be objective possible. After providing your explanation, please rate the response on scale 1 5 by strictly following this format: "[[rating]]", for example: "Rating: [[3]]".
Instruction: (instruction)
{input_ contuxt)
Model-gemerated Output: (response_output)
Figure 6: The prompt of eeference-free rating evaluation.
Please act as an impartial judge and evaluate the quality of the response generated by an AI assistant to the instruction with source input displayed below. Your evaluation should consider the following aspect: (rubric}.
Begin your evaluation by comparing the Model-generated Output with the Reference output and provide short explanation. Be objective possible. After providing your explanation, please rate the response on scale to by strictly following this format: "[[rating]]", for example: "Rating: [[3]]".
Instruction: (instruction)
(input_contexx)
Model-generated Output: (response_outpput
Reference output: (ref_output)
Figure 7: The prompt of reference based score rating evaluation.
Please act as a powerful reviser to revise the response generated by an AI assistant to the instruction displayed below. You should revise the response to follow the user's instructions and answer the user's instruction better. Your revision should consider factors such as the (rubric). If the original response is good enough, simply output the original answer.
**Instrucion: (iintruuctio))
#Model-Generatee Response: ** (response_outputt
I also give another model-generated answer, which is not necessarily of reference for your revision, and you can draw its strengths and avoid its weaknesses. ** **nnother ** (respossseoooooppprrr
better quality,
Do NOT provide any explanation for your response.
ONLY output the complete revised answer without saying anything else.
Figure 8: The prompt LLM-as-a-reviser for pairwise comparison.
17

---

Published as a conference paper at ICLR 2025
Please act as a powerful reviser to revise the response generated by an AI assistant to the instruction and source input displayed below. Please revise this output to be more (rubric). If model-generate output is already good enough, simply output that original output. innitruuctii:iinnsnttutttttt (input_context)
=*Model-generate, Output:* (rrepoonse_uutput)
Do NOT provide any explanation for your response. ONLY output the complete revised answer without saying anything else.
Figure 9: The prompt of LLM--a-a-reviser for score rating.
(instruction)
Figure 10: The prompt of LLM direct response instruction.
Table The Statistics of NLG Evaluation Training Data.
Task
Aspects
Samples Items
Evaluation Items
Summarization
fuonyyy.ossiccneeooorrrreeeeeeee accuracy
2886
11544
Translation
6000
6000
Data2Text
accuracy.fluengg
3098
6196
Story Generation
fluemcy.consistencyy.yylll matching
1052
3156
B INFERENCE SETTING FOR PROPRIETARY MODEL
Base Model. We choose GPT-4 as the base model for our evaluation and revision. For repro- ducibility, used the GPT-4 version T0TT---TRR--2024-0044 with temperature setting of 0.0.
C FINETUNING SETTING FOR OPEN-SOURCED MODEL
Base Model. We choose LLAMA 3.1-8B 1.1-8B-INST as the base model for our evaluation and revision. Here, we want to clarify that we choose the INSTRUCT model as the base model because finetuning on this model yields better evaluation and revision results than the PRETRAINED model.
Training Setting, We followed the common setup for supervised instruction finetuning, with context length 2048, epochs 3, batch size 128, and learning rate 2e
Distilling Setting. Whether finetuning open-source models for evaluation revision capabilities, the training data comes from the generation of a powerful model prompted by the same data source. The model choose distill is still GPT-4, with the same version and inference settings as mentioned above.
Distilling Data Source. We depict the distilling data flow in Figure 11. For NLG evaluation, ideally, would have variety of erroneous samples along with human evaluation scores for them. However, such data typically exists only in test sets, making it unavailable for training and often limited in quantity. Therefore, choose MetricInstruct 2, proposed by Jiang et al. (2024a),
ama.3.3mmm3mmmmmmmmmm -- 1 Instuct
htttps://hugiiingffac http:///ggggnaaaeeec///aaaaaaaaa
18

---

Published as a conference paper at ICLR 2025
Propriety L.M (Inference)
Opn (finctuning)
Instruction
Instruction
Revise
Respanse A/B
Evaluate
Diseal
Response
RevisEval reviser
Response
evaluation
Response adaptivve
Respons-adxptivee
OOO
RevisEval evaluator
Response
Metrielastrvet
Instruction
Instruction
Revise
Respanse
Evaluabe
Response
Finetune
ReviEEval reviser
Response
respone-adaptivv
evaluation
OO0
Repoonie-dxppiie reference
RevisEval eveluator
ReviÉval evalustion
Figure 11: In our data distillation process for open-source LLMs, we utilize HH-rlhf and MetricIn- struct the primary data sources. We then employ proprictary LLM to perform RevisEval, generating both the revisions and corresponding evaluation outputs. Finally, we fine-tune the open- source LLM using this enriched dataset.
training data source. This dataset provides a large volume of diverse erroneous texts, which serve as the basis for evaluation. From the 40K*+ data points, we filter out other NLG tasks and apply our previously mentioned prompts with corresponding aspects, maintaining the same inference settings to generate evaluation scores and reasoning for these error samples. Detailed statistics are presented in the Table 9. Although the overall dataset size relatively small compared to other works specifically designed to train evaluators, the NLG evaluation data assess remains held-out from these training samples.
Unlike NLG evaluation data that lacks human- evaluation, preference data typically contains substantial
preference annotations without evaluation. We choose the most commonly used hh-rlhf (Bai al., 2022) dataset, applying the aforementioned prompts and inference settings to conduct evaluations on this data to get the evaluation. We select the preference correctness intersection of reference- free evaluation, reference-basee based evaluation, and gold preference annotations, ensuring both accuracy and fairesss when comparing the performance of LLMs under different evaluation methods post-traine trstitraining.
the end, we selected 000 samples, each containing corresponding revisions, reference- free evaluation, and reference- evaluation. These three sets were then used train the same model separately, ensuring that no new information is introduced to alter the distribution.
Finetuning LLM-as-a-Judde adaptive to NLG. Each instance in filtered Metriclnstruct is tuple (response, input context, task instruction, aspect), then use GPT-4 get the corresponding (reference-Fie ffrencee-feeeeeeeo evaluation, revision text, REVEVO evaluation) to each instance. Subsequently, separately use these data to finetune LLM to get the reference-free evaluator, LLM-as-a-reviser, and reference-based evaluator.
Each instance in filtered hh-rlhff a tuple (instruction, response a, response b), then use GPT-4 to get the corresponding (reference- ffre evaluation, revision text, REVEvO evaluation) each instance. Subsequently, we separately use these data to finetune LLM to get the reference- evaluator, a.L.Maa-a-reviierr, and reference-based evaluator.
Decoding Setting. When the finetuned model executes the evaluation and revision tasks, the de- coding setting uses a greedy decoding strategy with a max output length = 1024 and temperature = 0.01.
19

---

Published as a conference paper at ICLR 2025
Table 10: The Statistics of NLG Evaluation Benchmarks.
Task
Benchmark
Response Source
Inputs Items
Samples Items
Summarization
SummEval (Fabbri al.. 2021)
16 Models
100
1600
Translation
WMT-22 22 (zh-en) (Kocmi ct al., 2022)
18 Models
1875
33750
Duta2TTxi
WebNLG G-2020 (Zhou Lampouras, 2020)
18 Models
179
2848
Story Generation
OpenMEVA (ROC) (Guan al., 2021)
5 Models
200
1000
D BENCHMARKS
D.1 NLG EVALUATION BENCHMARKS
Traditional text generation tasks and their corresponding evaluation benchmarks are highly diverse. Based on the varying degrees of freeform in text generation tasks, we select four representative Machine Translation, Data-to-Text, Summarization, and Story Generation. We follow the experiment setting of Tigerscore and choose specific benchmarks for each task, and their statistics are shown the Table 10.
D.2 INSTRUCTION FOLLOWING PREFERENCE BENCHMARKS
With their powerful generalization capabilities, LLMs have become the focus of research NLP generation abilities. Evaluating LLMs requires more challenging tasks that can assess their gen- eralization capabilities. Here, we selected three representative benchmarks: MT-BENCH (abbr., MT-BENCH BEEANCGHHMMANJJUDGGAAATT
ALPACAFARM, and L.LMAR..
MT-Bench. This dataset comprises 3.3K expert-level pairwise human evaluations of model re- sponses, generated by six models across 80 MT Bench questions. The six models include GPT-4, GPT-3.5, CLAUDE-V1, VICUNA- B3, ALPACA- and LLAMA- resentation of powerful language models. The topic of subtasks is consisted of Writing, Roleplay, Reasoning Math, Coding, Extraction, STEM and Humanities. MT-Bench is the most common bench- mark for evaluating LLM-as-a-Judge, and our results validate the feasibility of our experiments. We select the first round of dialogues from this dataset as the evaluation data, containing 1284 cases.
offering a diverse rep-
set specifically designed for evaluating the reliability of evaluators, following the ALPACAFARM process. Fach instance in this dataset contains cross-annttations from 4 human experts. Additionally, the tasks in this dataset are more diverse, open-ended, and challenging, making the preference annotations more reliable. Notably, since four experts conduct the preference annotations, some instances resulted in ties, where two annotators favored the first option and the other two favored the second. We filtered out these tied cases, leaving a final evaluation dataset of 501 instances.
AlpacaFarm. We utilize
LLMBar. LLMBar is meta-evaluation benchmark designed to test how well LLM evaluators can identify instruction-following outputs. It consists of two parts: (1) The Natural set, which gathers instances from existing numan-preferee for each instance. (2) The Adversarial set, where the authors intentionally create misleading outputs that superficially seem good but deviate from the instructions, challenge the evaluators. The Natural set measures performance in weal-world conditions, while the Adversarial set tests evaluators' ability detect true cnrrructio-follwiing The overall size 419.
datasets, filtered and adjusted ensure clear preference
E BASELINES IN NLG EVALUATION TASKS
N-gram Metrics. N-gram text generation metrics are commonly used evaluate the text quality generated by models, especially in tasks like machine translation and summarization. While these metrics are simple and efficient, they come with notable limitations: they are highly sensitive to
Pttpps: http://ggggg..eeeee/aa/aa/tt/ata http:///ggggg..a...oo.ooaaatttts hbtp://aaaalllaaeaaaaaaaaaaaaa paca_eva 1/b1 /b1 ob/ma / 11paaa_faat www.:aaahmaannorrssaaoooooooooaa
abbaa
jaon
20

---

Published as a conference paper at ICLR 2025
Table 11: The Lists of weak LLM-as-aaJudge.
Model
Base Model
Instruction
Response Annotation
Evaluation Scheme
Training Samples
Alpuca GPT4, Dolly-15K...
models
Judgel.M
Vicuna 7B
(Alpaca. Vicuna...)
GPT-4 Pairwise Grading
100K
Pundal.M
ILaMA-7B
Alpaca 52K
(LLaMA, Bloom...)
models
GPT3.5 Pairwise Selection
300
Chathot Arena, OpenAl WebGPT...
Human Pairwise Selection,
Auto-J
HaMMA--13B-cha
Preference Datasets
Pointwise Grading
4396
Prometheus
LLaMA2-7B-chal
GPT-4 Generated
GPT- Generated
GPT4 Pointwise Grading
100K
GPT-4 Pointwise Grading
Prometheus-2
Mistral-7R-v2.0
GPT-4 Generared
GPT-4 Generated
300
surface-level differences, such as word order or vocabulary choice, which may fail to capture the true meaning or fluency of the generated text. In our evaluation, we use the most widely adopted metries: BLEU, ROUGE-L, and METEOR.
Model-based Metrics. To capture the semantic-level meaning of generated text, researchers have started using models like BERT and BART as the foundation for text evaluation. Typical examples include BERTScore, BARTScore,, and Moverscore. BERTScore computes the similarity between two text sequences based on the contextual embeddings from BERT, while Moverscore enhances this by adding many-to-one alignment. BARTScore, on the other hand, uses BART to calculate the probability of converting the response to the reference text as score. All of these are reference-based model metrics.
Additionally, there are reference-free model-based metrics. These metrics are trained on specific task datasets, allowing the model to internalize relevant information. As a result, the model can generate evaluations without needing reference texts and transform them into scores. For instance, UNIEVAL uses data augmentation expand a task-specific dataset to 30K examples and fine-tunes on T5, which is why performs exceptionally well in summarization tasks.
LL---a--a-Judge Following the exciting advancements large language models (LLMs), most straightforward approach has been to replace the models in previous model-based metrics, such as BART, with larger models. GPTScore follows this concept, and despite its simplicity. delivers notable results. Moreover, leveraging the vast internal knowledge of open-source LLMs, more powerful and interpretable evaluators can be developed, such as INSTRUCTScore and TIGERScore.
Xu al. (2023) argue that performing error analysis given reference texts enhances evaluation explainability and reliability. They used NLG evaluation data and employed GPT-4 to perform error-based assessments. These outputs were then paired with the response train Llama model, resulting in a training dataset of 10K examples. TIGERScore goes a step further by proposing a reference-free approach. Using similar strategy, they collected 40K data points for training, masking the reference text during the process.
F BASELINES IN INSTRUCTION FOLLOWING PREFERENCE BENCHMARKS
LLM-aa--aJuudeer
The baseline models are listed in the Table 11. Considering scalability and cost, researchers have long sought to achieve evaluation performance on weaker L.MMs that is comparable to that of stronger LLMs. The most straightforward approach to this challenge has been to automatically generate more preference-rllate data, and many these efforts have followed this strategy.
N-gram Metrics & Model-based Metries. Unlike the NLG-Evaluation benchmark, the LLM-as- Judge benchmark has largely moved away from using n-gram metrics and model-based metrics. This shift is due to several characteristics of the test samples in such benchmarks:
(1) The response space is extremely large and unconstrained, making reference annotations both unhelpful and prohibitively expensive; (2) These metrics do not perform well for tasks such as coding or math, where even models like BERT struggle capture semantic-level meaning. In this study, we apply these metrics
21

---

Published as a conference paper at ICLR 2025
Table 12: Pearson correlation coefficients comparing non-reference, static-reference, and dynamic- reference methods across various text generation tasks and Instruction-Folloiing Benchmarks. This table summarizes the performance of these methods in generating summarization, translation, data- to-text, and story-generation tasks.
Methods
SUMMARIZATION
TRANSLATION
DATA2TENT
STORY GENERATION
Avg.
D-gram Metrics
BLEU
14.13
17.47
34.29
-3.89
15.50
ROUGE
15.36
16.26
35.85
-0.22
16.81
METEOR
18.69
18.80
36,30
-1.02
18.19
Reference- Metrics
BERTScore
26.26
37.65
48.22
26.58
34.68
BARTScore
19.73
29.04
47.89
17.76
28.61
UniEval
53,22
23.11
51.14
43.09
GPTScore
13.47
21.05
48.70
18.94
25.54
InstructScore-7B
27.40
51.55
47.28
12.81
34.76
TIGERScore-7B
43,95
37.70
49.13
39.90
42.67
Llama-3.1 8B-Instruct
25.89
27.84
31.15
31.04
28.98
Open-Sourced L.M-as-a-Juuhe
Ref-Free
33.61
14
$3.36
35.02
36.78
Ref-Based
42.32
29.99 99
48.52
11.92
33.19
REVEvO(OUrs)
39.69
29.58
53.87
29.04
38.05
Proprictary LLM-as-a-Judge
Ref-Free
42.12
41.35
54.26
33.50
43.56
Ref-Based
43.3
44.11
53.98
24.63
41.51
REVEvo(Ours) 43.81
43.92
54.25
35.07
44.26
to demonstrate that, when reference texts are highly relevant, these otherwise "inapplicable" metrics can be reactivated and produce meaningful results.
G META EVALUATION
Meta-evaluation aims to assess the performance of automated metrics by measuring how well the automated evaluations Yauto align with human evaluations Nhuman For score ratings, we calculate the correlation values across all N samples, represented as:
Corr (ly U [y (uuuman
Fruman))
where can adopt various correlation functions, e.g., Spearman. For pair-wise comparison evaluations, accuracy is typically used as the evaluation metric.
Acc
N Eitoo M[vinoo p²]
NLG Tasks. In the NLG evaluation task, it's crucial evaluation process. For the four selected tasks, we' ve outlined the specific rubrics to be evaluated in the accompanying Table In both the SummEval and Story-Generation tasks, we evaluate multiple rubrics independently, calculating the correlation coefficient for each one. Subsequently, we compute the average correlation coefficient across all rubrics to obtain an overall assessment for each task. This comprehensive approach ensures more nuanced and accurate evaluation of the model's performance across different dimensions.
various rubrics of the text during the
H PEARSON CORRELATION IN NLG EVALUATION TASKS
We also supplement the Pearson Correlation results in the NLG Evaluation Tasks.
22

---

Published as a conference paper at ICLR 2025
Table 13: Statistics on the differential proportion between REVISEVAL and each of the other two evaluation methods. A higher ratio indicates a greater evaluation difference between the two mecha- nisms.
Comparative Evaluation
MT-BENCH
ALPACAFARM
LLMBAR
Ref-Free
8.1
22.8
16.9
Ref-Based (GPT-4 Direct Response)
9.7
21.6
16.2
I OTHER ANALYSIS
I.1 THE BENEFITS OF TRAINING RESOURCE SCALE FOR LIM-AS-A--UDCH ARE QUESTIONABLE.
As shown in Table and 11, Despite being trained with extensive evaluation-speecffc resources, these LLM--as--- L-M-as-a-Judge baselines fail to achieve evaluation performance comparable to GPT-4, particularly on the adversarially designed LLMBar, where they perform worse than random selection. While substantial effort is put into designing and generating a large amount of training data for these LLMs, the results are even less effective than our evaluator trained on just 10,000 samples from the hh-rlhf dataset. The possible reasons for this could be: 1. The inherent capabilities of the base model play more crucial role; 2. Simply increasing the volume of training data does not yield significant benefits; 3. Efforts should be focused on other potentials to enhance the evaluator, such the method we propose.
1.2 OTHER REVISION STRATEGIES FOR PAIRWISE COMPARISON
When revising two given responses to generate a reference, we experiment with two different revision strategies. In work, adopt strategy where text is randomly selected as the primary text to be revised, while the other serves as the revision guidance. In addition to this, we try the following two prompt strategies: a) Revising single text based both responses. b) Revising each response separately. However, the outcomes of these two strategies are unsatisfactory. Strategy (a) exhibit a tendency forcibly merge the two responses during the revision process, resulting in a generated text that lacked logical consistency. Strategy (b), on the other hand, lead to a revised text with very low similarity the other response, inevitably favoring the one chosen for revision. Neither method is capable of producing a satisfactory reference. We anticipate that future research might provide more revision strategies approaches to more effectively combine multiple responses and generate high-quality reference. We look forward to seeing further developments in this area.
1.3 EVALUATION DIFFERENTIAL DIFFERENT METHODS
We analyse the differential between REVISEVAL and two other evaluation methods, showcasing the proportion of samples where the evaluation decisions differed. higher proportion indicates greater difference in the evaluation mechanisms of the two methods. The Table 13 present that REVISEVAL and the other two evaluation methods make different decisions on 22% of the samples in AlpacaFarm and 16% of the samples in LLMbar. This indicates that our evaluation method has compared to the other two methods. Furthermore, suggests aggregating these differing decisions could potentially lead more reliable final evaluation.
a significant difference in mechanism-
THE CORRECTNESS STUDY OF RESPONSE-ADAPTER RESPONSE
Revision is a post-edit modification method aimed improving the quality of the response. The ability to correct subtle errors in the text is key factor we observe to determine the effectiveness of the revision. Experiments on LLMBar and BLEU/ROUGE provide relevant evidence to support this to some degrees: i) LLMBar Benchmark: the performance of the judge on LLMBar relies its ability to detect subtle errors in instruction following. As shown Table REVISEVAL enhances the LLM- as-a-Judge's performance
LLMBar, demonstrating that our references effectively validate these errors. ii) BLEU/ROUGE: as N N-gram metrics, completely depend on references to identify subtle emors. In Sec. 4.3 and 4.4, we input the references generated by REVISEVAL. into BL BLEU/OOUEE to
23

---

Published as a conference paper at ICLR 2025
evaluate their effectiveness. By computing accuracy and the correlation human scores, show that our references successfully capture and validate these errors. Although our existing method has demonstrated advantages in both LLMBar and classic n-gram metrics, will provide more direct study and analysis here.
J.1
CASE STUDY
On LLMBar, we present response samples modified by REVISEVAL.
Question:
Convert from celsius to fahrenheit. Temperature in Celsius: 15
Response: 7700b0F
Response-Adapted Reference: 5900b0F
Sort them in lexicographical order: nnapples, grapes, bananas, oranges, kiwi, pears, plums, oranges, cherries, apples, tangerines, peaches
Question:
Response: apples, bananas, cherries, grapes, kiwi, oranges, peaches, pears, plums, tangerines
Response-Adapted Reference: apples, apples, bananas, cherries, grapes, kiwi, oranges, oranges, peaches, pears, plums, tangerines
We can observe that compared with the response, the adapted reference contains fewer subtle errors. This provides the guidance to make llm-as-a-judge casier to evaluate, as containing errors directly affects the quality of the response, leading higher alignment with human evaluators than reference-free evaluators.
J.2 DIRECT QUALITY EVALUATION ON RESPONSE-ADAPTE REFERENCES
We directly score the quality (1-5) of the adapted references on the correctness aspect compared to the original responses using LLM-as-a-Judge
Table 14: Comparasive Correctness Rating to the original responses and response cespons--adapted references on LLMBar. Here, we directly use a-deeee (GPT-40) to rate the responses and references on correctness.
ADVIRSARIAL NESGEBOR
ADVERSARIAL GPTNST
ADVERSABIAL GPTOUT
ADVERSARIAL MANUAL
NATURAL
OVERALL
Response Heqgonse
3.27
383
3.47
3.39
2.9
3.19
3,16
301
333
3.47
3.25
Rlopune-ulaptel Refermnce
In this Tab. 14, Response I and Response 2 refer to the position of this response in the pairs. On 5 subsets of LLMBar, the references (revised by REVISEVAL wAL---EEEA--------- -ur-gg------ have consistently better correctness than the responses, which means EAL detects and corrects the subtle errors.
K EVALUATION FOCUSING ON FACTUAL ACCURACY
While historical work and our experiments have validated that revision is an effective approach for improving text generation, in this section, further examine whether revision remains effective in modifying text errors in factual correctness.
First, previous studies (Pan al., 2024; Gao et al., 2023) have demonstrated that revision can be applied to correct factual errors, such as hallucination. Here, we provide a more direct experiment: using JudgeBench (Tan et al., 2024), benchmark for LLM-as-a-Judge to directly evaluate factual emrrrs in model-gen generatee text, we test whether REVISEvAL can improve the effectiveness of LLM- as-a-Judge in this context.
24

---

Published as a conference paper at ICLR 2025
Table 15: Comparative Performance on JudgeBench. Here, the metric is Accuracy.
METHOD
KNOLWEDGE
MATH
REASONING
CODING
OVERALL
LLM--as-a-Judge (gpt-4-turbo)
48.05
69.64
56.12
38.09
52.57
RevisEval (gpt-4-turbo)
64.29
64.29
70.41
45.24
63.71
LLM-as-a-Judge (gpt-4o)
53.2
55.4
49.0
35.7
50.3
RevisEval (gpt-4o)
72.7
58.9
66.3
33.3
64.0
I.M--ss---Judee (gpt-4o-mini)
64.3
62.5
65.3
42.9
61.7
RevisEval (gpt-40)
70.8
64.3
63.3
54.8
65.7
Here, JudgeBench includes multiple samples about factual correctness across various domains, and REVISEVAL. demonstrates strong performance evaluating factual correctness on different domains Tab. 15.
MULTIPLE REFINED-GRAINED REFERENCES
In the traditional NLG-Evaluation Tasks, e.g., Machine Translation (Freitag al., 2020), multiple references have been proven to be simple yet effective method to improve the reliability evaluators/metries. And, Tang et al. (2024) further uses LLMs to paraphrase the pre- existing human-annotated references to generate multiple references for the following evaluation. Based on this, we hope to verify whether REVISEVAL. works for multiple references and more refined-graines references, and further extend the applicability of REVISEVAL.
In the original REVISEVAL setting, we revise the response once to generate one reference, wherein reviser prompt, "Your revision should consider factors such the helpfulness, relevance, accuracy, depth, creativity, and level of detail of their responses", we include aspects in one prompt. Now, conduct separate fine-grad tine-graned revisions to generate multiple references, where each reviser prompt only includes one aspect (helpful, accuracy, relevance, depth, creativity), then use them in the following evaluation separately. For each case, will evaluate it based on each reference, and will get multiple predicted preferences or scores. For the preference evaluation task, we will run majority-voting get final preference; for the score-rating evaluation task, we will obtain a mean score. We name this new evaluation pipeline as FINE-GRAINED REVISEVAL, and test its accuracy in the MT-Bench as follows,
Table 16: Performance comparison of evaluation methods on MT-Bench. The results show that FINE- GRAINED REVISEVAL, which incorporates multiple refined-grained references, further improves evaluation accuracy over REVISEVAL and LLM-as-a- auddgee demonstrating the effectiveness of this extended strategy.
METHOD
GPT-4-TURBO
GPT-4O-MINI
LLM-as-a-Judge RevisEval
81.18
80.29
83.01
81.38
Finegrained-Reviss RevisEval
84.13
81.99
As presented this Tab. 16, REVisEVAL. has been improved in the multiple refine-grained references setting, indicating it works for this classic strategy.
M COMPARISON WITH DIV-REF
Both RIV (Tang et al., 2024) and ours leverage references to enhance evaluation performance. However, there are some differences between the two methods:
i) Methodology: DIV-REF diversifies pre-labeling references (through paraphrasing), it cannot support reference-free benchmarks, e.g., MT-Bench, AlpacaFarm. and this work also only tests in NL.G-tasks. In contrast, find that the reason for the ineffectiveness of pre-existing references is
25

---

Published as a conference paper at ICLR 2025
lacking relevance to the response, as shown in Fig 1. Therefore, REVISEVAL proposes to revise the response and create a post-generated reference with higher relevance towards itself than pre-existing ones. Additionally, REVISEVAL supports reference- free benchmarks, while DIV-REE can not.
ii) Performance: To comprehensively validate the effectiveness of the references generated by both mechanisms, we employ classic N-gram metrics (ROUGE), model- metries (BERTSScore and MoverScore), and an LLM evaluator (GPT-4-Turbo, aligned with our version) for testing.
For Tang al. (2024)'s method, diversify pre-existing human-labeles reference ten times to produce ten references, followed by running the reference-based metric separately to get 10 scores and calculating the mean score. For REVISEVAL (ours), do rely human references; instead, revise the response to generate one response-adaplee reference. We then conduct the reference-baat metric obtain a score. For each specific aspect (e.g., fluency), we compute the correlation between the predicted score and human evaluation scores, then average the correlation values across these aspects to derive the final performance score for this benchmark. We compare two methods in SummEval and Story Generation, and the correlation choose Spearman, the results are as below:
Table 17: Comparison of DIV-REF and REVISEVAL on SummEval and Story Generation tasks. REVISEVAL achieves superior results most cases, particularly story generation, highlighting its advantage in addressing the limitations of pre-existing references.
Methods
SummEEal
Story Generation
ROUGE
BERTSCORE
MOVERSCORE
GPT-4-TURBU
ROUGE
BERTSCURE
MOVERSCORE
GPT-4-TURSU
Human-Refeeene
DIV-REF
18.25
28.13
23:47
43.82
1.53
25.79
15.48
27.38
REVISEVAL
19.65
29.47
25.85
41.15
17.24
25.84
26.89
35.26
As we can observe in the Tab. 17, both DIV-REF and our REVISEVAL outperform human references, indicating the effectiveness of both methods. Comparably, method is the best-performed one in most cases, indicating its effectiveness, especially in story generation. In summary, DIV-REF is a piece of solid evidence of the ineffectiveness of pre-existing references. However, REVISEVAL differs from DIV-REF in how we address such ineffectiveness and whether supporting reference-free benchmark. We will incorporate this comparative experiment in the next version.
N PERFORMANCE ON REWARDBENCH
While we have tested on LLMBar, a challenging enough benchmark, to verify this issue, Reward- Bench (Lambert et al., 2024) is the latest popular challenging benchmark covering more challenging domains, such as CHAT-HARD, REASONING, and SAFETY subsets. So, we decide to choose the RewardBench extensive experiment, the result is below.
Table 18: Comparative Performance on RewardBench.
METHOD
CHAT
CHAT-HARD
SAFETY
REASONING
OVERALL
LLM-as-a-Judge (gpt-4-turbo)
97.76
80.04
88.51
90.01
89.04
RevisEval (gpt-4-turbo)
97.76
80.04
88.51
90.01
89.04
LLM-as-a-Judge (gpt-4o)
98.60
79.17
92.03
94.47
91.96
RevisEval (gpt-4o)
97.76
83.55
93.51
95.53
93.47
LLM-as-a-Judde (gpt-4o-mini)
96.37
60.09
91.89
81.21
82,45
vev (gpt-4o)
93.30
65.13
93.08
86.35
85.63
REVISEVAL can improve LLM-ass--Judge consistently on RewardBench. Especially in challenging subsets, REVISEVAL surpasses I.I I.LM-as-a-Judge stably.
26

---

Published as a conference paper at ICLR 2025
O PERFORMANCE ON CHALLENGING REASONING BENCHMARKS
To further explore the potential boundaries of our method, extremely challenging reasoning-basee generation benchmarks serve as an excellent testbed. We introduce new benchmarks, GPQA (Rein et al., 2024) and Omni (Gao et al., 2024a), both of which are state-of-the-art and highly challenging in the field of reasoning.
However, the initial setups of these benchmarks were not designed for evaluating LLM-as-a-Judge Specifically, they lack positive- neggaivee response pairs for the judge discriminate between. To address this, we modify the benchmark setups by using L.MSs to generate positive-negative response pairs for each question.
Negative responses are relatively easy to generate. We used GPT-4o to repeatedly sample and produce incorrect responses.
Positive responses, being harder to create due to the complexity of the questions, were generated through correcting the negative responses with the oracle solution-as-referennes by GPT-4o.
As result, obtain responses with similar styles but differing levels of accuracy.
In this experiment, we adapt GPQA and Omni to test the reliability of LLM-as-a-Judge. The evaluation task assesses whether LLM-as-a-Judge can more accurately select the positive response without access to the oracle answer. We introduce two baselines for comparison, 1) Vanilla: LLM-as- a-Judge directly selects the better response without any reference; 2) Ref-based: LLM answers the question first, and its response is used as the reference.
Table 19: Comparison of LLM-ass--Judde Baselines and REVISEVAL. on GPQA and Omni bench- marks.
GPQA
Methods
Omni
UTQA_EXTENDER
GPQA_ MAIN
GPQA DIAMOND
OVERALL
Vanilla(gpt-to-mini)
53.11
54,69
58.59
54.61
51.99
Rei-basel(gpptto-minn)
52.01
50.89
54.05
51.93
47.38
RevisEvalggp-4oommin)
54.21
53.79
58.59
54.78
53.00
Vanilla(gpt-4o)
67.39
71.21
70.71
69.39
60.86
Ref-based(gpt-44)
59.89
60.26
61.62
60.32
57.99
RevisEval(gpt-4o)
69.23
70.98
75.76
70.91
61.43
Vanilloggpt-4-turbo
70.33
66.52
68.68
68.62
61.11
Ref-based(gpt--tuuro)) 54.58
57.14
56.57
55.79
59.01
RevisEvallgpt-4ttubo)) 71.06
67.19
68.69
69.21
62.37
We draw the following conclusions from the Table 19: i) In the absence of Oracle solution, our method remains effective even on extremely challenging benchmarks. ii) Compared to having the LLM directly answer the question, REVISEVAL provides more effective solution for generating references with LLMs.
P JUSTIFICATION OF USING REVISION AS RELIABLE METHOD TO ENHANCE TEXT QUALITY
The revision process in LLMMs not simple copy operation. We emphasize that our revision mechanism incorporates information from two responses rather than solely one response being revised (shown in Sec. 3). Due to the training objective of LLLMs-LLMs are predominantly trained to generate high-quality, human-like outputs, with limited exposure to the distribution of low-quality or flawed data the probability of retaining correct segments is significantly higher than that of retaining erroneous segments during the revision process. This observation is also utilized by numerous studies (Yang et al., 2022; Akyurek et al., 2023; Guo et al., 2024; Gao et al., 2023) that post-revision LLMs is reliable mechanism improve response quality.
27

---

Published as a conference paper at ICLR 2025
For example, let E represent erroneous segments in the response, and H represent high-quality segments. Consider two responses: R₁: E1. Hz, H3; R2: H2, H3, H₂ and Rg has a higher quality. After revision, the reference R is likely retain H2, H3. H4 while eliminating E₁- Then LLM-as- a-Judge will pick the R2, as R would align more with high-quality segments overall. If the LLM fails to remove and retains it, the resulting R E1; H2, H3, H4 would be equally closer to R1 and R2. Then, inputting the reference adapted from Ri into LLM-as-a-Judee would make no difference from LLM-as-a-Judge without reference.
28


Reviews:
[{"review_id": "JgZog0JBKW", "reviewer": "Reviewer_Rcn4", "summary": "The paper proposes an interesting method “RevisEval” which explores a new approach to performing reference-based evaluation by modifying references based on the responses to be evaluated. The authors show that this improves the reliability of LLM-based evaluators as compared to using static references, by hypothesizing that an effective reference must be closely relevant to the response to be evaluated. Authors show many interesting observations and analysis across various standard NLG tasks as well as open-ended generative tasks and also evaluate various metrics (both standard and LLM-based). Authors also show that these adapted references can even boost the efficacy of standard metrics.", "questions": "1. While the overall paper is well-written, mentioning what the numbers mean in each table and how they have been calculated in the captions or in the text may improve the readability of the paper to a general user. For eg:  mentioning that the values in Table 2 is the accuracy against human preferences... \n2. As mentioned in the weaknesses, please provide details of any experiments that were conducted to study the soundess of this approach for factual responses (where the generated response contains errors which get normalised in the adapted reference).", "limitations": "", "rating": 5, "confidence": 5, "soundness": 1, "presentation": 4, "contribution": 3, "strengths": "1. The paper motivates the problem very well by identifying the issues with current reference-based evaluation paradigms. The idea of dynamically generating contextually relevant references is creative and interesting. It aims to address very important and quite relevant aspects of using LLM as evaluators. \n2. Extensive experiments have been conducted across different tasks as well as various metrics have been evaluated. The authors also show the generalizability of their approach to different metrics.\n3. The paper also considers and accounts for the various biases present in LLM Evaluators and also considers the cost of conducting evaluations (which is often ignored in a lot of works)\n4. Many interesting insights have been reported by the authors, including using these contextually relevant references to improve the standard n-gram and model-based metrics.", "weaknesses": "While I agree with the motivation behind the paper, I am not sure about the soundness of the  methodology followed to generate the reference answers:\n1. Using the response itself to generate an \"adapted reference\", the evaluation might indirectly validate the response’s content and structure. This may lead to artificially inflated evaluations, as the evaluator is essentially comparing the response against a modified version of itself, which serves as the reference.\n2. If the response contains subtle errors, the adapted reference “might” effectively validate or normalize these errors. These is no study around whether the reviser indeed accounts for or corrects for these errors.\n3. While this approach may work well for evaluations of standard NLG tasks as well as some open-ended tasks that care about the language generation capabilities, but for evaluations that care about the factual accuracy of the responses (something where LLMs are overall known to hallucinate), this simple revision may not be robust.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ewNUmOkuOr", "reviewer": "Reviewer_9THu", "summary": "Recently, LLM-as-a-Judge has been gaining popularity for evaluating language generation tasks, but still has a few reliability challenges compared to human evaluation. This paper proposes RevisEval, a text evaluation method that can be used for LLM-as-a-Judge methods as well as more traditional reference-based evaluation metrics, such as BLEU and BERTScore. The core of the method is to use LLMs to revise the response (system output) based on the human reference, called reponse-adapted references, which is then used as a new reference in the downstream evaluation, be it LLM-as-a-Judge or traditional evaluation metrics. Through experiments, the authors showed that 1) the proposed method RevisEval showed improved correlation with gold standard compared to reference-free and baseline reference-based evaluation methods, and 2) on preference tasks, RevisEval outperforms baselines including fine-tuned LLM-as-a-Judge models, 3) the proposed method reduces the positional bias compared to reference-free methods as well as conventional reference-based method.", "questions": "* How would the proposed method work for multiple references? For open-ended text generation tasks, including MT, multiple references are often used.\n* What's the metric used in Table 2? Accuracy?\n* Figure 3 — it looks like which metrics are most effective (and closest to GPT-4 performance) vary based on the specific metrics used. Would you provide some general guidelines which metric(s) are most effective in general when combined with RevisEval? Or simply doing majority voting is a good strategy?\n* In the last paragraph of Section 4.2 — \"significantly\" is used two times. Are they used in a statistical sense? If not, they are simply very subjective adverb and I would advise against using it in this context\n* Same for the section title of 4.3 — what do you exactly mean by \"activating?\" I would rephrase with something simpler, e.g, \"Improving\"\n* What are some examples of future work of RevisEval? The conclusion section only provides the summary of the findings.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "* Simple yet effective method — the core idea of the proposed method, RevisEval, is very simple—simply \"rewrite\" the response based on the human-written reference (and the rubric) and use it as a new reference. The method is also effective for many settings including LLM-as-a-Judge and traditional reference-based metrics. It is easy to imagine that the proposed method is used in evaluation of many NLG tasks going forward.\n* Good ablation studies — the paper provides a wide set of ablation studies to show the proposed method's effectiveness. It shows evaluation results on scoring tasks as well as pairwise preference benchmarks. I also liked the bias analysis (Section 4.5) as well as the detailed analysis of concrete examples (Section 5).\n\nOverall, the paper is overall well written and provides enough evidence that the proposed method is simple, effective, and widely applicable.", "weaknesses": "No major weakness as far as I see. Here are some minor weakness points:\n\n* Unclear names—personally I find \"response-adapted references\" very confusing. It sounds like the method adapt references based on response, but actually it's the other way around. It is actually reference-adapted responses, but I'm not sure if this is a better way of describing it (I don't have any better ideas).\n\n* Unclear description of the experiment settings—the main body of paper benefits a bit of description about the benchmarks. It is based on Tigerscore, but the paper provides very little information re: the specific datasets used and their sizes. Importantly, I think the variety and the quality distribution of responses matter a lot for the evaluation of evaluation methods, and a few sentences about the quantity and the quality of the benchmark datasets would be very helpful.\n\n* Future prospect—this is a bit hypothetical, but the very reason why RevisEval works at all is that current LLMs are in general better at generation rather than discrimination, as the authors state in Section 4.4. Does this mean that in the future, if we have more powerful LLMs at discrimination, would the proposed method still be useful, since future LLMs can simply \"guess better\" using the reference and the response? \n\n* Applicability — the paper already shows experimental results on a wide range of tasks and benchmarks, but I'm suspecting they are all English tasks (only exception is the source sentences of MT, which are in Chinese). It doesn't have to be done in this paper, but it would be valuable to test the effectiveness of RevisEval in a wider range of tasks (e.g., image captioning) and languages.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IhumxBiYgg", "reviewer": "Reviewer_6W2K", "summary": "The work proposes a simple and straightforward evaluation method that involves modifying and enhancing the output text to be evaluated, using it as the reference for further evaluation, motivated by the potentially unsatisfactory quality of traditional references. They experiment with various setups, including using strong and weak LLMs as revisors and employing both traditional evaluation metrics and LLM-based evaluators.", "questions": "Please refer to Weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 2, "strengths": "The proposed method is intuitive and reasonable, with a straightforward implementation that advances previous work using LLMs to generate references for evaluation. They also consider a comprehensive range of experimental setups, baseline methods, and evaluation benchmarks to verify the effectiveness of their method, resulting in solid experimental analyses.", "weaknesses": "Given that previous studies have already utilized LLMs to generate higher-quality references as replacements for traditional references (Tang et al., 2024), the innovation and contribution of this method are somewhat diminished. I believe they could further enhance the analysis by more comprehensively comparing these two approaches for generating references (generation as reference vs. revision as reference). Additionally, I suggest exploring the use of more refined response-adapted references, such as having the revisor focus on specific dimensions during evaluation, to allow for a richer and more diverse discussion.\n\nThe experiments in this work are thorough, but they may be somewhat distracting. First, the main experimental results presented in Tables 1 and 2 involve some inconsistent demonstrations; for example, both tables include the \"Open-Source LLM-as-a-Judge\" part, but the types of methods involved seem different. In Table 1, it’s unclear whether \"Ref-Based\" refers to references generated by the corresponding LLMs or the original references, which is important. And Sections 4.3 and 4.4 may not be as critical and could be moved to the appendix, given the availability of stronger evaluation methods; this would allow space for more in-depth experiments and analysis.\n\n**Reference**\n\nNot All Metrics Are Guilty: Improving NLG Evaluation by Diversifying References (Tang et al., NAACL 2024)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "q9cb9GU0qG", "reviewer": "Reviewer_zE3r", "summary": "The paper proposes a simple solution to enhance reference-based evaluation of LLM-as-a-Judges. Instead of using pre-made references, the work introduces a novel evaluation paradigm, \"Revise-and-Evaluation,\" where an LLM revises the provided input to generate a reference answer. The authors note that this method is effective in creating a reference similar to the response in terms of style and different artifacts, effectively accounting for the quality of the answer only. The methodology can be expanded to classical evaluation methodologies like BLEU, ROUGE, and more. The methodology is tested on diverse benchmarks.", "questions": "1. While the improvements look promising, there are some questions about the effectiveness of the proposed solution. Recent meta-evaluation works like Reward Bench [1] show that reward models are much more powerful than LLM-as-Judges in proxying human responses. Does the proposed methodology have a benefit against RMs? \n\n2. Automated evaluators are also widely used as a proxy for human preference in RLHF. An additional step to generate revisions makes the whole process slower and expensive. Hence, while the performance may be promising, it seems like it limits the usage of automated evaluators. Where do you expect this methodology to be used? \n\n3. Mandating a revision step before evaluation assumes that the revising model can refine the answer better. What if the question-response pair is too difficult for the model to revise? Will the methodology still be effective?\n\n\n[1] https://arxiv.org/abs/2403.13787", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The paper proposes a simple and straightforward solution to improve reference-based evaluation. The methodology is easy to implement and shows promising improvement in quality on different benchmarks. \n\n2. The methodology shows strong robustness, naturally controlling for style, and shows nice performance on adversarial benchmarks like LLM Bar, despite relatively small training.", "weaknesses": "Please see the questions section.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)