ReFeR: Improving Evaluation and Reasoning through Hierarchy of Models
Authors: Yaswanth Narsupalli
Abhranil Chandra
Sreevatsa Muppirala
Manish Gupta
Pawan Goyal
Venue: Accepted by TMLR ()

Abstract: Assessing the quality of generative model outputs from large language models (LLMs) or vision-language models (VLMs), poses significant challenges. Traditional evaluation methods either rely on human assessment which is resource-intensive and not scalable or on automatic metrics that often correlate poorly with human preferences. Another approach is to train dedicated neural evaluators, but this typically requires substantial training data and compute. In this study, we thus introduce ReFeR, a tuning-free framework for evaluating generative outputs including both text and images, using a two-level hierarchy of pre-trained LLM and VLM evaluators. This multi-agent hierarchical strategy leverages additional compute at inference time by orchestrating multiple models and utilizing the increased test-time reasoning to boost performance. By having models themselves provide feedback and final judgments, ReFeR reduces the dependence on human evaluation. We rigorously evaluate ReFeR on four diverse evaluation benchmarks, where it surpasses prior methods in accuracy while also generating constructive feedback useful for downstream distillation and self-improvement via finetuning. Interestingly, ReFeR is also applicable for reasoning tasks - experiments on four reasoning benchmarks show ReFeR’s superior collective reasoning abilities. We present two variants of the framework: ReFeR-Turbo, optimized for accelerated performance, and ReFeR-Lite, offering a more test-time compute efficient solution. ReFeR-Lite is $\sim12-14\times$ more compute efficient than previous works while being comparably accurate to ReFeR-Turbo.

Published in Transactions on Machine Learning Research (08/2025)
ReFeR: Improving Evaluation and Reasoning through Hier- archy of Models
Yaswanth Narsupalli ITT Kharagpur
yosshu. saashha..gsswanthh@@maalccoommm
Abhranil Chandra Universily of Waterloo
abtrmuil.chendrru@@maai.ccc
Sreevatsa Muppirala IIT Kharagpur
sreevotso220022@mmaa..oooo
Manish Gupta Microsoft, India
gmanissh@@mierrooof..ccom
Pawan Goyal IIT Kheragpur
pawang@ese.iiee withsee..itttttooo sittkgp..........
Reviewed on OpenReview: https: // openreview. net/ forum? id= o tSHFeBulff
Abstract
Assessing the quality of generative model outputs from large language models (LLMs) or vision-language models (VLMs), poses significant challenges. Traditional evaluation meth- ods either rely human assessment which is resource-intensive and not scalable or automatic metries that often correlate poorly with human preferences. Another approach is to train dedicated neural evaluators, but this typically requires substantial training data and compute. In this study, we thus introduce ReFeR, a tuning-free framework for eval- uating generative outputs including both lext and images, using two-level hierarchy of pre-trained LLM and VLM evaluators. This multi-agent hierarchical strategy leveragos additional compute at inference time by orchestrating multiple models and utilizing the increased test-time reasoning to boost performance. By having models themselves provide feedback and final judgments, ReFeR reduces the dependence on human evaluation. We rigorously evaluate ReFeR on four diverse evaluation benchmarks, where it surpasses prior hods in accuracy while also generating constructive leedback useful for downst dis- tillation and self self-improvement via finetuning. Interestingly, ReFeR is also applicable for reasoning tasks exporiments on four reasoning benchmarks show ReFeR's suporior collec- live reasoning abilities. We present for accelerated performance, and ReFeR-Lite, offering solution. ReFeRRLite is ~12-14x more compute efficient than previous works while be- ing comparably accurate to ReFeR-Pro. We make a PIP package, code and data publicly available
variants of the framework: ReFeRt-Poo, optimized
test-time compute efficient
"Denotes Equal Contribution
!Equal Supervision http:s////ttttuo.coolyasaetttt-s
httpp://ppp.orr//ppje///rrtt agents

---

Published in Transactions on Machine Learning Research (08/2025)
1 Introduction
The rapid production of content by LLMs and VLMs poses a challenge to traditional human-centric eval- uation methods and conventional automatic metrics. Metrics like BLEU (Papineni et al. 2002). ROUGE [Lin 2004), and METEOR (Banerjee & Lavic, (005) for textual evaluation and CLIPScore (Hessel et al., 2022 for image to text evaluation, often misalign with human judgment and face limitations in assessing creative or nuanced responses. Recent studies suggest using LLMs as novel, reference-indees independent evaluators by assessing text quality based on predicted sequence likelihoods, bypassing the need for direct reference comparisons (Chen et al.) 2023a). This has motivated researchers @Liu et al. 2023c Chiang & Lee
2023
to work on improving the evaluation capability of individual LLMs on text evaluation. Zhang el al. (2024)
highlight that large models align more closely with human perceptual processes, thereby enhancing the eval- uation of multimedia quality. Consequently, Chen et al. (2023b) leverage vision language models to provide explainable image quality evaluation by generating textual explanations, assessing fidelity, aligument, and aestheties.
Surprisingly, despite the potential for improved performance by using ensembles of multiple vision-langae -sionnlanguage models or large language models, there has been limited research on how to align evaluations from multiple VLMs or LLMs with human judgments. The recent advances in inference time compute methods have shown a promising direction leverage additional compute generaling more responses from the same or multiple models to improve performance in generative tasks (Wang et al. 2023 Yao et al. 2023 The prospect of thus taking inspiration from these developments and using multiple VLMs or LLMs together in a society of minds (Minsky 1988) setup to solve the complex and nuanced problem of automatic and scalable evaluations is very promising. But it also introduces several uncertainties, including how to select the models, how many models to use, how to manage communication between different models and what prompting structure should be used to maximize the collective effect.
In this paper, we introduce from the academic peer review process to enhance the evaluation of multimodal generative outputs like text generated by an LLM, an image generated by any model, or caption of an image generated by a VLM. By using multiple LLMs or VL.Ss as evaluators and feedback providers in a system akin to academic peer review, RoFoR enables a comprchensive evaluation of generative outputs across various domains, promoting model sell-improvement, reasoning behind evaluation, and consistent score across runs. The paper outlines ReFeR's methodology, including its new prompting schema and the strategic use of LLMs or VLMs in roles parallel to peer reviewers and area chairs, facilitating multi-dimensional evaluation through a hicrarchical framework consisting of two levels: evaluation at the peer level and evaluation at the area chair level. ReFeR thus leverages increased computation at inference time by coordinating multiple models during evaluation to improve performance and by having models themselves produce feedback and final assessment, we introduce an automated model-based evaluation pipeline, reducing the dependence on human evaluators, thereby helping towards scalable oversight of generative model contents (Bowman et al. 2022 Kenton al. 2024
Feeionn-Feddbeck-Reeeeee (ReFeR), multi-agent framework drawing inspiration
The framework is tested across two natural language generation (NLG) evaluation and two multimodal eval- uation tasks. Interestingly, the ReFeR framework is generic enough to exhibit strong transfer to reasoning benchmarks as well. The multi-agent hierarchical sctup, where multiple peer models deliberate, overseen by the arca-chair model, promotes structured reasoning and refloctive feodback, enabling complex reasoning and maeciiin-mmking:: we also lest. its reasoning ability on four reasoning benchmarks. This deliberative pro- cess also serves a scalable generator of high-quality rich beelback-annottter synthetic instruction-tuning datasets. This imbues smaller models with evaluative capabilities more aligned with human preferences. This enables a scalable method to distill the automatic evaluation capabilities of ReFeR-like frameworks into smaller efficient models by fine-tuning them on these datasets, to achieve better correlation with human evaluation.
We present two variants of our proposed framework, ReFeR-Pro and ReteR-Lite. ReFeR-Lite is ~44 more test-time compute efficient than ReFoR-Pro. Both the variants outperform strong basclines on both text evaluation datasets: TopicalChat (Mehri & Eskenazi 2020) and SummEval (Fabbri et al. 2021). ReFeR also beats baselines like Clipscore (Hessel et al. 2022), ImageReward (Xu et al. 2023a) and others on caption quality and image generation quality evaluation using ICQD (Levinboim et al.) 2019) and AGIQA (Zhang) et al. 2023) datasets respectively by large margins. Lastly, ReFeR also beats single agent methods (zero-shot CoT Koojmaa et al. 2023), self correction (Huang et al. 2024)), and multi-agent methods like multi-agent
2

---

Published in Transactions on Machine Learning Research (08/2025)
Generative-Outie
ReFeR Variants
TopicaiChat
RESNINEE
PHASE-1 Reason
REVIEW BODY
PEER REVIEW BODY
A
-Response contains
The response lacks proper structure & Score 2,5/5
-Re Responne cowers the main tople key poi... -Score 4/5
points...
Peer
Feedback
2
Peer
PHASE-2:
Area Chair
ReFail-Ture
BufeR-Lite
AREA CHAIR
AREA CHAIR
GPT-4o-mh
The evoluation can benelitted by impro Final Evaluation 3.3/5
Tuning
#iteppmanee
Herarchy Agents
Figure 1: Illustration of the ReFeR Framework on the TopicalChat dataset. Refer to Fig. 5 (in the appendix) for illustration of ReFeR, for multimodality and Algorithm showing the framework's 's working. We use the predictions from AC to create an Instruction tuning dataset which can be used to improve the performance of smaller models as evaluators, shown in Appendix J
debate (Du et al. 2023) and multi-agent peer review (Xu et al. 2023b on 3 out of 4 reasoning datasets, clearly outperforming on average while incurring lower costs than baselines.
To summarize, the primary contributions of our research are as follows: (1) Introduction of general- purpose tuning-free multi-agent hicrarchical framework leveraging inference time compute, called ReFeR, with its two variants, ReFeR-Pro and ReFeR-Lite. (2) We develop novel prompting schema, with a novel evaluation guidelines component, specifically designed to improve the effectiveness of our framework both evaluation and reasoning tasks. (3) Empirical validation of the framework's evaluation and reasoning skills on four benchmarks each. (4) We conduct an in-depth analysis of our multi-agent framework, addressing key questions such as how to select models, how many models to use, and other critical aspects of model interactions.
2
Methodology
2.1 ReFeR Framework
Evaluating generative outputs in unverifiable domains without predefined correct answer, such assessing the quality of rescarch paper open-ended responses, presents significant challenges. Inspired by the hicrarchical peer review process in academia, we propose the ReFeR framework, which leverages a hicrarchy of language models to systematically evaluate generative outputs. The ReFeR framework consists of two main modules, as depicted Figure [
1. Peer Review Body
Let G denote the generative output to be evaluated, Ep represent the prompt for the peer and EAC represent the prompt for the area chair. Tet (P1, ,,,...,PP Px} be a set of K peer agents, where each P; a language model acting a peer reviewer. Each peer agent independently evaluates G according to Ep, producing a comment C and score S₁ E R. This process formalized follows.
(C,, S.) Evaluate,, Ep), Ei (1,2,..., K)
(1)
2. Area chair Evaluation
An area chair agent AC, typically larger or more capable language model, synthesizes the peer reviews provide the final evaluation. The area chair considers the generative output G, the prompt EAC, and the set of peer reviews ((C,, K producing final comment Crnat and final score Stnal- is hyperparameter
3

---

Published in Transactions on Machine Learning Research (08/2025)
that denotes the number of responses for given prompt.
s [[C AC. b
Evaluato (1) (G, EaC, (CC, (.)
(2)
Shnal
56
(3)
AC
n
1=1
2.2 Prompting Schema
An essential aspect of assessing generative outputs with language model agents involves crafting prompts that elicit high-qualily evaluations. Prior work, such as G-Eval by Liu et al. (2023e), introduced structured evaluation schema, organizing the prompt into sections: task introduction, evaluation criteria, steps for evaluation, imput presentation (context and target), and an evaluation form designed to output numerical rating only. Subsequently, Chiang & Lee (2023) proposed an adjusted schema named Analyze- Rate,, which prioritizes an analytical review followed by scoring, showing improved performance over G-Eval's prompt.
To further refine this approach, we introduce a new module in the evaluation schern called Evaluation Guidelines to enhance the model's understanding of the scoring criteria, akin to guidelines provided in traditional academic review processos. Evaluation guidelines can be automatically generated by prompting a language model with the prompt structure and some examples from the dataset and we call this process auto prompt. We give an example of this process Appendix F showing the Auto Prompl for Engagingness prompt for TopicalChat. Alternatively, manually written human annotation guidelines of the dataset can be used. We also modified the evaluation form to include critical comment or reasoning for the given score. The proposed evaluation schema is shown in Figure in the Appendix.
2.3
ReFeR Variants
2.3.1
ReFeR-Pro
ReFeR-Pro leverages a hyperparameter n, representing the number of responses generated by the area chair agent. This variant generates multiple responses (n = 20) for each prompt,, applying a scoring function that averages the scores across all generated responses, as described in Eq. 3
The final comment Canal is the list of all individual comments from the area chair evaluations. While ReFeR-Pro provides superior performance due to generating more evaluations per prompt, it incurs higher computational costs. Additionally, the use of n = 20 is often constrained to models from the OpenAl API, as other APIs supporting large models do not support this level of multiple response generation directly. Although it possible to generate multiple responses by making repeated calls to the model (e.g., running the model 20 times with the same prompt), this approach computationally expensive and less practical for large-scale evaluation tasks. This usage of the hyperparameler was first suggested by G-Eval and later used by Analyze-Race Rate.
2.3.2 ReFeR-Lite
To enhance flexibility and reduce computational overhead, we developed ReFeR-Lite, which removes the dependency on the paramcter n for the given performance. In this variant, only a single response (n = 1) is generated for each prompt, or n is completely removed. This reduction in response generation is reffected in Eq 2, where n set to 1.
(Crnal: Stnai) Evaluate F,, ((C,, ((C
(4)
By relying on just one evaluation per prompt, ReFeR-Lite can be used with a wider variety of models, including open-source models, which do not natively support the generation of multiple responses with a single prompt. Despite generating fewer responses, ReFeR-Liite maintains competitive performance and offers significant cost savings. This makes it more test-time computing-cfficient solution for tasks where compulational resources are limited or where evaluating large nunbers of samples is required.
Both ReFeR-Pro and ReFeR-Lite use the same peer evaluation structure, but differ primarily in the area chair's response generation and model compatibility. ReFeR-Pro, with 20, offers potentially higher performance due to generating more evaluations but restricted to models that support simulate
4

---

Published in Transactions on Machine Learning Research (08/2025)
multiple response generation with single prompt. In contrast, ReFeR-Lite provides greater flexibility and cost-efficiency by generating only a single response (n = 1) per prompt, making it more suitable for resource-oonstrained environments.
3 Experiments
3.1
Datasets
For NT.G evaluation, we test our framework SummEval (Fabbri al. 2021 for summarization evaluation, and TopicalChat (Mehri & Eskenazi 2020) for dialogue generation evaluation. For multimodal evaluation, we compare our framework on evaluating two types of task, image-to-text using ICQD (Image Caption Quality Dataset) (Levinboim et. al., 2019) and text-to-image generation using AGIQA-1k by Zhang et al. (2023). For ICQD, we score model-generated captions and compare them with the average human annotated rating for the same. In AGIQA, we assess the quality of Al-generated images in reference to given prompt and compare it with the mean opinion score (human annotations).
We also test our framework on 4 reasoning datasets: AQuA (Ling et al. 2017 BBH-DU Srivastava et al. 2023), CSQA (Aggarwal et al.) 2 and GSM8k (Cobbe et al. 2021 which cover various reasoning tasks like Math, Commonsense and Date Understanding. Statistics and details about all the datasets are provided in Table [ For more details about the datasets, refer Appendix G We test our framework on these reasoning tasks, where our framework answers a reasoning question with the label or numerical Answer after giving the reasoning. We calculate the accuracy of our answers in reference the gold answers.
Table 1: Dataset Statistics. We list all the lasks we lackle in our paper and the datasets we used to show
results with the number of samples used
Dataset
Domain
Samples
Answer
Scale
TopicalChst
Dialogue Generation
NLG Evalustion
360
Rating (on 4 metrics)
1-3
SummEval
Summarixation
NI.G Evaluation
1600
Rating (on 4 metrics)
1-5
ICQD
Image-to-Text
Multimodal Evaluation
864
Caption Score
0-100
AGIQA
Text-to-Image
Multimodal Evaluation
500
Generation Score
0-5
AQuA
Math
Reasoning
Option
CSQA
Commonsense
Reasoning
100
Option
A-E
BBH-DU
Date Understanding
Reasoning
Option
A-F
GSM8k
Math
Reasoning
100
Number
3.2 Baselines
NLG Evaluation
While the current landscape of models for evaluating NL. responses includes reference-based methods such as BERTScore (Zhang et al. 2020a). UniEval (Zhong et al. 2022) and reference-fres methods like GPTSccree Fu et al. 2023). we do not consider these models as baselines given they were clearly surpassed by G-Eval
[Liu et al.
2023c and later works (Chiang Lee 2023 Given our work primarily proposes a reference- free
ULM-basee evaluation for NLG, we do comparative analysis primarily against G-Eval (Liu et al.. 2023c)
and Analyze-Rate (Chiang & Lee] 2023) only.
G-Eval performs evaluation by deploying a single LLM agent. This agent employs Auto-CoT (chain of thought) reasoning and a form-fillig paradigm to ascertain the quality of NLG outputs, delivering only scores for the specific dimensions under scrutiny. They use 'n' hyperparameter to generate 20 responses and take the average score.
Rate builds upon G-Eval, advocating for enhanced prompt structure. This methodology incorporates preliminary analysis phase before scoring, aiming to enrich the evaluative process for NLG tasks. Following G-Eval, they also consider the average score of 20 responses for each sample.
LLM-as-Judge [Zheng et al. 2023) is designed to evaluate LLMs and rank them potentially creating ChatBOT Arena. We replicated the original setup for TopicalChat dataset.
Foor Reasoning, random subset of 100 was sampled from the original datasets, following (Chen al. 2024a). 500 random samples were selected from the original AGIQA-II Ik get well-distributed dataset. We use 864 samples with usable image urls from the ICQD dataset. We use the full test. sets for the NLG Evaluation datasets.
5

---

Published in Transactions on Machine Learning Research (08/2025)
EvatEval Chan al. 2023 proposed single model multi-agent framework with varied persona to evaluate various NI.G responses on open-ended questions and traditional NLP tasks by leveraging debating structure among the agents. We reproduced their most optimal setup mentioned in the paper as baseline with 3 roles i.c. 3 agents and 2 discussion turns.
MultiModal Evaluation
For multimodal evaluation, several works like HyperIQA (Su et al. 2020 DBCNN (Zhang et al. 2020b), IP-IQA (Qu et al. 2024) were proposed for image quality assessment, but all of these works are deep barning-basee methods which leverage and depend on training capable model. Hence we do not compare our framework against them directly.
CLIP Score (Ileseel al., 2022) evaluates how well an image aligns with a text description by using the CLIP model, which computes similarity scores between images and text embeddings.
2023a is a scoring model trained to assess the quality quality/alignnenn /aligmment of generated Image Reward (Xu et al.)
images with text by comparing them against reference images using a reward model.
Pick Score (Kirstain et al.)
2023) is another scoring model for the task of image text alignment, which is trained on human preference images 'picked' for a given text.
X-IQE (Chen et al., (223b)) leverages VL.Ms to evaluate text-to-image generation methods by generating textual explanations. We implement their Alignment dimension experiments compare with our results on text-to-image generation dataset (AGIQA).
Reasoning
We compare our framework against a variety of baseline methods across different categories. For single- agent methods, we select zero-shot Chain-of-Thought (CoT) and Self-Correct. For multi-agent frameworks, we compare against Multi-Agent Debate and Multi-Agent Peer Review, both of which use a single model acting as multiple agents.
Zero-shot CoT (ooimaa et al. 2023 utilizes chain-of-thought prompting to generate reasoning processes and answers using a single agent.
Self-Correct (Huang et al.) 2024) is a single-agent approach that enables an LLM to iteratively evaluate its own outputs, identify errors, and refine its responses through self-reflectonn.
Multi-Agent Debate (Du el al., 2023) involves group of agents, where each agent observes the solutions provided by others, updates its own solution accordingly, and repeats this process through multiple iterations. Multi-Agent Peer Review (Xu et al. 2023b) is multi-agent system in which each agent independently generates solution, reviews the solutions of others, and assigns confidence scores to its reviews. Agents then revise their initial solutions based on the received peer reviews. This revision is repeated through multiple iterations/rounds of peer review. We used the default number of rounds (3) mentioned by the authors.
3.3 Implementation Details
NLG Evaluation: Our framework for NLG evaluation employs Llama-3, 1....11183 1-8B--Instrue AMeta-Al 2024 Mistral-Nemo-122 (Mistral-AT 2024) and Gemma-2-9B (Google-Research 2024 as the peer models and GPT-4o-mini (OpenAl| 2024b as the area chair model. We use Together-AAl (2023)'s 's API for the peer models, but since these are small open-source models, they can also be deployed locally. For the baselines, we follow the original setups (with GPT-4o-mini) proposed by Liu et al. (2023c) and Chiang & Lee (2023). As mentioned in Scction 2.3 we vary the hyperparameter for the two ReFeR variants. For more details
other hyperparameters, refer to Appendix C
Multimodal Evaluation: For multimodal evaluation, our framework uses only peers: Gemini-1.5-Flashh [DeepMind] 2024) and GPT-4o-mini [OpenA] 2024b). We use GPT-4o (OpenAI (024a) as the area chair model. We choose only peers for multimodal evaluation setup considering the cost and availability of VLMs of similar strength. More details the number of peers and how to choose peers described in Scction
5 The baselines like CLIPScore (Hessel et al.) 2022). ImageReward (Xu et al. 2023a), PickScore (Kirstain) et al. 2023) are implemented following the codes provided in their official repositories.
Reasoning: We use the same setup as our NLG evaluation for all our reasoning experiments following similar prompting structure except. using evaluation guidelines which is irrelevant in reasoning tasks. All the

---

Published in Transactions on Machine Learning Research (08/2025)
baselines were implemented and evaluated using the scripts provided by Xu et al. (2023b) in their official repository.
4 Results and Discussions
This section presents the experimental results evaluating ReFeR's effectiveness in assessing text, multimodal outputs, and reasoning capabilities. Experimental details are provided in Section 3.3. hyperparameters in Appendix C
and prompts in Appendix R
4.1 NLG Evaluation
We evaluate ReFeR's performance on two datasots: TopicalChat and SummFval. For TopicalChat, we assess dialog system responses based on four metrics: Coherence, Engagingness, Groundodness, and Naturalness. For SummEval, we evaluate article summaries using Coherence, Consistency, Fluency, and Relevance metrics. Following (Liu et al. 2023c) and (Chiang & Lee 2023), compare the generated scores with human- annotated ground truth using Spearman (p) and Kendall-tau (r) correlations. Results for TopicalChat are presented in Table [ with SummEval results in Appendix F All results are averaged over three runs. The table first shows individual peer performances using our peer prompt, followed by baselines Analyze-Rate (Chiang & Lee 2023) and G-Eval (Liu et al. 2023c). ReFeR Pro outperforms all baselines on most metrics and excels on average. ReFeR Lite, our cost-effective model, ranks second on average despite generating a single response instead of 20 like G-Eval and Analyze-Rate. G-Eval sometimes outperforms Analyze-Rate despite only generating scores, while both Analyzo-Rate and ReFeR provide analysis in addition to scores, offering the potential for model improvement. The key findings from this experiment are: (1) Both ReFeR Pro and ReFeR Lite outperform baselines. (2) ReFeR Lite with n = also achieves comparable performance while being significantly cheaper.
While generating multiple responses (e-g., 20 as in G-Eval) is theoretically possible with any LLM, it poses substantial practical challenges. For instance, evaluating the TopicalChat dataset (360 samples, 4 metrics) would require approximately 28,800 model calls with an average input token size of 675 for TopicalChat. This approach becomes impractical in terms of cost, time, and computational resources, especially for models without the throughput. (n = 20) capabilities of the OpenAT API. Hence ReFeR-Lite can an option in such cases.
Table 2: Comparison of ReFeR with baselines for NLG evaluation on the TopicalChat dataset (all baselines and ReFeR uses GPT-4o-mini as the backbone). Results are averaged across 3 runs. The best results are bolded, and the second-best are underlined. *Costs for ReFeR Pro and ReFeR Lite include only AC APT cost, as open-source peer models can be deployed locally and so do not involve API costs. Peer model costs based on API pricing from services like (Together-AT] 2023) are also provided for reference. Relative costs are shown fractions of the most expensive method. GPT-4o-mini row denotes the performance of the AC model with the peer setup, Peer Average row denotes the correlation when the average of the 3 peer scores is considered.
Method
Coherence
Engagingness
Groundedness
Naturalness
Average
Cost
p
P
[Relative]
Llama-3.1-84
0.350
0.324
0.300
0.312
0.444
0.111
U.320
0.268
0.386
0.337
0.13
Peer Agents
Mistral Nemo-12B
0.409
0.346
0.402
Gomma---991
0.536
0.453
0.615
0.527
0.582
0.545
0.519
0.430
0.563
0.489
0.20
GPT-4o-miai
0.518
0.438
0.618
0.527
0.589
0.549
0.540
0.457
0.566
0.493
0.13
Peer Average
0.547
0.433
0.648
0.519
0.577
0.510
0.512
0.396
0.539
0.447
0.56
Analyzo-Rate
0.551
0.465
0.638
0.544
0.615
0.569
0.562
0.476
0.501
0.5
0.77
Baselines
G-Eval
0.581
0.493
0.636
0.546
0.593
0.555
0.558
0.470
0.592
0.516
0.13
LLM-ae-Judge
0.510
0.145
0.393
0.519
0.556
0.467
0.534
0.471
0.548
0.476
0.13
ChatEval
0.551
0.471
0.624
0.538
0.522
0.428
0.557
0.478
0.564
0.479
0.78
Ours
ReFeR Pro
0.592
0.158
0.677
0.536
0.6415
0.588
0.616
0.473
0.632
0.514
1.0"
ReFeR Lite
0.561
0.479
0.636
0.543
0.618
0.575
0.591
0.416
0.602
0.503
0.13*
4.2 Multimodal Evaluation
To assess the multimodal applicability of ReFeR, we conducted experiments on two tasks: image generation quality evaluation using the AGTQA dataset (text-to-image setting) and image caption evaluation using
7

---

Published in Transactions on Machine Learning Research (08/2025)
the ICQD dataset (image-to-text setting). Table
3 presents the results of these experiments. Following
previous deep learning-based works such as Zhang et al. 2023 we report Spearman's and Kendall's т rank correlations. Key findings include the following.
ICQD dataset: Both variants of ReFeR, outperform all baselines. Notably, although individual peers show low correlations, AC effectively countered this, resulting in better correlation.
AGIQA dataset: ReFeR Pro outperforms all basclines, while ReFeR Lite outperforms ClipScore and X-IQE but falls short of ImageReward and PickScore.
We attribute the performance difference in the AGIQA dataset to the fact that both ImageReward and PickScore involve training based on human preferences, which may have contributed to their superior per- formance compared to our ReFeR Lite variant. But, our ReFeR-Lite has clearly surpassed single VLM based method X-IQE by a large margin showing the effectiveness of the framework.
Table 3: Multimodal Evaluation Results. Comparison of caption quality and image generation quality score correlations with human scores on ICQD and AGIQA datasets, respectively. *X-IQE is texi-lo-image VLM-based method, so we don't 't show it for Caption Quality.
Method
Caption Quality
Image
Quality
(Relative)
Peer Agents
GPT-to-mini
Gemini- 15- Flash
0.200
0.135
0.145
0.098
0.341
0.502
0.268
0.392
0.01
0.07
Clip Score
0.310
0.233
0.522
0.366
Baselines
Rewar
0.433
0.302
0.634
0.451
Pick Store
0.352
0.241
0.627
0.442
X-IQE#
0.410
0.307
0.05
ReFeR
Ours
BeFRR Lite
0.459
0.336
0.599
0.442
0.14
4.3 Reasoning
We hypothesize that our framework enhances the overall reasoning capabilities area chair by utilizing multiple models collaboratively, leading improved docision-making. To verify this, we compare ReFeR's reasoning capabilities against other frameworks, including zero-shot-CoT,, single-agent frameworks, and same- model multi-agent. frameworks. Table 4 presents the results of these experiments, with all results averaged across 3 runs, following the setup in (Chen et al. 2024a) Key observations:
On average, ReFeR outperforms all other basclines across the tested benchmarks. In the BBH Date Understanding benchmark, debating-type frameworks like Multi-Agent Debate show better results than ReFeR. This may be attributed to the nature of the benchmark, which involves understanding dates and resolving conflicts. Such tasks benefit from inter-agent discussions, which are possible in debating setup but not in ReFeR's hierarchical framework.
ReFeR outperforms baselines on the AQuA benchmark because the hierarchical structure allows the area chair to synthesize peer inputs efficiently, avoiding confusion. In contrast, debate formats may cause models to introduce conflicting reasoning, which less effective for tasks requiring precise reasoning like AQuA.
Considcring overall cost and performance, both variants of ReFeR demonstrate significant advantages in terms of cost-efficiency compared to corresponding multi-agent models.
Further analysis on response quality between ReFeR and other methods on GSM8k dataset using reference rationale as premise we get НHEM score (Bao et al. 2024h of 0.33 whereas other baselines have a score ranging from 0.102 to 0.115. Higher HHEM score indicates better response quality. More details on the analysis are provided in the Appendix-()
5 Analysis
In this section, we perform an analysis of the framework to understand the impact of different components and choices.
8

---

Published in Transactions on Machine Learning Research (08/2025)
Table 4: Experimental results on Reasoning tasks. Comparison of ReFeR performance (accuracy)
with single-agent and multi-agent method baselines. All results are averaged across 3 runs. Cost* Costs are shown as relative the most expensive method.
Method Type
Methods
AQuA
BEH DU
CSQA
GSM8k
Average
Cost
Llama-3.1-82
26.3 5.1
28.0
08.3
40.0 11.4
40.1
0.03
Peer Agents
Mistral Nemo-12B
3.6
4.6
65.7
14.7
54.8 + 6.5
0.05
Gemma-2-9B
5..7
6.55
7..
79.3 1 4.0
69.0
1.00 4.3
0.04
Single Agent
scro-shot-CoT
1.5
88.0
76.0
95.3
80.0
Self Correction
7..7
92.7
54.0 1.0
93.3 1.5
79.7
0.20
Ageti-Agent
Multi-Agent Debate
54.0 4.4
94.3
79.0 1: 1.7
95.7 1 0.6
80.8 1:311:111 1.6
0.97
Multi-Agen Peer Review
0.6
77.0
1.0
96.3 + 1.5
81.5
1.0
Ours
ReFaR Pro
85.0 1.0
92.01.2.0
79.3 1 1.2
96.0 0.0
88.
1.0
0.93
ReFeR Lite
81.0 2.0
2..0
79.3
93.3 2.1
86.2
0.18
Peers Qwen
Gemin
Spearman Correlation
Llama Gemma Nemo
4 Peers+GGemint
0.63-
Nemo
Mixtral
+Gemma
Gemma
Lama
Nemno
0.62
Average
Peers
Llama
0.61
+Gemma
Average
Peer
0.60
Nemo
Jama
5
Number of Peers
Figure 2: Framework Ablation. Results obtained on ReFeR-Pro by progressively adding different peers for the TopicalChat Dataset. The points in the figure indicate the performance of ReFeR when specific labelled peers were used in conjunction with the area chair (GPT-4o-mini), "3 Peers" refers to the Llama, Nemo, and Gemma models being used as peers. "4 Peers" includes the same 3 peers along with the Mixtral model added as fourth peer. Detailed results are presented in Table
5.1 Peer Ablation
To evaluate the impact of number of peer agents and composition on ReFeR's performance, we conducted a peer ablation study using the TopicalChat dataset shown in Fig. ?
Our findings indicate that increasing the number of peers generally improves the framework's 's overall corre- lation, evidenced by the main branch in Fig. 2 We experimented with varying peer combinations and numbers to distinguish between the effects of adding another peer versus a better-performing peer. Due to the impracticality of exploring all possible combinations with six peers, we selected a subset based on individual performances, costs, and model sizes.
Fig. 2 demonstrates that while the framework's average performance generally increases with more peers, adding a relatively weaker model can result in better performance than the base (1 peer) but not necessarily the highest overall. For instance, with five peers, the combination of four peers plus Qwen yields the best performance, whereas four peers plus Gemini (weaker at this task) performs closer to the three-peer configu- ration. Notably, the performance gain from four or five peers compared three peers is not substantial. This observation suggests that using three peers may be an optimal choice, balancing performance improvements with computational efficiency.
5.2 Selecting Peers and area chair
For optimal peer selection, we recommend using a group of capable peers chosen based on their individual performances in performance assessment. After assessing individual performances, top-performing peers
9

---

Published in Transactions on Machine Learning Research (08/2025)
be selected considering both cost and performance. As shown in Table G Gemma2-9B is the top performer across all metrics, while Mistral Nemo 12B and LJama-3.1 8B offer comparable performance at lower costs. Consequently, we selected Gemma2-9B, Llama-3.1 8B, and Mistral Nemo 12B as our peers. This selection is also crucial for enabling local GPU deployment of the peers.
To understand the framework's effectiveness under various conditions, we conducted a study by fixing the peers and changing the area chair. Partieularly, we choose area chair which is relatively weaker than not just GPT-4o-mine mini but also our best peer, Gemma2-98 9rmmm at this task. Hence, we choose Qwen1.5 - 72B. Table 5 presents the results using the ReFeR Lite setting on the TopicalChat dataset. We observed that although wC used Qwen as AC (whose individual performance is less than the best pecr), wc get improved performance compared to the respective individual performance. But we see that one of the peer's (Gemma2-9BB has a correlation of 0.568 hence we deduce that if the AC model is relatively stronger than most of the peers then we get improved performance but to get the best results out of the framework we see that we need a larger or better model AC to better utilize the evaluations done by the peers and incorporate them in it's own evaluation.
Table 5: Results on TopicalChat. using the open-source model Qwenl.5-72B the area chair. We were unable to include results for ReFeR Pro with Qwen the area chair due to the limitation of not being able to use n=20.
Method
Coherence
Engagingness
Groundedness
Naturalness
Avg
(Peer) Llama-3,1-80
0.417
0.357
0.118
0.357
0.488
0.155
0.346
0.289
0.117
0.365
(Peer) Mistral Nemo-12B
0.416
0.352
0.567
0.424
0.396
0.458
0.397
(Peer) Gomman-299
0.549
0.465
0.623
0.534
0.583
0.545
0.520
0.431
0.568
0.494
Qwen (Individual Porformance)
0.465
0.399
0.524
0.450
0.471
0.441
0.508
0.437
0.492
0.434
ReFeR Lite (Qwen)
0.496
0.422
0.609
0.522
0.587
0.550
0.527
0.450
0.555
0.486
This observation aligus with the original analogy of research paper peer review, where the area chair is typically a senior researcher with a potentially better understanding than most peer reviewers, thus being given more importance or final judgment authority. In cases where performance assessment. is not feasible to determine the most suitable models, the LLM Leaderboard on Hugging Face (2024) can be consulted to select appropriate models based on the specific task requirements, cost considerations, GPU availability, and time constraints.
Table G shows the individual performances of the 6 open-source models we chose and then the ablation of an increasing number of peers and the ablation of adding different models. The models we used are Llama-3.1-8B, Mistral-Nemo-1125 Gemma-2-9BB, Mixtral-8x7BB, Gemini-1.5-Pro, Qwen-1.5-72B.
We have also performed AC ablation experiments with another AC model to show the difference of perfor- mance based on the model chosen. We have chosen our best peer model Gemma2-9B as the AC model while using the same peer models and their responses, our original setup. We use the same hyperparameters for the AC modee((Gemma) as our original setup. We also show the original CPPT-4o-mini results to compare. From the results of Table 7 we can see that the peer performance of Gemma2-92 is relatively lesser than that of GPT-4o-mini. This same trend is observed when using these models as AC in the ReFeR framework. The better performing model GPT-4o-mini gives better correlation the AC too. This further supports our statements in Section 5.2. where we show that stronger model has to be chosen as an AC to get the best performance from the framework.
5.3 Error Analysis
To assess the framework's effectiveness in both evaluation and reasoning tasks, we conducted an error analysis, with results shown in Fig. 3 In this analysis, a TopicalChat sample's evaluation score for each metric is considered correct if it falls within a given threshold range of 25%. In the TopicalChat dataset evaluation, the area chair provided correct scores 42.6% of the time when one or two peers provided a correct answer, demonstrating the AC's ability to leverage partially correct peer scores effectively. The AC made mistakes only 11.9% of the time when at least one peer was correct. However, the AC was correct only in 2% of the cases where all the peers were incorrect, suggesting that the AC may require at least one correct peer input to avoid confusion and give a correct score. For reasoning tasks, the AC was incorrect for only 2.9% of cases where atleast one peer is correct, showing similiar observation evaluation. And the AC was
10

---

Published in Transactions on Machine Learning Research (08/2025)
Table 6: Ablation analysis of the effect of different models used. Comparison of ReFeR Pro results on TopicalChat with Different Peer Configurations. The method column shows what peers were used with the AreaChair -o-Ti---mini).. 4 Peers in the last rows denotes the 3 peers (Llama, Nemo, Gemma) and Mixtral. *These rows show the individual performance of the peers, not the framework's performance when the peer is used.
Peers Used
Coherence
Engagingness
Groundedneas
Naturalnest
Average
P
P
P
Llama
0.417
0.357
0.418
0.357
0.488
0.455
0.346
0.289
0.417
0.365
Nemo
0.416
0.352
0.567
0.475
0.453
0.424
0.396
0.339
0.458
0.397
Gemma
0.549
0.465
0.623
0.534
0.583
0.545
0.520
0.431
0.568
0.494
Individual results'
Mixtral
0.440
0,373
0.552
0.467
0.491
0.458
0.169
0.390
0.488
0,422
Gemini
0.352
0.300
0.460
0.387
0.498
0.466
0.419
0.352
0.432
0.376
Qwen
0.465
0.399
0.524
0.459
0.471
0.441
0.508
0.437
0.492
0.434
Average
0.440
0.374
0.024
0.446
0.497
0.405
0.443
0.373
0.476
0.416
Llama
0.542
0.423
0.603
0.479
0.628
0.556
0.599
0.460
0.593
0.479
1 Peer
Gemma
Nemo
0.558
0.564
0.448
0,440
0.684
0.680
0.548
0.552
0.617
0.635
0.533
0.578
0,539
0.622
0.481
0.414
0.625
0.599
0.489
0.515
Average
0.555
0.437
0.656
0.528
0.626
0.563
0.587
0.452
0.606
0.494
Llama Gemmaa
0.565
0.440
0.656
0.524
0.593
0.535
0.621
0.481
0.609
2 Porrs
LJama + Nemo
0.577
0.450
0.692
0.553
0.621
0.520
0.621
0.480
0.627
0.513
Nemoo Gemma
0.567
0.443
0.685
0.547
0.622
0.573
0.632
0.490
0.627
0.513
Average
0.570
0.444
0.677
0.541
0.612
0.559
0.624
0.484
0.621
0.507
3 Peers
Llama + Gemmua+ + Nemo
U.589
0.458
0.689
0.550
0.623
0.574
0.626
0.486
0.632
0.517
4 Peers
Peers + Mixtral
0.596
0.463
0.682
0.541
0.629
0.572
0,634
0.494
0.635
0.517
3 Peers + Gemini
0,601
0.469
0,688
0,530
0.644
0.590
0.623
0.485
0.639
0.523
5 Peers
4 Peers + Gemini
0.584
0.455
0.686
0.545
0.623
0.572
0.640
0.495
0.633
0.517
Peers + Qwen
0.601
0.467
0.682
0.545
0.646
0.592
0.637
0.498
0.642
0.526
Table 7: Choosing model for AC: Below Table shows individual performance of AC model and model
acting as AC in the ReFeR Framework. The purpose of this table is to understand correlation of capability of the AC and our framework. We See that if gpt-4o-mini is better than gemma in base performance then gpt-4o-mini is better at being AC than gemma
Coherence
Engagingness
Groundedness
Naturalness
Average
Method
p
Comma-2-9B (Peer Sctup)
0.536
0.453
0.615
0.527
0.582
0.545
0.519
0.430
0.563
0.489
Gemma-2-9B (n=20)
0.356
0.466
0.617
0.521
0.577
0.338
0.530
0.434
0.570
0.490
ReFeRt-Pro (Gemma AC)
0.569
0.410
0.684
0.540
0.643
0.581
0.590
0.451
0.621
0.503
ReFoRt-Lite (Gemana AC)
0.552
0.463
0.624
0.533
0.607
0.567
0.574
0.484
0.589
0.512
GPT-4n-mini (Peor Sctup)
0.518
0.438
0.618
0.527
0.589
0.549
0.540
0.457
0.566
0.493
ReFeR-Pro (gpt-4o-mini as AC)
0.585
0.454
0.673
0.635
0.628
0.577
0.625
0.484
0.628
0.513
ReFeR-Lite (gpt-lo-mini as AC)
0.352
0.467
0.640
0.550
0.596
0.358
0.599
0.505
0.597
0.520
correct 14% of the time, even when all peers were incorrect, indicating a better ability to disregard clearly
incorrect answers from the smaller peer models. This suggests that the AC, when using reasoning tasks, may not always rely on peers and can function independently in such cases.
More analysis about prompt optimization, communication strategics, inference and test-time compute, and statistical signifieance tests can be found in Appendices D K
and M respectively.
6 Related Work
NLG & Multimodal Evaluation: Recent advancements in NLG evaluation include GPTSScre Fuu et al. 2023), which uses generative pre-training models to assess text quality, and G-Eval Liu et al. 2023c
employing a chain-of-thouglie approach with form-filling methodology. Chiang & Lee (2023) highlighted limitations in G-Eval's automated CoT alignment with human evaluations. TIGERScore (Jiang et al., 2023) offers detailed error analysis using fine-tuned Llama-2, while EvsionEval (Shu et al.
2024) integrates
auxiliary evaluators with primary LLM for scoring. X-Eval (Liu et al. 2023a) introduces two-stage
instruction tuning framework for diverse evaluation dimensions. CvatEval (Caa et al. 2023 proposes a
multi-agent referce system using autonomous debating among agents with different personas to evaluate response quality. While similar to our approach, it primarily relies on debate methodology using the same models under varied personas, whereas our method employs diverse models as peers and area chairs with a richer prompting schema. Mixture of Agents (MoA) (Wang et al.] (024bb) is another work which explores the how to harness the collective expertise of multiple LLMs. In multimodal evaluation, CLIP Score (Hessel
et al. 2022 Image-Reward (Xu et al. 2023a and Pick Score (Kirstain et. al. 2023) assess image-text
11

---

Published in Transactions on Machine Learning Research (08/2025)
Comto
D Connek
Ae Careet
9.1%
AC Cornec
26.5%
255%
23%
2 Carest AC reorect
6.%%
2P C-Camet Incmet FFOrect, incoreet
23.3%
3.%
9.5%
Connet
1. Conech Insa
Al: incemee
20.9%
Coreet
C
AC
(a) Evaluation (TopicalChat)
(b) Reasoning
Figure 3: Performance analysis wrt framework scale. Pie-charts showing Peer and AC performance on evaluation and reasoning tasks. (P- Peer model, AC- area chair Model)
alignment using pre-trained models. Deep learning methods like HyperIQA (Su et al. (Qu et al.) 2024) have shown improvements in this domain. Later, X-IQE (Chen et al. using VLMs for the task of image-quality assessment.
2020
and IP-IQA
2023b introduced
Reasoning using LLMs: Single-agent methods like Zero-shot CoT Kojima et al. 2023 have improved language models' reasoning capabilities using Chain-of-Thoughh prompting. Self-correction (Huang et al.
2024) mimics human self-reflection to address reasoning errors. In multi-agent frameworks, Du et al. (2023) introduced a same-model approach using peer solutions for individual improvement, while Pham et al. (2023) proposed embedding-base communication to optimize reasoning. Xu et al. (2023b) developed a framework inspired by academic peer review, emphasizing iterative improvement through peer feedback. This differs from our method, which involves an area chair reviewing peer responses without direct inter- peer communication. ReConcile (Chen et al. 2024a) structured multi-model, multi-agent framework as round table conference, demonstrating enhanced reasoning through discussion and consensus. We expand on why we did not consider ReConcile as a baseline in Appendix H Wang et al. (202cc) proposed selecting the consensus-building building and improving reasoning accuracy. For more recent related works discussion, please refer to Appendix
most coherent response from multiple reasoning chains, offering an alternative approach to consensus-
7 Conclusion
a hicrarchical model framework that utilizes smaller yet capable models as peers and powerful model the area chair. The area chair leverages the reasoning and feedback from the peer models to produce final review for evaluating given images texts. We demonstrate ReFeR's efficacy across two NLG evaluation tasks, two multimodal evaluation tasks, and four reasoning tasks, outperforming various basclines while maintaining favorable cost-performance trade-olls. We also present two variants: ReFeR-Pro and ReFeR-Lite. Notably, our Lite version achieves performance comparable to other works and to ReFeR-Pro, while being significantly more efficient: in terms of compute used.
In this work, we propose ReFeR Seasoon-Feddaack-eeee
Our framework, while robust in many aspects, has some limitations. One notable constraint is the potential computational cost when using large models as both peers and area chairs, especially in resourco-limited environments. Additionally, the framework currently lacks an interactive discussion phase between peer models, which could further improve collective reasoning. In some scenarios, such as when a weaker model is used as the area chair, the performance may not be optimal. Lastly, while our framework has shown promising results on text and image evaluation tasks, it remains untested in other modalities, which could present unique challenges in scaling and complexity and is interesting area for future research.
12

---

Published in Transactions on Machine Learning Research (08/2025)
Broader Impact Statement
This work adheres to the TMLR Ethics Guidelines, ensuring that all evaluations and methodologies applied in the ReFeR framework were conducted with fairness, transparency, and integrity. Since ReFeR operates framework for evaluating machine- generated content, the primary ethical concerns are related to ensuring unbiased assessments and avoiding unintended model biases in evaluations. We carefully selected models to minimize potential biases, but the limitations of the models used could still introduce unintended biases, which we will continue to address in future improvements. No human subjects were involved in the exper- immnts conducted for this study. Additionally, we committ to making our code and datasets available for further research, study and improvements.
To ensure the reproducibility of our results, we provide a detailed description of the ReFeR framework, including the structure of the hierarchical evaluation system and its variants. All hyperparameters, evaluation criteria, and the models used are described in the main text and appendices. The datasets utilized for evaluation and reasoning tasks are publicly available, as mentioned in Section 3.1. Additionally, we will release the source code, along with instructions for running the experimenss, on an anonymous repository. Clear explanations for the model selection process, evaluation metrics, and experimental sctups are also included to facilitate replication by other researchers.
8 Acknowledgments
We would really like to thank Microsoft. Research for their Accelerating Foundation Models Research initiative for generously funding our research and helping us with the GPUs and API credits. We would also like to thank Ganesh Jawahar for helping us with brainstorming the problem statement initially.
References
Shourya Aggarwal, Divyanshu Mandowara, Vishwajeet, Agrawal, Dinesh Khandelwal, Parag Singla, and Dinesh Garg. Explanations for CommonsenseQAA: New Dataset and Models. In Chengqing Zong, Fei Xia, Wenjic Li, and Roberto Navigli (eds.), 1 Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Pupers), pp. 3050-3065, Online, August 2021. Association for Computational Linguistics. act-loo...... 238
doi: 10.18653/v1 122/////1/2211////// lacllo.... 238. URL https://aclaantholog...
Satanjcev Banerjec and Alon Lavic. METEOR:
An automatic metric for MT evaluation with improved correlation with human judgments. Jade Goldstein, Alon Lavie, Chin- Yew Lin, and Clare Voss (eds.), Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Transla- tion and/or Summarization, pp. 65-72, Ann Arbor, Michigan, June 2005. Association for Computational .rg. 0rg/w05-0909
Linguistics. URL https:://ccaatttol/
Forrest Bao, Miaoran Li, Rogger Luo, and Ofer Mendelevitch. HHEM-221-Open, 2024. URL https:// huggingface huggingface.co/vecttaaaa wtt..////rettraaaaaaaaaaaaiaaia htw.://wwnnaaitooooooaaaaaooooaa eewllatioon
Samuel R. Bowman, Jecyoon Hyun, Ethan Percz, Edwin Chen, Craig Pottit, et al. Mcasuring progress on scalable oversight for large language models. arXiv preprint arXiv:2211003540, 2022.
Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu. Chateval: Towards better basebased evaluators through multi-agent debate, 2023.
Justin Chih-Yao Chen, Swarnadeep Saha, and Mohit Bansal. Reconcile: Round-table conference improves reasoning via consensus among diverse llms, 2024a.
Pei Chen, Boran Han, and Shuai Zhang. Comm: Collaborative multi-agent, multi-reasoningg path prompting for complex problem solving, 2024b. URL httts://aaxxiv.oog//b////0//
Yi Chen, Rui Wang, Haiyun Jiang, Shuming Shi, and Ruifeng Xu. Exploring the use of large language models for reference-free text quality evaluation: An empirical study. In Jong C. Park, Yuki Arase, Baotian Hu, Wei Lu, Derry Wijaya, Ayu Purwarianti, and Adila Alfa Krisnadhi (eds.), Findings of the Association for Compulational Linguisties: IJCNLP-AACH 2023 (Findings), pp. 361-374, Nusa Dua.
13

---

Published in Transactions on Machine Learning Research (08/2025)
Bali, November 2023a. Association for Computational Linguistics. URL https://aclanntoooogy..ooo 2023. f findings--ijennl
Yixiong Chen, Li Liu, and Chris Ding. X-iqe: explainable image quality evaluation for texl-to-image gener- ation with visual large language models, 2023b. URL pttpps//aaxxxv.oorr//////330 10843
Cheng-Han Chiang and Hung-yi Lee. A closer look into automatic evaluation using large language models, 2023.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichrr Nakano, Christopher Hesse, and John Schulman. Training verifiers solve math word problems, 2021. URL http:///aaaraa..r///a/////...../
Google Dee..ind. Gemini 1.5-flash.
2024.
Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Tgor Mordatch. Improving factuality and reasoning in language models through multiagent debate, 2023. URL https://arxii org/abs/23055
14325
Sugyoong Eo, Hyconscok Moon, Evelyn Hayoon Zi, Chanjun Park, and Heuiscok Lim. Debate only when necessary: Adaptive multiagent collaboration for efficient Il reasoning, 2025. URL https://axxii:: abs/2504.05041
Alexander R. Fabbri, Wojeiech Kryáciński, Bryan McCann, Caiming Xiong, Richard Socher, and Dragomir Radev. Summeval: Re-evaluating summarization evaluation, 2021.
Jinlan Fu, See-Kiong Ng, Zhengbao Jiang, and Pengfei Liu. Gptscore: Evaluate you desire, 2023.
Googlo-Research. 1emma--2-9b.
2024.
Karthik Gopalakrishnan, Behnam Hedayatnia, Qinlang Chen, Anna Gottardi, Sanjocv Kwatra, Amu Venkatesh, Raefer Gabriel, and Dilek Hakkani- Tür. Topical-Chan Towards Knowledge-Groo Grounded Open- Domain Conversations. In Proc. Interspench 2019, pp. 1891-1895, 2019. doi: 10.21437 00.21437/Interspeeed /Interspeecch
2019-3079. URL http://dx doi org/10. 21137/ 111144/IInneeeeeeeiiia wwtpeechhh.1193077770 2019-3079
Karl Moritz Hermann, Tomáš Kočiský, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. Teaching machines to read and comprehend, 2015.
Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation metric for image captioning, 2022. URL https://arxiv.orr httg/abb/2100..885 08718
Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. Large language models cannot self-correct reasoning yet, 2024. URL https://arxiv.orr abs/ /ttp/2111..00000
HuggingPace. Open Ilm leaderboard. https://huggingfaceccoo http://inggaacccooooceoeeeeseeee 1epimml1mmmlaaalee 1emdeeadebbaard 2024.
Dongfu Jiang, Yishan Li, Ge Zhang, Wenhao Huang, Bill Yuchen Lin, and Wenhu Chen. Tigerscore: Towards building explainable metric for all text generation tasks, 2023.
Zachary Kenton, Noah Y. Siegel, Janos Kramar, Jonah Brown-Cohen, Samuel Albanic, Jannis Bulian, Rishabh Agarwal, David Lindner, Yunhao Tang, Noah D. Goodman, and Rohin Shah. On scalable oversight with weak 1ms judging strong 1lms. In Proc. of the 37th Conference Neural Information Processing Systems (NeurIPS), 2024.
Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. Pick-a-pic: An open dataset of user preferences for text-to-image generation, 2023. URL https://arxiv.. abs/ 2305. 01569
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners, 2023. URL http:::://axii.or///bbbb http://aas/2...... 11916
14

---

Published in Transactions on Machine Learning Research (08/2025)
Shanu Kumar, Akhila Yesantarao Venkata, Shubhanshu Khandelwal, Bishal Santra, Parag Agrawal, and Manish Gupta. Sculpt: Systematic tuning of long prompts. arXiv preprint orXiv:24 10.222800.7888 2024.
T. Levinboim, A. Thapliyal, P. Sharma, and R. Soricut. Quality estimation for image captions based on largo-scale human evaluations. arXiv preprint arXiv:1909. 0909009366, 2019.
Chin-Yer Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pp. 74-81, Barcelona, Spain, July 2004. Association for Computational Linguistics. URL https:
//aclanthology. hww.aaattoooooy.ogg/A0------0000 org/W04-1013
Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. Program induction by rationale generation: Learning to solve and explain algebraic word problems. In Regina Barzilay and Min-Yen Kan (eds.): Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 158-167, Vancouver, Canada, July 2017. Association for Computational Linguistics. doi: 10.18653/ 10.18653/v1/P117 101.17-1015. URL htttps:/acclatthollgg -00.
Minqian Liu, Ying Shen, Zhiyang Xu, Yixin Cao, Eunah Cho, Vaibhaw Kumar, Reza Ghanadan, and Lifu Huang. X-eval: Generalizable multi-aspect text evaluation via augmented instruction tuning with auxiliary evaluation aspects, 2023a.
Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost the middle: How language models use long contexts, 2023b. URL https: org/ abs/2307.03172
Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. G-eval: Nlg evaluation using gpt-4 with better human alignment, 2023c.
Shikib Mehri and Maxine Eskemazi. Usr: An unsupervised and reference free evaluation metric for dialog generation, 2020. URL httpg://aaxxvvrrrr/////00...0000
Meta-Al. Ulamm--.1-Sb-instrucc httpp://wwgiigggee...oerreeaeaaa Instructsttttt 2024.
Marvin Minsky. Sociely of mind. Simon and Schuster, 1988.
Mistral-AI.
httpe://hugiiggiccc.o///iitttttt
-estrall-Nmoo--nsttru 2024.
OpenAl. Gpt-4o.
2024a.
OpenAL. Gpt-4o-mini. http:://pppatttooooopaaaaaaaaaaa
2024b.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, ACL '02, pp. 311 318, USA, 2002. Association for Computational Linguistics. doi: 10.3115/
1073083.1073155. URL https://doi. httg//0.3115./0 1111111111111101111111 1073135
Chau Pham, Boyi Liu, Yingxiang Yang, Zhengyu Chen, Tianyi Liu, Jianbo Yuan, Bryan A. Plummer, Zhaoran Wang, and Hongxia Yang. Let models speak ciphers: Multiagent debate through embeddings, 2023.
Priya Pitre, Naren Ramakrishnan, and Xuan Wang. Toward deliberative ai: Multi-agent llms for real-world reasoning. https://vtechhorkks lib. vt. wttp///tes//449/0055555
2025.
Reid Pryzant, Dan Iter, Jerry Li, Yin Lee, Chenguang Zhu, and Michael Zeng- Automatic prompt optimization with "gradient descent" and beam search. In Houda Bouamor, Juan Pino, and Ka- lika Bali (ods.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Pro- cessing. pp. 7957-7968, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1 http//2.eemlmmmmin.40000 URL wtt.hhtsss/accaaahollloo
e0:0002..mmllpmmmmmmmmmmmmmmllm1 main.494.
15

---

Published in Transactions on Machine Learning Research (08/2025)
Bowen Qu, Haohui Li, and Wei Gao. Bringing textual prompt to ai-generated image quality assessment, 2024. URL httpp:////axiio.//babb////////// /t/pb///4438187100
Lei Shu, Nevan Wichers, Liangchen Luo, Yun Zhu, Yinxiao Liu, Jindong Chen, and Lei Meng. Fusion-eval: Integrating evaluators with Ilms, 2024.
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R. Brown, and ct cl. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models, 2023. URL https://arxiv, http://axxvvvo..aaaaa/ awgggbba/206..6. 04615
Shaolin Su, Qingsen Yan, Yu Zhu, Cheng Zhang, Xin Ge, Jinqiu Sun, and Yanning Zhang. Blindly assess image quality in the wild guided by self-adaptive hyper network. In IEEE/CVR Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
Together-AI. Together-ai.
togetheertttrrrrrrtooooo hit/ 2023.
Haotian Wang, Xiyuan Du, Weijiang Yu, Qianglong Chen, Kun Zhu, Zheng Chu, Lian Yan, and Yi Guan.
Learning to break: Knowledge-enhanced reasoning in multi-agent debate system, 2024a. URL https: //arxiv. http://bs/231...000 04854
Junlin Wang, Jue Wang, Ben Athiwaratkun, Ce Zhang, and James Zou. fuxture-of-agentt enhances large language model capabilities, 2024b. URL httts: org/ attp//bb/////////////// 04692
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In Proc. of the 11th International Conference on Learning Representations (ICLR), 2023.
Zhenhailong Wang, Shaoguang Mao, Wenshan Wu,, Tao Ge, Furu Wei, and Heng Ji. Unleashing the emergent cognitive synergy in large language models: A task-solving agent through multi multi-persona collcbollaoo,tionn, 2024c.
Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong, Imageroward: Learning and evaluating human preferences for text-to-image generation, 2023a. URL
httpa:://xxxvv..r...e 2204. 23077059777
Zhenran Xu, Senbao Shi, Baotian Hu, Jindi Yu, Dongfang Li, Min Zhang, and Yuxiang Wu. Towards reasoning in large language models via multi-agent, peer review collaboration, 2023b. URL https://
arxiv. www./orgg///////eeeeeee
Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V. Lc, Denny Zhou, and Xinyun Chen. Large language models as optimizers, 2024. URL https://arxiv. wwwp:///wwww.aab/......b http://b//33..... 03409
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. In Aduances in Neural Infor- mation Processing Systems 36 (NewrIPS 2023), 2023.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert, 2020a.
Weixia Zhang, Kode Ma, Jia Yan, Dexiang Dong, and Zhou Wang. Blind image quality asscssment using a deep bilinear convolutional neural network. IEEE Trunsuctions on Cirewils and Systems for Video Technology, 30(1):36-42 January 2020b. ISSN 1558-2205. doi: (0.1109/ccsv..21882888738 28877777.. URL http:
//dx.doi. or0/10..0011111111 11w0/TCSTTT... 2018. 2886771
Zicheng Zhang, Chunyi Li, Wei Sun, Xiaohong Liu, Xiongkuo Min, and Guangtao Zhai. A perceptual quality assessment exploration for aige images, 2023. URL
www.aaiii.orgr////////a1 12618
Zicheng Zhang, Yingjie Zhou, Chunyi Li, Baixuan Zhao, Xiaohong Liu, and Guangtao Zhai Quality assess- ment. in the era of large models: A survey, 2024. URL httpp::/arxxv..rr//bbs/////
16

---

Published in Transactions on Machine Learning Research (08/2025)
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ton Stoica. Judging llm-as- a-judge with mt-bench and chatbot arena, 2023. URL
Ming Zhong, Yang Liu, Da Yin, Yuning Mao, Yizhu Jiao, Pengfei Liu, Chenguang Zhu, Heng Ji, and Jiawei Han. Towards unified multi-diimensionll evaluator for text generalion, 2022. URL https: htttp://aaxiiiii w/tp:/aaa.vvooo//oo org/ abs/2210.0713
Yongchao Zhou, Andrei loan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. Large language models are human-level prompt engineers, 2023. URL https: pctps://arxivg wwapxiii.o.//..e. abs/ 2211.01910
A
Future Works
Future research can explore incorporating additional elements from the academic peer review process, such as the author discussion phase, to simulate more interactive review environment. Expanding the framework include evaluations beyond text and images, such as video and audio content, could further enhance its applicability. Another promising direction is develop various communication strategies between peers and the area chair to optimize evaluation and feedback cycles. Moreover, experimenting with different numbers of area chairs of varying strength could help in understanding the impact of multiple, potentially conflicting, judgments on the final evaluation outcomes.
INPUT CONTEXT Original Article
INPUT TARGET
Summary of that Article
TASK INTRODUCTION
You will be given one summary written for news article. Your task is to rate the summary on one metric
EVALUATION CRITERIA
which the summary evaluated)
Fluency (1-5): Evaluate sentence quality based on readability, grammar, and structure; high score reflects well- constructed, free sentences for easy reading.
Àuto CoT
Auto Prompt
EVALUATION STEPS
EVALUATION GUIDELINES
(Steps the model should follow to evaluate) Read the news articie understand the overall and tone. the quality construction, grammar, and overall readabllity,
(Guidelines model should follow to score the article]
5:0: Exceptional, flawless grammar, clear structure.
summary.
sentence
Mostly well-constructed,
<= Score Mix of good and poor sentences, noticeable errors, <n Score poorly constructed sentences, frequent errors.
Rate the summary's fluency scale of with the being most fluent.
Score Predominantly poor, error-ridden, to grasp.
EVALUATION FORM
Criticca comment Analysis/ Reasoning (Between
Figure Prompting Schema
17

---

Published in Transactions on Machine Learning Research (08/2025)
Outprt
ReFeR Variants
AGIOA
Image & Text
PHASE-1: Reason
GPT-4o-mini
Geminl-1.5-aag
PEER REVIEW BODY
Image is blury and doesn't follow the....
Image aligns with prompt, but misses.... Rating:
Rating
Peer Body
Poer 1
RefeR-Turbo
ReFaR-Lite
AREA CHAIR
AREA CHAIR
GPT-4o
-Image aligna with prompt but misses details like.. -Rating:3
#Rasponone::0
#Reepenssent
Hierarchy Agents
Onl OpenA API
Any LLMa
Figure 5: Illustration of ReFeR for Multimodal evaluation shown AGIQA datasct. A similar version of ReFeR working textual TopicalChat dataset is shown in Fig. 1
18

---

Published in Transactions on Machine Learning Research (08/2025)
B
ReFeR Algorithm
Algorithm 1: ReFeR Framework for Evaluating Generative Outputs
Input
Generative output 0 (text or image)
Prompting schema Ep for peers and EAC for area chair
Peer Models P (P,, P2, Pa...., (K Peers)
Area Chair Model AC
Variant V [ReFeR-Proo, RefeRRLite)
Number of responses (only for ReFeR-Pro)
Output:
Final Evaluation Score Stiall
Constructive Feedback Cinal
Peer Review Body Evaluation; foreach peer model P; P do
Phase 1:
// Each peer independently evaluates 0 using prompting schema Ep (c) + Evaluatep, Ep);
end
Phase 2: Area Chair Evaluation; if V ReFeR-Lite Lite then
1;
end else
n + 20;
end
for j =1 to do (li)
AC AC + Evaluate (G, (G S()
end
// Compute final score
Sinal 1 n クド Ac:
Canal Choose randomly whose score is closest to Stnal (C .c...., C(n))
return Final Evaluation Score Sanat, Constructive Feedback Ciiaal
Mathematical Notation Summary:
G: Generative output to be evaluatod.
AC: area chair agent.
Ep: Prompt of peer.
Number of independent evaluations by AC in
E Prompt of area chair.
ReFeR-Pro
P:: Peer agent i, for i i= 1, K.
c(u) s(J),
ACy AC: Comment and score from the j-th eval- uation by AC.
K: Number of peer agents.
C: Comment from peer agent P;-
Ctnal, Sinat: Final comment and score.
Sy Score from peer agent P..
In summary, the ReFeR framework formalizes the evaluation of generative outputs by modeling the process after the hicrarchical peer review system, with mathematical rigor to facilitate clarity and reprodueibility. This approach only enhances the evaluation accuracy but also provides constructive feedback, aligning closely with human judgment and expectations in complex evaluation scenarios.
19

---

Published in Transactions on Machine Learning Research (08/2025)
C Hyperparameters
For the ReFeR NLG Evaluation setup, following Analyze-Rate (Chiang & Lee 2023), we set these hyper- parameters as follows, for the AreaChair GPT-4o-mini model- temperature=44 1, mar tokens=256, top_p=1, pennvencypenall-ff., presence__penalllye__ penally=0, stop=None, n=20 (varies for ReFeR Lite and Pro). For the peer models, we use the default hyperparameters except for the max tokens= 128. For multimodal evalu- ation, we use the same setup for the AC, but for the peers, we increase the maz maz_tokens from 128 to 192 tokens. For reasoning tasks, we follow the NLG evaluation setup for the area chair, but we don't 't set any limit on the mar_tokena hyperparameter. For the peer modcls, we increase mar mar_tokens to 256 and set the seper-parameters temperature-1, lop p=1.
D Prompt Optimization
Prompt optimization methods utilizing LLMs, such as OPRO Yang et al. 2024), APE [hoou et al. 2023 ProTeGi (Pryzant et al. 2023), and SCULPT Kumar et al. (2024), often employ text-gradient or feedback- based techniques to refine prompts. These methods typically involve providing a capable LLM with error examples and obtaining feedback, which serves as text gradient to adjust the prompt for improved perfor- mance. Such approaches have demonstrated effectiveness for short questions/requests and relatively simple lasks.
Table compares our prompt with an optimized prompt using ProTeGi on the TopicalChat dataset. We utilized the default ProTeGi settings with GPT-4o-miní as the optimizer LLM. Our findings indicate that prompt optimization is time-consuming and incurs higher costs than evaluation itself due to the iterative improvement process of prompt optimization over a test set, and that too with long inputs for complex tasks like this. Morcover, our prompts with the proposed structure yield better correlation than the optimized prompts from ProTeGi.
We attribute this outcome to the limitations of these methods when dealing with extensive inputs, such conversation history and responses in TopicalChat, where entire dialogues provided to evaluate and rate NLG output various metrics. Unlike G-Eval, which only provides scores, methods like Analyze-Rate and ours improve scores based on generated analyses. In these cases, both the analyses and ratings are crucial for understanding errors, as the singular numerical rating value offers insufficient insight into prompt issues. Additionally, even when detailed analyses are provided in multiple error examples for the prompt optimization, the gradient-based approach may struggle with long contexts, making it challenging for the model to identify specific prompt deficiencies and provide useful feedback.
Table 8: Analysis of Prompt Optimization. Comparison of Average (across 4 metrics) results for different prompts on TopicalChat dataset.
ProTeGi
Method
Llama-3 1-8B
0.317
0.303
0.386
0.337
Mistral-Nemo-
0.387
0.336
0.402
2-9mma-2-913
0.511
0.444
0.563
0.489
ReFeR Pro
0.625
0.511
0.628
0.513
The table shows the results of average results across 4 metrics for peers and the framework for Prompt optimized by ProTeGi vs Our Prompt gencrated through our prompting schema. We can see that the peers' performance declines with the optimized prompts. These prompts were the best prompts after 3 rounds of Prompt-Optimizatin with ProTeGi. But still their performance falls short our prompting schema. And even though the framework is relatively close, it would still make point on how the effort and costs for the prompt optimizations would not be worth it. Running ProTeGi prompt optimizations alone for peers and area chair costs ~ 4 times the cost of evaluating using ReFeR-Pro.
Prompt Ablation
Table 9 shows the ReFeR framework performance on TopicalChat Dataset with different prompts. We can see from the results that our prompting Schema gives the highest performance with Analyze-rato being the second. From the significant difference, we can see that G-eval scores only prompting doesn't work for our ReFeR framework.
20

---

Published in Transactions on Machine Learning Research (08/2025)
Table 9: Analysis of Prompt Ablation.
Average results (across metrics) on TopicalChat Dataset of
ReFeR-Pro with different Prompt structure.
Average
Prompt Schema
p
G-Eval
0.568
0.464
ReFeR-Pro
Analyze-Rate
0.592
0.510
Ours
E
Auto Prompt
Example Implementation of Autoprompt:
Input AutoPrompt:
Auto Prompt for Engagingness Evaluation for TopicalChat
You are tasked with creating a clear and concise prompt for task based the provided prompt structure and examples from dataset. The prompt should be writlen in such way that it con be easily understood and followed by another LLM human user performing the lask. Your prompl should include the following:
A brief overview of the task.
Evaluation criteria explaining what metric the evaluation is going to be
Clear instructions for how to approach the task or evaluation steps.
Use the ezamples of dataset, analyze and undersland how it is evaluated for the given metric and give a detailed Evaluation guidelines which will tell when to give a particulor score.
Example Prompt Structure:
You will presented with a conversation between two individuals and given a potentiol response for the next turn in the conversation, along with a fact that the response based on.
Your lask is to evaluate the response on single metric: Eagaginggess. The rating must be given after giving the analysis too.
Evoluation Criteria:
Engagingness (1-3): Assess whether the response dull, moderalely interesting, or highly engaging. Evoluation Steps:
1. Read the conversation, the corresponding fact and the response carefully.
2. Rate the response on scale of 1-3 for engagingness, according to the criteria above.
Please ensure the prompt explains the rating scale from 1 to 3 clearly.
Examples for Task:
Example 1:
Conversation History:
Person A: "I heard that flamingos gel their pink color from their diet."
Person B: "Really? 1 didn't 't know that! What do they eat to turn pink?"
Fact: Flamingos get their pink color from consuming shrimp and algae rich in carotenoids.
Response: "Yes! They eat shrimp and algae, which contain pigments called carotenoids that give them their pink color."
Rating: 3
Example 2:
21

---

Published in Transactions on Machine Learning Research (08/2025)
Conversation History:
Person A: "I just rewatched Interstellor. The way they depict time dilation near a black hole is incred- ible."
Person B: "Yeah, the science behind it is pretty solid. What did you think of the emotional storyline with Cooper and his doughter?"
Person A: "I thought it was powerful how they tied the human element with such a complex scientific concept." "
Person B: "Do you think the film's portroyal of black holes was accurate?"
Fact: Interstellar consulted physicist Kip Thorne to ensure the black hole depiction was as scientifically accurate as possible, especially with the lime dilation effects.
Response: "It was fine."
Rating: 1
The above prompt is an example of Auto Prompt for the Engagingness metric in TopicalChat dataset. The text marked in blue, i.e, the example prompt structure and the examples are the inputs to be given by the user to get the evaluation guidelines to be used in proposed prompting schema. In the example above use the prompt structure from Analyze-Rate engagingness prompt. Hence by prompting capable LLM with an Auto Prompt crafted for task giving the prompt structure and examples, we can obtain a prompt following our prompting schema for that task, including evaluation guidelines. Output based on the above prompt:
Engagingness Peer Prompt
You will given a conversation between two individuals. You will then be given one potential response for the nert turn in the conversation. The response concerns an interesting fact, which will be provided as well.
Your task is to rate the responses on one metric. Please make sure you read and understond these instructions carefully. Please keep this documend open while reviewing, and refer to it as needed. Evoluation Criteria: Engagingness (1-3): Assess whether the response is dull, moderately interesting, or highly engaging.
Evoluation Steps:
Read the conversation, the corresponding fact and the response carefully.
2. Rale the response scale of 1-3 for engagingness, according to the criteria above.
Evoluation Guidelines:
A score of 1 (dull) means that the response is generic and dull.
A score of (somewhal interesling) means the response is somewhal interesting and could engage you in the conversation (e.g., an opinion, thought).
A score of (interesting) means the response is very interesting or presents an interesting fact.
F SummEval Results
We test our framework on the SummEval Dataset, comparing with the baselines G-Eval and Analyze- Rate. We first show our individual peer performances, then the baselines, and finally, the two variants of ReFeR framework. Before delving deep into the results of this benchmark, it is important to discuss the dataset distribution of SummEval. As shown in Fig. G the datasct is highly skewed for the consistency and fluency metrics, with almost 1300+ and 1100+ samples having score of 5 for consistency and fluency, respectively.
The skewed distribution in the Summeval dataset creates an imbalance in correlation evaluations. Smaller models, such Gemma-2-9B, which may lack sensitivity to subtle differences in the data (e.g., article and
22

---

Published in Transactions on Machine Learning Research (08/2025)
Table 10: Performance analysis on SummEval dataset. Comparison various methods for NLG
evaluation on SummEval.
Method
Coherence
Consistency
F Fluency
Relevance
Average
P
P
7
P
p
Llama-3.1-8B
0.351
0.287
0.425
0.381
0.307
0.277
0.301
0.295
0.361
0.310
Peer Agents
Mistral Memo-12B
0.367
0.296
0.383
0.310
0.239
0.211
0.368
0.303
0.339
0.287
Gemmaa-244
0.474
0.433
0.422
0.484
Baselines
Analyze- Hate
0.533
0.392
0.382
0.305
0.353
0.283
0.430
0.320
0.425
0.325
G-Eval
0.509
0.387
0.475
0.356
0.334
0.290
0.571
0.433
0.472
0.374
Refell
Ours
ReFeR, Lite
0.483
0.400
0.472
0.420
0.360
0.324
0.472
0.397
0.447
0.385
Consistency
Fluency
1200
1000
800
of Samples
of Samples
800
600
200
1.0
1.5
2.0
2.5
3.0
3.5
4.0
4.5
5.0
1.0
15
2.0
2.5
3.0
3.5
4.0
4.5
5.0
Rating
Rating
(a) Consistency
(b) Fluency
Figure 6: Ratings Distribution. We show the distribution of human annotations for Consistency, FLuency metrics the SummFval dataset.
summary), often give uniformly high scores that mimic the skewed human annotations, resulting in higher correlations. However, this does not reflect the modcl's true ability to understand and follow instructions. In contrast, larger models like GPT-4o-mini, which adhere more strictly to evaluation guidelines, tend generate more varied scores. This often leads lower correlations due the skewed nature of the human annotations. Additionally, in cases where the ReFeR framework provides consistent scores, the result is high number of tied predictions, which further lowers Kendall's tau coefficient due to the large number of tied pairs. This complicates the interpretation of performance for larger models and more advanced frameworks, as the lack of variability in the dataset hampers an accurate assessment of model effectiveness. Given these challenges, although SummEval is a popular benchmark dataset used for NLG evaluation, we not consider Summeval to an appropriate benchmark for testing our methods, unless a uniformly distributed sample can be extractod-a difficult task given the inherent skewness of the annotations. Hence, even though ReFeR-Pro outperforms other baselines on this dataset, we present these results only in the appendix.
G Datasets
NLG Evaluation
SummEval Fabbri et al. 2021) provides human asscssments on four critical dimensions of summarization quality: Coherence, Consistency, Fluency and Relevance, utilizing the CNN Mail dataset. (Hermann) et al. 2015) as its foundation.
TopicalChat (Gopalakrishnan et al. 2019 dataset of conversations. We use the dataset created by Mchri & Eskenazi (2020) using the TopicalChat dataset in which they give possible next response generated by language model for given conversation history, and the human annotation score of the
23

---

Published in Transactions on Machine Learning Research (08/2025)
response five attributes: Coherence, Engagingness, Groudednnsss, Naturalness, and Understandability. We exclude Understandibility, following the previous works G-Eval, Analyze-Rate and Uni-Eva
MultiModal Evaluation
ICQD (Image Caption Quality Dataset) (Levinboim et al. 2019) focuses on the task of Quality Estimation (QE) for image captions. We use the test dataset which provides human ratings (0/1) on quality, We scale these average ratings to a scale of 0-100 for our evaluation.
AGIQA (AI Generated Image Quality Assessment) (Zhang et al. 2023) presents a AGI quality assessment database, AGIQA-IK, which consists 1,080 AGIs generated from diffussio models. They provide MOS (Mean Opinion Score) in the range of 0-5. We have observed that the dataset is skewed around certain scores around 3-3.5. So to test samples, such that the data more less equally spread on the rating range (0-5).
subsct which has variance of image quality ratings, we sclect 500
Reasoning
AQuA (Algebra Question Answering) (Ling et al. 2017 dataset is designed to assess model's reasoning abilities in solving algebraic word problems. It consists of multiple-choice math questions, where the model must understand and compute the correct answer from several options.
BBH-DU (Big Bench Hard Date Understanding) (Srivastava et al. 2023) datasct is part of the BIG- Bench benchmark. It focuses on testing a model's ability to comprehend and reason about date-related information, such as calculating durations and interpreting dates.
CSQA (CommonsenseeAA) (Aggarwal et al. 2021) dataset designed to test a model's understanding of commonsense knowledge through multiple-choies questions. Each question requires reasoning over general world knowledge, with answer choices based on various plausible but nuanced options, lesting the model's 's ability to pick the most commonsensical answer.
GSM8k (Grade School Math 8K) (Cobbe et al. 2021) dataset is collection of 8,000 challenging grade- school-level math word problems. It is designed to test a model's ability to perform multi-step arithmetic reasoning and solve math problems requiring logical thinking and numerical computation.
H
Note on ReConcile
ReConcile (Chen el al. 2024a) is another relevant multi-agent framework that utilizes different LLMs with similar capabilities to engage in discussions and reach consensus. However, we exclude ReConcile from our baselines because its use of 3 LLMs of similar capabilities and makes it an unfair comparison to our framework, which employs 3 smaller models as peers and 1 larger model as the area chair. Simulating ReConcile with our setup would require excluding one of the models, either from the peer group or the area chair, which would lead to unbalanced debate. In particular, if we use 2 smaller models and large model, the debate would be dominated by the larger model, resulting in not include ReConcile as direct baseline.
biased outcomes. For these reasons, we do
More Related Works
In this section we discuss more recent works in the field of multi-agent reasoning. While our ReFeR frame- work proposes specific two-level hierarchy inspired by academic peer review, the broader field of multi-agent reasoning has explored several alternative collaborative architectures. Many recent works focus on achieving consensus through unstructured or semi-structured debate. For example, ReConcile (Chen et al., 2024a) im- plements a "round-table conference" where diverse LLM agents iteratively discuss and refine their answers to reach a consensus, castiag with ReFeR's single-pass hierarchical synthesis. Similarly, CONSENSAGENT Piitre al.) 2025) focuses on mitigating "sycophancy" in multi-agent debate through dynamic prompt re- finement achieve more efficient and effective consensus, whereas ReFeR's structure bypasses the need for peer consensus by vesting final authority in the area chair. Other frameworks introduce adaptive mechanisms to balance performance and efficiency. DOWN (Debate Only When Necessary) (Eo et al. 2025) proposes a dynamic framework that triggers a multi-agent debate only when an initial agent's 's confidence score is low, offering a different approach to efficiency than ReFeR-Lite's fixed, lightweight structure. Another direction
Uni- Eval shows results on the 4 metrics and uses the Understandability metric for transfer experiment, hence only dimensons are shown in the following works. Refer Zhong al. 2022 for more details.
24

---

Published in Transactions on Machine Learning Research (08/2025)
involves diverse reasoning paths and role-playing. CoMM (Chen et al. 2024b prompts agents to adopt different expert roles, encouraging varied problem-solving strategies, which differs from ReFeR's functionally defined roles of peer evaluator and synthesizer. Lastly, some frameworks integrate external knowledge, such as MADKE (Wang et al. 2024a), which provides agents with a shared knowledge pool to enhance their debating capabilities
clear distinction from ReFeR's reliance on the internal knowledge of its constituent models. These kind of reasoning frameworks can be used for specific use cases where you have a defined knowledge pool.
J Finetuning
Utiliziig Analysis from larger LLMs (* ("Area Chair"), we enhance smaller LLMs through instruction-tuii -umiin using a dataset crafted from comprehensive evaluations. We use the analysis feedback generated within the ReFeR framework, transforming it into useful resource for instructional tuning. This fine-tuning signifi- cantly improves smaller models performance, enabling them to reach or surpass their larger counterparts in evaluation tasks. We use Mistral-7B, since it can easily deployable on a small GPU and finetune. We used the Instruction-tuning dataset (final output of Area Chair) of ReFeR framework the training data.
Table 11: Improving smaller models via instruction-tuning. Finctuning Results for Mistral-7B model on TopicalChat Dalaset
Model
Coherence
Engagingness
Groundedness
Naturalness
Avg
p
Mistral-7B No Finetune
0.124
0.102
0.167
0.134
0.078
0.009
0.100
0.081
0.117
0.096
7BB Pinetuned
0.457
0.348
0.626
0.486
0.487
0.437
0.493
0.377
0.516
0.412
K
Communication Strategies:
Table 12: Communication Strategies. Results on TopicalChat showing different generation and commu-
nication strategies for ReFeR-Pro.
Communication
Coherence
Engagingness
Groundedness
Naturalness
Avg
Peer Feedback AreaChair
P
p
Comment Only
0.602
0.471
0.635
0.502
0.661
0.590
0.587
0.454
0.621
0.504
Score Only
0.585
0.454
0.673
0.535
0.628
0.577
0.625
0.484
0.628
0.513
Both Comment Score
0.580
0.453
0.642
0.512
0.605
0.345
0.555
0.127
0.596
0.181
The type of feedback provided by peers to the area chair plays a crucial role in determining overall effective- ness. We explored three communication strategies: passing only scores, passing only comments, and passing both comments and scores. Table 12 presents the impact of different feedback strategies on the framework's performance. The results indicate that passing only scores to the AC yields the best performance, with passing only comments being a close second. This is likely because when both comments and scores are passed, the AC model becomes more prone to confusion due to conflicting analyses scores, and the longer prompt inputs negatively affect its decision-making Liu et al. (2023b). Based on these findings, we adopt the scorcs-only strategy for all subsequent experiments with our framework.
L Inference and Computation
Fig. presents the time taken per instance for ReFeR Variants and baseline models. G-Eval demonstrates the fastest inference speed, as only generates scores. In contrast, Analyze-Ras takes nearly twice as long as G- Eval, since produces both an analysis and a rating. ReFeR-Lite and ReFeR-Pro require only approximately 1.5 times the duration of Analyze-Rate, despite being multi-model frameworks. Notably, there is minimal difference between the Lite and Pro variants due to the influence of the hyperparamcter, indicating that the bulk of the processing time arises from the involvement of multiple models in the framework.
Although we show the monetary cost comparison of our framework vs the other methods, not the only cost for evaluation. We need to also consider cest-time compute metric like FLOPs processed. So to know all parameter counts and calculate the FLOPs processed for each method, use open source model
25

---

Published in Transactions on Machine Learning Research (08/2025)
Figure 7: Inference speed comparison with baselines
Inference Speed Comparison
1.271
12-
1300
Time Instance (Seconds)
0.824
0.8-
0.6
0.479
0.2
G-éval
Analyze-Rate
ReFoR-LIte
RaFeR
Method
Qwen-2.5-72B, as our AC model (with the same peer models) and for G-Eval Analyze-rate. We calculate the FLOPs with this approximation formula:
FLOPs = layers x [4 sequence sequence_length &model + 8 duuoeel ffn dim]
The components involved in the FLOPs calculation are as follows: layers refers to the number of transformer layers, sequence_ _enngh length the length of the input sequence, d model is the dimensionality of the model (hidden size), and ffn dim represents the dimensionality of the feed-forward network, which is typically 4 times dmodel- For the self-attention mechanism, the cost of gencrating the query, key, and value matrices is given by:
3 da model sequence_ Jength.
The cost of the scaled dot-product attention, which involves computing attention scores and applying them, is approximately:
(oodel sequence length.
Therefore, the total cost for the self-attention mechanism is:
4 x df model sequence length.
In the food-forward network (FFN), which consists of two lincar layers with a ReLU activation in between, the cost for each linear layer is approximately:
2 x dmodel ffn dim.
Thus, the total cost for the FFN, considering both the forward and backward passes, is:
8 x dmodee ffn dim.
Although the exact model architectures vary for different models, and hence the actual FLOPs vary, but believe this approximation formula helps to give enough information for comparison of the computation. heaessccocreeeccnggginnnnnnnnn
The given Total Input tokens is for all 4 mentioned, we use the same model Qwen-2.5 our AC and for G-Eval, Analyze-Rate.
for cach model/method. As
We modulate the performance with the hyperparameter 'n' see whether our method consistently is above the baselines in the performance-comptee trade off and we present these results in Table 14 But since the open models don't 't have 'n' hyperparamater, do calls to get the responses.
26

---

Published in Transactions on Machine Learning Research (08/2025)
Table 13: Model Comparison of FLOPs. ReFeRt-Lite is calculated as the cost the peer models(first 3
rows) and AC. ReFeR-Pre Pro is calculated as summation of the peers and 20 AC calls. G-Eval, Analyzo-Rate Rate are calculated as 20 calls.
Model
Amodel
ffn
dim
layers
Total Input tokens
FLOPs (x (x10
Llama-3. 1-8B
1096
14336
32
970720
2.11
Mistral-Nemo-112
14336
970720
Gemmaa-299
3584
28672
42
970720
2.14
(Qwen-2.55722 (nm1) 1)
8192
1016279
Analyze- 2
8192
29008
80
856100
372.16
G-Eval(n=20)
8192
29568
888500
386.07
ReFeRt-Pro
8192
29568
449.31
ReFeR-Lite
8192
29568
80
30.40
Table 14: Performance and Computation of methods across different n-values. correlation across all metrics.
is average spearman
Method
P
FLOPs (x10*5)
ReFoR-Lite
0.620
30.41
Analyze Rate
0.545
18.61
G-Eval
0.608
19.3
ReFelt
0.639
74.51
Analyse
G-Eval
0.626
57.9
ReFeR
0.646
118.61
Analyse Rate
0.504
93.05
G-Eval
0.633
96.5
ReFeR
U.648
184.76
Analyze Rate
0.542
148.88
G-Feal
0.636
154.4
ReFelt
10
Anslyze Rate
0.541
186.1
G-Eval
0.637
193.0
Performance vs Compute Cost
n=
n=10
0.64
n=
n=5
n=2
Spearman Correlation
0.62
n=3
ReFeR
n=1
G-Eval
Analyze-Rate
0.58
0.56
n=]
n
n=8
n=10
100
150
200
250
Inference compute/FLOPs (x 1025)
Figure 8: Performance vs Compute Cost
For better understanding and visualization, we show the performance (spearman correlation) FLOPs in
Figure
8
27

---

Published in Transactions on Machine Learning Research (08/2025)
From Figure g and Table 14 observe that ReFeR method is above the other methods in terms of performance-compulee trade off. We can see that although ReFeR is relatively computationally costlier than the other methods, it has achieved higher correlations with relatively lower values of n, as we see that n=8 of ReFeR gives better performance than n=10 of G-eval while maintaining lesser FLOPs compared to G-Eval's 1010. We observe overall that ReFeR-Pro (or n=10 here,) has more FLOPS than G-Eval, but ReFeR gives more correlation than G-Eval. Whereas ReFeR-Lite (n=1) has significantly lesser FLOPs but gave very high correlation although didn't cross in this experiment with Qwen-2.5-72B model. But, we can also some other observations here in this setting with an open source model. Both ReFeR and G-Eval follow the same trend where see increase in performance with increase in 'n', but we don't see this trend in Analyze-rate. This leads us to the observation that some prompting schemna are not consistent across different models.
Overall do think that ReFeR-Lite is significantly better than other methods considering the overall FLOPs to performance ratio.
M Statistical Tests
Table 15: Statistical Tests. p-values for statistical tests between ReFeR Pro and baselines on TopicalChat.
Metrie
ReFeRt Analyze
ReFeR G-Eval
Coherence
2.31 x 10
4.29 10-13
Engagingness
3.70 x 10
1.01 10-17
Groundedness
1.15 x 10
6.31 x 10 12
Naturslness
1.19 x 10
0.0736
We conducted the paired t-test to compare our method with the baseline and we report the p-values. As we can see for only one metric in G-Eval, the p-value is greater than the typical significance level (0.05). Hence, from the p-values, we can see that ReFeR is, in general, statistically significantly better than G-Eval and Analyze Rate.
N Example of Reasoning improved by Framework
Example from GSM8K
Solo has to read 4 pages from his Science tertbook, 20 pages from his Social Studies tertbook, pages from his History textbook and 8 pages from his Geography textbook. Solo read 15 pages Monday. If he has 4 more days to complete all his reading, how many pages does he need to read, on average, in one day?
Question:
Gold Answer: 6 pages
Peer I's Response:
To find the total number of pages Solo needs to read, we add the pages from each teztbook: 4 (Science) + 20 (Social Studies) + 7 (History) + 8 (Geography) = 39 pages. Solo needs to read, on average, 75 pages per day. Answer: 9.75
Peer 2's Response: Total pages to read: 4 (Science) + 20 (Social Studies) 7 (History) + 8 (Geography) = 40 pages. After subtracting 15 pages already read, Solo needs to read 25 more pages. He needs to read an average of 6.25 pages per day. Answer: 6.25
Peer 3's Response: Total pages to read: 4 (Science) + 20 (Social Sludies) (History) 8 (Geography) = 39 pages. Solo needs to read, on average, 9. 75 pages per day, Answer: 9.75
Area Chair's Response: Total pages to read: 4 + 20 + + = 39 pages. After subtracting 15 pages already read, Solo needs to read 24 more pages. He needs read an average of pages per day- LLM and LLM 3 provided the correct colculations but made mistake in their final answers by not subtracting the number of pages Solo read Monday from the total number of pages. They both stated that Solo needs to read, average, 9.75 pages per day, which is incorrect. LLM 2 made calculation mistake. My solution corrects these errors and provides the accurate calculation for the average number of pages Solo needs to read per day- Answer: 6
28

---

Published in Transactions on Machine Learning Research (08/2025)
Example 2 from GSM8K
Question: Conor lives near a beach and loves going there every day lo have fun. On a particulor week, he found 50 people at the beach on the first day. The next day, 20 more people were present at the beach than on the first day, and on the third day, the total number of people increased to twice the number that were there on the second day. If Conor saw an average of 60 people each day the rest of the week, calculate the total number of people Conor at the beach that week.
Gold Answer: 500
Peer I's Response:
Analysis: To solce this problem, will first determine the number of people present the beach on each of the first three days. Then, I will calculate the total number of people Conor saw at the beach Jor the entire week by adding the number of people on each of the firsi three days to the average number of people he saw each day for the rest of the week. Since there are 5 more days in standard week, I will multiply the average daily number of people by 5 to find the total number of people Conor saw for the remaining days. Finally, will add the total number of people seen on the first three days to the total number of people seen during the rest of the week fond the overall total.
The first three days:
Daa 1: 50 people
-Day 2: 20 more people than the first day, so 50 20 70 people
-Day 3: Twice the number of people on the second day, so 2 x 70 140 people
Total ramberr of people on the first three days: 50 + 70 + 140 = 260
Average number of people on the rest of the week (5 days): 60 people/ day
Answer: 560
Peer 2 Response:
Analysis: On the first day, there were 50 people at the beach. On the second dag, there were 20 more people than the first day, making it 70 people. On the third day, the number of people doubled from the second day, resulting in 140 people. For the remaining days, Conor saw overage of 60 people each day, so the total for these days is 4," 60 = 240 people. Adding all these together gives the total number of people Conor saw that week. Answer: 450
Peer 3's Response:
First, we calculate the number of people at the beach on the second and third days. The second day had 50 + 20 70 people. The third day had 70 140 people. Then we calculate the total rumber of people the remaining five days 60 5 300. Finally, we add up the people from all seven days: 50 + 70 140 + 300 560.
Analysis:
Answer: 560
Arca Chair's Response:
Analysis: Let's lel's break down the problem step by step.
1. On the first day, Conor saw 50 people at the beach.
2. On the second day, there were 20 more people than the first day, so the lotal for the second day
is: 50 + -00 = 70 people.
3. On the third day, the number of people doubled compared to the second day, giving 70 2
140 people.
29

---

Published in Transactions on Machine Learning Research (08/2025)
4. For the remaining four days of the week, Conor saw average of 60 people each day. The total for these Jour days is: 60 x 4 240 people.
Now, let's add all these numbers together to find the total number of people Conor saw at the beach over the week: 50 (day 1) + 70 (day 2) + 140 (day 3) + 240 (remaining days) = 500.
Ample insighls were provided by the LLMs, parlicularly the correct calculation for each day. However, LLM 2 had a calculation mistake when reporting the final total as 450, which is inaccurate. LLM correctly adds up the totals but does not account for the final calculation correctly, as it appears to calculate five days instead of the four remaining after the first three. Thus, compiling our information correctly gives the right total: Answer: 500
Hallucination
To understand the difference in response quality for framework and the baselines, used the HHEM-2.1 model by [ao et al. (2024). The HHFM-2.1 model measures the hallucination in the hypothesis (generated text) based on the premise (reference text). We did this analysis on the GSM8k experimental results, as the dataset provides the human annotated cuasoning/rationaal along with the answers. Hence we use this reference human reasoning as the premise and the generated reasoning in the final responses by different methods as the hypotheses. HHEM-2.1 gives scores the range (0,1) where, mcans that the hypothesis is not evidenced all by the premise and means the hypothesis is fully supported by the premise. Hence, the higher the HHEM score is, the lower the method's responses hallucinated. We weren't 't able to test this on other benchmarks where human annotation reasoning/cvalaation evaluation is not given.
Table 16: Comparison of average hallucination scores across methods on the GSM8k Benchmark. GPT-40- mini is the HHEM score of GPT--4-mini using our peer setup.
Method
HHEM Score
GPT-lo-mini
0.207
Zero-Shot-Con
0.115
Self Correction
0.136
Multi-Agent Debate
0. 0.102
Multi-Agent Peer ReFeR
0.330
We can see that ReFeR clearly outperforms all the baselines and GPT-4o-mini'' our peer setup) hhem score. Considering the GPT-4o-min''s score baseline, see that ReFeR has reduced hallucination, whereas the other baseline methods have increased hallucination significantly. This shows that ReFeR produces better rationale in responses compared to the baselines.
P Homogeneous Experiments
We have demonstrated ReFeR primarily as heterogeneous framework, where we leverage different models for peers and AC. Hence, did this homogeneous experiment to test the framework's effectiveness when use the same model for both the peers and AC. We use our best peer model Gemma2-91 as all peers & AC for this experiment, and to have diversity in responses among the peers and AC, we use the temperature hyperparameter, choosing temperatures /0.25,0. 0,5,0.5,0 0. 75) for the peer models. We retain the temperature=1 for AC from our original sctup. We can see the results for the homogeneous experiment in Table 17
Table 17: Performance of ReFeR framework with same model, Gemma2-9B as both peers and AC. Varying temperatures were used for diversity among peers.
Method
Coherence
Engagingness
Groundedness
Naturalness
Average
P
P
Gemmna (emp=0.25)
0.359
0.175
0.611
0.525
0.563
0.329
U.536
0.146
0.568
U.194
Gemma (temp=0.0)
0.548
0.465
0.611
0.525
0.571
0.535
0.540
0.452
0.568
0.494
Gemma (temp=0.75
0.517
0.463
0.626
0.536
0.582
0.541
0.126
0.566
ReFoR-Pro
0.587
0.468
0.681
0.543
0.628
0.572
0.597
0.466
0.623
0.512
ReFoR-Lite
0.563
0.476
0.648
0.557
0.614
0.557
0.574
0.480
0.600
0.517
30

---

Published in Transactions on Machine Learning Research (08/2025)
As the framework has still shown to improve the performance using the same model for both peers and AC. Hence, we can see that the framework is effective even with using same model varying temperatures) for peers and AC. This setting is particularly helpful when one wants to utilize a single model on the GPU for the entire evaluation.
Q Additional Experiments
Averaging Ensemble
We also considered an ensemble method where we combine (average) the higher performing AC model with the peers by giving higher weightage to the AC and how this kind of direct numerical ensembling gives result, compared to our hierarchical review framework. In this ensemble method results in Table [8 where AC weightage is given, the remaining weightage (out of 1) will be split equally between the remaining peers, Equal weightage refers to the case where all the Peers and AC model are given equal weightage. As the weightages increase, the performance increases ever so slightly but plateaus later. We can see that this baseline still falls short to some of the other baselines like G-Eval & Analyze-Rate. This shows how simple ensembling like averaging of a higher models' evaluation might work.
Table 18: Average Spearman and Kendall-Tau correlations across metrics for different average ensemble weightages on TopicalChat. AC, refers to the weightage of the AC in the average and the remaining weightage is equally split between the peers.
Method
Spearman (p)
Kendall-Tau (r)
Llama-3.
0.386
0.337
Peer Agents
Mistral-Nemo
0.464
0.402
Gemma-2-9B
0.563
0.489
Baselines
G-Eval
0.592
0.516
Analyze-Ro
0.591
Chair
ReFeRt-Lite
0.602
0.503
AC with Peer Setup
0.5661
0.4932
Average
0.539
Equal Weightage
0.5787
0.4735
(AC 0..4)
0.5864
0.4781
Average
(AC 0..5)
0.586
0.1766
(AC... 0.66
0.5866
0.4769
Ensemble
(AC 0.7)
0.5864
(AC_. 0.8)
0.5864
0.4769
(AC 0.9)
0.5857
0.4753
Multiple Area Chairs
We also experimented on a variant of RelFer where we have multiple ACs, and then have each of them give the scores with the ReFeR framework, but in the end, combine the scores by averaging them. In the results in Table 19 GPT AC rows refer to our original setup of GPT-4o-miní as our AreaChair model, and Claude AC refers to the Claude-3.5-haiku model as AC. We can observe that with the suggested method of 2 area chair ReFeR-Lite version, achieve slightly more than the ReFeR-Lite method, whereas for both AC, the ReFeR-Pro gives significant gains, compared to the AC, These results demonstrate how the ReFcR method is effective even with multiple Arca chairs.
that
Table 19: Average Spearman(p) and Kendall- Kendall-Tau(7) correlation across metrics for different ACs and Multiple AC. The Multiple AC row refers to ReFeR-Lite variant.
Method
AC Model GPT-4o-mini GPT-4o-mini
Spearman
Kendall-Tau (7)
ReFoR.Pro
0.6352
0.5188
ReFaR-Lite
0.6827
0.5041
ReFeett-Poo
Claudo-3.5 haileu
0.6593
0.5414
ReFoR-Lite
Claude-3.5-haile
0.6325
0.5525
Multiple AC
GPT Claude
0.6327
0.5378
Method Ensemble
We performed these experiments to observe if /ennebbiign other methods such as G-Eval has any improvement over the existing framework due to the possibility of the other method being complementary.
31

---

Published in Transactions on Machine Learning Research (08/2025)
Table 20: Comparison Average Spearman correlation for Methods' ensemble across 4 metrics.
Method
Coherence
Engagingness
Groundedness
Natursiness
Average
G-Eval (n=20)
0.596
U,503
0.549
0.616
0.566
ReFeR-Pro
0.589
0.689
0.637
0,626
0.635
ReFeR-Lite
0.520
0,669
0.600
0.575
0.591
G-Eval + ReFeR-Pro
0.609
0.630
0.635
0.634
0.627
G-Eval ReFeR-Lite
0.593
0.628
Weighted Ensemble
0.601
0.543
0.576
0.635
0.589
Inverse Weighted
0.616
0.385
0.617
0.637
0.611
We ensemble by taking the average of the scores, in case of G-Eval and ReFeR-Pro, we average over the 20 20 scores, and in case of RefeR-Lite, we just take the average of the ReFeR--Lite and G-Eval. In case of the weighted G-Eval, ReFeR-Pro Avg, have chosen the weights using variance based confidence, basically assuming lower variance over 20 responses meaning higher confidence/weighiee /weightage (This method is chosen because that is the only external factor to do some kind of ensembling other than simple average). From the results in Table 20 we can see that, for ReFeR-Pro, this ensemble deteriorates the performance, but for ReFeR-Lite, slightly improves the performance. the performance drastically. But surprisingly case of the inverse weighted avg, where higher variance was preferred, the performance reductions are not that high.
the case of the weighted average, it reduced
ReFeR with Debate
We performed experiments to see if including debate into the ReFeR framework has any improvement. And the results in Table show that even though there are slight improvements over the normal framework with the debate in 2 metrics, overall, just the hierarchical structure without the Debate is better. We also see that the correlation of peers' scores drops drastically after debate. We have adopted the debate framework from the Multi-Agent Debate framework (Du et al. 2023) from our reasoning baselines, but where we have single debate round and then the AC review.
Table 21: Comparison of ReFeR and ReFeR Debate variants across multiple evaluation dimensions.
Category
Method
Coherence
Engagingness
Groundedness
Naturalness
Average
Llama-3.1-80
0.312
0.451
0.395
0.330
0.372
Mistral Nemo
0.441
0.623
0.429
0.374
0.467
ReFeR
Gemma-2-9B
0.505
0.603
0.582
0.516
0.552
ReFeR Pro
0.626
0.635
ReFelt Lite
0.537
0.630
0.583
0.565
0.579
Llama-3.1 1-8B
0.211
0.158
0.130
0.224
0.181
ReFeRt
Mistral Nemo
0.134
0.034
0.012
0.109
0.072
+ Debate
Gemma-2-9B
0.466
0.149
0.436
0. 0:133
0.446
ReFelt Pro
0.586
0.618
Refelt Lite
0.514
0.552
0.586
0.537
0.547
R
Prompts
R.1 NLG Evaluation
TopicalChat
Coherence Peer Prompt
You will receive dialogue between two people. Following that, there will be one suggested reply for the next part of the conversation, along with related interesting fact.
Your job is to assess how coherent the suggested reply is, focusing on its ability to seamlessly continue the dialogue while also considering the overall context of the conversation, including the provided fact. Please read and understand these instructions carefully. You may refer back to them needed. Assessment Criteria:
Coherence (1-3): How well does the response continue the conversation? A score of 1 (no) indicates that the reply significantly shifts the topic disregards the ongoing conversation entirely.
32

---

Published in Transactions on Machine Learning Research (08/2025)
A score of (somewhat) suggests that the response makes vague reference to the conversation but fails to effectively engage with the dialogue or the accompanying fact.
A score of 3 (yes) signifies that the response stays on topic, acknowledges the previous dialogue, and draws a clear and relevant connection to the interesting fact provided while maintaining the overall consersational flow.
Assessment Process:
1. Review the conversation history for contert and flow, focusing on how well the suggested reply relates to the previous exchanges.
2. Examine the suggested reply for its relevance and engagement with the ongoing dialogue.
3. Consider how well the reply connects with the interesting fact while also evaluating its contri- bution to the conversation as a whole.
4. Assign coherence score of I, 2, or 3, taking into account both the conversational progression and the connection to the fact.
Exumple:
Conversation History: HCooversationn]] Corresponding Fact: {[Contertual F
Response: [[Generated Response]]
Evoluation Form (Answer by slarting with "Analysis:" to analyze the given exumple regarding the evaluation criteria as concise as possible, and then give the nameric rating on the next line by "Rating".)
Coherence:
Coherence AreaChain Prompt
Navigate through a simrulated conversation between two individuuls, followed by a provided potential response incorporating intriguing fact. Your role is to assess the responses based the coherence metric.
Alongside your evaluation, you will also receive inilial evaluations from three large lunguage models, referred as the assistants" evaluations. Please read the instructions and crileria below carefully and use them as guide in your evaluation, critically assessing the conversation, and the assistants' inputs. Ensure meticulous understanding of the instructions. Keep this document accessible for reference during the evaluation.
Evaluation Criteria:
Coherence (1-3): Assess whether the response seamlessly continues the conversation history. A score of (no) denotes a significant shift topic or disregard for the conversation history- A score of 2 (somewhat) indicates response with limited reference the conversation history and noticeable shiftt in lopic.
A score of 3 (yes) signifies on-topic response that strongly acknowledges and builds upon the conversation history.
Evoluation Steps:
Thoroughly read the conversation history.
2. Examine the potential response.
3. Evaluate coherence based on the conversation history.
33

---

Published in Transactions on Machine Learning Research (08/2025)
4. Assign coherence score of 1,
Eromple:
Conversation History: [Conversation]]]
Corresponding Fact: f[Contertual Fact))
Response: [[Generated Response)}
First Assistant's 's Evaluation: {[Peer (lPPerrresponselle
Second Assistant's 's Evaluation:
Third Assistant's 's Evaluation: {(Peer_ rerpor_rss))s)3])
Evaluation Form (Answer by slarting with "Analysis:" " to analyze the given exumppl regarding the evaluation criteria as concise as possible, and then give the numeric rating on the next line by "Rating".)
Coherence:
Engagingness Peer Prompt
You will be given conversation between two individuals. You will then be given potential response for the next turn in the conversation. The response concerns an interesting fact, which will be provided as well.
Your lask is rale the responses on one metric. Please make sure you read and undersland these instructions carefully. Please keep this document open while reviewing, and refer to it needed. Evaluation Criteria:
Engagingness (1-3): Is the response dull/interesting?
A score of 1 (dull) means that the response is generic and dull.
A score of 2 (somewhat interesting) means the response is somewhat interesting and could engage you in the conversation (e.g., an opinion, thought).
- A score of 3 (interesting) means the response is very interesting or presents an interesting fact.
Evaluation Sleps:
1. Read the conversation, the corresponding fact and the response corefully.
2. Rale the response on a scule of 1-3 for engagingness, according to the crileria above.
Exomple:
Conversation History: {Convveraation)}
Corresponding Fact: {CContertual Fact]}
Response: ([Generated Response]]
Evaluation Form (Answer by starting with "Analysis:" to analyze the given exomple regarding the evaluation criteria as concise as possible, and then give the rameric rating on the next line by "Rating") ".) Engagingness:
Engagingness AreaChair Prompt
Navigate through a simallated conversation between two individuals, followed by a provided potential response incorporating an intriguing fact. Your role to assess the responses based on the engagingness metric.
Alongside your evaluation, you will also receive initial evaluations from three large language models, referred to as the assistants' evaluations. Please read the instructions and criteria below carefully and use them as a guide your evaluation, crilically assessing the conversation, and the assistants' inpuls.
34

---

Published in Transactions on Machine Learning Research (08/2025)
Ensure meticulous understanding of the instructions. Keep this document accessible for reference during the evalualion.
Evaluation Crileria:
Engagingness (1-3): Is the response dull or interesting?
A score of 1 (dull) means that the response is generic and uninteresting.
A score of 2 (somewhat interesting) means the response is somewhat engaging and could capture interest (e.g., on opinion or thought).
A score of (interesting) means the response is highly engaging or presents an intriguing fact.
Evoluation Steps:
1. Read the conversation, the corresponding fact, and the response carefully.
2. Rale the response scale of 1-3 for engagingness, according to the crileria above.
Eromple:
Conversation History: [[Conversation:]
Corresponding Fact: f[Contertual Fact)}
Response: [[Generated Response]}
First Assistant's Evaluation: [[Peer_responee
Second Assistant's 's Evaluation: (Peeer_repoonsel
Third Assistant's Evaluation:
Evoluation Form (Answer by starting with "Analysis:" to analyze the given eromple regording the eval- uation criteria as concisely as possible, and then give the numeric rating on the next line by "Rating", ".) Engagingness:
Groundedness Peer Prompt
You will be given a conversation between two individuals. You will then be given one potential response for the next turn in the conversation. The response concerns an interesting fact, which will be provided as well.
Your task is to rate the responses instructions carefully. Please keep this document open while reviewing, and refer to it needed. Evaluation Criteria:
metric. Please make sure you read and understond these
Groundedness (0-1) given the fact that this response is conditioned on, delermine whether this response. uses that fact.
A score of 0 (no) means the response does not mention or refer to the fact all.
A score of 1 (yes) means the response uses the fact well.
Evaluation Sleps:
1. Read the conversation between the two individuals.
2. Identify the fact that provided for the potential response.
3. Read the potential response.
4. Determine if the potential response uses or mentions the fact.
5. Assign a score of 0 or 1 for groundedness based on whether the response uses the fact.
Example:
35

---

Published in Transactions on Machine Learning Research (08/2025)
Conversation History: [Convvrrsatinn]]
Corresponding Fact: {CContertall Fact]}
Response: [[Generated Response]}
Evaluation Form (Answer by starting with "Analysis:" to analyze the given ezomple regarding the evaluation criteria as concise as possible, and then give the rameric rating on the next line by "Rating" ".)
Groundedness:
Groundedness Area Chair Prompt
Navigate through a simalated conversation between two individuals, followed by provided potential response incorporaling an intriguing fact. Your role is to assess the responses based on the groundedness metric.
Alongside your evaluation, you will also receive initial evaluations from three large language models, referred to the assistants' evaluations. Please read the instructions and criteria below carefully and use them as guide in your evaluation, critically assessing the conversation, and the assistants' inputs. Ensure meticulous understanding of the instructions. Keep this document accessible for reference during the evaluation.
Evoluation Criteria:
Groundedness (0-1): Given the Sact that this response is conditioned on, delermine whether this response that fact.
A score of 0 (no) means the response does not mention refer to the fact at all.
A score of 1 (yes) means the response uses the fact well.
Evaluation Steps:
1. Read the conversation between the two individuals.
2. Identify the fact that is provided for the polential response.
3. Read the potential response.
4. Determine if the potential response uses or mentions the fact.
5. Assign score of 0 1 for groundedness based whether the response the fact.
Erample:
Conversation History: [[Conversation]]
Corresponding Fact: {(Contertual Fact))
Response: [fGenerated Response}}
First Assistant's 's Evaluation:
{(Peer_responee responsel]]
Second Assistant's 's Evaluation: [
Third Assistant's Evaluation:
Evaluation Form (Answer by starting with "Analysis:" to analyze the given exomple regording the eval- ualion crileria as concisely as possible, and then give the numeric raling on the next line by "Raling" ".) Groundedness:
Naturalness Peer Prompt
You will be given conversation between two individuals. You will then be given potential response for the next turn in the conversation. The response concerns an interesting fact, which will be provided as well.
36

---

Published in Transactions on Machine Learning Research (08/2025)
Your task is to rate the responses metric. Please make you read and understond these instructions carefully. Please keep this document open while reviewing, and refer to it needed. Evaluation Crileria:
Naturalness (1-3) Is the response naturally written!!
A score of 1 (bad) means that the response is unnatural.
A score of 2 (ok) means the response. is strange, bul not entirely unnatural.
A score of 3 (good) means that the response natural.
Evaluation Steps:
1. Read the conversation between the two individuals.
2. Read the potential response for the next turn the conversation.
3. Evaluate the response based on its naturalness, using the provided criteria.
4. Assign a rating score of 1, 2, 3 based on the evaluation.
Example:
Conversation History: { [Conversation]]
Corresponding Fact: {[Contertual Fact]}
Response: [[Generated Response]}
Evaluation Form (Answer by starting with "Analysis:" analyze the given example. regarding the evaluation criteria concise as possible, and then give the rumeric rating on the next line by "Rating" ".)
Naturalness:
Naturalness Area Chair Prompt
Novigale Uhrough a sinvulated conversation between two individuals, followed by a provided polential response incorporating on intriguing fact. Your role to assess the responses based on the naturalness metric.
Alongside your evaluation, you will also receive initial evaluations from three large language models, referred to as the assistants' evaluations. Please read the instructions and criteria below carefully and use them as a guide in your evaluation, crilically assessing the conversation, and the assistants' impuls. Ensure meticulous understanding of the instructions. Keep this document accessible for reference during the evaluation.
Evoluation Criteria:
Naturalness (1-3): is the response naturally written?
A score of 1 (bad) means that the response is unnatural.
A score of 2 (ok) means the response is strange, but not entirely unnatural. A score of 3 (good) means that the response is natural.
Evaluation Steps:
1. Read the conversation between the lwo individuals.
2. Read the potential response for the next turn the conversation.
3. Evaluate the response based on its naturalness, using the provided criteria.
4. Assign rating score of 1, 2, 3 based on the evaluation.
37

---

Published in Transactions on Machine Learning Research (08/2025)
Exomple:
Conversation History: { [Covversation]]
Corresponding Fact: {[Contertull ]
Response: [[Generated Response)}
First Assistant's 's Evaluation: reppor_essons
Second Assistant's Evaluation:
Third Assistont's Evaluation: ([Peer_respones
Evaluation Form (Answer by starting with " "Analysis:" to analyze the given exomple regording the eval- ualion crileria as concisely as possible, and then give the numeric raling on the next line by "Raling" ".
Naturalness:
R.2
MultiModal Evaluation
R.2.1
ICQD
Caption Quality Peer Prompt
Your lask carefully evaluate the alignment between an image and corresponding caption based on the provided criteria. Pay close attention to the instructions to ensure an accurate and ruanced assessment. Instructions:
1. Examine the image closely, identifying its key visual elements, objects, actions, and overall contert.
2. Scrutinize the caption, comparing it the visual content of the image, and identifying any inaccuracies, omissions, or misleading information. Consider both the explicit details and the overall contert of the image.
3. Rate the caption on scale of 1-100 according to the Evaluation Criteria, where indicates very poor match and 100 indicates perfect match.
Evoluation Crileria:
Rating (0-100): Evaluate the extent to which the caption aligns with the visual content of the image. high roting should be given if the caption accurately reflects the main elements, actions, and contert of the image, even if uses concise language or omits minor details. Deduct points for inaccuracies, misleading descriptions, or significant omissions that distort the intended message of the image.
90-100: The caption perfectly or almost perfectly captures the image's 's content.
70-89: The cuption is mosily accurale, with only minor inaccuracies or omissions.
50-69: The coption has notoble inaccuracies or omissions but still partially represents the image.
30-49: The caption poorly represents the image, with significant inaccuracies or misleading elements.
0-29: The caption is almost entirely inaccurote irrelevant to the image.
Example:
Image:
[Image will be provided separately]
Caplion: {[Cuplion))
Evaluation Form (Answer by starting with "Anabysis:" to analyze the provided example regarding the evaluation criteria as concisely as possible, and then give the numeric rating on the next line by "Rat- ing".)
Caplion Quality:
38

---

Published in Transactions on Machine Learning Research (08/2025)
Caption Quality Area Chair Prompt
You will be given an image, ils caplion, and you will also receive inilial evoluations from lawo large language models, referred as the assistants' evaluations.
Your task is to rate the caption Please read the instructions and criteria below carefully and use them as a guide in your evaluation. Evoluation Criteria:
metric.
Relevance (0-100) Assess how well the caption aligns with the content of the image. The caption should accurately describe or complement the visual elements and contert of the image. Consider the caplion captures the key uspects of the image, its mood, and its intent, and whether it adds value by enhancing the viewer's underslanding or experience of the image.
Evaluation Guidelines:
1. Examine the Image: Carefully observe the image to undersland ils main elements, contert, and message. Review the Caption: Analyze if the caption accurately and effectively describes or complements the image. Consider the appropriateness of the language, tone, and whether the caption adds meaningful context insight.
3. Rate the Caption's Relevonce
Scale of 0 to 100:
90- 10:: The caption is highly relevant, fully capturing the essence of the image with precise and insightful description or commentary, adding signúficant value to the image.
80-89: The caplion is mostly relevanl, capluring most key elements of the image with minor omissiion slightly less impaciful language, still adding clear value.
70-79: The caption is somewhat relevant, capturing some key aspects but missing others, or includes minor irrelevant details, with noticeable but limited enhancement to the image. 50-69: The caplion has limiled relevance, covering only few elements of the image or providing a description that is either too generic or somewhat off-target, adding minimal value.
30-49: The caption is marginally relevant, with sigréficant omissions or 'naccuracies, possibly detracting from the image by misrepresenting or providing little no useful contert.
10-29: The caption is largely irrelevand, missing the key aspects of the image, with significant imaccuracies muisrepresentations, adding value even confusing the viewer. 0-9: The caplion is completely irrelevand nonsensioal, with connection to the image, possibly confusing or misleading the viewer.
Example:
Image:
[Image is attached below]
Caption: [(Caption]}
First Assistant's 's Evaluation: ((Peer_Response Response1]] Second Assistant's 's Evaluation: [{Peer Response2]]
Evoluation Form (Answer by starting with "Analysis:" " to analyze the provided example regarding the evaluation criteria, incorporating the peer rotings, and then give the numeric rating the next line by "Raling".)
Caption Quality:
R.2.2 AGIQA
Image Quality Peer Prompt
You will be given on image generated based on an input prompt. Your task to rate the image on one metric.
39

---

Published in Transactions on Machine Learning Research (08/2025)
Please make you read and understand these instructions carefully. Please keep this document open while reviewing, and refer to it needed.
Evaluation Crileria:
Quauee_uallittyy (0-5) the overall visual coherence and alignment with the input prompt. This rating should reflect how well the image matches the prompl, considering the clarity, relevance, and composition of the image.
Evoluation Steps:
Review the "Input Prompt" carefully to understand the intended content, theme, and style. Examine the generated image and compare it to the "Input Prompt". Check if the image accurately represenls the prompl, is visually cleur, and the composition aligns with the expected oulcome. Assign score for Image Quality on scale to 5, where is the lowest and is the highest based the Evaluation Criteria.
Example:
Input Prompt: {[Input_ Prompt]}
Generated Image:
[Tmage is allached below]
Evaluation Form (Answer by starting with "Analysis:" to analyze the given example regarding the evaluation criteria concise as possible, and then give the rumeric rating on the next line by "Rating" ".)
Image Quality:
Image Quality Chair Prompt.
You will be given an image generated based on an input prompt, along with initial evaluations from two assistants, referred to as the assistants' evaluations.
Your lask is to rale the image on one metric.
Please read the instructions and criteria below carefully and use them as guide in your evaluation. Evaluation Criteria:
Smage_Qualite (0-5) Assess the visual coherence and alignment of the image with the input prompt. The image should reflect the content, theme, and style described in the prompt, and be visually clear and well-composed.
Evoluation Guidelines:
Review the "Input Prompt" to understand the intended content, theme, and style. Examine the generated image and analyze how well it represents the "Input Prompt" in terms of accuracy, clarity, and composition.
Rate the image's quality on a scale of to 5, with 0 being the lowest quality and being the highest quality.
Scoring Guidelines:
-Score 5.0: The image fully caplures the essence of the prompt with high level of accuracy, clarily, and visual appeal, without any significant errors or irrelevant elements.
4 I Score < The image mostly aligns with the prompt, with minor inaccuracies or less relevant detaúls, but still maintoins a generally high quality.
3 s Score 4: The image partially represents the prompt, with noticeable inaccuracies irrele- vant details, and less coherent visual presentation.
2 < Score < 3: The image has signifieant deviations from the prompt, with major inaccuracies, irrelevant elements, and a disjointed visual composition.
1 S Score < 2: The image fails to represent the prompt accurately, lacks visual coherence, and includes significant errors or irvelevant elements.
40

---

Published in Transactions on Machine Learning Research (08/2025)
0 Score 1: The image is completely unrelated to the prompt.
Example:
Inpul Prompt: ((Inpul }rompl))
Generated Image:
[Tmage is attached below]
First Assistant's ' Evaluation: ((Peer responsel]}
Second Assistant's 's Evaluation: [[Peer response2]] Please provide your analysis and rating as follows:
Evaluation Form (Answer by slarling with "Analysis:" to analyze the given eammpll regarding the evaluation criteria concise as possible, and then give the nameric rating on the next line by "Rating") ) Image_Qualee
R.3
Reasoning
R.3.1
AQuA
AQuA Peer Prompt
You will be provided with a problem that requires logical reasoning, mathematical calculation, or both. Your task is to solve the problem accurately, providing not just the correct answer but also clear explanation of the sleps laken to reach that answer.
It is crucial thoroughly understand the problem and opply the correct principles or formulas to solve it.
Instructions:
Read the problem statement carefully, ensuring you understand all the details and what is required for the solution.
Work through the problem logically and methodically, explaining your reasoning and the steps you take to solve the problem.
Procide the final answer clearly, specifying it by choosing one of the provided options (e.g., A, B, C, etc.).
Problem Stalement: {question)}
Provided Options: {[ppions)} Evoluation Form:
Analysis: [Start with "Analysis:" to provide concise and structured explanation of the steps and reasoning used to solve the problem. Ensure your analysis is clear and follows a logical sequence.] Answer: [Clearly state the final answer only (e.g., A, B, C, etc.) on the line after your analysis.]
AQuA area chair Prompt
You will be provided with problem that requires logical reasoning, mathematical calculation, or both. Along with the problem, you will also receive solutions from three other Language Models (LLMs). Your task is to solve the problem accurately, using the peer responses to inform your approuch. Apply the correct principles or formulas to arrive at the solution, while taking note of any useful insights or errors the peer responses. Instructions:
Understond the Problem: Read the problem statement carefully, ensuring you grasp all details.
41

---

Published in Transactions on Machine Learning Research (08/2025)
Review Peer Responses:
Consider the solutions provided by the LLMs, noting useful approaches
any errors.
Solve the Problem: Work through the problem logically, explaining your reasoning and steps. Utiliie
the peer responses as needed but ensure your solution accurate and complete.
Final Answer:
Clearly state the fonal unswer, choosing one of the provided options (e.g., A, B, C,
etc.).
Problem Statement: [question]}
Provided Options: {(ptions}}
Solutions by Other LLMs:
LLM Answer: {[Peer {[Peer_responae responsel]
LLM Answer:
LLM 3 Answer:
((Peer_responsse responsee]
Evoluation Form:
[Start with "Analysis:", provide a concise explonation of your reasoning and sleps, inte- grating relevant insights from the LLMs' responses.]
Analysis:
[Clearly state the final answer label ONLY (e.g., A, B, C, etc.) on the line after your analysis. (DO NOT GIVE ANYTHING ELSE).]
Answer:
R.3.2 BBH_DU
BBH DU Peer Prompt
You will be provided with problem that requires understanding and interpreting dates or times logically. Your task is to solve the problem accurately, providing not just the correct answer but also clear eoplonation of the steps taken to reach that answer.
It is crucial to thoroughly understand the problem, applying the correct principles or formulas to arrive at the solution. Instructions:
Read the problem statement carefully, ensuring you understand all the details and what required for the solution.
Work through the problem logically and methodically, explaining your reasoning and the steps you take to solve the problem.
Provide the final answer clearly, specifying it by choosing one of the provided options (e.g., A, B, C, elc.).
Problem Statement: {fquestion}}
Evaluation Form:
Analysis: [Start with "Analysis:" to provide concise and structured explanation of the steps and reasoning used to solve the problem. Ensure your analysis is clear and follows logical sequence.]
Answer: [Clearly state the final answer only (e.g., A, B, C, etc.) on the line after your analysis.)
BBH DU AreaChain Prompt
You will be provided with a problem that requires understanding and interpreting dates or times logically. You will also receive the final onswers from three other Language Models (LLMs).
42

---

Published in Transactions on Machine Learning Research (08/2025)
Your task is to solue the problem accurately, using the answers provided by the LLMs to inform your reasoning. Provide clear explanalion of your approach, and orrive al your own. final answer. Instructions:
Understand the Problem: Read the problem statement carefully, ensuring you grasp all details.
Review Peer answers: Consider the final answers provided by the LLMs, noting any patterns or outliers.
Work through the problem logically, explaining your reasoning and steps. Use the peer answers as reference but ensure your solution accurate and complete. Clearly state the final answer, choosing one of the provided options (e.g., A, B, C,
Solve the Prohlem:
Final Answer:
elc.).
Problem Statement: {[question}}
Answers from Other LL.Ms:
LLM answer: ([Peer responsessss }
LLM answer: ((Peer response2] }
LLM 3 answer: [iPPeer_repponee]]
Evaluation Form:
Analysis: [Start with "Analysis:" to provide a concise explanation of your reasoning and steps to solve the problem, using the peer answers as a reference.]
[Clearly state the final answer label ONLY (e.g., A, B, C, etc.) on the line after your analysis. (DO NOT GIVE ANYTHING ELSE).]
Answer:
R.3.3 CSQA
CSQA Peer Prompt
Evaluate the question by selecting the best option from the provided choices. Your task is to understand the contert and vuances of the question, utilize your knowledge of the topic, and determine the most appropriate onswer based on the options given. The gool to select the most relevant and correct option that aligns with the question's intent. Instructions:
Understand the Question: Read the question carefully to comprehend all aspects and the contert in which it is asked.
Consider the Options: Analyze each provided option carefully. Think about how each option relates to the question and the scenario it presents.
Select the Best Option: Choose the option that best onswers the question, based on your analysis. Focus on the logic or knowledge that supports this choice.
Problem Statement: {qquestion}}
Provided Options: {(options}} Evaluation Form:
Analysis: [Begin with "Analysis:" to provide structured and clear erplanation of your reasoning process. Your analysis should logically explain why the chosen option is the most appropriate onswer to the question.)
Answer: [Clearly state the final answer only (e.g., A, B, C, etc.) on the line after your analysis.)
43

---

Published in Transactions on Machine Learning Research (08/2025)
CSQA area chair Prompt
You will be provided with question that requires careful evaluation to select the best oplion from the provided choices. You will also receive the final answers from three other Language Models (LLMs). Your task is to determine the most oppropriate answer, using the onswers provided by the LLMs to inform your reasoning. Provide clear explanation of your thought process and select the option that best aligns with the question's 's intent. Instructions:
Understand the Question: Read the question carefully to comprehend all aspects and context.
Review Peer Answers: Consider the final onswers procided by the LLMs, noting any potterns or outliers.
Select the Best Option: Based on your understanding and the peer onswers, choose the option that best answers the question.
Final Answer: Clearly state the final answer, choosing one of the provided options (e.g., A, B, C, etc.).
Problem Stalement: [[quesiion}}
Provided Options: {options}]
Answers from Other LLMs:
LLM Answer: {[Peer responsel]: }
LLM Answer: [(Peer_responsee
LLM Answer: respossss))))
Evaluation Form:
Analysis: [Start with "Analysis:" to provide concise and clear explanation of your reasoning, using the peer answers reference.]
Answer: (Clearly slate the final answer label ONLY (e.g., A, B, C, elc.) on the line after your
analysis. (DO NOT GIVE ANYTHING ELSE).]
R.3.4 GSM8k
GSM8k Peer Prompt
You will be provided with problem that requires logical reasoning, mathematical calculation, both. Your task to solve the problem accurately, providing not just the answer but also a clear explanation of the steps taken reach that answer.
It is crucial to understand the problem thoroughly and apply the correct principles or formulas to solve it.
Instructions:
Read the problem statement carefully, ensuring you understand all the details and what is required for the solution.
Work through the problem logically and methodically, ezplaining your reasoning and the steps you take to solve the problem.
Provide the final answer clearly, specifying it numerical value specific explanation required by the problem statement.
Problem Statement: {(question}}
Evoluation Form:
44

---

Published in Transactions on Machine Learning Research (08/2025)
Analysis: [Start with "Anolysis:" to provide concise and structured erplanation of the steps and rensoning used to solve the problem. Ensure your onalysis is clear and follows logical sequence.]
Answer: [Clearly state the final answer only (neumber) on the line after your analysis.]
GSM8k area chair Prompt
You will be provided with a problem that requires logical reasoning, mathematioal calculation, or both. You will also veceive the final onswers from three other Language Models (LLMs).
Your task is to solve the problem accurately, using the peer answers to inform your reasoning. Provide clear explanation of your thought process and the steps taken to arrive at the solution. Ensure that your reasoning sound and the final onswer is correct. Instructions:
Understond the Problem: Read the problem statement carefully to ensure you grasp all the details and what is required.
Review Peer Answers: Consider the final answers provided by the LLMs, noting any potterns or outliers.
Work Through the Problem: Solve the problem methodically, using the peer onswers reference. Explain your reasoning clearly.
Final Answer: Provide the final onswer clearly, specifying it as a numerical value or as required by the problem statement.
Problem Slatement: [{queslion}}
Answers from Other LLMs:
LLM Answer: {[Peer responsel]]
LLM 2 Answer:
LLM Answer:
respose_))))))
Evoluation Form:
Analysis: [Start with "Analysis:" to provide concise and clear explanation of your reasoning, using the peer answers as a reference.]
Answer: [Clearly state the final answer ONLY (number) the line after your analysis. (DO NOT
GIVE ANYTHING ELSE).
45


Reviews:
[{"review_id": "RbGeweYFtn", "reviewer": "Reviewer_SJ6p", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strengths\n1. The question of how we can evaluate LLM outputs, including in cases where there is no reference text, is an increasingly important and timely one. As human evaluations are costly, using evaluator LLMs to judge the outputs of other LLMs is an important & active research area, where the proposed approach improves upon the state of the art.\n\n2. The proposed approach is simple, intuitive, effective, and easy to implement, which increases the likelihood of being adopted more broadly in the field.\n\n3. The paper features evaluations on multiple tasks & benchmarks, in addition to thorough ablations. This improves the credibility of the findings, and the ablations are useful to better understand and pinpoint the optimal hyper-parameters (how many peer reviewers, which model to use as the area chair, etc.) for using the proposed approach.\n\n# Weaknesses\n1. Results on the Topical Chat dataset are slightly worse than the G-Eval baseline in terms of the average Kendall-Tau correlations (Table 2, \"Average\" column), even though the ReFeR Turbo & Lite results are markedly better than the baseline in terms of the Spearman correlation. It'd be nice to understand & analyze why this is the case.\n\n2. While there is a baseline that simply averages the 3 peer review evaluator LLMs (e.g. Table 2, \"Peer Average\" row), this is not exactly a fair comparison, because the ReFeR approach uses 4 models (3 peer reviewers + 1 area chair), rather than 3. It would be good to add a baseline that averages all 4 models, possibly with a higher weighting for the area chair model, which is the strongest one. This would help ascertain that the ReFeR approach is doing more than a simple averaging.\n\n3. I still have a lot of minor comments regarding the terminology, typos, presentations, and potential extensions. These are listed under the \"Suggestions / Questions / Typos\" section below.\n\n# Suggestions / Questions / Typos\n1. This is a minor terminology point, but I am a little ambivalent about the term \"ReFeR-Turbo\". The reason is that the word Turbo is typically associated with increased speed, which I don't think is what the authors mean (there is a ReFeR-Light version for increased speed / less computational load). Maybe \"Pro\" or something like that?\n\n2. Rather than taking n=20 samples from the area chair model (as is the case for ReFeR-Turbo), which is problematic because: (i) many of the models do not support taking 20 samples, and (ii) the samples are obtained from the same model, which means that they are likely to be correlated, it would be great to see results with multiple (e.g. 2) area chair models. Given that using n=1 (ReFeR-Lite) is not much worse than ReFeR-Turbo, I think using n=1 from 2 different, held-out area chair models could potentially yield better / competitive results than using n=20 from the same area chair model.\n\n3. Rather than using manually defined prompts to create the reviewers & area chair model, would using automated prompt discovery methods (e.g. prompt tuning & prefix tuning) and / or parameter-efficient fine-tuning (e.g. LoRA, adapters, etc.) yield better peer reviewers & area chair models?\n\n4. Could the peer reviewers / area chair model generate out-of-range scores (e.g. \"6\" while the range is 1-5)? If so, what happens in that case?\n\n5. How much worse would the result be if ReFeR uses different personas of the same model for the peer reviewers, rather than 3 different models?\n\n6. Typo: On page 6, \"... comparable performance which being significantly cheaper\". **which** should be replaced by **while**.\n\n7. Would it be possible to ensemble different metrics, e.g. G-Eval & ReFeR Turbo / Lite? It would be great if we can get better results with ensembling, which suggests that the metrics are complementary.\n\n8. Typo: On page 8, \"... can result in performance better than the base ....\". **performance better** -> **better performance**.\n\n9. In Table 7, the row below GPT-4o-mini just says \"ReFeR\". This is ambiguous; do the authors mean \"ReFeR-Turbo\"?\n\n10. Would the results be improved if the peer reviewers first do debating, and then revise their scores accordingly? Then the area chair evaluator LLM can take these post-debate peer reviewers' output, and then synthesize & adjudicate those instead of the pre-debate ones.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces ReFeR, a new method for conducting reference-free evaluation of LLMs & VLMs. To that end, the proposed approach leverages multiple **evaluator** LLMs in a setting that resembles peer review, where a few of the evaluator LLMs act as \"peer reviewers\" that each assign a score & a comment of the evaluated output. These scores & comments are then sent to an \"area chair\" evaluator LLM, which takes into account and adjudicates the peer reviewers' scores & comments, and then outputs a final score & comment for the evaluated output. To reduce variance & improve performance further, the more powerful version of ReFeR, denoted as ReFeR-Turbo, takes multiple outputs (up to n=20) from the \"area chair\" LLM, and produces a final score by doing a simple averaging of the area chair LLM's generated scores.\n\nExperiments on various NLG, multimodal, and reasoning benchmarks demonstrate that the proposed approach (ReFeR-Turbo & ReFeR-Lite) generally outperforms other evaluator LLM baselines like G-Eval & LLM-as-Judge (for NLG evaluations) and zero-shot CoT (Kojima et al., 2023) for reasoning, and also outperforms each of the individual area chair / reviewer model performance. The paper then proceeds to conduct various ablations, concluding that: (i) going beyond 3 peer reviewer evaluator LLMs consisting of strong models provides only marginal gains; (ii) the strongest model should be the area chair, and that a group of capable peers should be chosen based on their individual performance; and (iii) the area chair has a higher probability of producing the correct answer if at least one of the peer reviews is correct, demonstrating that the area chair evaluator LLM *does* take into account the peer reviewer evaluator LLMs' output into account, although this effect is weaker for the reasoning tasks, suggesting that the area chair LLM uses more of its own knowledge for these types of tasks.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. **Critical**: Elaborate & analyze on why the G-Eval baseline is performing slightly better than ReFeR-Turbo in terms of Kendall-Tau correlation in Table 2. (Weakness 1)\n\n2. **Critical**: Add another baseline that averages the 3 peer reviewer scores, plus the area chair score (i.e. average the scores from 4 models, rather than 3), potentially, with a larger weighting for the area chair model which is the strongest one. (Weakness 2)\n\n3. **Strongly recommended**: Address the comments, suggestions, potential extensions, and typos on the suggestions section above.", "broader_impact_concerns": "No broader impact concerns come to mind.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "6pgnJUFGeC", "reviewer": "Reviewer_mdx1", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\nComprehensive evaluation across multiple domains (NLG, multimodal, reasoning)\nStrong ablation studies examining peer selection, number of peers, and area chair choice\nDetailed analysis including FLOPs vs accuracy comparison\nWell-documented implementation details and prompting strategies\n\nWeaknesses:\n\nThe \"inference time compute\" framing appears superficial - the approach essentially involves calling multiple APIs, which is not a novel contribution\nThe comparison between ReFeR-Lite (n=1) and baselines (n=20) is somewhat obvious; claiming \"12-14x compute efficiency\" provides limited additional insight\nThe framework represents primarily an engineering and empirical setup rather than offering fundamentally new knowledge to the community\nThe paper focuses heavily on accuracy improvements but doesn't explore other important production metrics like output consistency\nMissing analysis on how peer reviewers complement the area chair's capabilities - are they covering different aspects/expertise or just adding another call (for instance, what if we just replace 1 call to peer with another gpt4o-mini as a peer? ). It would otherwise not been convincing.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces ReFeR (Reason-Feedback-Review), a hierarchical multi-agent framework for evaluating generative model outputs. The framework uses smaller language models as \"peer reviewers\" and a larger model as an \"area chair\" to evaluate text and image generation quality. The authors present two variants: ReFeR-Turbo (using n=20 responses from the area chair) and ReFeR-Lite (n=1). The main contributions are: (1) A hierarchical evaluation framework inspired by academic peer review, (2) Evaluation on NLG tasks (TopicalChat, SummEval), multimodal tasks (ICQD, AGIQA), and reasoning benchmarks", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Provide deeper theoretical or methodological insights beyond the empirical setup. What fundamental principles make this hierarchical approach work better?\nInclude analysis of output consistency across multiple runs - in production settings, consistent scoring is crucial\nCharacterize what specific expertise or perspectives the peer reviewers contribute that the area chair (GPT-4o-mini) lacks. The accuracy boost alone doesn't capture this important aspect", "broader_impact_concerns": "not applicable.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "0i6WrqVAlZ", "reviewer": "Reviewer_EWhs", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strengths\n- **Well-motivated framework design**: The academic peer review analogy is intuitive and provides clear structure for the multi-agent approach. The hierarchical design with specialized roles is conceptually sound.\n- **Comprehensive evaluation**: The paper demonstrates effectiveness across diverse tasks (NLG evaluation, multimodal assessment, reasoning) with consistent improvements over baselines.\n- **Thorough analysis**: The ablation studies on peer selection, communication strategies, and error analysis provide valuable insights into framework behavior.\n\n# Weaknesses\n- **Limited technical novelty**: While the peer review framing is intuitive, the core contribution is essentially ensemble learning with LLMs. The technical advances beyond existing multi-agent approaches are incremental.\n- **Questionable baseline comparisons**: For multimodal tasks, comparing against training-free methods while excluding trained models like ImageReward/PickScore creates unfair comparisons; The reasoning baselines are somewhat limited - missing recent strong multi-agent reasoning methods\n- **Cost-benefit analysis unclear**: ReFeR-Turbo uses 4 models vs single model baselines, making cost comparisons misleading; The \"12-14× more efficient\" claim for ReFeR-Lite needs clearer justification; Table 2 shows ReFeR-Turbo as most expensive (1.0 relative cost) but claims about efficiency are confusing", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces ReFeR, a hierarchical multi-agent framework for evaluating generative outputs (text and images) and enhancing reasoning tasks. ReFeR employs a two-tiered structure: multiple \"peer\" models (e.g., smaller LLMs/VLMs) independently evaluate outputs, and a more capable \"area chair\" model synthesizes their feedback to produce final scores and explanations. The framework offers two variants: ReFeR-Turbo (high accuracy, higher compute) and ReFeR-Lite (compute-efficient, comparable performance). Experiments across four NLG/multimodal evaluation benchmarks and four reasoning tasks demonstrate ReFeR’s superiority over single-model and multi-agent baselines. The authors also highlight its utility for generating feedback-rich datasets to distill evaluation capabilities into smaller models.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Missing discussion of some recent multi-agent reasoning works in related work and ensemble learning approaches in NLP.\n- How does performance scale with number of peers beyond the tested range? The diminishing returns observed suggest optimal peer counts may vary by task.\n- The finding that \"scores only\" works best is interesting but counterintuitive - this deserves deeper investigation.", "broader_impact_concerns": "No", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)