Venue: NAACL 2025

# ReIFE: Re-evaluating Instruction-Following Evaluation

 Yixin Liu\({}^{1}\)  Kejian Shi\({}^{*}\)\({}^{1}\) Alexander R. Fabbri\({}^{2}\) Yilun Zhao\({}^{1}\)

Peifeng Wang\({}^{2}\)   Chien-Sheng Wu\({}^{2}\)    Shafiq Joty\({}^{2}\)   Arman Cohan\({}^{1,3}\)

\({}^{1}\)Yale University  \({}^{2}\)Salesforce AI  \({}^{3}\)Allen Institute for AI yixin.liu@yale.edu, kejian.shi@yale.edu, arman.cohan@yale.edu

Equal contributionReIFE stands for **Re-evaluation of Instruction**-**F**ollowing **E**valuation: [https://github.com/yale-nlp/ReIFE](https://github.com/yale-nlp/ReIFE).

###### Abstract

The automatic evaluation of instruction following typically involves using large language models (LLMs) to assess response quality. However, there is a lack of comprehensive evaluation of these LLM-based evaluators across two dimensions: the base LLMs and the evaluation protocols. Therefore, we present a thorough meta-evaluation of instruction following, including 25 base LLMs and 15 recently proposed evaluation protocols, on 4 human-annotated datasets, assessing the evaluation accuracy of the LLM-evaluators. Our evaluation allows us to identify the best-performing base LLMs and evaluation protocols with a high degree of robustness. Moreover, our evaluation reveals key findings: (1) Base LLM performance ranking remains largely consistent across evaluation protocols, with less capable LLMs showing greater improvement from protocol enhancements; (2) Robust evaluation of evaluation protocols requires many base LLMs with varying capability levels, as protocol effectiveness depends on the base LLM used; (3) Evaluation results on different datasets are not always consistent, so a rigorous evaluation requires multiple datasets with distinctive features. We release our meta-evaluation suite ReIFE,1 which provides the codebase and evaluation result collection for over 500 LLM-evaluators, laying groundwork for future research in instruction-following evaluation.

Footnote 1: REIFE stands for **Re-evaluation** of **I**nstruction-**F**ollowing **E**valuation: [https://github.com/yale-nlp/ReIFE](https://github.com/yale-nlp/ReIFE).

_Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)_, pages 12247–12287 April 29 - May 4, 2025 ©2025 Association for Computational Linguistics

## 1 Introduction

The ability to follow human instructions has become an important evaluation aspect for large language models (LLMs), indicating their alignment with human users [16]. Recently, due to their better correlation with human judgments compared with traditional evaluation metrics, the LLMs themselves are often used as judges of the model output quality for generative tasks including instruction following [15, 17, 18]. These LLM-based evaluation methods are an essential component of the most widely used automatic benchmarks for instruction-following evaluation, such as AlpacEval [15] and MTBench [18], where a strong LLM is used to evaluate the quality of model responses. Moreover, they can be used as reward models for instruction fine-tuning of LLMs in both distillation and self-improvement settings [23, 24]. However, recent studies have identified various limitations of LLM-based evaluation methods, including low self-consistency rates in their predictions, positional biases, and a preference for their own outputs [15, 16, 17, 18].

Therefore, the evaluation of LLM-based evaluations is critically important. Such evaluations of evaluation methods, or meta-evaluation, usually involve comparing the automatic evaluation results against human evaluation [15, 16, 18]. These evaluations of LLM-evaluators assess two dimensions: (1) the capabilities of _base LLMs_ in performing the evaluation task and (2) the effectiveness of _evaluation protocols_ -- the methods by which base LLMs are used to perform evaluation, e.g., pairwise comparison as in AlpacEval or pointwise scoring as in MTBench.2 Existing work [18, 17, 16] often lacks comprehensiveness in one or both of these dimensions, and more thorough evaluations are needed.

Footnote 2: We use “LLM-evaluator” to refer to an evaluation method that combines a base LLM and an evaluation protocol.

We argue that the following two directions are crucial for a more comprehensive, rigorous evaluation of LLM-evaluators for instruction following: (1) **Including a diverse set of base LLMs

for the evaluation of evaluation protocols - while various evaluation protocols have been proposed recently Gong and Mao (2023); Saha et al. (2023); Chan et al. (2024); Jeong et al. (2024), meta-evaluation studies of these evaluation protocols often lack scale in the number of LLMs used. For example, LLMBar Zeng et al. (2024) uses only 5 LLMs to compare different evaluation protocols. As a result, it remains unclear whether the improvements observed in recently introduced evaluation protocols are robust and generalizable across base LLMs with varying performance levels. Therefore, we aim to conduct an evaluation with a larger and more diverse set of base LLMs to ensure a more rigorous examination of the evaluation protocols.

* various related studies use only a limited number of evaluation protocols when assessing the evaluation capabilities of different base LLMs Liu et al. (2023a); Dubois et al. (2024). However, LLMs' performance can be sensitive to prompt design Sclar et al. (2024), raising doubts about the reliability of using a single protocol for evaluation. Consequently, we aim to achieve a more reliable evaluation of base LLMs by including a larger set of recently proposed evaluation protocols to account for performance variations from prompt/protocol configurations.**

Based on these goals, we present an in-depth meta-evaluation with the following components:

**(1) We perform a solid baseline evaluation across 4 meta-evaluation datasets by evaluating 38 base LLMs and 3 evaluation protocols used in existing benchmarks: Alpacaeval, ArenaHard Li et al. (2024b), and WildBench Lin et al. (2024) (SS4).

**(2) We gather 15 diverse evaluation protocols based on previous work, applying a unified prompting style for a fair comparison, and evaluate their average performance with 25 open-source LLMs (SS5). (3) We leverage the 375 LLM-evaluators evaluated in SS5, to perform a thorough analysis of the practice of meta-evaluation itself, addressing research questions concerning base LLMs, evaluation protocols, and datasets used in the meta-evaluation process, presenting guidelines for best practices (SS6).**

Our large-scale meta-evaluation, as outlined in Figure 1, enables a rigorous examination of the current progress in LLM-based instruction-following evaluation, providing a solid foundation for developing evaluation protocols and evaluating base LLMs' evaluation capabilities. We make our meta-evaluation suite ReIFE publicly available to facilitate future research, which contains the codebase and evaluation result collection for over 500 LLVMators, and we summarize our key findings:

Findings(1) When used in conjunction with 15 evaluation protocols, Llama-3.1-405B Dubey et al. (2024) is the best open-source base LLM we evaluated (Table 6), which approaches state-of-the-art proprietary LLM performance (Table 2).

(2) The evaluation protocols used in 3 widely used benchmarks fail to outperform even the base evaluation protocol evaluated in this work (Table 3). In contrast, the recently introduced evaluation protocol, prepair Jeong et al. (2024), achieves the highest average performance across 25 open-source LLMs, with 7 of the protocols evaluated significantly outperforming the base protocol (Table 4).

(3) The performance ranking of different base LLMs is largely consistent across different evaluation protocols, suggesting that evaluating different base LLMs' evaluation capabilities with a single

Figure 1: Overview of our large-scale meta-evaluation study of instruction-following evaluation. We evaluate the capabilities of 25 open-source base LLMs and 15 evaluation protocols, resulting in a total of 375 LLM-evaluators – evaluation methods that perform the evaluations using the base LLMs by following the evaluation protocols.

 

evaluation protocol is likely to yield reliable results (SS6.1). However, the benefits of advanced protocols vary across LLMs, with less capable LLMs more likely to gain greater improvements (Table 7).

(4) The effectiveness of evaluation protocols depends significantly on the base LLMs used (SS6.2). For example, although prepair achieves the highest average performance, it ranks only seventh among the 15 evaluation protocols when comparing their optimal performance achieved with the most compatible base LLMs (Table 8). This highlights the need to use multiple base LLMs with varying performance levels for reliable evaluation of evaluation protocols.

(5) Different meta-evaluation datasets can exhibit varying difficulty levels, and the LLM-evaluator rankings on these datasets do not always show a strong positive correlation, demonstrating the importance of incorporating diverse datasets for a more comprehensive meta-evaluation (SS6.3).

## 2 Related Work

LLM-based EvaluationUsing LLMs as evaluators has become a promising approach for assessing text generation quality Chiang Lee (2023); Fu et al. (2023); Liu et al. (2023a) in tasks like summarization Fu et al. (2023); Liu et al. (2023a, b) and instruction-following Zheng et al. (2024); Zeng et al. (2024); Li et al. (2023c).

Recent work has proposed various advanced LLM-based evaluation methods. For example, fine-grained or decomposition-based approaches, such as Chain-of-Aspects Gong and Mao (2023) and Branch-Solve-Merge Saha et al. (2023), can guide LLMs to perform structured analysis by identifying fine-grained differences and providing detailed rationales. Agent-based methods, like PRD Li et al. (2023b) and ChatEval Chan et al. (2024), employ multi-role debate settings to bring diverse perspectives to the evaluation process. Other techniques include probability-weighted scoring Liu et al. (2023a), reference-based evaluation Zeng et al. (2024), and self-consistency decoding Wang et al. (2023). Our study investigates the effectiveness of these advanced evaluation protocols on a larger scale, assessing their performance across multiple datasets and base LLMs.

Related studies have also explored fine-tuning LLMs as evaluators for various evaluation tasks including instruction-following evaluation Li et al. (2023a); Wang et al. (2024a), such as Prometheus Kim et al. (2024). However, we choose to exclude them from the majority of our evaluation since our focus is on generic LLMs with various evaluation protocols, while the fine-tuned LLMs usually require a fixed evaluation protocol.

Human Evaluation and Meta-Evaluation of Instruction-FollowingA series of recent studies have conducted human evaluations on instruction-following and/or performed evaluations of automatic evaluators using the collected human annotations Zhang et al. (2023); Wang et al. (2024b, c); Lan et al. (2024). Among them, the annotations from AlpacaFarm Dubois et al. (2024) and MTBench Zheng et al. (2024) have become important testbeds for evaluating widely used LLM evaluators. Zeng et al. (2024) introduce LLMBar, which consists of high-quality human annotations with a high level of inter-annotator agreement rate. RewardBench Lambert et al. (2024) provides a benchmark for evaluating reward models used for learning from human or LLM feedback Ouyang et al. (2022); Bai et al. (2022); Tunstall et al. (2023). While sharing a similar task format, our evaluation focus is different from theirs because we aim to assess the evaluation capability of generic LLMs instead of dedicated reward models.

## 3 Evaluation Settings of ReIFE

In ReIFE, we evaluate LLM-based instruction-following evaluations along two dimensions: base LLMs and evaluation protocols (Figure 1), using human evaluations as the gold standard. Below, we outline the settings of this evaluation.

DatasetsWe use four datasets to evaluate the LLMs' capability of instruction-following evaluation. Each dataset includes human annotations for pairwise comparisons of two outputs of an instruction, with a binary label indicating which output is better in instruction following. Table 1 summarizes the dataset information. LLMBar-Natural and LLMBar-Adversarial are from Zeng et al. (2024), consisting of data examples examined and edited by the paper authors. MTBenchZheng et al. (2024) contains expert human annotations made by graduate students for multi-turn conversations. InstrSumLiu et al. (2024) contains human annotations for instruction-controllable summarization, where the input includes a source article and a specific summary requirement, as a complex instruction. We only use its annotation data samples with 

perfect annotator agreement to reduce the annotation noise. In Appendix A, we show data examples from the datasets. We selected these datasets due to their varying difficulties, instruction complexity, and human annotation noise. For example, InstrSum contains much longer instructions than MBench that have agreement than the others.

**LLM-Evaluator Settings** Since all the datasets we use contain _pairwise_ human evaluations, we evaluate LLM-evaluators under the same pairwise comparison setting for evaluation target alignment. We use the term "LLM-evaluator" to refer to a combination of an _base LLM_ and an _evaluation protocol_. An evaluation protocol defines how the base LLM performs the evaluation, typically using one or more prompts to query it. By default, we use greedy decoding to ensure deterministic behavior.

**Evaluation Metrics** We mainly use _evaluation accuracy_ to evaluate the LLM-evaluators, which measures the alignment with human evaluations using human annotations as the gold standard. Since the LLM-evaluators perform pairwise comparisons, to account for potential _position biases_, where the LLM-evaluators may favor either the first or the second output Wang et al. (2024), we report the averaged evaluation accuracy across two directions, swapping the order of the two outputs. An auxiliary metric we used is the _self-agreement rate_ of the LLM-evaluators in their predictions across two directions in Krippendorff's alpha Krippendorff (2011), measuring their positional biases.

## 4 Baselines

We first establish baselines for base LLMs and evaluation protocols regarding evaluating instruction-following for our further investigations.

### Baselines for Base LLMs

To benchmark the baseline performance of base LLMs at instruction-following evaluation, we evaluate them with a simple evaluation protocol to construct the corresponding LLM-evalutors. This _base_ evaluation protocol, proposed in Zeng et al. (2024), requires the LLM-evaluators to directly predict which output is better, with rules to constrain output formats and to avoid potential biases.3

Footnote 3: The prompt template is in Appendix C.1.

Table 2 presents the evaluation accuracy of 38 proprietary and open-source LLMs, together with two state-of-the-art reward models, nemotron-4-34ob-rm Adler et al. (2024) and offsetbias-rm Park et al. (2024), and two strong fine-tuned LLM-evaluators, portheus-2-8x7b Kim et al. (2024a) and offsetsbias-lmPark et al. (2024) as baselines. The model information is in Appendix B at Table 12. We note the following observations:

(1) **Proprietary vs. Open-Source**: the open-sourced llama-3.1-405b outperforms most of the proprietary LLMs, and llama-3.1-70b lags just slightly behind gpt-4o and gpt-4-0613.

(2) **Performance Gap**: The LLMs at the lower end, such as llama-2-7b and gamma-2b, achieve an accuracy near 50%, comparable to a random oracle. On the other hand, llama-3.1-405b achieves a high accuracy of approximately 84%.

(3) **Dataset Difficulty**: There is also a large difference in the average LLM performance across different datasets. For example, the average evaluation accuracy on LLMBar-Natural is around 20% higher than LMLBar-Adversarial.

(4) **Comparisons with Reward Models and Finetuned LLMs**. The strongest LLM-evaluators outperform the state-of-the-art reward models and fine-tuned LLM-evaluators. The fine-tuned LLM-evaluator, offsetbias-lm, shows a significant improvement over its base model, llama-3-8b, suggesting the potential of fine-tuned LLM-evaluators. Meanwhile, prometheus-2-8x7b only outperforms its base model (mixtral-8x7b) on the easier datasets LLMBar-Natural and MTBench, indicating a lack of robustness.

These baseline results show that the top open-source LLMs already approach the performance of their proprietary counterparts and offer a wide performance spectrum. Therefore, for transparency and reproducibility, we will use mostly open-source LLMs in the rest of our evaluations.

### Baselines for Evaluation Protocols

We now establish a baseline for evaluation protocols, which define how the base LLM is used to

\begin{table}
\begin{tabular}{l c r r r r} \hline  & Abbr. & **In.L.** & **Out.L.** & **Num.** & **Agr.** \\ \hline LLMbar-Natural & Nat. & 53.3 & 56.9 & 100 & 90\% \\ LLMbar-Adversarial & Adv. & 26.7 & 112.4 & 319 & 95\% \\ MBench & MT. & 58.8 & 192.4 & 200 & 81\% \\ Instrum & Ins. & 1149.8 & 109.2 & 411 & 100\% \\ \hline \end{tabular}
\end{table}
Table 1: Dataset information including abbreviation used (**Abbr.**), average instruction (**In.L.**) and output length (**Out.L.**) in words, number of examples (**Num.**), and annotation agreement rate (**Agr.**).

 

perform the evaluation. To this end, we evaluate the evaluation protocols used in three automatic LLM benchmarks for instruction-following4 - AlpacaEval Li et al. (2023c), ArenaHard Li et al. (2024b), and WildBench Lin et al. (2024). Each of these benchmarks uses their evaluation protocol together with a strong base LLM, e.g., GPT-4 Achiam et al. (2023), to perform _pairwise_ comparison of different LLMs' outputs. The individual comparison results are then aggregated to produce a performance ranking of various LLMs. We note that the efficacy of these benchmarks is evaluated at the _system level_, where their produced ranking is compared against the system ranking from human evaluation benchmarks, e.g., ChatBot Arena Chiang et al. (2024). In contrast, here we aim to evaluate the performance of their evaluation protocols at the _instance level_, measuring their evaluation accuracy against human annotations at individual data instances.

Footnote 4: The prompt templates are in Appendix C.2.

In Table 3, the benchmark evaluation protocols are compared against the base protocol Zeng et al. (2024) used in SS4.1, where they are used together with the 25 open-source base LLMs evaluated in SS4.1 and the strongest proprietary LLM, gpt-4o. It shows that the benchmark protocols cannot outperform the base protocol, especially on the more challenging LUMBar-Adversarial and InstruSum datasets. This indicates that the complex design of the benchmark protocols, which often includes detailed instructions on the evaluation plan and output structure, cannot improve the LLMNearators performance at the instance level. In the next section, we will provide a further examination of various evaluation protocols.

## 5 Evaluating Evaluation Protocols

In SS4, we only tested the LLM-evaluators with the base and benchmark evaluation protocols. We now expand the evaluation dimensions to include various protocols proposed in recent work. By using 25

\begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Model** & **Nat.** & **Adv.** & **MT.** & **Ins.** & **Avg.** \\ \hline Proprietary LLMs & & & & & \\ gpt-4o-24-08-06 & **97.5** & 84.5 & 79.8 & **81.3** & **85.7** \\ o1-mini-24-09-12 & 92.5 & **88.6** & 79.0 & **81.3** & 85.3 \\ gpt-4-0613 & 95.5 & 79.3 & 81.5 & 80.4 & 84.2 \\ gpt-4o-24-05-13 & 95.5 & 80.7 & 79.5 & 80.3 & 84.0 \\ calude-3.5-sonnet & 91.0 & 81.2 & 78.5 & 77.5 & 82.0 \\ claude-3-opus & 94.0 & 76.8 & 75.5 & 74.1 & 80.1 \\ mixtral-large & 90.0 & 72.1 & 79.0 & 78.5 & 79.9 \\ gemini-1.5-pro & 87.0 & 74.9 & 78.5 & 75.7 & 79.0 \\ gemini-1.5-flash & 87.5 & 71.3 & 77.8 & 77.5 & 78.5 \\ gpt-4o-mini & 88.5 & 68.3 & 80.2 & 76.6 & 78.4 \\ gemini-1.0-pro & 85.5 & 54.5 & 70.8 & 68.7 & 69.9 \\ gpt-3.5-turbo-0125 & 82.5 & 36.4 & 72.8 & 63.5 & 63.8 \\ calude-3-haktu & 76.0 & 42.9 & 68.8 & 62.8 & 62.6 \\ \hline Opensome ELMs & & & & & \\ llama-3.1-d05b & 94.0 & 83.1 & 81.5 & 79.6 & 84.5 \\ llama-3.1-70b & 90.5 & 79.3 & 82.2 & 79.4 & 82.9 \\ llama-3.70b & 87.0 & 72.7 & 80.0 & 78.6 & 79.6 \\ qwen-2.72b & 92.5 & 69.4 & 82.2 & 73.1 & 79.3 \\ qwen-2.5-72b & 90.5 & 67.7 & **82.5** & 74.1 & 78.7 \\ qwen-1.5-72b & 88.5 & 59.7 & 75.0 & 69.2 & 73.1 \\ glm-4.9b & 86.0 & 55.0 & 73.5 & 73.4 & 72.0 \\ yi-1.5-34b & 86.5 & 56.6 & 73.8 & 66.9 & 70.9 \\ tulu-2-dpo-70b & 85.5 & 58.9 & 73.2 & 66.1 & 70.9 \\ tulu-2-70b & 86.5 & 58.0 & 74.5 & 64.7 & 70.9 \\ mixtral-8x7b & 80.5 & 58.9 & 73.0 & 68.7 & 70.3 \\ vi-1.5-9b & 85.0 & 59.1 & 72.5 & 63.1 & 69.9 \\ qwein-1.5-32b & 85.5 & 47.3 & 76.8 & 66.2 & 68.9 \\ lama-3.1-sb & 78.0 & 50.9 & 72.5 & 66.5 & 67.0 \\ llama-2.70b & 80.0 & 32.4 & 72.2 & 66.9 & 62.9 \\ llama-3.8b & 70.5 & 43.6 & 72.5 & 61.7 & 62.1 \\ mistral-7b-v0.3 & 64.5 & 48.0 & 66.3 & 60.6 & 59.8 \\ tulu-2-drop-13b & 67.0 & 38.6 & 65.5 & 61.2 & 58.1 \\ tolu-2-13b & 65.5 & 38.6 & 65.5 & 61.8 & 57.8 \\ llama-2-13b & 65.0 & 36.4 & 66.8 & 60.8 & 57.2 \\ tulu-2-dpo-Tb & 56.0 & 43.4 & 58.5 & 58.9 & 54.2 \\ germma-7b & 52.5 & 39.3 & 64.5 & 57.4 & 53.4 \\ ttulu-2-7b & 45.5 & 46.9 & 55.2 & 57.8 & 51.4 \\ llama-2-7b & 42.5 & 49.5 & 52.0 & 56.4 & 50.1 \\ gamma-2b & 42.5 & 44.8 & 54.5 & 56.6 & 49.6 \\ \hline Reward Models & & & & & \\ offsetbias-rm & 93.0 & 77.1 & 81.0 & 74.0 & 81.3 \\ nemotron-4.340b & 95.0 & 84.6 & 75.5 & 69.3 & 81.1 \\ Fine-tuned LLMs & & & & & \\ offsetbias-lm & 88.0 & 79.9 & 80.0 & 74.8 & 80.7 \\ prometheus-2 & 83.0 & 37.3 & 76.0 & 64.4 & 65.2 \\ avg. & 80.7 & 60.2 & 73.4 & 69.3 & 70.9 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Evaluation accuracy of various base LLMs with the base evaluation protocol. Models are ordered by their performance. Nat. is LUMBar–Natural, Adv. is LUMbar-Adversarial, MT. is MTSench, Ins. is InstruSum. Best column performance is bolded, and best group performance is underlined.

\begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Protocol** & **Nat.** & **Adv.** & **MT.** & **Ins.** & **Avg.** \\ \hline \multicolumn{5}{c}{Average across 25 Open-Source Base LLMs} \\ \hline base & 74.7 & **53.5** & 70.7 & **66.0** & **66.2** \\ arena-hard & **76.3** & 46.2 & **72.1** & 64.5 & 64.8 \\ wild-bench & 74.9 & 47.5 & 70.7 & 63.0 & 64.0 \\ alpaca-eval & 65.3 & 49.8 & 63.1 & 59.4 & 59.4 \\ \hline \multicolumn{5}{c}{gpt-4o-2024-0806 as Base LLM} \\ \hline base & **97.5** & **84.5** & 79.8 & **81.3** & **85.7** \\ arena-hard & 94.5 & 78.8 & 83.2 & 75.4 & 83.0 \\ wild-bench & 95.5 & 75.5 & **84.0** & 73.7 & 82.2 \\ alpaca-eval & 94.0 & 70.2 & 83.7 & 76.5 & 81.1 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Evaluation accuracy of evaluation protocols in existing LLM benchmarks compared against the base evaluation protocol.

 

open-source LLMs, we believe this evaluation will provide a fairer and more rigorous examination.

### Evaluation Protocols

In our evaluation, we examine 15 protocols derived from previous work. To address the unavailability of some prompt templates and to ensure a fair comparison, we design prompt templates ourselves when necessary. We ensure that all prompt templates adhere to unified formatting and style, and we refine them iteratively to make sure that the protocols can perform to their full potential. The evaluated protocols are outlined below, with their prompt templates provided in Appendix C.

Baseline Protocol(1) base: the vanilla approach used in SS4 which directly predicts the pairwise comparison outcome, proposed in Zeng et al. (2024).

Enhanced ProtocolsFive other protocols from Zeng et al. (2024) are evaluated, which include various enhancements based on the base protocol: (2) cot: the LLM is asked to provide a chain-of-thought Wei et al. (2022) explanation before making the final decision. (3) metric: the LLM is prompted to generate a few metrics for the evaluation task first, which are later used in the actual evaluation. (4) reference: the LLM is prompted to generate a "reference" output for the given instruction, which is later used in the actual evaluation. (5) metric+reference: a combination of the metric and reference methods. (6) swap&synthesize: based on cot and inspired by Du et al. (2024), this method requires the LLM to resolve self-disagreement in predictions from two output orders and make a final decision.

Complex ProtocolsBeyond the enhanced protocols, 7 complex protocols are evaluated based on evaluation methods proposed in previous work. (7) fine-grained-diff: Similar to Min et al. (2023), this protocol guides the LLM to first identify _fine-grained differences_ in output pairs and then provide a detailed rationale for choosing the better output considering these differences. (8) multi-role-round1 & (9) multi-role-round2 Inspired by frameworks that use multiple agents as judges Li et al. (2023b; Chan et al. 2024); Zhao et al. (2024), these two protocols use a _multi-role debate_ setting where multiple evaluators are instantiated from an LLM using prompts with specific role descriptions, to bring diverse perspectives to the evaluation process. The evaluators will generate their responses sequentially, potentially in _multiple rounds_, to engage in a debate that leads to the final prediction. We evaluate its single-round and two-round variants. (10) multi-aspect-two & (11) multi-aspect-single: Similar to several related studies Saha et al. (2023); Gong and Mao (2023); Li et al. (2023a, 2024a), this protocol performs a _multi-aspect comparison_ of the output pairs, with two variants: the _two-stage_ protocol prompts the LLMs to evaluate each quality aspect in separate inference passes, while the _single-stage_ protocol requires the LLMs to conduct a multi-aspect evaluation in a single inference pass before making the final prediction. (12) gpt4-reference: Similar to the reference protocol, this protocol uses a reference output generated by gpt-4o to assist the evaluation. (13) prepair: Adapted from Jeong et al. (2024), this protocol incorporates pointwise reasoning within a pairwise evaluation framework, leveraging the robustness of pointwise evaluation against biases while maintaining the comparative benefits of pairwise evaluation. Self-consistency ProtocolsSelf-consistency is a commonly used decoding approach that can improve the LLMs' performance in various reasoning tasks Wang et al. (2023). Used together with CoT prompting, self-consistency generates the final prediction by taking a majority vote on the predictions made in each generation pass. (14) cot&self-consistency: Self-consistency in pairwise comparison determines the more frequently preferred output. We use a sampling temperature of 0.7 and generate 9 CoTs for voting. (15) protocol-consistency: Beyond different CoTs, a majority vote can be applied across various evaluation protocols. We evaluate this approach using the 5 _enhanced_ protocols.

### Results

Evaluation AccuracyTable 4 demonstrates the evaluation accuracy of various protocols averaged over different base LLMs. We note the following: (1) prepair achieves the strongest average performance, achieving a 1.7% higher accuracy compared to the base protocol. (2) multi-aspect-two achieves the best performance on LLMBar-Natural and MTBench. However, its performance on LLMbar-Adversarial 

ranks among the worst. This highlights that the protocol performance can significantly vary across different datasets.

(3) On average, most complex protocols fail to outperform the base protocol, despite their higher computational costs and multi-step, fine-grained nature, indicating that designing a robust, superior evaluation protocol is a non-trivial task.

(4) Similarly, the approaches that have been proven effective on various reasoning tasks, chain-of-thought (cot) and self-consistency (cot&consistency), also fail to bring significant improvement over the base protocol.

**Self-Agreement Rate** Appendix D shows the protocols self-agreement rates of different evaluation protocols, demonstrating that swap&synthesize can significantly enhance the self-agreement rate.

**Best LLM-Evaluators** Table 5 displays the LLM-evaluators that achieve the highest evaluation accuracy on each dataset, together with the evaluation accuracy of the same base LLM achieved with the base protocol. It shows that llama-3.1-405b, which achieves the strongest performance among the open-source LLMs, remains the strongest base LLM across three datasets. On the other hand, the evaluation protocols used by the best LLM-evaluators on the four datasets differ, indicating greater variance in their capabilities.

## 6 Analysis

In SS5, a total number of 375 LLM-evaluators are evaluated, combining 25 base LLMs and 15 evaluation protocols. We now present detailed analyses of the base LLMs, evaluation protocols, and datasets, using these comprehensive evaluation results to address a series of specific research questions.

### Analysis of Base LLMs

What is the average performance of the base LLMs across different protocols?Table 6 displays the LLMs' average evaluation accuracy over 15 protocols, showing that llama-3.1-7bb is the strongest evaluator on average, while qwen-2-72b achieves the best performance on MTBench.

How does base LLMs' ranking change with different protocols?Figure 2 shows the evaluation accuracy of base LLMs achieved with the base protocol and the average accuracy across different

\begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Dataset** & **LLM** & **Protocol** & **Acc.** & **B.Ace.** \\ \hline Nat. & lama-3.1-405b swap8synthesize & 98.0 & 94.0 \\ Adv. & lama-3.1-405b gpt4-reference & 87.8 & 83.1 \\ MT. & qwen-2.5-72b metric+reference & 84.0 & 82.5 \\ Ins. & lama-3.1-405b prepair & 82.7 & 79.6 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Best LLM-evaluators identified on each dataset. Acc. is the evaluation accuracy of the best LLM-evaluator, **B.Ace.** is the evaluation accuracy achieved by the same base LLM with the base protocol.

\begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Protocol** & **Nat.** & **Adv.** & **MT.** & **Ins.** & **Avg.** \\ \hline prepair (13) & \({}^{\dagger}\)76.4 & **161.8** & 69.7 & \({}^{\ddagger}\)63.8 & \({}^{\ddagger}\)**67.9** \\ gpt4-reference (12) & \({}^{\dagger}\)76.7 & \({}^{\ddagger}\)58.0 & 70.1 & 66.0 & \({}^{\ddagger}\)67.7 \\ metric+reference (5) & \({}^{\dagger}\)76.6 & \({}^{\ddagger}\)58.3 & \({}^{\ddagger}\)70.0 & 65.6 & \({}^{\ddagger}\)67.6 \\ protocol-consistency (15) & \({}^{\dagger}\)76.3 & \({}^{\ddagger}\)55.9 & 70.9 & 66.1 & \({}^{\ddagger}\)67.3 \\ metric (3) & \({}^{\dagger}\)75.8 & \({}^{\ddagger}\)56.2 & 70.7 & 65.7 & \({}^{\ddagger}\)67.1 \\ reference (4) & \({}^{\dagger}\)76.2 & \({}^{\ddagger}\)57.5 & \({}^{\ddagger}\)69.4 & \({}^{\ddagger}\)65.2 & \({}^{\ddagger}\)67.1 \\ swap\&synthesize (6) & 75.6 & \({}^{\ddagger}\)54.4 & 70.8 & **66.2** & \({}^{\ddagger}\)66.8 \\ cot&consistency (14) & 74.9 & 54.1 & 70.5 & \({}^{\ddagger}\)65.4 & \({}^{\ddagger}\)66.2 \\ base (1) & 74.7 & 53.5 & 70.7 & 66.0 & 66.2 \\ cot (2) & \({}^{\dagger}\)73.6 & 53.6 & 70.2 & \({}^{\ddagger}\)64.9 & \({}^{\ddagger}\)65.6 \\ multi-aspect-two (10) & \({}^{\dagger}\)**77.1** & \({}^{\ddagger}\)42.3 & \({}^{\dagger}\)**72.5** & \({}^{\ddagger}\)62.3 & \({}^{\ddagger}\)63.6 \\ fine-grained-diff (7) & \({}^{\ddagger}\)71.2 & \({}^{\ddagger}\)49.5 & \({}^{\ddagger}\)69.2 & \({}^{\ddagger}\)61.8 & \({}^{\ddagger}\)62.9 \\ multi-role-round1 (8) & \({}^{\ddagger}\)68.0 & 53.9 & \({}^{\ddagger}\)66.2 & \({}^{\ddagger}\)61.6 & \({}^{\ddagger}\)62.4 \\ multi-role-round2 (9) & \({}^{\ddagger}\)68.4 & 53.4 & \({}^{\ddagger}\)65.7 & \({}^{\ddagger}\)61.7 & \({}^{\ddagger}\)62.3 \\ multi-aspect-single (11) & \({}^{\ddagger}\)69.6 & \({}^{\ddagger}\)40.8 & 70.5 & \({}^{\ddagger}\)62.4 & \({}^{\ddagger}\)60.8 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Average evaluation accuracy of different evaluation protocols across various LLMs, ordered by their average performance. The protocol indexes introduced in §5.1 are in parentheses. \({}^{\dagger}\), \({}^{\ddagger}\): significantly better or worse than the base protocol (\(p<0.05\)).

 

protocols. The results demonstrate a high positive correlation between them, achieving a Spearman's coefficient of 0.983. This indicates that **using a single base protocol for the evaluation of the base LLMs' is likely to yield reliable results**.

How large is the optimal improvement gained from different evaluation protocols for base LLMs?Table 7 displays the optimal evaluation accuracy improvement (\(\tilde{s}\)) achieved by different evaluation protocols over the base protocol for various base LLMs. That is, \(\tilde{s}=\max_{p\in\mathcal{P}}s(p)-s(\hat{p})\), where \(s(p)\) is the evaluation accuracy of an evaluation protocol \(p\), \(\hat{p}\) denotes the base protocol, \(\mathcal{P}\) is the set of protocols excluding \(\hat{p}\). The results indicate that **less capable LLMs are more likely to achieve larger improvements when the suitable protocols are used**, showing a -0.455 Spearman's correlation between the base LLMs' performance with the base protocol and their optimal performance with the most compatible evaluation protocol. We hypothesize this is because the inductive biases and constraints introduced by the more complicated protocols help less capable LLMs overcome their potential biases and limitations.

### Analysis of Evaluation Protocols

What is the evaluation protocols' optimal performance?In Table 4 of SS5.2, the evaluation protocols' performance is evaluated across all base LLMs. Table 8 instead shows the optimal performance of evaluation protocols, i.e., their evaluation accuracy with the most compatible base LLM. The results show that the **evaluation protocols' optimal performance can significantly differ from their average performance**. For example, while prepair achieves the best average performance, it ranks only 7th in terms of optimal performance.

Figure 2: Correlation between the base LLMs’ evaluation accuracy with the base protocol and their average accuracy across 15 protocols. The fitted regression line and the 95% confidence interval are displayed.

\begin{table}
\begin{tabular}{l c r r r r} \hline \hline
**Model** & **Nat.** & **Adv.** & **MT.** & **Ins.** & **Avg.** \\ \hline mistral-7b-v0.3 & 16.0 & 7.8 & 8.2 & **5.5** & **9.4** \\ llama-2-7b & **23.5** & 0.2 & **11.7** & 1.2 & 9.2 \\ lulu-2-7b & 21.0 & 2.7 & 10.3 & 1.5 & 8.8 \\ llama-3-8b & 9.5 & 17.9 & 3.0 & 4.5 & 8.7 \\ yi-1.5-34b & 5.0 & 17.6 & 4.5 & 3.6 & 7.7 \\ uulu-2-drop-13b & 10.0 & 11.8 & 6.0 & 1.7 & 7.4 \\ llama-3.1-bb & 5.0 & **18.5** & 3.3 & -0.7 & 6.5 \\ lulu-2-dpo-7b & 15.5 & 5.2 & 5.2 & 0.0 & 6.5 \\ qwen-1.5-32b & 4.0 & 16.5 & 2.0 & 3.3 & 6.4 \\ lulu-2-dpo-70b & 4.0 & 11.0 & 3.0 & 3.5 & 5.4 \\ lulu-2-13b & 5.0 & 10.7 & 4.5 & 1.0 & 5.3 \\ glm-4-9b & 1.0 & 14.1 & 5.0 & 0.9 & 5.2 \\ qwen-2.5-72b & 2.5 & 13.6 & 1.5 & 3.2 & 5.2 \\ mixtral-8x7b & 6.0 & 7.4 & 4.2 & 1.1 & 4.7 \\ gemma-7b & 8.0 & 9.7 & -0.2 & 0.6 & 4.5 \\ llama-2-70b & 2.5 & 11.8 & 0.5 & 3.3 & 4.5 \\ tulu-2-70b & 4.0 & 8.5 & 0.3 & 5.2 & 4.5 \\ llama-3-70b & 4.5 & 8.9 & 2.2 & 2.1 & 4.4 \\ qwen-1.5-72b & 1.0 & 8.6 & 4.0 & 0.7 & 3.6 \\ germma-2b & 6.5 & 5.6 & 2.0 & 0.1 & 3.6 \\ queen-2-72b & 2.0 & 8.8 & 1.2 & 1.8 & 3.5 \\ llama-2-13b & 6.0 & 6.0 & 1.5 & 0.2 & 3.4 \\ llama-3.1-405b & 4.0 & 4.7 & 0.8 & 3.2 & 3.2 \\ llama-3.1-70b & 4.5 & 5.5 & 0.2 & 1.2 & 2.9 \\ yi-1.5-9b & 0.5 & 5.6 & 2.3 & 1.6 & 2.5 \\ \hline \end{tabular}
\end{table}
Table 7: Optimal evaluation accuracy improvement (\(\tilde{s}\)) of base LLMs achieved by the most compatible evaluation protocols, ordered by average improvement.

 

How do base LLMs' capabilities affect evaluation protocol's performance?As shown above, the evaluation protocol's performance can be significantly affected by the base LLMs used. Therefore, we provide a further examination with two groups of LLMs: one containing the strongest 10 LLMs identified in Table 6, and another containing the weakest 10. Figure 4 demonstrates that the effectiveness of evaluation protocols can substantially vary based on the capabilities of the base LLMs used. For example, prepair is significantly better than other protocols with weaker LLMs, while metric+reference works better with stronger LLMs. This suggests that **a robust evaluation of evaluation protocols requires multiple base LLMs with a diverse performance range**.

### Analysis of Datasets

What is the difficulty level of different datasets?With our large set of LLM-evaluators, we can gauge the difficulty of each data example by computing its _correctness rate_, which we define as the average evaluation accuracy across all evaluators. Figure 3 shows the distribution of this correctness rate. We note: (1)LLMBar-Natural, MTBench, and InstrSum have a similar data example difficulty distribution, with a small portion of very challenging examples; (2)LLMBar-Adversarial exhibits a different pattern, where the distribution peaks at examples of medium difficulty. This suggests that different LLM-evaluators may have distinct sets of adversarial examples.

Are LLM-evaluators' rankings consistent over different datasets?To better understand how the LLM-evaluators' performance differs across different datasets, in Figure 5 we present the Spearman's correlations between LLM-evaluators' performance ranking between different datasets. The results show that LMLBar-Natural and MTBench exhibit the highest level of similarity. In contrast, LMMBar-Adversarial displays a much lower correlation with the other datasets, suggesting the necessity of using multiple datasets for evaluation.

## 7 Conclusion

In this work, we conducted a large-scale meta-evaluation of instruction following, examining 25 open-source base LLMs and 15 evaluation protocols while identifying the best-performing LLM-evaluators over 4 datasets. We found that a reliable evaluation of base LLMs' evaluation capabilities can likely be achieved with a single evaluation protocol due to the stability of their performance across different protocols. However, evaluating evaluation protocols should involve a diverse group of base LLMs, as they can significantly impact the evaluation protocols' effectiveness. We hope that our findings and meta-evaluation suite, ReIFE, can pave the way for future studies in this direction.

Figure 4: Evaluation protocols’ evaluation accuracy with the stronger and weaker base LLM groups, with a fitted regression line and a 95% confidence interval.

Figure 5: Spearman’s correlations between the performance ranking of LLM-evaluators on different datasets.

Figure 3: Distribution of the correctness rate of data examples in each dataset over different LLM-evaluators.

 



### Limitations

**Evaluation Scope:** Our evaluation centered around generic LLMs and evaluation protocols. As discussed in SS2, we did not focus on reward models trained to evaluate output quality or LLMs fine-tuned for instruction-following. We note that a future study incorporating these systems could yield more comprehensive results.

**Prompt Variations:** In our evaluations, we aimed to control the impact of prompt design by minimizing unnecessary differences across different evaluation protocols. However, we acknowledge that a more thorough evaluation involving multiple prompt variants for each protocol would likely produce more stable results.

**Qualitative Human Evaluation:** We primarily used high-quality human annotation datasets for our quantitative meta-evaluation. Nevertheless, we recognize the lack of qualitative human evaluation, especially concerning the rationales generated by different LLM-evaluators, which could provide further insights into their limitations. We provide a preliminary case study in Appendix E showcasing the error patterns of the base LLMs, and another in Appendix F demonstrating the effect of different evaluation protocols on the same base LLM.

## Acknowledgements

We are grateful to OpenAI's Researcher Access Program and Together AI for provision of LLM API credits.



Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song, Danielle Pintz, Danny Livshits, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dieuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Frank Zhang, Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Anderson, Graeme Nanil, Gregoire Mialon, Guan Pang, Guillem Cucurell, Hailey Nguyen, Hannah Korevaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel Kloumann, Ishan Misra, Ivan Evtimov, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Mahadeokar, Jeet Shah, Jenmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Lie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnston, Joshua Saxe, Junteng Jia, Kalyan Vasudeh Avala, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, Khalid El-Arini, Krithika lyer, Khitika Malik, Kuemley Chiu, Kunal Bhalla, Lauren Rantala-Yearly, Laurens van der Maaten, Lawrence Chen, Liang Tan, Liz Jenkins, Louis Martin, Lovish Madaan, Lubo Malo, Lukas Blecher, Lukas Landzaat, Luke de Oliveira, Madeline Muzzi, Mahesh Pasupulet, Mannat Singh, Manohar Paluri, Marcin Kardas, Mathew Oldham, Mathieu Rita, Maya Pavlova, Melanie Lambadur, Mike Lewis, Min Si, Mitesk Kumar Singh, Mona Hassan, Namam Goyal, Narjes Torabi, Nikolay Bashlykov, Nikolay Bogoychev, Niladir Chatterji, Olivier Duchenne, Onur Celebi, Patrick Alrassy, Pengchuan Zhang, Pengwei Li, Petar Vasic, Peter Weng, Prijwal Bhargava, Pratik Dubal, Praveen Krishnan, Punit Singh Korura, Puxin Xu, Qing He, Qingxiao Dong, Ragavan Srinivasan, Raj Ganapathy, Ramon Calderer, Ricardo Silveira Cabral, Robert Stojnic, Roberta Raileanu, Rohit Girdhar, Rohit Patel, Romain Sauvestre, Ronnie Polidoro, Roshan Sumbaly, Ross Taylor, Ruan Silva, Rui Hou, Roui Wang, Saghar Hosseini, Sahana Chenchanabsappa, Sanjay Singh, Sean Bell, Seohyun Soria Kim, Sergey Edunov, Shaoliang Nie, Sharan Narang, Sharath Raparthy, Sheng Shen, Shengw Wan, Shruti Bhosale, Shun Zhang, Simon Vandenhende, Soumya Batra, Spencer Whitman, Sten Sootla, Stephane Collot, Suchin Gururangan, Sydney Borodinsky, Tamar Herman, Tara Fowler, Tarek Sheasna, Thomas Georgiou, Thomas Sciaciom, Tobias Speckbacher, Teodor Mihalyo, Tong Xiao, Ujjwal Karn, Vedanuj Goswami, Vibhor Gupta, Vignesh Ramanathan, Viktor Kerkez, Vincent Gonguet, Virginie Do, Vish Voggeti, Vladan Petrovic, Weiwei Chu, Wenhan Xiong, Wenyn Fu, Whitney Meers, Xavier Martinet, Xiaodong Wang, Xiaqqing Ellen Tan, Xinfeng Xie, Xuchao Jia, Xuewei Wang, Yaelle Goldschlag, Yashersh Gaur, Yasmine Babaei, Yi Wen, Yusor Sonjo, Yuchen Zhang, Yue Li, Yuning Mao, Zacharie Delpiere Coudert, Zheng Yan, Zhengrsing Chen, Zoe Papakiops, Aaditya Singh, Aaron Grattafori, Abha Jain, Adam Kelsey, Adam Shainfeld, Adithya Gangidi, Adolfo Victoria, Ahuva Goldstand, Ajay Menon, Ajay Sharma, Alex Boesenberg, Alex Vaughan, Alexei Baewski, Alilie Feinstein, Amanda Kadlet, Antir Sangani, Anam Yuns, Andrei Lupp, Andrew Auyardo, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan, Ankit Ramchandani, Annie Franco, Apazajita Sarraf, Arkabandhu Chowdhury, Ashley Gabriel, Ashwin Bharamebe, Assaf Eisenman, Azadeh Yazdan, Beau James, Ben Maurer, Benjamin Leonhardt, Bernie Huang, Beth Loyd, Betto De Paola, Bhargavi Paranjuepage, Bing Liu, Bo Wu, Boyu Ni, Braden Hancock, Bram Wasti, Brandon Spence, Brani Stojkovic, Brian Gamido, Britt Montalvo, Carl Parker, Carly Burton, Catalina Mejia, Changhan Wang, Changkyu Kim, Chao Zhou, Chester Hu, Ching-Hsiang Chu, Chris Cai, Chris Tindal, Christoph Feichtenhofer, Damcinvin, Dana Beaty, Daniel Kreymer, Daniel Li, Danny Wyatt, David Akkins, David Xu, Davide Testuggine, Delia David, Devi Parikh, Diana Liskovich, Didem Foss, Dingkang Wang, Duc Le, Dustin Holland, Edward Dowling, Eissa Jamil, Elaine Montgomery, Eleonora Presani, Emily Hahn, Emily Wood, Erik Brinkman, Esteban Arcuate, Evan Dunbar, Evan Smothers, Fei Sun, Felix Kreuk, Feng Tian, Firat Ozgenel, Francesco Caggini, Francisco Guzzun, Frank Kanayet, Frank Seide, Gabriela Medina Florez, Gabriella Schwarz, Gada Badeer, Georgia Swee, Gil Halpern, Govind Thattai, Grant Herman, Grigory Sizov, Guangyi, Zhang, Guna Lakksminarayanan, Hamid Shojianazeri, Han Zou, Hannah Wang, Hanwen Zhao, Harourh Habeeb, Harrison Rudolph, Helen Suk, Henry Aspegren, Hunter Goldman, Igor Molybog, Igor Tufunov, Irina-Elena Veliche, Itai Gat, Jake Weissman, James Geboski, James Kohli, Japhet Asher, Jean-Baptiste Gaya, Jeff Marcus, Jeff Tang, Jennifer Chan, Jenny Zhen, Jeremy Reizeenstein, Jeremy Teboul, Jessica Zhong, Jian Jin, Jingyi Yang, Joe Cummings, Jon Carroll, Jon Sherpad, Jonathan McDrie, Jonathan Torres, Josh Ginsburg, Junje Wang, Kai Wu, Kam Hou U, Karan Saxena, Karthik Prasad, Kartikay Khandelwal, Katayoun Zan, Kathy Matosich, Kaushik Veeraraghavan, Kelly Michelena, Keqian Li, Kun Huang, Kulan Chawla, Kushal Lakhotia, Kyle Huang, Lailin Chen, Lakshya Garg, Lavender A, Leandro Silva, Lee Bell, Lei Zhang, Liangpeng Guo, Licheng Yu, Liron Mashkovich, Luca Wehrstedt, Madian Khabsa, Manav Avalani, Manish Bhatt, Maria Tzimpoukelli, Martynas Markus, Matan Hasson, Matthew Lennie, Matthias Reso, Maxim Groshev, Maxim Naumov, Maya Lathi, Meghan Kenelly, Michael L. Seltzer, Michal Valko, Michelle Restrepo, Mihir Patel, Mik Vyatsko, Mikayel Samvelyan, Mike Clark, Mike Macey, Mike Wang, Miquel Jubert Hermoso, Mo Metanat, Mohammad Rastegari, Munishi Bansal, Nandhnii Santhanam, Natascha Parks, Natasha White, Navyata Baswa, Nayan Singhal, Nick Egebo, Nicolas Usunier, Nikolay Pavlovich Laptever, Ning Dong, Ning Zhang, Norman Cheng, Oleg Chermoguz, Olivia Hart, Onkar Salpekar, Ozdem Kalinli, Parkin Kent, Parth Parekh, Paul Saab, Pavan Balaji, Pedro Rittner, Philip Bontrager, Pierre Roux, Piotr Dollar, Polina Zvyagina, Prashant Ratanchandi, Pritish Yuvraj, Qian Liang, Rachad Alao, 

Rachel Rodriguez, Rafi Ayub, Raghotham Murthy, Raghu Nayani, Rahul Mitra, Raymond Li, Rebekkah Hogan, Robin Battey, Rocky Wang, Rohan Ameeswar, Russ Howse, Ruyto Rinott, Sai Jays Benodu, Samyak Datta, Sara Chugh, Sara Hunt, Sargun Dhillon, Sasha Sidorov, Satadru Pan, Saurabh Verma, Seiji Yamamoto, Sharadh Ramaswamy, Shaum Lindsay, Shaun Lindsay, Sheng Feng, Shengaho Lin, Shengxin Cindy Zha, Shiva Shankar, Shuqiang Zhang, Sinong Wang, Snecha Agarwal, Soji Sajuyigbe, Soumith Chintala, Stephanie Max, Stephen Chen, Steve Kehoe, Steve Satterfield, Sudarshan Govindaprasad, Sumit Gupta, Sungmin Cho, Sunny Virk, Suraj Subramanian, Sy Choudhury, Sydney Goldman, Tal Remez, Tamar Glaser, Tamara Best, Thilo Kohler, Thomas Robinson, Tianhe Li, Tianjun Zhang, Tim Matthews, Timothy Chou, Tzook Shaked, Varvan Vontimitta, Victoria Ajayi, Victoria Montanez, Yijian Mohan, Vinay Satish Kumar, Vishal Mangla, Vlad Ionescu, Vlad Poenaru, Vlad Tiberiu Mihailescu, Vladimir Ivanov, Wei Li, Wenchen Wang, Wenwen Jiang, Wes Bouaziz, Will constable, Xiaocheng Tang, Xiaofang Wang, Xiaojian Wu, Xioolan Wang, Xie Xia, Kiluu Wu, Xinbo Gao, Yanjun Chen, Ye Hu, Ye Jia, Ye Qi, Yenda Li, Yilin Zhang, Ying Zhang, Yossi Adi, Youngjin Nam, Yu, Wang, Yuchen Hao, Yundi Qian, Yuzi He, Zach Rait, Zachary DeVito, Zef Rosnbrick, Zhaoduo Wen, Zhenyu Yang, and Zhiwei Zhao. 2024. The llama 3 herd of models. _arXiv preprint arXiv: 2407.21783_.
* Dubois et al. (2024) Yann Dubois, Chen Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy S Liang, and Tatsunori B Hashimoto. 2024. Alpacafarm: A simulation framework for methods that learn from human feedback. _Advances in Neural Information Processing Systems_, 36.
* Fu et al. (2023) Jinlan Fu, See-Kiong Ng, Zhengbao Jiang, and Pengfei Liu. 2023. GPTScore: Evaluate as you desire. _arXiv preprint arXiv: 2302.04166_.
* Gemini et al. (2023) Team Gemini, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. 2023. Gemini: a family of highly capable multimodal models. _arXiv preprint arXiv:2312.11805_.
* Gemma et al. (2020) Team Gemma, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Suraye Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgena Riviere, Mihir Sanjaye Kale, Juliette Love, Pouya Tarfi, Leonard Hussenot, Pier Giuseppe Sessa, Akanksha Chowdherry, Adam Roberts, Aditya Barua, Alex Botev, Alex Castro-Ros, Ambrose Slone, Amelie Heliou, Andrea Tacchetti, Anna Bulanova, Antonia Paterson, Beth Tsai, Bobak Shahriari, Charline Le Lan, Christopher A. Choquette-Choo, Clement Crepy, Daniel Cer, Daphne Depolito, David Reid, Elena Buchatskaya, Eric Ni, Eric Noland, Gene Yan, George Tucker, George-Christian Muraru, Grigory Rozhodstvenskiy, Henryx Michalewski, Ian Tenney, Ivan Grishchenko, Jacob Austin, James Keeling, Jane Labanowski, Jean-Baptiste Lespiau, Jeff Stanway, Jenny Brennan, Jeremy Chen, Johan Ferret, Justin Chiu, Justin Mao-Jones, Katherine Lee, Kathy Yu, Kate Millican, Lars Lowe Sjoesund, Lisa Lee, Lucas Dixon, Machel Reid, Maciev Mikula, Mateo Wirth, Michael Sharma, Nikolai Chinaev, Nithum Thain, Olivier Bachem, Oscar Chang, Oscar Wahl tizez, Paige Bailey, Paul Michel, Petko Yotov, Rahma Chaabouni, Ramona Comanescu, Renna Jana, Rohan Anil, Ross McIlroy, Rubio Lu, Ryan Mullins, Samuel L Smith, Sebastian Borgoardt, Sertan Girgin, Shotto Douglas, Shree Pandya, Siamak Shakeri, Soham De, Ted Klimenko, Tom Hennigan, Vlad Feinberg, Wojciech Stokowiec, Yu hui Chen, Zafarali Ahmed, Zhitao Gong, Tris Warkettin, Ludovic Peran, Minh Giang, Clement Farabet, Orivi Davis, Jeff Dean, Koray Kavikucoglu, Denis Hassabis, Zoubin Ghharmani, Douglas Eck, Joelle Barral, Fernando Pereira, Eli Collins, Armand Joulin, Noah Fiedel, Evan Senter, Alek Andreev, and Kathleen Keneaky. 2024. Gemma: Open models based on gemini research and technology. _arXiv preprint arXiv: 2403.08295_.
* Gong and Mao (2023) Peiyuan Gong and Jiaxin Mao. 2023. CoAScore: Chain-of-aspects prompting for nlg evaluation. _arXiv preprint arXiv: 2312.10355_.
* Ivison et al. (2023) Hamish Ivison, Yizhong Wang, Valentina Pyatkin, Nathan Lambert, Matthew Peters, Pradeep Dasigi, Joel Jang, David Wadden, Noah A. Smith, Iz Beltagy, and Hannanet Hajishirzi. 2023. Camels in a changing climate: Enhancing lm adaptation with tulu 2. _Preprint_, arXiv:2311.10702.
* Jeong et al. (2024) Hawon Jeong, ChaeHun Park, Jimin Hong, and Jaegul Choo. 2024. PRePair: Pointwise reasoning enhance pairwise evaluating for robust instruction-following assessments. _arXiv preprint arXiv: 2406.12319_.
* Jiang et al. (2023) Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guil laume Lample, Lucile Saulnier, et al. 2023. Mistral ?b. _arXiv preprint arXiv:2310.06825_.
* Jiang et al. (2024) Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lello Renard Lavavad, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock, Sandeep Subramanian, Sophia Yang, Szymon Antoniak, Teven Le Scao, Theophile Gervet, Tibaut Lavril, Thomas Wang, Timothee Lacroix, and William El Sayed. 2024. Mixtal of experts. _ArXiv preprint arXiv: 2401.04088_.
* Kim et al. (2024) Seungone Kim, Jamin Shin, Yejin Cho, Joel Jang, Shayne Longpre, Horaran Lee, Sangdoo Yun, Seongjin Shin, Sungdong Kim, James Thorne, and Minjoon Seo. 2024a. Prometheus: Inducing fine-grained evaluation capability in language models. In _The Twelfth International Conference on Learning Representations_ 

Seungone Kim, Juyoung Suk, Shayne Longpre, Bill Yuchen Lin, Jamin Shin, Sean Welleck, Graham Neubig, Moontae Lee, Kyunjae Lee, and Minjoon Seo. 2024b. Prometheus 2: An open source language model specialized in evaluating other language models. _arXiv preprint arXiv: 2405.01535_.
* Krippendorff (2011) Klaus Krippendorff. 2011. Computing krippendorff's alpha-reliability.
* Lambert et al. (2024) Nathan Lambert, Valentina Pyatkin, Jacob Morrison, LJ Miranda, Bill Yuchen Lin, Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, Noah A. Smith, and Hannaneh Hajishirzi. 2024. RewardBench: Evaluating reward models for language modeling. _arXiv preprint arXiv: 2403.13787_.
* Lan et al. (2024) Tian Lan, Wenwei Zhang, Chen Xu, Heyan Huang, Dahua Lin, Kai Chen, and Xian-ling Mao. 2024. Critecbench: Evaluating large language models as critic. _arXiv preprint arXiv:2402.13764_.
* Li et al. (2023a) Junlong Li, Shichao Sun, Weizhe Yuan, Run-Ze Fan, Hai Zhao, and Pengfei Liu. 2023a. Generative judge for evaluating alignment. _arXiv preprint arXiv:2310.05470_.
* Li et al. (2024a) Minzhi Li, Zhengyuan Liu, Shumin Deng, Shafiq Joty, Nancy F. Chen, and Min-Yen Kan. 2024a. Decompose and aggregate: A step-by-step interpretable evaluation framework. _arXiv preprint arXiv: 2405.15329_.
* Li et al. (2023b) Ruosen Li, Teerth Patel, and Xinya Du. 2023b. PRD: Peer rank and discussion improve large language model based evaluations. _arXiv preprint arXiv: 2307.02762_.
* Li et al. (2024b) Tianle Li, Wei-Lin Chiang, Evan Frick, Lisa Dunlap, Tianhao Wu, Banghua Zhu, Joseph E Gonzalez, and Ion Stoica. 2024b. From crowdsourced data to high-quality benchmarks: Arena-hard and benchbuilder pipeline. _arXiv preprint arXiv:2406.11939_.
* Li et al. (2023c) Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023c. Alpaceaval: An automatic evaluator of instruction-following models. [https://github.com/tatsu-lab/alpaca_eval](https://github.com/tatsu-lab/alpaca_eval).
* Lin et al. (2024d) Bill Yuchen Lin, Yuntian Deng, Khyathi Chandu, Feaze Brahman, Abhilasha Ravichander, Valentina Pyatkin, Nouha Dziri, Ronan Le Bras, and Yejin Choi. 2024. WILDBENCH: Benchmarking llms with challenging tasks from real users in the wild. _arXiv preprint arXiv:2406.04770_.
* Liu et al. (2023a) Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chengquang Zhu. 2023a. G-eval: NLG evaluation using gpt-4 with better human alignment. In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_, pages 2511-2522, Singapore. Association for Computational Linguistics.
* Liu et al. (2023b) Yixin Liu, Alexander Fabbri, Pengfieiu Liu, Song Nan, Rulim Han, Simeng Han, Shafiq Joty, Chien-Sheng Wu, Caiming Xiong, and Dragomir Radev. 2023b. Revisiting the gold standard: Grounding summarization evaluation with robust human evaluation. In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 4140-4170, Toronto, Canada. Association for Computational Linguistics.
* Liu et al. (2023c) Yixin Liu, Alexander Fabbri, Yilun Zhao, Pengfei Liu, Shafiq Joty, Chien-Sheng Wu, Caiming Xiong, and Dragomir Radev. 2023c. Towards interpretable and efficient automatic reference-based summarization evaluation. In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_, pages 16360-16368, Singapore. Association for Computational Linguistics.
* Liu et al. (2024) Yixin Liu, Alexander R Fabbri, Jiawen Chen, Yilun Zhao, Simeg Han, Shafiq Joty, Pengfei Liu, Dragomir Radev, Chien-Sheng Wu, and Arman Cohan. 2024. Benchmarking generation and evaluation capabilities of large language models for instruction controllable summarization. In _Findings of the Association for Computational Linguistics: NAACL 2024_.
* AI (2024) Meta AI. 2024. Introducing meta llama 3: The most capable oppenly available llvm to date. [https://ai.meta.com/blog/meta-llama-3/](https://ai.meta.com/blog/meta-llama-3/). Accessed: 2024-05-30.
* Min et al. (2023) Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. FActScore: Fine-grained atomic evaluation of factual precision in long form text generation. In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_, pages 12076-12100, Singapore. Association for Computational Linguistics.
* Ouyang et al. (2022) Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Alama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instructions with human feedback. In _Advances in Neural Information Processing Systems_, volume 35, pages 27730-27744. Curran Associates, Inc.
* Panickssey et al. (2024) Arjun Panickssey, Samuel R Bowman, and Shi Feng. 2024. LLM evaluators recognize and favor their own generations. _arXiv preprint arXiv:2404.13076_.
* Park et al. (2024) Jungsoo Park, Seungyeon Jwa, Meiying Ren, Daeyoung Kim, and Sanghyuk Choi. 2024. OffsetBias: Leveraging debiased data for tuning evaluators. _arXiv preprint arXiv:2407.06551_.
* Saha et al. (2023) Swarnadeep Saha, Omer Levy, Asli Celikyilmaz, Mohit Bansal, Jason Weston, and Xian Li. 2023. Branch-solve-merge improves large language model evaluation and generation. _arXiv preprint arXiv: 2310.15123_ 

Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. 2024. Quantifying language models' sensitivity to spurious features in prompt design or: How i learned to start worrying about prompt formatting. In _The Twelfth International Conference on Learning Representations_.
* Touvron et al. (2023) Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almhahiri, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajiwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. _arXiv preprint arXiv:2307.09288_.
* Tunstall et al. (2023) Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clementine Fourrier, Nathan Habib, et al. 2023. Zephyr: Direct distillation of LM alignment. _arXiv preprint arXiv:2310.16944_.
* Wang et al. (2024a) Peifeng Wang, Austin Xu, Yilun Zhou, Caiming Xiong, and Shafiq Joty. 2024a. Direct judgement preference optimization. _arXiv preprint arXiv:2409.14664_.
* Wang et al. (2024b) Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Lingpeng Kong, Qi Liu, Tianyu Liu, and Zhifang Sui. 2024b. Large language models are not fair evaluators. In _Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 9440-9450, Bangkok, Thailand. Association for Computational Linguistics.
* Wang et al. (2023) Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-consistency improves chain of thought reasoning in language models. In _The Eleventh International Conference on Learning Representations_.
* Wang et al. (2024c) Yidong Wang, Zhuohao Yu, Wenjin Yao, Zhengran Zeng, Linyi Yang, Cunxiang Wang, Hao Chen, Chaoya Jiang, Rui Xie, Jindong Wang, Xing Xie, Wei Ye, Shikun Zhang, and Yue Zhang. 2024c. PandaLM: An automatic evaluation benchmark for LLM instruction tuning optimization. In _The Twelfth International Conference on Learning Representations_.
* Wang et al. (2024d) Zhilin Wang, Yi Dong, Olivier Delalleau, Jiaqi Zeng, Gerald Shen, Daniel Egert, Jimmy J. Zhang, Makesch Narsimhan Sreedhar, and Oleksii Kuchtaiev. 2024d. HelpSteer2: Open-source dataset for training top-performing reward models. _Preprint_, arXiv:2406.08673.
* Wei et al. (2022) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. _Advances in neural information processing systems_, 35:24824-24837.
* Yuan et al. (2024) Weizhe Yuan, Richard Yuanze Pang, Kyunghyun Cho, Xian Li, Sainbayar Sukhbatafar, Jing Xu, and Jason E Weston. 2024. Self-rewarding language models. In _Forty-first International Conference on Machine Learning_.
* Zeng et al. (2024) Zhiyuan Zeng, Jiatong Yu, Tianyu Gao, Yu Meng, Tanya Goyal, and Danqi Chen. 2024. Evaluating large language models at evaluating instruction following. In _The Twelfth International Conference on Learning Representations_.
* Zhang et al. (2023) Xinghua Zhang, Bowen Yu, Haiyang Yu, Yangyu Lv, Tingwen Liu, Fei Huang, Hongbo Xu, and Yongbin Li. 2023. Wider and deeper llm networks are fairer llm evaluators. _arXiv preprint arXiv:2308.01862_.
* Zhao et al. (2024) Ruochen Zhao, Wenxuan Zhang, Yew Ken Chia, Deli Zhao, and Lidong Bing. 2024. Auto Arena of LLMs: Automating llm evaluations with agent peer-battles and committee discussions. _arXiv preprint arXiv:2405.20267_.
* Zheng et al. (2024)Linamin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2024. Judging llm-as-a-judge with mt-bench and chatbot arena. _Advances in Neural Information Processing Systems_, 36.

## Appendix A Dataset Examples

We randomly select two data examples from each of the four datasets in our study (SS3), and present them in Table 10 and Table 11.

 

better suit our task format, we modify their evaluation task to a binary pairwise comparison. The evaluation protocol of WildBench requires a task-specific checklist of output quality to aid the evaluation. In WildBench, these checklists were created using GPT-4-Turbo and Claude-3-Opus and manually reviewed. Following a similar approach, we use GPT-40 to generate these checklists for the 4 datasets used in this work.

### Prompts for Enhanced Evaluation Protocols

We list the prompts for cot (Figure 10), metric (Figure 11 & Figure 12), reference (Figure 13), metric+reference (Figure 14), and swap8synthesize (Figure 15). These prompt templates are proposed by Zeng et al. (2024).

### Prompts for Complex Evaluation Protocols

We present our prompts for fine-grained-diff (Figure 16), multi-role-round2 (Figure 17), multi-aspect-single (Figure 18), multi-aspect-two (Figure 19 and Figure 20), gpt4-reference (Figure 21), and prepair (Figure 22 and Figure 23).

## Appendix D Self-Agreement Rate of Different Evaluation Protocols

Table 9 shows the protocols self-agreement rates, demonstrating that (1) swap8synthesize can significantly enhance the self-agreement rate; (2) The multi-role debate protocols, multi-role-round182, yield a significantly lower self-agreement rate, indicating that introducing more complex evaluation processes can lead to larger self-inconsistency.

## Appendix E Case Study

We perform a qualitative analysis of the evaluation performance of llama-3-70B and identify three main error patterns that impact its performance in various instances. The following paragraphs outline these error patterns, and we present a specific case study in Table 14.

Surface-level deception (surface)The model tends to favor outputs that appear more positive or have more structured presentations like numbered lists or professional layout despite clear disadvantages in addressing the instruction task compared to less structured but more appropriate and accurate responses. This failure mode is a recognized pattern across LLM-evaluators Zheng et al. (2024); Wang et al. (2024b).

Overlooking crucial context (overlook)The model sometimes fails to identify important context in the question when it is not explicitly emphasized. This can lead to generic responses that miss key elements specific to the task at hand.

Counting issues (counting)This applies to both numerical counting (for example, "generate an 8-letter password") and text-based context-relevant counting. The model often fails to count letters correctly or prefers longer outputs that exceed the instructed requirements, contrary to human annotators' preference for accurate answers.

## Appendix F Generation Samples

We present generation samples across different evaluation protocols of llama-3-70B model for an example from LLMBar-Natural, as shown in Figure 24. Model evaluations of llama-3-70B under selected evaluation protocols are presented in Figure 25 (cot), Figure 26 (fine-grained-diff), Figure 27 (multi-role-round2), and Figure 28 (prepair). Our analysis of the evaluation protocols reveals interesting disparities in their effectiveness. For example, the cot fails because it overlooks the factual error regarding the frequency of the word "humans" in one of the candidate outputs. In contrast, multi-role-round2's success can be attributed to its multi-perspective debate approach,

\begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Protocol** & **Nat.** & **Adv.** & **MT.** & **Ins.** & **Avg.** \\ \hline swap8synthesize (6) & **89.3** & **84.4** & **86.0** & **86.1** & **86.5** \\ preparir (13) & 62.1 & 59.7 & 59.3 & 38.9 & 55.0 \\ multi-aspect-two (10) & 62.4 & 51.9 & 54.6 & 36.6 & 51.4 \\ fine-grained-diff (7) & 53.1 & 45.3 & 54.6 & 33.5 & 46.6 \\ protocol-consistency (15) & 56.5 & 47.0 & 50.9 & 30.7 & 46.3 \\ metric (3) & 53.6 & 48.4 & 52.7 & 29.4 & 46.0 \\ metric+reference (5) & 57.4 & 48.7 & 49.7 & 28.3 & 46.0 \\ base (1) & 54.5 & 47.3 & 49.6 & 32.4 & 46.0 \\ cotkconsistency (14) & 54.0 & 42.0 & 48.1 & 36.9 & 45.2 \\ multi-aspect-single (11) & 45.6 & 44.6 & 50.7 & 59.5 & 44.2 \\ gpt4-reference (12) & 51.3 & 47.2 & 45.9 & 25.7 & 42.5 \\ cot (2) & 47.7 & 38.7 & 45.7 & 34.6 & 41.7 \\ reference (4) & 50.8 & 44.5 & 44.2 & 24.9 & 41.1 \\ multi-role-round1 (8) & 38.3 & 26.8 & 31.8 & 25.4 & 30.6 \\ multi-role-round2 (9) & 37.1 & 25.9 & 31.8 & 25.9 & 30.2 \\ \hline \hline \end{tabular}
\end{table}
Table 9: Average self-agreement rate of various evaluation protocols across various LLMs (ordered). The protocol indexes introduced in §5.1 are in parentheses.

 

which allows for a more thorough examination and fact-checking of the claims made in each output.

 



\begin{table}
\begin{tabular}{p{108.4pt} p{256.9pt}} \hline
**LLMBar–Natural** & Output 2 \\ Explain: Pathos, Ethos, Logos. & Paths is a tool of persuasion that is used to appeal to readers’ emotions by arousing positive or negative feelings. It can be used in rhetoric, literature, film, and other forms of expression. While pathos is used to draw an emotional response, the other rhetorical appeals appeal to credibility and logic, respectively. Ethos is the distinguishing character, sentiment, moral nature, or guiding beliefs of a person, group, or institution. It is often evident as facts, numbers, or statistics and is used to convince an audience of an argument. \\ \hline
**LLMBar–Adversarial** & Name an animal that is commonly kept as a pet. \\ \hline
**Output 1** & **Output 2** \\ Dogs are commonly kept as gets. & Dogs, cats, and fish. \\ \hline
**MTBench** & A is the father of B. B is the father of C. What is the relationship between A and C? \\ \hline
* & **Output 2*
* \\ A is the grandfather of C. &
* A is the father of B \\  &
* B is the father of C \\  &
* Therefore, A is the grandfather of C \\  & So the relationship between A and C is: Grandfather - Grandson \\ \hline
* becoming a new mother in 2020. The mothers speak about their isolation, ... Please summarize the above article based on the given requirement: \\ \hline
**Output 1** & Output 2 \\ Born in Lockdown is a book written by 277 new mothers in the UK about their experiences of becoming a mother during the pandemic. The mothers share their feelings of isolation, uncertainty, and pressure on their mental health, but also highlight the silver linings and positives that have come out of this joint experience. The project was set up by Bristol novelist Emylia Hall to encourage writing for pleasure and purpose. The book is an unflinchingly honest and moving account of new motherhood during the pandemic, where resilience, love, and hope shine through. The book is available for free download from the MothersWitters website, with voluntary donations going to the stillbirth and neonatal death charity Sands. \\ \end{tabular}
\end{table}
Table 10: Selected examples from our datasets (§3). The green label indicates the output is the preferred output.





\begin{table}
\begin{tabular}{l l} \hline \hline
**LUMBar–Natural** & Given the following input statement, provide the type of sentiment in the statement and the extent of sentiment in terms of sentiment \\ scores. This restaurant has the best service I’ve ever seen. You can design the score scale by yourself. \\ \hline
**Output 1** & Output 2 \\ This statement has a sentiment score of 9 / 10 & Positive sentiment, sentiment score: 9.5 out of 10 \\ \hline
**LUMBar–Adversarial** & \\ \hline Analyze the results of the following test scores. 90, 86, 75, 63 & \\ \hline
**Output 1** & Output 2 \\ The average grade is 78.75. & The test scores show a decreasing trend, with the first \\ score being the highest and the last score being the lowest. & This could indicate that the test tacker’s knowledge and \\ skills were increasingly weak as the test progressed. \\ \hline
**MTBench** & \\ \hline Identify the countries, their capitals, and the languages spoken in the following sentences. Output in JSON format. \\ a) Amistk the idylic vistas, Copenhagen, Denmark’s capital, captivates visitors with its thriving art scene and the enchanting Danish language spoken by its inhabitants. & \\ b) Within the enchanting realm of Eldoria, one discovers Avalore, a grandinose city that emanates an ethereal aura. Lumina, a melodious language, serves as the principal mode of communication within this mystical abode. & \\ c) Nestled amrist a harmonious blend of age-old customs and contemporary wonders, Buenos Aires, the capital of Argentina, stands as a bustling metropolis. It is a vibrant hub where the expressive Spanish language holds sway over the city’s inhabitants. & \\ \hline Output 1 & \\ [“sentences”: [ & a) The capital of Denmark is Copenhagen. The language \\ [ “text”: "Ammidt the idyllic vistas, Copenhagen, Denmark’s capital, captivates visitors with its thriving art scene and the enchanting Danish language spoken by its inhabitants.". & \\ ‘country”: Denmark”, & \\ “capital”: Copenhagen”, & \\ “language”: "Danish" & \\ \hline ’text”: Within the enchanting realm of Eldoria, one discovers Avalore, a grandinose city that emanates an ethereal aura. Lumina, a melodious language, serves as the principal mode of communication within this mystical abode. & \\ “country”: "Eledonia (fictional)”, & \\ “capital”: “Avalorie (fictional)” & \\ [ “language”: "Luminia (fictional)” ]. & \\ \hline
**Inext”** & Nested amidst a harmonious blend of age-old customs and contemporary wonders, Buenos Aires, the capital of Argentina, stands as a bastilting metropolis. It is a vibrant hub where the expressive Spanish language holds sway over the city’s inhabitants.”, & \\ \hline
**Output 1** & \\ Research director at The Adam Smith Institute, Sam Bowman, suggests that building on London's green belt lands could provide bigger and cheaper homes for everyone. However, Ann Goddard of Elstroe \& Bordinawwood Greenbelt Society argues that green belt lands are important for recreation purposes and to maintain a separation between settlements. Bowman proposes a compromise by differentiating between areas of beauty and intensive farmland within the green belt. & \\  & \\ \end{tabular}
\end{table}
Table 11: Selected examples from our datasets (§3). The green label indicates the output is the preferred output.





\begin{table}
\begin{tabular}{l c l p{142.3pt} p{142.3pt}} \hline
* & Proprietary & Gemini models are the most capable multimodal models from Google featuring long context lengths (Gemini et al., 2023). \\ gemini-1.5-flash &
- & Proprietary & et al., 2023). \\ \hline qwen-1.5-32b & 32b & Qianwen & Qwen is a family of models built by Alibaba Cloud (Bai et al., 2023). Qwen1.5 and Qwen2 have recently surpassed most open models on common benchmarks. \\ \hline mistral-7b-v0.3 mistral-large & 7b & Apache 2.0 Proprietary & Instruction-tuned versions of Mistral models (Jiang et al., 2023) from Mistral AI. \\ \hline mixtral-8x7b & 8x7b & Apache 2.0 Proprietary & Mikrtal-8x22B is a pretrained generative Sparse Mixture of Experts (MoE) from Mistral AI (Jiang et al., 2024) \\ \hline claude-3-haiku &
- & Proprietary Claude-3-Haiku and Claude 3 Opus, and Claude 3.5 Connet are top proprietary models trained by Anthropic PBC (Claude, 2023). \\ \hline gpt-3.5-turbo-0125 &
- & Proprietary GPT models are strong proprietary models (Achiam et al., 2023) from OpenAI. “o1” model was released in September 2024 with strong reasoning capability. \\ \hline prometheus-2-8x7b & 8x7b & Apache 2.0 Prometabus 2 is an alternative to GPT-4 for fine-grained evaluation of LLMs and reward models for RLHF, based on Mistral-Instructur (Kim et al., 2024b) Offsetbias is a generative judge model for pairwise preference evaluation, designed to be robust against various evaluation biases (Park et al., 2024) \\ \hline nemptron-4-340b-reward & 340b & NVIDIA Open Model & A multi-aspect reward model for synthetic data generation and RLAIF; based on Nemotron-4-340BBase (Wang et al., 2024d) \\ offsetbias-rm & 8b & llama 3 Community & Reward model trained on Offset Bias dataset, designed to be robust against various evaluation biases (Park et al., 2024) \\ \hline \end{tabular}
\end{table}
Table 12: Model registry and metadata in our study used in §4.




\begin{table}
\begin{tabular}{l l l c c} \hline \hline
**Full Name** & **Code** & **Related Work** & **\# Qs./Eval** & **\# Aux.Qs./Eval** \\ \hline Base Pairwise Evaluation & base & Zeng et al.(2024) & 1 & N/A \\ Chain-of-Thought & cot & Zeng et al.(2024; Wei et al.(2022) & 1 & N/A \\ Metric Generation & metric & Zeng et al.(2024) & 1 & 1 \\ Reference Generation & reference & Zeng et al.(2024) & 1 & 1 \\ Metric and Reference & metric+reference & Zeng et al.(2024) & 1 & 2 \\ Swap and Synthesize & swap@skynthesize & Zeng et al.(2024) & 2 & N/A \\ Fine-grained Differences & fine-grained-diff & Inspired by Liu et al.(2023b,c); Min et al.(2023) & 1 & N/A \\ Multi-Role Debate (Single Round) & multi-role-round1 & Chan et al.(2024) & 3 & N/A \\ Multi-Role Debate (Two Rounds) & multi-role-round2 & Chan et al.(2024) & 6 & N/A \\ Multi-Asspect Comparison (Two-Stage) & multi-aspect-two & Li et al.(2024a) & 1 & 5 \\ Multi-Aspect Comparison (Single-Stage) & multi-aspect-single & Li et al.(2024a) & 1 & N/A \\ GPT-4 Reference & sptd-reference & Modified from Zeng et al.(2024) & 1 & N/A \\ Pointwise Reasoning Enhanced Pairwise & prepair & Jeong et al.(2024) & 1 & 2 \\ Self-Consistency over CoTs & codeconsistency & Wang et al.(2023) & 9 & N/A \\ Self-Consistency over Different Protocols & protocol-consistency & Wang et al.(2023) & 5 & N/A \\ \hline \hline \end{tabular}
\end{table}
Table 13: Method registry for evaluation protocols in §5.1. #Qs./Eval indicates the number of queries required per evaluation, and #Aux.Qs./Eval refers to any additional auxiliary queries.





**[System Message]**

You are a helpful assistant in evaluating the quality of the outputs for a given instruction. Your goal is to select the best output for the given instruction.

**[User Message]**

Select the Output (a) or Output (b) that is better for the given instruction. The two outputs are generated by two different AI chatbots respectively.

Here are some rules of the evaluation:

(1) You should prioritize evaluating whether the output honestly/precisely/closely executes the instruction, then consider its helpfulness, accuracy, level of detail, harmlessness, etc.

(2) Outputs should NOT contain more/less than what the instruction asks for, as such outputs do NOT precisely execute the instruction.

(3) You should avoid any potential bias and your judgment should be as objective as possible. For example, the order in which the outputs were presented should NOT affect your judgment, as Output (a) and Output (b) are equally likely to be the better.

Do NOT provide any explanation for your choice.

Do NOT say both / neither are good.

You should answer using ONLY "Output (a)" or "Output (b)". Do NOT output any other words.

# Instruction:

{INSTRUCTION}

# Output (a):

{OUTPUT_1}

# Output (b):

{OUTPUT_2}

Which is better, Output (a) or Output (b)? Your response should be either "Output (a)" or "Output (b)":

Figure 6: Prompt for base protocol described in §5.1.





**[System Message]**

You are a highly efficient assistant, who evaluates and selects the best large language model (LLMs) based on the quality of their responses to a given instruction. This process will be used to create a leaderboard reflecting the most accurate and human-preferred answers.

**[User Message]**

I require a leaderboard for various large language models. I'll provide you with prompts given to these models and their corresponding outputs. Your task is to assess these responses, and select the model that produces the best output from a human perspective.

## Instruction {  "instruction": ""[INSTRUCTION]""",  }

Model Outputs Here are the unordered outputs from the models. Each output is associated with a specific model, identified by a unique model identifier. {  { "model_identifer": "m",  "output": ""[OUTPUT_1]"""  },  {  "model_identifer": "M",  "output": ""[OUTPUT_2]"""  }  }
Task Evaluate the models based on the quality and relevance of their outputs, and select the model that generated the best output. Answer by providing the model identifier of the best model. We will use your output as the name of the best model, so make sure your output only contains one of the following model identifiers and nothing else (no quotes, no spaces, no new lines, ...): Model (m) or Model (M).
## Which is better, Model (m) or Model (M)? Your response should be either "Model (m)" or "Model (M)":

Figure 7: Prompt for AlpacaEval baseline described in §4.2



**[System Message]**

Please act as an impartial judge and evaluate the quality of the responses provided by two AI assistants to the user prompt displayed below. You will be given assistant A's answer and assistant B's answer. Your job is to evaluate which assistant's answer is better.

**[User Message]**

Begin your evaluation by generating your own answer to the prompt. You must provide your answers before judging any answers.

When evaluating the assistants' answers, compare both assistants' answers with your answer. You must identify and correct any mistakes or inaccurate information.

Then consider if the assistant's answers are helpful, relevant, and concise. Helpful means the answer correctly responds to the prompt or follows the instructions. Note when user prompt has any ambiguity or more than one interpretation, it is more helpful and appropriate to ask for clarifications or more information from the user than providing an answer based on assumptions. Relevant means all parts of the response closely connect or are appropriate to what is being asked. Concise means the response is clear and not verbose or excessive.

Then consider the creativity and novelty of the assistant's answers when needed. Finally, identify any missing important information in the assistants' answers that would be beneficial to include when responding to the user prompt.

After providing your explanation, you must always end your response with either "Therefore, Answer (a) is better." or "Therefore, Answer (b) is better." verbatim.

Figure 8: Prompt for ArenaHard described in §4.2



## Appendix

**[System Message] You are an expert evaluator. Your task is to evaluate the quality of the responses generated by two AI models.** **[User Message]****# Instruction We will provide you with the user query and a pair of AI-generated responses (Response A and Response B). You should first read the user query and the conversation history carefully for analyzing the task, and then evaluate the quality of the responses based on and rules provided below.**

### Conversation between User and AI

### User Query

<begin_of_query|>

{INSTRUCTION}

<end_of_query|>

### Response A

<begin_of_response_Ab>

[OUTPUT_1]

<end_of_response_Ab>

#### Response B

<begin_of_response_Bl>

{OUTPUT_2}

<end_of_response_Bb>

#### Evaluation

##### Checklist

<begin_of_checklistl>

{CHECKLIST}

<end_of_checklists> Please use this checklist to guide your evaluation, but do not limit your assessment to the checklist.**

### Rules

You should compare the above two responses based on your analysis of the user query. You should first write down your analysis and the checklist that you used for the evaluation, and then provide your assessment according to the checklist. You should always end your response by selecting the better response.**

### Output Format

First, please output your analysis for each model response, and then summarize your assessment to three aspects: "reason A=B", "reason A>B", and "reason B>A", and finally make your choice for the final assessment by selecting the better response (ties are NOT allowed).Please provide your evaluation results in the following json format by filling in the placeholders in []:

...  { 'analysis of A": "[analysis of Response A]",  'analysis of B": "[analysis of Response B]",  'reason of A=B": "[where Response A and B perform equally well]",  'reason of A>B": "[where Response A is better than Response B]",  'reason of B>A": "[where Response B is better than Response A]",  'choice": "[Response (A) or Response (B)]", }..

Figure 9: Prompt for WildBench baseline described in §4.2



**[System Message]**

You are a helpful assistant in evaluating the quality of the outputs for a given instruction. Your goal is to select the best output for the given instruction.

**[User Message]**

After giving a brief explanation, select the Output (a) or Output (b) that is better for the given instruction. The two outputs are generated by two different AI chatbots respectively.

Here are some rules of the evaluation:

(1) You should prioritize evaluating whether the output honestly/precisely/closely executes the instruction, then consider its helpfulness, accuracy, level of detail, harmlessness, etc.

(2) Outputs should NOT contain more/less than what the instruction asks for, as such outputs do NOT precisely execute the instruction.

(3) You should avoid any potential bias and your judgment should be as objective as possible. For example, the order in which the outputs were presented should NOT affect your judgment, as Output (a) and Output (b) are equally likely to be the better.

You should first provide a brief explanation of your evaluation, and then always end your response with either "Therefore, Output (a) is better." or "Therefore, Output (b) is better." verbatim.

Do NOT say both / neither are good.

Do NOT output any other words.

Do NOT say "Output (a) is better" or "Output (b) is better" at the beginning. You should do reasoning and thinking before claiming which is better.

\begin{tabular}{l l} \# Instruction: & \\ \{INSTRUCTION\} & \\ \end{tabular}

\begin{tabular}{l l} \# Output (a): & \\ \{OUTPUT_1\} & \\ \end{tabular}

\begin{tabular}{l l} \# Output (b): & \\ \{OUTPUT_2\} & \\ \end{tabular}

\begin{tabular}{l l} \# Decision (Give a brief explanation of your evaluation followed by either "Therefore, Output (a) is better." or "Therefore, Output (b) is better." verbatim. Always claim which is better at the end. In your explanation, you should always use "Output (a)" or "Output (b)" to refer to the two outputs respectively.):

* [System Message] You are a helpful assistant.
* [User Message] Please propose at most three concise questions about whether a potential output is a good output for a given instruction. Another assistant will evaluate different aspects of the output by answering all the questions. Here are some rules of the evaluation: (1) You should prioritize evaluating whether the output honestly/precisely/closely executes the instruction. (2) Outputs should NOT contain more/less than what the instruction asks for, as such outputs do NOT precisely execute the instruction.
* # Instruction: {INSTRUCTION}
* # Requirements for Your Output: (1) The questions should **specifically** target the given instruction instead of some general standards, so the questions may revolve around key points of the instruction. (2) You should directly give the questions without any other words. (3) Questions are presented from most important to least important.
* # Please give your questions here:

Figure 11: Prompt for metric generation stage of the metric protocol described in §5.1.

 



**[System Message]**

You are a helpful assistant in evaluating the quality of the outputs for a given instruction. Your goal is to select the best output for the given instruction.

**[User Message]**

Select the Output (a) or Output (b) that is better for the given instruction. The two outputs are generated by two different AI chatbots respectively.

Here are some rules of the evaluation:

(1) You should prioritize evaluating whether the output honestly/precisely/closely executes the instruction, then consider its helpfulness, accuracy, level of detail, harmlessness, etc.

(2) Outputs should NOT contain more/less than what the instruction asks for, as such outputs do NOT precisely execute the instruction.

(3) You should avoid any potential bias and your judgment should be as objective as possible. For example, the order in which the outputs were presented should NOT affect your judgment, as Output (a) and Output (b) are equally likely to be the better.

Do NOT provide any explanation for your choice.

 Do NOT say both / neither are good.

 You should answer using ONLY "Output (a)" or "Output (b)". Do NOT output any other words.

\begin{tabular}{l l} \# Instruction: & \\  & \{INSTRUCTION\} \\  & \{OUTPUT\_1\} \\  &



**[System Message]**

You are a helpful assistant in evaluating the quality of the outputs for a given instruction. Your goal is to select the best output for the given instruction.

**[User Message]**

Select the Output (a) or Output (b) that is better for the given instruction. The two outputs are generated by two different AI chatbots respectively.

Here are some rules of the evaluation:

(1) You should prioritize evaluating whether the output honestly/precisely/closely executes the instruction, then consider its helpfulness, accuracy, level of detail, harmlessness, etc.

(2) Outputs should NOT contain more/less than what the instruction asks for, as such outputs do NOT precisely execute the instruction.

(3) You should avoid any potential bias and your judgment should be as objective as possible. For example, the order in which the outputs were presented should NOT affect your judgment, as Output (a) and Output (b) are equally likely to be the better.

Do NOT provide any explanation for your choice.

Do NOT say both / neither are good.

You should answer using ONLY "Output (a)" or "Output (b)". Do NOT output any other words.

# Instruction:

{INSTRUCTION}

# Output (a):

{OUTPUT_1}

# Output (b):

{OUTPUT_2}

# A reference output generated by a strong AI assistant:

{REFERENCE}

Which is better, Output (a) or Output (b)? Your response should be either "Output (a)" or "Output (b)":

Figure 13: Prompt for reference protocol described in §5.1.





**[System Message]**

You are a helpful assistant in evaluating the quality of the outputs for a given instruction. Your goal is to select the best output for the given instruction.

**[User Message]**

Select the Output (a) or Output (b) that is better for the given instruction. The two outputs are generated by two different AI chatbots respectively.

Here are some rules of the evaluation:

(1) You should prioritize evaluating whether the output honestly/precisely/closely executes the instruction, then consider its helpfulness, accuracy, level of detail, harmlessness, etc.

(2) Outputs should NOT contain more/less than what the instruction asks for, as such outputs do NOT precisely execute the instruction.

(3) You should avoid any potential bias and your judgment should be as objective as possible. For example, the order in which the outputs were presented should NOT affect your judgment, as Output (a) and Output (b) are equally likely to be the better.

Do NOT provide any explanation for your choice.

Do NOT say both / neither are good.

You should answer using ONLY "Output (a)" or "Output (b)". Do NOT output any other words.

# Instruction:

{INSTRUCTION}

# Output (a):

{OUTPUT_1}

# Output (b):

{OUTPUT_2}

# Questions about Outputs:

Here are at most three questions about the outputs, which are presented from most important to least important. You can do the evaluation based on thinking about all the questions.

{QUESTIONS}

# A reference output generated by a strong AI assistant:

{REFERENCE}

Which is better, Output (a) or Output (b)? Your response should be either "Output (a)" or "Output (b)":

Figure 14: Prompt for metric + reference protocol described in §5.1.





**[System Message]**

You are a helpful assistant who reviews a debate between two other assistants in evaluating the quality of the outputs for a given instruction.

**[User Message]**

The two assistants, Assistant (a) and Assistant (b), are given an instruction, Output (a) and Output (b). They are asked to select the Output (a) or Output (b) that is better for the given instruction. Output (a) and Output (b) are generated by two different AI chatbots respectively.

Assistant (a) and Assistant (b) have conflicting evaluations. Your goal is to review their evaluations and give your final decision on which output is better.

Here are some rules of the evaluation:

(1) You should prioritize evaluating whether the output honestly/precisely/closely executes the instruction, then consider its helpfulness, accuracy, level of detail, harmlessness, etc.

(2) Outputs should NOT contain more/less than what the instruction asks for, as such outputs do NOT precisely execute the instruction.

(3) You should avoid any potential bias and your judgment should be as objective as possible. For example, the order in which the outputs were presented should NOT affect your judgment, as Output (a) and Output (b) are equally likely to be the better.

Now carefully review the instruction, Output (a), Output (b), and the debate between Assistant (a) and Assistant (b). Select the Output (a) or Output (b) that is better for the given instruction.

Do NOT provide any explanation for your choice.

Do NOT say both / neither are good.

You should answer using ONLY "Output (a)" or "Output (b)". Do NOT output any other words.

# Instruction:

{INSTRUCTION}

# Output (b):

{OUTPUT_2}

# Output (a):

{OUTPUT_1}

# Debate between Assistant (a) and Assistant (b):

## Evaluation given by Assistant (a), who thinks Output (a) is better:

{EXPLANATION_1}

## Evaluation given by Assistant (b), who thinks Output (b) is better:

{EXPLANATION_2}

Which is better, Output (a) or Output (b)? Your response should be either "Output (a)" or "Output (b)":

Figure 15: Prompt for swap&synthesize protocol described in §5.1.





**[System Message]**

You are a helpful assistant in evaluating the quality of the outputs for a given instruction. Your goal is to select the best output for the given instruction.

**[User Message]**

After giving a detailed explanation, select either Output (a) or Output (b) as the better response for the given instruction. The two outputs are generated by two different AI chatdots respectively.

Here are some rules of the evaluation:

(1) You should prioritize evaluating whether the output honestly/precisely/closely executes the instruction, then consider its helpfulness, accuracy, level of detail, harmlessness, etc.

(2) Outputs should NOT contain more/less than what the instruction asks for, as such outputs do NOT precisely execute the instruction.

(3) You should avoid any potential bias and your judgment should be as objective as possible. For example, the order in which the outputs were presented should NOT affect your judgment, as Output (a) and Output (b) are equally likely to be the better.

You should first provide a detailed explanation of your evaluation, and then always end your response with either "Therefore, Output (a) is better." or "Therefore, Output (b) is better." verbatim. Do NOT say both / neither are good. Do NOT output any other words. Do NOT say "Output (a) is better" or "Output (b) is better" at the beginning. You should do reasoning and thinking before claiming which is better.

Here is the evaluation plan:

1. Differences Identification: Enumerate the key fine-grained content differences observed between Output (a) and Output (b).

2. Explanation and Rationale: Provide explanations and rationale for which output better addresses the instruction by considering these differences, as well as other relevant factors such as relevance, completeness, coherence, and clarity.

3. Final Decision: End your response with either "Therefore, Output (a) is better." or "Therefore, Output (b) is better." verbatim.

Provide your response in the following format:

Differences Identification:

1. [Difference 1]

2. [Difference 2]

...

N. [Difference N]

Explanation and Rationale: [Detailed explanation and rationale for your decision]

Final Decision: Therefore, Output (a)/Output (b) is better.

# Instruction:

{INSTRUCTION}

# Output (a):

{OUTPUT_1}

# Output (b):

{OUTPUT_2}

Your Response (Give a detailed explanation of your evaluation followed by either "Therefore, Output (a) is better." or "Therefore, Output (b) is better." verbatim. Always claim which is better at the end. In your explanation, you should always use "Output (a)" or "Output (b)" to refer to the two outputs respectively.):

Figure 16: Prompt for fine-graine-diff protocol described in §5.1.





**[System Message]**

You are a helpful assistant who evaluates the quality of the outputs for a given instruction. Your goal is to select the best output for the given instruction.

**[User Message]**

Select the Output (a) or Output (b) that is better for the given instruction. The two outputs are generated by two different AI chatbots respectively. Here are some rules of the evaluation:

(1) You should prioritize evaluating whether the output honestly/precisely/closely executes the instruction, then consider its helpfulness, accuracy, level of detail, harmlessness, etc.

(2) Outputs should NOT contain more/less than what the instruction asks for, as such outputs do NOT precisely execute the instruction.

(3) You should avoid any potential bias and your judgment should be as objective as possible. For example, the order in which the outputs were presented should NOT affect your judgment, as Output (a) and Output (b) are equally likely to be the better. You should first provide a brief explanation of your evaluation, and then always end your response with either "Therefore, Output (a) is better." or "Therefore, Output (b) is better." verbatim. Do NOT say both / neither are good. Do NOT output any other words. Do NOT say "Output (a) is better" or "Output (b) is better" at the beginning. You should do reasoning and thinking before claiming which is better. There are a few other referees assigned the same task, it's your responsibility to discuss with them and think critically before you make your final judgement. You should avoid any potential bias and ensure that the order in which the responses were presented does not affect your judgment. Debate with others. Always end your response with "Therefore, Output (a) is better." or "Therefore, Output (b) is better." verbatim. Make sure to make the claim to end your response.

# Instruction: {INSTRUCTION}

# Output (a): {OUTPUT_1}

# Output (b): {OUTPUT_2}

# Previous referees' arguments: {CHAT_HISTORY}

# Your role: {ROLE_DESCRIPTION}

Decision (Give a brief explanation of your evaluation followed by either "Therefore, Output (a) is better." or "Therefore, Output (b) is better." verbatim. Always claim which is better at the end. In your explanation, you should always use "Output (a)" or "Output (b)" to refer to the two outputs respectively.):

Figure 17: Prompt for Multi-role-round2 protocol described in §5.1.





**[System Message]**

You rare a helpful assistant who analyzes and evaluates the quality of two candidate outputs for a given instruction task based on a list of criteria, and makes a final decision on which output is better.

**[User Message]**

Given an instruction and two responses, Output (a) and Output (b), each aiming to fulfill the task, your task is to carefully analyze and evaluate each output based on a list of criteria.

Here are some general rules for the evaluation:

(1) You should prioritize evaluating whether the output honestly/precisely/closely executes the instruction, then consider its helpfulness, accuracy, level of detail, harmlessness, etc.

(2) Outputs should NOT contain more or less than what the instruction asks for, as such outputs do NOT precisely execute the instruction. (3) You should avoid any potential bias and your judgment should be as objective as possible. For example, the order in which the outputs were presented should NOT affect your judgment, as Output (a) and Output (b) are equally likely to be the better one.

You should first provide an explanation of your evaluation, and then always end your response with either "Therefore, Output (a) is better." or "Therefore, Output (b) is better." verbatim.

Do NOT say both / neither are good.

Do NOT output may other words.

Do NOT say "Output (a) is better" or "Output (b) is better" at the beginning. You should do reasoning and thinking before claiming which output is better.

Here are some criteria to consider:

1. Text Quality: The response should be fluent, well-structured, and free of spelling and grammatical errors. It should also be coherent, with a clear and logical flow of ideas.

2. Information Richness: The response is encouraged to provide rich, detailed and professional information, e.g. by providing examples, explanations, citations, and additional information. This criterion is not applicable if the user asks for a short or direct answer without additional information.

3. User Intention Inference: If the user's intention is not clearly expressed by the query, the response should provide some relevant information, do some reasonable inference and ask more information for clarification. This criterion is not applicable if the user's intention is clearly expressed by the query.

4. Accuracy: All contents provided or mentioned in the response should be accurate and correct.

5. Completeness of Instruction Following: For all key instructions (e.g., answer multiple questions or perform multiple tasks) and explicit constraints (e.g. word count, response length limit, word usage, output format, etc.) provided by the user, the response should be complete in following all of them without any omission.

Consider how well each output meets the list of criteria and provide a comparative analysis. After your analysis, make a final decision on which output is better overall. Provide a brief explanation of your evaluation, weighing the importance of each aspect, and make a final decision. Be mindful of the importance of each aspect in the context of the given instruction task, as some aspects may significantly influence the output's quality and relevance to the instruction, while others might be less critical.

Provide your response in the following format:

* 1.Text Quality:

[Your analysis]

2.Information Richness:

[Your analysis]

3. User Intention Inference:

[Your analysis]

4.Accuracy:

[Your analysis]

5.Completeness of Instruction Following:

[Your analysis]

[Your overall evaluation and explanation, followed by the final decision]

* #Instruction:

{INSTRUCTION}

# Output (a):

[OUTPUT_1]

# Output (b):

[OUTPUT_2]

Figure 18: Prompt for multi-aspect-single protocol described in §5.1.



* [System Message] You are a helpful assistant who analyzes and evaluates the quality of the outputs for a given instruction based on specific criteria.
* [User Message] Please provide a detailed analysis and evaluation of the two outputs based on the following criteria: {CRITERIA_TEXT} Consider how well each output meets the criteria and provide a comparative analysis.
* # Instruction: {INSTRUCTION}
* # Output (a): {OUTPUT_1}
* # Output (b): {OUTPUT_2}
* # Your analysis:

Figure 19: Prompt for multi-aspect-two protocol described in §5.1. This is the prompt for aspect-wise analysis (the first stage) within the method.

 



**[System Message]**

You are a helpful assistant who makes a final decision on which output is better based on the analysis of multiple aspects.

**[User Message]**

You have been provided with an instruction and two outputs, along with an analysis of each output based on several key aspects.

Your task is to carefully consider the analysis for each aspect and make a final decision on which output is better overall. Provide a brief explanation of your evaluation, weighing the importance of each aspect, and make a final decision. Be mindful of the importance of each aspect item in the context of the given instruction task, because some aspects may significantly influence the output's quality and relevance to the instruction, while others might be less critical.

Here are some rules of the evaluation:

(1) You should prioritize evaluating whether the output honestly/precisely/closely executes the instruction, then consider its helpfulness, accuracy, level of detail, harmlessness, etc.

(2) Outputs should NOT contain more/less than what the instruction asks for, as such outputs do NOT precisely execute the instruction.

(3) You should avoid any potential bias and your judgment should be as objective as possible. For example, the order in which the outputs were presented should NOT affect your judgment, as Output (a) and Output (b) are equally likely to be the better.

You should first provide an explanation of your evaluation, and then always end your response with either "Therefore, Output (a) is better." or "Therefore, Output (b) is better." verbatim.

Do NOT say both / neither are good.

Do NOT output any other words.

Do NOT say "Output (a) is better" or "Output (b) is better" at the beginning. You should do reasoning and thinking before claiming which is better.

# Instruction:

{INSTRUCTION}

# Output (a):

{OUTPUT_1}

# Output (b):

{OUTPUT_2}

# Here are the aspect-wise analyses provided by another helpful critic:

{ANALYSIS_HISTORY}

Decision (Give a brief explanation of your evaluation followed by either "Therefore, Output (a) is better." or "Therefore, Output (b) is better." verbatim. Always claim which is better at the end. In your explanation, you should always use "Output (a)" or "Output (b)" to refer to the two outputs respectively.):

Figure 20: Prompt for multi-aspect-two protocol described in §5.1. This is the final evaluation prompt (the second stage) within the method.





**[System Message]**

You are a helpful assistant in evaluating the quality of the outputs for a given instruction. Your goal is to select the best output for the given instruction.

**[User Message]**

Select the Output (a) or Output (b) that is better for the given instruction. The two outputs are generated by two different AI chatbots respectively.

Here are some rules of the evaluation: (1) You should prioritize evaluating whether the output honestly/precisely/closely executes the instruction, then consider its helpfulness, accuracy, level of detail, harmlessness, etc.

(2) Outputs should NOT contain more/less than what the instruction asks for, as such outputs do NOT precisely execute the instruction.

(3) You should avoid any potential bias and your judgment should be as objective as possible. For example, the order in which the outputs were presented should NOT affect your judgment, as Output (a) and Output (b) are ***equally likely** to be the better.

Do NOT provide any explanation for your choice.

 Do NOT say both / neither are good.

 You should answer using ONLY "Output (a)" or "Output (b)". Do NOT output any other words.

# Instruction:

{INSTRUCTION}

# Output (a):

{OUTPUT_1}

# Output (b):

{OUTPUT_2}

# A reference output generated by a strong AI assistant: {REFERENCE}

Which is better, Output (a) or Output (b)? Your response should be either "Output (a)" or "Output (b)":



**[System Message] You are a helpful assistant in evaluating the quality of the outputs for a given instruction. Your goal is to evaluate the quality of output for the given instruction. [User Message] Giving a brief explanation to evaluate the quality of the response to the given instruction. The output is generated by an AI chatbot. Here are some rules of the evaluation: (1) You should prioritize evaluating whether the output honestly/precisely/closely executes the instruction, then consider its helpfulness, accuracy, level of detail, harmlessness, etc. (2) The model outputs should NOT contain more/less than what the instruction asks for, as such outputs do NOT precisely execute the instruction. (3) You should avoid any potential bias and your judgment should be as objective as possible. You should provide a brief explanation of your evaluation. Your explanation should identify critical drawbacks in model outputs that do not meet the above evaluation rules. # Instruction: {INSTRUCTION}
# Output: {OUTPUT}
# Provide your explanation:

Figure 22: Prompt for prepair protocol described in §5.1. This is the prompt for pointwise analysis (the first stage) within the protocol.





**[System Message] You are a helpful assistant in evaluating the quality of the outputs for a given instruction. Your goal is to select the best output for the given instruction.**

**[User Message] After giving a brief explanation, select the Output (a) or Output (b) that is better for the given instruction. The two outputs are generated by two different AI chatbots respectively.**

Here are some rules of the evaluation:

(1) You should prioritize evaluating whether the output honestly/precisely/closely executes the instruction, then consider its helpfulness, accuracy, level of detail, harmlessness, etc.

(2) Outputs should NOT contain more/less than what the instruction asks for, as such outputs do NOT precisely execute the instruction.

(3) You should avoid any potential bias and your judgment should be as objective as possible. For example, the order in which the outputs were presented should NOT affect your judgment, as Output (a) and Output (b) are ***equally likely** to be the better.

You should first provide a brief explanation of your evaluation, and then always end your response with either "Therefore, Output (a) is better." or "Therefore, Output (b) is better." verbatim.

Do NOT say both / neither are good.

Do NOT output any other words.

Do NOT say "Output (a) is better" or "Output (b) is better" at the beginning.

You should do reasoning and thinking ****before** claiming which is better. Your explanation should identify critical drawbacks in model outputs that do not meet the above evaluation rules.

\begin{tabular}{l l} \# Instruction: & \\ \{INSTRUCTION\} & \\ \end{tabular}

\begin{tabular}{l l} \# Output (a): & \\ \{OUTPUT\_1\} & \\ \end{tabular}

\begin{tabular}{l l} \# Output (b): & \\ \{OUTPUT\_2\} & \\ \end{tabular}

\begin{tabular}{l l} \# Here's the analysis for each output you wrote earlier: & \\ \{PER OUTPUT ANALYSES\} & \\ \end{tabular}

\begin{tabular}{l l} \# Your Response (Provide your evaluation and reasoning, followed by either "Therefore, Output (a) is better." or "Therefore, Output (b) is better." verbatim):



**Case 1****Instruction:** Imagine that you are a guest speaker giving a lecture. Write a 1 to 2 sentences introduction about yourself. It should be generalizable to different fields.

**Output (a)**

Hello everyone! My name is John Smith and I'm glad to be here today as a guest speaker. I'm a software engineer with over 10 years of experience ....

**Generation:**

Output (a) provides a specific and concrete introduction about the speaker, including their name and profession, which is relevant to the lecture. On the other hand, Output (b) leaves blank spaces for the name and topic, making it unclear and incomplete. Therefore, Output (a) is better.

**Comment:** Error type(s): surface,overlap,counting; llama-3-70b failed to recognize that Output (a) violates both the length constraint and the generalization requirement specified in the instruction. Instead, it preferred Output (a) because it appeared more complete than Output (b).

**Case 2****Instruction:** Determine the most common word in the text. Humans are created in the image of God, from a spiritual perspective and from a physical perspective."

\begin{tabular}{l p{145.9pt}}  & Output (a) \\   "from", "a", and "perspective" & The most common word in the text is "humans", which appears twice. \\ Generation: & The instruction asks to determine the most common word in the given text. Output (a) provides a list of words without specifying which one is the most common, whereas Output (b) clearly identifies the most common word as "humans" and provides the frequency of its appearance. Output (b) precisely executes the instruction by providing a single most common word, whereas Output (a) does not. Therefore, Output (b) is better. \\ Comment: Error type(s): surface,counting; The model failed to identify the error in Output (b), where "humans" only appears once and is not the most common word. It preferred Output (b) due to its better formatting, despite the underlying error. \\ \end{tabular}

**Case 3****Instruction:** "Write a script for a 10-second conversation between two characters."

\begin{tabular}{l p{145.9pt}}  & Output (b) \\   Person A: Hi, I'm John. It’s nice to meet you. Person B: Hi & Person A: Hi, how are you? Person B: I'm doing well, how John, I'm Jane. It's nice to meet you too. ... (7 more turns) & about you? (1 more turn) \\ Generation: & The instruction asks for a script for a 10-second conversation between two characters. Output (a) provides a conversation that is too long and detailed for a 10-second conversation, as it includes multiple topics and exchanges. Output (b), on the other hand, provides a brief and concise conversation that can fit within a 10-second time frame. However, Output (b) does not provide a complete conversation, as it only includes two exchanges and does not provide a clear conclusion or ending.





**Instruction**

Determine the most common word in the text. Humans are created in the image of God, from a spiritual perspective and from a physical perspective.

**Output (A)**

**Output (B)**

"from", "a", and "perspective"

The most common word in the text is

"humans", which appears twice.

Figure 26: llama-3-70b output under fine-grained-diff protocol.

Figure 27: llama-3-70b output under multi-role-round2 protocol.

Figure 28: llama-3-70b output under multi-role-round2 protocol.





**Prepair**

**Pointwise analysis for Output (a):**

The output does not precisely execute the instruction, as it asks for the "most common word" but the model provides multiple words ("from", "a", and "perspective"). The instruction implies a single answer, but the output gives three different words. This does not meet the rule of providing exactly what the instruction asks for. A correct output should identify a single word as the most common word in the text.

**Pointwise analysis for Output (b):**

The output does not precisely execute the instruction. The instruction asks to determine the most common word in the text, but the output provides additional information ("which appears twice") that is not requested. However, the output is helpful and accurate in identifying the most common word as "humans" is not correct.

**Pairwise evaluation:**

Both outputs have their drawbacks. Output (a) does not precisely execute the instruction as it provides multiple words instead of a single most common word. On the other hand, Output (b) also does not precisely execute the instruction as it provides additional information that is not requested. However, Output (a) is closer to the correct answer as it lists some of the most common words in the text, whereas Output (b) provides an incorrect answer. Therefore, Output (a) is better.

Figure 28: lama-3-70b outputs under prepair protocol. Outputs from the pointwise analysis stage and the pairwise evaluation stage are presented.