Paper: ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate (ICLR-2024)

Reviewer comment (Presentation):
The specific examples of three different communication strategies are lacking, which can be put into the appendix.

Author response:
Currently, the comprehensive algorithm is detailed in the appendix, and we will include illustrative examples in the appendix of our next version to facilitate a better understanding.

Reviewer comment (Evaluation):
The resource cost of ChatGPT/GPT-4 is a problem that needs to be considered, and the paper does not compare the resource consumption of previous methods with the proposed method.

Author response:
We have assessed the resource expenditure, and the following table shows our method can substantially lower both time and labor costs in comparison to the employment of human annotators.

|                | Cost    | Time   |
|-------------------|---------|--------|
|Human             | \$90    | 240min |
|gpt-4 (SA)    | \$2.90  | 10min  |
|gpt-4 (MA ensemble)  | \$8.70  | 10min  |
|gpt-4 (MA chateval)  | \$12.30 | 36min  |
|Chatgpt(SA)   | \$0.11  | 3min   |
|Chatgpt(MA ensemble) | \$0.33  | 3min   |
|Chatgpt(MA chateval) | \$0.41  | 11min  |

Reviewer comment (Experiments):
About fairness comparison. The setting of ChatEval is 2 agents with 2 discussion turns. It is not clear what the setting of Multi-Agent (Ensemble) is like.

Author response:
Dear reviewer:

We express our heartfelt thanks for your prompt response. Your diligent efforts in reviewing our work are highly respected and we sincerely appreciate the affirmative evaluation you've provided for our paper thus far. In accordance with the cost analysis of these baseline methods, we have included an additional table on the FairEval benchmark for your reference.

It is important to highlight that the mentioned methods require multiple inferences per instance, which we have assumed to be run in an asynchronous manner. Consequently, their time cost would be that of an SA COT method, consistent with our reporting for our ensemble methods previously discussed.

Furthermore, regarding the g-eval method, which necessitates sampling 20 times to estimate the token probabilities, the incurred cost is literally 20-fold that of an SA COT method.

We hope that these additional results address your concerns. If our recent revisions and clarifications meet your expectations, we would be grateful if you would consider supporting the paper by further increasing the score.


|                | Cost    | Time   |
|-------------------|---------|--------|
|G-eval-3.5 | \$2.2   | 3min |
|G-eval-4    | \$58  | 10min  |
|FairEval-chatgpt | \$0.34  | 3min  |
|FairEval-gpt4  | \$6.38 | 10min  |

Reviewer comment (Reproducibility):
More detailed about the Ensemble method.

Author response:
To clarify the use of the ensemble method in our paper, the term 'ensemble' as used in our context refers to the procedure where we apply the SA Chain of Thought (COT) method multiple times using different role prompts across the same instance and then aggregate their results through averaging to derive the final outcome. We will amend our paper to explicitly define this process and ensure that the methodology is understandable for all readers.

Reviewer comment (Experiments):
The experiments are conducted only on two datasets that are relatively small. On Topical-Chat, the win/loss of single-agent and multi-agent is mixed on different rubrics, and ChatGPT and GPT-4 seem to have different behaviors. Some more analysis on each rubric will be helpful rather than simply comparing the improvement on average.

Author response:
Our findings indicate that on average, our MA approach surpasses the SA method in performance when using both ChatGPT and GPT-4. However, there are subtle variations in performance across different sub-metrics between the two LLMs. Table 2 shows that for ChatGPT, MA slightly underperforms in Naturalness and Coherence but shows improvements in Engagingness and Groundedness. In contrast, MA of GPT-4 displays a more uniform improvement across three sub-metrics, with only a minor decline in Groundedness. It is also important to highlight that such variability is observed not only within our proposed metrics but also within traditional metrics and previous methods like g-eval. For instance, g-eval-4's performance dips in Engagingness and Groundedness compared to g-eval-3.5. It is notable that our GPT-4 (ChatEval) consistently excels over ChatGPT (ChatEval) in both SA and MA configurations, which is a trend we consider to be a reasonable and scalable behavior.

Reviewer comment (Experiments):
The experiments are conducted only with ChatGPT and GPT-4, thus it is unclear whether the proposed method could generalize to other LLMs, or is limited to GPT models. Evaluation on more models of different capacities could help understand when the multi-agent framework would work, and whether there is an emergent ability that requires a model with decent performance to have constructive discussion and evaluation.

Author response:
We have now included additional results for Llama2-chat-7b and Vicuna-7b-v1.5 and attached these new findings for your review. As indicated in the updated table, Llama2 achieves only marginal performance under single-agent CoT methods. Despite reaching an accuracy of 37.5%, the negative kappa coefficient suggests that the observed agreement is less than what would be expected by chance. However, it's noteworthy that through the application of a multi-agent debate, Llama2 demonstrates a modest improvement, albeit still marginally above chance levels.

In contrast, Vicuna exhibits more robust performance improvement compared to Llama2. ChatEval notably enhances its capabilities, achieving an accuracy of 52.3% and a kappa coefficient of 0.19, indicative of fair agreement beyond chance. While both models show limitations in their performance, **these results demonstrate the effectiveness of ChatEval and underscore its positive impact**.

We are grateful for the reviewers' insights regarding the application of ChatEval to less powerful models. Our findings demonstrate that ChatEval can still significantly enhance performance, a point we believe holds substantial value, as also echoed by the reviewers.

Furthermore, we hope that our revisions and comprehensive responses have satisfactorily addressed the initial concerns. If so, we would be deeply appreciative of your consideration in revising your evaluation of our paper upwards. 

Thank you once again for your invaluable feedback and guidance in refining our study.


|                             | Acc. | Kap.  |
|------------------------------|------|-------|
|Llama2-chat-7b (Single)   | $37.5_{\pm 1.3}$ | $-0.01_{\pm 0.01}$ |
|Llama2-chat-7b (ChatEval) | $40.0_{\pm 1.1}$   |  $0.05_{\pm 0.01}$  |
|Vicuna-7b-v1.5 (Single)       | $45.0_{\pm 1.5}$ |  $0.12_{\pm 0.01}$  |
|Vicuna-7b-v1.5 (ChatEval)     | $52.3_{\pm 1.3}$ | $0.19_{\pm 0.01}$  |

Reviewer comment (Evaluation):
There is no discussion on the limitation of the proposed method. For example, the inference speed/cost will be affected given that multi-agent requires more rounds of generation with the LLM.

Author response:
Here we include the cost analysis table. The results indicate that while the MA debate inherently incurs higher costs, it remains significantly more cost-effective than using human annotators, both in terms of time and expense. We will incorporate this cost analysis into the forthcoming version of our manuscript. 

|                 | Cost    | Time   |
|-------------------|---------|--------|
|Human             | \$90    | 240min |
|gpt-4 (SA)    | \$2.90  | 10min  |
|gpt-4 (MA ensemble)  | \$8.70  | 10min  |
|gpt-4 (MA chateval)  | \$12.30 | 36min  |
|Chatgpt(SA)   | \$0.11  | 3min   |
|Chatgpt(MA ensemble) | \$0.33  | 3min   |
|Chatgpt(MA chateval) | \$0.41  | 11min  |

Reviewer comment (Experiments):
In section 3.2, it is mentioned that the default results are obtained with 2 agents. What are these 2 agents? The appendix describes the implementation of 5 agents, while in section 4.1 it is mentioned that there are three different roles.

Author response:
In the experiments involving two agents, we utilize the roles of 'general public' and 'critic'.

Reviewer comment (Presentation):
In section 4.1, the authors mention comparison between specific roles and the default setting, but it seems Figure 3 only includes results of specific roles.

Author response:
In Figure 3, we have compared specific roles against the default setting. Here, 'general' on the x-axis refers to the default setting.

Reviewer comment (Reproducibility):
What is the specific version of ChatGPT and GPT-4 used in the paper, since that will affect model performance as well.

Author response:
Regarding the models referenced, "ChatGPT" corresponds to gpt-3.5turbo-0301 and "GPT4" to gpt-4-0314.

Reviewer comment (Evaluation):
In the abstract, the authors mention that one of the drawbacks of using humans in the pipeline is the time and labor cost. The analysis of the proposed framework would benefit significantly if there is any analysis in terms of time spent by humans for annotation vs LLM.

Author response:
We extend our gratitude to the reviewer for highlighting the importance of a time cost analysis in supporting our proposed framework. In the appended table, it is evident that the evaluator based on LLMs significantly reduces both time and financial expenditures. Although the chateval framework incurs higher costs than the SA  and MA ensemble methods due to multiple inference rounds necessary for final judgment derivation, it remains more cost-effective than employing human evaluators. We consider this an acceptable trade-off for the benefits provided.

 |                | Cost    | Time   |
|-------------------|---------|--------|
|Human             | \$90    | 240min |
|gpt-4 (SA)    | \$2.90  | 10min  |
|gpt-4 (MA ensemble)  | \$8.70  | 10min  |
|gpt-4 (MA chateval)  | \$12.30 | 36min  |
|Chatgpt(SA)   | \$0.11  | 3min   |
|Chatgpt(MA ensemble) | \$0.33  | 3min   |
|Chatgpt(MA chateval) | \$0.41  | 11min  |

Reviewer comment (Experiments):
The framework seems to be suited only for short conversations based on the Analysis in Section 4.3. This would be an issue when extending this framework for evaluating longer dialogue conversations.

Author response:
We acknowledge the reviewer's point regarding the challenges of evaluating prolonged conversations, a concern shared by both academic and industrial spheres.  Within our framework, we have incorporated a summary-style communication strategy specifically designed to mitigate the issues presented by lengthy historical context. Specifically, at the end of each iteration of the debate, we will prompt an extra LLM to summarize the messages conveyed so far and use this condensed information as its historical context. The results in section 4.3 show that summary-style has a better scalable behaviour. Additionally, we would like to emphasize that the principal aim of our work is to establish a more comprehensive mechanism for evaluating textual content. The experimental results indicate that the application of a debating format can indeed catalyze improved performance. As more capable LLMs emerge, particularly those adept at managing exceptionally long contexts, our framework is inherently equipped to generalize these advancements. The integration of such LLMs is anticipated to unlock the full potential of our framework further.

Reviewer comment (Experiments):
In Tables 1 and 2 why is only Multi Agent (ChatEval) using the strategy of One-by-one mentioned? Why are results from other strategies not provided in the table?

Author response:
Initially, we highlighted the 'one-by-one' strategy for ChatEval as it was our primary focus and the most direct method to benchmark against existing frameworks. We agree that it is important to show a comprehensive view of the performance across different strategies. To this end, we have now included results from additional strategies in the revised tables. The data presented in the table indicate that each of the strategies yields a statistically consistent enhancement over the SA method, affirming the efficacy of our approach. For an in-depth examination of communication strategies, we recommend that reviewers refer to Section 4.2.


|                             |                       | Acc.        | Kap          |
|------------------------------|------------------------|-------------|--------------|
|ChatGPT                      | SA         | $53.7_{\pm 1.4}$ | $0.27_{\pm 0.02}$ |
|ChatGPT                      | one-by-one           | $60.0_{\pm 0.9}$ | $0.30_{\pm 0.02}$ |
|ChatGPT                      | simultaneous-talk | $59.3_{\pm 1.0}$ | $0.29_{\pm 0.02}$ |
|ChatGPT                     | simultaneous-talk-with-summarizer | $58.8_{\pm 1.3}$ | $0.28_{\pm 0.03}$ |
|GPT-4                         | SA         | $60.8_{\pm 0.7}$ | $0.36_{\pm 0.01}$ |
|GPT-4                         | one-by-one           | $63.8_{\pm 0.9}$ | $0.40_{\pm 0.01}$ |
|GPT-4                         | simultaneous-talk | $63.2_{\pm 0.6}$ | $0.38_{\pm 0.02}$ |
|GPT-4                         | simultaneous-talk-with-summarizer | $63.4_{\pm 0.8}$ | $0.39_{\pm 0.02}$ |

Reviewer comment (Evaluation):
How effective is the framework at detecting issues on hallucination which is quite common along LLMs?

Author response:
The issue of hallucination in LLMs is indeed prevalent, and past research has suggested that multi-agent debates can mitigate this to a certain extent https://arxiv.org/abs/2305.14325. However, we consider a thorough investigation into this matter to be beyond the scope of the current paper. We believe that integrating external resources, such as search engines or databases, could be an effective strategy to curtail hallucinations. We plan to explore these possibilities and assess their compatibility with our debate framework in subsequent research endeavors.

Reviewer comment (Reproducibility):
The paper may benefit from providing more in-depth technical details about the ChatEval framework. While a prompt template is present in the appendix, the determination of LLM roles and the potential impact of varying roles and orders remain unclear.

Author response:
**Q1: The origin and the impact of varying role design as well as order issue**

We choose the role design introduced in https://arxiv.org/abs/2303.15078 as a starting point to specify role prompts and facilitate a fair comparison with prior literature. As for roles design alternatives, we believe that the experiments presented in Section 4.1 probe the impact of different roles design. We design roles for different groups, and these roles perform better in their respective categories. As evident in the results, we believe the impact of role design is significant. In terms of the order issue, we have proposed "simultaneous-talk" communication strategy which represents a pragmatic solution to alleviate strict order constraints in contrast to one-by-one communication. It employs asynchronous functions to merge historical messages, shuffling the order of messages within the history at each round. This approach enables a flexible and dynamic interactions among agents. We will make it clearer in our next version of the paper.

Reviewer comment (Theory):
As mentioned in section 3.2, two agents are employed in the implementation of ChatEval. Can they always reach an agreement at the end of debates?

Author response:
**Q2: Inter-annotator agreement among agents**

In response to your question about agreement, we provide an analysis of the inter-annotator agreement experiment results in the following table. We employed ICC (Intraclass Correlation Coefficient) to measure the agreement among annotators. From the table, it is evident that the annotators are generally able to reach a consensus by the end of their debate. We also observe that as we increase the number of roles, there is a decrease in agreement. It is reasonable considering that it's not also easy for a team of people to reach a consensus. Still, the overall ICC values remain at a high level, indicating strong agreement among annotators.


  |     | 2 roles | 3 roles | 4 roles | 5 roles |
|-----|---------|---------|---------|---------|
| ICC | 0.981   | 0.955   | 0.890   | 0.881   |

Reviewer comment (Experiments):
Considering the performance gap between G-EVAL-3.5 and ChatGPT(MA) in Table 2 for dialogue response generation, it appears that the effectiveness of the proposed framework is influenced by the chosen LLM. Has there been an evaluation of ChatEval's generalization ability?

Author response:
**Q4: Generalization of ChatEval and its performance compared to G-eval**

We would like to clarify that our method is also applicable to a range of LLMs and it operates orthogonally to SA prompting techniques. It complements and extends these techniques by integrating them within a MA debate context, which can leverage the unique strengths of different LLMs in a collaborative setting. Looking ahead, we are excited about the potential of more specialized and expert-driven LLMs in different domains, which promise to further generalize our framework.

 It is important to note that while we benchmarked our approach against the g-eval method, the comparison may not be entirely straightforward. The g-eval method, which achieved a better 3.5 performance score, does not solely rely on COT prompting; instead, it requires access to the language model's logits. These logits are then used to perform a weighted calculation, which differs from our methodology. However, our proposed MA approach does not necessitate access to the model's logits—a requirement that is often impractical in real-world applications. Consequently, it is expected that there would be performance discrepancies between our method and g-eval.

Additionally, our experimental results demonstrate that the MA approach (MA ChatEval), surpasses the performance of the SA COT method referenced in our main text. This responds to the reviewers' queries by confirming that our MA approach surpasses the SA CoT method, as clearly demonstrated by our experimental results.

Reviewer comment (Evaluation):
It seems that the G-Eval framework can be easily adapted to open-ended QA evaluation. From the performance differences in Table 2 between G-Eval and ChatEval, I wonder whether the multiple-agent debating framework really works better than the single-agent COT framework?

Author response:
**Q4: Generalization of ChatEval and its performance compared to G-eval**

We would like to clarify that our method is also applicable to a range of LLMs and it operates orthogonally to SA prompting techniques. It complements and extends these techniques by integrating them within a MA debate context, which can leverage the unique strengths of different LLMs in a collaborative setting. Looking ahead, we are excited about the potential of more specialized and expert-driven LLMs in different domains, which promise to further generalize our framework.

 It is important to note that while we benchmarked our approach against the g-eval method, the comparison may not be entirely straightforward. The g-eval method, which achieved a better 3.5 performance score, does not solely rely on COT prompting; instead, it requires access to the language model's logits. These logits are then used to perform a weighted calculation, which differs from our methodology. However, our proposed MA approach does not necessitate access to the model's logits—a requirement that is often impractical in real-world applications. Consequently, it is expected that there would be performance discrepancies between our method and g-eval.

Additionally, our experimental results demonstrate that the MA approach (MA ChatEval), surpasses the performance of the SA COT method referenced in our main text. This responds to the reviewers' queries by confirming that our MA approach surpasses the SA CoT method, as clearly demonstrated by our experimental results.