Paper: DebateGPT: Fine-tuning Large Language Models with Multi-agent Debate Supervision (ICLR-2024)

Reviewer comment (Novelty):
If I'm understanding correctly, the Multi-Agent debate method used to generate data seems to follow the approach in [1]. However, it's important to note that when comparing it to the baseline models, we're only looking at GPT-3.5 and GPT-4, and there's no direct mention of [1].

Author response:
**Answer**: We thank the reviewer for the valuable comment. We have added the results of fine-tuning GPT3.5 using the data generated by the method proposed by [1] on three evaluation benchmarks. Our method consistently outperforms [1].
| Model | AlpacaEval | Arithmetic | Winogrande |
| -------- | --------------- | -------------- | ---------------- | 
| DebateGPT-3.5 Original Debate[1] | 85.64 | 50.30 | 84.19 |
| DebateGPT-3.5 (Ours) | 91.91 | 55.30 | 86.97 |

Reviewer comment (Evaluation):
In the main contributions of the paper, it claims to "offers a more economical alternative to leveraging models like GPT-4". However, when we look at Figure 6, we see that both options are similarly priced. Furthermore, taking into account their divergent performance makes it difficult for me to fully support this claim.

Author response:
**Answer**: Figure 6 shows that on 500 examples, generating responses to 500 examples with multi-agent debate costs 30 dollars less than GPT-4. As the number of examples increases, this cost will add up considerably. In our case with generating a dataset of 5K examples, we saved approximately 300 dollars using our approach with multi-agent debate. 

Furthermore, our performance is comparable with GPT-4. Table 1 shows that our method has a win rate accuracy of 88.6 and GPT-4 is 89.0. Table 6 in Appendix F (pasted below) shows that our method is consistently better than other methods at generating high quality data.

| Model                            	| Win Rate (higher is better) |
|--------------------------------------|----------|
| GPT-3.5                          	| 74.2 	|
| Zero-Shot Chain of Thought       	| 78.4 	|
| Multi-Agent Debate (Du et. al. 2023) | 77.2 	|
| Multi-Agent Debate (Ours)        	| 81.2 	|

Reviewer comment (Reproducibility):
Which LLMs are included in the Multi-Agent used for data generation?

Author response:
**Answer**: As stated in Section 3.2, we use GPT-3.5 in the multi-agent debate for data generation.