Revisiting Multi-Agent Debate as Test-Time Scaling: A Systematic Study
  of Conditional Effectiveness
Authors: name: Yongjin Yang, name: Euiin Yi, name: Jongwoo Ko, name: Kimin Lee, name: Zhijing Jin, name: Se-Young Yun

The remarkable growth in large language model (LLM) capabilities has spurred
exploration into multi-agent systems, with debate frameworks emerging as a
promising avenue for enhanced problem-solving. These multi-agent debate (MAD)
approaches, where agents collaboratively present, critique, and refine
arguments, potentially offer improved reasoning, robustness, and diverse
perspectives over monolithic models. Despite prior studies leveraging MAD, a
systematic understanding of its effectiveness compared to self-agent methods,
particularly under varying conditions, remains elusive. This paper seeks to
fill this gap by conceptualizing MAD as a test-time computational scaling
technique, distinguished by collaborative refinement and diverse exploration
capabilities. We conduct a comprehensive empirical investigation comparing MAD
with strong self-agent test-time scaling baselines on mathematical reasoning
and safety-related tasks. Our study systematically examines the influence of
task difficulty, model scale, and agent diversity on MAD's performance. Key
findings reveal that, for mathematical reasoning, MAD offers limited advantages
over self-agent scaling but becomes more effective with increased problem
difficulty and decreased model capability, while agent diversity shows little
benefit. Conversely, for safety tasks, MAD's collaborative refinement can
increase vulnerability, but incorporating diverse agent configurations
facilitates a gradual reduction in attack success through the collaborative
refinement process. We believe our findings provide critical guidance for the
future development of more effective and strategically deployed MAD systems.