Voting or Consensus? Decision-Making in Multi-Agent Debate (2025)

Abstract: Much of the success of multi-agent debates depends on carefully choosing the right parameters. The decision-making protocol stands out as it can highly impact final model answers, depending on how decisions are reached. Systematic comparison of decision protocols is difficult because many studies alter multiple discussion parameters beyond the protocol. So far, it has been largely unknown how decision-making influences different tasks. This work systematically evaluates the impact of seven decision protocols (e.g., majority voting, unanimity consensus). We change only one variable at a time - the decision protocol - to analyze how different methods affect the collaboration between agents and measure differences in knowledge and reasoning tasks. Our results show that voting protocols improve performance by 13.2% in reasoning tasks and consensus protocols by 2.8% in knowledge tasks compared to other decision protocols. Increasing the number of agents improves performance, while more discussion rounds before voting reduce it. To improve decision-making by increasing answer diversity, we propose two new methods, All-Agents Drafting (AAD) and Collective Improvement (CI). Our methods improve task performance by up to 3.3% with AAD and up to 7.4% with CI. This work demonstrates the importance of decision-making in multi-agent debates beyond scaling.

## introduction humans are inherently social, and collaboration has been key to innovation and progress. we know that generating solutions together is only beneficial if we can effectively select, agree, and commit to them. history, sociology, and psychology have long demonstrated how different decisionmaking processes influence collective outcomes @xcite . multi-agent systems form a parallel to human behavior by solving problems collectively via debate. however, so far, few ## methodology in the following, we explain the multi-agent environment, decision protocols, response generators, and datasets used in our experiments. ## experiments we use a set of diverse reasoning and knowledge tasks to explore the effectiveness and limits of decision protocols in multi-agent debate. 3 approval voting is left out as it consistently fails to reach a voting decision as described in section 4.1. ## conclusion we systematically evaluated the role of consensus and voting decision protocols across three knowledge and three reasoning tasks. our study assessed how the number of discussion rounds and agents influences task performance. we proposed two new methods to improve answer diversity during multiagent discussions and decisions, i.e., all-agents drafting (aad) and collective improvement (ci). aad requires each agent to contribute draft ideas at the beginning of the discussion, and ci encourages independent reasoning steps by limiting communication between agents and only allowing them to exchange possible solutions after each turn. our findings show that voting performs well on reasoning tasks, outperforming consensus by up to 13.2%, and outperforming a single cot baseline by 10.4%. this is likely because voting-based protocols allow agents to explore multiple reasoning paths instead of a single one, as in consensus. in comparison, consensus outperforms voting in knowledge tasks by up to 2.8%, because it improves fact-checking by requiring at least the agreement of the majority of agents. increasing the number of agents in the discussion improved task performance, while increasing the number of discussion rounds before voting decreased performance. one possible reason for that could be problem drift, a recent limitation found in long multi-agent debate @xcite . aad improved performance by up to 3.3%, and ci by up to 7.4% over multi-agent debate baselines, and 6.1% and 10.2% over single model cot baseline respectively. our methods enhance answer diversity and reveal a connection between answer diversity and task performance. future work could explore other characteristics influencing decisions, such as power relations between managers and employees. this could also involve examining personas within this hierarchi-cal structure to investigate whether dominant or affectionate leaders are more effective in leading discussions @xcite . we recommend using voting in reasoning tasks and consensus in knowledge tasks, scaling up the number of agents instead of the number of rounds, and increasing the diversity of answers between agents using aad and ci.