Improving Multi-Agent Debate with Sparse Communication Topology (2024)

Abstract: Multi-agent debate has proven effective in improving large language models quality for reasoning and factuality tasks. While various role-playing strategies in multi-agent debates have been explored, in terms of the communication among agents, existing approaches adopt a brute force algorithm – each agent can communicate with all other agents. In this paper, we systematically investigate the effect of communication connectivity in multi-agent systems. Our experiments on GPT and Mistral models reveal that multi-agent debates leveraging sparse communication topology can achieve comparable or superior performance while significantly reducing computational costs. Furthermore, we extend the multi-agent debate framework to multi-modal reasoning and alignment labeling tasks, showcasing its broad applicability and effectiveness. Our findings underscore the importance of communication connectivity on enhancing the efficiency and effectiveness of the “society of minds” approach.

## introduction large language models (llms) have demonstrated exceptional performance in natural language understanding and generation tasks. recently a paradigm shift towards prompting llms has emerged as a significant and influential research area. by leveraging the in-context learning (icl) capabilities of llms, these models can be adapted to various tasks such as reasoning, factuality, and ai feedback. several prompting methods have been developed to enhance llm performance by optimizing their icl capabilities. notable techniques include chain-of-thought (cot) @xcite , self-consistency (sc) @xcite , and selfcritique (madaan et al., 2024; welleck et al., 2022; † correspondence: yunxuanli@google.com @xcite . recently, the multi-agent debate (mad) framework is proven to be an innovative approach. similar to a human discussion process, mad employs multiple llm agents to engage in discussions with one another, combining their reasoning and critical thinking abilities to produce high-quality results. specifically, given a question, each agent first generates their own solutions to the question and then takes other agents' solutions as reference to update its own answer. this process can be repeated for several rounds. mad has demonstrated significant improvement on factuality and reasoning tasks. while the debate process is highly productive, it is also very costly: as the number of llm agents and debate rounds increase, the input context expands significantly. inspired by the intensive computational cost of mad, a natural question arises: what if we reduce the number of reference solutions visible to each agent? we conduct a systematic study on the sparsity of the multi-agent communication topology. surprisingly, we find that sparse communication connectivity can deliver comparable or superior performance while significantly reducing inference costs. figure 1 presents a comparison between fully-connected mad and neighborconnected mad. compared to fully-connected mad, neighbor-connected mad achieves an improvement of +2% on the math dataset and maintains the same accuracy on gsm8k. meanwhile, the average input token cost for reasoning tasks is reduced by over 40%. mad can also be a promising approach for reinforcement learning with ai feedback (rlaif) @xcite and weak-tostrong generalization @xcite . by delivering better reward signals, mad has the potential to significantly aid in aligning large language models. to assess this, we first extend the mad framework to alignment labeling tasks, demonstrating its effectiveness compared to single-agent setups. additionally, we verify that the advantages of sparsity observed in the reasoning tasks experiments also apply to alignment labeling tasks. our experiments on the anthropic-hh datasets show an improvement of +0.5% in helpfulness and +1.0% in harmlessness, while reducing costs by 50.0% and 53.3%, respectively. we find that when agents are instantiated by different llms within the mad framework, interactions among multiple llms allow weaker models to be progressively strengthened through engagement with stronger models. in non-regular graph settings, assigning stronger llms to agents with higher centrality consistently yields better performance. in summary, our contributions are listed as follows: (1) we demonstrate that sparse communication topology enhances both effectiveness and efficiency of the multi-agent debate framework; (2) we thoroughly evaluate sparse mad for text-only and multimodal reasoning tasks, showing its advantage over standard mad; (3) we extend the mad framework to alignment labeling tasks, showing the effectiveness of standard mad and further performance improvement introduced by sparse mad; (4) we provide insights that explain the effectiveness of sparsity in mad; (5) we find that assign-ing stronger llms to agents with higher centrality yields better overall performance in multiple llm debate setup. ## experiments: mad with multiple llms previous sections focus on the mad with agents instantiated by the same llm. in this section, we explore the scenario when multiple llms are available. with agents instantiated by different llms, the permutation invariance symmetry is broken, and the regular graph may not be optimal. a natural question is: how to design the communication topology given a mad framework of n agents, in which m instantiated by the stronger llm and n -m instantiated by the weaker llm? assigning stronger llms to agents with higher centrality yields better performance: we conducted experiments on harmlessness alignment labeling task, involving 6 agents, with 1 agent utilizing gpt-3.5 (the stronger llm) and the remaining 5 agents utilizing mistral 7b (the weaker llm). we tested two setups on the isotropic communication topology: one where the stronger llm had a degree of 1 (indicating low centrality) and another where it had a degree of 5 (indicating high centrality), as illustrated in figure 5 . the experimental results presented in the results above underscore the importance of information flow in the design of communication topology. figure 6 illustrates the average accuracy of weaker agents with respect to the number of debate rounds. when the stronger agent has a degree of 5, it can effectively disseminate its knowledge to weaker agents in just one debate round, resulting in a sharp increase in the average accuracy of weaker llms. in contrast, when the stronger agent has a degree of 1, the process requires two rounds: first, the information is transmitted to the central weaker agent in the first debate round (round 2), which then shares it with other weaker agents in the next round (round 3). this two-step process leads to greater information loss. ## conclusion in this paper, we show that sparse communication topologies can improve the multi-agent debate framework significantly. our results indicate that sparse mad configurations achieve comparable or superior performance to standard mads while greatly reducing computational costs. we also extend the mad framework to alignment labeling tasks, demonstrating the benefits of mads over single-agent setups and self-consistency and further highlighting the benefits of sparse mads over fully-connected configurations. we present case-study insights that explain the effectiveness of sparse mads. additionally, we investigate the impact of communication topology design with multiple large language models (llms), finding that assigning stronger llms to more connected agents enhances overall performance. in summary, our work paves the way for more efficient and effective multi-agent systems by leveraging sparse communication topologies. future studies could focus on deepening our understanding of the underlying mechanisms and developing strategies for optimal topology design in multi-agent frameworks.