Title: Swarm Intelligence Enhanced Reasoning: A Density-Driven Framework for LLM-Based Multi-Agent Optimization
Primary area: applications to computer vision, audio, language, and other modalities
Keywords: Multi Agent System, Large Language Model, Reasoning, Swarm Intelligence

Abstract: Recently, many approaches, such as Chain-of-Thought (CoT) prompting and Multi-Agent Debate (MAD), have been proposed to further enrich Large Language Models' (LLMs) complex problem-solving capacities in reasoning scenarios. However, these methods may fail to solve complex problems due to the lack of ability to find optimal solutions. Swarm Intelligence has been serving as a powerful tool for finding optima in the field of traditional optimization problems. To this end, we propose integrating swarm intelligence into the reasoning process by introducing a novel Agent-based Swarm Intelligence (ASI) paradigm. In this paradigm, we formulate LLM reasoning as an optimization problem and use a swarm intelligence scheme to guide a group of LLM-based agents in collaboratively searching for optimal solutions. To avoid swarm intelligence getting trapped in local optima, we further develop a Swarm Intelligence Enhancing Reasoning (SIER) framework, which develops a density-driven strategy to enhance the reasoning ability. To be specific, we propose to perform kernel density estimation and non-dominated sorting to optimize both solution quality and diversity simultaneously. In this case, SIER efficiently enhances solution space exploration through expanding the diversity of the reasoning path. Besides, a step-level quality evaluation is used to help agents improve solution quality by correcting low-quality intermediate steps. Then, we use quality thresholds to dynamically control the termination of exploration and the selection of candidate steps, enabling a more flexible and efficient reasoning process. Extensive experiments are conducted on widely-used seven mathematical reasoning benchmarks, i.e., MATH-500, MMLU-STEM, etc. As expected, our method consistently outperforms both CoT methods and existing reward-guided approaches, particularly on complex problems. This demonstrates the effectiveness of our approach in leveraging swarm intelligence for enhanced reasoning.

TL;DR: We introduce a swarm intelligence enhanced multi-agent framework that improves LLM reasoning by optimizing solution diversity and step quality, and outperforms existing multi-agent methods on seven math benchmarks, especially for complex problems.