AgentsNet: Coordination and Collaborative Reasoning in Multi-Agent LLMs
Authors: name: Florian Grötschla, name: Luis Müller, name: Jan Tönshoff, name: Mikhail Galkin, name: Bryan Perozzi

Large-language models (LLMs) have demonstrated powerful problem-solving
capabilities, in particular when organized in multi-agent systems. However, the
advent of such systems also raises several questions on the ability of a
complex network of agents to effectively self-organize and collaborate. While
measuring performance on standard reasoning benchmarks indicates how well
multi-agent systems can solve reasoning tasks, it is unclear whether these
systems are able to leverage their topology effectively. Here, we propose
AgentsNet, a new benchmark for multi-agent reasoning. By drawing inspiration
from classical problems in distributed systems and graph theory, AgentsNet
measures the ability of multi-agent systems to collaboratively form strategies
for problem-solving, self-organization, and effective communication given a
network topology. We evaluate a variety of baseline methods on AgentsNet
including homogeneous networks of agents which first have to agree on basic
protocols for organization and communication. We find that some frontier LLMs
are already demonstrating strong performance for small networks but begin to
fall off once the size of the network scales. While existing multi-agent
benchmarks cover at most 2-5 agents, AgentsNet is practically unlimited in size
and can scale with new generations of LLMs. As such, we also probe frontier
models in a setup with up to 100 agents.