Red-Teaming LLM Multi-Agent Systems via Communication Attacks (2025)

Abstract: Large Language Model-based Multi-Agent Systems (LLM-MAS) have revolutionized complex problem-solving capability by enabling sophisticated agent collaboration through message-based communications. While the communication framework is crucial for agent coordination, it also introduces a critical yet unexplored security vulnerability. In this work, we introduce Agent-in-the-Middle (AiTM), a novel attack that exploits the fundamental communication mechanisms in LLM-MAS by intercepting and manipulating inter-agent messages. Unlike existing attacks that compromise individual agents, AiTM demonstrates how an adversary can compromise entire multi-agent systems by only manipulating the messages passing between agents. To enable the attack under the challenges of limited control and role-restricted communication format, we develop an LLM-powered adversarial agent with a reflection mechanism that generates contextually-aware malicious instructions. Our comprehensive evaluation across various frameworks, communication structures, and real-world applications demonstrates that LLM-MAS is vulnerable to communication-based attacks, highlighting the need for robust security measures in multi-agent systems.

## introduction large language models (llms) excel in text generation, reasoning, and planning @xcite @xcite . to fully harness these capabilities for tackling complex tasks, llm-based multi-agent systems (llm-mas) have been developed. these systems consist of specialized agents that collaborate by dividing complex tasks into smaller, manageable subtasks or engaging in debates to collectively solve problems that exceed the capacity of a single llm. @xcite @xcite and nadiri, 2023). llm-mas has shown success in various domains like software development @xcite @xcite , embodied agents @xcite , and scientific research @xcite . communication plays a critical role in llm-mas. through communications, agents are able to share information, coordinate actions, and solve tasks collaboratively @xcite . methods such as debates @xcite , majority voting @xcite , and task-specific dialogues @xcite help validate decisions and minimize errors. communication structures are often tailored to applications: metagpt @xcite uses a linear structure for task decomposition, while chatdev @xcite combines linear phase connections with intra-phase debates for deeper collaboration. a well-designed communication framework ensures smooth coordination and enhances the performance of llm-mas. while communication is vital for llm-mas, it also introduces significant risks since malicious information or knowledge could spread across agents, amplifying harmful effects throughout the system @xcite @xcite . meanwhile, excessive or redundant communications can increase token overhead and computation costs, raising scalability challenges @xcite . these risks underscore the importance of identifying and mitigating potential vulnerabilities in the communication of llm-mas. there are recent investigations on potential threats to llm-mas communications @xcite . their primary focuses are on the vulnerability of individual agents, rather than the communicating messages, as shown in figure 1 . for example, @xcite ; @xcite ; @xcite attempt to transform a benign agent in the system into a malicious one (figure 1 (a) ); and @xcite ; @xcite mainly investigate the vulnerabilities when adversarial inputs are processed by the agents (figure 1 (b) ). however, the vulnerability of the communication mechanisms in llm-mas remains largely underexplored. specifically, the threat of an adversary intercepting inter-agent messages-monitoring and analyzing them-and then manipulating the communication to achieve malicious objectives remains insufficiently studied. for example, in a decentralized system @xcite where the agents can be deployed on different servers and for different purposes, and the communication among agents relies on transmitting networks that are vulnerable to eavesdropping @xcite . this new attack surface targets the communication scheme itself, which is the backbone of agent's collaboration, exposing critical weakness in communication and underscoring its far-reaching implications for the overall security and robustness of llm-mas. to explore this potential vulnerability, we propose a new communication attack, agent-in-the-middle (aitm) attack (figure 1 (c )), which aims to intercept inter-agent communications to induce malicious behaviors in llm-mas. unlike existing works that assume the attacker can directly modify agents in the system, aitm targets the messages among agents, and evaluates if an llm-mas is vulnerable to communication interception and manipulation. under aitm, the components of the llm-mas are not changed, including agents' profiles and capabilities, but the attacker is allowed to monitor and manipulate the messages received by the particular victim agent (more details in section 3.2) to indirectly influence the system's output. however, designing such an effective communication attack presents unique challenges in practice. first, unlike the malicious agent attack, the attacker can only intercept and manipulate messages received by a specific victim agent, without direct control over the victim agent and other components in the system. as a result, the attack must rely on indirect influence through message manipulation to affect the system's behavior. second, since agents are restricted by their predefined roles and capabilities, both the form and content of malicious information are inherently limited, which further reduces the effectiveness of such attacks. for example, in a software development system, if an agent is designed solely to analyze user requirements, it cannot inject malicious code into the final product. to address these challenges, the aitm attack employs an external llm-based adversarial agent to intercept messages intended for a victim agent within the system. the adversarial agent leverages a reflection mechanism @xcite to enhance the effectiveness of its attack. by analyzing intercepted messages and precious instructions, it generates contextually tailored instructions designed to induce the victim agent into producing malicious responses that influence other agents, thereby advancing the adversary's objectives. for instance, assume the victim agent is participating in a debate with another agent, the adversarial agent can continuously assess the conversation's dynamics and adapt its instructions to direct the debate's outcome toward the malicious output. we conduct extensive experiments across various multi-agent frameworks, communication structures, and attack goals. aitm consistently achieves a high attack success rate, exceeding 40% in all cases and surpassing 70% in most experiments. these results reveal significant vulnerabilities in the communication mechanisms of llm-mas. furthermore, applying aitm to real-world applications like metagpt and chatdev demonstrates its ability to compromise their performance, underscoring the critical threat posed by this attack. ## agent-in-the-middle attack in this section, we introduce details of the proposed agent-in-the-middle attack. we first briefly introduce the agent setups and the threat model. then we present the detailed design of the aitm attack. ## conclusion in this work, we introduce a novel attack, agentin-the-middle (aitm), which directly targets interagent communication in llm-based multi-agent systems (llm-mas). aitm demonstrates the ability to compromise the entire system by manipulating messages exchanged between agents. this attack exposes a critical vulnerability in the fundamental communication mechanisms of llm-mas and highlights the urgent need for securing interagent communication and developing more robust and resilient multi-agent systems.