The Five Ws of Multi-Agent Communication: Who Talks to Whom, When, What, and Why - A Survey from MARL to Emergent Language and LLMs
Authors: Jingdi Chen
Hanqing Yang
Zongjun Liu
Carlee Joe-Wong
Venue: Accepted by TMLR ()

Abstract: Multi-agent sequential decision-making underpins many real-world systems, from autonomous vehicles and robotics to collaborative AI assistants. In dynamic and partially observable environments, effective communication is essential for reducing uncertainty and enabling coordination. Although research on multi-agent communication (MA-Comm) spans diverse paradigms, we organize this survey explicitly around the Five Ws of communication: who communicates with whom, what is communicated, when communication occurs, and why communication is beneficial. This lens provides a coherent structure for synthesizing diverse approaches and exposing shared design principles across paradigms. Within Multi-Agent Reinforcement Learning (MARL), early work relied on hand-designed or implicit communication protocols, followed by trainable, end-to-end mechanisms optimized for reward and control. While effective, these approaches often yield task-specific and weakly interpretable communication, motivating research on Emergent Language (EL), where agents develop more structured or symbolic protocols through interaction. EL methods, however, still face challenges in grounding, generalization, and scalability, which have driven recent interest in large language models (LLMs) as a means to leverage natural language priors for reasoning, planning, and coordination in open-ended multi-agent settings. This progression motivates our survey: we analyze how communication paradigms evolve in response to the limitations of earlier approaches and how MARL, EL, and LLM-based systems address complementary aspects of multi-agent communication. This paper provides a unified survey of MA-Comm across MARL, EL, and LLM-based multi-agent systems. Organized around the Five Ws, we examine how different paradigms motivate, structure, and operationalize communication, reveal cross-paradigm trade-offs, and identify open challenges in communication, coordination, and learning. By offering systematic comparisons and design-oriented insights, this survey helps the community extract effective communication design patterns and supports the development of hybrid systems that combine learning, language, and control to meet diverse task, scalability, and interpretability requirements.

Published in Transactions on Machine Learning Research (02/2026)
The Five Ws of Multi-Agent Communication: Who Talks to Whom, When, What, and Why A Survey from MARL to Emergent Language and LLMs
Jingdi Chen'
jingdicBuuizonaaco edu
University of Arizona Tucson, AZ 85719
Hanqing Yang
hangingg3eendrev.coom cmaa, eduu
Carnegie Mellon Universily Pillsburgh, PA 15213
Zongjun Liu
manlin@@arizona.ci
University of Arizona Tucson, AZ 85719
Carlee Joe- Wong
cjoewongSendreu.cmon
Carnegie Mellon University Pittsburgh, PA 15213
Reviewed on OpenReview:
https: openreview. net/ permno id= LGse@000gg
Abstract
Multi-agent sequential decision-making underpins many real-world systems, from au- tonomous vehicles and robotics to collaborative AT assistants. In dynamic and partially observable environments, effective communication is essential for reducing uncertainty and enabling coordination. Although research on multi-agent communication (MA-Comm) spans diverse paradigms, we organize this survey explicitly around the Five Ws of communica- tion: who communicates with whom, what is communicated, when communication occurs, and why communication is beneficial. This lens provides a coherent structure for synthe- sizing diverse approaches and exposing shared design principles across paradigms. Within Mulli-Agen Reinforcement Learning (M),L), early work relied on hand-designed or im- plicit communication protocols, followed by trainable, end-to-end mechanisms optimized for reward and control. While effective, these approaches often yield task-specific and weakly in- terpretable communication, motivating research Emergent Language (EL), where agents develop more structured or symbolic protocols through interaction. F.. methods, however, still face challenges in grounding, generalization, and scalability, which have driven recent interest in large language models (LLMs) as a means to leverage natural language priors for reasoning, planning, and coordination open-ended multi-agent settings. This progres- sion motivates our survey: we analyze how communication paradigms evolve in response to the limitations of carlier approaches and how MARL, EL, and LLM-based systems ad- dress complementary aspects of multi-agent communication. This paper provides a unified survey of MA-Comm across MARI, EL,, and LLLM-based multi-agent. systems. Organized around the Five Ws, we examine how different paradigms motivate, structure, and opera- tionalize communication, reveal cross- paradiim trade-offs, and identify open challenges in communication, coordination, and learning. By offering systematic comparisons and design- oriented insights, this survey helps the community extract effective communication design
*This work was done during Jingdi Chen's postdoctoral research at Carnegie Mellon University.

---

Published in Transactions on Machine Learning Research (02/2026)
patterns and supports the development of hybrid systems that combine learning, language, and control to meet diverse task, scalability, and interpretability requirements.
Contents
Introduction
4
Evolution of Mueti-Agent Communication Learning
5
1.2
The Need for Dedicated Survey
7
2
Foundations of Agent Communication
8
2.1
Foundations of Commuication as Action
8
2.2
From Speech Acts to Learned Message Actions
9
3
Related Work: Surveying the Three Dimensions of Mlulti-Aent Communication
10
3.1
Surveys on Communication in MARL
3.2
Surveys on Emergent Communication
3.3
Surveys LLM- Based Communication
11
Methodology and Scope of the Survey
13
Search Strategy and Temporal Scope
13
Inclusion and Exclusion Criteria
13
Coverage Across Domains and Select ion for Detailed Discussion
Limitations of Scope
14
Communication in Multi-Agon Reinforcement Learning
14
5.1
Background of MARL-Comnl
15
5.2
Overview and Taxonomy of MARL-Comm
17
5.3
Who, Whom and What to Communicate: Information and Recipient Selection Relevance Filtering of Messages and Recipients
19
5.3.1
5.3.2
Redundancy Reduction via Targeted Communication
21
5.3.3
Compression and Encoding via Message Weighting
23
5.4
When lo Communicate: Timing and Adaptation under Constraints
26
5.4.1
Learning When to Communicate under Constraints
26
5.4.2
Timing of Message Integration
................................
28
5.5
Why and How to Communicate: Purpose Driven Protocol Design
................................
30
5.5.1
Training Frameworks for Communication Learning
31
5.5.2
Communication Objectives and Control Shaping
36
5.5.3
Why and How Communcattion Formats Matter
40
5.6
Discussions
44
2

---

Published in Transactions on Machine Learning Research (02/2026)
5,6.1
Who Communicates with Whom and What Is Shared
46
5.6.2
When Communication Is Necessary
46
5.6.3
Why Communication Helps and How It Is Used
46
5.7
7
Bridge: From MARL-Comm to Emergent Language
46
6
Emergent Language in Multi-agent Systems
47
Background and Foundations of Emergent Language
50
6.2
Overview and Taxonomy of Emergent Language
51
6.3
Who, Whom and What to Communicate: Preliminary Considcrations
................................
53
6.3.1
Population Size and Communicative Roles
53
6.3.2
Cooperation Level and Recipient Selection
54
6.3.3
Architectural Mechanisms for Encoding What Communicated
................................
55
6.3.4
Multi-Step Interaction and Dialog Structure
56
6.3.5
Vocabulary Size and Message Structure
56
6.3.6
Message Structure and Compositionality Toward Like Language
57
6.3.7
Metrics for Evaluating What Is Communicated
58
6.3.8
Discussion: Who-Whom: What in Emergent Language
6.4
When and Where Communicate: Experimental Environments
62
6.4.1
Standardized Toolkits for Controlling Interaction Timing and Topology
63
6.4.2
One-Shot and Heraliie Communication
63
6 6.4.3
Grounded and Embodied Environments
64
6.4.4
Discussion: When and Where Communication Occurs
65
6.5
Why and How to Communicate: Motivations and Mechanisms
66
6.5.1
Motivations for Emergent Communication
66
6.5.2
Interpretability and Structured Protocols
67
6.5.3
Robustness and Generalization
67
6.5.4
Discussion: Why and How Commmication Emerges
68
6.6
Discussions
69
6.6.1
Who Communicatos with Whom and What Is Shared
69
6.6.2
When and Where Agents Communicate
6.6.3
Why Communication Emerges and How It is Learned
70
6.7
7
Bridge: From MARI-Comm and Emergent Language to LLM-B Based Communication
70
LLM-Powered Multi-Agent Communication
71
Background and Foundations of LLMss and L.M.M Agents
7.1.1
Large Language Models (LLMs)
74
7.1.2
LLM Agent
74
3

---

Published in Transactions on Machine Learning Research (02/2026)
7.2
Who, Whom, and What to Communicate
74
7.2.1
Communication Topology and Interaction Structure
75
7.2.2
Agent Configurations and Communication Capabilitics
80
7.2.3
Message Design and Communicative Content
................................
82
7.2.4
Lramework-Lovel Design of Communication Structure
................................
82
When and Where LLM Agents Communccate
85
7.3.1
Motivations for Communccation Timing
................................
85
73..2
Environmental Triggers and Context
................................
88
7.3.3
Evaluation Settings for Communication
7.4
Why and How LLM Agents Communicale
94
7.4.1
Structure of Communication
94
7.4.2
Why Communicate: Static and Dynamic Communication Protocols
97
7.5
Discussions
100
7.5.1
Who Communicates with Whom and What Is Shared
100
7.5.2
When and Where Communication Is Necessary
100
7.5.3
Why Communication Used and How Implemented
100
7.6
Bridge:
MARL and Its Limitations
101
7.6.1
Why and How Communication Emerges in LLM MARL Hybrid Systems
102
7.0
7.6.2
Comparative Pattern Across Hybrid Systems
104
7.6.3
LLM Limitations in MARL Settings
106
7.6.4
LLM Limitations from a Decentralized Control Perspective
108
Discussion, Open Problems, and Future Directions
109
8.1
Discussions Multi-agent Communications
109
8.2
Beyond ML: Epistemic Perspectives on Agent Commmication
113
8.3
Perspectives on Future Directions/
114
8.3.1
Future Research on T Theoretical Guarantees
114
8.3.2
Future Research on Algorithm Developments
115
8.3.3
Future Works on Benchmarking.
117
8.3.4
Future Research on Human-Centric Multi-Agent Communication
118
Conclusion
120
1 Introduction
Multi-agent decision-making plays a critical role in a wide range of real-world applications, including robotics Gu et al. (2016): Zhang et al. (2055). such as navigation Candido & Hutchinson (2011) and ma- uipulation Pajarinen & Kyrki (2017); autonomous systems Talpaert et al. (2019), including autonomous driving Wang al. (2019a); and planning under uncertainty Wang et al. (2019b); Cheng al.) (2018). In these settings, multiple agents must act independently while interacting with both the environment and

---

Published in Transactions on Machine Learning Research (02/2026)
another. Depending on the task, agents may cooperate to achieve shared objectives, compete over limited resources, or engage in mixed-motive interactions that involve both collaboration and competition. A central challenge in these systems is uncertainty, which can arise from multiple sources: limited or partial observ- ability of the global state or of other agents' intentions, as well as inherent stochasticity in euvironment dynamics. Effective multi-agent systems must therefore address problems of coordination, decentralized control, and decision-making under uncertainty, often in real time and with limited communication.
To address these challenges, reinforcement learning (RL) has been widely explored as framework for training agents to make sequential decisions through trial and error. Deep Reinforcement Learning (DRL).
in particular, has been extensively studied in both simulated MMih et al. (2013); Lillicrap et al. (2015)
and real-world Gu al. (2016); Zhang ct al. (2015); Qureshi ct al. (2017): Meng et al. (2019) sccnarios.
While many DRL algorithms assumne fully observable environments modeled Markov Decision Processes
(MDPs) Lample & Chaplot (2016); |Schulman et al. (2015): Pipattanasompor et al.
2009 Li et al.
(002)
real-world multi-agent systems often involve Partially Observable Markov Decision Processes (POMDPs),
where each agent has only a limited view of the system state. In such cases, inter-agent communication can help mitigale partial observability by allowing agents to exchange information and build a more complete representation of the environment Chen et al. (2024b). As a result, an important research direction in MARL focuses on designing effective communication strategics to improve dccision-making in complex environments.
Effective communication is a core capability in multi-agent systems, particularly in scenarios requiring co- ordination, negotiation, or competition among autonomous agents. In MARL, agents must make decisions based on local observations while interacting with others in partially observable and often stochastic environ- ments. Communication enables agents to exchange task-relevant information, such as observations, goals, intentions, or strategies, which helps them align actions, resolve uncertainty, and coordinate more effectively. While this is critical in fully cooperative settings like multi-robot collaboration Li et al. (2002) or smart grid control Pipattanasomporn et al. (2009), communication also plays a nuanced but important role in competitive and mixed-molive scenarios. For example, in adversarial games like StarCraft [Sanvelyan et al. (2019) or Dota, Berner et al. (211)), agents may engage in strategic signaling, misinformation, or negotiation to manipulate opponents or form temporary alliances. In such cases, communication becomes not just a tool for cooperation, but also mechanism for influencing, deceiving, or adapting to others' behaviors in strategic interactions.
To move beyond manually designed protocols, recent research has focused on learned communication, where agents develop communication strategies through interaction. A widely used approach involves en- abling gradient. flow between agents during training, allowing them to optimize both message content and timing based task performance. This line of work offers scalable and flexible alternative to hand- crafted rules, enabling agents to discover efficient and adaptive messaging schemes tailored to their goals and environmental dynamics, whether cooperative, competitive, or somewhere in between.
1.1 Evolution of Multi-Agent Communication Learning
Communication in MARL Research on learning multi-agent communcation has evolved significantly, mirroring broader advances artificial intelligence and multi-agent systems. Initial efforts in this area focused on enabling communication in MARL enhance coordination under partial observability. Early studies addressed foundational questions such as who should communicate and what information should be
shared Paulos et al. (2119) Lowe et al. (2017): Foerster et al. (201 6ab; Sukhbaatar et al. (2016) Jiang & Tu
(2018) b Das et al. (2018); Rangwala & Williams (2020). Many of these early approaches employed Centralized Training with Decentralized Execution, where a centralized coordinator facilitates communication during
training while agents act independently during execution. For example, Comm.Ne SSkhbaatar et al. (2016)
and /C3Net Singh et al. (2018) aggregate hidden states across agents to produce shared representations. Later methods introduced more flexible mochanisms, such attention- based message routing in TarMA CMMMMADaaa Das et al.
(2018) and graph-structured message propagation in DICG Li et al.] (2020), allowing agents to dynamically select communication partners. While these models often rely on centralized components during training, they still support decentralized execution, preserving scalability and autonomy in deployment. While these neural architectures enabled learned communcaation, they often lacked interpretability, as deep neural networks generate complex messages that are difficult to analyze Brown et al. (2020); LeCun et al. (2055)
5

---

Published in Transactions on Machine Learning Research (02/2026)
Moreover, most approaches assumed agents could exchange continuous, real-valued messages, ignoring the practical constraints of real-world communication networks, which typically rely on discrete and
bandwidth-limited transmissions Foerster et al. (2016a); Lowe et al. (2017): Mordatch & Abbeel (2018):
Freed et al. (2020b a). These limitations motivated research into Emergent Discrete Communication,
where agents develop structured, interpretable protocols for exchanging information.
While MARL has made substantial strides in learning communication protocols within RL frameworks, alternative approaches for enabling agent cooperation have increasingly drawn attention. One such direction is Emergent Language (EL), which studies how agents can develop structured communication prolocols through repeated interaction. This line of work provides insights into how meaningful communication can emerge organically in coopcrative settings, without requiring pre-defined languages or protocols Lazaridou & Baroni (2020); Li et al. (2022). This shift was molivated by several limitations of traditional MARL- based commumication: (I) Although end-to-end learned communication can be highly effective for task optimization, it often produces contimuous, opaque message representations that are difficult to interpret, verify, or reuse across lasks and agent populations. (2) Moreover, MARL communication protocols are typically tightly coupled to specific environments and reward structures, making them brittle under changes in agent composition, task semantics, or deployment conditions. These challenges motivated researchers study communication itself a learning outcome in EL rescarch, rather than mercly internal signal optimized for reward in MARL settings.
Communication in EL Learned communication in EL has been a promising avenue for improving inter- pretability and structure in multi-agent systems. To move beyond opaque continuous message spaces, carly research explored how agents could develop discrete communication protocols through interaction- using one-hot messages Chaabouni et al. (2019); Kottur et al. (2017;; Havrylov & Titov [001]]; Lazaridou et al.
(2016) Lee et al. (2017), binary signals Focrster et al. (2016a), or other constrained message formats EEcles et al. (209)). These eflorts made agent communication more interpretable to humans, but often revealed coor- dination challenges, such as zero-shot failures where independently trained agents could not understand each other's learned protocols Hu et al. (020b). To address this, rescarchers have begun to design environments and training methods that encourage more robust, generalizable, and human-aligned communication Bullard let al. [2021 2020). While many of these efforts originate in the context of MARL, their implications extend to multi-agent decision-makinng more broadly, including settings beyond traditional reinforcement learning, For example, some works have aimod align emergent communication with natural language Lee et al.. (2019) Lowe et al. (2020), while others integrate pre-trained language models to introduce linguistic priors and promote structured, interpretable coordination Lazaridou et al. (2020); Tucker et al. (2021). These developments reflect growing trend toward bridging learned communication protocols with human lan- guage to support not only efficient coordination in MARL, but also broader agent collaboration and human-A AI interaction.
More recently, Large Language Models (LLMs) have emerged as a natural next step in this trajectory, By leveraging strong language understanding, reasoning capabilities, and extensive world knowledge acquired from large-scale pretraining, LLMs provide new foundation for multi-agent communication. Unlike tra- ditional MARL or EL agents, LLM-based systems can communicate directly in natural language, infer shared goals, and adapt to novel tasks and teammates with minimal additional training Yang et al. (2025). This capability makes LLMs particularly attractive for settings that require flexible coordination, zero-shot generalization, and interaction with humans.
Communication in LLMs The rise of LLMs has introduced a new paradigm for multi-agent communi- cation, expanding beyond traditional MART, frameworks. Unlike communication in MARL, where agents develop protocols from scratch through reinforcement learning, based agents leverage pre-trained lin- guistic knowledge to engage in structured, natural language-based exchanges and decision-making Du et al.) h2023); Liang et al. (2023); Wang et al. (2023c). This is distinct from EL research, where the focus is on how agents develop communication protocols in task-driven settings without any pre-existing linguistic knowledge. Recent work has proposed a range of communication architectures for LLM-based multi-agent systems, including direct messaging, chain-of-thought interactions, hierarchical structures, and graph-based
exchanges Zhang et al. (2023c) Du et. al. (2023) Qian al. (0023): Hong al. 2023): Holt et. al. Wu

---

Published in Transactions on Machine Learning Research (02/2026)
et al. 2003bb Jiang et al. 2023 Chan et al. (2023): Qian et al. (2024b) Zhuge et al. (0024b) These frameworks enable agents to collaborate, debate, plan, and reason across diverse environments while offering scalability and adaptability across tasks. A key distinction is that EL research investigates how communication emerges from scratch through interaction and learning, whereas LLM-based communication builds on pre-existing natural language capabilities. Despite this difference, EL offers valuable insights for LLM-based systems, particularly in understanding how structured, compositional, and task-relevant com- munication can arise from coordination pressures. Bridging these two lines of work suggests a promising path toward hybrid communication frameworks that combine learned behaviors, emergent structure, and the expressiveness of pre-trained language models. LLMs have become attractive for multi-agent systems in part because they address several limitations that persist in EL approaches. Although EL improves inter- pretability relative to purely latent MARL communication, it typically requires training from scratch, careful environment design, and extensive interaction to stabilize meaningful protocols. In contrast, LLMs provide agents with immediate access to rich linguistic structure, commonsense reasoning, and broad world knowledge, enabling zero-shot or few-shot coordination without explicit protocol learning. As a result, the role of communication shifts from a task-specific signaling mechanism to a reusable, semantically grounded interface that can support flexible coordination across tasks and agents.
EL and LLM-based agents highlight broader shift in how multi-agent communication is conceptualized, exlending beyond the assumptions of conventional MARL frameworks. Rather than supplanting earlier paradigms, each line of work emerged in response to concrete limitations in expressiveness, interpretability, scalability, or adaptability encountered at increasing levels of task and interaction complexity. Viewed to- gether, these developments reveal communication not as a fixed design choice, but as an evolving mechanism that co-adapts with agent capabilities, environments, and coordination demands. This evolution motivates unified treatment of multi-agent communication that explicitly traces how communcattion mecha- nisms transform as systems move from closod, task-specific settings toward open-ended, human-facing, and deployment-orientee scenarios.
1.2 The Need for a Dedicated Survey
Despite substantial progress, most cxisting work on multi-agent communication is discussed as a subsection within broader MARL surveys WWong et al. (2023); ; Gronauer & Diepold (2022); Oroojlooy Hajinezhadl 2023 15 Nguyen al. (2020); Hernandez-Leal et al. (2019); Zaiem & Bennequin (2019). Given the increasing complexity and diversity of approaches, from MARL-based communication and EL to LLM-drrven coor- dination, there is growing need for a dedicaled and structured review of MA-Comm. This survey aims to formalize the field, highlight key research challenges, and provide a roadmap for future research at the intersection reinforcement learning, natural language processing, and multi-agent cooperation. The main contributions of this paper are as follows:
A Unified Survey of Multi-Agent Communication Across Paradigms: We present the first survey that systematically unifics MARL-based communication, EL, and LLM-powered multi-agent systems, moving beyond traditional MARL-centric views to capture the full evolution of communication in multi-agent decision-making.
A Five Ws-Driven Analytical Framework: We organize the literature using the Five Ws of communication, who communicates with whom, what is communicated, when communication occurs, why communication is needed, and how it is motivated and operationalized, providing a consistent lens for comparison across paradigms.
Cross-Paradigm Synthesis and Bridging Analysis: Beyond cataloging methods, we introduce dedicated bridging sections that explain how limitations in MARL-CCmm motivated EL, and how gaps in MARL and EL led to LLM-based and hybrid LLM MARL systems, clarifying complementary roles rather than treating paradigms in isolation.
Foundational, Formal, and Game-Theuee Theoretic Perspectives: We ground modern communica- tion methods in classical views of communication as action, add concise mathematical formaliza- tions of communication structures, and connect mixed-motive and competitive settings to Nash and Bayesian Nash equilibrium concepts, strengthening theoretical clarity without excessive formalism.
7

---

Published in Transactions on Machine Learning Research (02/2026)
Open Challenges and Future Directions: We identify key open problems spanning grounding, interpretability, generalization, efficiency, and theoretical guarantees, and outline future research directions for algorithm design, benchmarking, and human-centric multi-agent communica- tion in increasingly open-ended and safety-critical settings.
Survey Structure and Organization In the order of MARL with Communication, EL, and LLM- based Multi-Agent Systems, we structure the paper as follows. Section 2 reviews the foundations of agent communication, showing how modern MARL, EL, and LLM-based approaches extend classical theories of communication as goal-directed, belief-shaping action. Section 3 reviews existing surveys on MARL, emer- gent communication, and LLM-based agents, identifying the need for unified perspective on multi-agent communication (MA-Comm). Section f outlines the methodological framework and
criteria guiding the selection of papers in this survey. Sections 5 6 present the three core paradigms of multi-agent commmicationn, MARL with learned communication protocols, EL, LLM-powered multi-agent communication. Fach section organized explicitly around the Five Ws of communi- cation, who communicates with whom, what communicated, when communication occurs, why communication is needed, how communication is motivated and operationalized, providing unified analytical lens across paradigms. We first introduce essential background, then categorize represen- tative methods according to these dimensions, highlighting how different communication motivations (why) shape concrete design choices (how). To further strengthen cross porss-aar coherence, we add dedicated bridging subsections the end of each major section, together with a comprehensive Bridge section that integrates MART-, FI-; L.MM-based communication, clarifying their relationships, limitations, and complementary roles. Finally, Section S synthesizes insights across the survey, discussing key challenges, limitations, and open problems in multi-agent communication, extending beyond implementation-leww issues to epistemic perspectives and outlining future research directions on theory, algorithms, benchmarking, and human-centric communication across MARL, EL, and LLM--baodd systems.
2
Foundations of Agent Communication
2.1 Foundations of Communication as Action
The study of agent communication builds on long-standing intellectual tradition that predates modern multi-agent systems, reinforcement learning, and large language models Russell et al. (1995). Early phi- losophy of language framed communication not as passive information transfer, but as a form of action. Speech act theory formalized this view by arguing that utterances perform goal-directed acts such as re- questing, committing, or commanding Searle (1969); Austin (1775). Related work by Wittgenstein and Grice further emphasized that meaning depends on context, intention, and shared conventions rather than syntax alone Grice (1957 Arrington (2016). Together, these ideas established foundation for viewing communication as a deeision-deependent behavior chosen to influence others' beliefs, intentions, and actions.
This perspective strongly influenced early artificial intelligence research on situated language use, where communication was embedded within perception, reasoning, and action rather than treated as an isolated symbolie process Hobbs al. (193).. In parallel, formal models of language structure such
context-free grammars and their computational extensions provided tools for representing and processing linguistic structure Chomsky (1956); Pereira & Warren (1880); Colmerauer (2005). while formal semantics
connected language to logic and reasoning through explicit moaning reprosentations Tarski (1956): Montague
(9770g Montague & Thomason (1978); Woods (1978). These frameworks enabled principled reasoning
about communication but relied heavily on hand-crafted rules and domain-specific representations, limiting
scalability in dynamic and multi-agent environments.
Modern learning-based approaches revisit these foundational ideas under new computational regimes. Rather than prescribing communication semantics a priori. MARL and EL research allow com- munication protocols to arise through interaction and optimization SSukhbattar et al. (2016); Lazaridou et al.) (2018). Recent surveys on foundation agents further contextualize this evolution by framing com- munication as one component of modular, brain-inspired agent architectures that integrate memory, world
8

---

Published in Transactions on Machine Learning Research (02/2026)
modeling, goals, reward processing, and social interaction Liu et al. (2025b) From this perspective, contem- porary LLM-based and multi-agent systems do not abandon classical views of communication as action, but extend them by coupling language, learning, and coordination within scalable, adaptive, and collaborative agent societies. Many persistent challenges, such as grounding, interpretability, belief alignment, and coordi- nation under uncertainty, can thus be seen as modern instantiations of long-standing foundational questions about how agents act, reason, and commanicate to achieve collective objectives.
2.2 From Speech Acts to Learned Message Actions
Classical speech act theory views communication as a form of action chosen to achieve specific social or pragmatic effects, such as requesting, committing, or influencing beliefs Austin (1975): Searle (1969). This perspective maps naturally onto modern MART, where messages are treated as decision variables selected by agents to influence future system trajectories Bernstein et al. (2002); Sukhbaatar al. (2016). In MARL- Comm, sending message can be understood executing learned speech act whose value is defined by its contribution to long-term reward, coordination stability, or belief alignment (Sinhh et al.) (2018) Das et al. (2018). Unlike classical speech acts, which rely on shared linguistic conventions and intentional reasoning Grice [1977, MARL message actions derive their semantics implicitly through interaction and optimization. This shift replaces explicit pragmatic rules with reward-driven learning, but preserves the core insight that communication is not passive information exchange; it is action taken under uncertainty to shape the behavior of others RRusell et al. (1995). This connection clarifies why issues such grounding, interpretability, and generalization arise in learned communication: without explicit semantic constraints, speech-act-like behaviors must be rediscovered through experience rather than assumed a priori Kottur et al.
20170; Lazaridou et al. 2018
Table 1: Foundational assumptions about communication across classical AI, MARL-based systems, and LLM-based multi-agent systems.
Aspect
Classical Agent Commu- nication
MARL-Based Communi-
LLM-Based Communica- tion
cation
View of com- munication
Symbolic action governed by explicit rules (speech acts, logic)
Learned action optimized for reward
Language generation condi- tioned prompts and con- text
Semantics
Explicit, hand-defined, logic- based
Implicit, emerges from inter- action
Implicit, inherited from pre- training
Grounding
Assumed via symbolic world models
Grounded through environ- mental interaction
Weakly grounded; relies on textual abstractions
Interpretability
High (human-readable rules and meanings)
Low to moderate (latent discrete learned signals)
High surface level, uncer- tain at control level
Adaptation
Manual rule design
Policy learning via reinforce- ment signals
Prompling, fine-tuning, or hybrid RL integration
Primary limi- tation
Poor scalability and brittle- ness
Task specificity and data inef- ficiency
Lack of control guarantees and grounding
Table I contrasts foundational assumptions about communication across classical A1, MARL-based systems, and LLM-based multi-agent systems. Classical approaches treat communication as a symbolic action with explicitly defined semantics and grounding, offering strong interpretability but limited scalability and adapt- ability Russell al. (1995): Austin 1975] Searle [1969). MARL reframes communication as learned action optimized directly for reward, enabling agents to adapt communication through interaction, but often at the cost of interpretability and generalization beyond co-trained teams Sukhbaatar et al. (2016); Singh et al. (2018); Kottur et al. (017): Lazaridou et al. (2018). LLM-based systems inherit rich linguistic structure from large-scale pretraining, yielding fluent and human-readable communication, yet they lack strong grounding in environment dynamics and formal control guarantees Park et al. (2023): Mahowald al. (2024). This comparison highlights that modern hybrid LLM MAM-MARI.... systems occupy a middle ground, seeking to balance
9

---

Published in Transactions on Machine Learning Research (02/2026)
expressiveness, grounding, and control by combining learning- based communication with language priors
FAIR): Slumbers et al. (2023): Estornell et al. (2025)
3 Related Work: Surveying the Three Dimensions of Multi-Agent Communication
Prior efforts to survey communication in multi-agent systems have evolved along several lines. In this work, we focus on three prominent and interrelated dimensions: communication in MARL, emergent language sys- tems, and LLM-driven communication frameworks. While not exhaustive, these dimensions capture a wide spectrum of recent research exploring how agents exchange information support coordination, learning, and reasoning. Existing MARL. surveys often address communication only indirectly, emphasizing policy learning while overlooking communication-speciii on language development and interpretability but are typically siloed from broader multi-agent learning discussions. More recent reviews on LLM-based agents emphasize architectural workflows and coordination capabilities, yet often remain disconnected from prior frameworks. Our survey aims to bridge these per- spectives through unifying framework (Fig. D that supports comparative analysis and highlights shared trends, challenges, and open questions across these three infiuential threads of multi-agent communication research.
challenges. Surveys emergent communicattin focus
3.1 Surveys on Communication in MARL
Previous surveys MARL have primarily provided broad overviews of the field, only occasionally addressing communication as a secondary or supporting topic. Hernandez-Leal et al. Hernandez- et al. (2019) reviewed various MARL methodologies, emphasizing algorithmic frameworks and their critiques, with limited exploration agent communication strategies. Similarly, Nguyen et al. [guyen et al. (2020) presented comprchensive analysis of MARL, highlighting challenges and general solutions but allocating minimal attention to the specifics of communication mechanisms.
Gronauer and Diepold Gronauer & Diepold 2022) conducted an extensive survey focusing predominantly on deep reinforcement learning techniques and their multi-agent extensions. They discussed communication briefly, primarily the context of agent coordination tasks, without providing an in-depth or structured ex- ploration. Wong et al. Wong al. (2023) also outlined various challenges and future directions multi-agent systems, briefly touching upon comminication but primarily maintaining focus general reinforcement learning issues rather than systematically reviewing communication techniques.
Recent surveys like that of Oroojlooy and Hajinezhad Oroojjooy & Hajinczhad 2023) continue this trend, examining cooperation
mechanisms extensively while only superficially addressing com- munication methodologies, and Beikmohammadi [Beikmoham- madi (2024) presents systematic review of 16 MARL communi- cation studies from 2016-2021, focusing on their methodological rigor and identifying gaps such as a lack of standardized envi- ronments and consideration of real-world communication con- straints. Even the works specifically addressing communication, such as Zaiem el. al. Začem & Bennequin (2019) and Zhu et al. Zhu et al. (2024) have limitations, either narrowly focusing on MARL-specific methods or not incorporating recent advance- ments involving emergent languages and large language models (LLMs).
MARL-Comm
EL-Comm
LLM-Comm
Figure Three core dimensions multi- agent communication surveyed in this work: MARI-basod, emergent language, and LLM-driven communication.
In contrast, our work uniquely addresses the significant gap in the literature by providing the first dedicated, structured survey explicitly focusing on Communication. Our contributions differ significantly from prior surveys in sev- eral ways. First, we establish a novel three-dimensional review framework that integrates MART.bbased communication approaches, emergent language paradigms, and recent advancements in LLM-driven agent communication. Secondly, we systematically categorize and comprehensively analyze key works across these dimensions, highlighting essential trends, similarities, and differences. Finally, we explicitly outline open
10

---

Published in Transactions on Machine Learning Research (02/2026)
challenges and suggest clear research directions to enhance the efficiency, interpretability, scalability, and applicability of MA-Comm in real-world multi-agent systems.
3.2 Surveys on Emergent Communication
Rescarch on emergent language in multi-agent systems has gained traction as promising pathway toward interpretable and grounded communication. While Boldt and Mortensen Boldt & Mortensen (0024 provide a broad survey covering emergent communication across machine learning, natural language processing, and cognitive science, their review spans beyond the scope of decision-making. In contrast, our focus centers specifically on how emergent language facilitates coordination and policy learning in sequential decision- making settings. They emphasize emergent communication's potential for explainability and human-agent interaction, while also categorizing practical and theoretical use cases. Peters et al. Peters et al. (2024)
present & large-scale review of 181 papers, proposing a taxonomy for discrete emergent language. They systematically analyze evaluation metrics and identify challenges in language comparability, compositionallty, and semantic alignment. Their contribution is methodological, aiming standardize how emergent language is assossed. Wolff ct al. Wolff et al. (2024) address the limitations of traditional reference games and introduce more realistic, situated environments for emergent language learning. Their work highlights the emergence of bidirectional and sparse communication strategies that arise naturally under coordination pressure in complex MARL settings. Other focused surveys and domain-specific efforts include Chafii et Ch.fi et al. (2023), which explores EC-MARL in future 6C wireless networks, and Gupta et al Gupta et al. (2020). who analyze emergent communication grounded in graph- multi-agent topologies. While valuable, these works are contextually bounded and do not offer general-purpos,, integrative perspective.
Prior surveys on emergent language have primarily focused on the linguistic and structural properties of com- munication protocols, such as synlax, semantics, compositionally, and their resemblance to human language. These studies are often motivated by cognitive science and language understanding, aiming to investigate how structured language can emerge in artificial agents through interaction, without necessarily grounding that communication in task-driven decision-making. In contrast, Boldt and Mortensen's 's recent survey [oldd & Mortensen (2024) is among the few that explicitly address emergent communication in the context of multi-agent coordination, i.e., how structured language can emerge through agent interactions when mul- tiple agents interact and communicate with each other. This highlights its role not just as a linguistic phenomenon but as a mechanism for enabling effective joint decision-making.
The inclusion of EL in this survey is motivated by the pioncering work of Mordatch & Abbeel (2018). which first established a connection between emergent communication and MARL methods through MAD- DPG Lowe et al. (2017). This work demonstrated how grounded, compositional language can emerge among agents as a mechanism to complete tasks and achieve goals in multi-agent environments. The symbolic and discrete nature of the communication protocols developed in this setting subsequently inspired a range of MARL communication methods, such as Havrylov & Titov (2017): Chen et al. (2024b), which adopted dis- crete messaging to reduce communication overhead and enhance interpretability. Summarizing E.I, research in the context of multi-agent communication thus offers an interpretable framework for agent collaboration and promotes the development of broader goal of cemann-cenered AI.
communication protocols, contributing toward the
Therefore, our survey complements and extends these efforts by incorporating EL as one of the three central dimensions of multi-agent communication. Unlike prior reviews that emphasize emergent communication in isolation, provide unifidd framework that links MARL commmication, emergent language, and LLM- driven communication strategies. This broader integration facilitates comparisons, highlights cross-cutie cross-cutting challenges, and sets the stage for designing generalizable and human-alignod commuication protocols in future multi-agent systems.
3.3 Surveys on LLM-Based Communication
Recent surveys have extensively explored the integration of LLMs into multi-agent systems (MAS), particu- larly focusing on enhancing inter-agent communication, coordination, and collaboration. Wang et al. [Wang et al. (2024a) provide comprehensive review of L.L.-based autonomous agents, categorizing core capabili-
11

---

Published in Transactions on Machine Learning Research (02/2026)
Table 2: Comparison of Surveys Across MA-Comin Dimensions: our survey uniquely bridges three domains, HARR-CComm, EL, and LLM-based MAS, offering unified framework that highlights cross-cutting trends, systematic methodologies, and open challenges for building scalable, interpretable, and human-centric multi- agent communication systems.
Focus
Communication Focused
LLM Focused
Emergent Language
Systematic Framework
Survey
Area
Hernandez- Leal
MARL
Igguyen
MARL
Grousuer & Diepold 2022
MARL
Wong et a.. 2023
MARL
X
Oroo loay & haizhad 2023
MARI
K
Bellkmohhammmn
MARI.-Comm
Zalem
Hennequin 2019
MARL-Como
V
Zhu el al (2224
MU22
MARL-Comu
V
Boldt & Mortensen
2024
Emergent
X
Peters et a.
2024
Emergent
X
Woll
2021
Emergent
Chalil
2024
Emergent (Wireless)
Gupta
2020
Emergent (Graphs)
Wang ct ml
2124
LLM
La ot
2024
L.MM
Gao os
202-1h
Y1
LLM (Dialogue)
Gunn
20225
LLM (Dinlogue Eval.)
Ho al.
20244
LLM (Games)
Tran ct al
2022ue
LLM (Collaboration)
Sun et s.
2025b
LLM (Coordination)
20218
LLM (MARL)
Mosquera ct
22024
LLM (Evaluation)
Comnri ct ml
2025b
LLM (Evaluation)
/
L:t of al.
202on
L.M (Future)
Araschige & 1minn
(2025
LLM (Technological Aspects)
Survey
Above
lies such as communication, planning, memory, and tool use. Li et al. Li et al. (2024e) offer workflow-centric perspective, outlining key infrastructure components including profile modeling, self-action reasoning, per- ception, interaction, and learning in LLM-based MAS.
Guo et al. Guo et al. (024bb survey the progress and challenges of large language model-based multi-agents, covering aspects such as reasoning, cooperation, and agent modularity. In the context dialogue systems, Yi et al. et al. (2024) and Guan et al. Guan et al. (2025) analyze recent advances and evaluation strategies for LLM-based multi-tun turn conversations, highlighting communication bottlenecks and coordination failures. Hu et al. Hu et al. (2024b) focuses L.L.-based agents in game environments, discussing planning, interaction dynamics, and decision-making complexity. While these surveys highlight the growing role of LLMs in multi- agent decision-making, they largely treat communication as a secondary concern rather than a primary focus, leaving open the need for a more targeted synthesis of communication strategies in LLM- -LM--owered multi-agent systems.
Several surveys have examined collaboration and coordination among LLM agents more directly. Tran al. Tran et al. (2025a) review mechanisms for multi-agent collaboration using LLMs, while Sun et al. Sun et al. 2025b survey coordination strategies across diverse application domains. Sun et al. Sun et al. (2024a) spocifically explore how LLMs are being integrated with MARL, summarizing current progress and proposing future research directions. Aratchige and Immini Aratchige Ilmini (2025) further examine the technological challenges in building effective LLM- LLM-based multi-agent systems, addressing issues like communication reliability, shared memory, and uncortainty handling.
Complementary to these broad surveys, several evaluation studies have investigated the fundamental lim- itations of T.IM-based multi-agent cooperation. Mosquera et al. Mosquera et al. (2024) empirically assess cooperative behaviors in LLM-augmented agents using the Melting Pot benchmark, while Cemri et al. Cemri et al. (2025b) provide an in-depth analysis of why multi-agent LLM systems often fail, identifying critical
wea knessess in alignment, communication consistency, and robustness. Moreover, Liu et al. Liu et al. 2025a
12

---

Published in Transactions on Machine Learning Research (02/2026)
present broader vision of 'foundation agents', exploring brain-inspired architectures, evolutionary collabo- ration mechanisms, and the pursuit of safe, scalable multi-agent intelligence.
In contrast to existing surveys that primarily focus on architectural capabilities, evaluation benchmarks, or emerging challenges LLM-based multi-agent systems, our work frames LLM-enabled communication as one of three central paradigms within a broader taxonomy of MA-Comm. By connecting LLM-driven interaction with traditional MARL communication protocols and EL systems, provide an integrative perspective that spans diverse rescarch communitics. While MA-Comm addresses broad range of coordination and decision- making objectives beyond human involvement, the incorporation of LLMs introduces new opportunit ies for more natural, interpretable, and potentially human-aligned communication. This broader lens helps uncover shared challenges and promising directions for advancing communication strategics across agent types and application domains.
Table 2 provides a structured comparison of major surveys across the key dimensions of MA-Comm. While earlier works focus narrowly on either MARL communication, emergent language, or LLM-based systems, none comprehensively address all three paradigms together. In contrast, our survey uniquely bridges these domains, offering a unified framework that highlights cross-cutting trends, systematic methodologies, and open challenges for building scalable, interpretable, and human-centric multi-agent communication systems.
4 Methodology and Scope of the Survey
This survey follows a systematic litcrature review methodology to analyze research on multi-agent com- munication across three closely related but traditionally separate domains: commuication in multi-agent reinforcement learning (MARI-Comm), emergent language (MA-FI.), and large language model- based multi- agent communication (LLM-Comm). Our goal is not to exbaustively enumerate every paper in these areas, but to provide a principled, representative synthesis that highlights core modeling assumptions, methodolog- ical trends, and open challenges across domains.
4.1 Search Strategy and Temporal Scope
We conducted a structured literature search covering works published before 2025. To capture foundational context, a small number of carlier seminal works (c.g., Dec-POMDPs, classical team decision thoory, and communicttion-constraiiii
control) were included where necessary to ground later developments. Primary searches were performed using Google Scholar with combinations of keywords such as multi-agent commu- nication, MARL communication, emergent language, signaling games, multi-agent LLM, and language-based coordination. In addition, we manually examined proceedings of major Al and machine learning venues (NeurIPS, ICML, ICLR, AAAI, AAMAS) and relevant workshops. Backward and forward citation tracking was used identify influential works not surfaced by keyword search alone. For each search term, the stopping criterion the absence of new relevant papers full results page.
4.2 Inclusion and Exclusion Criteria
Papers were included this survey based on the following criteria: (1) the work explicitly trcats com- munication between agents as core component of the problem formulation or solution, rather than peripheral implementation detail; (2) the setting involves multiple decision- makiin agents, including cooperative, competitive, or mixed-motive scenarios; (3) the paper presents methodological contribu- tion, empirical evaluation, or formal framework related to commiicatiin learning, reprosentation, or usage; and (4) the work is published in a peer-reviewed journal, conference, workshop, thesis, or book chapter, or is a technically substantive arXiv preprint included selectively reflect recent advances in fast-moving subfields (c.g., LLM-based multi-agent communication) that are not yet fully represented in archival venues.
Conversely, papers were excluded if (1) communication was mentioned only superficially without being modeled or analyzed; (2) the work focused solely on single-agent learning or fully centralized control without decentralized execution; (3) the paper was purely conceptual or a position piece without technical substance; (4) the work was available only a non-peer-reviewed preprint and lacked sufficient technical depth,
13

---

Published in Transactions on Machine Learning Research (02/2026)
empirical validation, methodological clarity to warrant inclusion, despite the fast-paced nature of emerging areas such as LLM-based multi-agent agent communication.
These inclusion and exclusion criteria were applied consistently across all three domains considered in this survey (MARL-Comm, MA-EL, and LLM-Comm), with domain-specific judgment used only to account for differences in terminology, experimental conventions, and evaluation practices.
4.3 Coverage Across Domains and Selection for Detailed Discussion
The initial search yielded several hundred candidate papers across the three domains. After applying the inclusion and exclusion criteria, we retained a curated corpus of representative works spanning MARL- Comm, emergent language, and LLM-based multi-agent systems. Within this corpus, subset of papers is discussed in depth in the main text and tables. These papers were selected based on one or more of the following factors: (1) the introduction of a widely adopted commmicatiin architecture, framework, or benchmark; (2) an explicit or implicit investigation of who communicates with whom, what, when, why, and how agents communicate, especially works that analyze how the motivation for communication (why) directly informs the design of communication mechanisms (how); such works are discussed and compared in depth in the main text; (3) demonstrated influence on subsequent research, as evidenced by citation patterns or follow-up works; and (4) the ability to illustrate contrasts or connections across domains (e.g., between MARL-Comnn and LLmmm..m). The remaining works are cited or summarized more briefly provide breadth and contextual coverage without redundancy. This balance reflects an intentional trade- off between comprehensive coverage and analytical depth, ensuring that core ideas are examined rigorously while situating them within the broader literature.
4.4 Limitations of Scope
We note that the rapid pace of research particularly in LLM-based multi-agent systems means that new results continue to emerge beyond the temporal scope of any fixed survey. Accordingly, this survey reflects the state of the literature up to August 2024 and prioritizes works that clarify fundamental modeling as- sumptions, communication mechanisms, and design trade-ofls, rather than attempting exhaustive coverage of all contemporaneous results. The methodological criteria described above are intended to make these se- lection decisions transparent, principled, and reproducible. In total, this survey considers approxinately 400 works across MARI-Comm, MA-EL, and LLM-Comm, with approximately 100-130 representative papers examined in depth through detailed laxonomies, comparative tables, and focused discussion.
5 Communication in Multi-Agent Reinforcement Learning
Communication in Multi-Agent Reinforcement Learning (MARI-Comm) plays central role in en- abling agents to coordinate, share information, and act collectively in dynamic and partially observable environments. As agents operate under docentralized exccution, effective communication must address fun- damental challenges such as limited observability, bandwidth constraints, and non-stationarily induced by learning teammates. Early MARL systems relied on hand-designed protocols, but recent advances have enabled agents to learn when, what, and how to communicate end-to-end part of the docision-making process. This shift has positioned communication a first-class control mechanism, lightly coupled with policy learning and hori-norizon coordination.
Review Scope and Structure This section begins with background and foundational concepts in MARL- Comm (Section 5. , followed by an overview of related work and key research directions (Section 5.2 We then organize the MARL- Comm literature using an updated Five Ws perspective. Specifically, we examine who communicates with whom and what is shared through message relevance, recipient selection, and information encoding (Section 5.3 when communication occurs under timing, frequency, and resource beneficial and how it is operationalized via
constraints (Section 5.4); and why communication training objectives, protocol design, and control shaping (Section 5.5). We conclude with a discussion that synthesizes insights across the Five Ws (Section 5.6 and a bridge section that motivates the transition from MARI-Comm emergent language (Section 5.7 Figures 2 and 3 summarize the resulting taxonomy and
14

---

Published in Transactions on Machine Learning Research (02/2026)
end-to-end MARL communication pipeline. Throughout the section, highlight representative algorithms, evaluation settings, and design trade-offs, grounding all discussions within the Markov Decision Process framework Puterman (2014) to provide a unified and extensible foundation for commmicationn-ontiic MARI. research.
Suthtant et al (2010) Boog et (5017] Tark et
2015 Das
Relevance Filter- ing (Sec. 5.3.1
<t (2013) Manrams di (231)) Toupss 6 (elrs
Comnte dtat
(2000) (Pig 101
Staing
Daars Lo 2013
(322))
Kmm cl
(Araen my Tange Shhap et [222] ehr 2020)
10 el
Who/What What Comm. (Tab. 3
Targeted Com- munication (Sec. 0.3.2
Riit
p10 [[mm Triishiii if p000 100011220
Wars at 20000
[iu cl al (00)
Nu 202) [Mw it 2020
Toender et
20.N
Pa at
Freed at
(0at Kiits & Momians
200 Rkm
Calblartt at al Aub Aubbiiing SSina at bult
Message Integra- tion (Sec. 5.3.3
Zhon o 24 (eJju 2oreef
Wanr
guUu Gupl
Lunny dilinn Lang Loi2e0 Ding
Sheng
Learning Commu- nication Timing (Sec. 6.4.1
Sulehbaster
201e
Foeeter a amste
Arn
When to Comm (Tab. 1
MARL-Comm
Inner Integra- tion Timing (Sec. 5:112
Sinph
Keng
2023
Foenter
Rrasmer
Agaamal 2010
ct (2000) Oo 2020 (Matohore di fumm
Training Frame- work (Sec. 5.5.1
No [201
200
Ho A0ur
Map
700
2010)
(lrl) page Mairina PiPes)
Topeder Fpitaa
Pars Perr
2017 Das
Why/How to Comm. (Tab. ()
Communication Shaped by Re- ward Objectives (Sec. 5.5.2
Charg
#
202u Str del
201g 200
Yun da a (ol
Miltinn Mestina ai Sheee
41
Treed a
20al Fi at
Cama di 1A2
272h
Smy af
2ols Pay
Au Jong st 201
(.m at
Kong af
01 2 72011
Lo
Hang
Message Format Ohjectives (Sec.
5mgh al 2015 Das et 2015 Lows al 2017 B Foerser
4 a22222 Krspppor et ad all 2l27/Shh SSrw dl Mlae
Lzams
4 al (2515) Kergee d a d. Tun et (2000) (oappnn
(22 (onguai Willin (i55mea (A00
Wartam
Figure 2: MARL-Comm Agents Taxonomy
5.1 Background of MARL-Comm
Reinforcement learning trains models to make sequential decisions in uncertain environments by maximiz- ing cumulative rewards through trial and error [Sutton & Barto (2018). While is traditionally designed for single agents, many reinforcement learning applications involve multiple agents, making MARL a more appropriate framework Busoniu et al. (2008). MARL studies maaisiionmmkinng among agents in shared en- vironments, with applications in nication networks Cortes et al. (2004); Choi et al. (2009), and social science Casteelfannhii (2001); Leibo et al. (017)). MARL problems can be fully cooperative, fully competitive, or mixed, depending on the agents' objectives. Multi-agent communication is a fundamental aspect of MARL, enabling agents to share information, coordinate strategies, and achieve collective objectives efficiently. Tn MARL, agents operate in
systems Adler & Blue (2002); Wang et al. (2016). commu-
15

---

Published in Transactions on Machine Learning Research (02/2026)
Sec. 5.3
Relevance Filtering
Targeted Communication
Message Encoding
Past Knowlcdge
Future Knowledge
Nearby Agents
Concatenation
Leaming Agents
Equally Valued Integration
With Representative Agents
Without Representative
Agents
Representative Agents
Weighting Prioritization
Sec. 5.4
Communicate
Limited Bandwidth
Scheduling Communication
Resource
Policy-
Combined Policy and Value
Value- level
level
5.5
Cooperative
Agent
Competitive
Agent N
Mixed
Explicit Implicit
Leaming Training
Parameter Sharing
Execution
Centralized
Centralized
Concurrent Leaming
Decentralized
Decentralized
Figure 3: Organization of the key dimensions of communccation design in Sec. what and whom to
communicate with (Sec. 5.3 when to communccaee under resource constraints (Sec. 5.4). why communicate
and how communication shaped by interaction scenarios and learning architectures (Sec. 5.5) Tt highlights
how various communication mechanisms are conditioned on task structure, bandwidth limitations, agent relationships, and training/exccuttan exccution paradigms.
16

---

Published in Transactions on Machine Learning Research (02/2026)
shared environment, and their joint actions influence the state transitions and rewards. Communication allows agents to overcome partial observability, align their policies, and avoid conflicts or redundant actions.
Formal Framework Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs) Bern- stein et al. (2002) are a multi- agent extension of a partially observable Markov decision process, which models cooperative MARL, where agents lack complete information about the enviromment and have only local observations. Communication is crucial for strategy coordination in this scenario. A Dec-POMDP with commuiccttion is often modeled as tuple Chen et al. (2024b) D = (S, P.9, O, I,,,,, R, f), where I= -1.2....,, ] is a set of n agents, S is the joint state space and A = A x A2 x ..... x An is the joint action space. Here a = (a1, ual action space of agent The joint action influences the environment, which transitions to new state and provides a global reward r. P(s'|s, a) S x A x S + [0, 1] is the state transition function. Ω is the observation space. iss S x 1 - Q is a function that maps from the joint state space to distributions of observations for each agent i. R(s,a) : :S 55x x A + R is the reward function in terms of state and joint action a, and y is the discount factor. Communication can be modeled as an additional action space, where agents send and receive messages my to share information. At each time step, agent i generates a message m, fi h.) based on its local observation o E O and internal state h2 E H, where fi O x H + M is a message-generation function that maps observations and internal states to message space M, often implemented as neural network. Each agent receives messages {myhee, from a (possibly dynamic) subset of other agents N₁ S (1,....,N] NT} \ (i), and aggregates them into a context vector Q: = (:mmsbee)), where 9. is an aggregation function such as averaging, concatenation, or attention. The agent's policy Ti uses the aggregated context e; along with its local observation o, to select an action a, ~ (T(,, This framework highlights the dual role of communication: (1) enabling agents to share local observations and (2) facilitating coordination by aligning their policies based on shared information.
an) E A denotes the joint action of all agents and A, is the individ-
Based on this background, we provide a review of communication--asee MARL algorithms in the following subsections, with a particular focus on works whose primary novelty lies in the design of communication strategies. One category of such algorithms aims to enhance cooperative multi-agent learning by improving the efficiency and relevance of the communication process. These works either introduce mechanisms such as gating and selective message filtering to reduce redundant communicttio or leverage advanced techniques like attention mechanisms and graph-based communication protocols to ensure meaningful message exchange. Another category focuses on extended MARL sctups, such as hicrarchical or task-specific communication, utilizing the idea that communication messages can serve as embeddings or representations of tasks or roles within the multi-agent system.
5.2 Overview and Taxonomy of MARL-Comm
Communication plays a crucial role in MARL by enabling agents to share information, coordinate effectively, and tackle complex tasks in partially observable environments. Early foundational works, such as Comm- Net |Sukhbaatar et al. (2016), DIAL, and RIAL Foerster et al.] (2016a), pioncered communication lcarning for cooperative agents. CommNet uses a shared neural network to process local observations and aggregates messages via mean pooling for decentralized gles with scalability and communication delays. DIAL and RIAL introduced more explicit communication protocols, where agents exchange discrete or continuous messages and jointly learn both their policies and communication strategies through reinforcement learning. These approaches, while innovative, require fully connected communication networks, leading to high bandwidth demands as the number of agents increases.
making..... While effective in small-scale settings, it strug-
To address the scalability and efficiency challenges, subsequent research has focused on adaptive and selective communication mechanisms. For instance, TC3Net (Singh et al. (2018) employs gating mechanisms to allow agents to dynamically decide whether to communicate based on their current states, reducing redundant messages. TarMAC Das et al. (2018) introduces multi-headed attention to improve the relevance of com- municated information, enabling selective sharing among agents. Similarly, ATOC [Jiang & Lu (2018) forms dynamic communication groups based on agent proximity and employs bi-directional T.STMs to aggregate messages within groups. More advanced methods, such as SARNet [Rangwala & Williams (202)), incorpo- rate structured reasoning and attention mechanisms to assess the relevance of received messages and past memories, enabling more nuanced decision-making. These methods collectively explore key questions such
17

---

Published in Transactions on Machine Learning Research (02/2026)
when, what, and with whom agents should communicate, offering significant performance improvements in MARL tasks.
Recent work also explores communicttin constraints, such as limited bandwidth and noisy channels. For example, SchedNet Kim et al. (2019a) selects only a subset of agents to communicale, optimizing within bandwidth limits, while IMAC Wang et al. (2020b) regularizes message entropy to reduce communication overhead. These approaches emphasize the tradc-offs between communication efficiency and performance, paving the way for practical MARL systems in real-world scenarios. To further advance MARL-Comm, new methods must address challenges such as scalability, interprelability, and performance guarantees under real- istic constraints Chen et al. 2024bb. Future work should explore integrating sparse, discrete communication protocols with thcoretical guarantoes, such as regret minimization, to enhance robustness and applicability in large-scale environments Chen al. (0244b).
Based on the foundational advancements in MART-Comm, we provide a review of communication--asse MARL algorithms in the following subsections, with a particular focus on works whose primary novelty lies in designing efficient and scalable communication strategies. One category of such algorithms seeks to enhance MARI-Commm by introducing mechanisms like attention or gating to optimize when, what, and with whom agents should communicate. These approaches aim reduce redundant communication, improve message relevance, and address practical constraints such as limited bandwidth or noisy channels.
Key Challenges Despite its potential, multi-agent communication MARL faces several core challenges: (1). Scalability in Large State and Agent Spaces: In large-scale environments, each agent typically observes only small portion of the global state. Agents often have limited local observations, making it difficult to infer the global state or the intentions of other agents. Communication is essential for enabling co- ordination under these conditions, but exchanging high-dimensional or unfiltered observations among many agents can be inefficient and computationally prohibitive Cao et al. (2013) The challenge lies in designing scalable protocols that selectively share information while preserving task-relevant signals. (2). Communi- cation Overhead: As the number of agents increases, communication complexity grows rapidly, leading to significant overhead. Techniques such message filtering, attention mechanisms, and hierarchical structures aim to mitigate this. For instance, Comm Sukhbaattar et al. (2016) uses shared centralized architecture to enable contimuous communication, but it does not scale well with the number of agents. MADDPG Lowe et al. (2017) introduces a centralized critic and decentralized actors, where agents generate communication as part of their actions. However, its policy network often overfits to specific agent configurations, limit- ing generalizability in large-scale scenarios. (3). Non-Stationarity: In MART, each agent operates in non-stationary environment because other agents are concurrently learning and updating their policies. This dynamic context makes it difficult to learn stable communication strategies, as the meaning and utility of messages may shift over time. Credit Assignment: In cooperative settings, determining the contribution of each agent to the collective reward is non-trivial, yet. essential for effective learning. Without proper credit assignment, agents cannot accurately update their policies based on individual impact, which can lead to suboptimal or unstable learning dynamics. Commuication can help agents better coordinate their actions, but it also introduces additional dependencies between agents" behaviors and messages, making it more complex to attribute success or failure to specific contributions. (5). Interpretability: Learned communication protocols in MARL are often embodded in latent continuous spaces, making them hard to interpret. Unlike hand-crafted protocols with explicit semantics, learned messages lack transparency, making it difficult to analyze their meaning, influence on decision- or alignment with human reasoning. (6).
Lack of Theoretical Guarantees: Many MARL-Comm methods rely on heuristic mechanisms for learning continuous or discrete communication Focrster et al. (0166)); Singh et al. (2018); [iian & Lu (2018) Das
et al. 22018; Rangwala & Williams 20020); Havrylov & Tit.ov (2017); Mordatch & Abbeel (2018). While
some approaches model discrete, human- like messages, they often lack formal guarantees such as policy convergence or regret minimization, limiting their theorctical robustness and practical reliability.
Key Methodologies Several approaches have been proposed to address these challenges, which can be broadly categorized as: (1). Centralized Communication, such as CommNet (Sukhbaatar et al. (2016)). uses a shared neural network to aggregate messages from all agents and compute a global context. The mes- sage generation and aggregation steps are formalized as: m, hi); G == ==N-T Ejjt i mj, where is the
18

---

Published in Transactions on Machine Learning Research (02/2026)
averaged message from all other agents. While this approach simplifies coordination, it introduces single point of failure and may not scale well to large systems. (2). Decentralized Communication, such as Tar- MAC (Das et al. (2018)) ) enables agents to communicate directly with each other using attention mechanisms. The message aggregation step is formalized as: G; = = Q:y][ aij = softmaxx("; Wu;), where aij is the attention weight between agents and j. 4 and uj are learned embeddings, and W is a learned weight matrix. This approach is more scalable but requires robust protocols to handle message delays and redundancies. (3). Learning to Communicate, such as IC3Net (Singh et al. (2018)), focuses on end-to-end learning of communication protocols. The policy update step is formalized as: 0g (T: (i), G = (( hy4)), where 9s is a learned aggregation function. This approach allows agents to develop communication strategies that are tailored the specific task and environment. (4). Emergent Communication, such as in Mordatch & Abbeel (2018); Lazaridou et al. (2016), studies how agents can develop their emergent communication protocols through interaction, without explicit supervision. The message generation step is formalized as: m, = hi), fi is learned through reinforcement learning. This approach is often studied in the context of cooperative gaines or language learning tasks. We will use a separate section to introduce this line of work in Sec. 6 (5). LLM-Guided Communication: Approaches such as Li et al. leverage Large Language Models (L.Mss) to guide agent communication toward more interpretable, natural language-like structures. The message generation process is formalized as m, = fi h4, LLM), where fa is a function conditioned not only on the agent's observation o, and internal state h,, but also informed by a pre-trained LLM. This integration allows agents to generate structured, iuman--nderstandable messages while preserving or even enhancing task performance. LLM-guided communication has shown promise in tasks like collaborative planning and ad-hoc teamwork, leading to improved generalization and faster communication emergence. We provide a detailed discussion of this research direction in Sec. 7 (6). Return Gap Minimization, such in Chen et al. (2024b), quantifies the gap between the optimal expected average return of ideal policy with full observability, i.e., x' = [x] (a.lol..... on), Vi] and the optimal expected average return of a coammmnication-cnabble
policy, i.c., т = [T;[a,]o;, m),), vi]. This result enables such works to recast multi-agent communication into a novel online clustering problem over the local observations at each agent, with messages as cluster labels and the upper bound on the return gap as clustering loss.
partially-observabe
5.3 Who, Whom and What to Communicate: Information and Recipient Selection
One of the main challenges in MARL-Comm is determining what information should be shared among which agents in order to maximize coordination while minimizing redundant or unnecessary communica- tion. Previous surveys have often treated what to communcaat and who or whom to communicate with as separate topics, even though the two are tightly coupled in both problem formulation and algorithmic design. In practice, selecting message content and selecting recipients are usually handled jointly Das et al.) (2018); Rangwala Williams (2020), and many methods learn these choices together. This section therefore presents a unifidd discussion in which both information selection and recipient selection are viewed as part of the question of 'what to communicate'. We identify three key challenges in jointly determining what to communicate and to whom: (1). Relevance Filtering: Determining which information is essential for coordination and selecting the recipients who will benefit from receiving it. (2). Avoiding Redundancy: Ensuring messages are sent only to agents who cannot infer the information independently, thereby reducing unnecessary or duplicated communication. (3). Compression and Encoding: Constructing compact mes- sage representations tailored to the intended recipients while meeting bandwidth and scalability constraints.
5.3.1 Relevance Filtering of Messages and Recipients
A central component of determining what to communicate is identifying which agents should receive each piece of information. Relevance filtering therefore involves jointly selecting message content and appropriate recipients, ensuring that communication is both informative and directed toward agents for whom the in- formation is actionable. After communication links are established by a communication policy, agents must determine which information should be shared and which recipients will benefit from receiving it. Under partial observability, local observations become especially valuable, and agents often rely on historical ex- perience, intended actions, or anticipated future plans to generate informative messages tailored to specific recipients.
19

---

Published in Transactions on Machine Learning Research (02/2026)
Encoding Past Knowledge Past observations, actions, and internal states provide a rich source of in- formation for relevance filtering. These signals are frequently encoded into compact messages that are selectively routed to relevant recipients. Models from the Recurrent Neural Networks (RNN) family (e.g.. Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU)) are especially effective for this purpose, as they capture temporal structure and selectively preserve information that may be useful for downstream decision-making. Representative works encode either historical observations Sukhbaatar et al. (2016) Kong
et al.
(2017): ;; Jiang et al. (2018); Das et al. (2018): ; Malysheva et al. (2018); Kiline & Montana (2018); Singh
ct al.
2018): Pesce & Montana 2020a]; ;: Zhang et al. (2019); Wang et al. (2020b); Liu et al. (2020); Mao et al.
(2020g; Sheng et al. [2022];
; Freed et al. (2020a); Hu et al. (0200a); Zhang et al. (202)] Gupta et al. (2023
Niu et al. (2021) ;: Du et al. (2021 : Yun et al. (2021) or action obbeevatono histories Foerster et al. (2016a)
Peng et al. (2017). depending on how message relevance is defined for different recipients. Nevertheless, if
there is representative agent, messages will be generated and transformed from agents to the representative
agent, and then from the representative agent to agents. Therefore, we differentiate recent works on using
existing knowledge as messages into the following two cases.
Through Representative Agents: When using a represenlative agent, communicated messages are generated in two stages. First, local observations can be encoded (Sukhbaatar et al. (2016); Kong et al. (2017); Jiang et al. (2018); Das et al. (2018); Malysheva ct al. (2018) or directly sent Das al. (0018); Malysheva et al. (2018) to the representative agent. Then, the representative agent aggregates these local (encoded) observations and produces either single global message for all agents Gupta et al. (2023) or individualized messages tailored to each agent SSukhbaatar et al. (2016); Kong ct al. (2017); Jiang ct al. (2018); Das et al. (2018); Malysheva et al. (2018). This mechanism removes the burden message integration from individual agents. Because the representative agent is responsible for determining both the structure and the routing of the aggregated information, the design intrinsically links what information is communicated with whom it is intended for. Systems such as CommnNt (Sukhbaatar et al. (2016) and MS-MARL-GCM Kong et al. (2017) generate a shared global message that captures information relevant to all teammates, while architectures like HAMMER Gupta et al. (2023) or TarMAC Das et al. (2018) enable agent- specific messages, reflecting differentiated informational needs across recipients. In these settings, the representative module operationalizes recipient selection by shaping the content and distribution of messages simultaneously.
Without Representative Agents: Without a representative agent, messages are exchanged di- rectly among agents. DIAL and RIAL Focrster et al. (2016a) encode past observations and actions into messages. BiCNet Peng et. al. (217)) shares both local and global observations among agents. Other approaches directly communicate raw observations T.uu et al. (2020), or rely on feed-forward networks Liu et al. (2020]; Mao et al. (2000); Sheng et al. 2002); Freed al. (2020a). MLPs Freed
ct al. 2020a) : Hu et al. (2020a), autoencoders Freed et al.
(2020a). , CNNs Liu et al (220 Mao et al.
[2020], RNNs SSkhhaaaaar et al. (2016); Kong et al. 2017 ; Jiang et al. (2018):
; Das et al. (2018)
Malysheva et al. (2018), or GNNs Liu et al. (2020): : Freed et al. 2020a to encode local observa-
tions for communication. In decentralized settings, selecting message content is inseparable from broadcasting, neighborhood-basee routing, or attention-guider message passing all implicitly define whom the information is meant for. Methods such as DIAL Foerster et al. (2016a), BiCNet [Peng et al. (2117), or GNN-based communication Liu et al. )2000); Freed et al. (2020a) thus jointly learn the relevance of information and the appropriate recipients, rellecting a direct coupling between what is transmitted and who benefits from it.
determining which agents require that information:
In addition, some methods communicate more specialized information. For example, in GAXNet Yun et al. (2021), agents exchange attention weights to coordinate the combination of hidden states from neighboring agents, providing fine-grained recipient-aware message content.
Encoding Future Knowledge We use Futture Knowledge' to refer to either intended actions J Jiang & Lu (201)) a policy fingerprint (i.e., current action probabilities in a particular state) Chu et al. (2020): Qu et al. (2020), or future plans Kim et al. (2020). which can be generated by simulating a model of environment dynamics Kim et al. 2020). As intentions and plans are state-related, recent works usually
20

---

Published in Transactions on Machine Learning Research (02/2026)
encode intentions together with local observations to generate more relevant messages. Because future- oriented signals are meaningful only to agents whose decisions depend on anticipating the sender's behavior, this category naturally couples what is communicated with whom it is communicated to. In intention- based methods such as ATOC Jiang & Lu (0018) or fingeprintt-sharing approaches Chu et al. (2020); Qu et al. (2020), the shared information is selectively transmitted to teammates whose policies must coordinate tightly with the sender. Planning-hasse based communication Kim et al. (2020) similarly directs predictive rollouts abstracted plans to agents whose future actions are interdependent. Across these approaches, future knowledge functions as a targeted commmicatiion mechanism in which message content is shaped by the specific recipients who benefit most from predictive information.
5.3.2 Redundancy Reduction via Targeted Communication
A key feature of human communication is the ability to engage in targeted interactions. Instead of broad- casting messages to all agents, as explored in previous work CommNNe (Sukhbaatar et al. (2016), RIAL and DIAL Foerster al. (2016a), and IC3Net [iinhh et al. (2018), directing specific messages to intended recipients can enhance collaboration in complex environments. For instance, in team of search-and-rescue robots, message like 'smoke is coming from the kitchen' is relevant to firefighter but irrelevant to bomb defuser. Targeted communication therefore directly reflects the "whom" " dimension, specifying which agents are expected to receive and act on a given piece of information. In this section, we focused on how agents learn whom to communicate to while performing cooperative tasks partially obsorvable environments. The communicatee lype defines which agents are potential recipients of messages in a MARL-Comm system. In the literature, communicatee types can be categorized based whether agents communicate directly with one another or through a representative agent.
Communicating with Nearby Agents In many MARL systems, communication is only allowed between nearby agents. Nearby agents are defined in various ways, such as agents that are observable Yun et al. (2021), agents within a certain distance Jiang et al. (2018); Agarwal et al. (2019b); Sheng et al. (2022). or agents that are neighbors on a graph Chu et al. (2020): : Qu et al. (2020). Neighboring agents can also emerge during learning instead of being predefined, as in GA-Comm Liu et al. (2020), MAGIC Niu let al. (2021), and FlowComm Du et al.] (2021), which explicitly learn a graph structure among agents. In these works, "whom" is determined either by spatial proximity or by learned interaction graph, binding message relevance to a specific recipient set. However, in GA-Comm and MAGIC, central unit (e.g.. GNN) learns the graph structure and coordinates messages, so the agents do not communcate directly and are instead managed through a representative agent. Dynamic Graph Networks (DGN) Jiang et al. (2018) restrict communication to the three closest neighbors based on a predefined distance metric. This locality- based communication is particularly important in dynamic environments, where agents' positions and states change frequently, requiring timely and context-specific message exchanges to maintain effective coordination and avoid communication bottlenecks. Similarly, the Agent-Entity Graph Agarwal et al. (209bb) also uses distance to measure proximity, allowing communication between agents as long as they are close to each other. MAGNet-SA-GS--N
Malysheva et al. (2018) uses attention mechanisms to optimize communication by selecting the most relevant messages, with a pretrained graph limiting communication to neighboring agents. Learning Structured Communication (LSC) Sheng et al. (2022) enables agents within cluster radius to decide whether to become leader agent, with all non-leader agents communicating with the leader in that cluster. This structured communication protocol improves both communication efficiency and interpretability. NeurComm Chu et al. (2020) and Intention Propagation (IP) Qu et al. (2020) restrict communication to neighbors on a preset graph structure during learning, facilitating coordination by sharing decision decision-making policies and intentions (i.e., action probabilities) with neighboring agents. FlowComm Du et al. 2202)) p dynamically manages communication using messago-passii passing techniques in a flow-based system, optimizing the routing of information based on environmental flow, while also learning graph structure to identify neighboring agents. Lastly, GAXNet Yun et al. (2021) uses atteitoon mechanisms to combine hidden states from observable neighboring agents, allowing for selective message passing that helps agents focus on critical information while filtering out irrelevant data. Across these methods, both the message content and the chosen recipients are co-determined by spatial, structural, or learned adjacency relations.
21

---

Published in Transactions on Machine Learning Research (02/2026)
Communicating with Other Learning Agents When nearby agents are not explicitly defined (e.g.. via proximity-based graphs), communication typically occurs among all learning agents or through mech- anisms that selectively determine recipients. Several methods support communication in this scenario by either broadcasting messages or learning whom to communicate with. DIAL (Differentiable Inter-Agent Learning) Foerster et al.] (2016a) enables differentiable communication between agents, allowing them to op- timize message content jointly via. backpropagation. RIAL (Reinforced Inter-Agent Tearning) Foorster et al.. (2016a) instead uses reinforcement learning to select messages that maximize collective rewards, assuming fixed set of recipients. CommNet (Sukbaatarr et al. (2016) facilitates communication by averaging hidden states across all agents, implicitly assuming full communication among teammates. BiCNet (Bidirection- ally Coordinated Network) Peng et al. (2017) applics RNNs to support recurrent communication of local and global observations, again relying on shared message exchange among all agents. TarMAC (Targeted Multi-Agent Communication) Das et al. (2018) introduces learned attention weights that determine whom to send messages to at each step, enabling targeted communication and reducing unnecessary message passing. MADDPG-M Aalulti-Ageni DDPG with Messages) Kiline & Montana (2018) extends the MADDPG frame- work with continuous message exchange but does not explicitly control recipient selection, assuming messages are accessible to all agents. IC3Net (Implicit Communication in Collective Intelligence) (iighh et al. (2018) enhances communication efficiency by allowing agents to learn a gating mechanism that determines when to communicate, reducing bandwidth usage. SchedNet Kim et al. (2019a) implements a scheduler that selects which agents can communicate in each timestep, effectively learning both when and who communicates un- der constrained bandwidth. DCC-MD Kim et al.] (2019b) scales message exchange by dynamically dropping messages based on importance, which indirectly controls the recipient set. VBC (Value- Communi- cation) Zhang et al. (2019) uses value function information to prioritize which state information to share, helping agents coordinate by highlighting key states without broadcasting all messages. Diff Discrete Freed et al. (2020a) and I2C (Intentional Information Communication) Ding et al. (2020) reduce communication load by encouraging sparse or goal-relevant communication. I2C, in particular, guides agents to share only information aligned with their current intentions, which can implicitly filter recipients. Information Shar- ing (IS) Kim et al. (020) focuses on transmitting internal state representations to synchronize behavior in cooperative tasks. ETCNet (Event-Triggers Communication Network) Hu al. (2020a) triggers communi-
cation only upon detecting significant environmental changes, thus reducing redundant message broadcasts. Finally, Variable-length Coding Freed et al. (2020a) and TMC (Temporal Message Compression) Zhhag et al. 2020) compress historical information into concise messages, improving efficiency but still assuming full communication unless paired with selective mechanisms. In all these settings, redundancy reduction hinges on selecting both what to send and to whom it should be directed, linking content relevance to intended recipients.
Communicating through Representative Agents In systems where agents communicate via a repre- sentative agent, communication between agents is mediated by intermediary or representative agent. This intermediary coordinates, processes, and distributes messages to and from individual agents, enhanc- ing commuicatiin efficiency and often simplifying the coordination process. Several approaches have been proposed for this type of communication, as summarized here. MS-MARI-GGMM [Kong et al. (2017) intro- duces a master agent who gathers local observations and hidden states from agents within the environment. The master agent then sends a common message back to each agent, thereby reducing the need for direct communication between individual agents. This centralized communication model improves coordination in partially observable environments. Similarly, HAMMER Gupta et al. (2023) employs central representative agent that collects local observations from agents in the multi-agent system (MAS) and distributes both a global and individualized message to each agent. This ensures efficient communication by aggregating and distributing relevant information to agents, facilitating better decision-making. ATOC Jiang & Tu (2018) uses an LSTM to link nearby agents who choose to join a communication group. The representative agent in this system sends coordinated messages to the group members, streamlining communication and improv- ing the ability of agents to act cohesively based on shared information. MD-MADDPG |Pesce & Montana (2020a) maintains a shared memory among agents, allowing them to selectively store local observations in the memory. The representative agent then loads and relays this information, facilitating communication while reducing message overload. This method ensures that only relevant information is communicated to the nec- essary agents. IMAC Wang et al. (2020b) defines scheduler that aggregates encoded information from all
22

---

Published in Transactions on Machine Learning Research (02/2026)
agents and sends individualized messages back to each agent. This centralized scheduling of communication helps agents focus on their individual tasks while still benefiting from global knowledge. In GA-Commn Liu et al. (2020) and MAGIC Niu et al. (2021), global message processor is used to aggregate information from agents and then distribute messages based on agent-specific weights. This enables more customized com- munication, where each agent receives a message tailored to its unique context within the system. Finally, Gated-ACMI. Mao et al. (2020) takes slightly different approach, allowing agents to decide whether not to communicate through message coordinator. This introduces an additional level of decision-making, where agents evaluate the necessity of communication based on the relevance of the information at hand. These methods illustrate the utility of using representative agents to enhance communication in multi-agent systems by centralizing, filtering, and distributing messages in ways that improve overall system efficiency and agent coordination. Because the representative agent explicitly determines recipients, targeted commu- nication becomes part of the central design choice: the intermediary decides whom each synthesized message is intended for, embedding the "whom" dimension into the communication architecture itself.
5.3.3 Compression and Encoding via Message Weighting
Recent studies in MARL-Comm often handle multiple incoming messages collectively. Message Integration refers to the method by which received messages are combined before being passed into the agents' internal models. In cases where representative agent is used, the representative agent typically consolidates and coordinates messages, providing a unified single message to each agent, this climinates the need for individual agents to manage the combination of messages. However, when no representative agent is involved, each agent must independently determine how to merge multiple incoming messages. Since the communicated messages reflect the sender's personal interpretation of the learning process or the environment, some messages may be prioritized over others. These integration mechanisms primarily shape what information is ultimately preserved, compressed, or emphasized, although in some architectures, e.g., attention-based weighting, the prioritization of messages can indirectly influence which agents' information becomes most influential during coordination.
Message Concatenation In this approach, incoming messages are concatenated into a single input vector without applying explicit preferences or weighting. This ensures that no information is discarded, but it increases the input dimensionality, which may become problematic the number of agents grows. Methods such as DIAL Foerster et al. (2016a), RIAL Foerster et al. (2016a), ETCNet Hu et al. (2020a), Variable- length Coding Freed et al. (2020a), MADDPG-M Kiline & Montana (2018), and Diff Discrete Freed et al. (2020a) adopt this approach. For instance, DIAL and ETCNet concatenate scalar message values, preserving simplicity while retaining essential information. SchedNet Kim et al. (2019a) uses shorter message vectors to reduce communication overhead before concatenation, allowing agents to operate more efficiently within limited bandwidth settings. Since all messages are treated equally without prioritization, this method is particularly suitable for small-scale systems, where preserving the entirety of transmitted data is more beneficial than selectively filtering or compressing messages. Because concatenation retains the full set of incoming messages, it directly shapes what information enters the agent's policy without making any distinction about whom the preserved content originated from; thus, concatenation reflects a content- preservation choice rather than recipient-selection mechanism.
Equally Valued Message Integration When agents are assumed to contribute equally, their messages are integrated using permutation-invarian operations such as summation or averaging, rather than concate- nation. While concatenation also treats messages uniformly in the sense that each is included, it preserves ordering and increases dimensionality, whereas averaging or summing compresses the messages into a fixed- size representation without assuming any order. Techniques such as CommNet [Sukhbaatar et al. (2016) IC3Net (Singh et al. (2018), and VBC [Zhang et al. (2019) employ this form of integration. In Net, all incoming messages
averaged to produce single vector, ensuring equal influence from each sender. IC3Net uses a similar averaging mechanism over messages from neighboring agents. FlowComm Du et al. (202)) instead sums the messages, maintaining equal contribution while enabling gradient-based message refinement. This approach is especially effective in cooperative tasks where all agents are assumed to have similarly relevant information, allowing simple yet effective message integration without the need for atten- tion or learned weighting. Because every incoming message is weighted identically, this integration strategy
23

---

Category Published in Transactions on Machine Learning Research (02/2026)
reflects specific choice of what information the agent retains, namely, aggregated summary of all inputs, while implicitly treating all senders as equally relevant, without introducing any mechanism to differentiate who or whom the information came from.
Weighted Message Prioritization In this category, agents assign different levels of importance to re- ocived messages based on their relevance or utility to the current task. Some approaches rely on handcrafted rules to selectively prune messages deemed less useful. For example, DCC-MD Kim ct al. (2019b) and IMC/Zhang et al. 2020 implement manual thresholds on message magnitude or relevance scores to discard low-salience messages, thereby reducing commuicatiion overhead. More commonly, modern approaches uti- lize attention mochanisms to learn these weights dynamically. Attention-based models such as TarMAC Das et al. (0018), Agent-Entity Graph Agarwal el al. (2019b), and MAGNeI-SA-GS-ME Malysheva et al. (2018) compute relevance scores for each incoming message and combine them using a weighted sum. This enables the receiving agent to focus the most informative messages while suppressing noise from less critical sources. By assigning different weights to messages, this approach specifies what information should be emphasized and what should be ignored. It also introduces an implicil notion of whom the agent al alnds to, because messages from more relevant senders receive higher importance and have a stronger effect on the final representation.
The Use of Neural Networks Neural networks are frequently employed to automatically learn the importance of messages, enabling agents to extract and prioritize useful information without hand-crafted rules. In IMAC Wang et al. (2020b), GA-Comm Liu et al. (2020). and HAMMER Gupta et al. (2023), simple multilayer perceptrons (MLPs) are used to integrate messages while implicitly learning which ones are most valuable. Gated- ACML Mao et al. (2020) takes a similar approach but introduces a gating mechanism to regulate communication links with a central representative agent. For more complex sottings, convolutional neural networks (CNNs) are used to capture spatial dependencies among agents, as in DGN Jiang et al.) (2018), which is particularly beneficial in dynamic environments where agent positions change frequently and updated spatial context is essential. Recurrent neural networks (RNNs), including Long Short-Teme Memory (LSTM) variants, are widely applied for sequential decision-making tasks in communication-contic coordination. These are used in ATOC Jiang & Lu (2018), T2C Ding et al. (2020). NeurComm Chu el al. (2020 BiCNet [Pnn et al. (2017), MD-MADDPG Pesce & Montana (2020a), and GAXNet Yun et al. 22021 where they capture temporal dependencies across communication rounds, allowing agents to make informed decisions based evolving message histories. Lastly, graph neural networks (GNNs) have gained popularity in works such as MAGIC Niu et al. (2021). GA-Comm Liu et al. (2020), and LSC [heng et al. (2022). These approachos utilize a learned graph structure over agents to model intoractions and assign relevance to neighboring messages, often combining topological information with atlention mechanisms to improve communication efficiency and task performance. These models determine what information should be extracted from messages, because they learn to highlight useful patterns and suppress noise. They also inffuence whom the agent effectively listens to, since learned graph edges, attention scores, or recurrent dependencies determine which senders have a stronger impact on the final communication representation.
Table 3: Taxonomy of Approaches to What to Communicate" (Sec. 5.3 in MARL-CCmm. The taxonomy also illustrates how message content is closely linked to the intended recipients, since relevance filtering, targeted communication, and woighting mechanisms depend on whom each message is for.
Approach
Representative Works
Relevance Filtering and Recipient Matching (Sec. 5.3.1
Encoding Past Knowledge
With representative agents:
Sukhbaatar
(information selection tied to fixed or learned recipients) (with/without representative agents)
et al.
2016); Kong et al. (2017): Jiang et al.
2018
Das et al. (2018) Malysheva et al.
2018
Gupta et al. (2023)
Without representative agents:
Foerster
et al.
(2016a) Peng et al. (2017): Liu et al.
2020
Mao et al. (2020): Sheng et al.
2022
Freed et al. 2000 : Hu et al.
2020a
Yun et al. (021
Continued next page
24

---

Published in Transactions on Machine Learning Research (02/2020)
Table continued from previous page
Category
Approach
Representative Works
Encoding Future Knowledge (shared with specific. down- stream recipients) (intended actions, finger- prints, plans)
Jiang & Lu 2018): : Chu et al 2020
Qu et al.
2020
Kim al. 2020
Targeted Communication and Recipient Selection (Sec. 5.3.2
Nearby Agents
Static:
Jiang et al. (2018); Agarwal et al.
(proximity- recipient lection) (static/dynant proximity)
(2019b
Sheng et al.
(2022);
Chu et al.
2020
Qu et al. (2020
Yun et al. (2021)
Dynamic: Liu et al.
2020
Nuu al.
2021
Du al. 20021
Other Learning Agents (learned recipient sets)
Foerster et al.
(2016a):
Sukhbaatar et al.
[2016); Peng et al.
[2017 Das et al.
(2018)
(learned communcatee sets)
Kiline Montana (2018): Singh al.
2018
Kim et al.
(2019a by;
Zhang et al.
2019
Freed et al.
2020a Ding et al. (2020): Kim
et al. 2020
Hu et al. (2020a] Zhang et al.
2020
Representative Agents (centralized routing of recipi- ents)
Kong al.
2017 Gupta et al.
2023
Jiang
& Lu
20188
Pesce & Montanal
2020a
Wang
al.
(2220b):
[iu et al. (2020):
Niu et al.
(centralized message process-
2021
Mao al. (020
ing)
Message Integration and Recipient- Aware Weighting (Sec. 5.3.3
Concatenation
Foerster et al.
2016a): Hu et al. (2020a) Freed
(scalar/shor) vectors) (no recipient differentiation)
al.
(2000a)
Kiline & Montana (2018
Kim
et al.
2019a
Equal Weighting (averaging/summnil (treating all senders equally relevant)
Sukhbaatar Zhang et al. (0019] Du el al. (0021)
(0016) Singh et al. 2018
Weighted Prioritization (manual or attention-based) (recipient-dependent message importance)
Manual pruning: Kim al.
2019b Zhang
al. (2020)
Attention: Das et al. (2018):
Agarwal al.
2019b)
Malysheva et al.
2018
Neural Integration Models
MT.Ps:
Wang et al. (200b))
Liu et al.
(MLPs, CNNs, RNNs, GNNs) (learning sender-recipient rel-
2020
Gupta et al.
(002)
Mao et al.
2020
evance patterns)
CNNs: Jiang et al.
2018
RNNs
LSTMs:
Jiang & Lu
2018
D Ding
et al.
2020); Chu al. (2020)
Peng et al.
(2017 7 Pesce & Montana
2020a
Yun
et al.
(002))
GNNs: Niu et al. 2021);
Liu al.
2020
Sheng et al. (0022)
Table 3 presents a taxonomy of representative approaches to the problem of 'what to communicate" in MARL-Come Comm. organizes key strategies into three overarching categories: relevance filtering, targeted communication, and message integration. Within each category, we highlight how agents address information selection and compression by focusing different communication design choices. For relevance filtering,
25

---

Published in Transactions on Machine Learning Research (02/2026)
methods grouped by whether they encode past knowledge (e.g., histories or observations) future knowledge (e.g., intentions or plans), and whether communication occurs via representative agents or in a peer-to-peer manner. Targeted communication is further classified by proximity-based strategies (static vs. dynamic neighborhoodss) interaction with other learning agents, and centralized communication through representative agents. Finally, message integration methods are categorized by how agents combine received messages: concatenation, equal weighting, weighted prioritization (manual or attention-based), and neural architectures such as MLPs, CNNs, RNNs, and GNNs. Across all categories, the choice of what information to communicate is closely tied to who that information is sent to, since message relevance, filtering, and weighting depend on the intended recipients. This connection illustrates how communication content and communication partners must be considered together when designing scalable and effective MARL-Comm systems. This structured overview reveals trends in how communication is selectively managed to balance informativeness, efficiency, and scalability in multi-agent systems.
5.4 When to Communicate: Timing and Adaptation under Constraints
A fundamental challenge in MARI-Comm is determining when agents should exchange information. While continuous communication keeps agents fully informed, it can be costly, inefficient, or even counterproductive in decentralized environments with limited bandwidth or computational resources. Thus, effective commu- nication timing is critical for balancing the benefits of coordination with practical constraints. Instead of communicating at every time step, agents must learn to identify key moments when communication provides meaningful advantage, enabling better decision-making and collaboration under resource constraints. Beyond general bandwidth ronstraints, sharing a comumunication tion medium introduces additional complexity, particu- larly when multiple agents must. avoid message collisions. While protocols like Wi-Fi and LTT. manage this at the packet level through well-established scheduling mechanisms, such low-level access control is typically handled separately from decision-making in MARL systems. Instead, the key challenge in MARL-Comm is higher-level: determining when to communicate so that information is shared efficiently and meaningfully without overwhelming the channel. This involves two main concerns: (1) selecting concise, resk-relevant messages, and (2) coordinating communication timing to minimize redundancy and interference Kim et al.. (2019a). Rather than optimizing communication protocols directly, recent MARL approaches address these issues by integrating message selection and scheduling into agents' learning policies.
5.4.1 Learning When to Communicate under Constraints
Certain constraints, such as discrete decisions about whether or when to communicate, can make com- munication timing doeedifferentiable, posing challenges for gradient- based learning algorithms. This non- differentiability hinders end-to-end training of communication protocols. One common workaround is cen- tralized training with decentralized execution (CTDE), where agents are trained in centralized manner with full access to global information and differentiable approximations of discrete choices (e.g., via soft gating or policy gradients). This allows agents to learn effective communication strategies during training, even if message transmission is limited or constrained during execution. At test time, agents deploy these learned strategies under stricter, often non-differentiabl,, conditions. This section reviews methods that ad- dress communication constraints by learning when and how to communicate, including approaches based on CTDE, differentiable gating mechanisms, public belief modeling, and distributed communication scheduling.
Learning to communicate through backpropagation Sukhbaatar et al. Sukhbaatar et al. (2016) introduced 'Commuiccaion Network' (CommNet) that processes partial observations from multiple agents to determine their actions. Each layer in this multi-layered network has several cells, where each cell takes as input its own previous hidden state h and aggregated messages en, from other agents' hidden states. The updated hidden state is computed as hit1 aC H!hh))) where o non-linear activation function. The final layer outputs the agent's action, and the entire network is trained using backpropagation based on a reward signal. This approach outperformed baseline models where communication within the network occurred through discrete symbols. Subsequent research shifted toward centralized training with decentralized execution, as demonstrated by Deep Recurrent Q-Networks (DRQN) Hausknecht & Stone (2015). Unlike independent Deep NeNttworks (DQN) van Hasselt et al. (2015). DRQN does not assume full observability and is better suited for partially observable multi-agent environments. By maintaining internal memory
26

---

Published in Transactions on Machine Learning Research (02/2026)
through recurrent units, DRQN helps mitigate issues of non-stationarity arising from concurrent policy updates across agents, making it more robust in decentralized multi-agent settings.
Foerster et al. Foerster et al. (2016b) extended this by proposing the two following centralized communication schemee where agents share network weights and learn to communicale effectively in partially observable environments with shared rewards.
Deep Distributed Recurrent Q-Network Foerster et al. (2016b): The Deep Distributed Recurrent Q-Network (DDRQN) extends Deep Recurrent Q-Networks (DRQN) by adapting them to multi-agent settings through modifications to the independent Q-learning paradigm. Specifically, DDRQN introduces three key modifications to standard independent, Q-learning: (1) experience replay is disabled, as concurrently learning agents invalidate each other's past experiences; (2) each agent's previous action is appended to the current input to capture temporal dependencies; and (3) a single Q-network is shared across all agents, rather than maintaining separate nelworks. This weight sharing improves learning efficiency and reduces computational overhead.
Despite using a shared network, each agent receives individualized imputs inppts-inchuueeee including its own obser- vation, previous action, and agent cowtffier-allwwing the shared Q-network to specialize behavior for each agent. The Q-function learned by the Recurrent Neural Network (RNN) is defined as:
Qio]", 1h-11 m, a2-1, :0);
(1)
where of and al" are the observation and current action of agent m at time t, of is its previous action, h is the hidden state, and denotes the shared network parameters.
Training proceeds in two stages: (1) agents interact with the environment using an e-greedy policy based on the current Q-function, and (2) the shared network is updated using the Bellman equation, optimizing temporal-difference error over episodes.
Diffcrentiable Inter-Agent Learning (DIAL) Focrster et al. (2016a): DIAL introduces a framework where, at each time step, agent m outputs not only an enviromment action at but also a communication action ci" During ralized training, communication is unrestricted and differentiable, enabling gradient flow across agents; during decentralized execution, messages are constrained to bow-bandwitth discrete channels. Unlike traditional Q-networks that only estimate value functions, DIAL's Q-uetwork additionally outputs real-valued messages of', which are sent time + 1 to other agents' Q-networks (all agents share weights).
Two types of gradients flow through the network: (1) the reward gradient from the agent's own Q- learning loss and (2) an error gradient from message recipients, which propagates back through the communication channel. This allows the sending agent to directly adjust ils messages to minimize downstream DQN loss, accelerating the emergence of effective communication.
Experimental results show that both DIAL and DDRQN eventually converge to similar perfor- 100% of Oracle rewards for = 3 agents, and 90% for n 4. However, DIAL converges substantially faster, requiring only 20,000 episodes compared to DDRQN's 500,000. For n = 4, DIAL maintains reliable convergence, while DDRQN may fail to converge aero The perfor- mance gains are attributed DIAL's differentiable message passing, which enables more direct and efficient coordination learning during training.
Learning to communicate through parameter sharing Besides the differentiable inter-agent learning (DIAL.) proposed in Foerster et al. (2016a), the authors of this paper also proposed another communication approach based on centralized learning decentralized execution using deep Q-learning with a recurrent net- work to address partial obscrvability, i.c., reinforced inter-agent lcarning (RIAL). RIAL has two variants, using either independent or centralized Q-learning. In independent Q-learning, each agent learns its own network, treating others as part of the environment. Another variant trains a single shared network for all agents. During execution, agents act independently based on their unique observations. RIAL combines DRQN with independent Q-learning to learn both environmental and communication actions. Each agent's is Q-network, Q", takes as input the hidden state hg, observation of, and the messages mi_1 received from
27

---

Published in Transactions on Machine Learning Research (02/2026)
other agents at the previous time step. To manage the potentially large action space, RIAL factorizes Q" into two separate output heads: Qu for environmental actions uf E U and Qt for communication actions mi E M, where U and M denote the respective action spaces. An c-greedy policy is used to select both types of actions, requiring the Q-network to produce |U]+ + |M] outputs per time step. Both QN and Qm are trained using DQN, with two key modifications: disabling experience replay to address non-stationarity, and feeding the actions ue and my as imputs at the next time step to account for partial observability. Even though the agents learn independently, the decentralized execution is identical to the training phase.
Foerser et al. (2016a) assumes Learning to Communicate through Public Belief The DIAL method Foerster
the presence of an idcalized communication channel where agents can freely exchange information without affecting the environment. However, in many practical scenarios, explicit communication is unavailable or constrained. To address this, Foerster et al. Foerster et al. (2019) introduced the concept of public belief. which captures a shared probabilistic estimate over latent state features, based on common observations. At time step i, a latent fcature fa is inferred, with its publicly observable part denoted as f!ubb The public belief updated as:
B, = = P(fofft fgub. fPub).
Building this, they proposed the Bayesian Action Decoder (BAD), method that uses public belief to enable coordination without direct communcatiion. Each agent selects actions based on its private ob- servation and a shared virtual policy म., computed from the public belief and known policies of all agents. This virtual policy maps private observations to likely actions, enabling agents to infer others' private states from their observable actions. Specifically, after observing an action at" an agent updates its belief about another's s private state ft" using:
P(SE"la"" BL, S! #2) I -f P(S S("hh
This approach was evaluated on the cooperative card game Hanabi, where players cannot see their own cards and must rely on hints and inference. BAD achieved state-of-the-art performance in two-player settings, with agents learning implicit conventions (e.g., using a hint like "red" signal which card to play) that facilitated non-verbal coordination. The success of BAD demonstrates how shared beliefs can support effective communication even in the absence of explicit channels.
Scheduling Communication in a Distributed Manner A common approach to enhancing coordina- tion in multi-agent systems is by enabling distributed communcation, where agents exchange information to act cohesively without relying paradigm Lowe et al.] (2017), which is also adopted by many of the previously discussed methods in this section. SchedNet Kim et al. (2019a) exemplifies this approach by using reinforcement learning to learn a scheduling policy that determines which agents should communicate at each time step. This dynamic schodul- ing is particularly important practical scenarios where communication occurs over a shared medium such as a wireless frequency channel introducing two key constraints: limited bandwidth and contention for medium access. SchedNet addresses these constraints by jointly learning (1) message encoder, (2) a policy for scheduling which agents can broadcast, and (3) an action selector that uses both local observations and received messages to make decisions.
centralized controller. This is often achieved through the CTDE
5.4.2 Timing of Message Integration
Message integration plays critical role in determining not only how, but also when communicated informa- tion affects an agent's behavior during both learning and execution. By message integration, we refer to the process of incorporating received communication into an agent's internal computation, such as its policy or value function, during a training round or decision cycle. In most MARL-Comm methods, communication is treated as additional input and incorporated into either the policy network, the value network, or both. The specific timing of this integration integration-whetherr whether messages influence immediate action selection, delayed value up- dates, or both-affects how quickly agents adapt to new information and coordinate their behavior. Poorly timed integration may delay responses to critical cues or cause misaligned coordination. To examine these effects, we categorize recent approachess into three strategies: policy-level integration, valuo-level integration, and joint policy-and-value integration Zhu et al. (2022)
28

---

Published in Transactions on Machine Learning Research (02/2026)
Policy-Level Communication Messages Integration Incorporating communication messages into the policy model directly influences when agents adjust their decisions based on received information. By con- ditioning their next action on the latest messages from other agents, policies can dynamically adapt to evolving environments and coordination needs in real time. Typically, messages are concatenated with an agent's local observations or hidden states and fed into the policy network. This integration allows the agent to immediately update its behavior at every decision step based on the most recent communicated information, rather than relying solely on its own partial observations. In policy gradient methods, such as REINFORCE Sukhbaatar et al. (2016); Liu et al. (2020); ; Singh et al. (2018); Kong et al. (2017), the timing of communication affects the entire trajectory distribution: agents collect rewards over episodes while continuously adjusting their policies based on ongoing message exchanges. Communication influences the probability of action sequences during learning, making timely message integration critical for effective gra- dient estimation and coordination. Actor-critic approaches Du et al. (2021): Wang et al. (2020b); Jiann & Lu 20188; Kim et al. (2019a); Ding et al. (2020); Chu et al. (2020); Kim et al. (2020); Pesce & Montana (2020a); ); Freed et al. (2020a); Yun et al. (2021) ) Gupta et al. (2023) further emphasize timing by using critics that evaluate future returns conditioned on both local observations and communication messages. Here,
agents can adapt their action choices at. each step based on newly received messages, optimizing not only for immediate rewards but also for longer-term cooperation based on shared information. Thus, in policy-level integration, commmnication timing determines when new information influences the agent's 's decision-making pipeline, enabling agents to react to dynamic changes in their teammates' states and strategies throughout the learning and execution process.
Value-Level Communication Messages Integration In value-level integration, communication mes- sages are incorporated into the value function (or action-value function), directly influencing how agents evaluate the long-term utility of their actions. By conditioning value estimates on both local obscrvations and received messages, agents can better assess expected returns under partial observability and dynamic conditions, Many works following DQN-style frameworks adopt this strategy Foerster et al. (2016a): Jiang ct al. 2018): ; Zhang et al.] (2019); Agarwal et al. (2019b); Sheng et al. (2022): Kim ct al. (2019b). Typically, messages are concatenated with local observations or embedded as additional features for the value network.
This allows incoming communication ion to immediately affect an agent's Q-value estimates, which in turn guide future action selection via e-greedy or similar policies. Unlike policy-level integration, where messages di- roctly shape the action distribution, valuo-level integration influences behavior indiroctly, through changes in the estimated returns of different actions. In both cases, the timing of message reception malters. However, in value-level integration, updates often occur after reward feedback is received, introducing temporal delay in how communicated information affects future decisions. This lag can influence when agents benefit from new information, particularly in enviromments with fast-changing dynamics. Incorporating communication into the value function allows agents to reason about the long- consequences of actions informed by peer input, thereby enhancing coordination in soquential decision-making tasks
Combined Policy and Value Communication Messages Integration Tn combined integration ap- proaches, communication messages are incorporated into both the policy and value models, enabling agents to adjust not only their immediate action selection but also their evaluation of future outcomes based on received information. This dual integration enhances the agent's ability to dynamically adapt its decisions when new messages are received, influencing both short-term reactions and long-term planning. This ap- proach is often based on actor-critic methods, where messages are taken as additional inputs to both the actor and the critic. Some works feed the messages separately into the policy and value networks Agarwal et al. (201b)); Peng et al.] (2017), allowing independent, processing of communication information in action generation and value estimation. Others first combine messages with local observations to form a shared internal representation, which is then jointly used by both the actor and critic models Niu et al. (2021)
Das et al. (2018); Liu et al.) (2020): Freed et al. (2020a). By integrating communication into multiple stages of decision-making, agents can react promptly to new information at the policy level while simultaneously refining their value predictions. The timing of communication thus has a compounded effect: when mes- sages arrive at critical decision points, they can immediately inffuence an agent's behavior, either by directly altering action probabilities (in policy-level integration) or by updating value estimates that drive action
29

---

Published in Transactions on Machine Learning Research (02/2026)
selection (in value-level integration, e.g., via Q-learning). At the same time, these updates shift agents' expectations of long-term rewards, enhancing coordination over both immediate and future timescales.
Category
Approach
Representative Works
Learning
Backpropangation-basse Communica-
Sukhbaatar et al. 2016 Foerster et al.
Communication Timing
tion
(2016a
(Sec. 5.4.1
Parameter Sharing (RIAL/DRQN)
Foerster et (0066a
Public Belief Modeling
Foerster et
2016a
2019
Distributed Scheduling (MAC)
Lowe et al.
(017)) Kim al. (2019a)
Inner Integration Timing (Sec. 5.4.2
Polecy-Level Integration
REINFORCE Sukhbaatar et al.
2016
Liu et al.
2020
Singh et al. 2018
Kong al.
7
Wang approaches Du et al.
Actor-critic
2021
(2020b Jiang & Lu
2018
Kim et al.
[2019a] Ding et al.
2020
Chu
[2020] Kim al. [2020
Pesce
& Montana (2020a]: | Freed et al.
[2020a)
Yun al. (2021); Gupta al.
(2023)
Value-Level Integration
DQN-like
frameworks Foerster et al.
(2016a) ; Jiang et al. (2018): Zhang al.
2019
Agarwal et al.
(2019b] Sheng 2019b
et al. (2022); Kim al.
Combined Policy-Value Integration
Feeding the messages separately into the
policy and value networks Agarwal al.
(201bb); Peng et al. [2017]
Combining messages with local observa- tions to form a shared internal repre- sentation, which is then jointly used by both the actor and critic models Niu al. 2202]; Das al. (2018): Liu al.
{2020 Freed et al. (020a))
Table 4: Taxonomy of Approaches to 'When to Communccate' in MARL-Comm
Table summarizes key approaches determining when agents should communicate multi-agent rein- forcement learning systems. The first part of the table highlights learning- based strategies for adapting communication under bandwidth resource constraints, including backpropagation-enablel frameworks like CommNet and DIAL, centralized training with shared parameters, public belicf modeling for environ- ments without explicit channels, and distributed scheduling using reinforcement learning. The second part focuses on the timing message integration within an agent's learning model, either at the policy level (directly influencing action selection), at the value level (shaping return estimates) or both. Together, these approaches aim to balance timely and efficient communication with coordination demands, especially in dynamic, partially observable, bandwidth-constrainee settings.
5.5
Why and How to Communicate: Cirpose-Driven Protocol Design
After identifying what to communicate, who should communicate, whom commincatee to, and when munication should occur, the remaining question is how agents should structure and exchange information. The choice of protocol is not only a matter implementation. Tt reveals the underlying purpose of com- munication and the specific advantages agents seek to obtain. In practice, how agents communicate reflects why they communicate in the first place. For example, agents may communicate reduce uncertainty, coordinate joint actions, to stabilize learning under partial observability, and each of these goals motivates
30

---

Published in Transactions on Machine Learning Research (02/2026)
different protocol design. For this reason, we discuss how and why together in this subsection and treat protocol design as a purpose-driven driven process that links communication structure with communication intent. Effective protocols must support coordination, preserve interpretability, and remain scalable as the number of agents increases. They must also determine whether information exchange should rely on centralized support or be achieved through fully decentralized interactions, and whether messages should use explicit human-interpretable symbols or implicit latent representations optimized for learning.
5.5.1 Training Frameworks for Communication Learning
The training framework in an MARI-Comm system defines how agents use the collected experiences, includ- ing observations, actions, rewards, and messages, to improve performance. It also implicitly determines why communication is needed and how communication mechanisms should be designed, since different frame- works create different informational needs and coordination requirements. Different frameworks specify how this information is processed and shared during learning, shaping both the motivation for communication and the mechanisms through which agents exchange information.
In decentralized learning, each agent is trained independently using only its respective experience. Be- cause no centralized information is available, communication is motivated by the need to reduce uncertainty caused by partial observability and non-stationarityy Agents communicate in order to stabilize learning and anticipate the behaviors of others. This approach allows agents to learn their individual policies based solely local observations and rewards. However, decentralized learning faces significant challenges in multi- agent scttings, primarily due to the non-stationary environment introduced by constantly adapting agents. As agents evolve, the enviromment each agent perceives is continuously changing, complicating the learning process.
In centralized Icarning, all agents are trained jointly using the combined experience of every agent. Here, communication during execution is not needed to supply missing global context, because global information is already used to construct joint policies. Instead, the reason for communication shifts toward enabling the central learner to shape coordinated local behaviors. This approach transforms the multi-agent system into a stationary environment where global information can be utilized to guide the training process. However, centralized learning suffers from the curse of dimensionality, as the joint policy space grows exponentially with the number of agents, making it computationally expensive and difficult search.
To address the drawbacks of purely decentralized or centralized learning, the Centralized Training with Decentralized Execution (CTDE) paradigm Foerster et al. (2016a); Kraemer & Banerjee (2016) has become the dominant training framework for MARL-Comm. In CTDE, communication serves two distinct purposes. During training, communication is shaped by centralized information that guides the learning of local policies. During execution, communication helps agents reconstruct or approximate the centralized information that is no longer available. This dual purpose leads to diverse communication mechanisms, including learned message encoders, attention-based routing, and critic-informed message usage. In CTDE, agents learn their local policies with the benefit of centralized information during training, such as access to global observations or the actions of other agents. However, during execution, agents operate autonomously, using only their local information. This approach balances the benefits of centralized training with the flex- ibility of decentralized execution, allowing agents to learn more robust policies while avoiding the challenges of joint policy search during execution.
In addition, Parameter Sharing schemes Foerster et al. (2016a) have emerged as a solution to further reduce the complexity of multi-agent systems. Here, the motivation for communication is diffcrent: because all agents share the same model, communication becomes a means for a shared policy to differentiale agent roles or contextual behaviors. This motivation influences the design of communication mechanisms that help distinguish agents that otherwise follow identical network paramcters. In these methods, agents share common set of parameters (such neural network weights), effectively training single model that can control all agents. While this reduces the computational overhead of training multiple policies, it may limit the diversity of agent behaviors, as all agents are governed by the same model.
Lastly, Concurrent Learning addresses situations where agents must simultancously learn to interact with each other while managing their own policies. Methods like MD-MADDPG Pesce & Montana (2020a) and
31

---

Published in Transactions on Machine Learning Research (02/2026)
IS Kim et al. 20200] allow agents to train in tandem, coordinating their learning processes while maintaining individuality in decision-making, Here, communication is motivated by the need for agents to track each other's evolving behaviors without a shared replay buffer or central controller, which leads to mechanisms that propagate up-to-date observations, actions, or intentions among agents.
Before examining each training framework in detail, important to clarify how they jointly determine both why agents communicate and how communication mochanisms ultimately designed. The training setup shapes what information is available, what information becomes unavailable at exccution, and what gaps agents must fill through communication. Centralized learning reduces the need for communication during training but creates a pressure for communication at execution time to approximate the global context that is no longer accessible. Decentralized learning creates the opposite pressure: agents begin with only local views, communication becomes necessary stabilize learning and reduce uncertainty. CTDE blends these motivations by using centralized information to train policies while requiring agents to reconstruct or approximate that information during exccution. Across these frameworks, the underlying reason why communication is needed varies, and this difference naturally produces different communication mechanisms. When communication is required to overconee partial observability, agents exchange local signals. When it is required to propagate global structure learned during centralized training, communication becomes critic- shaped implicit. When it is roquired to diversify behaviors under shared paramcters, communication emphasizes identity and role dilferentiation. In the following content, we examine each training framework in depth, compare their assumptions and communication implications, and highlight how their distinct motivations lead to diffcrent choices for how agents communicate.
Environment
Environment
(01. .2,...,. an)
(21,02..... 0n)
01
Qg
Policy
Policy 2
Policy n
Policy
Policy 2
Policy m
Update with (Pa.7as...,
Update with Update with 72
Update with r.
(a) Centralized Learning
(b) Decentralized Learning
Figure 4: Centralized and Decentralized Learning. In contralized learning (a), policics are jointly optimized with shared knowledge and gradient flow. In decentralized learning (b), each agent operates and learns independently using local observations and rewards.
Centralized Learning In the centralized learning paradigm, the experiences of all agents, including their obscrvations, actions, rewards, and messages, are aggregated by central unit that Iearns to optimize the collective behavior of the system. As rllustrated in Fig. 4a this centralized controller leverages a global view of the environment to coordinate agents and compute joint policies or value functions. By having access to system-wide information, the learning process can exploit dependencics and correlations that may be inaccessible in decentralized settings. This global access also shapes why communication is needed at execution. Because the central learner integrates information from all agents during training, individual agents do not learn to infer missing global context on their own. Once the central unit is removed at execution, communication bocomes a mechanism for agents to reconstruct the global structure they relied on during training. In this way, centralized learning implicitly motivales communication as a tool to approximate training-tine information using decentralized signals.
Historically, carly centralized-learning communication works such CommNet Sukhbaatar al. (0066) provided a single continuons communication channel trained through centralized backpropagation. This addressed why to communicate by assuming that all agents benefit from sharing a unified global feature space, and it addressed how to communicate by averaging hidden states across agents. BiCNet Peng et al. (2017) improved this by introducing a recurrent structure across agents, motivated by the need propagate richer dependencies through the centralized model. The communication mechanism therefore
32

---

Published in Transactions on Machine Learning Research (02/2026)
became sequential and contextual, reflecting the centralized model's need to capture ordered interactions among agents.
DIAL Focrster et al. 20166) strengthened the link between why and how by making the communication channel diflerentiabl. Because centralized learning used gradient flow to shape the content of messages, DIAL justified why communication needed as a way to convey representations that directly support joint optimization. Its communication mechanism then became real-valued messages shaped by gradient signals. IC3Net Singh et al. (2018) added gating module to learn when commuication is helpful. This responded to the observalion that centralized (raining revealed mixed incentives in some tasks, which motivated the need for selective communication as part of the how.
The next generation of cnttralized-learming works used attention. TarMAC Das et al. (2018) showed that centralized gradients could train attention weights to choose informative senders. This created strong con- nection wetween why and how: the reason to communicate became the need to capture targeted relevance under global supervision, and the communication architecture became a learned attention mechanism. Sim- ilarly, graph-based methods such as ATOC Jiang et al. (0188) used centralized feedback to discover which neighborhoods of agents should interact. This motivated communication as a way to share local group- specific context, while the communication protocol became graph attention guided by centralized training. MAGIC Niu et al.] (2021) contimued this trend by learning dynamic graph structures under centralized gra- dients, again tying the motivation for communication to the need to recover global information using local graph-based message passing. Finally, representative-agent models such as HAMMER Gupta et al. (2023) used a central module to aggregate information and broadcast processed guidance, making communication necessary to distribute the centrally computed strategy. The communication mechanism therefore became anntraaliied-t--deeentrallniii
broadcast, path.
In practice, centralized learning is typically confined to the training phase. Most recent works assunne that, during execution, agents operate independently without access to a central controller. This decoupling, commonly referred to as CTDE, reflects the fact that maintaining a centralized controller at runtime is often infcasible in real-world deployments, particularly in dynamic, distributed, or bandwiidth-constrained enviromments. As such, centralized learning serves as practical way to facilitate coordination during training while preserving scalability and autonomy during execution.
Because centralized learning does not require agents to exchange explicit messages during training, the com- munication that emerges afterward is often implicit, critic-shaped, or encoded in latent vectors rather than explicit symbolic messages. This explains why centralliedd-eerriing methods frequently produce commu- nication protocols that rely on hidden representations, shared critics, or centralized guidance rather than explicit message structures. The distinction between why communication is needed and how it is im- plemented becomes clearer: the training process reveals what global information agents must reconstruct, and the communication architecture becomes the tool that allows them to recover this information without a central controller.
Decentralized Learning As Illustrated in Fig. 4b decentralized learning frameworks enable each agent to collect experience and optimize its policy independently, using only locally available information. Agents aaariion-accioo--vard----eee
learn from their own troller or shared global knowledge. This paradigm is suitable for scenarios where communication is con- strained, delayed, or unreliable, and where agents must operate under partial observability or localized perspectives. In decentralized settings, the reason why communication emerges is directly tied to agents lacking access to the information that other agents obscrve. Agents must therefore create message exchanges that compensate for missing state information. The structure of these messages reflects how decint systems attempt share only what is necessary support coordination.
trajectorics, without reliance on a centralized con-
Several recent approaches adopt this decentralized formulation, including MAGNet-SA-GS-MM Malysheva et al. (2018), Egent-Entity Graph Agarwal et al. (2019b). DCC-MD Kim et al. (2019b), NeurComm Chu et al. 22000) Intention Propagation (IP)Qu Qu et al. (2020), and Diff Discrete Frere et al. (020aa). These works share a common challenge: each agent must infer missing information from peers without a centralized learner, which clarifies why explicit communicatin bocomes critical and how different solutions attempt to encode that needed information.
33

---

Published in Transactions on Machine Learning Research (02/2026)
Environment
Environment
41
01
02
02
On
On
01
01
02
02
an On
Policy
Policy 2
Policy Y
Policy 1
Policy 2
Policy n
Update
Update
Update
Update
Central Unit
Central Unit
Global Info.
Global Info. e,g, rewards
e.g., rewards
(a) Individual Parameters
(b) Parameter Sharing
Environment
Policy
Policy 2
Policy
Global Info. (e.g., rewards)
Update
Update
Update
Central Unit
Central Unit
Central Unit n
(c) Concurrent
Figure 5: Three types of CTDE Scheme.
MAGNet-SA-GSS-MG Malyshova et al. (2018) employs a modular attention mechanism to choose relevant senders and receivers. The challenge lies in determining which information should be exchanged and which should be ignored, clarifying why communication must be selective rather than uniform. The architecture specifies how this selectivity is implemented through multi-scale attention over spatially varying contexts. This improves over earlier work by enabling dynamic sender-receiver assignment rather than fixed commu- nication channels.
Agent-Entity Graph Agarwal et al. 2019b introduces a dynamic graph representation to handle interactions based on proximity. The challenge motivating communication that agents cannot jointly track entity relationships from partial views. The method clarifies why messages needed: agents must share ionalional cues that reveal structured dependencies. It also specifies how this is achieved: through graph- based message passing between agent-cntity nodes. Compared to carlier fully connected commmication schemes, this work improves scalability by allowing messages only along relevant edges rather than across all agent pairs.
DCC-MD Kim et al. (2019b) addresses the difficulty of scaling communication in large multi-agent popula- tions. The challenge here is bandwidth, which explains why only subset of messages should be transmitted. The method defines how pruning is performed through relevance-driven message dropping. Compared with MAGNet-SA-GS-MG.. DCC-MD focuses more directly on communication efficiency and scalability rather than relational or spatial structure.
NeurComm Chu et al. 2020) learns decentralized communication policies directly through neural encoders. Here the challenge that agents cannot predict future behavior of their teammates, which explains why communication must encode policy-relevant latent information. The method explains how agents exchange compact neural messages that approximate their internal policy state. Relative to Agent-Entity Graph, NeurComm improves expressiveness by learning latent intention cues rather than relying only on structural proximity.
Intention Propagation (IP) Qu al. (2020) targets the difficulty of anticipating others' immediate actions.
This makes clear why communication is used: agents need way to reduce uncertainty about teammates'
34

---

Published in Transactions on Machine Learning Research (02/2026)
next moves. The method defines how this is done by passing explicit intended actions across local neigh- borhoods. Compared with NeurComm, which uses latent embeddings, IP improves transparency and inter- pretability by exchanging explicit one-step action predictions.
Diff Discrete Freed et al. (2020a) aims to produce compact, discrete communication protocols that improve interpretability. The challenge that motivates communication is the need for discrete coordination signals that are robust under noise. This clarifies why soft continuous messages may be insufficient. The method shows how to generate discrete messages through a differentiable Gumbol-softmax relaxation. This approach improves over DCC-MD by producing structured, interpretable messages rather than only pruning continuous ones.
Together, these decentralized methods demonstrate that the challenges agents face under partial obsorvability shape both why communication is needed and how communication is designed. As the field evolves chrono- logically, methods progress from structural message passing (Agent-Fatity Graph) to latent policy-aware messaging (NeurComm), explicit intention sharing (IP), selective attention mechanisms (MAGNet-SA-GS- MG), scalable pruning techniques (DCC-MD), and finally discrete, interpretable protocols (Diff Discrete). Fach step refines prior approaches by addressing limitations in expressiveness, scalability, or interpretability.
Centralized Training with Decentralized Execution (CTDE) In the CTDE framework (Fig. 5 the experiences from all agents are made available for optimizing their individual policies. This design addresses a core challenge of multi-agent learning: agents must coordinate despite receiving only partial obscrvations during exccution. CTDE clarifics why communication becomes necessary at execution. During training, agents benefit. from global information through the centralized critic, but during execution this global structure disappears. Agents therefore must exchange messages to reconstruct or approximate the information they relied on during training. By using gradicnts computod from the combined experiences of all agents, CTDE guides each agent's 's local policy while still allowing decentralized execution in the environment. The key benefit is that CTDE exploits rich global information during training without requiring centralized control at runtime.
One critical component of CTDE is parometer sharing Foerster et al. (2016a), which improves data efficiency by using a single set of parameters (such as a shared Q-lunction or shared policy) across agents instead of maintaining separate models. Even with shared parameters, agents can still learn diverse behaviors because they encounter different observations and roles in the environment. Parameter sharing also informs how communication is structured. Since agents operate under identical models, communication often focuses on exchanging role-identifying information or contextual cues that help them break symmetry during execution.
Based on these mechanisms, recent CTDE-based works can be grouped into three main approaches:
(1). Individual Policy Parameters. In this category, each agent maintains its own policy parameters while central unit aggregates experiences from all agents to compute global guidance signals, shown
in Fig. 5a This family of methods typically uses policy gradient algorithms such as REINFORCE Kong
et al. 2017 or actor-critic variants Wang et al.] (2020b); ; Kim et al. (2019a); Mao et al. 2020);; Yun el al.
22021). The challenge motivating communication here is that agents must act under partial observability even though training relics on access to full system information. This explains why agents later need to communicate: they must approximate the influence of centralized gradients using messages. The training setup also shapes how communication emerges, often in the form of critic-shaped messages or latent signals that mimic the centralized feedback agents received during learning. Relative to decentralized learning approaches, these methods improve stability by grounding each local update in global training data, which reduces the non-stationarity caused by concurrently learning policies.
(2). Parameter Sharing. Here, single set of parameters is shared across all agent policies or critic networks, as illustrated in Fig. 5b] Parameter sharing improves data efficiency and can accolerate training. When DiNelike methods are used, agents may learn a unified Q-function from all experiences Foerster et al.) 2016a); Jiang et al. (2018); : Sheng et al. (2022), or a second global Q-function may be added to complement local updates Zhang et al. (2019 2020 Actor-critic methods instead train one shared actor using all observation-action samples while relying on a centralized critic to provide gradients Niu et al. (2021); Dul
et al. 2021 Das et al. (2018); Jiang & Lu (2018); Hu et al. (2020a); Mao et al. (2020) In some cases,
35

---

Published in Transactions on Machine Learning Research (02/2026)
REINFORCE-styl updates are still used, relying solely episode-level rewards Sukhbaatar et al. (0016)
Liu et al. 2020); Singh et al. (2018). Parameter sharing highlights why communication becomes essential: if all agents are governed by the same network, they must exchange contextual information to differentiate their
behavior. The approach also influences how communication is designed, often producing clentity-indicaainn messages or disabbiisaniintton signals that help agents specialize. Compared with individual parameter methods, parameter-sharing techniques improve sample efficiency and reduce computational cost, but require more deliberate communication to avoid homogenized behaviors.
(3). Concurrent Learning. In scenarios where storing all experiences in a centralized buffer is impractical. each agent maintains its own buffer while assuming access to the actions and observations other agents. These decentralizod backups are used train a centralized critic per agent, as shown in Fig. 5c Methods such Kim et al. (2020); Pesce & Montana (2020a) adopt this variant. The challenge arises when agents must learn concurrently but still use global information during training. This explains why communication is needed: agents require way to maintain coordination despite updating policies at different times. It also shapes how communication is exchanged, often through implicit synchromization signals or messages that refine the critic's predictive accuracy. Relative to parameter-sharing approaches, concurrent CTDE offers greater behavioral diversity and avoids the restrictions of shared policy class, while still maintaining the advantages of centralized critic guidance.
CTDE balances centralized training and decentralized execution, becoming a standard framework for multi- agent reinforcement learning. This flexibility is particularly valuable in environments where agents must coordinate effectively while preserving individual autonomy during execution. Across all CTDE variants, the underlying training architecture clarifies why communication is required and how communication protocols are shaped by the structure of the centralized information available during learning.
5.5.2 Communication Objectives and Control Shaping
Agent
Profstoe
Pray 3
Pray
Predator 2
Produtor
Tirget
Agent
Pray
Prey 2
Agent 2
Predator 3
Produtoe
(a) Cooperative
(b) Competitive
(c) Mixed
Figure 6: Communcation shaped by reward objectives: cooperative (a), competitive (b), and mixed (c).
The design of reward functions plays central role in shaping how agents communicate and coordinate in multi-agent systems. By tailoring these reward signals, agents can be incentivized to pursue specific inter- action patterns and group dynamics. Depending on how these rewards are structured, agent interactions tend to fall into one of three primary categories: cooperative, competitive, or a combination of both, referred soppentice-competitiee to as mixed or cooperative- competitive scenarios Ning & Xie (2024): Busoniu et al. (2008:: Matignon et al. (2012). Each paradigm requires distinct. communication strategiie to support the underlying objective. For instance, fully cooperative tasks encourage agents to share informative messages that maximize joint utility, while competitive tasks may discourage communication or foster deception. A number of recent approaches validate their adaptability by testing across multiple behavioral regimes Liu et al. (2020): Das et al. (2018): Singh et al. (2018); Jiang et al. 2018); Sheng et al. (2022). These differences across reward structures clarify why communication is encouraged or suppressed each setting and how agents adapt their messaging pat- terns to suit the underlying incentive landscape. In cooperative settings (Fig. (6)). agents are encouraged to work hegether to maximize a shared objective. A common approach is to assign the same reward to all
36

---

Published in Transactions on Machine Learning Research (02/2026)
agents, i.e., 71 = 12 YN, that collective performance is prioritized over individual gain. This equal reward distribution incentivizes agents to support one another and avoid failures that could harm the group's overall outcome, thereby fostering stronger coordination. Under this incentive structure, commu- nication becomes useful because agents benefit jointly from shared information, and coordination-orientee message passing naturally emerges as the most effective mechanism for improving collective outcomes. Com- petitive settings (Fig. (bb.. by contrast, emphasize individual success. Each agent seeks to maximize its own reward, potentially at the expense of others. In fully competitive tasks often modeled zero-sum games-the total reward is fixed, such that Σx= = O. Here, one agent's 's gain directly results from an- other's loss, and the objective becomes outperforming opponents rather than collaborating. This creates a very different motivation for communication: sharing information may disadvantage the sender, so agents often reduce, gate, or strategically time their messages. The resulting communication protocols therefore reflect adversarial incentives rather than cooperative ones. Mixed settings (Fig. (6c), also known general- sum games, incorporate both cooperative and competitive dynamics. In these environments, agents have partially aligned or independent goals, and rewards are neither fully shared nor strictly opposed. Rather than involving multiple tasks, agents in mixed settings may find themselves cooperating in some parts of a task while competing in others. For example, in autonomous driving, vehicles may cooperate to avoid collisions and maintain smooth traffic flow (shared safety objective), but compete for limited road resources such as merging lanes or parking spots. Similarly, in multi-player video games robot soccer, teams may exhibit internal cooperation while competing against opposing teams. These scenarios reflect many realistic multi- agent applications and require more nuanced communication strategies to manage the trade-offs between collaboration and competition. Because incentives shift over time, agents must learn why communication is beneficial in some contexts but risky in others, and how to adjust message content, timing, or selectivity accordingly. This leads to more complex, context-aware communication patterns than those found in purely cooperative competitive environments. In the following content, review representative methods from each category in depth and analyze how their communication strategies arise from the underlying reward structures.
Cooperative Scenarios In cooperative scenarios, the central challenge is that agents must align their behaviors to maximize shared objective, often under partial observability or limited local information. This structure ereates a clear reason for communication: agents benefit when they share information that reduces uncertainty and improves group coordination, which explains why communication is encouraged in these environments, A common formulation assigns a shared team reward, where all agents receive the same signal regardless of individual contributions Foerster et al. (2017)s Das et al. (2018); Waan et al. (2020bls Jiang & Lu
6201 SSkhbaataar et al. (2016); Peng et al. 20188: Kim et al. (2]19a);
Mao et al. (2020):
Ding et al. (2020): Agarwal et al. (2019b); Kim et al. (2020): ;: Zhang et al. (2019 2020 g Yun et al. 2021
Kiline& & Montana
(2018); Sheng et al. (2022); Freed et al. (020a);; Hu et al. (2020a); Gupta et al. (0023
This equal reward structure naturally promotes cooperation because agents succeed only when the team
succeeds.
An alternative design uses individualized rewards that still depend on the performance of others Liu et al. (2020 Singh et al. (2018): Peng et al. (2017); Jiang at al. (2018); Frred et al. (2020a): Kim et al. (2019b): Kong el al. (2017); ; Gupta al.] (2023). This does not force identical incentives, but agents remaii moti-
vated to support teammates because their own returns are tied to overall group performance. Many works
strengthen this coupling by adding penalty terms, such as collision costs or inefficiency penalties Niu et al.
(2021) 0: Das et al. (2018): Liu et al. (2020);; Jiang & Lu (2018) ; Hu ct al. (2020a); Kim et al. (2020); [Jang et al.
220nn):Jiang
(2018g Kim et al. (2019b), while othe use neighborhood hoodbbbbeoood-bbeeeeee reward sharing to promote local cooperation
among nearby agents Chu et al. (2020); Qu al. (2020). These reward designs highlight why communica-
tion becomes useful: agents need access to each other's observations, planned actions, intentions to avoid
penalties and improve joint outcomes.
Farly cooperative communication frameworks approached this need with simple differentiable messaging. DIAL Foerster et al. (2016a) and CommNet [Sukhbaatar et al. (2016) introduced end-to-end trainable com- munication protocols that allowed agents to exchange continuous messages. These methods were among the first. to demonstrate how commuication can directly improve collective performance by enabling shared
37

---

Published in Transactions on Machine Learning Research (02/2026)
latent representations. Their simplicity made them widely applicable, but message exchange dense and often unnecessary.
Later works improved communicatio efficiency by making message exchange selective. ATOC Jiang & Lu (2018) introduced an attention mechanism that lets agents decide when communication is useful. This innovation addressed the challenge of unnecessary communication and provided a more scalable solution. IC3Net (Sinhh et al. (2018) expanded on this by giving each agent communication gating function that learns to open or close communication channels based on individual returns, which offered a principled way to adapt messaging to diverse cooperative lasks.
Subsequent methods developed more structured communication to improve coordination in complex envi- ronments. GA-Comm Liu et al. (2020) aligned individual incentives with team outcomes through reward shaping, which reduced conflicts among agents and encouraged more meaningful messages. TarMAC Das et al. (2018) introduced targeted communication using multi-head attention, allowing agents to direct dif- ferent parts of their messages to different recipients. MAGIC Niu et al. (2021) built on these ideas with modular communicatin components and dynamic attention to handle dense multi-agent interactions more effectively.
Across these developments, the trend is clear. Early methods provided continuous messaging without struc- ture. Later approaches introduced selective, targeted, and modular mechanisms that better match the cooperative incentive structure. These improvements reflect how communication strategies evolve as the underlying cooperative challenge becomes more complex and as agents learn when and what to share to maximize joint performance.
Competitive Scenarios Competitive MARL presents a distinct challenge because agents pursue conflict- ing objectives. Each agent aims to maximize its own reward while possibly obstructing the progress of others. This creates instability in learning, as policies must adapt to dynamic opponents that continually change their behaviors. These adversarial incentives explain why communcatiin becomes delicate. Messages help an agent anticipate threats or opportunities, but they can also reveal private intentions that opponents might exploit. As a result, communication must be selective and strategically controlled to avoid harming individual performance.
Most benchmarks for competitive MARL, such as the StarCraft Multi-Agent Challenge (SMAC) Zhang et al. 2019 20 Samvelyan et al. (2019), primarily evaluate coordination within single team under adversarial pressure. They do not model communication across opposing leams, since sharing intentions with adversaries is rarely beneficial. This means that explicit competitive communication remains understudied, even though adversarial dynamics highlight the importance of selective information sharing.
IC3Net Singh et al. (0188) provides one of the first concrete examples of how controlled commmicatiin can emerge in competitive settings. Tt introduces learnable gating mechanism that allows each agent to decide when communication is advantageous, This mechanism helps agents avoid revealing unnecessary information, while still exchanging useful signals in moments that directly improve their own utility. IC3Net demonstrates that even relf-interested agents can benefit from conditional communication, such as sharing information just. before seizing a goal or evading pursuit. Compared with carlier cooperative-oriented methods, IC3Net improves robustness by allowing agents to suppress communication when it creates vulnerability.
MADDPG Lowe et al. (017)) addresses another fundamental challenge in competitive tasks: unstable learn- ing due to interdependent strategies. It adopts a CTDE paradigm with a centralized critic that observes the global state and all agent actions during training. This helps agents learn stable policies in environments where opponents' decisions strongly influence the optimal response. Although MADDPG does not impose an explicit communication channel, its training structure clarifies why communication would later become useful. Agents Iearn policies that depend on global information during training, and message passing execution time provides a decentralized method for reconstructing that missing context.
Differentiable Inter-Agent Learning (DIAL. Foerster et al. (2016a) offers a complementary perspective by enabling end-to-end learning of communication through differentiable channels. In competitive settings, this capability can yield private or encoded communication strategies that improve coordination within a team without leaking exploitable information. This reflects different notion of how communication can operate
38

---

Published in Transactions on Machine Learning Research (02/2026)
in adversarial environments: not merely sharing raw information, but transmitting abstract encrypted signals shaped by competitive pressure.
Further insights come from the work of Deka and Sycara Deka & Sycara (2021). who explore the emergence of diverse roles in compelitive leams. Their graph neural nelwork architecture and decentralized training process encourage agents to specialize, leading to heterogeneous strategies that counter varied opponent tac- tics. Specialization offers another path toward improved communication, because agents can tailor messages to complement distinct roles rather than sharing uniform signals.
These works collectively illustrate that communication in competitive MARL follows different principles from cooperative settings. It must be selective, strategically motivated, and often implicit. 1C3Net provides con- ditional sharing, MADDPG stabilizes adversarial learning and motivates communication during execution, DIAL explores optimized latent channels, and role-specialization methods highlight structured coordination under competition. Although competitive communication is still in its early stages, these foundational stud- ies reveal how adversarial pressures reshape communication needs and offer promising directions for future research, including deception-resisee resistance, privacy maintenance, and selective signaling.
Mixed Scenarios Mixed scenarios involve agents that pursue individual reward functions that may be partially aligned, entirely independent, or conflicting. These environments blend cooperative and compet- itive dynamics, requiring agents to adapt their strategies and communication behaviors based on context. This hybrid structure explains why communication becomes more nuanced than in purely cooperative competitive tasks. Agents must decide whether sharing information will produce mutual benefit or expose vulnerabilities, and they must recognize how communication affects both short-term and long-term incen- tives. Mixed settings can also be viewed as a generalization of competitive scenarios because they relax the zero-sum constraint, allowing cooperation and adversarial behavior to emerge simultaneously. The central challenge is determining when commuicattion is beneficial, what information should be shared, and how agents balance self-interest with opportunities for collaboration.
One of the foundational works for mixed scttings is MADDPG Lowe et al. (017), which introduces a CTDE paradigm. By allowing each agent to condition its policy on other agents' actions during Training via a centralized critic while preserving decentralized execution, MADDPG provides a robust framework for learning in cooperativ--oopcittiee environments. This structure clarifies how communication may later be used at execution time: agents rely on global information during training, and message passing becomes a way to approximate that structure once the centralized critic is removed.
RIAL and DIAL Foerster et al. (2016a) extend this perspective by supporting both private and shared communicatio channels, enabling agents to negotiate implicitly and adopt dynamic roles. Although initially designed for cooperative tasks, their communication flexibility helps explain how agents adapt signals to mixed-reward conditions, selectively sharing withholding information based on changing incentives.
Liu et al. Liu et al.] explore the emergence of team-oriented behaviors in competitive multi-agent soccer through decentralized population- training and reward shaping. Their results illustrate that agents can progress from self-interested strategies toward cooperative patterns without explicit communication, highlighting different mechanism for coordination that complements message-based approaches.
DGN Jiang et al. (0018) considers environments where agents choose between collecting food for positive rewards or attacking others for greater gain. Its graph-based communication mechanism enables agents to shift from competitive to cooperative behavior, demonstrating an improvement over carlier architectures by exploiting relational structure and proximily-based interactions. IC3Net (Singh et al. (2018) contributes another dimension by allowing agents to learn when to communicate, which is particularly useful in mixed tasks where signaling must be conditional on the reward context.
TarMAC Das et al. (2018) proposes targeted communication and evaluates it in Predator-Prey settings, showing that addressing specific receivers improves performance when cooperation and competition must coexist. 12C Ding et al. (2020) builds upon this by introducing incentive-based messaging that accounts for communication cost, encouraging agents to exchange information only when the expected benefit outweighs the overhead. MAGIC Niu et al. (2021) further expands scalability by combining modular communication
39

---

Published in Transactions on Machine Learning Research (02/2026)
with dynamic attention, helping agents focus information flow under shifting cooperative and competitive pressures.
Ryu et al. Ryu et al. 2021 | incorporate cooperative and competitive biases into training to improve adapt- ability, oflering a principled way to shape communication tendencies. Santos et al. [aantos el al. (2022) investigate dynamic communication availability at execution time, showing how agents can adjust fluctu- ations in bandwidth connectivity.
Together, these studies show that communication in mixed settings depends on reward design, environmental structure, and temporal context. They highlight how agents learn to determine when collaboration is beneficial, when independence is necessary, and how to navigate the uncertainty that arises when cooperation and competition coexist.
5.5.3 Why and How Communication Formats Matter
In multi agent reinforcement learning, communication formats determine how agents share information and resolve coordination challenges under partial observability. This makes it essential to understand why agents need to communicate and how the chosen format operationalizes that need. The central difficulty comes from decentralized execution, where agents hold only partial and often inconsistent local views of the environment. As a result, communication formats arise as different solutions to this problem. Explicit communication compensates for missing information through direct message exchange, while implicit com- munication compensates by allowing agents to interpret or infer hidden intentions from observed behavior. In the following, examine both formats and show how each method addresses a specific challenge, why munication is needed in that scenario, and how the architecture implements its solution. Cooperative multi agent reinforcement learning has proven effective for collaborative decision-male in diverse domains Canese et al. 2021 Oroojlooy & Hajinezhad (002)). Multi agent environments introduce additional challenges such as partial observability and non stationarity. To address these issues, many approaches adopt the centralized training with decentralized execution paradigm, where agents are trained with access to global information but must operate based only on local observations during execution Rashid et al. (2018) Wang et al. (020a): Yu et al. (2022a). This information gap explains why communication becomes necessary: agents lose access to global state at execution time and must reconstruct missing information through interaction. The how depends on whether the system uses explicit messages or behavioral cues.
Explicit Communication Protocols Explicit communication directly addresses the challenge of partial observability by enabling agents to transmit structured messages. The why is to supply missing information that cannot be inferred locally, and the how is realized through differentiable message passing, attention mechanisms, or selective routing modules. Fach method chooses its architecture according to the severity of partial observability and the coordination demands of the task. Explicit communication allows agents to exchange observations, latent features, or intentions. Although this enhances coordination, it increases bandwidth cost and may weaken decentralization. Below, we review representative methods and explain their challenge, why to communicate, how to communicate, and how each improves over earlier work.
DIAL Foerster et al. 2016a).. The challenge is learning meaningful messages that support coordination. The why that agents require information beyond their local view. The how is through differentiable communication channels that allow gradients to shape message content. This improves over earlier discrete protocols by enabling end to end learning of compact and task specific messages.
CommNet Sukhbaatar et al. 2016). The challenge is scalability when many agents communicate si- multancously. The why that agents must share aggregated context to align behavior. The how is through averaging hidden states into shared representation. This improves efficiency but. sacrifices expressiveness, motivating later advances.
ATOC Jiang & Lu 20188. The challenge is eliminating unnecessary communication that causes noise or congestion. The why is that communication should occur only when coordination difficulty is high. The how is through learned attention gate that identifies beneficial communication moments. This improves over CommNet by introducing selectivity and reducing redundant messaging.
40

---

Published in Transactions on Machine Learning Research (02/2026)
TarMAC Das et al. 2018). The challenge is routing messages to the correct recipients in heterogeneous teams. The why is that agents often need information from specific teammates rather than broadcast messages. The how is through multi head target addressing using attention. This improves over ATOC by making communication directional and role aware.
IC3Net Singh et al. (2018). The challenge is handling enviromments where communication may be harmful or risky. The why is that agents should share information only when it improves their own utility. The how is through a learnable commuiiattin gate that decides whether to speak or stay silent. This improves over TarMAC by enabling context dependent suppression of communication,, which is valuable in competitive noisy environments.
The evolution from CommNNt to ATOC to TarMAC to IC3Net shows increasing refinement in both why communication occurs and how messages are structured and routed. Early works use simple broadcast commuicattin,, while later works carefully regulate message timing, direction, and necessity.
Implicit Communication Protocols Implicit communication begins with a different challenge. In many enviromments, explicit message passing is limited by bandwidth, connectivity, or strategic concerns. The why for commuication is therefore to preserve coordination under communication constraints. The how is to encode information in observable actions or learned latent beliefs, allowing agents to infer intentions from behavior. Implicit communication avoids explicit message passing. Instead, agents learn to interpret movements, actions, or representation shaped cues as informative signals Oliehock et al. (0066). This reduces bandwidth cost but requires stronger inference capabilities. Below, we summarize main methods with their challenge, why; how, and improvement.
Action based implicit communication. Knepper et al. Knepper et al. (017)), Shaw et al. Shaw et al. (2022), and Tsiamis et al. Tsiamis et al. (2015). The challenge is coordinating when explicit messaging is unavailable. The why is the need to reveal intent through observable behavior. The how is choosing actions that encode information for teammates to interpret. These works improve over purely reactive behaviors by formalizing intention revealing actions.
Reprosentation based implicit communication. Tian et al. Tian et al. (000)) introduce belief modules and auxiliary rewards to help agents infer hidden information. The challenge is reconstructing others' states without explicit signals. The why is the demand for coordination in settings without communication channcls. The how is learning internal beliefs that summarize inferred information. This improves over action based approaches by providing more expressive internal models.
Grupen et al. Grupen et al. 2021 show that spatial cues act implicit signals in fully decentralized systems. Their improvement is removing reliance on centralized components.
Li et al. Liet al. (2023a) propose TACO, which begins with explicit commuication but gradually removes this information as agents learn to infer missing context. The challenge is transitioning from explicit to implicit communication. The why is to reduce reliance on global information while maintaining coordination quality. The how by training agents to approximate global information through local inference. This improves scalability and robustness in decentralized settings.
Together, these methods progress from simple behavioral cues to increasingly sophisticated latent inference, showing how communication can be embedded implicitly without dedicated channels.
Comparison and Discussion Explicit and implicit communication reflect different answers to the ques- tion of why communication is needed. Explicit communication compensates for severe partial observ- ability by exchanging structured messages. Implicit communication compensates for communication limits strategic risks by encoding signals in behavior learned beliefs. They also reflect different answers to how communication should occur. Explicit communication emphasizes message content, routing, and compression. Implicit communication emphasizes behavioral expressiveness and inference. Across both categories, improvements move toward greater selectivity, robustness, and decentralization.
41

---

Published in Transactions on Machine Learning Research (02/2026)
Table 5: Summary of Approaches to 'How to Communicate?" in MARL-Comm
Category
Sub-Category
Representative Approaches
Training Framework (Sec. 5.5.1
Centralized Learning
Foerster
(0166a
Kraemer & Ranerjee
2016
Decentralized Learning
Agent-Entity Graph Agarwal et al. (2019b),
NeurComm Chu et al. (2020). Intention Prop-
agation (IP) Qu et al. (2020), MAGNet-
SA-GS-MG Malysheva
al. 2018), DCC-
MD Kim et al. (2019b). Diff Discrete Freed
et al. (2020a).
Centralized Training with Decentralized
Individual Policy
Parameters:
REIN-
Execution
FORCE Kong et al.
(2017
Actor-critic-
(CTDE)
based Wang et al.
(202bb) Kim al.
(019aa): Mao et al.
(2020): Yun et al. 2021
Parameter Sharing:
local Q-function Fo-
erster et al. (2016a): Jiang
al. (2018); SSheng et al. (2022), global Q
function Zhang al. (2019 2020), shared
actor receiving gradient guidance from a centralized critic [Nu al. (2021): Du
let. al.
(2021); Das et al.
(2018): Jiang
& Lu
2018);
; Hu et al.
(2020a)s Mao
el al.
(0200) rewards sampled over entire
episodes
REINFORCE Sukhbaatar
el al.
(2016): Liu al. (2020) Singh et al.
20188
Concurrent Learning: centralized critic to
help guide the optimization of its local pol-
icy Kim et al. (2020) Pesce & Montana
(2020a)
Continued next page
42

---

Published in Transactions on Machine Learning Research (02/2026)
Table continued from previous page
Category
Approach
Representative Works
Communication
Cooperative Scenarios.
Shared team reward, all agents receive the
Shaped by Reward
signal regardless of individual contri-
butions Foerster et al. [2016a) Sukhbaatar
Objectives
et al.
(2016): PPng et al. (2017 Das al.
(Sec. 5.5.2
(2018
Wang et al.
(2020b); Jiagg & Lu
(2018
Kim et al.
(2019a)) Mao al.
(2020
Ding et al.]
(2020); Agarwal et al.
(2019b)
Kim et al. (2020); Zhang et al.
(0019
2020); Yun et al. (2021); Kiline &
Montana (2018) 3 Shhnng al. (2022) Freed
et al. (2020a); Hu et al. (2020a) Gupta
al.
(2023)
Individualized rewards that are still depen-
dent on the performance of other agents Liu
et al.
(2020); SSigh et al. (2018): Peng el al.
2017 Jiang et al.
[0018;
Freed el al.
2020a):
Kim et al.
(2019b) Kong et al.
2017
Gupta al.
(2023
Incorporating penal ties Niu et 2021
Das et al.
(2018): ; Liu et al. [2020]; Jiang &
Lu (2018); Hu et al.
(2020a):
Kim et al.
2020
Jiang al.
(2018): Kim al.
2019b)
Neighborhood-bbssel reward sharing
strengthen localized cooperation Chu et al.)
2020
Qu al. (2020): Sukhbaalar al.
2016 Jiang Lu (2018).
Competitive Scenarios
Learnable gating mechanism enabling
agents to selectively communicate:
IC3Net Singh al. 2018
Centralized critics during training: MAD-
DPG Lowe et al. (2017)
End-to-end learning of communication
messages through differentiable channels:
DIAL Foerster et al. (2016a)
Leveraging graph neural networks and de-
centralized training Deka & Sycara 2021
Continued on next page
43

---

Published in Transactions on Machine Learning Research (02/2020)
Table continued from previous page
Category
Approach
Representative Works
Mixed Scenarios
Centralized training with decentralized ex-
ecution paradigm: MADDPG Lowe et al.
22017
Learn both shared and private communica- tion channels, supporting implicit negotia- tion and role emergence in partially aligned settings: RIAL and DIAL Foerster et al. (2016a)
Decentralized population rapulatton-based training and reward shaping Liu et al. Jiang al. (2018)
Gating mechanism for communication:
IC3Net Singh et al. (2018)
Targeted communication architecture eval-
uated on mixed scemarios: TarMAC Das
al. (2018): Ding al. (2220)
Incorporate cooperative
and compelitive
binses into agent training Ryu al. 2021
Dynamic communication availability at ex-
ecution time
Santos et al.
2022
Message Format Objectives (Sec. 5.5.3)
Explicit communicatio
Singh al.
2018); Das al.
2018
h Lowe
et al. 2017
Foerster et al. 2016a
Chen
(2024b
Breazeal et (2005)
Implicit communication
Oliehoek et al.
(2016 Knepper et al.
2017
Shaw et al. 20022)
Tsiamis et al. (2015
Knep-
per al. (017): Tian al. (2020):
Grupen
et al. 2021
Table 5 provides a structured summary of the key approaches addressing the question of how to com- municate in MARL-Comm. The taxonomy organized into four major categories: training frameworks, reward-shaped communication strategies, communication message formats, and protocol designs. Each cat- egory is further subdivided into specific approaches, such centralized learning, decentralized learning, and the increasingly popular centralized training with decentralized execution. Representative works are pro- vided for each sub-category, highlighting influential algorithms and methods from the literature. The table captures both explicit and implicit communication protocols, as well as variations tailored for cooperative, competitive, and mixed-motive scenarios. This summary offers comprehensive reference for understanding how communication mechanisms are operationalized in MARL systems, balancing coordination, scalability, and interpretability.
5.6 Discussions
To provide a unified perspective on MARL-Comm, this subsection synthesizes communication methods using a structured lens: what information is exchanged, who speaks, whom it is directed to, when communication occurs, and finally how and why communication is carried out. While Table 6 retains its original three columns for clarity, our discussion expands these categorics into more complete framework that reflects recent conceptual developments in MARL-Comm.
Table illustrates the diversity of communication design choices. Early methods emphasize simple message content and uniform broadcasting, while recent works introduce selective roles, structured recipients, and more expressive encoding mechanisms. Below, we reinterpret these trends using the order of what, when, and how, with each part incorporating the related dimensions of who, whom, and why.
44

---

Published in Transactions on Machine Learning Research (02/2026)
Table 6: Representative MARI-Comm methods categorized by what they communicate, whom they com-
municale with, and how messages are encoded aggregated.
Method
What Comm.
Whom to Comm.
How Comm.
DIAL
Foerster al. 2016a
Past Knowledge (w/o rep)
Other Learning Agents
Concatenation
RIAL Foerster et al. 2016a
Past Knowledge (w/o rep)
Other Learning Agents
Concalenation
Comm
Sukhbaatar et
Past Knowledge (rep)
Other Learning Agents
Equal Weighting
(0016)
IC3Net Singh et al. 2018
Past Knowledge (rep)
Other Learning Agents
Equal Weighting
TarMAC Das et al 2018
Past Knowledge (rep)
Other Learning Agents
Weighted
MADDPG-M Kiline
Mon
Past Knowledge (w/o rep)
Other Learning Agents
Concatenation
tana
(2018)
ETCNet Hu al.
2020a
Past Knowledge (w/o rep)
Other Learning Agents Representative Agent Representative Agent
Concatenation
HAMMER Cupta et al. 2023
Past Knowledge (rep)
Weighted
GA-Comm
et 2020
Past Knowledge (rep)
Weighted
MAAIC el al. 2021
Past Knowledge (rep)
Representative Agent
Weighted
ATOC Jiang & Lu 2018
Future Knowledge
Representative Agent
RNN-based
FlowComm Du el a. [2021]
Past Knowledge (w/o rep)
Nearby Agents
Equal Weighting
GAXNet Yun et al. 2021
Past Knowledge (w/o rep)
Nearby Agents
RNN and Attention
DGN Jiang et al.
2018)
Past Knowledge (w/o rep)
Nearby Agents
CNN
Ding et al.
2020
Future Knowledge
Other Learning Agents
RNN
MD
MADDPC
Pesce & Mon-
Past Knowledge (rep)
Representative Agent
RNN
tana
2020a
Gated-ACMI.
Mao et al.
2020
Past Knowledge (rep)
Representative Agent
Gated NN
MAGNet-SA-
GS-MG
Maly-
Past Knowledge (rep)
Nearby Agents
Attention-
sheva et al.
2018)
NeurComm
Chu et
2020
Future Knowledge
Nearby Agents
RNN
Agent- Entity
Graph
gar
Past Knowledge (w/o rep)
Nearby Agents
Attention-
et al. 2019b
45

---

Published in Transactions on Machine Learning Research (02/2026)
5.6.1 Who Communicates with Whom and What Is Shared
The first. design decision concerns the nature of the information being exchanged. Methods transmit local observations, latent encodings, predicted intentions, or learned value-related features. What agents choose to broadcast is tightly connected to who is responsible for speaking and whom the message is intended for.
In early systems such as DIAL Foerster et al. (2016a) and RIAL Foerster et al. (2016a), all agents share low-level hidden states, reflecting a simple assumption where every agent serves as both sender and receiver. CommNet Sukhbaatar et al. (2016) introduces representation-based sharing, enabling agents to reason over pooled latent feature but still maintaining fully inclusive broadcasting pattern. As the field evolves, newer models refine both the content and the routing structure. For instance, TarMAC Das et al. (2018) assigns messages to specific teammates, guided by role or spatial relevance. MAGIC Niu et al. (2021 and GA-Comm Liu et al. (2020) reduce redundancy by designating representative agents that compress group- level information. IC3Net Singh et al. (2018) and Gated-ACML [ao et al. (2020) further introduce learned speaking decisions, where only agents that expect their information to be useful choose to communicate. This progression reflects shift toward more deliberate and purpose-driven messaging.
5.6.2 When Communication Is Necessary
Timing is increasingly treated as an independent design dimension. In many classical architectures, com- munication is contimuous and unconditional. However, always-on communication rarely efficient and can overwhelm learning dynamics. Recent methods identify the need to transmit information only when it becomes valuable.
Event-driven communication, seen in ETCNet Hu et al. (2020a) activates messaging only when significant envirommenal changes occur. Predictive communication in ATOC Jiang & Lu (2018) triggers information exchange before agents enter cooperative interactions. Gating mechanisms in IC3Net Singh et al. (2018) and
other selective frameworks learn to suppress communication in uninformin uninformative moments, reducing bandwidth and improving robustness. These advances show that the temporal aspect of communication not merely optimization detail but fundamental part of multi-agent coordination.
5.6.3 Why Communication Helps and How It Is Used
The underlying reason for communicating determines the structure of the commmication protocol. When the purpose is to reduce partial observability, methods favor explicit message passing with expressive encoders and learnable aggregation functions. When the goal is to align distributed representations or coordinate long-horizon planning, models adopt iikentioonbased or role-aware mechanisms that shape the influence of messages. When communication must be conservative strategically safe, selective gating or reprosentative- based designs become preferable.
Concatenation, equal weighting, and simple pooling dominate early algorithms such as DIAL Foerster et al. (2016a), RIAL Foerster et al. (2016a), and MADDPG-M Kiline Montana (2018). providing scalable but coarse message integration. Attention-based methods such as TarMAC Das et al. (2018) MAGIC Niu et al.) (2021) , GAXNet Yun et al. (2021) ,, and MAGNNe--SA-GS-MM
Malyshhva et al. (2018) introduce finer-grained routing that reflects task relevance. Neural architectures such as RNNs used in 12C Ding et al. (2020) and Neur Comm Chu et al. (2020), CNNs used in DGN Jiang et al. (2018). and gated networks as in Gated- ACML Mao et al. (2020) offer richer temporal or spatial structure. Across all these approaches, the choice of how to encode and aggregate messages is closely connected to why communcatiin is beneficial in the first
place.
Overall, framing MARL-Comm through what, when, and how clarifies the role of each design element. Future work is likely to explore more adaptive message content, richer temporal scheduling, and tighter alignment between communication purpose and communication protocol.
5.7 Bridge: From MARL-Comm to Emergent Language
46

---

Published in Transactions on Machine Learning Research (02/2026)
Table 7: From MARL-based communication to emergent language: limitations in MARL-Comm that moti- vated EL research.
Aspect
MARL-Based Communica-
Motivation gent Language
Toward Emer-
tion
Message form
Continuous or latent vectors; Entangled with policy net- works.
Discrete symbols or tokens; Explicit communicative units.
Interpretability
High task performance; Low human interpretability.
Focus semantic structure; Human-readable protocols.
Generalization
Jointly trained agents;
Partner generalization;
Weak zero-shot transfer.
Population-level consistency,
Semantic ground- ing
Implicit, reward-driven sc- mantics.
Explicit symbol-state align- ment.
Core limitation
Opaque, non-transferable sig- nals.
Study why and how language emerges.
Although MARL-Comm has achieved strong empirical performance in cooperative control, much of this progress relies on learning task-specific signaling protocols that are tightly coupled to the environment, reward design, and training population. Consequently, the learned messages are often difficult to interpret, hard to verify, and challenging to transfer new tasks or unseen teammates. Morcover, many MARL- Comm methods assume continuous high-bandwidth communication channels, which mask the semantic structure of what is being communicated and abstract away practical constraints.
These limitations motivated research on emergent language, which shifts the focus from optimizing task return to understanding how discrete, structured communication protocols arise through interaction. particular, emergent language studies aim to characterize when commiication becomes necessary, what compositional structure can emerge, and how protocol stability and generalization behave across different games, environments, and agent populations. This perspective complements MARL-Comm by emphasiz- ing interpretability and generalization as first-class objectives, rather than byproducts of task optimization. Table 7 highlights how limitations in MARL-based communication, particularly interpretability and gen- eralization, motivated the emergence of discrete and semantically grounded communication research.
6 Emergent Language in Multi-agent Systems
Deep learning advances in natural language processing and multi-agent reinforcement learning have enabled a new paradigm for studying how communication systems can arise through interaction. This line of research, commonly referred to as Emergent Language (EL) or emergent communication, uses learning- based agents simulate the formation of human-like signaling behaviors without predefined semantics [oldt & Mortensen (2024). Large-scale self-play systems, such as AlphaZero Silver al. (2017) and OpenAl's hide- and-seek agents Baker et al.] have demonstrated that complex coordination strategies can emerge purely from environmental dynamics and reward feedback. Building on these insights, early EL studies extended self-play to discrete communication settings, showing how symbolic protocols can arisc from task-driven interaction Foerster et al. (2016a): Lazaridou et al. (2016); Havrylov & Titov (2017) ; Mordatch Abbeel (2018h. While modeling language emergence offers compelling lens for understanding communication, EL research has also become increasingly relevant for designing scalable, interpretable coordination mechanisms in artificial multi-agent systems.
47

---

Published in Transactions on Machine Learning Research (02/2026)
Buck et al. (2018) Steinert- Threlkeld
2022 :
Number of Agents.
Lazaridou et al. 2010 Mordatch & ADoee
2018 )
Lowe et al.
2 Ren
2202
Van Eecke & Beuls 2020
Graesser ct al
2019)
Levela of Co- operation
Noukhovitch et al.
2021 Lowe et al.
2019)
Liang et al. 2020
Noukhovitch et al. 2021
Who/ What to Comm. (Tab. g)
Communication Model Types
Lazaridou et al.
2016 2018
Evtimova et al.
2017
Comm. Steps Counts
Jorge et al.
2016 Ca ct al 2018
Comm. Vo- cabulary Size
Foerster et al
201 Lazaiddou at al. avrrylov & 11ov (017
[201 2018
Resembling Hu-
Ren et al. 2020 :
Kirby et al. 2007
2008
Languages
Resnick
2019
Lazanioou et 2018
Prokopenko Wang (2003 ; Zaiem Bennequin (2019)
Lowe et al. (k011);; Brighton & Kirby (2006] ].] Lazari-
dou et 3l. (2018) ! Kriegeskorte et al. [2008]] Luna
[2020]] Chaabouni al. (2220) Bogin
[2018 Kucinski et al. (2020) Andreas (2019 y Korbak
Evalunting metrics
(2020 /Kharitonvv Baron (2020 Chaabouni
et al (2020) Korbak et all (2019): Chaabouni et al
2020 Denamganai Walker (2020a) ] Kharitonov
& Baroni (2020): Reenick et al. (2019 Perkins
Emergent Language Communication in Multi-Agent Systems
[2021a]; [Bullard [202]
202 1g Mu
& G.oodmann (021 Baroni 2020
Yao et al 2022
Chaabouni et al. (2019): Dessi et al. 2019))
Emergence Language in Games
Chaabouni et al. (2020]
Auersperger & Pecina
(222 Kharisonov
(2020) |Prrins (0021b
Ikram et al. (2021
Eutimova et al. [2017 Mu
Goodman
Noukhovisch et al. 2021
Lazaridou et al. 2016
Bouchacourt & Baroni 2018): De-
When/ Where to Comm. (Tab. g
namganai al. (2023]
Carmeli et al. (2923 Slowik
Signaling Game and Ita Variants
Kang ct al. [2020) Denamganal & Walker [2020b F
Evtimova et al [2017] Rita et al. (202) Chaabouni
et al. 2020 Kharitonor et al (2019 ] Guo 2019
Bouchacourt Baron] (2019)
Blumenkamp Prorok [202]] ): Kolb al (2119]
Grid World Games
Simoes et all (2200) [] Gupta [2020] | SSkhhbattar
et al. 2010 Kalinowska et al. (202)] Wang et al.
2019c
Pesce Montana
2020b Patel et al. 2021
Why /How Comm. (Tab. 10
Scalability to General- Purpose Tasks
Mul et al (2019) Li et al 2022 |. Nisioti et al.
[2023
Bulllrdd (2021)
Cope Schoots
2021
Wang et al. (2024e] lucker et al. 2021
Interpretability
Jorge et al. 20166:
Choi et al.
2018
Lazaniou
2016
Robust and Generalization
Cope & Schoots 202 Wang et al 2024e
Bullard et al. 2021
Cope Schoots 2021
Figure 7: EL-Comm Agents Taxonomy
48

---

Published in Transactions on Machine Learning Research (02/2026)
Multi-agent
Sec.
Agents
Communication
Sec.
Who communicate
objectives
Number of agents
Scalability
Images
Messages
ABC
ABC
Previous Actions
Embedding Networks
Human-like Multi-Step
Evaluating
Interpretability
Reward
Sec.
Testing Environment
6.4
Robustness
Signaling Games
Grid-World Games
EL Games
Generalization
Figure Illustrating the organization of Sec. under the Five Ws lens. We analyze who commu- nicates with whom and what is communicated via population roles, recipient selection, and mes- sage encoding/metrics (Sec. 6.3 including multi-step dialogue and compositionality). Next, examine when and where communication occurs through standardized toolkits, one-shot versus ilerative proto- cols, and grounded or embodied environments (Sec. 6.4). Finally, we discuss why communication emerges and how it learnod and shaped, covering motivations, interpretablityy/strruuuu protocols, and robust- ness/generalization (Sec. 6.5
49

---

Published in Transactions on Machine Learning Research (02/2026)
Review Scope and Structure Emergent Language Communication (EL-Comm) investigates how agents develop communication protocols from interaction alone, without explicit supervision or predefined mean- ing. This section begins with foundational background and core concepts (Section 6.1 followed by an overview and taxonomy of the EL literature (Section 6.2 We then organize prior work using the Five Ws perspective: who communicates with whom and what is shared (Section 6.3 when and where communication occurs through interaction structure and environment design (Section 6.4 and why and how communication emerges via learning objectives, protocol structure, and generalization properties (Section 6.5 Section 6.6 synthesizes these dimensions, and Section 6.7 connects EL-Comm to LLM-based communication, highlighting limitations that motivate Language-groundee and hybrid approaches. Figures] 7 and g summarize the taxonomy and end-to-end EL pipeline, guiding readers through key design choices, environments, and evaluation criteria.
6.1 Background and Foundations of Emergent Language
Emergent language has become a rapidly advancing area of research within artificial intelligence, particularly in the realm of multi-agent reinforcement learning. Historically, the study of language emergence focused on understanding the origins of human language, with limited attention to its applicability for artificial agents. However, recent advancements in reinforcement learning aim to enable agents to develop communication protocols with capabilities comparable to or even exceeding human language. These efforts extend beyond the statistical approaches commonly used in natural language processing, prompting critical questions about the conditions for language emergence and how to effectively evaluate its success. To provide context for the taxonomy and analysis, this subsection outlines the fundamental concepts of communicttion and linguistics and offers background knowledge of emergent language research Peters et al. (2024).
Communication fundamentally involves the exchange of signals that convey information, whether intentional, such as speech, or unintentional, like bodily reactions. According to Watzlawik's 'Interactional View* [Wat-] zlawick et al. (2011), all behavior communicates, making communication universal and essential. It occurs through various channels and can be broadly categorized into intrapersonal, interpersonal, group, public, and mass communication, depending on its purpose and audience Peters et al. (2024). In emergent language research, two forms of communication are commonly studied: interpersonal and group communictiinn In- terpersonal communication involves interactions between entities that influence each other, with each entity perceiving its own environment, often overlapping through shared commuicatiin channel. Noise may affect the process by distorting perception or communication. Group communication extends this to multi- ple entities, focusing on collective goals or tasks, making it more formal than interpersonal communication, which often has a social character. Most population-based EL research focuses on group communication, while intrapersonal, public, and mass communication remain largely unexplored.
Natural language is one of humanity's greatest. achievements, serving as the foundation for all forms of communication. It enables us to convey highly complex information through discrete and humanly man- ageable amount of utterances. Much AI rescarch focuses on developing natural language models for tasks like translation and texl generation Wolf et al. (2020); Brown el al. (2020); Lauriola et al.] (2022); Khurana et al. 2023). However, many in the Al community argue that current statistical models trained on static datascts lack true language undorstanding for effective human cooperation Mordatch & Abboel (2018); Choi eel al.] (2018) ; Lazaridou et al. (2021); MMerrll el. al. (2021). Emerging EL research in Al aims to enable agents to communicate like humans, enhancing cooperation, performance, and generalization, and fostering meaningful interaction between humans and Al Mordatch & Abbeel (2018). While explicit forms of EC are widely studied, some survey papers excluded implicit communication Peters et al. (2024), such as spatial positioning in multi-agent systems Grupen et al. (2022)
Emergent Language (EL) is a form of communication that develops naturally among artificial agents through interaction, without explicit programming. It ariscs from the agents' need to cooperate and solve tasks in their environment Brandizzi (2023). EL involves agents creating, adapting, and refining linguistic structures to exchange information effectively Lipowska. & Lipowski (2022). Research focuses on understanding how
elements like syntax Ueda et al. (2022). ) semantics Colas et al. (2020); Qiu et al. (2022), and pragmatics Lowe let al. (2019) emerge and enhance agent performance and cooperation. A natural language-like communica- tion system would make artificial agents more accessible, comprehensible, and powerful Tocchi et al.) (022
50

---

Published in Transactions on Machine Learning Research (02/2026)
Noukhovitch et al. (2021 Lazaridou & Baroni (2020). Initially focused on the origins of language Steels 1997p, EL research now emphasizes enabling agents to harness communication mechanisms akin to natu- ral language Lazaridou & Baroni 2020). Modern EL in computer science involves self-learned, reusable,
teachable, and interpretable communication protocols Nowak Krakauer (1999); Lazaridou et al. (2016)
Li & Bowling (2019). The ultimate goal is seamless and extensible communication between machines and humans Lazaridou & Baroni (2020): Yao et al.]
In this section, we review papers that proposed agent-based computer models in which communication emerges as a general paradigm for coordinating behavior in multi-agent. systems Boldt & Mortensen (2024).
These models feature agents with communication channels that generate discrete message symbols, aiming to mimic human-like languago. The structure and content of the messages are not prodefined but instcad emerge naturally during training, shaped by the agents' interactions and the characteristics of their environments. Such approaches have been explored within variety of agentic frameworks, often leveraging deep learning methods and neural networks optimized through gradient descent.
6.2 Overview and Taxonomy of Emergent Language
Communication arises from conventions and rules developed through the need or benefit of coordination.
Lewis Lewis (2008) formalized such scenarios as 'coordination problems' and introduced a simple signaling
game, where a speaker describes an object and a listener identifies it among options. This game significantly
influenced emergent language (EL) research in computer science. Early studics focused on specific aspects of
emergent communication (EC) through hand-crafted simulations Wagner et al. (2003): Steels (1977
Nowak
& Krakaner |1999; ; Kirby (2002); Cangelosi & Parisi [2012]; Christiansen & Kirby 2003 Batali
19988
Oliphant & Batali 1997); Steels 1995); [Skyrms (2002): ; Smith et al. (2003), often relying on supervised
learning and non-situated settings, which limited their exploration of complex linguistic features Wagner
et al. (2003). Recently, EL research has gained momentum Foerster et al. (2016a Lazaridou et al. (0016)
Havrylov & Titov (0077); Bouchacourt & Baroni 2018); Cao et al. (2018); Mordatch & Abbeel
(0118
Das et al. (0017) leveraging multi-agent reinforcement learning (MARL.) approaches (arooachhs//grwwall Agarwal et al.
2019a
Blumenkamp & Prorok (2021): Brandizzi et al. 2022); Iocchi et al. (2022) Chaabouni et al. (2022) Gupta
et al. 20021]
Lo & Sengupta, Lowe et al. 2019 3 Vanneste et al. (202bb) Yu et al. (002bb to study more
intricate features of language emergence.
A key goal of EL rescarch in MARL is to enable agents to autonomously develop communication systems that support both agent-lo-agent and agent-to-human interaction in natural language (NL)-like manner Wag-
ner et al. (2003): Bouchacourt & Baroni (2018); Iocchi et al. (2022); Lo & SSnnuptaa Bogin et al. (018
Noukhovitch et al. (2021). Reinforcement learning (RL) methods are particularly promising for this purpose.
They not only create agents capable of flexible, practical communication [Lzaridou & Baroni (2020) but
also offer insights into the evolution of NT. itself Galke et al.] Unlike NLP models, which rely on static
datasets to mimic NL NL[Steinert, Steinert--Thelleele et al. 2022]; Bender & Koller (2020), EL focuses agents design-
ing and lcarning their own communication systems through active, goal-driven experiences Lemon (2022)
This experiential approach cont rasts with the shallow understanding of language seen in large language models (LLMs) Manning & Schutze| [1999): Qiu et al. (2020); Wolf et al. (220)). In EL, agents develop communication while solving tasks, mirroring natural processes Lewis (2008), leading to a deeper grasp of
their enviromment [ogiin et al. (2018). Advances in EL pave the way for innovative multi-agent systems and
more human-centrr AI applications Lazaridou & Baroni 2020
Various research areas and questions have emerged in EL. Recent studies explore diverse settings, including semi-cooperative environments Noukhovitch et al. (2021): Liang et al. (2020). adversaries Blumenkamp & Prorok (2021) : Yu et al. (2022b). noise in messages Cope & Schoots (2020), and social structures Dubova et al. Fitzgerald & Tagliabue (2022). Grounding EL has been approached through representation learn- ing Lin et al. (2021), combining supervised lcarning with self-play Lowe et al. (2020) using EL agents for fine-tuning NL models Yao et al.) Other work examines the emergence of NL-like characteristics, such as internal and external pressures Luna et al.] (2020): Kalinowska et al. (2022); Dagan et al. (2021) Too-
chi et al. (2022): ; Rita et al.) (2022) compositionality Resnick et al. (2020) generalization Chaabouni et al. (2022), expressivity Guo et al.) and the connection between compositionality and generalization Chaabouni et al. 20200_ Unlike NL.P models like L.MSs, which imitate language statistically without addressing the
51

---

Published in Transactions on Machine Learning Research (02/2026)
functional purpose of communication Mordatch & Abbeel 20118 EL treats language a tool for achieving meaningful outcomes Brandizzi et al. (2022). Agents must learn EL through necessity or benefits Luna et al.
(2020), in settings that encourage communication, such as cooperative scenarios Noukhovitch et al. (2021) However, EL faces significant challenges. Encouraging communication can lead to task-specific gibberish rather than NL-like language Mu & Goodman (2021), making proper incentives critical. Additionally, mea- suring successful communication and assessing language properties like syntax, semantics, and pragmatics are essential for guiding and evaluating EL development Lowe et al. (2019); Chaabouni et al. (2020). The
following sections delve into these challenges and related approaches.
Existing survey publications on EL research address similar topics but differ in scope. Peters et al. Peters
ct al. (2024) categorized these surveys into three main directions, and we expand on this framework by including more recent works and providing broader descriptions. Surveys focusing on learning settings
explore the design of environments and learning processes, as seen in works Van Eecke & Beuls (2020)
Lipowska & Lipowski
(2022); Denamganai & Walker (202bb). Those emphasizing methods review learning
and evaluation techniques, inc iucluding studies Lowe Lowe et al. (2019); Lemon 22022) Korbak et al.
(2020); LaCroix
(019 9: Mihai & Hare (2021 Galke & Raviv (2024); Vanneste el al. (2022a). Lastly, surveys offering general
overviews provide broad insights into the EL field, as demonstrated in locchi et al. (2022); Lazaridou &
Baroni ;020); Galke et al. Hernandez-Leal et al. 2019); Fernando et al. (2020 Brandizzi 2023 Boldt &
Mortensen [2024]]
Peters el al. (2024).
There are also several surveys papers that reviews the EL research. Some surveys focus on learning problems, environments, and language learning design. van Eecke and Beuls Van Eecke & Beuls (2020) reviewed the
language game paradigm, categorizing experiments and highlighting propertics like symmetric agent roles and autonomous behavior. Our survey extends beyond the language game paradigm, providing more details about RL- communication protocols. Similarly, Lipowska and Lipowski Lipowska & Lipowski (2022) explored EL in MARL, emphasizing naming games, protolanguage explainability, and sociocultural factors like migration and teachability. While these are included in our analysis, we integrate them into a broader context. Denamganai and Walker Denamganai & Walker (2020b) reviewed referential games, introducing the ReferentialGym framework and metrics like positive signaling and listening Lowe et al. (2019). Our
survey includes referential games but covers wider range of metrics and approaches.
Some surveys focused on learning and evaluation methods. Korbak et al. Korbak et al. 2020 discussed compositionality metrics, introducing the tree reconstruction error to address gaps in analyzing semantic aspects. LaCroix LaCroix (2019) critiqued the focus on compositionality in EL, suggesting reflexivity as a better direction, though metrics for this remain undeveloped. Lemon [emon (2022) reviewed language grounding, emphasizing the need for better data collection, but did not propose metries. Lowe et al. Lowe et al.) (019) highlighted language utility over semantics, proposing metrics like positive signaling and listen- ing, and Mihai and Hare Mihai & Hare (2021) emphasized exploring semantic factors rather than low-level hashes but did not introduce new metrics. Galke and Raviv Galke & Raviv (dentiidettffied pressures and biases aligning neural EL with human NL, focusing on NL phenomena rather than training biases. Vanneste et al. Vanneste et al. (2022a) reviewed discretization methods for MARL-based EL, recommending DRU, straight-through DRU, and Gumbel-Softmaa for general use, though these are not central to our survey.
There are also more existing surveys providing general overviews or focus on unique aspects of EL rescarch that do not fit into the seltings or methods categories. Hernandez-Leal et al. Hernandez-Leal el al. (2019) offered a broad review of multi-agent reinforcement learning (MARL), categorizing work into emergent be- havior, communication, cooperation, and agent modeling. While their survey is valuable for understanding MARL's historical context, it lacks detailed analysis of EL-specific metries, which we address in this work. Brandizzi and Iocchi Iocchi et al. (2022) emphasized lighting the underrepresentation of human-Al Al communication. They explored interaction types but did not provide a comprchensive revicw of EL papers, as we do. Moulin-Fricr and Oudcyer Moulin-Frier & Oudeyer (2020) connected MARL to linguistic theories, identifying challenges like decentralized learning and intrin- sic motivation but omitted detailed metric discussions. Galke et al. Galke et al. reviewed RL-based EC approaches, focusing on the gap between emergent and human language, particularly in compositionality. Fernando et al. Fernando et al. (2020) proposed novel drawing- communication system, which, while interesting, falls outside our survey's scope. Suglia et al. Suglia et al. (2024) reviewed visually grounded lan-
interactions in EC' research, high-
52

---

Published in Transactions on Machine Learning Research (02/2026)
guage games, categorizing tasks and models but primarily focusing on multimodal grounding rather than EL itself. Zhu et al. Zhu et al. (2024) structured EC works along nine dimensions but focused more on learning tasks than EL emergence. Lazaridou and Baroni Lazaridou & Baroni (2020) provided a summary of the EL field but did not emphasize metrics or taxonomy, which are central to our work. Similarly, Brandizzi Bran- dizzi (2023) categorized EC research and identified open challenges but lacked a structured taxonomy or a systematic literature approach, which we include with more publications. Boldt et al. Boldt & Mortensen (2024) offer review of EL research, however, it mainly focused specifically on the goals and applications of EL research.
6.3 Who, Whom and What to Communicate: Preliminary Considerations
In emergent communication systems, the foundational questions of who communicates with whom and what information is exchanged play decisive role in determining whether structured protocol can arise at all. Unlike MARL-Comm where these dimensions are often shaped by explicit architectural design, emergent language settings require these components to be discovered through interaction. Thus, separating the analysis along the axes of 'who/wlin whom' and 'what' is essential for understanding how communication emerges from minimal assumptions.
We group who/ Acho/whot /whom and what together in this subsection since these dimensions are tightly coupled in emergent communicatin: the identity and number of interacting agents directly constrain the informational content that must be transmitted, and conversely, the nature of the information structure shapes which agents must communicate and how frequently. Studying them in isolation would obscure these dependen- cies, whereas a unified treatment allows us to characterize how agent roles, task structure, and message representations co-evolve.
To provide principled organization of this space, we examine the following dimensions that collectively address the core who/w whh/whom// he/hho/oowwtttt /wwat challenges: (1) specifying the communicating parties (who/ (hho//hom,, (2) determining how social incentives shape when and why communication is beneficial (why + whom), (3) identifying the representations used to encode messages (what), (4) structuring the interaction protocol through which information is exchanged (what + whom), (5) regulating the expressive capacity of the communication channel (what), and (6) shaping the form and linguistic structure of emergent messages (what). (7) evaluating the structural and functional propertics of the resulting messages through formal metrics that assess informativeness, compositionally
and generalization (what). Each of these dimensions reflects a distinct foundational challenge that any emergent communication system must resolve.
We therefore structure this subsection around these perspectives, as each cluster of works addresses a dif- ferent component of the broader htt///ho//whhhtte problem. In the following paragraphs, we summarize representative studies within each category and highlight how they refine our understanding of how agents decide who communicates with whom and what information is transmitted.
6.3.1 Population Size and Communicative Roles
A central challenge emergent communication is determining which agents participate in communica- tion and how their roles structure the emergence of a shared protocol. Unlike MARL-Comm, where communication roles are often tied to task-driven coordination needs, research in EL has historically treated "who communicates with whom' as a structural design choice. This choice directly constrains what messages can be produced, how interaction unfolds, and the expressive capacity of the resulting commuicatiin system. Solutions in the EL literature lypically instantiate one of three communication configurations: single-agent, dual-agent, or population settings Peters et al. (2024), each progressively increasing the complexity of the tho//hhmms structure and the flexibility available for language-like behavior to emerge.
Early work adopts single-agent settings, which are comparatively rare but useful when the goal is to model muman-machine interfaces. Buck et al. Buck et al. (2018) use a single agent to learn interpretable representations that facilitate human Al interaction, while Steinert et al. [Steiner--Threee Threlked et al. (2022) focus on model fine-tuning where the emergent messages serve as internal scaffolding rather than inter- agent communication. These works address the /ho/whom challenge by effectively removing multi-agent
53

---

Published in Transactions on Machine Learning Research (02/2026)
interaction, showing that emergent symbol structures can arise even without conversational partner, but at the expense of neglecting genuine dialogue dynamics.
The field then evolved toward dual-agent communication, which became the dominant paradigm for studying emergent protocols. Lazaridou et al. Lazaridou et al. (2016) introduced the canonical preaker-listenee referential game, establishing clear communicative roles: one agent selects a message and the other interprets it. This structured who/whom /whom configuration enabled rigorous study of how symbolic communication emerges from cooperative pressures. Subsequent work Mordatch & Abbeel (2018) deepened this analysis, examining how variations in input distributions and (raining dynamics influence emergent ut- terances. The same speaker- listener paradigm was later adopted in grounded settings such as Lowe et al. (2017) where messages come from a fixed K-dimensional onc-hot vocabulary. While this restricts what can be communicated, it isolates how role asymmelry shapes the form of emergent messages. Across these dual- agent systems, the improvement over single-agent designs lies in introducing role specialization, which allows rescarchers to disentangle message production from message interpretation and thus study the structural properties of emergent language.
More recent works explore population-based comminication, where multiple agents interact simultane- ously. These settings address a new challenge how protocols scale and stabilize when many agents must coordinate or compete. Agarwal et al. ? show that populations enable community regularization, which sta- bilizes emergent languages and prevents degenerate conventions from dominating. Ren el al. Ren et. al. (2020) demonstrate that multi-agent populations naturally support language evolution through iterated learning, allowing compositional structure to emerge over generations of agents. Additional studies Grresssr et al. (2019); ; Van Eecke & Beuls (2220) examine how larger populations provide more opportunities to shape the emergent protocol through selective partner interaction, variation in roles, and exposure to heteroge- neous communication partners. Population settings therefore improve over earlier paradigms by revealing how social srructure-not just pairwise pardination-affeca which protocols are selected, transmitted, and stabilized.
Together, these works define the 'who/whom' landscape by showing how the number of interacting agents, the asymmetry or symmetry of their roles, and the social structure of the population constrain the interaction dynamics and shape the space of possible emergent messages.
6.3.2 Cooperation Level and Recipient Selection
Another challenge in emergent communication research is understanding how social incentives that are en- coded through the reward structure shape both who communicates with whom and what information must be exchanged. Because communication is costly, ambiguous, or strategically manipulable, different coop- crative structures impose distinct pressures on the emergence, stability, and semantic content of messages. Thus, the degree of cooperation in the environment is not merely a contextual detail but primary determinant of when communication becomes necessary and which communication strategies are viable.
To examine this challenge, prior work has analyzed emergent communication under three primary incentive structures: fully cooperative, semi-cooperative, and fully competitive environments. Each structure provides a different answer to the 'who/w hhho/whom' and 'what' dimensions, as the reward topology dictates which agents depend on one another and what types of information are beneficial to share.
In early work, fully cooperative settings dominate the literature because agents share global reward and therefore rely on communicatinn to jointly reduce uncertainty and coordinate successful task completion. Studies such as Noukhovitch et al. NNouhhovith et al.] (2021) show that when all agents reccive identical rewards and have no individual incentives, communication naturally emerges as a tool for aligning beliefs and actions. Here, the 'who communicates with whom' dimension is simple: every agent's payoff is intertwined while the 'what' dimension centers on transmitting task-relevant referential or relational features needed for group success.
Semi-cooperative environments, examined in works such as Lowe et al. Lowe et al. (2019) and Liang et al. Liang et al. 2020), introduce both shared and individual rewards. This incentive structure complicates who should communicate with whom: agents must decide whether sharing information benefits the group, themselves, or potentially empowers competitors. The challenge therefore shifts toward balancing cooperative
54

---

Published in Transactions on Machine Learning Research (02/2026)
information exchange with self-interested decision-making: These studies demonstrate that communication content becomes more selective and strategic, which means the agents often share partial or compressed signals that advance both personal and collective utility.
Fully competitive environments are comparatively rare emergent communication research. Noukhovitch et al. NNuukhovitch et al. (2021) present one of the only systematic examinations. Here, the incentive struc- ture makes the who/whon dimension adversarial: agents may choose to conceal information communicate deceptively, as sharing accurate messages directly benefits opponents. As a result, meaningful communication often fails to emerge unless some cooperative substructure exists. These results highlight the limits of emergent communication under purely competitive incentives and highlight that 'what' is communicated bocomes strategically suppressed or intentionally mislcading.
even
Taken Logether, these works show that the levels of cooperation fundamentally determine the communication topology (who speaks to whom) and the semantic structure of messages (what must be conveyed). Cooperative environments encourage broad, informative communication; semi-cooperative environments in- duce selective and self-interested signaling: and competitive environments inhibit communication altogether or promote deceptive strategies.
6.3.3 Architectural Mechanisms for Encoding What is Communicated
Representational choices strongly influence how emergent communication develops: before agents can form a protocol, they must. first. determine what information from raw perceptual input should be encoded and transmitted. Existing EL systems approach this problem by designing neural architectures that extract, compress, and structure evidence for communication. We group these works together because they address the same underlying question of how architectural choices determine the content of messages, their design decisions also collectively define the dominant strategies for specifying 'what' is communicated and how this interacts with the roles of speaker and listener.
CNN-based Communication Architectures One early line of work focuses on improving the percep- tual encoding available to the speaker, thereby refining what information enters the communication channel. In the referential games of Lazaridou al. Lazaridou et al. (2016), Convolutional Neural Networks are used to transform raw images into higher-quality visual features. This architectural change strengthens the commu- nicative 'what' by ensuring that messages are grounded in more discriminative perceptual representations, enabling the listener to resolve the target more reliably.
Attention-based Communication Architectures Subsequent work by Evtimova el al. Evtimova el al. (2017) incorporates an attention mechanism into the gressag--eneration process, allowing the encoding net- work to emphasize salient object regions. Attention not only clarifies what information should be highlighted in the message but also implicitly addresses whom the speaker is informing, since the mechanism prioritizes features most useful for the listener's decision-making process. The resulting messages generalize better to unseen objects precisely because they reflect feature relevance rather than uniform encoding.
LSTM-based Communication Architectures Lazaridou et al. Lazaridou al. (2018) further extend the representational pipeline by introducing an LSTM-based recurrent decoder, drawing on DRQN princi- ples Hausknecht & Stone (2015). Unlike earlier one-shot symbol generation, the recurrent policy maintains a temporal momory of previously produced tokens and intermodiate states. This enables mossages to form multi-token sequences, enriching the expressive space available for communication and allowing 'what' is communicated to unfold over time. Recurrent encoding thus supports more structured and compositional message formation, property important for later EL work.
Across these studies, the architectural template is consistent: a CNN encoder produces a hidden state hr.zi a recurrent decoder generates a message m = g(hnn, and a listener-side LSTM encodes this message into = h(m,00) before comparing with candidate images to select the correct target. Speaker and listener parameters are optimized jointly without weight sharing, reflecting their asymmetric contributions to the communication pipeline.
55

---

Published in Transactions on Machine Learning Research (02/2026)
Collectively, these architectures clarify the 'what' dimension of emergent communication by showing how perceptual encoders, attention mechanisms, and recurrett policies influence the semantic content of messages and determine how effectively information is conveyed between agents.
6.3.4 Multi-Step Interaction and Dialog Structure
A well-established model of language emergence was outlined above, but the literature has since expanded to include several variants that incorporate more interactive, human-like communication patterns. One line of work, exemplified by Jorge et al. (2016) and Cao et al. (2018), introduces multi-step exchanges that more closely resemble dialog.
In Jorge et al. (2016), the task remains referential, identifying target images (celebrities), yet both agents participate in message generation. The guessing agent begins the interaction by posing questions derived from the candidate images. The answering agent, who knows the target, responds with binary signals such as 'yes' or 'no', similar to the mechanics of the Guess Who game. The most effective performance is achieved with two rounds of questioning and answering. The authors report that the questioning agent tends focus distinct visual attributes across the two questions, thereby accumulating information in structured manner.
Cao et al. Cao et al. 2018) also examine multi-step communication but shift from referential games to negotiation tasks in which agents bargain over items with different utilities. Their framework introduces two complementary communication channels: symbolic channel for exchanging abstract information and proposal channel that communicates trade offers that may be accepted or rejected by the partner. Under cooperative reward structures, the symbolic channel helps agents reveal preference information, enabling more efficient negotiation. Their findings show that this additional communicative structure helps agents converge loward a Nash equilibrium and reduces variability in joint optimality, improving the robustness of the learned bargaining strategies.
Taken together, these studies demonstrate how multi-step dialogs broaden the space of 'what' can be commu- nicated by allowing agents to iteratively refine exchanged information, and they clarify 'whom' each message is directed to at different stages of the interaction protocol.
6.3.5 Vocabulary Size and Message Structure
A persistent challenge in emergent communication is determining how much expressive bandwidth agents need to convey task-relevant information. Vocabulary size directly constrains what agents can communi- cate: too small a symbol set limits expressiveness and forces overly compressed encoding strategies, whereas larger vocabularies or sequence- based messages allow richer semantic distinctions and the emergence of struc- tured communication. We group these works together because they examine how expanding or restricting the channel capacity shapes the semantic space of messages, thereby defining the limits of what information agents can reliably exchange.
Simple 1-Bit Messages The vocabulary size significantly impacts the effectiveness of communication between agents. Foerster et al. Foerster et al. (2016a) initially experimented with simple 1-bit messages in scenarios requiring only minimal information exchange, such as solving the switch riddle or determining numerical parity. These settings illustrate how extremely limited vocabularies restrict agents to transmitting single binary distinctions, sufficient only for tasks with low informational complexity.
Single-symbol Messages with a Larger Vocabulary Size As tasks became more complex, researchers investigated whether agents required larger symbol sets to convey more nuanced distinctions. Tazaridou et al. Lazaridou et al. (2016) explored single-symbol messages with vocabulary sizes ranging from to 100, showing that increasing the cardinality of the message space allowed agents to encode more fine-grained referential information. This line of work demonstrated that vocabulary size acts as design lever governing what types of attributes or relational distinctions agents can represent.
NLP-introduced Message Sequences Subsequent research inspired by natural language introduced recurrent architectures capable of generating symbol sequences rather than isolated tokens Havrylov &
56

---

Published in Transactions on Machine Learning Research (02/2026)
Lazaridou et al. (2018) In these models, the vocabulary remains fixed, but the combinatorial power of sequences vastly expands the potential message space. It is important to note that these symbols initially lack inherent meaning; their association with perceptual or task-specific features emerges entirely through training. One of the key advantages of sequence based communication that it enables composi- tional structure to appear. For example, a speaker may produce description such as 'red box' by combining learned representations for 'red' and 'box,' even if that specific attribute combination was not seen during training.
Titov 20017]
Taken together, these works show how vocabulary size and message structure determine the expressive boundary of what agents can communicate. By regulating the symbol budget or enabling combinatorial soquences, rescarchers shape the granularity, compositionality, and generalizability of emergent languages, thereby addressing the fundamental question of what information the communication channel can support.
6.3.6 Message Structure and Compositionality Toward Human-Like Language
A persistent challenge in emergent communication is understanding what structural properties languages acquire when agents learn to communicate from scratch. Earlier work in EL primarily examined whether agents could transmit lask-relevant signals at all, emphasizing referential accuracy or cooperative success. Yet the resulting protocols often remained brittle, uninterpretable, and highly specialized their training environments. These limitations motivated a shift toward investigating whether emergent languages can develop human-like structural propertics that support gencralization, interpretability, and systematic reuse of concepts. This line of research moves beyond the mechanics of who communicates with whom and what information is conveyed, and instead asks what form the communication system itself ultimately takes and whether resembles linguistic systems observed humans.
Human language exhibits low-level regularities, such as compositional semantics, morphology, and syntax, as well as higher-level pragmatic behaviors shaped by social interaction and cultural transmission. These properties allow humans to generalize to unseen contexts, compose meanings productively, and coordinate effectively with diverse partners. Accordingly, researchers explore whether emergent protocols can develop analogous linguistic structure and identify the environmental, architectural, or learning pressures under which such structure arises. By examining these questions, the field aims not only to obtain deeper theo- lack
retical understanding but also to address long-standing shortcomings of carlier EL systems-inchinn of compositionality, limited transfer, and the opacily of learned messages by grounding emergent language design in principles that characterize natural communication systems.
Emergent communication research therefore seeks to understand how and why certain linguistic traits arise, ranging from compositional semanties and syntactic regularities to pragmatic behaviors influenced by social context. The goal is not to recreate human language verbatím, but rather to study how different task settings, agent architectures, and learning dynamics generate pressures that lead to more structured and interpretable emergent protocols. This perspective treats language emergence as a complex adaptive process shaped by interactions among agents rather than explicit engineering of linguistic rules.
Despite the ambition of this research dircction, no existing study attompts full reconstruction of human language. Most works locus on a single linguistic property al time, which risks overlooking the deep in- terdependencies among linguistic phenomena. For example, Ren et al. Ren et al. (2020) show that iterated learning imperfect transmission of language across generations induces compositionality, consistent with longstanding rcsults from cognitive science Kirby et al. (2007) 2008). However, many studics of composition- ality rely on porul-population settings with no generational transmission, raising questions about whether observed compositional structure truly arises from factors such as representational capacity [Reniick et al. (209)) perceptual features Lazaridou et al. (018)), or whether important cultural pressures are being omitted.
This fragmentation highlights a broader challenge: human-like linguistic structure likely emerges only when multiple pressures are jointly present, e.g., expressivity, learnability, cultural transmission, environmental diversity, and social incentives. Research that isolates only one such factor may therefore produce languages with limited resemblance to natural communication systems, even if they succeed specific tasks.
57

---

Published in Transactions on Machine Learning Research (02/2026)
Taken together, these studies address the highest-level component of the 'what' dimension: not merely what information agents choose to convey, but what structural form the communication system itself acquires. They collectively demonstrate that human-like linguistic properties emerge only under specific combinations of environmental pressures and learning dynamics, highlighting the need for more integrated and comprehensive approaches studying emergent language.
6.3.7 Metrics for Evaluating What Is Communicated
Evaluating emergent communication requires more than verifying whether agents succeed on a task. Early studies relied heavily on task accuracy (e.g., correct image selection) as the primary indicator of successful communication, but such measures offer limited insight into whether the learned messages are interpretable, structured, or capable of generalizing beyond the training environment. This limitation prompted a shift toward formal evaluation metries that capture what properties the emergent messages themselves exhibit. By moving beyond task reward, these metrics operationalize the 'what' dimension of the Five- Ws framework and provide a principled basis for diagnosing weaknesses in communication protocols, comparing competing approaches, and assessing whether an emergent language possesses meaningful linguistic structure rather than task-specific heuristics.
Entropy Choice accuracy remains a widely used baseline, but metrics such as purity have limited ability to assess interpretability. For example, forcing agents to describe broad categories (e.g., 'dog") instead of fine-grained features (e.g.. color or shape) can inflate success rates while obscuring the actual content of communication. This limitation has motivated the development of metrics that assess communication in- dependently of downstream task reward. Prokopenko and Wang Prokopenko & Wang (2003). , for instance, analyze public belief entropy, showing that it decreascs as communication conventions stabilize. Similarly, Zaiem et al. Zaiem & Bennequin 2019) quantify how much uncertainty message reduces, following sug- gestions by Lowe et al. Lowe et al. (2019) to measure informativeness rather than task reward alone.
Beyond Information-Cheorele metrics, researchers have developed increasingly fine-grained lools to measure structural properties of emergent languages. Whereas concrete properties like vocabulary size are easily quantified, abstract ones, such as compositionality and generalizability, require careful formalization, and their evaluation has become central to the field.
Compositionality Compositionality refers to the principle that complex utterances derive meaning from the combination of simpler components (c.g., 'red car' car that is red), distinguishing structured communi- cation from holistic signaling. The most common metric is topographic similarity Brighton & Kirby (2006):
Lazaridou et al. (2018), which measures correlation between distances in the referent feature space and message space. Lazaridou et al. Lazaridou et al. (2018) apply Spearman's 'ss rank correlation (p) using cosine distances for referents and Levenshtein distances for message strings. More sophisticated metrics include representation similarity analysis Kriegeskorte al. (2008); Luna et al. (2020), which compares internal metrics evaluate whether message com-
agent representations with referent features. Disentanglementt-bann
ponents correspond to independent attributes via positional and bag-of-words disentanglement Chaabouni et al. 2020), context independence [Bgii et al. (2018), or conflict count Kucinski et al. (2020). Tree re- construction error Andreas (2019) examines how well learned messages match a compositional semantic grammar. Meta-analyses by Korbak et al. Korbak et al. (2020) show that many metrics capture only basic forms of compositionality, while Kharitonov and Baroni Kharitonov & Baroni (2020) and Chaabouni
al. Chaabouni et al. 2020) challenge whether such metrics reliably predict generalization to novel objects.
Generalization measures how well agents communicate in conditions not seen during training. In emergent communication, common benchmark is whether agents describe objects composed of novel attribute combinations Korbak et al. (2019); Chaabouni et al. (2020): Denamganai & Walker (2020a):
Generalizability
Kharitonov & Baroni (2020): Resnick et al. (2019) Perkins| (2021a). Other forms of generalization include
robustness to new communica tion partners/ Bullard et al. (2021), adaptation to new environments Guo et al.
2022 bg Mu & Goodman (2021). and separation of syntax from semantics [Baoon (2020). Yao et al. Yao et al.
2022 propose data-driven metric that compares an emergent language's machine translation performance
58

---

Published in Transactions on Machine Learning Research (02/2026)
to human language, under the intuition that emergent protocols closer to natural language better support transfer and broad applicability.
Overall, evaluation research specifies what properties an emergent communication system must possess to be considered successful. Through metrics that assess informativenesss, compositional structure, and generalization, this line of work clarifies the operational meaning of the 'what' dimension and provides rigorous criteria for analyzing and comparing emergent languages.
6.3.8 Discussion: Who-Whom-What in Emergent Language
Table Taxonomy of the Who/Whom Whom and What Components in Emergent Commuication: Population, Incentives, Architectures, Protocols, Expressive Capacity, Structure, and Evaluation Metrics (Sec. 6.3
Category
Sub- Cub-Category
Representative Approaches
Population Size and Communica- tive Role Structure (who/wa (who/whom) (Sec. 6.3.1
Single agent
Human-machine interfaces Buck et al.
(2018)
Model fine-tuning Steinert- Siannert-Thnee Threlkeld al.
2022
Dual-agent settings /speaker and listener
Each with distinct roles, emergent ulter- ances come from a fixed set and the commu-- nication symbols
in the K-dimensional one-hot encoding sample Lazaridou et al.
(2016): Mordatch & Abbeel 2018 Lowe
et al. (007)):
Larger groups of agents: regularization
language evolution Ren et al.
2020 influ-
encing the emergence process Van Becke
BBeuls (2020); Graesser et al. 2019)
Social Incentive Structures and
Fully cooperative settings
Agents share rewards entirely and lack individ- nal incentives and rely on shared language
Their Influence on Communication
coordinate, and commiicatiin does not de-
velop dominance can achieved without
(who/whom whom what)
+
it Noukhovitch et al. 2021
(Sec. 6.3.2
Semi-cooperative setups
Both shared and individual rewards, introduc- ing the challenge balancing collective and personal objectives Lowe et al. 2019 Liang
al. (2020).
Fully competitive teitings
Agents compete for rewards without shared objectives, often leads to deceptive language, making meaningful communication unlikely without cooperative elements Noukhovitch
al. (2021).
Architectural Mechanisms for Encoding Commu- nicative Content
CNN-based Visual Feature Extraction
Lazaridou et al. (0016) used CNNs to pro- cess image imputs, yielding more informative features that improved referential task perfor- mance.
(what)
(Sec. 6.3.3
Recurrent Policies for Tempo- ral Encoding
Lazaridou al. (2018) introduced RNN-based decoders to maintain temporal memory and enhance the generation of longer communica- tion sequences.
Continued on next page
59

---

Published in Transactions on Machine Learning Research (02/2026)
Table continued from previous page
Category
Approach
Representative Works
Attention Attention-Eahanced Commu- nication
Evtimova et al. (017)) added attention mech- anisms guide symbol generation, improving generalization by focusing on salient features in visual inputs.
Jointly Sender Receiver tectures
Optimized Archi-
These works train speaker and listener net- works end-to-end using only final task success as the learning signal, without weight sharing between agents.
Multi-Step Dialog Protocols and In- al. (2016) teraction Structure (who/whow +
Identifying targel lask Jorge
Task: Identifying target images (e.g., celebrities).
Agent Roles: Asymmetric - guesser initi- ales questions, answerer replics.
what)
Communication:
Binary responses
(Sec. 6.3.4
('yes'/'uo") across two interaction rounds. Architecture: MLPs for perception, 2-layer GRUs for maatemesssgee updates, DRU for discretization.
Key Findings: Later questions refine earlier guesses; multi-step dialog improves target accuracy and dialogue consistency.
Agents bargain items that hold different utilities for each party Cao et al. 2018)
Task: Negotiation over items with differing utilities.
Agent Roles: Symmetric- both agents ne- gotiate cooperatively.
Communication: Two channels - symbolic (linguistic) and action-based (trade propos- als).
Architecture: RL agents with value net- works and discrete messaging.
Key Findings: Linguistic messages clarify preferences; leads to Nash equilibrium and reduces variance in joint rewards.
Expressive Capac- ity of the Channel: Vocabulary and
Simple 1-bit messages
Foerster et al. Foerster al. (016aa) initially experimented with simple 1-bit messages in scenarios that required minimal information
Message Structure (what)
(Sec. 6.3.5
Single-syunbol messages
Lazaridou et al. Lazaridou el al. (2016) ex-
plored the of single-symbol messages with
vocabulary sizes ranging from 0 to 100.
Message sequences
Research Lazaridou et al. (2018): Havrylov &
Titov 2017 inspired by natural language, which proposed the adoption of LSTM-based encoder-decoder architectures create mes- sage sequences.
Linguistic Struc- ture and Human- Like Properties of Emergent Messages (what)
Imperfect language transmis- across generations drives compositionality
Ren et al. Ren et al. (2020) demonstrate that iterated learning- -inngg---mpeetttete transmission of language across tionality, as supported by earlier studies Kirby
ceitratioonnrrivee composi-
el al. 2007 2008).
(Sec. 6.3.6
Continued next page
60

---

Published in Transactions on Machine Learning Research (02/2020)
Table continued from previous page
Category
Approach
Representative Works
Fixed-population settings
Many studies fix population size, overlooking iterated learning as a sufficient driver com- positionality, thereby questioning claims that attribute to model capacity [Resnick et al. 2019 or perception Lazaridou et al. (2018).
Evaluation Metrics for Assessing Com- municative Proper-
Public belief entropy
Prokopenko and Wang Prokopenkoo & Wang 2003) analyzed public belief entropy, which decreases communication conventions form
ties (what)
(Sec. 6.3.7
Using entropy to quantify how much a message reduces uncertainty
Zaiem et al Zaiem & Bennequin (2019) used entropy to measure how much a message re- duces uncertainty, metric Lowe al Lowe et al. (2019) also suggested for evaluating commuicattin protocols beyond task perfor- mance
Compositionality
Topognaphic similarity: measures the cor- relation between distances in the referent feature space and message space Brighton & Kirby (2006); Lazaridou et al. 2018);
Spearman's ronk correlation coeficcint (p): with cosine similarity for feature space dis- tances and Levenshtein distance for mes-
sage space distances: Lazaridou et al.
2018
Represenlation
simallariy
analysis:
Kriegeskorte el.
2008 ; Luna al.
22020);
Disentanglernent-uuses Melrics:
Evaluate
whether individual message components
represent distinct attributes, including:
Positional and bag-of-wor words disentan-
glement Chaabouni et al. (0000)
Context independence Hogin
al.
(2018)
Conflict count Kucińiski et al. 2020
Tree Reconstruction Frror: Measures how well compositional semantics model re- constructs the strueture of the language produced by agents Andreas (2019).
Meto- Metoo-Anaysiis of Metrics: Korbak al. Korbak et al. (2020) found that exist- ing metrics capture basic compositionality but struggle to detect more complex forms. Limitations of Compositionallity Met- rics: Kharitonov and Baroni KKaritonov & Baroni (2020) and Chaabouni al. Chaabouni et al. (2020) questioned
whether current metrics reliably assess generalization to novel objects.
Continued next page
61

---

Published in Transactions on Machine Learning Research (02/2020)
Table continued from previous page
Category
Approach
Representative Works
Generalizabllity
Unseen Altributes: Agents general- ize to novel object attribute com- binations Korbak et al. (2019);
Chaabouni et [2020g] Denamganai & Walker (2020a]; Kharitonov & Ba-
roni (2020); Resnick et al. (2019);
Perkins [2021a].
Partners and Environnentss: Agents adapt to unfamiliar part-
New
ners
Bullard et al. (2021) and set-
Lings
Guo el al. (2021: Mu & Good-
m1811
2021).
Seporation: Gen-
eralization across linguistic struc- ture Baroni (2020).
MT-basee Human Alignment: Trans- lation performance used to assess similarity human language Yao et al. 2022).
Overall, Table 8 synthesizes the seven foundational perspectives introduced in Sec. 6.3 providing an integrated view of how emergent communication systems address the coupled questions of who commu- nicates with whom and what is communicated. The table mirrors the conceptual structure of the subsection: it begins with (1) population size and role structure, which determine the basic commuicaation topology; followed by (2) cooperation levels, which shape the incentives that govern when and why munication is beneficial; then (3) architectural mechanisms that specify what perceptual or latent features enter the channel; (4) multi-step dialog protocols that clarify whom each message targets at different inter- action stages and refine what information is exchanged across rounds; (5) expressive capacity constraints, such as vocabulary size and sequence structure, which bound the richness of what can be communicated; (6) human-like linguistic structure, which examines what form the overall communication system acquires; and finally (7) evaluation metrics, which operationalize the what dimension by defining how to measure informativeness, compositionality, and generalization. Together, these perspectives illustrate that emergent communication does not arise from any single design choice. Rather, it is shaped jointly by social structure, representational capacity, interaction protocols, expressive constraints, and the criteria used to evaluate the resulting language. By situating representative works within this unified taxonomy, Table g clarifies how each research thread contributes to resolving a different facet of the broader who/whom// whom/what wwhat challenge and how these threads collectively advance our understanding of structured communication in multi-agent systems.
6.4 When and Where to Communicate: Experimental Environments
In addition to clarifying who communicates with whom and what is communicated (Sec. 6.3 emergent language research must also specify when agents are allowed exchange messages and where these in- teractions are embedded within environment. In practice, the temporal schedule of interaction (c.g., one-shot vs. multi-round exchanges, episodic vs. continual tasks) and the spatial or task context in which communication is grounded (e.g., images, text, graphs, or navigation worlds) together determine the op- portumities agents have to develop and refine a protocol. We therefore treat 'when' and 'where' as tightly coupled: the design of an experimental environment simultancously fixes when along an episode messages can be sent and where in the observation-actinn action loop communication influences behavior.
62

---

Published in Transactions on Machine Learning Research (02/2026)
From this perspective, standardized experimental environments serve not only to simplity implementation, but also to make explicit the assumptions about when and where communication occurs. Well-designed environments allow researchers to vary the timing of messages, the sequence length of interactions, and the grounding context, while keeping other factors fixed. This supports reproducibility, reduces implementation bugs, and enables more reliable cross-paper comparisons. At the same time, emergent communication is highly sensitive to such design choices, so care is needed to avoid introducing systematic biases through environment defaults. The following subsubsections review three prominent families of environments and games, emphasizing how each family encodes assumptions about the temporal and spatial structure of communication.
6.4.1 Standardized Toolkits for Controlling Interaction Timing and Topology
A first challenge is to provide reusable infrastructure in which researchers can flexibly specify at which points in an episode messages are exchanged and which observation spaces they refer to. General-purpose toolkits answer this challenge by abstracting common emergent communication patterns into configurable modules, thereby standardizing both when and where communication is inserted into the learning loop.
Emergence of Language in Games (EGG) Kharitonov et al. (2019) is the most widely used framework for deep learning- based emergent communication, It provides straightforward Python interface for common emergent commmication games, agent architectures, and metrics, making it easy to define at which time step a sender observes a stimulus, when a message is produced, and when receiver acts on it. Studies that implement new games with EGG, such as CChaaooui et al. (2019); Dessi et al. (2019); Chaabouni et al. (2020); Auersperger & Pecinal (2022) : Kharitonov et al. (2020), expand the range of temporal and environmental configurations supported by the toolkit and enrich the library of metrics for probing resulting protocols.
Other lools larget specific aspects of emergent communication experiments while still providing rolled settings for when and where language grounded. TexRel (2021b) offers a synthetic dataset in which compositional language can describe constructed imagos, effectively fixing the visual 'where' while allowing researchers to vary the message schedule. HexaJungle [krrm et al. (2021) introduces a suite of grounded environments for emergent communication, exposing richer spatial structure and longer interaction horizons. For experiments beyond standard platforms, many researchers develop custom implementations of games and agents EEtimova al. (2017); Mu & Goodman| (2021): Noukhovitch et al.) (2021) on top of general- purpose deep learning libraries such PyTorrch| Paszke el. al. (2019), explicitly encoding when messages are sent relative environment transitions.
Across these efforts, standardized frameworks primarily address the 'when/ wwhhn/wwhree' whhre'' question at the infras- tructure level: they provide common hooks for inserting communcctiin into different stages of the percep- tion-action cycle and consistent abstractions for grounding messages in visual, textual, or other modalities. This enables systematic exploration of how varying the timing and context of communication affects the emergence and stability of learned protocols.
6.4.2 One-Shot and Iterative Communication
A second challenge is to design interaction games that expose specific temporal structures for communication. Signaling games and their variants address this challenge by defining how many messages can be sent, in what order, and against which backdrop of stimuli. In these games, 'when' is operationalized through the number of rounds and their sequencing, while 'where' is determined by the modality of the inputs (c.g.. images, texl, graphs) that messages are meant to describe.
The classical signaling game generally involves two agents, a sender and a receiver Lazaridou et al. 2016 The receiver must identify target sample from set that may include distractors using only the sender's message. The sample set can contain images Lazaridou et al. (2016); Bouchacourt & Baroni (2018) Denam- ganaï et al. 2023), object feature vectors (texts) Carmeli et al. (2023) or graphs Slowik et al. (2021) Here, communication typically occurs in a single round: the sender observes the target (and sometimes distractors) and then produces one message, fixing both when the communication happens (once per episode, before the receiver's choice) and where it is grounded (in the representational space of the stimuli). Key design
63

---

Published in Transactions on Machine Learning Research (02/2026)
decisions include whether the sender sees only the correct sample also distractors, potentially different from those given to the receiver Lazaridou et al. (2016), and how many distractors or near-duplicates the
receiver must choose among Kang et al. (2020). Although a referential Gym implementation of the signaling game Denamganai & Walker (2020b) exists, it has seen limited adoption in practice Evtimova et al. (2017)
A closely related variant is the reconstruction game. Instead of selecting from collection, the receiver must reconstruct a sample from the sender's message. The objective is to reproduce the original sample shown to the sender as accurately as possible Rita et al. (2022); Chaabouni ct al. (2020). This sctup resembles an autoencoeer in which the latent boltleneck mimics or supports language, as implemented in EGG Kharitonov et al. (2019). Compared to referential games, reconstruction games change both when and where communication acts: messages are producod once but evaluated via reconstruction quality rather than discrele choice, and the 'where' the continuous feature space of the input rather than discrete index set Guo 2019
The question answer game extends these designs by introducing iterative, bilateral communication over multiple rounds Bouchacourt & Baroni (2019). Unlike original signaling and reconstruction games, this framework allows the receiver to ask follow-up or clarifying questions Das et al. (2017); ? so that the timing of messages becomes dynamic: agents alternately decide when to query and when to reply. This setting has sparked interest in the symmetry of emergent language Das et al. (2017), even though it remains less widely adopted. By enabling agents to adapt their messages over several lurns, question question-answer answer games provide a richer answer to the 'when' dimension and allow researchers to study how protocols evolve within an extended dialog.
Overall, signaling games and their variants operationalize 'when to communicate' through the number and ordering of interaction rounds and specify 'where' communication is grounded via the choice of referents reconstruction targets. They thereby expose how different temporal schedules from one-shot to multi-round dialogs affect the emergence, refinement, and symmetry of learned communication protocols.
6.4.3 Grounded and Embodied Environments
A third challenge is to understand communication in temporally extended, spatially structured worlds where agents must. act and coordinate over many steps. Grid-world and continuous environments tackle this challenge by embedding communication within navigation, control, or manipulation tasks. In these settings, 'when' refers to which time steps within an episode agents choose (or are allowed) to communicate, while 'where' refers to the physical or simulated enviromment in which their messages are grounded.
Grid-world games simulate simplified 2D environments to model scenarios such as warehouse path plan- ning Blumenkamp & Prorok (022)), object movement Kolb et al. (2019), traffic management Simöes et al. (2020 Gupta et al. (2020), or maze navigation Sukhbaatar et al. (2016): Kalinowska et al. (2022). These games offer substantial design flexibility: agents may communicate with teammates while moving through the grid or act external supervisors that issue instructions. Designers can decide whether messages sent at every time step, only when certain events occur, or at sparse decision points, thereby explicitly shaping when information flows. The complexity of the environment, the observability of local neighborhoods, and the placement of communication channels together determine where messages are grounded and which spa- tial relations they must encode. Despite their popularity, grrd-world implementations vary greatly, making this highly diverse category.
Contimuuus-wol world games extend these ideas to richer physics and observation spaces, introducing greater complexity into the learning process Wang et al. (2019c): : Pesce & Montana (2020b). In emergent language approaches, such environments often support multi-task learning in which agents must both interact with the environment and develop communication policies. Contimuous 2D or 3D settings Patel al. (2021) increase realism: agents must decide when to exchange information while navigating, manipulating objects, coordinating with others under partial observability and noisy dynamics. This increased temporal and spatial complexity raises the bar for effective 'when/ when/where" /where' design but also enhances the potential for deploying emergent communication in real-world scenarios.
Taken together, grid-world and continuous environments ground emergent communication in embodied in- teraction. They reveal how the timing of messages over long horizons and the spatial structure of the world
64

---

Published in Transactions on Machine Learning Research (02/2026)
jointly inffuence what kinds of protocols agents discover, highlighting that decisions about when and where to communicate are inseparable from the tasks in which communication is embedded.
6.4.4 Discussion:
When and Where Communication Occurs
Category
Description
Representative Works
Standardized Toolkits for Controlling When and Where Agents Interact (when + where)
General frameworks and toolkits signed for building, training, and eval-
EGG: symbolic and neural agent
toolkit Kharitonov et al. (2019);
uating agents in emergent communica- tion tasks. These platforms standardize interfaces, simplify development, and encourage reproducibility.
Studies that implement with EGG, such as Chaabouni
games
(Sec. 6.4.1
et
al. (2019): Dessi et al. 2019):
Chaabouni et al. ; Auersperger
& Pecina (2022) Kharitonov al.
(2020). expand its accessible range of games and metrics;
TexRel:
compositional
image- Perkins
language benchmark
(202bb);
HexaJungle: diverse grounded com-
munication tasks Ikram et al. (2021)
Custom codebases using Py-
Torch [Etimova et al. (2017); Mu
Goodman (2021) Noukhovitch et al. [2021).
One-Shot and Itera-
Sender-receiver interaction games where the goal is either identify target, construct message, or iteratively refine communication. These variants differ in message structure, agent roles, and level of interactivity.
Signaling Game: sender encodes target,
tive Signaling Games:
receiver selects Lazaridon
Scheduling When Mes-
et al. 2016 Bouchacourt Baroni
sages Are Exchanged
2018
Denamganai et al. (2023):
(when)
Carmeli et al. (2023); Slowik al. Kang et al. (2020);
(Sec. 6.4.2
2021
Signaling game
referential
Gym
Denamganai & Walker
2020b Evtimova al. (2017
Reconstruction Game: receiver re-
constructs input Rita al. 2022
Chaabouni et al. (020)]
Question-Answer
Game:
multi-
round clarification Bouchacourt &
Baroni 2019); Das et al. (2017): ?
Temporally Extended Embodied Environments: Where Communication Grounded in Space and Time (where + when) (Sec. 6.4.3
2D spatial environments for grounded interaction, object manipulation, or path planning. These games range from
Pathfinding and object move- ment Kolb et al. (2019): Sukhhaatar
et al. (2016):
discrete to continuous settings, often used to study embodiment and commuu-
Traffic coordination and network
control Simöes et al. 2020 Cupta
nication under navigation coopera- tion tasks.
et al. (2020):
Complex planning in mazes Blu-
menkamp & Prorok (001))
Continuous 2D/3D worlds for real-
iam Wang al.
2019c): Pesce &
Montana (000b
Patel et al. 2021
Table 9: Summary of When and Where (Experimental Environment. Settings) for Emergent Communication
(Sec. 6.4
Table 9 synthesizes the three major perspectives introduced Sec. 6.4 each addressing complementary aspects of when agents exchange messages and where communication is grounded. Together, these per- spectives highlight that experimental environments are not merely platforms for implementing emergent
65

---

Published in Transactions on Machine Learning Research (02/2026)
communication systems; they actively shape the temporal structure of communication, the spatial ground- ing of observations, and the coordination pressures that determine whether communication emerges at all.
The first category, standardized toolkits, provides researchers with environments that make the notion of 'when' explicit through configurable interaction loops and message schedules, while also specilying 'where' communication anchored: whether in symbolic referential settings, image-based benchmarks, or custom grounded tasks. These frameworks lower barriers to experimentation and ensure reproducibility, but they also impose default assumptions about temporal sequencing and perceptual context that influence the emergence of language.
The second category, signaling games and their variants, directly operationalizes the when dimension through earefully designed communication protocols. One-shot signaling games define a single, fixed moment for communication, whereas multi-round question question-answer answer variants introduce iterative temporal structure, enabling agents to refine and negotiate meaning over time. These settings specify 'where' communication takes place through symbolic inputs, images, graphs, or feature vectors, but their defining characteristic is their explicit control over communication timing, which allows researchers to probe how emergent languages depend on interaction order and available rounds of exchange.
The third category, spatial and continuous environments, integrates communication into embodied lasks, making 'where' central component of the learning problem. Here, agents operate in environments where observations, goals, and opportunities for communication unfold over time and space. These settings naturally expose more complex when--o--oommnicat decisions: agents must determine not only what information useful, but when during a trajectory it becomes relevant and where within the euvironment it must be grounded. As task complexity increases from discrete grid-worlds to rich continuous domains, do the temporal and spatial dependencies that drive the emergence of structured communication.
Viewed together, the three categories illustrate that 'when's and 'where' cannot be treated independently. Temporal scheduling constrains which observations ions are available and thus where communication must. be grounded; conversely, the spatial structure of the environment determines when communccation becomes nccessary for coordination. By organizing representative environments into this unified taxonomy, Table g clarifies how experimental design choices implicitly prescribe distinct pressures on emergent communication and, in doing so, shape the languages learned by multi-agent systems.
6.5
Why and How to Communicate: Motivations and Mechanisms
Having examined who communicates with whom and what is communicated in Sec. 6.3 as well experi- mental settings in Sec. 6.4 the next foundational perspective concerns why communication is needed in multi-agent systems and how communication should be implemented to serve those needs. These two dimensions are tightly connected: the reasons agents must communicate directly influence the design of the communication mechanisms, while the available mechanisms constrain what motivations can be satisfied. Treating these dimensions jointly clarifies how communication arises not an optional feature but task-driven, efficiency-driven, and robustness-driven behavior.
Traditional multi-agent systems rely either on handcrafted communication protocols or on learned latent- vector protocols Boldt& & Mortensen (2024). Handcrafted protocols specify rigid rules for how to communicate but require significant expert effort and do not scale open-ended, dynamic environments. Automatically learned continuous communication relaxes these constraints but often produces uninterpretable, brittle rep- resentations.
Emergent communication provides an alternative: agents learn why to communicate based on task incentives and how to communicte based on the structural pressures in their environments. This subsection examines three perspectives that together explain why communication necessary and how effective communication protocols can be constructed: (1) communication for scalability general-purpose tasks, (2) communication for interpretability and structured meaning, and (3) commmication for robustness and generalization.
6.5.1 Motivations for Emergent Communication
66

---

Published in Transactions on Machine Learning Research (02/2026)
One fundamental reason agents communicate is to enable coordination in environments too complex for isolated reasoning. Communication becomes necessary when tasks exhibit partial observability, dynamic uncertainty, or heterogencous roles. In such cases, agents must exchange information to align beliefs, nego- tiate goals, or distribute sub-tasks. Thus, the why rests on the inadequacy of individual observation and planning.
To address these challenges, emergent communication rescarch investigates how communication protocols should be designed to scale across diverse situations. Open-world or open-ended environments such as Minecraft, Starcraft, or Dota 2 contain rich contextual variation, forcing agents to learn communication strategies that adapt flexibly across different states and task phases. Hierarchical or structured communi-
cation protocols in referential and navigation tasks Mul al. (2019): Li ct al. (002)); Nisioti et al. (2023
illustrate early attempls develop scalable mechanisms..
In signaling-based environments Bullard et al. 2021 Cope & Schoots 2021 Wang et al. 2002ee Tucker
et al. [021], adaptability arises from enforcing communication across diverse partners or contexts. Tech-
niques such as symbol remapping or channel randomization reveal how commuication mechanisms can be structured to avoid overfitting to a single partner or environment.
Thus, communication improves scalability (why) by providing a flexible means of exchanging situation- ally relevant information, and scalable mechanisms (how) must support general-purpose, conlext-dependent adaptation.
6.5.2 Interpretability and Structured Protocols
A second motivation for communication is the need for interpretable coordination. When agents must collaborate, monitor each other, or provide explanations to humans, messages must carry structured, mean- ingful content. Communication becomes necessary not only to exchange information but also to make that information understandable and verifiable.
The mechanisms for achieving interpretability revolve around how to constrain communication. Discrete symbol vocabularies, bounded message lengths, and structured composition promote clarity and trans- parency, reducing the opacity associated with continuous-vector protocols. These mechanisms enable re- searchers to detect when a message encodes properties such as attributes, preferences, or task intentions.
Empirical studies illustrate the interpretability benefits of symbolic communication: Jorge al. Jorge et al.) (0066) analyzed agent messages to identify semantic alignment with features such hair color. Choi al. Choi et al.] (2018) demonstrated that dynamic environments challenge stable meaning formation, high- lighting the need for structured protocols. Purity analyses Lazaridou et al. (2016) show that symbolic messages can somctimes align with human-like categorical distinctions.
In summary, interpretability motivates commuication (why) by promoting transparent coordination, and structured symbolic protocols provide mechanisms (how) for agents to produce messages that analyzable, compositional, and meaningful.
6.5.3 Robustness and Generalization
A third motivation for communication arises from the need for robust and generalizable coordina- tion. Real-world multi-agent systems encounter uncertainty, noise, new partners, and shifting environments. Agents must communicate in ways that remain reliable under such variability. Communication is valuable because it allows agents to share stabilizing information, correct misperceptions, and maintain coordination even in novel conditions.
To support robustness, communication mechanisms must prevent overfitting and encourage generalizable message structures. Techniques explored in the literature include: Channel noise injection, symbol permuta- tions, or message scrambling Cope & Schoots (2021), which force agents to rely on stable relational structure rather than arbitrary encodings; Population cycling and bottleneck architectures Wang et al. (2024e), which prevent coordination from becoming agent-specific; Protocols that enable zero-shot commmication with unseen partners Bullard et al. (2021).
67

---

Published in Transactions on Machine Learning Research (02/2026)
Thus, robustness explains why communication is essential in umpredictable environments, and mechanisms such as noise tolerance, population diversity, and structured bottlenecks explain how systems can maintain communicative effectiveness across contexts.
Challenge Category
Approach
Representative Works
Why Communication
Use of wornd, task-rich environ-
Hierarchical protocol learning in referen-
Improves Scalability and How Protocols Adapt (Sec. 6.5.1
ments to elicit adaptive and scalable communication strategies
tial games Mul et al. 2019
Grounding communication via autoen-
coders in navigation Li et al. (022
(why + how)
Autotelic reinforcement
learning for
adaptive goal generation Nisioti al.
(2023)
Dual-channel communication in negotia-
tion settings Tuccer et al. (2021)
Techniques to enforce protocol gener- ality diverse partners texts
Symbol remapping, channel randomiza-
tion Cope & Schoots (2021
Quasi-equivalence training across agent
populations Bullard et al. (022))
Why Interpretabil- ity Matters and How Structured Proto- cols Support Meaning (Sec. 6.5.2 (why + how)
Design protocols with discrete sym- bols and bounded vocabulary size to mimic human language
Grounding symbols in interpretable fea- tures (e.g., hair color) Jorge et al. 2016
Symbol purity and linguistic structure
analysis Lazaridou et al. (0016)
Zero-shot compositional lnguage assess-
ment Choi et al. 20018
Why Robustness Essential and How Communication Mech- anisms Promote Gen- eralization (Sec. 6.5.3)
Introduce agent cycling, bottlenecks, and symbol-level constraints to avoid
Population cycling and bottleneck archi- lectures Wang et al. (024c))
co-adaptation and foster generaliza- tion
Zero-shol communication with unseen
agents Bullard et al. (2021)
Channel noise injection and message
(why + how)
scrambling Cope & Schoots (0021)
Table 10: Summary of Why Communication Is Needed and How It Is Achieved in Emergent Multi-Agent Systems (Sec. 6.5
6.5.4 Discussion: Why and How Communication Emerges
Overall, Table 10 synthesizes the three foundational perspectives developed in Sec. 6.5 each addressing complementary aspect of why multi-agent systems require communication and how commmication mecha- nisms can be designed to meet these requirements. The subsection begins with (1) the scalability perspective, which explains why communication becomes essential in open-ended or task-diverse settings and highlights how agents develop adaptive protocols that generalize across partners, tasks, and environments. It then examines (2) the interpretability perspective, which clarifies why hunan-like structure desirable for trans- parency and verification, and how discrete, compositional protocols support meaningful communication. Finally, (3) the robustness perspective addresses why communication must withstand perturbations, agent turnover, and environmental noise, and how design interventions such as population cycling, bottlenock structures, or noise injection promote generalization rather than co-adaptation.
Together, these perspectives demonstrate that emergent multi-agent communication is shaped by the align- ment between communicative motivation (why agents need to exchange information) and communicative mechanism (how this exchange is implemented). Scalability motivates flexible protocol formation; inter- pretability motivates symbolic and structured message spaces; and robustness motivates architectures and training regimes that remain effective beyond the training distribution. By organizing the literature ac- cording to these why and how dimensions, Sec. 6.3 highlights how different research threads contribute constructing multi-agent systems capable of reliable, interpretable, and generalizable communication.
68

---

Published in Transactions on Machine Learning Research (02/2026)
6.6 Discussions
1990-2000
2000-2010
2010-2015
2015-Present
Early Language
Emergence of Statistical Learning
Deep Learn- ing Revolution
Games & Hand-
Modern EL Research
crafted Simulations
Basic Signal-
Supervised Learning
Neural Networks MARL.
Advanced
ing Games
Frameworks
Foundational Theory
End-to-Fnd tend--dn Training
Human-like Language
Learning Models
Nowak Krakauer
Lazaridou al. (2016)
Chaabouni et
2020 ;
Steels
1195
1997
1999
Kirby 2002
Havrylov & Titov (2017
Mu & Goodman
2021
Figure 9: Evolution of Emergent Language Research: Timeline showing major milestones, approaches, and
achievements
Performance Metrics
Language
Generalization
Properties
Task Suc- Rate
Zero-shot
Compositionality
Performance
Communication Efficiency
Topographic Similarity
Crose-domann Transfer
Coordinstion Quality
Vocabulary
Partner Adaptation
Statistics
Lazaridou et al. 2016):
Brighton & Kirby
2006
Bullard et al. 202
Mordatch & Abbeel (0088
Chaabouni al.
2020
Cope & Schoots (0021
Figure 10: Taxonomy of Evaluation Metrics in Emergent Language Research
6.6.1
Who Communicates with Whom and What Is Shared
The first set of considerations in emergent communication concerns who participates in communication, whom messages are directed toward, and what information agents are capable exchanging. As detailed Sec. 6.3 population size, role structure, cooperation level, and message design jointly determine the topology and informational bandwidth of the communication process. These assumptions specify the communicative space in which protocols emerge.
Fig. Iillustrates how research has gradually expanded these foundational assumptions. Early signaling- game studies focused primarily on dyadic, role-specific communication and symbolic vocabularies. Subsequent neural approaches introduced richer architectural mechanisms (CNNs, RNNs, attention) that broadened what can be communicated. Multi-step protocols and expressive vocabulary choices further shaped whom messages target and how information structured across conversation.
69

---

Published in Transactions on Machine Learning Research (02/2026)
Evaluation practices (Fig. 10 matured alongside these developments. Metries such compositionality, topographic similarity, and vocabulary statistics emerged as tools for quantifying the structure and semantic content of messages, offering insights into what agents communicate and whether resembles human-like abstractions. These foundational who whom what dimensions form the basis on which more dynamic aspects of communication are built.
6.6.2 When and Where Agents Communicate
The second dimension concerns the timing and situational context of communication, addressing when com- munication is useful and where in an environment or task structure communicative bottlenocks emerge. Sec. 6.4 shows that emergent language is highly sensitive to the structure of the experimental setting. Com- munication does not arise uniformly; it appears in moments where coordinated behavior depends infor- mation that cannot be inferred from local observations passive cues.
Referential, reconstruction, question question-answer, grid-world, and continuous environments each create different temporal and spatial opportunities for communication. Multi-round dialog games enable clarification over time, while continuous control tasks require communication during navigation or manipulation. Grid-world tasks depend spatial grounding, constraining where information must be shared for success. These design choices influence not only the availability of communication but also its necessity and strategic value.
The historical development shown in Fig. reflects this progression from static symbolic scttings toward dynamic, embodied environments where timing, location, and partial observability determine when commu- nication becomes advantageous. Effective evaluation therefore requires metrics that capture performance across varied environmental contexts and task structures, organized in Fig. 10
6.6.3 Why Communication Emerges and How It Is Learned
The third dimension captures the functional drivers behind emergent communication: why communication is beneficial in task and how agents implement communicative strategies that lead to improved coordination, generalization, robustness, discussed in Sec. Communication emerges when independent decision- making is insufficient to solve open-ended or high-uncertainty tasks, when agents face partner variability or noise, or when concepts must. be abstracted and shared to achieve stable coordination.
The how dimension encompasses architectural and algorithmic mechanisms that support communication. These include discrete vocabularies, message bottlenecks, altention modules, agent cycling, and multi-round negotiation schemes. Such mechanisms influence the emergence of interpretability, enabling messages to develop structure similar to natural language. They also contribute to scalability and robustness; for example, population eycling or noise injection prevents co-adaptation and strengthens generalizen lization across agents and environments.
As summarized in Fig. 10 evaluation metrics for these functional properties have expanded beyond task success to include compositionality, partner adaptation, zero-shot gencralization, and cross-domain transfer. These metrics reflect the need to assess not only whether agents communicate but also whether the resulting protocols provide benefits that continuous or handcrafted systems cannot. Identifying benchmarks where emergent communication clearly outperforms such basclines remains an important open direction.
6.7 Bridge: From MARL-Comm and Emergent Language to LLM-Based Communication
While emergent language rescarch addressed key limitations of MARL-basod communication notably interpretability and discrete structure it introduced new challenges related to scalability, generalization, and semantic richness. Emergent communication protocols are typically learned from scratch through task- driven interaction, which makes them sensitive to training environments, agent populations, and reward structures. As result, many emergent languages struggle to generalize across tasks, adapt to unseen partners, or support open-ended reasoning beyond narrowly defined coordination problems.
At the same time, MARL-based communication continues to face challenges in interpretability, verification, and integration with human-centric decision-makkin pipelines. Continuous or latent message spaces remain difficult. to audit align with human semantics, limiting their applicability in safety-critical human-
70

---

Published in Transactions on Machine Learning Research (02/2026)
Table 11: From MARL-Comm and emergent language to LLM-based communication:
limitations that
motivate LLM-Comm.
Aspect
MARL-Comm tions
Limita-
Emergent Language Lim- ittiions
Motivation for LLM- Comm
Semantic struc-
Latent or continuous sig- nals;
Discrete symbols;
Rich,
compositional
ture
Limited expressiveness.
language:
Semantics hard inter- pret.
Human-alignn seman-
tics.
Generalization
Jointly trained agents; Poor transfer across lasks.
Populacion-specffic
lan-
Strong cross-task gener- alization; Pretrained prior.
guages;
Zero-shot failures.
coordination
linguistic
Scalability
Communication cost grows
language from scratch is costly.
Learning
Reusable language rep- resentations;
with agents.
Scales
across
agent
roles.
Human
align-
Hard to interpret or audit.
Limited grounding in nat- ural language.
Natural interface for hu-
ment
mans;
explanation and oversight.
Supports
Core gap
Task-optimized but opaque (MARL); Interpretable but brittle and narrow (EL).
Open-ended, reusable communication sub- strate.
in-the-loop settings. Together, these limitations reveal a gap between lask-optimized communication and semantically rich, reusable language.
LLM-based multi-agent communication emerges as response this gap. By leveraging large-scale pre- training natural language, LLMs provide agents with strong semantic priors, compositional structure, and the ability to generalize across tasks and domains without task-specific retraining. Unlike emergent language systems, LLM--basd agents do not need to rediscover linguistic structure through interaction, and unlike MARL-based communication, their messages are inherently interpretable and human-aligned. These properties motivate the growing adoption of LLMs as communication backbones for coordination, planning, and reasoning in multi-agent systems. Table 11 summarizes how limitations in MARL-Comm and EL-Comm jointly motivate the shift toward LLM-based multi-agent communication.
7 LLM-Powered Multi-Agent Communication
Large language models (LLMs) demonstrate strong performance across domains due to training large, diverse corpora, which yields emergent commonsense reasoning and robust human language understanding and generation capabilities Liang et al. (022)). Because human language is general and flexible medium for representing knowledge and intent, it provides a natural interface for reasoning, coordination, and information exchange. Building on these capabilities, LLMs are increasingly deployed as autonomous agents that plan, reason, and act, and more recently as teams of agents in multi-agent systems (MAS) that communicate via natural language. Language-baser based communication enables agents to coordinate and collaborate in an interpretable, task-agnostic manner without Hand-crafted protocols, supporting flexible interaction patterns and broad generalization Guo et al.
(2024a); Liu et al. (2025b). Despite challenges such as hallucination and multimodal LLMs further euhance robustness by integrating language (2024h. Compared traditional RL agents with fixed protocols, LLM- based agents offer general communication framework, and hybrid LLM-RL. approaches seek to combine
limited grounding Ahn et al. (2022).
with sensory inputs Durante al.
71

---

Published in Transactions on Machine Learning Research (02/2026)
linguistic reasoning with cettrooment-grouunde learning Sun al. (2024a]
Wu et al (2022)
Feng et al.
h024h.
LLM/
Decision Making
Memory/ Context Ma nagemeent
API
Tools
Topology:
Communiestion
Who to
Agent Components
Agent Motivations:
Strategy what communicate
Chain Topolagy
Nested Hierarchical
Structure of Communication:
What and how to communicate
Dynamic Policy
Agent
Configuration:
coordination among heterogenous
Decantralizoo ized Topology
Centralized Topology
Hateroga nous Agents
Multi-Tum Communication
agent
Polley:
LLM-Based Multi-Agemt System Design
Static compared
with dynamic
stratogies
</>
Challenges
Applications
Figure 11: LLM-Based Multi-Agent System Design and Challenges. In an LLM-based multi-agent system, each agent outputs decisions that can be enhanced through memory, context management, and the use of domain-specific tools. Agents communicate through different topologies and can operate in either static or dynamic configurations, with applications spanning diverse domains. Building functional multi-agent systems remains challenging due to numerous design considerations, and several key challenges are summarized.
Review Scope and Structure As interest in L...-powered multi-agent systems grows, communication has become central organizing theme. Natural language provides unified interface for agent agent-cnvvronment interaction, agent agent coordination, and human agent collaboration. This section reviews LLM-powered multi-agent communication by first introducing the foundations of LLMs and LLM-based agents (Sec- tion 7.1 then systematically organizing prior work around who communicates with whom and what is shared (Section 7.2 when and where communication occurs (Section 7.3 and why and how communica- tion is structured (Section 7.4p. We further discuss these dimensions in Section 7.5 and conclude by bridging LLM-based communication with multi-agent reinforcement learning, highlighting emergent communication patterns in LLM MARL hybrid systems and their limitations in decentralized settings (Section 7.6). Fig. provides high-level system view, illustrating how LLM agent components, communication topologies, agent configurations, and policies jointly shape multi-agent system design and challenges, and Fig. 12 presents a detailed taxonomy that categorizes existing work across communication topology, motivational setting, agent heterogeneity, and communication structure. We conclude by bridging LLM-based communication with multi-agent reinforcement learning, highlighting emergent patterns LLM-MARI. hybrid systems and their linitations in decentralized settings.
72

---

Published in Transactions on Machine Learning Research (02/2026)
Chain
Zhang
Hia
Raiis et
light
Kamnah
Wane ahreles
20:3 Cher
2026
Hantet di 1
Decentralized
(232) Nare
(inner Ef 2004
P00 (Npt vorato ar
Topology
2074 21 al
PRO at
In ot Pioth
Chan et
2024
Cong
No
Hors
Centralized
Che
2023
2204
Lnrst at
at A0X
Karnan at
Founty (000
Wung
20240 Chap
2024
Findnn
Hierarchical
Unrdin (204 (dnza levsa a chaarir 2 Vastmum
Resd Heaer (222) Mu (eoz5a Wer Aoole
Senti (101 Sun (irbe
Thanng (iopirs) S lnto at pirte
HH
Shared Message
ploi Hemoord Nithmar
Who/W What Comm.
List
(ien 20244)
Heterogencous
dintro commeral Priliein kiikw
2004
Wybb Caan liiiee giida at
Agent Coufguration
Chro rf (X94)) M0110 Swint] (107)
of PiPh Sin
Fan at (ossh TT (201)) 1 [2hing at 70730
Homogenous
Chen et
Gang
2054
Hi
2020
atal pins
Sun
Aradhr
120 Pir
Kn
laau Soabuu piota (Caml
Cooperative
[2:25]
(dhicee mi 2. benmmn 2104
2023 [Cher
Joven Mime Mimst &el
Mo Li rerpliia ln mnse
Rase Pin Pazzh Mashhien meng
41 a (2025) Uhhi oi (Apa0l (or & (asa Mure Area
LLM-Comm Multi-Agent Systems
2023
Wang
When/ Where Comm.: Motivations
Bails
2004
(Wagg 200th (Lght
2023
Du al
2023
Competitive
Laly (ols Liba (0220
Mblennat
di Aol
Lan (203) For A35
Chen
Aboelnat
2024
Mixed-motives
punn of
Tourtey Pereere at
Senty (ere
Au (2220)) (Prg of Aoiia de
Arri) (Chin
Multi-turn
linii et ((00]]] (i a
Elariel st 2225 Morn Coneei Misioo Rocaane
2 Zhara (200) Lnn Fi Lre Nost Wren
Interaction Length
log 2
Why/How to Comm. Structure Communication
Single-turn
2026
ct
Static
Protocol Adaptivity
Park
Mrotth
2004 Pori
al (200g) (ung
Dipto MottoWW Wi
Kunz 21 2iiz2zz iitin
1053 01 Peiit co 2e 10:4 Modiwa
7hent 2. 2lr5 1 Tron at Goeea Line at de BUbR Mamt
e
Dynamic
Surney
Figure 12: LLM Agent Communication Taxonomy
73

---

Published in Transactions on Machine Learning Research (02/2026)
7.1 Background and Foundations of LLMs and LLM Agents
7.1.1 Large Language Models (LLMs)
LLMs have emerged as transformative tools for agent- based systems, offering advanced capabilities such as reasoning, planning, and natural language understanding. These capabilitics arisc not from explicit program- ming but from the emergent properties of training on large-scale, diverse corpora Liang et al. (2022). When deployed as decision maaiing making agents, LLMs demonstrate robust adaptability in dynamic, multi-agent environ- ments by enabling flexible coordination and communication beyond rulc-based constraints: (1). Emergent Behaviors. LLMs display emergent abilities such as strategic planning, abstract reasoning, and social simulation that were not explicitly encoded during training. These behaviors empower agents to engage in high-level tasks including negotiation, cooperative problem-solving, and context-aware decision-making. In multi-agent systems, such behaviors are particularly valuable for fostering robust coordination strategies in unpredictable seltings. (2). In-Context, Few-shot, and Zero-shot Learning. LLMs are capable of adapting to new tasks via in-context learning, interpreting task instructions and examples directly from the prompt without parameter updates. This ficxibility extends to fow-shot and zcro-shot scttings, where agents can perform novel tasks using minimal or no prior training dala. Such capabilities significantly reduce development overhead and enable agents to operate in open- open-ended, evolving environments with task-agnostic generalization. (3). Natural Language Communication. The ability to understand and generate human language makes LLMs ideal for facilitating interpretable communication within multi-agent systems and between agents and humans. Agents can articulate goals, strategies, and rationales using natural language, improving both inter-agent alignment and human oversight. This is critical for collaboration, transparency, and trust in mixed-agent teams. (4). General-Purpose Reasoning Across Environments. Impor- lantly, LLMs are not pre-trained to optimize behavior for any specific enviromment (e.g., a particular game or graphical interface). Instead, they serve as general-purpose cognitive engines that can be fine-tuned or scaffolded to suit domain-specific requirements. This property supports reusability across diverse agent applications from GUI-based assistants to strategic game agents while maintaining task flexibility. In summary, the emergent reasoning, flexible learning paradigms, and language-driven communication of LLMs establish them as foundational components for next-generation multi-agent systems. By leveraging these attributes, agents can dynamically adapt to new objectives, collaborate through interpretable dialogue, and function effectively in complex, uncertain environments.
7.1.2 LLM Agent
An LLM-based agent is a modular system that integrates reasoning, memory, and task execution around a central language model Durante et al.] (2024): Wang et al. (2024a). As illustrated in Fig. 13 these agents are structured into four core components, each contributing to their ability to act coherently and adap- tively within dynamic environments: (1). Profile. This componen encodes the agent's identity, personality traits, and social attributes. Profiles influence interaction style and decision preferences, allowing agents to simulate diverse roles or personas. Profiles can be handcrafted, learned, or aligned with external datasets. (2). Memory. Memory systems enable agents to persist and retrieve historical information, such as past interactions or task outcomes. Implemented as embeddings, text logs, or databases, this component sup- ports long-tere term coherence and task continuity through read/write operations and reflective reasoning. (3). Planning. The planning module supports decomposition of goals into actionable steps and the formation of strategies to reach those goals. Plans may be iteratively refined based on agent self-evaluation, feedback from other agents, or environmental input, enabling responsiveness to dynamic task contexts. (4). Action. This component operationalizes plans into concrete actions. Actions may involve environmental manipulation, communication, tool usage, or internal reasoning. By integrating outputs from memory and planning, this module allows the agent to execute tasks effectively and adapt behavior in real time. Together, these compo- nents form cohesive architecture for LLM-based agents capable of multi-turn reasoning, human interaction, and decentralized coordination. This modular design facilitates extensibility and customization for a broad range of multi-agent settings.
7.2 Who, Whom, and What to Communicate
74

---

Published in Transactions on Machine Learning Research (02/2020)
Working memory
Long-ter memory
</>
Perception
T/F
Coding
Human
Action
Agent
Environment
Response
Tools
Figure 13: General Architecture of LLM-Based Agent. The figure depicts how an agent interacts with coding, human, virtual, and embodied environments through perception and action loops. Within the agent, customized modules can be employed to handle memory, decision making, response generation, and tool use, enabling flexible and adaptable performance across domains.
Communication in multi-LLM agent systems introduces challenges that differ substantially from those in traditional MARL or emergent language settings. Unlike gradient-trained policies, LLM agents communi- cate through open-ended natural language, causing issues such as ambiguity, verbosity, hallucination, uncontrolled information flow, and difficulty coordinating roles among heterogeneous agents. These problems often arise because the communication structure is under-specified: it is unclear who has the authority opportunity to speak, whom each agent should address respond to, and what information ought to be ex- changed for effective coordination. By explicilly analyzing who, whom, and what in communication design, we can systematically constrain interaction patterns, reduce uncertainty, and promote coherent multi-agent bchavior. Concretely, these dimensions help address several core challenges: (1) uncontrolled or redundant message generation, (2) misalignment between speakers and intended recipients, (3) information overload omission due to unclear message content, (4) degraded coordination in large teams with overlapping responsibilities, and (5) failure to maintain shared context or stable interaction protocols. The following subscctions examine how communication topology, agent-rolc configurations, message structure and communication framework each operationalize the who-whom- hhatt axes, and how these design choices help mitigate the fundamental challenges outlined above.
7.2.1 Communication Topology and Interaction Structure
Communication topology the primary structural mechanism through which LLM-based multi-agent sys- tems define who allowed or expected to speak, whom they address, and conscquently what information is most relevant for exchange. Unlike MARL, where communication edges often reflect spatial adjacency bandwidth constraints, and unlike emergent language settings where agent roles are fixed by the game structure, LLM-MAS topologies are logical and design-driven. They impose organizational scaffolding that prevents free-form natural language communication from becoming chaotic, redundant, or misaligned with lask demands. Below, we synthesize five major topology classes (Figures 14 18 and analyze how each governs who- interaction and influences what gets communicated.
Chain and Pipeline Topologies (Fig. 14 describes a linear communication pattern where each agent communicates exclusively with its immediate neighbors. This structure is particularly suited for pipelines or sequential workflows, ensuring messages flow in an organized manner. In sequential communication, agents operate in step-by-step chain, where each one handles part of the task and passes the result
75

---

Published in Transactions on Machine Learning Research (02/2026)
Figure 14: Chain
Figure 15: Decentralized
Figure 16: Hierarchical
Figure 17: Centralized
Figure 18: Shared Message Pool
to the next Zhang al. (024cc. In this fixed pipeline, the who agent may listen to is its immediate predocessor, and the whom it may speak to is its immediate successor. This rigid pairing shapes what is communicated: agents must pass only the intermediate artifact or reasoning needed by the next stage, and cannot rely on global context or cross-checking from non-adjacent agents. This setup breaks down long or complex problems into smaller parts that are easier to process. Some systems follow this pattern for multi-step workflows such as software development, with different agents handling design, coding, testing, and review in a clear sequence Qian et al. (2024a); Du et al. (2024). In these systems, the chain explicitly resolves the who talks to whom question by binding each stage to a single upstream provider and single downstream consumer, which reduces communication redundancy but also restricts what information can be transmitted, typically structured specifications, code drafts, or test results formatted for the next specialized agent. Compared with decentralized or shared-memory approaches, these chain-based workflows improve modularity and control but limit opportunities for global coordination corrective feedback, making them more susceptible to cascading errors when early-stage outputs are flawed.
Decentralized topology (Fig. 15 Decentralized topologies permit agents to communicate directly with multiple peers without centralized coordination, forming fully or partially connected interaction graphs. Unlike chain or hierarchical designs where who talks to whom is largely fixed by execution order or role assignment, decentralized systems allow communication partners to be selected dynamically based local decisions, strategic incentives, or environmental context. This flexibility enables richer and more adaptive interaction patterns, but it also shifts the burden of coordination, consistency, and filtering onto the agents thersselvs.
A first class of decentralized systems arises social and strategic game-based environments, such as Werewolf Bailis al. (2024), Avalon Wang (2023b), and related role-playing settings Xu et al. (202bb) Hua et al. (2023); : Light al. 20233 Kannan et al. (2023) y Lan et al. (2023) Chen et al. (0024a). In these settings, the who talks to whom pattern is shaped jointly by explicit game mechanics, such as turn-taking, voting rounds, and phase transitions, as formalized in Avalon- and Werewolf-style benchmarks Light et al. 2023 Bailis et al. (2024] ; Wang et al. (2023b), and by implicit social roles or personas (e.g., villagers, ad- versaries, or agents with special abilities) that condition agents' communicative incentives Xu et al. (2023b): Hua et al. (2023): Lan
(0023); Chen et al. (2024a). Within these settings, agents actively choose whom to address when accusing, defending, or probing others, docision shown to be central to strategic influence and belief shaping Xu et al. (202bb); Kannan et al. 2003). This partner selection directly determines what is communicated: agents generate persuasive arguments, deceptive narratives, and explicit belief statements to sway group decisions or obscure private information Hua et al. (2023); Bailis et al. (2024); Chen et al. (2024a). Across these works, consistent trend emerges: decentralized interaction enables intention-driven and socially expressive communication, but it also introduces higher ambiguity and vulnerability to misinfor-
76

---

Published in Transactions on Machine Learning Research (02/2026)
mation, as conflicting narratives persist without central mechanism for reconciliation verification Light
et al.
p023;; Wang et al. (002bb); ;; Lan et al. (023).
A sccond category consists of debatc-style and critique-basdd frameworks Li ct al. (2024f): Jin et al. (2024); Chen et al. (2023b); Du el al. (2023); ; Liang et al. (2023); Chan et al. (2023). which typically adopt fully connected communication graphs in which any agent may address any other. In systems such as LLM-Debate and related variants Du et al. 20233] Liang et al.
2023), who speaks to whom is governed by explicit debate protocols (e.g., challenger versus defender), while judge-based or roconcllation-orientt designs further structure interactions through evaluator roles Li et al.
(2024] Chen el al. (023b)). In these settings, what is communicated is explicitly argumentative, including critiques, counterexamples, evidential claims, and meta-rcasoning steps [Jin et al. (2024); Chan et al. (2023). Empirical evaluations consistently report improvements in reasoning quality and error detection relative to single-agent baselines, demonstrating
the benefit of decentralized peer critique for complex decision-maken making Du et al. (2023): Liang et al.) (2023)
Chan et al. (023)). At the same time, these works expose a key limitation of decentralized debate designs:
without carefully designed turn-taking rules, aggregation mechanisms, or stopping criteria, communication
can become repelitive or diffuse Li et al. (2024]) Chen et al. (023b), and becomes difficult to attribute performance gains to specific who whom interactions or individual argumentative contributions.
In contrast, embodied and task-grounded decentralized systems Ma et al. 2023 Zhang et al (023b
Wu et al. 2025); 1 Kannan et al. (2023); Liu et al.] (2024a) impose implicit structure on who talks to w whom through physical proximity, complementary roles, or local task dependencies. For example, agents coordinate primarily with spatially nearby peers or role-complementtry partners in embodied planning and navigation tasks Ma et al. (2023): Zhang et al.] (2023b). while multi-robot and tool-using systems often restrict com- or porreptiio-actionn coupling Kannan et al. (2023); ; Liu et al. (2024a). This partner-selection mechanism induces sparse but functionally meaningful communication graphs, reducing redundancy while preserving task relevance. As a consequence, what is communicated in these systems is lightly grounded in the environment. and task dynamics, typically consisting of spatial descriptions, action proposals, affordance information, or lo- cal state updates needed for immodiate coordination Ma et al. (2023); : Wu et al. (2025). Compared with social-deduction or debate-based decentralized settings, these systems deliberately trade expressive richness and open-ended dialogue for clearer grounding and tighter coupling between messages and environment dy- namics Zhang et al.] (2023b); Liu et al. (2024a). Empirically, this design yields more predictable and stable behavior, but evaluations are often limited to smaller numbers of agents and narrower task distributions, reflecting a scalability grounding trade-off inherent to lask-grounded decentralized communication.
munication links based on functional dependencies such as
Finally, large-scale agent societies and general-purpose decentralized frameworks Park et al.
Carroll et al. (2019)
Kaiya et al. 22023
Nascimento et al. (023bb Piatti et al. (0024) Zhang et al.
2024b a); Liu et al. (2024b) Li et al. (2024a];
Xiong et al. (2023): : Nascimento et al. (002a)): D D'Arcy et al.
b0244: Abdelnabi et al. 22024)) ; Li et al.
(2023c)
; AL et al. (2024): Liu et al. (2023b) examine communcaatio
at population scale, where who talks to whom is no longer fixed by task structure but emerges from learned utilities, social ties, or cultural and based signals. For instance, agent- based social simulations and gen- erative societies allow agents to initiate communication based perceived social relationshis, memory, or long-term preferences Park et al. (2023); Nascimento et al. (2023b a). Other systems explicitly model inter- action incentives or utility-driven partner selection, leading agents to contact peers that maximize expected payoff or coordination benefit Carroll et al. (2019); Piatti et al. 2 Abdelnabi et al. (2024). In these settings, what is communicated extends beyond immediate observations or actions to include recommen- dations, shared norms, role expectations, and high-level strategies that regulate collective behavior Li et al. (2024a); Zhang et al. (2024b ab. Several works demonstrate that such decentralized exchanges can give rise to emergent division of labor, social conventions, and adaptive group-level organization without centralized control Kaiya et al. 20233: Liu et al. (2024b): Li et al. (2023c); | [L et al. (2024). However, empirical analy- ses also reveal limitations unique to population-scall decentralization: communication becomes increasingly noisy or redundant as the number of agents grows, causal attribution of individual interactions becomes difficult, and system-level evaluation requires aggregate statistical metrics rather than trajectory-level
analysis Xiong et al. 2023 D'Arey et al. (024).
77

---

Published in Transactions on Machine Learning Research (02/2026)
Overall, decentralized topologies consistently enable flexible, context-dependent who-whom selection and richer what content than rigid pipelines or hierarchies. At the same time, the literature reveals a common design tension: greater expressive power and adaptability come at the cost of increased ambignity, coordi- nation overhead, and stabilization difficulty, motivating the use of filtering, aggregation, or hybrid control mechanisms in practical systems.
Hierarchical topology (Fig. 16 structures communication into layers, assigning distinet roles to agents: higher-level agents handle planning, abstraction, and task allocation, while lower-level agents are specialized for executing subtasks or domain-specifie actions. This design enables modularity and scalable coordina- tion by decomposing complex tasks. However, effective use relies on strong role specialization, coherent coordination across layers, and a shared understanding of task structure and progression.
Across hierarchical systems, the who and whom of communication are intentionally predetermined: su- pervisors decide who speaks, whom instructions target, and the structure of these interactions strongly constrains what is communicated. In multi-layer pipelines such as Magentic-Oni Fourney et al.) (2024), MegaAgent Wang al. (2024b), AutoAct Qiao et al. (2024), and CGMI Jinxin et al. (2023), higher-level agents issue abstract plans or lask recommossiiions; while lower-level agents respond with execution details or local updates. These works explicitly encode a downward flow of what: plans, constraints, and goals; and an upward flow of what: progress reports, clarifications, or exceptions. Compared with decentralized systems where who talks to whom emerges dynamically, these approaches eliminate ambiguity about communication partners, reducing coordination overhead and yielding more reliable task adherence.
Role-based hierarchical simulations further reinforce how fixed who talks to whom mappings shape what is exchanged. In hospital simulations Li et al. (2024c), legal-service workflows Sun et al. (2024b), and AutoMI. pipelines Trirat et al. (2024); Du et al. 2024h, agents are embedded in organizational structures mirroring real-world institutions. Here, who communicates is dictated by job role, and whom they address is defined by institutional hicrarchy, c.g., doctors report diagnoscs to coordinators, analysts return model results to pipeline managers. Consequently; what becomes highly structured: domain-specific evidence, pre- dictions, summaries, or structured updates. These systems improve reliability and interpretability compared to decentralized topologies, which generate more diverse but less predictable messages.
General-purpose hierarchical environments such as Agent Verse Chen et al. (023c)) and related agent orches- tration frameworks Ishibashi & Nishimura (2024); Rasal & Hauer (2024): Xu et al. (023a) extend hierar- chical communication beyond fixed pipelines toward adaptive teaming and organizational alignment. These systems emphasize explicit role assignment and supervisory coordination, showing that hierarchy is particularly effective when tasks require structured decomposition, global oversight, or consistent execution standards across agents Chen et al. (2023c); Zhhng et al. (2023). Several works further demonstrate how hierarchical control supports reliability and scalability in complex workflows. For example, nested or super- visor- worker designs Ahn & Sentis (2021): Wu al. (2024a) and multi-agent orchestration frameworks [Sun| et al. 2025c Guo et al. (2024c) highlight that fixing who talks to whom simplifies coordination and sta- bilizes what is communicated typically structured plans, constraints, summaries yielding predictable system behavior across long horizons. At the same time, these studies consistently reveal a key limitation of hierarchical communication. Because interaction patterns are rigidly defined, the space of what agents can express correspondingly constrained, limiting lateral information sharing, peer-to-peer critique, and creative recombination of ideas Xu et al. (2023a); ; |asall & Hauer (002)). As a result, hierarchical topologies are less effective in scenarios that benefit from spontaneous collaboration or exploratory reasoning, where decentralized or shared- posseeepos designs allow richer and more diverse communication strategies. Taken together, this line of work clarifies a fundamental trade-off: hierarchical systems favor predictability, control, and interpretability at the cost of expressive fiexibility. Compared with decentralized designs, they often achieve higher reliability and task adherence, but lower diversity in communication behaviors and reduced capacity for emergent, peer-driven reasoning.
Centralized topology (Fig. 17 involves a single central agent coordinating communication among all agents. This centralized coordinator manages information flow, assigns tasks, and synchronizes actions, simplifying coordination and promoting consistency, particularly beneficial in large-scale deployments.
78

---

Published in Transactions on Machine Learning Research (02/2026)
Centralized systems make the who and whom dimensions explicit: the central planner is always the who that initiates communiction, and all subordinate agents are the whom it addresses. This fixed mapping strongly constrains what is communicated. For instance, in Scalable Multi-Robot Collaboration Chen et al. (002g), the central controller allocates roles and synchronizes multi-robot teams, meaning the what passed downstream consists of structured task assignments and coordination directives, while the what returned upstream is limited to progress reports or sensor summaries. Compared with decentralized schemes where agents independently determine partners and message types, this reduces message diversity but substantially increases predictability and task adherence.
A similar pattern appears in MindAgent Gong et al. 2023) where a central manager orchestrates gaming agents by distributing observations and ensuring coherent actions. Because the who talks to whom mapping is fixed, agents rarely need negotiate or infer partners; instead, what they communicate is shaped by managerial demands: local states, strategy suggestions, execution feedback. These systems highlight both the strength and limitation of centralized topologies: despite efficiency and coherence, adaptability drops sharply if the central agent misinterprets context or becomes overloaded.
Centralized coordination is also widely used in general-purpose L.LM--AS frameworks including AutoGen, (023):Cheenee
MetaGPT, AutoAgents, and OpenAgents Wu et al. (2023b); Hong al. (2023); Chen et al. (2023a) Xie et al. (003)) In these systems, the central planner decomposes high-level goals into subtasks, selects specialized agents, and aggregates outputs. Here, the who is the planner generating decomposition instructions; the whom are the skill-specific agents; and the what becomes a structured interchange of plans, execution traces, and tool outputs. Because all communication flows through the planner, these frameworks gain strong global oversight but sacrifice robustness under noisy or ambiguous downstream messages.
Similarly, workflow-automation and system-integration platforms such Magentic-Onn AutoMI--Agent, and AgentCoort Fourney et al. (022)) ; Trirat et al. (2024); Pan et al. (2024) embed a centralized controller to schedule subtasks, manage dependencies, and enforce execution order. Compared with hierarchical topolo- gies, where information flows along multiple levels, these systems collapse hierarchy into a single locus of authority, simplifying who talks to whom but increasing pressure on the central agent to correctly inter- pret and route what. This concentration improves coordination efficiency but also narrows the expressive bandwidth of messages since agents tailor what they report to the controller's expectations.
In embodied and strategic game environments Kannan et al. (023); Gong et al. (2023) centralized over- sight ensures consistent high-level goals and prevents divergence among agents operating in complex spatial or adversarial settings. Yet this reliability comes at the expense of flexibility: unlike decentralized topolo- gies where agents dynamically choose whom address and decide what to share based on local needs, centralized systems impose rigid communication funncls that may bottleneck under high task complexity.
Overall, centralized topologies contrast sharply with chain, hierarchical, and decentralized structures: they minimize uncertainty in who talks to whom mappings and thus constrain what to well-defined task primitives, enabling consistency and fault detection at the cost of adaptability and emergent coordination. Their effectiveness therefore hinges on the competence and reliability of the central agent, which becomes both the enabling factor and the primary point of failure.
Shared message pool topology (Fig. 18] employs a global broadcast channel or shared repository where agents asynchronously publish messages and retrieve relevant information. This topology provides flexibility and loose coupling, but may face challenges such as message overload or latency in high traffic scenarios.
Unlike chain or hicrarchical topologics, where who talks to whom is specified by fixed edges, the shared message pool removes explicit addressing entirely. Instead, agents broadcast to a common memory and retrieve whatever they judge to be relevant. This indirection strongly shapes what is communicated, since messages must be self contained, interpretable, and useful to recipients that may not be known beforchand.
Dialogue based collaborative systems such as CAMEL Li et al. (023b), AutoGen Wu et al. 2023b Agent- Verse Chen et al. (2023c), and Reconcile Chen et al. (2023b) illustrate this pattern clearly. Because agents cannot target specific partners, the who is any agent contributing to the shared buffer and the whom is every agent capable of reading from it. As a result, what tends to include rationales, intermediate reasoning traces, and shared task context that can support the work of diverse collaborators. These systems improve
79

---

Published in Transactions on Machine Learning Research (02/2026)
robustness compared with chain topologies, since any agent can join or leave without breaking the communi- cation path. However, they suffer from ambiguity about which messages are relevant to which agent, which creates a need for retrieval heuristics and memory pruning.
Graph based coordination syslems Hu et al. (2024c); Jeyakumar et al. (2024); Yang el al. (2025) use a structured shared memory such as a knowledge graph. Here, the who an updating agent and the whom is the global state. Consequently, what agents communicate takes the form of structured updates, such node proposals or constraint asscrtions. This improves scalability and creates a more interpretable global representation, but requires all agents to align on schemas and update rules, which reduces the flexibility of free form natural language messages.
Task-oriented collaboration frameworks such as CommAI Chen et al. (2224)), MapCoder Islam et al. (0024)
MACRee Wang et. al. (2024T), AgentCoord Pan et al. (2024), and MedAgents Tang et al. (2023) rely on shared repositories where agents publish plans, intermediate results, or tool outputs. Since no explicit whom is specified, what is communicated must be anticipatory and modular so that heterogeneous agents can reuse information. Compared with decentralized debates, these systems reduce message conflict and support higher levels of parallelism, but can suffer from coordination bottlenecks if too many agents produce unfiltered content.
Emergent organization approaches such as Theory of Mind agents and self organizing societies Li el al. [2023c;; Ishibashi & Nishimura (2024) use the message pool to mediate adaptive roles and norms. Because the who and whom relations are fluid, what includes meta level information such as inferred intentions or social expectations. This enables rich emergent behaviors but also increases the risk of noise and instability.
Relative to decentralized topologies, where who selects whom, the shared pool simplifies coordination by allowing fully implicit partner selection but removes partner specificity. Relative to hierarchical structures, where what is shaped by manager worker relations, the shared pool encourages more symmetric communi- cation but provides less control over message relevance. The eveectiveness of this topology relies on relevance filtering, memory organization, and the ability of agents to prioritize among many concurrent messages.
7.2.2 Agent Configurations and Communication Capabilities
ALM-based multi-agent systems can operate with either heterogeneous or homogeneous agents. These config- urations determine how the system specifies who communicates with whom and what types of information are exchanged. Figures
19 and 20 illustrate these two settings.
Figure 19: Heterogeneous
Figure 20: Homogeneous
Heterogeneous agents In heterogencous systems, agents possess different roles, skills, knowledge sources, or identities. This diversity allows the system designer to specify who interacts with whom based on functional dependencies and also shapes what is communicated, since messages must be tailored to the needs of particular recipients. Many role based multi agent systems rely on this structure. Works such as
CAMET Li et. al. (2023b), ChatDev Qian et al. (2024a), social reasoning agents Xu al. (2023b): | Hual
et al.
2023 ; Lan et al. (2023): Park et al. (2023); Chen et al. (202cc); ; Li et al. (2024d); Ghafarollahi &
Buchler (2024g Cheng al. [2024);
; Chen et al. (2024ch, embodied collaborators Wu et al. (2025)
Stepputtis
et al.
(202)) Hu et al. (2024a); Cheng et al. (2024), and multi stage task teams Fourney et al.
2024h; Xie
et al. 2023 Hong et al. (2023): Sun et al. (2024b); AL et al. (2024): Schmidgall et al. (2025
construct
heterogeneity through explicit role prompts. This determines who initiates communication, whom they
direct it to, and what content is appropriate for that role. For example, in ChatDev Qian et al.
(2024a) the
designer agent sends specifications to the coder agent, who returns code to the tester agent. The who talks
80

---

Published in Transactions on Machine Learning Research (02/2026)
to whom mapping is therefore rigid and the what is specialized to each stage. CAMEL Li et al. similarly enforces explicit bilateral roles that shape the exchange of rationales and plans.
2023b
Some works introducc stronger heterogencity through tool calling or environment access Chen ct al.
2024c
Schmidgall et al. (2025); Hua et al.] (2023). Here, the who talks to whom structure emerges from capability boundaries. Agents direct messages to those uniquely able to perform actions such as web browsing or simu- lation control. Because tools require precise inputs, the what communicated becomes more structured and often constrained to parameterizable instructions. Compared with purely prompt based heterogencity, these systems reduce redundancy and improve reliability, although they also narrow communication flexibility.
A different branch of heterogeneous systems uses persona variation to simulate social diversity Carroll et al. p2019 Chen et al. (2024a); Ahn & Sentis (2021); Liu et al. (2025b); Sun et al. (2025c) [Tran al. (2025b); Li et al. (2024); Zhang et al. (2023b). In these settings, who communicates with whom is influenced by social compatibility, trust, or inferred preferences rather than static pipelines. As result, what is communicated often includes beliefs, intentions, or culturally grounded reasoning. These systems improve realism and emergent dynamics but introduce higher variability and reduced predictability compared with role fixed teams.
Strategic heterogeneity appears in debate, negotiation, or opinion shaping contexts Li et al. 2024f): Xu al. (2023b); Chen et al. 2024a). Agents differ in reasoning style, argumentative strength, decision heuristics, which creates richer information exchange. Here, the who talks to whom pattern becomes fully connected since each agent must respond to all others. Consequently, what is communicated tends to include evidence, counterarguments, and justification, Compared with persona based heterogeneity, strategic heterogeneity is better for convergence and critique but less effective at modeling long horizon social dynamics.
Overall, heterogeneous agent settings enable targeted, role dependent communication with clear pathways specifying who speaks to whom. They also shape what is communicated toward specialization, justifica- tion, tool use, or persona grounded interaction. Their main advantage is increased coordination power and expressivity. Their main limitation is reduced scalability and greater dependence on accurate role specifica- tion.
Homogeneous agents Homogeneous systems employ agents with identical architectures and comparable capabilities, without predefined role assignments or fixed communication hicrarchics. In such settings, there is no pre-specified who talks to whom structure; instead, symmetry allows any agent to communicate with any other, and interaction patterns emerge from task demands and envirommental context rather than from role differentiation. For example, Optima Chen et al. 2202ff demonstrates how homogencous LLM agents coordinate planning and decision-making through peer-to-peer communication without designated leaders, while scalable graph-based planners Hu et al. (202cc show that identical agents can dynamically form sparse communication graphs based on task relevance and dependency structure. VillagerAgent Dong ct al. (024) provides a concrete illustration of emergent who-whom selection: although all agents share the same architecture and reasoning capabilities, communication links arise from spatial proximily, shared observations, and immediate coordination needs in social-deduction environments. In this case, what is communicated consists primarily of local observations, suspicion signals, and coordination requests, rather than role-specific commands. Similarly, MedAgents Tang et al. (2023) employs homogeneous medical agents that exchange observations, hypotheses, and intermediate conclusions; despite identical capabilities, agents naturally differentiate their interactions based on case context and informational needs. Across these works, a common pattern emerges: homogencity simplifics system design and avoids brittle role engincering, but it also constrains what is communicated to relatively generic signals requests, observations, and coordination cues since no agent is explicitly responsible for specialized planning or supervision. As a result, meaningful interaction structures can still emerge, yet they rely heavily on task structure and environmental signals to induce elfective who talks to whom patterns.
Homogeneous collaboration remains effective even in embodied environments. Modular embodied agents Zhang et al.) (2023b) show that identical agents can jointly perform object manipulation and navigation when commmicattin is grounded in shared perception. This shifts what is communicated toward spatial descriptions or action proposals rather than strategy division.
81

---

Published in Transactions on Machine Learning Research (02/2026)
Compared with heterogeneous systems, homogeneous designs scale more easily, simplity message formats, and reduce dependency on accurate role prompting. However, they may lack the expressive power needed for complex task decomposition and often require additional coordination mechanisms to prevent redundancy or conflict.
In summary, the heterogeneous versus homogeneous distinction directly affects specification of who interacts with whom and the structure of what is communicated. Heterogencous teams excel at specialization and structured pipeline communcation,, while homogencous teams excel at scalability, symmetry, and emergent coordination. The appropriate choice depends on the demands of the LLM based multi agent system.
7.2.3 Message Design and Communicative Content
Beyond interaction frequency, the structure of communication also determines what types of messages agents exchange and how content aligns with addressing assumptions. Structured messages constrain what can be communicated to predefined schemas, reducing ambiguity and enabling reliable execution when who and whom are fixed. Unstructured messages, by contrast, allow agents to convey muanced reasoning and contextual information, which becomes necessary when addressing patterns are flexible dynamically determined.
Different content types refeect distinct assumptions about who needs access to information and what level of abstraction is required. Task execution messages explicitly align responsibilities between agents, as in AutoGen and GPT-in-the-Loop, where what is communicated consists of actionable instructions or task states directed to specific executors Wu et al. 2023b); Nascimento et al. (2023a] Knowledge exchange mechanisms emphasize shared understanding and collective reasoning, with agents broadeasting intermediate insights rather than targeting a single recipient, as in CoMM [Chen et al. (2024d). Negotiation and strategy- oriented communication emerges when who and whom have partially misaligned objectives, shifting what is communicated toward influence and compromise, exemplified by LLM-Deliberation Abdelnabi et al. 2023
State updates, feedback, and persona maintenance further illustrate how message content depends struc- tural assumptions. Systems such as MetaGPT and ReConcile rely on frequent status reporting to maintain global coherence across agents Hong et al. (023) 1 Chen et al. (202bb), while persona-driven systems require agents to communicate self-consistenn reflections to preserve identity and role stability [altaji et al. (2024): Wang et al. These designs implicitly assume that who receives a message can interpret it within a shared contextual frame, reinforcing the coupling between addressing structure and message semantics.
Taken logether, the structure of communication in LLM-based multi-agent systems reveals that decisions about who communicates with whom directly motivate what must be communicated. Environmental con- text further constrains these choices, as grounded and embodied scttings require spatially and temporally localized exchanges, while abstract workflows favor broadcast or pooled communication. Organizing interac- tion structure and message content jointly is therefore necessary to understand how communication supports coordination, scalability, and robustness in complex multi-agent systems.
7.2.4 Framework-Level Design of Communication Structure
This subsection focuses on existing LLM-based multi-agent frameworks because they provide concrete, system-level answers to the fundamental questions of who commmicates with whom, what information is exchanged, and ultimately why communication is introduced in the first place. Unlike algorithmic studies that isolate communication modules, frameworks operationalize communication choices through explicit de- sign decisions, such as agent roles, orchestration logic, message routing, and memory sharing. As a result, each framework implicitly encodes assumptions about who should be allowed to speak, whom they should address, and what kinds of information are necessary to support coordination, reasoning, task exccution. Examining these frameworks therefore reveals how different application demands molivate distinct answers to the why communicate question and how those motivations shape the resulting who-whome whom-what what configurations.
Table 12 summarizes representative LLM multi-agent frameworks through the lens of communication topol- ogy, number of agents, and application domain, providing compact view of how these systems instantiate
82

---

Published in Transactions on Machine Learning Research (02/2026)
communication structure at scale. The topology columns explicitly characterize constraints who communicate with whom, ranging from rigid chains and hierarchies to decentralized and shared- designs. The application domains further contextualize why these communication patterns are chosen, for example, reliability in software engineering workflows, diversity reasoning in debate settings, or autonomy in embodied environments. By juxtaposing these dimensions, the table highlights that frameworks support- ing similar tasks often converge similar answers to the who-whin whom question, while differing primarily in what information is exchanged and how strictly communication is regulated. This overview sets the stage for the detailed analysis that follows, where we compare how different classes of frameworks resolve these tradeoffs in practice.
Table 12: Comparison of LLM Multi-Agent Frameworks
Topology
Framework
Num Agents
Application
Chain
Hier.
Decent.
Cent.
Pool
LangGraph lan
Flexible Flexible
General
CrowAl
General
OpenAA
Warm
OpenAl
Flexible
General
SuperAGI
60p
Flexible
General
CAMEL
2023b
2
General
AutoGeu
Wu et AL
2023b
Flextble Flexible
Reasoning tasks
Agent
Chrn
21/3
General
MetaGPT
Long
5
Software dev & coding
MegaAgeni
Wang et a.
590+
Software dev & coding; policy simulation
ProAgent
Zlanz
Embbodied Overcooked.
AgentCF
Zhain et
20210
Users & Items 5
Recommender
ChatDev
Uuan et al.
M7246
Software dev & coding
COLEA Zhang
Emboodied and TDW-MAT
MedAgent:
Tany
22122
3
Clinical ressoning tasks
ReCoucile
3
Reasnning tacks
AutoAgenta
Flexible Flexible
General automation
DyLAN Liu et (ilkeb
General
OpenAgeuts Xie sl.
2023
3
Data Aualysis/ API Tools/Wol /Web assistant Recommendation system
MACRee Wang
5
Magentic
One Fourruyy ct ml.
2024
5
General
AgentCoord Pin al.
Flexible
Visualization roordination strategies
AntaML-Agen Lireat
2014
5
ML pipeline ddepl--xeplyyyment) Ressoning tasks
GPT-Swamn
Zhuge el
Flexible Flexible
MACNet Qien
A1744b
General
Theory-oFMinn/
2123e
3
Oustomized text game (search and rescue)
SelE-Oganinx!]
[Shibash
& Nicimmara 2024
Floxible
Software dev & coding
SMAITT-
Kantan ml. 2025
LLM
Multi-rooot task planning
Workflow oriented frameworks Workflow oriented systems such LangGraph lan (b), Crew AI cre
AutoAgents Chen et al. (2023a), Magentic-One Fourney et al. (2024), AutoML-Agent TTiiat et al. 2024
MetaGPT Hong et al. (2023), ChatDev Qian et al. (2024a), MegaAgent Wang et al. (2024b)
Self-
Organized Ishibashi & Nishimura (2024). OpenAT Swarm OpenAT and OpenAgents Xie et al. 2023 define
a relatively rigid pattern of who communicates with whom. A central planner or manager agent typically receives the user goal, decomposes it into subtasks, and assigns these subtasks to specialized worker agents, which then report back their outputs. This design constrains what communicated structured artifacts such as task specifications, intermediate results, bug reports, and tool invocation logs. MctaGPT Hong et al. (2023), MegaAgent Wang et al. (2024b), and ChatDev Qian et al. (2024a) emphasize strong role specialization in software engineering settings, which narrows what each role communicates and improves reliability for code production and review. In contrast, LangGraph lan (b.. CrewAT crc OpenAT Swarm OpenAl| and OpenAgents offer more flexible compositions of agents and tools, expanding the range of what be exchanged but increasing coordination overhead. AutoAgents Chen et al. (023a)), Magentic-Onn Fourney et al. (2024). AutoMT-AAeet Trirat et al. (2024), and Self-Organized further highlight that central orchestration simplifies debugging and monitoring but can create bottlenecks when many agents depend on a single coordinator. Across these systems, the main tradeoff is between strict control of who talks to whom for predictable behavior and looser structures that support richer but harder manage communication.
Collaborative reasoning and decision making frameworks Collaborative reasoning frameworks such CAMEL Li et al. (022b)) ReConcile Chen et al. (003b)) AutoGen Wu et al. (2023b), GPT-Swann Zhuge
83

---

Published in Transactions on Machine Learning Research (02/2026)
et al. (0244a), MACNet Qian et al. (2024b). and MedAgents Tang et al. (2023) relax some of the workflow constraints to encourage richer dialogue. They typically rely on chain or shared message pool topologies so that each agent can observe and respond to the evolving conversation. This broadens the possible whom targets for each agent and shifts what is communicated toward rationales, critiques, counterarguments, and confidence assessments. CAMEL Li et al. (2023b) keeps who and whom simple by pairing two role- playing agents, which facilitates convergence and grounded collaboration. AutoGen Wu et al. (2023b and MACNet Qian et al. (2024b) extend this to multiple agents with complementary roles, increasing reasoning depth but also introducing redundancy when many agents address the same issue. ReConcile Chen et al. (2023b) and Aega Wang et al. (2024b) use shared message pools to support multi perspective critique in reasoning and clinical settings, respectively, pushing what toward evidence based justification and error checking. GPT-Swarm Zhuge et al. (2024a) emphasizes scalability in the number of cooperating agents, showing that more agents increase coverage of the solution space but require mechanisms to filter low-quality contributions. Overall, these frameworks illustrate how enlarging the set of possible whom recipients and promoting argumentative what content can improve solution quality, at the cost of more complex aggregation and consensus procedures.
Embodied and interactive environment frameworks Embodied and interactive frameworks such as Prot Zhang et al. (2024a), COLFE Zhang et al. (2023b), Theory-of-Mind basedd agents Li et al. (2023e), and SMART-LLM Kannan et al. (2023) primarily operate in simulated or physical environments where communication is grounded in action and perception. Here, who communicates with whom is often determined dynamically by spatial relationships, task assignments, or inferred capabilities. In ProAgent Zhang et al. (2024a) and COLEA Zhang et al. (2023b), for example, a small number of agents coordinate in Overcooked or manipulation environments, and what is communicated focuses on local state, intended moves, or high-level intent lo avoid collisions and deadlocks. Theory-of- systems introduce richer internal models of other agents, so messages can include beliefs, goals, or inferred mental states, changing what from purely action-oriented content to explanations of reasoning. SMART- Kannan et al. (2023) uses central LLM planner with multiple robots, but at the level of the robots, communication still centers on motion plans and (ask assignments. Compared to workflow and reasoning frameworks, embodied systems are more constrained in what they must communicate, yet they are more sensitive to errors in who talks to whom, since misrouted information can directly degrade physical coordination.
Personalization frameworks Personalization--rrineeed systems such as AgentCF Zhang et al. (024bb) and MACRec Wang et al. (2024l) define who as specialized recommenders that each capture a distinct perspective users, items, or contexts, while whom is the aggregation mechanism that combines these perspectives into final recommendations. Because the goal is to keep the system efficient and interpretable, what is communicated is typically lightweight, including scores, embeddings, or short rationales rather than long free form messages. AgentCF Zhang et al.] (2024b) allows agents to attend to each other's intermediate outputs, which enriches what by incorporating cross-agent critique and adjustment, leading to more diverse recommendations. MACRec Waan et al. (2024l) intentionally restricts communication between agents to control noise and maintain stability, trading some expressiveness in what for improved robustness. Compar- ing these two frameworks highlights how communication density among specialized agents influences both accuracy and interpretability in personalization tasks.
General purpose frameworks General purpose infrastructures such as LangChain lan (a. AGI sup AutoGen Wu et al. (2023b), Veent Chen et al. (2023c) expose flexible building blocks rather than fixed communication patterns. They leave who communicates with whom largely under developer control through configurable graphs, routing policies, or orchestration logic. As a result, what can be com- municated spans tool calls, environment observations, intermediate reasoning traces, and meta level control signals. AgentVerse illustrates how this fiexibility supports heterogeneous teams of agents operating under different topologies, while AgentCoord visualizes emergent communication patterns to help diagnose whether information is being routed effectively. LangChain and SuperAGI focus on tool integration and workflow construction, letting developers tune who and whom for specific applications. AutoGen appears here as well because it provides both ready made interaction patterns and lower level primitives. These frameworks
Super-
84

---

Published in Transactions on Machine Learning Research (02/2026)
demonstrate the benefits and risks of high flexibility: they enable rapid experimentation with communication structures but also place the burden of designing stable whom-what what configurations on practitioners.
In summary, cxisting LLM based multi-agent frameworks differ systematically in how they organize who communicates with whom and thus structure what information flows through the system. Workflow and centralized frameworks fix these dimensions to ensure reliability and debuggability, collaborative reasoning frameworks broaden them to support critique and consensus, embodied frameworks tie them to environmental and physical constraints, and personalization frameworks optimize them for efficient user modcling. General- purpose platforms span these regimes but expose unresolved challenges in scaling communication without losing coherence. This diversity suggests that designing communication in Mas--sdedd multi-agent systems is fundamentally about controlling the interaction between who, whom, and what rather than only choosing topology in isolation.
7.3
When and Where LLM Agents Communicate
7.3.1
Motivations for Communication Timing
Motivational settings determine not only the alignment of agent objectives but, more fundamentally, when agents choose to communicate during interaction. In LLM--baed multi-agent systems, communication is often selective and costly, so agents do not exchange messages continuously. Instead, motivational structure governs the timing of communication, such as whether agents speak preemptively, reactively, strategically, or only at key decision points. We categorize motivational settings into three main types: competitive, cooperative, and mixed-motive (Figures 21 23
VS
Figure 21: Competitive
Figure 22: Cooperative
Figure 23: Mixed-Motives
Competitive settings This setting (Fig. 21 arises in adversarial confticting environments, fun- damentally altering the function and timing of communication. Unlike cooperative systems, where com- munication supports synchronization, in competitive settings when to communicate becomes a strategic decision: agents communicate selectively at moments where messages can influence beliefs, commitments, or outcomes. As a result, communication sparse, high-stakes, and tightly coupled to strategic leverage rather than routine state sharing. (1) Hidden-role and social-deduction games, including Werewolf Bailis let al. (2024), Avalon Wang et al. (2023b), and AvalonBench Light et al. (2023), exemplify this regime. In these environments, when agents communcaate is largely dictated by explicit game phases such as accusa- tion, defense, and voting. Communication is concentrated at points of maximal impact, where revealing, distorting, or withholding information can decisively shift collective beliefs. A recurring trend across these works is that communication prioritizes persuasion and deception over information completeness, making timing more critical than message frequency. (2) Debate-oriented competitive frameworks, such
LLM-Debate] Du et al. (023),, MAD Liang et al. (2023), and Fval Chan et al. (2023), impose externally scheduled communication rounds. Here, when communication occurs is fixed by debate protocols critique, rebuttal, and judgment phases-which which improves fairness, comparability, and evaluation consistency across agents. However, this structure limits adaptive timing strategies, preventing agents from opportunistically delaying, withholding, or escalating arguments based on opponent behavior. (3) Repeated adversarial benchmarks, including LLM Arena Chen et al. (2024c), embed agents in ongoing competition where when to communicate must be decided reactively. Unlike turn- barn-based debates, agents here choose communication timing in response to evolving opponent strategies and performance feedback. These settings expose failure modes such as over-persuasion, delayed response, or ineffective escalation, highlighting the difficulty of tim- ing communication in open-ended competitive interaction. (4) Strategic dialogue and critique-bar based
systems, such negotiation and reconciliation frameworks Abdelnabi et al. (0023): Chen et al. (2023b).
85

---

Published in Transactions on Machine Learning Research (02/2026)
treat conflict detection itself as the trigger for communication. Compared with hidden-role games that reward deception, these systems encourage agents to communicate when disagreements or inconsistencies are identified, shifting the timing objective from belief manipulation toward conflict resolution and outcome stabilization. (5) Dynamic competitive environments, including MindAgent [Gong et al. (2023) and extended Arena-style settings Chen et al. (2024c), further emphasize temporal adaptivity. In these systems, agents communicate in response to changes in environmental state or opponent strategy, making when communication occurs context-dependdnt rather than phase- or turn-based. This highlights the importance of rapid detection and response over deliberative scheduling. (6) Learning- based competitive frame- works Du et al. (2024): Feng et al. (2024) introduce adaptive timing policies learned through self-play or reinforcement learning- Here, agents discover when to communicate based on task difficulty, uncertainty, and opponent strength, often outperforming fixed or prompt-driven schedules. This trend suggests that optimal communication timing in competitive settings is itself a learned control problem rather than static design choice. Overall, competitive settings consistently concentrate communication at strategically critical moments. Compared with cooperative systems, communication is less frequent but more consequential: mist- imed messages can directly degrade performance, while well-timed interventions can decisively shift beliefs, votes, or strategic balance.
Cooperative settings This setting (Fig. 22 assumes aligned objectives, shifting the role of communica- tion from persuasion or negotiation toward coordination, synchronization, and error recovery. Importantly, even in fully cooperative systems, agents do not communicate continuously. Instead, when communica- tion occurs is typically event-driven, triggered by task decomposition boundaries, dependency resolution, replanning needs, or uncertainty about execution outcomes. (1) Structured task-driven systems in software engineering, healthcare, law, and scientific discovery such as CAMEL Li et al. (2023b). Chat- Dev Qian et al. (2024a), AutoML-style pipelines Du et al. (2024), hospital simulations Li et al. (2024c), legal workllows /Snn et al. (2024b), and scientific discovery agents Ghalarollahi & Buehler (2024) exhibit highly regular communication timing. In these systems, when agents communicate is tightly coupled to role hand- off points (e.g., planner to executor, executor to reviewer), which improves reliability and traceability but limits flexibility when task structure or execution order changes. (2) General cooperative benchmarks and coordination frameworks, including AgentCF Zhang et al. (2024b), ProAgent: Zhang et al./ (2044a) Coord Pan et al. (2024). Gentopia Xu et al. (2023a) ) MARG D D'Arcy et al. (2024), and large-scale evaluation studies Agashe et al. (2023); Zheng et al. (2023): Li et al. (2024a), shift communication timing away from fixed pipelines toward coordination boundaries. Here, agents communicate when shared plans must be aligned, responsibilities reassigned, or conflicts resolved, reflecting a trend toward adaptive but still structured synchronization rather than continnous exchange. (3) Hiuman-agent cooperative studies, such as those on utility alignment Carroll et al. (2019), preference elicitation Feng et al. (2024), reconcilia- tion and critique Chen et al. (2023b), negotiation Yin et al. (2023), reflective reasoning Shinn et al. (2024). and mixed-initiative collaboration Wu et al. (2024a), emphasize a different driver of timing: uncertainty reduction. In these settings, when to communicate is dictated less by task stage and more by ambiguity in intent, preferences, or trust, and communication quality is evaluated by its ability to reduce misalign- ment rather than task completion alone. (4) Embodied cooperative environments, including robotic construction Zhang et al. (2023b), scalable multi-robot coordination Chen et al. (024g), real-world manipu- lation Mandi et al. (2024), embodied agent teams Guo et al. (2024c), and control-oriented LLM systems Wang et al. 2203cc,, impose strict real-time constraints. In these works, agents communicate primarily at joint- action boundaries, replanning events, or failure recovery moments, making when communication occurs critical for safety, efficiency, and physical feasibility. (5) Self-organizing and adaptive team frame- works, such as self-organizing agents Ishibashi & Nishimura 2024), dynamic coordination models Liu et al. (2024b), adaptive navigation systems Rasal & Hauer (2024). )) long-horizan agent teams (Schmidgall et al. (2025), ; Optima [hen al.) (2024f), AutoAct Qiao et al. (2024), GPT-Swarm Zhuge et al. (2024a), au- tonomous multi-agent control Liu et al. (2024a). ) scalable planners Hu ct al. (022cc), VillagerAgent Dong ct al. (2024), AutoML coordination 'Trirat et al. 2024). and MedAgents Tang et al. (2023) further relax fixed schedules. Compared to static cooperative pipelines, these systems trigger communication during capability discovery, delegation, and long-hoo long-horizon state maintenance, highlighting a trend toward adaptive timing at the cost of higher coordination overhead. Finally, (6) reusable infrastructures such as Agent Chen
et al. 2023c AutoGen Wu et al. (202bb), MetaGPP Hong et al. (2023). and ProAgent Zhang et al. (203aa)
86

---

Published in Transactions on Machine Learning Research (02/2026)
generalize cooperative communication patterns across tasks and domains. While they broaden coverage of coordination scenarios, they also expose a recurring design tension: without explicit timing regulation via memory, scheduling, or attention mechanisms, cooperative systems risk over-communication that degrades efficiency rather than improving coordination.
Mixed-motine settings This setting (Fig. 23 combines cooperative and competitive incentives, mak- ing communication timing inherently context-dependdnnt. Unlike purely cooperative adversarial regimes, agents must continuously assess whether communication will strengthen alliances, signal commitment, repair trust, or expose strategic vulnerabilities. As a result, when to communicate is neither fixed nor spanse, but adaptively modulated by evolving social relationships and incentive structures. (1) Generative agent socicties and social simulations, such as role-based and persona-driven environments Laan et al. (2023): Hua al. (2023); Chen al.] (2024a);; [Abdelnabi al. (2024), illustrate this tension clearly. In these systems, communication is frequently triggered by social events e.g., alliance formation, reputation repair, betrayal, or conflict aecalation-rathen than by task decompasition alone. A recurring pattern is that early communication serves to establish social positioning and intent, while later communication reflects shifts in trust and strategic alignment. (2) Large-scale self-organizing and open-ended platforms Cheng et al. (024g; Liu et al. (2025b): Tran et al. (2025b); (Snn et al. (2025cp; Xie et al. (2023); AL et al. (2024); Hu al. (024a) emphasize the role of scale and temporal evolution. In these systems, when communication occurs changes over time: early-stage interactions focus on partner discovery, norm formation, and role negotiation, whereas later-stage communication shifts toward maintaining coherence, resolving disputes, and managing long-term dependencies across many agents. This temporal stratification of communication timing is largely absent in smaller or static environments. (3) Nested, modular, and multi-level architectures Fourney ee al. (0244); Xu al. (2023c); Gong et al. (2023); | Wu et al. (2025); Li et al. (2024(); Ahhn & Sentis (2021)
further refine communication timing by introducing structural layers. In these systems, agents communicate
frequently within local groups for coordination and execution, but communicate selectively across groups when incentives diverge or global alignment is required. This produces layered communication schedules, where when to communicate depends jointly on group membership, task phase, and strategic alignment. Overall, mixed-motive settings reveal that communication timing cannot be prescribed by a single prin- ciple. Compared with cooperative systems that emphasize coordination events and competitive systems that concentrate communication at leverage points, mixed-motive environments require agents to dynam- ically adapt when they communicate in response to shifting incentives, social structure, and strategic considerations.
TheereTheoettic View of Communication Timing in Mixed-Motive Settings Mixed-motive com- munication settings can be formally interpreted through the lens of dynamic games with incomplete in- formation. Let agents E W interact in extensive-form game characterized by state st, private in- formation 8,, actions a, and messages m(") Communication acts modify agents' information sets and beliefs, n Pr(O_: | h(")m" m!!),, thereby reshaping strategic incentives and equilibrium behavior. From this perspective, deciding when to communicate corresponds to selecting signaling actions at specific nodes of the game tree, balancing information revelation against strategic vulnerability. Equilibrium behavior in mixed-motive multi-agent systems is often emergent and heterogeneous. In large or role-structured environ- ments, different subsets of agents may converge local Nash or Bayesian Nash equilibria, while higher-level coordination arises through aggregation, mediation, or hierarchy. Many MARL and LLM-based systems implicitly approximate Nash or Bayesian Nash equilibria via self-play or reinforcement learning, even when equilibria are not solved explicitly. Recent work, such as Yi et al. (2025), formalizes multi-LLM coordination as an coampeete-infrmmaioon game and explicitly targets the Bayesian Nash equilibrium, demonstrating improved efficiency and regret. bounds relative to non-equilibrium coordination. Beyond learning- ap- proximation, several studies directly examine the ability of LLMs to compute or reason about equílibrium. Silva et al. Silva (2024) study LLM performance in mixed-strategy Nash equilibrium games and show that, while LLMs can identify equilibria in canonical settings, especially when equipped with code execution and structured prompts, their performance degrades sharply under slight perturbations of game structure or ran- domized strategies. These findings highlight both the promise and fragility of LLMs as equilibrium reasoners, underscoring the gap between surface-level strategic competence and robust thee-theoretic reasoning. More recent. frameworks further extend equilibrium analysis to the reasoning level. The L.LM-Nash framework Zhu
87

---

Published in Transactions on Machine Learning Research (02/2026)
2025) defines equilibrium over the prompt reasoning space, modeling bounded rationality by treating LLM inference itself as part of the strategic process. In this view, actions emerge as behavioral outputs of equilibrium reasoning trajectories, and equilibrium outcomes may diverge from classical Nash predictions due to cognitive constraints and epistemic learning. These perspectives suggest that communication timing in mixed- motive settings is best understood as an equilibrium- consussttntnttttttttto strategy across multiple levels: agents speak, delay, or withhold messages when doing so improves expected utility under evolving beliefs, whether equilibrium is approximated through learning, computed explicitly, realized through aasoonnnglevee adap- tation. This unifies modern MARL and LLM-based communication with classical notions of signaling, cheap talk, and equilibrium selection, while remaining compatible with data-driven multi-agent learning.
7.3.2 Environmental Triggers and Context
The environments which LLM agents operate play a central role determining when communication is necessary, optional, or costly. Unlike motivational settings, which describe agents' goals, application domains impose concrete constraints on observability, temporal coupling, action dependencies, and feedback latency. These factors directly influence when to communicate: agents may need to exchange information before acting, during execution, after observing outcomes, or only when coordination failures arisc. We therefore organize application domains as a lens on when communication is triggered by environmental structure rather than by incentives alone.
Table 13 summarizes major application domains for LLM-based multi-agent systems and highlights how different environments induce distinct commmication timing patterns. Across domains, commmicattion is not uniformly continuous. Instead, when agents communicate depends on where uncertainty arises, where dependencies between agents are strongest, and where delayed or missing information would cause coordination breakdowns.
Coding In coding-oriented multi-agent. systems, when communication occurs is primarily governed by task decomposition, artifact dependencies, and error propagation rather than social incentives or persuasion. Communication is therefore event-driven and tightly coupled to the software development lifecycle, with agents exchanging information when intermediate artifacts become available, inconsistent, or insufficient for progress. (1) Stage-based and role-driven coding pipelines, exemplified by early systems such as fixed
ChatDev Qian et al. (2024a), adopt predefined workflows that explicitly schedule communication stages (e.g., design, implementation, testing, and review). In these systems, when agents communicate is determined by role handoff points, ensuring clarity and traceability of information flow. This structured timing improves reliability and reduces ambiguity, but it also limits adaptability when task requirements change or unexpected dependencics arisc. (2) Parallel and multi-team coding frameworks, including systems that relax strict stage ordering Du et al. (2024); Li et al. (2024d), shift communication timing to- ward periodic synchronization and conflict resolution events. Here, agents or subteams work concurrently and communicate when inconsistencies, integration confficts, divergent design choices detected. This trend improves robustness through diversity and redundancy, but introduces additional coordination over- head and requires explicit mechanisms to manage synchronization frequency. (3) Self-organizing and uncertainty-driver coding agents, such as those explored in Ishibashi & Nishimura (2024), further de- couple commuication from predefined schedules. In these systems, when to communicate demand-driven: agents initiate communication only after delecting uncertainly, stalled progress, or insufficient confidence in their local solutions. This adaptive timing reduces unnecessary exchanges but places greater burden uncertainty estimation and progress monitoring. (4) Domain-specific coding environments, includ- ing automaled machine learning pipelines Triral et al. (2024) and competitive programming agents Islam et al. 2024), illustrate how artifact readiness directly governs communication timing. Agents communicate when concrete outputs, such as trained models, feature sets, compiler errors, or failing test cases, become available, making when to communicate tightly aligned with artifact maturity rather than abstract task phases. Overall, coding environments consistently favor communication at dependency boundaries rather than continuous dialogue. Across these systems, when communication occurs is dictated by artifact avail- ability, integration risk, and uncertainty, reflecting a broader trend toward event-driven and demand-aware communication policies rather than fixed or socially motivated schedules.
88

---

Published in Transactions on Machine Learning Research (02/2026)
Application Domain
Key Communication Characteris- tics
Timing, Structure, and Design Im- plications
Coding Systems
Role-based decomposition (planner,
Communication triggered at depen-
coder, reviewer) Qian et al. 2024a
dency boundaries or artifact readi- ness;
et al. (2023b):
Arlifact-centriic
coordination via
Fixed pipelines improve reliability
code, lests, models: Trirat al.
but reduce adaptability;
(024): Islam et al.
(2024);
Adaptive timing improves robustness
Increasing
sell-organizing
at the cost of coordination overhead.
or adaptive teams:
Ishibashi &
Nishimural (2021): Li al. (2021d).
Virtual Environments
Event-driven social interaction and
Communication timing driven by state changes or social events; Strict interaction rounds enable eval- uation but limit adaptivity;
memory updates: Park et (2023
Kaiya et al. (0023);
(lurnbbased strategic communication
in games: Bailis et al. (2024 Light
Scalability requires selective and de- layed communication.
et al.
2
Open-ended exploration and long-
horizon planning: Lan
2023
Hua et al. 2023
Embodied Environments
Grounded language-action
cou-
Communication aligned with percep-
pling: Ahn et al. 20022 Zhang et al.
tion-action loops;
(2023b):
Early communication aids planning,
Coordination under physical and temporal constraints: Mandi et al.
late communication aids recovery;
Over-communicttion degrades real-
(2024) Guo et al. (202cc):
time performance.
Mixed centralized and decentralized
control strategies: Kannan al. (2023).
Human-Agent Workflows
Mixed-initiative dialogue and prefer-
Communication timed to reduce
ence elicitation: Feng et al. 2024
certainty or cognitive load;
Wu al. 2002
Turn-taking improves reasoning but limits responsiveness;
Role- and persona-aware interac-
tion: Chen et. al. (222 Li
Effective timing balances informa- liveness and user trust.
(2024a]
Structured debale and critique mechanisms: Xu et al. (202bb) Jin
al. 2024
Table 13: Taxonomy of LLM-based multi-agent systems: linking application domains to communication characteristics and lining-driven design implications.
89

---

Published in Transactions on Machine Learning Research (02/2026)
Virtual Environments Virtual environments are characterized by evolving world states, partial observ- ability, and long- horizon interaction, making when communication occurs contingent on environmental events and state transitions rather than fixed task phases. Communication timing in these settings is there- fore predominantly event-driven, shaped by changes in social context, spatial proximity, resource availability, or agent beliefs. (1) Generative agent simulations and social worlds, such as Smallville-sttle environ- ments Park et al. (2023): ; Kaiya al. (2023), emphasize socially motivated timing. In these systems, agents communicate when new information is acquired, social relationships change, or long-term plans are revised. Communication is triggered by events like encounters with other agents, updates to memory or goals, and shifts in social context, reflecting a trend toward communication driven by internal state updates rather than externally imposed schedules. (2) Social deduction and strategic game environments, including Avalon and Werewolf benchmarks Lan et al. (2023); Light et al. (2023); [ailii et al. (2024), impose explicit temporal constraints communication. Here, when agents may communicate is tightly regulated by game mechanics, such as discussion rounds or voting phases. This structured timing enables controlled evaluation of reasoning, deception, and belief manipulation, but also restricts adaptive communication strategies that depend on continuous state monitoring. (3) Competitive and evaluative virtual benchmarks, such as LLMArena and related platforms Chen et al. (2024c); Agashe et al. (2025), systematically vary interaction schedules to probe agent robustness. These studies reveal that agents often struggle when communica- tion opportunities are sparse, delayed, or strictly time-limited, highlighting sensitivity to when information exchange is permitted rather than to message content alone.
(4) Large-scale virtual societies and task-oriented simulations, including institutional and project-based environments Li et al. (2024c); AT
et al. 2024);; Wang et al. (2023a); Zhu et al. (2023); Wu et al. (2024b), expose scalability-driven shifts in communication timing. As the number of agents and interactions grows, communication becomes increas- ingly selective and delayed, typically triggered by resource contention, coordination failures, or significant environmental changes. This trend reflects move away from frequent interaction toward sparse, high- impact communiattin events. Overall, virtual environments foreground timing as a central design variable. Across social simulations, strategic games, and large-scale virtual worlds, when communication occurs is governed by environmental dynamics and scalability constraints, revealing a fundamental tension between responsiveness and tractability that does not arise in more structured task-driven settings.
Embodied Environments Embodied environments couple communication tightly to real-time percep- tion-action loops, making when communication occurs a function of physical dynamics, sensing uncertainty, and safety constraints rather than abstract lask stages. In these settings, communication is costly in time and attention, and mistimed messages can directly degrade performance or safety. (1) Early embodied coordination systems, including multi-robot and embodied LLM agents Ahn et al. [2022]; Agashe et al. (2023); Zhhag el al.] (2023b), demonstrate that proactive communication before action execution often beneficial. Agents communicate when plans must be aligned spatially (e.g., shared navigation goals or ma- nipulation targets), reducing downstream conflicts and redundant motion. These works establish a baseline insight: in tightly coupled physical tasks, carly communication can prevent costly crrors. (2) Centralized
versus decentralized planning studies Mandi et al. (2024); Kannan et al. (2023); ; Chen et al. (024g);
Dong et al. (2024): : Guo et al. (2024c) reveal a systematic divergence in communication timing. Centralized or coordinator-bass based systems tend to communicate carly, aggregating observations and issuing joint plans before execution. In contrast, decentralized systems delay communication until local uncertainty, prediction error, coordination risk exceeds threshold. This comparison highlights key trade-off: early communication improves global consistency, while delayed communication preserves reactivity and scalability. crative embodicd benchmarks, such as Overcookod-style environments and related evaluations Agashe el al. 20223 Zhang el. al. (2024a), emphasize that communication frequency alone is not predictive of success. Instead, performance correlates strongly with when communication occurs particularly at joint- action boundarics, task handoffs, failure recovery moments. These benchmarks show that sparse but well-timed messages outperform continuous or poorly aligned communication.. Overall, embodied environ- ments make when to communicate inseparable from perception and action timing. Across centralized planners, decentralized agents, and cooperative benchmarks, the literature converges on a common design principle: effective embodied communication is event- tring-rrggeerd and environment-aware, prioritizing safety, coordination, and responsiveness over message volume Mandi et al. (2024); Agashe et al. (2023): Guo et al. (2024c).
(3) Coop-
90

---

Published in Transactions on Machine Learning Research (02/2026)
Human-Agent Workflows and Collaboration In human-agent workftows, when communication oc- curs is fundamentally constrained by human attention, trust calibration, and cognitive load, making timing as critical as content. Unlike purely automated systems, excessive or poorly timed communication can directly degrade human performance and trust. (1) Task-driven human agent workflows, such as CAMEL-style collaborative problem solving Li et al. (2023b) and preference-aware assistance systems FFnng et al. 2024), show that communication is most effective when triggered at decision-critical moments rather than continuously. These systems demonstrate that delaying communication until uncertainty, ambiguity, or irreversible choices arise reduces cognitive burden while preserving decision quality. (2) Interactive and mixed-initiative systems, including human-in-the-loop AT interfaces Wu et al. (2022) and adaptive guid- ance policies Feng et al. (2024), explicitly model when to communicate as a control problem. Here, agents learn request feedback or intervention only when confidence drops below a threshold or when predicted outcomes diverge, highlighting a trend toward demand-driven rather than schedule driven communication. (3) Persona-based and culturally grounded agents Chen et al. (2024a); Li et al. (2024a) further refine communication timing by adapting to user expectations, roles, and social norms. These works show that when communication is perceived as appropriate depends not only on task state but also on social context, with mistimed messages harming engagement even if informationally correct. (4) Human-in-the-loog
multi-agent debates and team-based reasoning environments Xu al. (2023b); Stepputtis et al. (2023;; Chen et al. (2023b) emphasize structured timing mechanisms such as turn-taking, moderation, or intervention windows. These studies consistently find that externally regulated communication timing improves reasoning quality and reduces redundancy, whereas unrestricted communication leads to repetition and coguitive overload. Overall, human agent col- laboration reframes when to communicate as a balance between responsiveness and cognitive efficiency. Across task-driven workflows, mixed-initiative systems, persona-aware agents, and debate settings, the litera- ture converges shared design principle: effective human agent communication is sparse, event-triggered, and socially aware, prioritizing human trust and usability over maximal information exchange.
(2023); Jin et al. (2024); Du & Zhang [2024]; Xiong et al.
|;
7.3.3 Evaluation Settings for Communication
Evaluating LLM-based multi-agent systems requires explicit reasoning about when communication occurs and where it is situated, because communication is rarely an isolated artifact. Instead, it is embedded within environment dynamics, task structure, and agent interaction schedules. Unlike single-agent evaluation, where outputs can be assessed independently, multi-agent evaluation must consider how information exchange unfolds over time and how the environment exposes or constrains that exchange.
Crucially, the choice of evaluation environment determines where communication can happen, such as text channels, embodied actions, or shared state, and this in turn determines when communication becomes observable, necessary, or consequential. As a result, benchmarks implicitly encode assumptions about com- munication timing even when communication itself is not an explicit evaluation target.
Planning and Collaboration Benchmarks Plamning-oriented benchmarks such as Collab- Overcooked Sun et al. (2025a) and large-scale coordination tasks Chen et al. (2024g) emphasize joint planning and cooperative execution, making when communication occurs tightly coupled to plan formation. In these environments, agents typically communicate before acting to align goals, assign roles, or synchronize plans. Compared to benchmarks that emphasize execution-time adaptation, these settings privilege early, front- loaded communication and implicitly assume that most coordination can be resolved prior to action. The where dimension in these benchmarks is usually a structured interaction loop with explicil planning and execution phases. Metrics such as task completion rate and efficiency indirectly reward timely communi- cation but do not distinguish whether communication was minimal, redundant, or delayed. As a result, these benchmarks underrepresent the value of corrective or opportunistic communication that occurs during execution rather than at planning time.
Embodied Interaction and Navigation Benchmarks In contrast to planning-centric benchmarks, embodied benchmarks such as A12-THOR Kannan et al. (2023) and C-WAH and TDW-MAT Zhang et al. (2023b) shift attention from plan formation to real-time perception- coupling, where when commu- nication occurs is tightly constrained by physical dynamics. Agents must. decide whether to communicate
91

---

Published in Transactions on Machine Learning Research (02/2026)
before acting to coordinate intent or after acting to repair mistakes, introducing contintous tradeoff between proactive and reactive communication. Here, where communication is embedded within action sequences rather than isolated dialogue turns. Evaluation metrics such as task success and efficiency implicitly penalize late communication, as mistimed exchanges directly translate into physical inefficiencies. Compared to sym- bolic planning benchmarks, embodied environments more clearly expose the cost of delayed or insufficient communication, even though they rarely measure communication timing explicitly.
Social Inference and Strategic Communication Benchmarks Unlike both planning-orienle and em- bodied benchmarks, social reasoning benchmarks such as Werewolf Arena Bailis et al. (2024) and Avalon puttis et al. (2023) externalize the decision of when to communicate by enforcing discrete, turn-basod discussion and voting phases. Communication timing is therefore no longer adaptive but institutionally imposed, shifting the learning problem from deciding when to communicate to deciding what to commu- nicate at strategically critical moments. The where aspect is constrained to shared text-based dialogue channels, enabling controlled evaluation of belief modeling, persuasion, and deception. Metries such win rale and role identification accuracy reflect whether communication was effective al specific moments rather than whether agents communicated frequently. Compared to continuous environments, these benchmarks allow clearer attribution between communication timing and outcomes but are limited stylized social interactions with fixed lemporal structure.
Long-Horizon and Open-Ended Environments Moving beyond fixed temporal structures, long- horizon environments such as MindCraft. [araa et al. (2021) and MineLand Yu et al. (2024) make when communication occurs an explicit strategic choice over extended time scales. Agents must decide whether to communicate early to establish shared understanding or delay communication until uncertainty, con- flict, or divergence emerges, introducing temporal credit assignment challenges absent from short-horizon turn-based benchmarks. In these settings, where communication is distributed across persistent. world states and repeated interactions, which obscures individual communication events. Metrics such survival rate or resource accumulation capture aggregate performance but fail to reveal how communication tim- ing contributed to coordination. Compared to episodic benchmarks, these environments highlight the nood for temporally sensitive evaluation metrics that can attribute long-term outcomes to earlier communication decisions.
Large-Scale Swarm and Infrastructure Benchmarks At even larger scales, swarm and infrastructure benchmarks such as SwarmBench Ruan et al. (0025) and Smart Streetlight IoT Nascimento et al. (022a)) introduce scalability as the dominant constraint shaping when communication occurs. Unlike small-team or horizon environments where communication supports shared understanding, these benchmarks require agents to communicate selectively, often only when local decisions have system-level consequences. The where dimension decentralized and localized, limiting global observability of communication patterns. Metrics such as coordination efficiency and energy usage indirectly reflect communication timing decisions but do not isolate communication events Zhu et al. (2025); Chen et al. (2024c). Compared to small-team benchmarks, these environments foreground tradeoffs between communication frequency, scalability, and robustness, emphasizing that effective systems must learn not only when to communicate, but also when not to.
Table 14: Multi-agent environment benchmarks and their characteristics.
Environment
# #Agents Observation Space
Action Space
Evaluation Method
TextStarCraft Ma
(0023)
partial; text-based
diacrete [macro/ / text commands)
rate; population ratio; resource ratio score; ToM accuracy; plan- ning accuracy
LLM- Coordiininotion
Agashe
partial; text-based
discrete (text-based
2023
commands)
O-WAH and TDW-MAT Zhang
partial; RGB-D
discrete (navigate; inser- act; communirate)
task completion rate; ciency
20265
AT2- HOR Kannan et al.
2023
partial; symbolic
discrete (primitive API calls)
success rate; task comple- tion; goal recall answer accuracy
GAIA Mialon et al.
2023
no liuit
partial; multi-modal)
discrete (tool
use;
browse; reason) discrete (move; place)
Box Warehouse, BoxLift Chen
32
full; symbolic; based
grid-
pick:
success rate; token usage; steps taken task score; efficiency
al.
2024g
Cuista World
Goug et al.
2023
full; baxt-
discrete (move; get/ppt;;
use tool)
92

---

Published in Transactions on Machine Learning Research (02/2020)
Environment
# Agents Observation Space
Action Space
Evaluation Method
RoCoBench Mandi et al.
2024
partial; symbolic
discrete (nuvee;
pick;
success rate; courdination
place) discrete
GOVSIM Platti al.
2024
partial; text-based
survival rate; efficiency;
equality completion rate; efficiency; workload balance
VillagerBench Dong et
2024
partial; text-based
discrete (move; place;
craft; manage] discrete (switch; adjust; communicate)
Smart Sireetlight 1oT
Nascimento
100
partial; simulated sen- sory inputs
fitness score; latency; energy usage
et al.
2023a
Warewoll Arena Bailis et al.
2024
partial; text-hased
discrete (vote; act; com- municate)
rate; strasegy quality;
gameplay metrics
Graphlustruct Hu et al
2024c
no limit
partial; graph- based (lo-
discrete (message pase- ing)
task accuracy
cal neighhorhoud) partial; text-hased
Agent Hospital
T. at al.
2024c
100
discrete (dingnose; pro-
diagnosis accuracy; MedQA score
scribe; examine)
LLMArena Chen et al.
2021c
partial; text-based
discrete (game moves; communicate) discrete (web naviga- tion; query tools)
TrueSkill score; win rate; skill metrics
BOLAA Liu ot al.
2023a
partial; text-
final reward; intermediate recall
Avalon (Role Identificatinn) Stepput-
partial; text-
discrete (vote; propose; communicate)
win rate; role identification
tis al. 2023
accuracy
SwarmBeuch Ruan et al.
2025
no limit
partial; grid-based
discrete (move; commu- nicate)
success rate; coordination metrics dishes delivered; efficiency
Collab- Ovelbb-Overrooeed Sun ctal.
2025
partial; grid-based
discrete (move; pick;
place; interact) discrete (move; shoot)
BattleAgeutBennce
Wang
et
partial; grid-based
mission progress; action ac- curacy; mission score sucress rate; steps human; error recovery task completion: milestone KPIs; collaboration quality ToM sccuracy; task success
(2024d)
PAICE Chang
2024
partial;
discrete (navigate; ma- nipulate; instruct)
MultiAgent Zhu et al.
2025
no limit
partial; best-
diacrete (task-dependent
actions: communicate) diacrete (move; build; communicate) discrete (move: gather; build; communicate)
MindCraft Bara et al
2021
partial; 3D
MineLand Yu ct al.
2024
48
partial; 3D
survival rate; resource coller- tion; social dynamics eplsode return; generaliza- tion to novel partners
MeltingPot Agapiou et al.
2022
partial; multi-modal
discrete (move; tag: ool- lect)
Table 14 provides a cross-sectional view of representative multi-agent benchmarks, revealing how environment design fundamentally shapes what aspects of communicttion can be evaluated. Across these benchmarks, variation agent scale, observability, action abstraction, and evaluation metrics leads to markedly different assumptions about how, when, and why agents communicate.
At high level, the environments in Table 14 span four broad benchmark regimes. Small-team, task- oriented environments (e.g., Collab- Overcooked, AI2-THOR, RoCoBench, PARTNR) emphasize tight coordination under partial observability, where communication is closely coupled to action timing and task progress. Social and strategic games (e.g., Werewolf Arena, Avalon, LLMArena) impose discrete inter- action rounds that externalize communication timing and enable clearer attribution between messages and outcomes. Long-horizon and open-ended worlds (e.g., MindCraft, MineLand, MeltingPot) evaluate communication over extended temporal scales, where timing decisions are strategic but difficult to isolate. Finally, large-scale swarm and infrastructure benchmarks (e.g., Smart Streetlight IoT, SwarmBench, MultiAgenttBench) foreground scalability constraints, forcing communication to be selective and localized.
A key trend evident from the table is that obscrvation space and action abstraction jointly do- termine how visible communication timing is to the evaluator. Benchmarks with symbolic or text-based observations and discrete actions (e.g., TextStarCraft II, VillagerBenchh, GOVSIM) make com- munication events explicit but often abstract away physical constraints. In contrast, embodied and 3D environments (c.g., C-WAH, A12-THOR, MineLand) embed communication within perception-action loops, where delayed or mistimed messages directly manifest as execution failures, yet. are rarely measured explicitly as communication errors.
Agent scale further shapes evaluation. Small-team benchmarks allow fine-grained analysis of pairwise group communication, but may overfit to dense interaction patterns. As the number of agents increases (e.g., Agent Hospital, Smart Streetlight ToT, SwaarBeenc)) communication overhead becomes a dominant concern, and benchmarks increasingly rely on system-level metrics such as efficiency, energy usage, or fairness. While these metrics reffect the consequences of communication decisions, they obscure causal links between specific communication events and outcomes.
93

---

Published in Transactions on Machine Learning Research (02/2026)
Taken together, Table 14 highlights fundamental evaluation gap: most benchmarks assess task perfor- mance under communication, rather than communication itself. Discrete, turn-based environments enable clearer attribution but are limited in realism, while contimuous and large-scale environments capture realistic constraints but lack metrics that disentangle communication timing, content, and necessity. This observa- tion motivates the need for benchmark designs and evaluation protocols that explicitly expose and measure communication structure across diverse environments.
7.4
Why and How LLM Agents Communicate
7.4.1
Structure of Communication
The structure of communication in multi-agent systems reflects both why agents need to communicate and how communication is operationalized to support coordination, reasoning, and task execution. Rather than being a purely architectural choice, interaction structure encodes assumptions about task cou- pling, uncertainly, error tolerance, and coordination cost. Different structural designs arise from different motivations for communication, and these motivations directly shape how interaction unfolds in practice. Table 15 provides consolidated view of these design choices, linking communication motiva- tions to structural patterns and their associated trade-offs.
Formally, communication in a multi-agent system can be modeled as a time-indexed signaling process. At timestep 6, each agent i emits message m E M according to a communication policym: ~ Ci(of, h M where of is the local observation and K4 summarizes past messages and actions. Actions then selected at (o), (m) I=+s,, making communication explicit information channel that shapes the joint decision process. Single-turn communication corresponds to the special case where C; is evaluated only once (c.g.. at / L= 0) and M = 0, so messages do not adapt to others' responses. Multi-turn communication allows repeated message generation over = I,..., enabling belief updates and iterative alignment through feedback loops. In grounded and embodied communication, message generation and action selection are tightly coupled in time, with mf constrained to influence immediate or near-term actions af under execution and latency constraints.
man
Figure 24: Single-turn
Figure 25: Multi-turn
Bongle-Turn Communication: Aggregation-Driven Coordination Single-turn communicatio is adopted when the primary why to communicate is to improve robustness, diversity, solution qual- ity through aggregation rather than explicit coordination. In this setting, agents do not need to adapt their behavior based on others' intermediate states. As a result, how communication is structured emphasizes minimal interaction, low latency, and independence across agents. representative example is More Agents Is All You Need Li et al. (202d)), where multiple independently sampled agents generate solutions without inter-agent interaction, and communication occurs only implicitly through a final voting or aggregation step. Compared to interactive approaches, this design reduces coordination overhead and avoids error propagation across rounds, while still improving performance through ensemble effects. However, because communication is non-adaptive, single-turn designs are limited in their ability to resolve ambiguity, handle dependencies, or correct mistakes during execution. This aggrogation-driven design pattern and its limitations are summarized in Table 15.
Multi-Turn Communication: Iterative Alignment and Coordination Multi-turn communccation arises when the why to communicate extends beyond one-shot information sharing to resolving uncer-
94

---

Published in Transactions on Machine Learning Research (02/2026)
tainty, managing inter-agent dependencies, sustaining coordination over time. In these settings, commu- nication is not an auxiliary signal but an iterative control mechanism: agents must repeatedly exchange messages to align beliefs, refine shared artifacts, or adapt plans as new information becomes available. Con- sequently, the how to communicate is shaped by explicit interaction loops that support feedback, revision, and convergence rather than single-pass transmission. (1) Artifact-centric collaboration systems repre- sent a first major class of multi-turn communication. In multi-agent software engineering, AutoMI. pipelines, and workflow-oriented design assistants, such automated coding and toolchain systems [uu et al. (2024a); (2024); Chen et al. (2023a): Wang et al. (2024b), project-level orchestration Fourney et al. (2024); Ahn & Sentis (2021). and scalable task decomposi- (2024g): Liu et al. (2024b), the primary why for communication is iterative refinement of shared artifacts. This motivates a how characterized by staged exchanges, versioning, and role-based feedback cycles. Compared to single-turn pipelines, these systems improve solution quality and robustness through repeated correction, but they also incur higher coordination cost and require explicit mechanisms to manage dependency tracking and convergence. (2) Interactive reasoning and collective sense-making systems form a second category, where the why to communicate is belief alignment or hypothesis refinement rather than artifact production. Multi-round dialogue enables agents to react to intermediate reasoning steps, challenge assumptions, and incorporate diverse perspectives, as seen in collab- orative reasoning benchmarks and social inference settings Lan et al. (2023); Xu et al. (2023e); Wu et al. (2023a); Händler (2023); Feng et al. (2024); Li et al. (2024a); Chen et al. (2024c]: Ghafarollahi & Buehler
Du et al. (2024): Trirat et al.
frameworks AL et al. (2024);
tion environments Chen et al.
2024). This motivation leads to a how that emphasizes turn-taking, explicit reasoning traces, and iterative updates, improving robustness over one-shot aggregation methods. A recurring insight across these works is that allowing agents to revise earlier conclusions through dialogue often more effective than increasing model capacity or ensemble size alone. (3) Debate-oriented and adversarial interaction frameworks further specialize multi-turn communication when the why to communicate includes persuasion, disagree- ment resolution, or strategic influence. Systems based on structured debate, critique, and reconciliation Jin
ct al. ) Li et al. (2024f): Chen et al. (202bb Du & Zhang (2024): ; Wang et al. (2024c): : Yin et al. (0023
Xiong et al. 2023) motivate how centered on adversarial turn structure, selective disclosure, and explicit
challenge response dynamics. Compared to cooperative dialogue, these designs improve performance under conflicting objectives and reduce correlated errors, but they also introduce higher variance and instability, as communication becomes sensitive to ordering, rhetoric, and strategic withholding of information. Across these categories, unifying pattern emerges: as the why to communicate shifts from information sharing to alignment, correction, or influence, the how necessarily evolves from single-turn messages to structured, multi-turn interaction protocols. While multi-turn communication consistently improves robustness and adaptability, the literature also reveals shared trade-off: greater interaction depth increases coordination overhead and amplifies the need for stopping criteria, memory management, and convergence guarantees. Table 15 highlights how these motivations map to iterative structures and their coordination trade-offs.
Grounded and Embodied Communication: Execution-Oriented Interaction Grounded and em- bodied communication arises when the why to communcatee is driven by real-time execution constraints rather than abstract reasoning or offline planning. In navigation, manipulation, and teamwork tasks, agents must synchronize actions under spatial, temporal, and perceptual uncertainty. As a result, the how to communicate prioritizes context-sensitive,, temporally aligned exchanges that directly support execution, error recovery, and action coordination. (1) Execation-drrven coordination systems constitute first category, where communication is motivated by immediate action alignment. In simulated and embodied environments such as cooperative task execution and multi-agent control settings Sun et al. (025cc); Piatti et al. (202)); Wu et al. (2 Cheng et al. (2024) agents communicate before or during action execution to synchronize movement, allocate subtasks, or resolve local conflicts. This why leads to a how characterized
by frequent, low-latency exchanges tied closely to perception-action loops, improving task success compared to static or single-ture turn designs. (2) Socially grounded and interactive environments extend execution- oriented communication into settings where physical or simulated actions interact with social dynamics. Tn
generative agent societies and interactive simulations Hua et al. (2023); Xu et al. (2023b a);; Wu et al. (2024a) agents communccate not only to coordinate actions but also to negotiate space usage, adapt to others' behav- ior, or maintain social coherence during execution. Here, the how reflects a blend of action-grounded signals
95

---

Published in Transactions on Machine Learning Research (02/2026)
and lightweight social reasoning, enabling agents to adapt to dynamic, multi-agent contexts at the cost of increased communication complexity. (3) Tool-augmented and hybrid execution frameworks further enrich grounded communication by integrating external tools or shared representations. Systems combining embodied action with tool use or shared knowledge structures Zhuge et al. (2024a] Li et al. (2023c] motivate how that interleaves execution-leve messages with higher-level coordination cues. This hybrid structure allows agents to adapt strategies during execution while preserving responsiveness, though it introduces addi- tional overhead in managing when to switch between action-centric and abstraction-centric communication. Across grounded and embodied settings, a consistent pattern emerges: as the why to communicate shifts toward real-time execution and coordination, communication becomes more frequent, localized, and tightly coupled to action timing. Compared reasoning- or planning-driven domains, these systems trade commil- nication efficiency for adaptability and safety, highlighting fundamental design tension between minimizing overhead and maintaining robust execution under uncertainty. This execution-orientee communication regime is contrasted with single- and multi-turn designs in Table 15
Table 15: Communication structure in multi-agent systems: motivations, structural choices, and trade-offs.
Structure
Why Agents Communicate
How Communication Is Structured (and Trade-offs)
Single-turn
Improve robustness or diversity via ag-
Independent generation followed by voting or selection: Li (2024d);
gregation: Li et 2024d);
Avoid inter-agent dependency track-
Minimal latency and coordination
ing or belief alignment;
overhead;
Suitable when coordination is unnec-
Limited ability to resolve ambiguity or
essary or harmful.
correct errors during execution.
Multi-turn
Resolve uncertainty and align beliefs:
Explicit: interaction loops with feed-
Tan et al.
2023 ; Xu et al. (003cc
back and revision: AT et al. (0024)
Feng et al.
2024;;
Fourney al. (2024): Chen al.
Manage inter-agent dependencies and
(2024g):
shared artifacts: Hu et al.
(2024a): Du
Turn-taking, staged workflows, or de-
et al. (2024); ;
Trirat et al.
2024]:
bate protocols: Li et al. (2024f) Du &
Support persuasion or disagreement
Zhang (2024);
resolution: Jin et al.
2024 Chen
Improved robustness and adaptability
et al. 2023b
at the cost of coordination overhead and convergence control.
Grounded
Real-time execution and action syn-
Frequent, low-latency, context-
& Embodied
chronization: Sun et al. 2025c Pi-
sensitive exchanges tied to execution:
atti al.
(2024);
Sun et al. (2025c); Wu al. (2025):
Error recovery under spatial and tem-
Communication aligned with action
poral constraints: Wu al. (2025)
boundaries and state changes;
Cheng et al. (2021)
Improved safety and adaptability, but
Coordination driven perception
higher bandwidth and timing sensitiv-
action coupling: Hua et al. 2023
Xu
ity: [hhuge et al. (2024a): Li et al. 2023c).
et al.
)
Analysis and Implications Table 15 reveals a consistent organizing principle across multi-agent systems: the structure of communication is not an arbitrary architectural choice, but a direct consequence of the un- derlying motivation for communication. When the goal robustness through diversity, systems favor single-turn aggregation with minimal coordination overhead. As the motivation shifts toward uncertainty reduction, dependency management, or joint reasoning, multi-turn structures emerge support feedback, revision, and convergence. In grounded and embodied settings, where communication directly affects phys- action loops, prioritizing responsiveness and safety over efficiency. Importantly, the table highlights a shared trade-off across all regimes. Richer communication structures improve adaptability, error correction, and coordination, but incur higher costs in bandwidth, latency, and convergence control. Conversely, lightweight structures scale well and are casier to stabilize, but struggle with ambiguity and dynamic dependencies. This perspective suggests that future
ical execution, interaction becomes tightly coupled to perception annception-aation
96

---

Published in Transactions on Machine Learning Research (02/2026)
MA-Comm systems should move beyond selecting single communication structure, and instead design adaptive or hybrid mechanisms that modulate interaction structure based on task phase, uncertainty level, and execution risk.
7.4.2 Why to Communicate: Static and Dynamic Communication Protocols
This subsection analyzes communication protocols through the joint lenses of why agents communicate and how communication is operationalized. These dimensions are inseparable: the motivation for communication determines whether interaction can be predetermined or must remain adaptive, while the protocol structure constrains which motivations can be effectively supported. We group static and dynamic protocols together bocause rigidity versus adaptability is not an abstract architectural preference, but direct response to lask uncertainly, environmental volatility, and coordination demands.
The where dimension plays a mediating role in this discussion. Communcatiion channels, observability, and interaction interfaces determine how motivations translate into concrete protocol behavior. Fixed workflows and well-defined interfaces favor static protocols, whereas open, partially observable, or socially complex environments necessitate dynamic communication to ensure timely and context-aware exchanges.
Static Communication Protocols Static communication protocols arise when the answer to why com- municate is stable and known in advance. Agents communicate primarily to transmit predefined information required by a fixed workflow, and commmiicttio is triggered predetermined stages rather than by runtime uncertainty. (1) Sequential and pipeline-based protocols exemplify this design. Chain-based prompt- ing and sequential reasoning frameworks Wu et al. (2022) Xiao et al. (2023) fix how communication unfolds as a unidirectional information flow, where cach agent processes an input and passes a structured output downstream. This protocol is motivated by a procedural why: decomposing a task into ordered subtasks. Such designs maximize reproducibility and interpretability, but assume early communication sufficient and cannot correct upstream errors once execution begins. (2) Fixed-turn debate and roundtable proto- cols adopt static structure for a different reason. In structured debate settings Li et al. (2024f), the why of communication is to expose diverse reasoning paths rather than to coordinate actions. Consequently, how communication is implemented relies on predetermined turn-taking and message formats. Compared to pipelines, these protocols allow richer content, but remain temporally rigid and insensitive to evolving task context. (3) Workflow-driven applied systems further illustrate slatic protocols motivated by procedural coordination. In systems such ChatDev Qian et al. (2024a), AutoML. pipelines Trirat et al.) (2024), and large-scale software agent frameworks SSchiddaal et al. (2225) agents communicate to exchange specifications, intermediate artifacts, or validation results according to predefined plan. This improves scalability and debuggability, but limits adaptability when task requirements change mid-execution. (4) Graph-structured and dependency-driven frameworks formalize static communication by fixing in- teraction graphs Qian et al. (2024b): Hu et al. (2024c); Yang et al. (2025). Here, the why for communication is localized dependency resolution, and the how is constrained by graph topology. While this supports par- allelism and scalability, it restricts cross-cutting information flow and makes late-stage adaptation costly,
Dynamic Communication Protocols Dynamic communication protocols emerge when the answer to why communicate cannot be determined a priori. Instead, communication is triggered by uncertainty, partial observability, strategic interaction, or evolving goals, requiring the how of communication to remain flexible and responsive at runtime. (1) Social and role-driven interaction systems represent a first class of dynamic protocols. Generative agent societics and rolc-playing environments Park et al. 22023 ; Xu
et al. 2023b);; Cheng et al. (2024) motivate communication through social context, memory updates, and changing interpersonal states. This why leads to a how based on open-ended dialogue, memory retrieval, and adaptive response generation rather than fixed pipelines. (2) Embodied and interactive environ- ments further amplify the need for dynamic protocols. In embodied task settings Zhang et al. (2023b): Dong et al. (2024): : Guo et al. (2024c); Wu et al. (2025), communication is motivated by perception errors, coordination failures, or environmental change. As a result, how communication occurs becomes tightly coupled to action timing, enabling plan repair and synchronization. Compared static workflows, delayed communication in these settings directly degrades performance. (3) Competitive and mixed-motive
97

---

Published in Transactions on Machine Learning Research (02/2026)
environments impose additional pressure protocol adaptability. In hidden-role games and adversarial benchmarks Bailis et al. (2024); Wang et al. (2023b) Light et al. (2023); Ma et al. (2023), the why of communication includes persuasion, deception, and strategic signaling. Consequently, the how must adapt message content, timing, and recipients based on opponent behavior, making static protocols ineffective. (4) Human-agent collaboration systems highlight complementary motivation. In interactive reasoning
and feedback-driven settings Feng et al. (2024); Du et al. (0024] Du & Zhang (2024); Chen et al. (2023b), agents communicate to align with human intent incorporate guidance at critical decision points. This mo- tivates how centered on interruptible reasoning, negotiation, and iterative refinement rather than one-shot exchange. (5) Large-scale and self-organizing systems push dynamic protocols toward selective and emergent communication. In scalable coordination frameworks and benchmarks Chen et al. (2024g); Zhuge et al. (2024a); Chen al. (2024c) Hu et al. (2024a), communication must. be sparse avoid overload, moti-
vating role specialization and partial observability. Fully self self-organizing systems Liu et al.
(2024b) Ishibashi further relax constraints,
& Nishimura (0024) ; Liu et al. 20023 AL et al. (2024): Jinxin et al. (2023
allowing communication structures themselves to emerge over time.
Table 16: Static vs. dynamic communication protocols, illustrating the motivations (why) and operational mechanisms (how) identified Section 7.4.2
Protocol Type
Why Communicate
How Communication is Structured
Static protocols
Predefined task decomposition and ordered subtasks: sequen- tial reasoning pipelines [Wn et al.
Fixed execution order or pipelines with
no backtracking Wu et al. 2022 Xiao
et al.
20233
(202)); Xiao et al. (2023)
Predetermined turn-taking and message
Exposure of diverse reasoning
formats Li et 20224
paths without runtime adapta-
Stage-based handoffs between predefined
tion: fixed-turn debates [i al. (024f)
roles Qian et al. (2024a): Trirat et 2024
Procedural exchange of artifacts
Static interaction graphs limiting cross-
in stable workflows: software, Au-
cutting information flow Qian et al. (2024b)
toML, and engineering pipelines
Hu el al. 2024c
Qian et al. (2024a]; Trirat et al. (2024): Schmidgaall et al. (2025)
Local dependency resolution in
known graphs workflows Qian
et al.] (2024b); Hu et al. 2024cc
Yang al. (2025)
Continueed on next page
98

---

Published in Transactions on Machine Learning Research (02/2026)
Protocol Type
Why Communicate
How Communication is Structured
Dynamic protocols
Social context
evolution and
Event-driven, open-ended dialogue with
memory updates in open-ended
memory and context tracking Park et al.
interaction Park et al. (2023 1 Xu
(2023 Xu al. (023b]
et al. 2223bb Cheng et al. (2021)
Action-timed communication tightly cou-
Coordination under partial ob-
pled to perception-actio
loops Zhang et al. (203bb); Guo et al. (2024c)
servability and perception uncer-
tainty embodied settings Zhang
Adaptive message timing, recipients, and
et al. (2023b): Dong et al. 2024):
content based on opponent behavior
Guo (0025)
al.
2024cb; Wi et
Bailis al. (024 Wang et al. (2023b
Interruptible reasoning, negotiation, and
Strategic persuasion, deception;
iterative refinement Feng al. (002
and opponent modeling in com-
Chen et al. (2023b)
petitive or mixed-motive environ-
Sparse, selective, and emergent commu- nication structures at scale Liu et al
ments Bailis et al. (2024); Wang
et (003bb) Light et 2023
(2024b): Ishibashi & Nishimura 2024
Ma et al. (2023)
Human-agent alignment, feed- back incorporation, and intent clarification Feng et [2024] Du
et al. (2024): Du & Zhang 2024);
Chen et 2023b
Selective
coordination and emergence in large-scale or self- organizing systems [Chen et al. (2024g); Zhuge et al. (2024a); Chen et al. (2024c); Liu et al.
2024b); Ishibashi & Nishinura 2024
Comparative Perspective Across these studies, static protocols prioritize efficiency, predictability, and control by assuming that the why of communication known in advance. Dynamic protocols embrace uncertainly, allowing the how of communication to be determined at runtime. This distinction reflects fundamental trade-off between coordination overhead and adaptability, summarized in Table I6 Crucially, this trade-off is mediated by where communication occurs. Fixed workflows and fully observable envi- ronments favor static designs, whereas open, embodied, or socially complex environments demand dynamic protocols to ensure timely and context-aware information exchange. Organizing protocol design around why, how, and where thus reveals static versus dynamic communication an environmental response rather than purely architectural choice.
Analysis Table 16 clarifies that the distinction between static and dynamic communication protocols is driven primarily by why agents need to communicate, rather than by architectural implementation convenience. When task structure is stable and dependencies are known in advance, static protocols align naturally with procedural coordination needs, offering efficiency, interpretability, and ease of scaling, In contrast, as uncertainty, partial observability, social interaction, or strategic behavior increase, dynamic pro- tocols become necessary to support context-sensitive, event-driven communication that cannot be scheduled priori. A key insight emerging from the table is a recurring design tension across domains: while dynamic protocols consistently improve robustness, adaptability, and error recovery, they introduce higher coordi- nation overhead and amplify the need for filtering, prioritization, and convergence control. This trade-off suggests that future MA-Comm systems should move beyond a binary choice between static and dynamic designs, and instead adopt hybrid communication protocols that adapt interaction structure based on task phase, uncertainty level, and coordination demands. Such hybrid designs offer principled pathway balance efficiency and flexibility across diverse multi-agent. settings.
99

---

Published in Transactions on Machine Learning Research (02/2026)
7.5 Discussions
7.5.1
Who Communicates with Whom and What Is Shared
The first foundational dimension in LLM-based multi-agent communication concerns who participates in communication, whom messages are directed toward, and what information is exchanged. Across the surveyed systems, these design choices are not incidental but tightly coupled task structure, coordination requirements, and system scalability. Unlike emergent language settings, where communication channels are typically constrained and learned from scratch, based systems inherit a rich linguistic prior, shifting the focus from inventing symbols to structuring interaction.
Early LLM multi-agent frameworks primarily adopt small, fixed agent sets with clearly defined roles, resulting in explicit and often static who whom relationships. Centralized and hierarchical systems constrain who can speak to whom through orchestrators or planners, while decentralized and sebate-based systems allow all agents to address each other freely. These choices directly shape what is communicated. Pipelino-style systems emphasize task instroction, intermediate artifacts, and execution summaries, whereas recentralized or debate-based systems encourage argumentative reasoning, persuasion, and belief updates.
As agent populations scale, several works shift from explicit addressing to implicit aggregation mechanisms, such as shared message pools or memory buffers. In these settings, whom a message intended for becomes ambignous by design, and what is communicated must be general enough to be interpretable by multiple recipients. This transition highlights core tension in LLM communication: increasing expressiveness and flexibility often comes at the cost of precision and control. Overall, LLM-based communication systems demonstrate that who, whom, and what are jointly determined by architectural constraints rather than learned implicitly, contrasting sharply with emergent language paradigms.
7.5.2 When and Where Communication Is Necessary
The second dimension concerns when communication is triggered and where it occurs within the task environment. In LLM-based systems, communiatiin is rarcly continuous. Instead, it is activated specific decision points, such as task decomposit ion, conflict resolution, uncertainty reduction, or coordination checkpoints. This selective invocation reflects both computational cost considerations and the assumption that language is most valuable at moments of ambiguity or coordination failure.
Different application environments impose distinct temporal and spatial pressures on communication. In workflow automation and coding systems, commuicattio typically occurs at stage boundaries, such re- quirement analysis, code review, integration. In embodied or interactive environments, communication is tied to spatial context and temporal urgency, for example during navigation, object handoff, or joint manip- ulation. Virtual social environments and games further complicate this picture by requiring commuication during belief formation, alliance shifts, or adversarial reasoning.
These patterns indicate that where agents are situated, whether in symbolic workflows, simulated worlds, physical environments, directly influences when commmncatiin becomes necessary. Unlike emergent lan- guage benchmarks, where communication frequency is often fixed by experimental design, LLM-based sys- tems increasingly treat communication as a resource to be scheduled. This observation motivates analyzing when and where jointly, as environmental structure determines not only the availability of communication but also its strategic value.
7.5.3 Why Communication Is Used and How It Is Implemented
The final dimension addresses why LLM agents communicate and how communication mechanisms are realized. In contrast to emergent language, where communication emerges to maximize reward under partial observability, LLM-based communication is typically introduced to compensate for limitations in planning, coordination, or generalization. Communication is used to externalize reasoning, align plans, negotiate roles, or improve robustness in the face of uncertainty.
The how of commuication reflects these motivations. Static protocols impose fixed interaction patterns that support reliability and reproducibility, particularly in workflows and pipelines. Dynamic protocols, by
100

---

Published in Transactions on Machine Learning Research (02/2026)
contrast, allow agents to adapt communication partners, frequency, and content in response to intermediate outcomes or environmental feedback. Hybrid systems combine both approaches, using structured stages augmented by opportunistic dialogue when conflicts or ambiguities arise.
Importanll, LLM-based systems rarely optimize communicatioo end-to-end. Instead, communication mech- anisms are hand-designed and layered on top of pretrained models. This design choice enables rapid deploy- ment but raises questions about efficiency, scalability, and interpretability. Compared to emergent language systems, where communication and policy are jointly learned, LLM-based systems trade optimality for flex- ibility. Understanding why this tradeoff' is beneficial and how it affects coordination quality remains an open challenge.
Taken together, the wwhomwhhee when where, and why-how dimensions reveal that LLM-basck multi-agent communication is best understood a systems design problem rather than a purely learning- driven phenomenon. Communication is introduced deliberately, structured explicitly, and evaluated function- ally. While this contrasts with the bottom-up nature of emergent language, it enables practical deployment at scale. Bridging these two paradigms remains a promising direction for future research.
7.6 Bridge: LLM-Grounded MARL and Its Limitations
Purpose of this bridge. This subsection connects MARL-Comm (Sec. 5 and EL-Comm (Sec. G with re- cent LLM- based multi-agent systems, clarifying why neither MARL- Com-Coom nor EL- Comm alone fully resolues commumication challenges, and motivating L.M-grrunded MARL as principled intermediate paradigm.
From MARL-Comn to EL-Comm: Progress and Persistent Gaps Communication is a fun- damental mechanism for collaboration in multi-agent systems, particularly in ad-hoc teamwork settings where agents lack shared protocols or prior agreements Mirsky et al. (2020). MARL has shown that agents can learn communication strategies end-to-end by optimizing task rewards, as exemplified by Comm- Net (Sukhbaatar et al. (2016) and IC3Net (Singh et al. (2018) (reviewed in Sec. 5 However, communication learned purely through task-driven optimization is often highly environment-specific, producing protocols that are difficult for humans or non co-trained agents to interpret or reuse Kottur et al. (2117; Li et al. (2024b). These limitations motivated EL research, which seeks to encourage agents to develop more struc- tured and interpretable communication through interaction. Nevertheless, many EL-Comm approaches, particularly referential games trained with symbolic or pixel-level inputs, still exhibit limited interpretabil- ity and poor generalization beyond co-trained agents Lazaridou et al. (2018) (reviewed in Sec. G At deeper level, this challenge stems from how communication is represented and learned: many EL-Comm methods rely on atomic or compositional symbols without explicit semantic grounding, as surveyed in Zhu et al. (2024), making the resulting protocols difficult to interpret or transfer. Although some works explore semantic representations that enable zero-shot EL communicttin Tucker et al. (2021): KKrten al. (2023) most. Fl-baeed protocols remain task-specific. Moreover, both MARL-Comm and BL-Coom typically require large amounts of interaction data to overcome joint exploration challenges, reflecting the data-hungry nature of reinforcement learning. Inductive-bias approaches, such as positive signaling and pos- itive listening Eooles et al. 2019), partially alleviate this issue but do not eliminate it. Attempts to further incorporate linguistic structure from human language Tucker et al.) Lazaridou et al. (2020); Lowe et al.)
(2 Agarwal et al. (2019a) face fundamental obstacles due the mismatch between data-intensive RL training and the limited availability of human-annotate interaction data Chaabouni et al.
20019 Lowe et al.
(2020) Taken her, these results indicate that learning communication solely through reward-driven in-
teraction, even with emergent structure, leaves a persistent gap between task efficiency, generalization, and human-aligned interpretability.
Why LLMs Became Attractive for Multi-Agent Communication These limitations motivated growing interest in LLMs complementary mechanism for multi-agent communication. LLMs provide access to rich, human-aligned language priors learned from large-scale text corpora, enabling agents to communicate in natural language, express intent, and generalize across tasks without requiring task-specific co-training. Recent instruction-tune tuned models demonstrate strong performance in cooperative and embodied tasks, producing fluent and context-aware communication Park et al. (2023); Li al. (2023c). In contrast to MARI-Comm and EL-Comm, T.L.--baedd agents can leverage linguistic structure and world knowledge
101

---

Published in Transactions on Machine Learning Research (02/2026)
that would be infeasible acquire through interaction alone. However, LLMs introduce different class of limitations. Their language generation is weakly grounded in environment dynamics, which can lead to hallucinated or inconsistent plans that are not executable real or simulated worlds Mahowald et al. (2024). While attempts have been made to ground LLMs with RL or interactive data collected from environments such as World Models Xiang et al. (2023), Interactive RL Environments Carta et al. (2023) and RL Embodied Environments Tan et al. (2024), these efforts largely focus on single-agent settings and none of them involve teamwork communication among multiple embodied agents.
Motivation for LLM-MARL Hybrid Systems Together, these observations motivate LLM- grounded MARL as bridge between task-optimized communication and langnage-grounder coordination. Rather than replacing MARL communication with unconstrained language, hybrid systems integrate LLMs high-level communicative or reasoning components while preserving MARL's grounding in environment dynamics and reward optimization. This hybrid perspective aims to combine the adaptability and inter- protability of natural language with the stability and control-oriented guarantees of MARL. At the same time, hybridization exposes unresolved tensions between linguistic abstraction (emphasized LLM-based systems), and control-oriented grounding (central to MARL). Understanding how these ten- sions shape communication behavior is essential for designing reliable multi-agent systems. The remainder of this subsection analyzes how such hybrid systems operationalize communication and what limitations remain.
EL and
7.6.1 Why and How Communication Emerges in LLM-MARL Hybrid Systems
This part examines reprosentative LLM-MARL hybrid systems through the lenses of why com- munication is required and how communication operationalized in our Five Ws Taxonomy. hybrid systems, communication not an auxiliary interface but mechanism shaped directly by environment dynamics, reward structure, and long- mor-horizon optimization constraints. LLMs contribute flexible language generation, abstraction, and social reasoning, which explains why communication is introduced in many MARL environments: to negotiate, explain intent, reduce coordination uncertainly, or reason under partial observability without exhaustive retraining. MARL, in contrast, constrains how communication is used: messages must be grounded in policies, influence future actions, or improve long-term reward. As a result, communication in LLM MARL systems occupies a middle ground between unconstrained natural language interaction and purely latent learned protocols.
Before analyzing individual systems, situate LLM MARL hybrids relative to pure MARL and pure LLM- based multi-agent systems along key communication dimensions in Table 17 This comparison clarifies that hybrid systems neither replace learned communication with free-form language nor simply add language on top of MARL; instead, they constrain natural language interaction through policy optimization, shaping both expressiveness and stability.
Aspect
Pure MARL
Pure LLM-MAS
LLM-MARL Hybrid
Message form
Latent vectors
Natural language
Hybrid (language grounded policy)
Communication driver Reward optimization
Reasoning and prompting
Reward optimization + rea- soning
Adaptivity mechanism
Learned end-to-end
Prompt-basee based adjustment
Policy-constrainee adapta- tion
Table 17:
Comparison of communication mechanisms across pure MARL, pure L.LM-based multi-agent
systems, and LLM MARL hybrid architectures.
We now illustrate these design principles through representative lighting how different commuication motivations lead to distinct structural choices.
hybrid systems, high-
Case Study: Cicero Cicero I (AIR)) exemplifies setting where the why to communicate is driven by strategic alliance formation, persuasion, and long-horizon commitment. management in the game of Diplo-
102

---

Published in Transactions on Machine Learning Research (02/2026)
macy. Communication is indispensable because successful play requires agents to coordinate intentions, negotiate agreements, and maintain consistency across many future decision steps under shifting alliances. The how of communication in Cicero tightly couples I.L.-generated natural language dialogue with MARL-trained strategic policies. The language model generates proposals, explanations, and commitments, while reinforcement learning constrains these utterances through policy optimization to ensure alignment with future actions. As summarized in Table 17 Cicero represents hybrid communication regime where natural language is explicitly constrained by long-horizon policy optimization. Compared to pure LLM-based negotiators, Cicero stabilizes dialogue by grounding language in reinforcement learning; however, the lan- guage and policy modules are trained largely separately and connected through intention estimates, limiting end-to-end alignment between communication learning and control.
Case Study: LangGround LangGround Li et al. (2024b) addresses a related but distinct motivation, where the why to communicate arises from the need for buman-interpretable and reusable communica- tion in ad-hoc teamwork scenarios involving unseen teammates and novel task states. Unlike Cicero, the primary objective is not strategic negotiation among agents alone, but aligmment between agent communi- cation and human language to support effective human-agent collaboration. The how of communication in LangGround differs fundamentally from Cicero by training action and communication end-lo-end within MARL. Communication policies are jointly optimized using reinforcement learning and supervised ground- ing losses derived from synthetic interaction data generated by LLM agents. This design aligus the learned communication space with human natural language while preserving reward-driven coordination. As sum- marized in Table 17 TangGround exemplifies a hybrid architecture where natural language grounding is integrated directly into MARL optimization rather than appended as a separate module. Compared to Cicero's modular design, LangGround enables tighter coupling between communication and control, sup- ports zero-shot generalization in ad-hoe teamwork, and facilitates translation between latent embeddings and human-readable language, making communication both task-effective and human-aligned.
Case Study: Werewolf Werewolf Xu et al. 2023c represents a contrasting hybrid setting where the why to commuicate is driven by deception, hidden roles, and belief manipulation rather than explicit coordination or alignment. Communication is essential for influencing other agents' beliefs, obscuring true intentions, and surviving under adversarial and mixed-motive incentives. The how of communication in Werewolf reflects this strategic objective through a tight integration of LLM reasoning and reinforcement learning-baseed decision control. The LLM is used to perform deductive reasoning and generate a diverse set of candidate language actions, helping mitigate intrinsic biases inherited from pretraining. A reinforce- ment learning policy then selects among these candidates to optimize long-term game outcomes. In this design, language functions as a probabilistic and strategic signal rather than a binding commitment, with RL shaping when agents speak truthfully, deceive, or remain silent across repeated interactions. Com- pared to Cicero, where language is used to form and maintain cooperative commitments aligned with long- horizon planning, communicttin in Werewolf is adversarial and belief-centric, emphasizing strategic signaling over coordination. In contrast to LangGround, which aims to align MARL communicatio with iemman-iterrretablle language for ad-hoc teamwork and generalization to unseen partners, Werewolf does not prioritize interpretability or transferability. Instead, its communication is optimized for in-game strategic advantage under fixed adversarial roles, with reinforcement learning directly regulating the deployment of language to maximize reward. As summarized in Table 17 Werewolf exemplifies a hybrid communication regime where natural language serves as a strategic signaling interface, while reinforcement learning governs action selection and policy consistency under adversarial incentives.
Case Study: FAMA FAMA Slumbers et al. (2023) represents a cooperative L.MM-MAR.. hybrid where the why to communicate is driven by the need for reliable coordination under partial observability across multiple cooperative tasks. Unlike social or adversarial sottings, communication in FAMA is motivated by execution-level coordination: individual agents lack sufficient local information to act optimally without exchanging state summaries, intentions, or intermediate reasoning. The how of communication in FAMA reflects this coordination-centric objective through two tightly coupled mechanisms. First, LLM-based agents are aligned with task-specific functional knowledge via a centralized on-policy MARL update rule, ensuring that communication remains reward-aligned and policy-consistent. Second, agents exchange information
103

---

Published in Transactions on Machine Learning Research (02/2026)
using natural language message passing, leveraging LLMs' linguistic strengths to convey task-relevant ab- stractions in an intuitive and interpretable form. This combination allows FAMA to outperform independent LLM agents and traditional symbolic MARL methods across diverse coordination tasks. Compared to Ci- cero, where communication supports long-horizon negotiation and commitment management, FAMA uses language strictly as a coordination signal rather than a tool for persuasion or alliance formation. Compared to Werewolf, where language functions as strategic and potentially deceptive belief signal, FAMA's 's com- munication is cooperative, non-adversarial, and execution-orientee.. Relative to LangGround, which focuses on aligning MARL communication with human language for ad-hoc teamwork and generalization to unseen partners, FAMA prioritizes functional alignment and task performance within fixed cooperative environments rather than zero-shot human interpretability. As summarized Table [7, FAMA exemplifies hybrid commuication pattern where natural language serves as an intuitive medium for state and intent sharing, while centralized MARL training constrains communication to support stable, coordinated policy optimization across tasks.
Case Study: ACC-Collab ACC-Collab Estornell et al. (2025) studies multi-agent collaboration in com- plex question-anan auestion-answeringg and reasoning tasks, where the why to communicate is driven by epistemic uncertainty and reasoning errors rather than environmental dynamics or partial observability. Individual agents may generate plausible but incorrect reasoning paths, making communication necessary to surface, evaluate, and correct intermediate conclusions through interaction. The how of communication in ACC- Collab is realized through an explicit aitor-critic dialogue framework inspired by MARL. One agent (the actor) generates candidate reasoning trajectories, while another agent (the critic) evaluates, critiques, and provides feedback, with reinforcement-stylle updates guiding improvement across iterative dialogue rounds. Unlike FAMA, which uses language to support coordination and execution in task environments, ACC- Collab operates in a purely línguistic setting without external state transitions. Unlike Werewolf, where language is strategically deceptive and belief-oriented, communication in ACC-Collab is corrective and accuracy-driven. Compared to Cicero, which grounds dialogue in long-horizon strategic commitments, ACC-Collab focuses on short-horizon reasoning refinement. In contrast to LangGround, which aims to align communication with human-inteerpeetable semantics for ad-hoc teamwork, ACC-Collab emphasizes im- proving solution quality through structured critique rather than semantic grounding. As summarized in Table 17 ACC-Collab exemplifies hybrid communication regime in which natural language interaction is structured by MART-inspired feedback mechanisms, enabling poliey-constrained reasoning refinement rather than action coordination or social negotiation.
7.6.2 Comparative Pattern Across Hybrid Systems
Table 18 makes clear that LLM MARL hybrid systems do not share a single communication template; in- stead, each system chooses communication strueture that matches its dominant. uncertainty source and incentive landscape. Cicero treats language commitment mechanism for long-horizon coordination: communication must remain policy-consistent so that promises map onto executable trajectories. Were- wolf flips this logic under adversarial incentives: language becomes a strategic belief signal, and RL is used to select among candidate language actions to counter intrinsic LLM bias and to manage when to reveal, distort, or withhold information. FAMA emphasizes cooperative exccution under partial obscrvabil- ity: communication is primarily a coordination substrate for sharing lask-relevant state and intent, while centralized MARL updates align the LLM agents toward functional knowledge for cooperation. ACC- Collab abstracts away the environment entirely and focuses on epistemic uncertainty: communication is structured as actor-critic critique loops that improve reasoning quality throngh iterative feedback, rather than through state sharing or negotiation.
Several design lessons for future LLM MARL co-design follow directly from these contrasts. (1) Match the communication role to the objective and uncertainty source. If the bottleneck is horizon coordi- nation, treat language as commitments and enforce message-action consistency (Cicero). If the bottleneck is adversarial belief dynamics, treat language as signals and let RI. learn when and how to deploy them under strategic pressure (Werewolf). If the bottleneck is decentralized execution under partial observability, treat language as structured state/intent summaries and couple it policy learning through coordination- centric updates (FAMA). If the bottleneck is reasoning reliability, treat language self-correction infras-
104

---

Published in Transactions on Machine Learning Research (02/2026)
Table 18: Communication motivations and mechanisms in representalive LLM MARM-MARLLL hybrid systems, high- lighting distinct functional roles of language across strategic, adversarial, cooperative, human-alii -lman-aligned, and reasoning-contee sottings.
System
Why Communication Needed
How Communication Structured
Cicero
FAIR
Strategic commitment: al- liance formation
Natural language negotiation
Policy-consistent dialogue
Lorg-horizon coordination
RI-constrained commitments
Managing promises under shifting coalitions
aligned with future actions
Werewolf
Xu
2023c
Adversarial belief dynam- ics: hidden roles
Language as a probabilistic be- lief signal
Belief manipulation and de- ception
RL-based selection among can- didate language actions
Strategic inference under certainly
Non-binding, strategically adap-
tive communication
LangGroue Ground
Li
al. (2024b)
Human--liggned coordina- tion: ad-hoc teamwork
End-to-end MARL training with language grounding
Interoperability with unseen teammates
Aligument between latent em- beddings and natural language
Bridging agent protocols and human language
Joint oplimization of task reward and linguistic inter- pretability
FAMA
Slumbers
et al.
2023
Cooperative execution:
Centralized on-policy MARL updates
partial observability
Task decomposition across agents
Natural language message pass- ing
Coordination under execution uncertainty
Functionally aligned, task-
relevant abstractions
ACC-Collab
Es-
fornell
8
Epistemic reliability: rea- soning uncertainty
Actor-critic dialogue roles
2025
Iterative
proposal-critique
Error detection and correction
rounds
howard-guides reasoning refine- ment
105

---

Published in Transactions on Machine Learning Research (02/2026)
tructure through explicit critique roles and reward-guided refinement (ACC-Collab). (2) Separate what LLMs are good at from what RL must guarantee, Across all systems, LLMs provide rich candidate messages and abstractions, while RL is most valuable when it (i) selects among candidates to reduce bias (Werewolf), (ii) constrains language to remain executable and temporally consistent (Cicero), or (iii) aligns communication with cooperative performance objectives (FAMA) and iterative improvement (ACC-Collab). (3) Prefer structured interfaces when stakes are high. The more communication must. drive control and coordination, the more these systems rely structured schedules, roles, interfaces (policy- consistency constraints in Cicero, centralized update structure in FAMA, eetor-critic roles in ACC-Collab), suggesting that future hybrids should avoid fully free-form chat and instead adopt protocol-aware natural language.
Overall, Table 18 supports a unified conclusion: communication in LLM MARL hybrids is best viewed reward-shaped, conttaol-o mechanism rather than generic language interface. Future LLM-MARL co-design should therefore start from the intended function of communication (commitment, signaling, coordination, critique) and then choose the minimal structure constraints, roles, and update rules- needed to make that function reliable, scalable, and learnable.
7.6.3 LLM Limitations in MARL Settings
While LLM-powered agents introduce expressive language, social reasoning, and rapid adaptation, their in- tegration into MARL settings exposes several fundamental limitations that shape both why communication is effective and how it can be reliably used. These limitations recur across different LLM MARL systems and evaluation settings, as summarized later in Table 19 (1) grounding and state alignment remain weak. In MARL environments, optimal communication is tightly coupled with latent environment states, long-horizon dynamics, and delayed rewards. LLMs, operating primarily on textual context, lack direct access to environment transition models and often rely on imperfect summaries or abstractions provided by other agents. As a result, language- baaee communication may drift from the true system state, leading to inconsistent coordination or compounding errors over time. This issue is particularly salient when lan- guage mediates strategic commitments or hidden information, as in Cicero and Werewolf] (FAIR); Xu et al. (203cc), and has also been observed in broader evaluations of LLM-augmented cooperation Mosquera et al. (2025). (2) policy consistency and temporal coherence are not guaranteed. MARL. communication protocols are learned jointly with policies, ensuring that messages influence future actions in predictable and reward-aligned manner. In contrast, LLM-generated messages are typically produced at inference time and may vary across identical states due to stochastic decoding or prompt sensitivity. This variability com- plicates credit assignment and undermines the stability required for long-horizon coordination. Even when hybrid systems constrain language to align with downstream actions (e.g., Cicero), this typically requires substantial scaffolding or spec-sp training (FAIR). Recent analyses of muulii-geent LLM failures further identify communication inconsistency and robustness key bottlenecks Cemri et al. (2025a)- (3) scalabil- ity and efficiency constraints differ sharply. MARL communication mechanisms are often explicitly designed to respect bandwidth limits, sparse messaging, and decentralized execution constraints. LLM-based communication, by contrast, incurs high token and computation costs and scales poorly with the number of agents or communication rounds. This limits its applicability in real-time, resourco-constrained multi-agent systems and motivates designs that restrict language to high-level negotiation or episodic coordination rather
than continuous control signals Shumbers et al. 2023 y Mosquera et al. (2025). (4) theoretical guarantees are largely absent. Classical MARL communicaation frameworks can be analyzed through Dec-POMDPss,
information bottlenecks, regret bounds, or return-gap guarantees. LLM-based communication currently lacks comparable formal guarantees regarding convergence, optimality, or robustness under partial observ- ability and non-stationarity. This gap is amplified in open-ended language settings where semannics are not optimized directly for control performance Cemri et al. (2025a). (5) failure modes are difficult to detect and correct. LLM agents may hallucinate, overgeneralize, ratives, especially in adversarial or mixed-moot -ottvee environments. Without explicit coupling to reward signals or environment feedback, such failures can persist undetected and propagate across agents. Mixed-motive dialogue-centric settings such as Werewolf make these risks especially clear, since communication can be strategically manipulated Xu al. (2023c), while broader empirical studies report fragility in cooperation and nobustness of L.M--based multi-agent behaviors Mosquera et al. (2025): Cemri et al. (025a
adopt persuasive but incorrect nar-
106

---

Published in Transactions on Machine Learning Research (02/2026)
Table 19: LLM limitations in MARL settings: manifestations, representative evidence, and practical impli- cations for hybrid
system design.
Limitation
Manifestation in MARL
Representative Works
Implication for De-
sign
Grounding
Language drifts from latent
Policy-aligned negotiation
Use structured state interfaces;
& state align- ment
environment state;
with partial grounding
Textual summaries omit long-
(Cicero) (FATR);
Restrict language
horizon dynamics;
Hidden roles and deceptive
high-level intent or episodic coordina- tion.
Errors compound time under partial observability.
signaling (Werewolf) Xu
et al. (003cc);
Empirical evaluation
Melting Pot Mosquera al. (2025).
Temporal coherence & consistency
Same state yields different
Long-horizon commitment alignment: requires scaf- folding (Cicero) (FAIR):
Constrain generation
messages;
strategies;
Unstable influence on down-
Enforce
message
stream actions;
Failure analysis highlights
action
consistency
Credit assignment becomes
communication inconsis-
checks;
ambiguous.
tency Ceuri et al.
2025a
Incorporate feedback-
driven updates.
Scalability & efficiency
Token and computation cost scale with agent count; Communication latency lim-
Centralized language (FAMA)
MARL with
Prefer sparse, event- triggered language; Keep low-level con- trol in MARL mod- ules.
messaging Slumbers al.
its real-Lime control.
2023):
Cooperation fragility
scale Mosquera et al. 2025).
Lack of guaran- tees
No convergence guarantees;
Multi-agent failure analy- sis emphasizes robustness gaps Cemri al. (025a))
Use MARL for control-critical chan- nels;
Weak robustness under non- stationarity.
Add verification or
safety layers.
Hard-to-detect failure modes
Hallucination and overgener-
Adversarial and mixed- motive signaling (Were- wolf) Xu et al. (2023c):
Add grounding and
alization;
consistency audits;
Persuasive but incorrect nar-
Use adversarial test-
ratives;
Robustness and cooper-
ing:
Strategic manipulation prop-
ation
failures Mosquera
Fallback to learned
agates across agents.
et al. 2025 Cemri al.
protocols when uncer- tain.
2025a
107

---

LLM/MAS Evidence of Limitation Published in Transactions on Machine Learning Research (02/2026)
Table 19 summarizes these limitations by mapping each failure mode to its typical manifestation in MARL settings, representative systems empirical studies, and the resulting design implications for hybrid LLM- MARL architectures. Taken together, these limitations indicate that LLMs are best viewed not as replace- ments for MARL communication, but as complementary components. Hybrid systems benefit most when LLMs are constrained high-level reasoning, abstraction, or negotiation (e.g., Cicero, ACC-Collab), while MARL mechanisms retain responsibility for low-level control, temporal consistency, and policy optimization) [AIR); Estornell et al. (2025); Slumbers et al. (2023). From a broader perspective, many of the limitations Identified above stem not only from current LLM architectures, but also from deeper mismatch between open-ended language generation and the assumptions underlying classical decentralized control and conmuiiatiion-consrrrnnn decision-making.
7.6.4 LLM Limitations from Decentralized Control Perspective
Table 20: LLM limitations through a decentralized control lens: each control requirement is paired with reprosentative MARL-Comm mochanisms and empirical evidence on LLM/MAS limitations.
Decentralized Con-
What It Implies
Representative
trol Requirement
for Communication
MARL-Comm Works (Align With Req.)
Well-defined semantics and grounding
Messages encode task-
Dec- PEMDPOMMP--ssse mod- eling
Cooperation may drift from la- tent state due
relevant information about
Bernstein et al.
latent state;
(0022)
Semantics are interpretable
Learned state sharing via
to weak ground- ing
under partial observability.
CommNet Sukhbaatar el al. 2016).
Mosquera el al. 2025
Reliability and predictability
Signals remain consistent across time;
Structured
aggregation attention in Tar-
Communication
and
inconsistency and brittleness
Coordination stabilizes under repeated interaction.
MAC Das et al.
(2018)
Stable learned coordina-
multi-agent LLM systems et al. 2025a
tion in TC3Net Singh et al. 2018
Cemri
Information efficiency and minimality
Messages are compressed and
Informatioon-ottlennck
Token-heavy
sparse;
inspired efficient com- munication Wang al. 2020b);
communica-
Bandwidth constraints are explicitly respected.
tion and scaling fragility Mosquera et al. 2025
Selective exchange mech- anisms in TarMAC Das et al. 2018).
Objective coupling (control optimality)
Messages optimized directly for control perfor- mance;
message learning with reward (CommNet,
End-to-end
Semantics arise from pretrain- ing rather than control objec- tives Cemri et al. (2025a)
feedback
Communication influences re- turn maximization.
IC3Net) Sukhbaatar et al.
2016 Singh et al. (2018).
Robustness under non-stationarity
Communication remains ef- fective as policies evolve; Coordination resists behav- ioral drift.
Decentralized policies commu-
Coordination instability and robustness break- downs Mosquera
with learned nication to miligate Singh
non-stationarity et al. 2018))
al. 2025
Cemri et
2025a
From the standpoint of decentralized control and decentralized information theory, communication is not merely an auxiliary interface but a control-relevant information channel with well-defined semantics,
whose design critically determines system performance Bakule & Papik (2012); [Trpakiss] (2004). Classical team decision theory and communictiion-constraaini control formalize this view by modeling communica-
108

---

Published in Transactions on Machine Learning Research (02/2026)
tion as structured signaling mechanism subject to partial observability, bandwidth, delay, and noise, and by explicitly characterizing how information flow affects optimal control policies Witsenhausen (1968) Ho
et al. 1972]; Tatikonda, & Mitter (2004). In these formulations, the value of a message is defined by its contribution to state estimation, coordination, and long-horizon control objectives, rather than by linguistic expressiveness.
This perspective is directly reflected in Dec-POMDPs and related decentralized control models, which assume that communicated signals are optimized jointly with control policies and can be analyzed in terms of sulficiency, redundancy, and performance loss (e.g., Dec-POMDP formalizations [Bernstein et al. (2002) Many MARL-Comn mechanisms align naturally with these assumptions: messages are learned end-to- end with policies and shaped by reward feedback, often under explicit efficiency or sparsity constraints. Represent alive examples include continuous message pooling in ComunNet /Sukhbaalar et al. (2016), selective exchange in TarMAC Das et all (2018), and learned gating or graph-based coordination in IC3Net (Singh et al. (2018). Morcover, several MARL-Comm studies explicitly adopt cntemattion-thooretie objectives, such as compression, minimality, or information bottlenecks, to control communication cost while preserving lask-relevant information Waang et al. (2020b).
In contrast, LLMM-ased communication departs fundamentally from this tontrol- paradigm. Lan- guage is treated as an open-ended interface whose semantics emerge implicitly from large-scale pretraining rather than being optimized for a specific decentralized control objective. As a result, it becomes difficult to formally reason about whether generated messages are sufficient, minimal, or reliable for coordination under partial observability and non-stationarity. Empirical studies of multi-agent LLM systems report fail- ure modes- including inconsistency across time, coordination fragility, and robustness breakdowns that directly violate predictability and reliability assumptions central to classical decentralized control Mosquera et al. (2025): Cemri et al. (2025a). Furthermore, stochastic decoding and prompt sensitivity introduce unbounded variability, which is incompatible with the stability requirements of long-horizon decentralized decision-making.
These obscrvations suggest a principled division of labor hybrid systems. LLMs are best integrated as high-level planners, negotiators, or abstraction mechanisms that operate above the control layer, while MARL-based communication remains essential for enforcing decentralized control laws, maintaining temporal coherence, and ensuring robustness under uncertainty. Bridging these paradigms, therefore, requires not only architectural interfaces but also new theoretical tools that reconcile open-ended language with the foundational principles of communication-constiin decentralized-conle requirements translate into communication desiderata, and how reported LLM multi- agent failure modes violate these desiderata relative to MARL-Comm mechanisms.
decentralized control. Table 20 summarizes how key
8 Discussion, Open Problems, and Future Directions
In this section, we provide an in-depth discussion multi-agent communication (MARL-Comm), emergent language in multi-agent systems (MA-EL), and multi-agent communication with large language models (LLM-Comm). These discussions expand on the main text and outline key challenges, methodologies, and future dircctions in each area. Our analysis focuses on how communication has been modeled, Icarned, and optimized within multi-agent systems, providing a comprehensive summary of this paper while offering insights for future research. For clarity, we use the following abbreviations: MARL-Comm Multi-Agent Agent Reinforcement Learning Communication, MA-EL- Emergent Language, and LLM-Comm Multi-Agent Communication with Large Language Models.
8.1 Discussions Multi-agent Communications
A Common Usage of Communication Models in Multi-Agent Systems Communication models play a crucial role in multi-agent systems by enabling agents to share information, coordinate actions, and enhance decision-making. These models can be broadly categorized based on their structure and learning mechanisms, and they are commonly used in MARL-Comm, MA-EL, LLM-Comm. In MARL, communi- cation models facilitate cooperation by allowing agents to exchange observations, share rewards, or align policies. They be integrated into centralized learning frameworks, where global controller aggregates
109

---

Published in Transactions on Machine Learning Research (02/2026)
and distributes information, into decentralized architectures, where agents autonomously decide whom and when to communicate with. To be specific, the communication process can be formulated as
m ~ C (oot"" .af- 1s)). a²" ~ m. (-1 St: Ne..mm/.laa (
ml)) mij E M,
where m represents the message sent from agent to agent j at time step t, C()) denotes the communication function that generates messages conditioned the current environment state 8, and the agents' actions from the previous timestep, and T. ( St, (m.,)) denotes the policy of agent i that selects an action at time & based on its current observation and the set of received messages. This time-indexed formulation makes explicit that communication and action selection are staged across timesteps, thereby avoiding circular dependencies within a single decision epoch. All communication models exhibit unique advantages and trade-offs: (1). Graph-based communication encodes agent interactions using predefined or dynamic graphs, allowing for structured message passing but requiring scalability optimizations. (2). Attention based communication uses transformer-baat based mechanisms to selectively attend to relevant information, improving robustness but increasing computational overhead. (3). Emergent communication enables agents to learn their own protocols, enhancing adaptability but often lacking interpretability. (4). LLM-based communication leverages pretrained language models to enhance expressiveness but may suffer from grounding issues in task-specific settings. Each of these models provides different capabilities depending on the environment and the desired level of communication efficiency. For instance, graph-based models are well-suited for structured multi-agent interactions, whereas LLM-based approaches excel in human-agent, communication and natural language coordination. Future research should focus integrating adaptive communication mechanisms that dynamically adjust based on task complexity, scalability requirements, and real-time constraints.
The unique usages of each communication model in multi-agent. systems Fach communication model has distinct advantages and is suited for different multi-agent scenarios. (1) Graph-based commu- nication structures agent interactions as a communication graph, where nodes represent agents and edges determine message passing. This structured representation is effective for tasks requiring topology-aware coordination, such as traffic control and distributed sensor networks. Graph-based models can be either static, where communication links are predefined, or dynamic, where agents learn whom to communicate with based on relevance and necessity. (2) Attention-based communication utilizes mechanisms such as transformers to allow agents to selectively attend the most relevant messages, improving scalability and robustness. By dynamically weighing information from different sources, attention-based communica- tion enhances coordination efficiency in large-scale multi-agent systems while reducing redundancy in message exchange. (3) Emergent. communication enables agents to develop their own communication protocols through interaction, rather than relying on predefined language or rule-based exchanges. This type of communication is particularly useful cooperative tasks where agents need to establish novel coordi- nation strategies in zero-shot or few-shot scenarios. However, ensuring compositionality and interpretability remains challenge. (4) LLM-based communication integrates large language models to facilitate natural language-based interactions among agents, making it particularly valuable for human-agent collaboration and generalist Al systems. LLMs can serve as a high-level reasoning module, allowing agents to infer intentions, interpret ambiguous instructions, and align communication with human-like stucturess. However, challenges include grounding in specific contexts and reducing hallucinations in generated messages. (5) Adaptive communication policies leverage reinforcement learning optimize when and whom to communicate with, reducing unnecessary message exchange and improving bandwidth efficiency. These policies are crucial in scalable multi-agent reinforcement learning (MARL), where excessive communication can lead to overhead and inefficiency. Future research should explore hybrid communication architectures that combine structured passsag-paasing mechanisms with learned communication strategies, ensuring scalability, adaptability, and robustness across different multi-agent domains.
Integrated use of different communication models in multi-agent systems The unique usages and advantages of each communication model form the basis for integrating different approaches in multi-agent communication, and several works have explored this direction. Here, we highlight some key examples. (1) Graph-based and attention-based communication can be combined to leverage structured message passing while dynamically selecting relevant information, enhancing scalability in large-scale multi-agent coordina- tion. (2) Emergent. communication can be integrated with reinforcement learning to allow agents learn
110

---

Published in Transactions on Machine Learning Research (02/2026)
communication protocols while optimizing their policies, enabling more adaptive and decentralized inter- actions in complex environments. (3) LLM- based communication can be used as a high-level reasoning mechanism, complementing low-level MARL-based message exchange to enable human-interppretable and goal-oriented communication. (4) Hybrid models incorporating both explicit and implicit communication strategies allow agents to switch between direct messaging and behavior-based signaling. improving efficiency in tasks requiring limited bandwidth privacy constraints.
(5) Adaptíve communication policies can be in- corporated into any of these models, allowing agents to dynamically decide when and whom to communicate with, balancing message complexity and coordination effectiveness. Future research should further explore the integration of structured, learned, and language-based communication, ensuring that multi-agent systems remain robust, interpretable, and scalable across diverse environments.
A summary of the base communication models in multi-agent systems The different communica- tion models used in multi-agent systems can be categorized based on their structure and learning mechanisms. (1) In MARL, communication strategies can be broadly classified into explicit and implicit communication. Explicit communication methods include graph-based and attention-based models, where agents directly ex- change messages to enhance coordination. Implicil communication, on the other hand, relies on agents learning to infer information from observed behaviors, reducing bandwidth but requiring stronger inference capabilitics. (2) In emergent communication, agents develop their own structured protocols through in- teraction, often optimizing for lask-specific coordination while balancing generalization and inderprelability Research has explored different learning objectives, such as maximizing mutual information, reinforcement learning rewards, or structured learning constraints to encourage compositionality and efficiency. (3) In large language model-based communication, pretrained models facilitate agent interactions through aamaan--nleepretablle messages, allowing for seamless integration of nalural language underslanding in multi- agent collaboration. However, ensuring grounding and alignment with task-specific requirements remains an open challonge. (4) Hybrid communication architectures combine multiple modcls, such as inte- grating learned allention mechanisms with structured graph-bused. message passing or combining emergent communication with LLM reasoning modules. These approaches aim to balance interpretability, efficiency, and scalability. (5) Finally, adaptive communication policies allow agents to dymamically decide when and whom to communicale wilh, reducing unnecessary message exchange while maintaining effective coordi- nation. Future research should focus on refining the balance between structured and learned communication approaches, ensuring that multi-agent systems achieve robust, interpretable, and scalable communication strategies across diverse applications.
Seminal works of communication models in multi-agent systems The development of communi- cation models for multi-agent systems has followed diverse trajectories, with some models sceing greater adoption in certain domains than others. The impact of a communication model often depends on whether there are seminal works in the category, as many subsequent research efforts extend these foundational studies. Here, we highlight seminal contributions in this area. Graph-based communication has been widely studied in reinforcement learning contexts, where agent interactions are modeled as graph structures, and information is propagated through message passing mechanisms. Many works formalize communica- tion as function over graph structure:h (1+1) = & (A1), -EEN(() v(h))) (1) mmjs)): where h; is the hidden state of agent i at timestep f, my.s is the message received from neighboring agent j, and & and yb are Iearned transformation functions. Attention-based communication leverages transformer-based archi- tectures to allow agents to dynamically weigh the importance of messages. This mechanism is particularly effective in large-scale systems where message relevance changes based on task demands. The communi- exp(f(h,,A,))
cation process can be modeled using attention weights: 0i.jj EvENTO, 34) where 0g represents
the attention weight assigned by agent i agent j, and f(h,, hy) a scoring function that determines the relevance of agent j's message. Emergent communication models have introduced new paradigms in reinforcement learning, where agents develop their own symbolic language through optimization. Re- search in this area often seeks to maximize the mutual information between the transmitted and received messages: max I (m; s) = (m) - [m]ss, where m represents the communicated message and s is the observed state. The challenge in emergent communication lies in ensuring compositionality, interpretability, and generalization across different. agent populations. LLM-based communication introduces pretrained
111

---

Published in Transactions on Machine Learning Research (02/2026)
language models as foundation for natural communication between agents. Large language models provide agents with rich semantic representations, enabling them to process and generate messages that align more closely with human language. However, challenges such as grounding messages in task-relevant information exp(f(s.a.m))
remain critical: p(mls,a) = Em exp(f(s.o.m"))' where p(m|s, a) represents the probability of generating mes-
sage m given state 8 and action a, and f(s, a, m) is a learned function capturing relevance. Some seminal works have pioneered new paradigms in multi-agent communication. Graph-based communication has been foundational in distributed reinforcement learning, while emergent language research has opened up avenues for self-learned symbolic exchanges. The use of large language models represents a shift toward integrating structured natural language into agent communication. Future research should continue exploring the inter- section of these methods, leveraging hybrid architectures that integrate structured message- passsnnggg with adaptive learning- based communication strategies.
A summary of the issues and extensions of communication models in multi-agent systems Communication models in multi-agent systems address various challenges and enable extensions that im- prove scalability, adaptability, and robustness. Below, we categorize key issucs these models aim to solve, along with potential research directions for further improvements. Specifically, six major extensions have been explored: multi-task (MT), multi-agent (MA). hierarchical (Hier) learning, model-based (MB) communication, policy safety, and policy generalization. While significant progress has been made, many challenges remain unsolved, requiring further exploration. (1) Communication efficiency and band- width constraints: Many multi-agent systems operate under limited communication resources, requiring models to minimize message exchange while maintaining effective coordination. Graph-based and attention- based commmication mechanisms mitigate unneccssary transmissions by dynamically selecting relevant information. The effectiveness of these approaches is often evaluated through message entropy and commu- nication sparsity: minge H(m[s,o) a) subject to 7(m; 8,;;8,0 a) 2 X, where H Hmm]s, a) represents the entropy of transmitted messages given state-action pairs, ensuring minimal redundancy while maintaining sufficient in- formation flow through a mutual information constraint I I(m; s; a). (2) Multi-modal and heterogencous input: Agents in real-world scenarios must process multiple input modalities, such as vision, language, and structured data. Communication models designed for nisms to integrate these diverse signals. A common approach is to embed multiplc modalitics into a shared latent space before generating communication messages: z = Smodality ($1, , .,); m = g(z), where fmodality encodes inputs from different modalities $2, , Sn, and g generates a structured message m. (3) Generalization to new partners and environments: Ensuring that communication models generalize beyond their training conditions is an ongoing challenge. Emergent communication systems often struggle with partner adaplation, requiring mechanisms lo align agent language representations dynamically. One approach involves optimizing for alignment loss between agents: minge DKL (p(m:) [s)[p((m)s)) Vi,j E A, where DKL is the Kullback-Leibler divergence ensuring that message distributions between agents i and j remain consistent. (4) Scalability in large-scale multi-agent communication: As the number of agents increases, traditional communication protocols face scalability issues due to excessive message over- head. Attention-based models address this by dynamically filtering irrelevant messages. Scalable commu- nication is often formalized as a minimization problem over message complexity while maintaining task performance: minre Σ |myl, subject to (T) 2 Juhreshold, where |m]] represents the message size of agent i, and (TT) is the expected return of the multi-agent policy, constrained to ensure effective coordi- nation. (5) Grounding and interpretabilitty in LLM-based communication: Large language models introduce challenges in aligning generated messages with lask-relevant information. Ensuring messages remain grounded in the environment requires integrating reinforcement learning with human feedback mech- anisms: maxx; Elrgrouuded (m,,, s,)] where Igrounded (m, 8, a) mcasures how well the message m aligns with meaningful environmental information. Future research should focus on hybrid architectures that com- bine structured message-passing with adaptive learning, ensuring that multi-agent communication is both efficcent and interpretable across diverse applications.
fusion use attention- based mecha-
Foundation models, Algorithms, and Data Among all the communication models in multi-agent sys- tems, transformer-based and LLM-driven communication approaches have demonstrated some of the most promising results. (1) Many algorithms leveraging attention-based communication have achieved success in compler mulli-agent coordination lasks, including real-world applications such as robotic teamwork, au-
112

---

Published in Transactions on Machine Learning Research (02/2026)
tonomous vehicle swarms, and large-scale multi-agent competitions. (2) The use of transformers enables agents to process long-range dependencies in communication, making them highly effective for handling sequential interactions and multi-modal inputs. Agents leveraging transformer-basee communication can dynamically adjust message relevance, allowing for efficient collaboration in dynamic envirouments. (3) In particular, large-scale foundation models such as GPT-based architectures have been explored for multi- agent dialogue and cooperalive problem-solning, where a single model can adapt to multiple domains, from robotic task ezecution to human-agent interaction. Recent research has explored the integration of pretrained language models with reinforcement learning for communication optimization, demonstrating cross- domain generulization effects. The ability to fine-tune large models for mullii-ggen reasoning suggests a paradigm shift towards data-driven communication learning, where pretrained models provide agents with rich seman- tic representations for structured coordination. However, despite these advancements, foundation models introduce new challenges, such computational efficiency, real-time adaptability, and grounding in task- specific enwironments. The question remains whether future development should be data-drinen (leveraging large-scale pretraining) or algorithm-driven (designing more efficient communication-speeifee architectures). Regardless, future research must emphasize scalability and computational efficiency, ensuring that commu- nication models remain robust, interpretable, and deployable in weal-world mulli-agent scenarios.
8.2 Beyond ML: Epistemic Perspectives on Agent Communication
While the preceding sections of this survey focus on algorithmic realizations of communication in MARL, emergent language, and LLM-based multi-agent systems, Communication has long been examined form segnitivescience) of belicf-aware reasoning in philosophy [ewis (2008); ; Hintikka (1962) , cognitive science Premack & Woodruff
1978 Tomasello (2009), and general artificial intelligence Russell el al. (195).. where communicative acts are treated as deliberate actions selected based on agents' beliefs about others' knowledge, intentions, and mental states. In these traditions, communication is not merely a channel for information exchange, but a deliberate action chosen based on an agent's 's beliefs about other agents' knowledge, intentions, and mental
states, formalized in speech act theory and cooperative models of meaning Austin 1975 Searle (9699
Grice 1957).
A central idea in this line of work is that agents communicate in order to influence or coordinate be- liefs. Epistemic and doxastic logics formalize this perspective by explicitly modeling agents' beliefs and knowledge Meyer] (2003), often using Kripke semantics to reason about nested beliefs (e.g., what one agent believes about another agent's beliefs) Orlowska (1990). Within these frameworks, commuicatiin decisions are naturally framed around questions of when to communicate, why communication is necessary, and to whom information should be conveyed sovveyed-dimensonn ddmensionss that closely align with the Five Ws organizing principle adopted in this survey.
From this perspective, many challenges observed in modern multi-agent learning systems can be reinterpreted instances of implicit. belief reasoning Levesque| (1984). Tn MARL-baaed communication, agents learn mes- sage policies that indirectly encode beliefs about other agents' future actions, but these beliefs remain latent and task-specific. Emergent language systems similarly rely on interaction to align internal representations, yet lack explicit mechanisms for modeling or reasoning about others' mental states. LLM-based agents, by contrast, exhibit strong surfaco-level theory-of-mind capabilities through language, but without formal guarantees that such reasoning is grounded in environment dynamics or aligned with control objectives.
Viewing communication through an epistemic lens helps clarify why issues such as grounding, interpretability, and generalization persist across learning- based approaches. Without explicit representations of belief, in- tent, or audicnce, modern systems must rediscover specch-act-like behavior through experience alonc. This observation suggests that future progress may benefit from lighter integration between learning- communication mechanisms and classical models of belief-aware reasoning, even if such models are used only as conceptual guides rather than explicit symbolic components.
Importantly, the goal of incorporating epistemic perspectives is not to replace data-driven or reinforcement learning approaches, but to situate them within a broader intellectual context. By connecting modern learning-bead based communication systems to foundational ideas in epistemic reasoning and coguitive science, we highlight that contemporary communication learning revisits long-standing questions about intentional action, belief alignment, and social reasoning under uncertainty. This broader view complements the technical
113

---

Published in Transactions on Machine Learning Research (02/2026)
focus of the survey and provides additional guidance for designing agents that communicate effectively, robustly, and intelligibly in complex multi-agent environments.
8.3 Perspectives on Future Directions
Based on the discussions above and the main content in previous sections, present some perspectives future research directions in multi-agent communication. We categorize these directions into four key areas: theoretical models, algorithm developments, general benchmarks, and human-centric research. We believe that more open challenges can be identified from the detailed discussions in carlier sections, and we have included specific comments on future work within the relevant sections. Each of these categories represents crucial aspect of advancing multi-agent communication, ranging from foundational theoretical advancements to practical considerations in real-world deployment.
8.3.1 Future Research on Theoretical Guarantees
Most works multi-agent communication focus algorithm design and empirical evaluation rather than theoretical analysis, highlighting a need for deeper theoretical research in this field. One promising direction is the formal derivation of convergence guarantees and performance bounds for key communication frameworks, such as attention- message passing and emergent communication protocols. Establishing rigorous mathematical foundations will enable researchers to design communication strategies that are both scalable and provably efficient.
Theoretical Guarantees for MARL Communication Most MARL communication studies focus on learning-basee methods that improve coordination, but thcoretical results on optimality, convergence, and performance bounds remain scarce. A fundamental question is: how much does communieation improve the return in cooperative MARL, and what is the theoretical gap between policies with and without commu- nication? Given a cooperative MARL setting, where J (T) is the expected return under optimal fully centralized policy TT,, and J(Tc) is the expected return under communcatioon-onstaainn policy TC, the return gap can be defined as:
AJ = - J"==)
(2)
where AJ quantifies the loss incurred due limited communication. A major research direction is deriv- ing upper bounds for AJ, showing under what conditions communication policies approach optimality, Additionally, scalability and communication efficiency are key concerns in large-scale MARL. Many existing methods use graph-based or attention-based message passing, but the theoretical trade-offis between band- width constraints, coordination performance, and sample efficiency remain unclear. A useful framework is minimizing communication costs while ensuring bounded performance loss:
min
C. subject to J(TT) - J(TC) Sc.
where C, represents communication costs at time t and e is the maximum allowable performance degradation. Future work should develop formal criteria for when and whom agents should communicate with to optimize efficiency. Finally, robustness in adversarial settings is crucial but underexplored theoretical problem. Communication channels can be subject to noise, delays, adversarial tampering. Establishing worst-case bounds on return degradation due to message corruption is essential:
T
J( = E
Σ
rı m f(mi n)
(4)
where mi is the ideal message, 7) represents a perturbation or adversarial noise, and f(m;. 11) is the re- ccived message. Theoretical guarantees on error correction, redundancy mechanisms, and robustness will be necessary for safe deployment of communcaation-hase MARI..
114

---

Published in Transactions on Machine Learning Research (02/2026)
Theoretical Analysis of Emergent Language (EL) Communication In emergent communication, agents develop their own protocols through interaction rather than being explicitly programmed. However, understanding why certain languages emerge, how they generalize, and how to ensure their efficiency remains a theoretical challenge. A major open problem quantifying the expressiveness and compositionality of emergent languages. Current emergent protocols often lack systematic structure and interpretability, leading to poor generalization across tasks and partners. One research direction is modeling emergent language information bottleneck problem:
max (m;;)) s) subject to H (mlss S 6,
(5)
mC
where [(m;)) s) measures the mutual information between message m and state while (m]ss) ensures that messages remain concise and interpretable. Establishing conditions under which compositionality emerges naturally will be crucial for developing generalizable emergent languages. Another key theoretical question is the slabilily and convergence of learned languages. Many current methods rely on reinforcement learning to evolve communication, but the non-stationarity of language evolution can lead to instability. Understanding the convergence properties of emergent protocols and the conditions under which languages remain stable is a crucial research area. Lastly, partner adaptation remains an open challenge in emergent communication. Most current models assume a fixed agent population, but real-world multi-agent systems must interact with new, unseen partners. Theoretically, ensuring zero-shot generalization across different agent communities requires defining formal notions of language adaptability:
min DEL (p)mm me
Vi.j E A,
(6)
where DKL represents the Kullback-Leblerr divergence between the language distributions of different agents. Establishing bounds on language alignment loss will be critical for ensuring communication effectiveness across diverse multi-agent populations.
Theoretical Challenges in LLM-Based Multi-Agent Communication Large language models (LLMs) introduce new possibilities for structured communication among agents, enabling human-like inter- action and symbolic reasoning. However, integrating LLMs into multi-agent systems presents new theoretical challenges related to grounding, consistency, and efficiency. One major issue is ensuring that L.M-Mgerratedd messages are grounded in task-specific knowledge rather than being syntactically plausible but semantically irrelevant. This can be formalized as a constrained optimization problem:
exp(s(s, a, m))
p(m|s,a) Em (x,([(s,a,
where p(mls, a) represents the probability of generating message m given state s and action a, and f(s, a, m) ensures message relevance. Future research should investigate ways to enforce consistency and aligroment between LLM-based communication and decision- making policies. Another key challenge is scalability and real-time performance. LLM-based communication is computationally expensive, and real-time multi-agent interactions require low-latency message generation. Developing bounded complezity guarantees for LLM inference in multi-agent settings is an important area of theoretical research. Finally, multi-modal commu- nication is an emerging field where LLMs integrate language with vision, gestures, and symbolic reasoning. Establishing a uniified theoretical framework for multi-modal communication will help in designing systems where agents can elfectively process and exchange diverse types of information.
8.3.2 Future Research on Algorithm Developments
From the discussions in previous sections and the main content in multi-agent communication (MA COMM), we can identify several potential future directions regarding algorithm design. Here, we outline some of these promising directions.
Under-explored categories in multi-agent communication. For existing categories of multi-agent communication algorithms, several areas remain under-explored, such reinforcement. learning- based
115

---

Published in Transactions on Machine Learning Research (02/2026)
communication without explicit message exchange, self-adaptive emergent communication protocols, and transformer-based communication learning that integrates structured memory for improved long-term plan- ning. Additionally, while large-scale multi-agent systems often suffer from communication bottlenecks, more efficient message-passing algorithms that balance communication cost and coordination effectiveness are needed. Specifically, attention- based message filtering and information-theoret, compression techniques could significantly improve the scalability of multi-agent. communication.
Integrated use of various communication paradigms in MARL Each category of communica- tion strategies such as explicit message-passing, emergent language, and LLM--ased communication has unique advantages. However, integrating these paradigms into unified multi-agent communication frame- work remains an open challenge. Future work could explore hybrid architectures that combine structured communication for task-specific scenarios with emergent protocols for adaptability. For instance, multi-agent reinforcement learning (MARL) systems could benefit from hierarchical communication learning, where low- level agents use emergent communication while high-level agents leverage structured messages generated by LLMs. Another promising research direction is improving multi-agent transformer-baset communication, where agents can dynamically decide when to communicate using learned importance scores:
exp(f(hs,hh) 04 ExENro) exp(f(h; hg))
where Oni represents the attention weight assigned by agent i to the message from agent j, and f(h; hy) is a learned relevance function. Developing adaptive atlention mechanisms that optimize communication efficiency while maintaining task performance remains a critical challenge.
Unsolved issues and extensions of multi-agent communication Several fundamental issues in multi- agent communication remain unsolved, requiring new algorithmic developments: (1). Scalability in large- scale MARL: Many existing methods struggle with scalability due to communication overhead. Efficient routing and selective message-passing strategies, such as reinforcement learning- based communication topolo- gies, could be explored. (2). Handling non-stationarity: In non-stationary environments where agents continually update their policies, communication protocols must adapt dynamically. One approach could be meta-learning- based strategies that allow agents to learn and adapt their communication strategies over time. (3). Multi-agent robustness under noisy communication: Real-world applications introduce message loss, delays, or adversarial tampering. Designing robust multi-agent communication frameworks that employ redundancy mechanisms and adversarial training techniques could improve resilience. Specifically, game-theoretic theoretic
MARL algorithms for fully competitive or mixed cooperative-compettiee tasks, grounded in game- principles, require more sophisticated communication strategies that balance cooperation and deception.
Development of generalist. multi-agent communication models One of the most exciting future directions in multi-agent communication is developing a generalist communication model that can adapt to variety of environments and tasks without requiring task-specific fine-tuning. This could be achieved by leveraging advances in foundation models from NLP and computer vision. To build generalist multi- agent communicttin systems, several challenges must be addressed: (1). Scalable architectures for universal communication: Designing models that can process multi-modal inputs, including textual, visual, and structured data, while maintaining efficient message encoding. (2). Pretraining and fine- tuning for generalization: As seen in large-scale foundation models, pretraining communication models diverse datasets followed by fine-tuning task-specific interactions could enable better generalization. (3). Learning from third-person demonstrations: Future research should explore whether multi-agent communication strategies can be learned from video datasets or human-agent interactions, allowing agents to infer communication protocols without direct reinforcement learning.
Future works in related areas Finally, there are several related research areas that could significantly impact the future of multi-agent communication: (1). Inverse reinforcement learning for communica- tion: Understanding how communication strategies emerge in expert demonstrations could improve learned policies. (2). Uncertainty-aware communication in MARL: Extending multi-agent communication to
116

---

Published in Transactions on Machine Learning Research (02/2026)
handle risk-sensitive decision-making could be critical for high-stakes applications such as autonomous driv- ing. (3). Incorporating new foundation models: The emergence of new architectures such as Mamba models and Poisson Flow models provides exciting opportunities to enhance communication strategies for real-time and data-efficient learning.
8.3.3 Future Works on Benchmarking
Development of more realistic, challenging benchmarks We observe that most multi-agent commu- nication (MA COMM) algorithms are evaluated on standard MARL benchmarks such as StarCraft Multi- Agent Challenge (SMAC), Multi-Agent Particle Environment (MPE), and Google Research Football (GRF). However, these benchmarks may not fully capture the complexity and challenges of real-world multi-agent communication. To advance the field and properly evaluate new methods, more comprchensive and chal- lenging benchmarks should be developed: (1). Large-scale datasets for communication evaluation: The scalability of MA COMM algorithms remains a key challenge. Benchmarks should include large-scale datasets that test how communication methods perform with increasing agent populations, diverse com- munication constraints, and extensive training data. Understanding the relationship between datasct size and communication elficiency can help assess whether learned communication strategies generalize well. (2). Multi-modal communication benchmarks: Real-world multi-agent communication often involves het- crogencous data sources, including language, vision, and symbolic information. Future benchmarks should include multi-modal environments where agents communicate using mix of textual, visual, and structured messages. For example, agents might receive partial visual observations and complement them with natural language messages to coordinate actions. (3). More realistic task scenarios: A major reason for the widespread success of deep learning in computer vision and NLP is that models are trained on large, real-world datasets. In rastrast, multi-agent communication is still largely evaluated in simulated environments that lack real-world complexity. Future benchmarks should include real-world datasets of human or agent inter- actions, such as autonomous vehicle coordination data, multi-robot task exccution logs, and human dialogue datasets for cooperative problem-solving, (4). Beyond return-based evaluation: Current benchmarks mainly evaluate performance based on cumulative rewards or task completion rates. However, effective com- munication should also be assessod in terms of interpretability, efficiency, generalization, and safety. Future benchmarks should include metrics that could evaluate problems like: Message efficiency, How much communication overhead is required for achieving optimal performance? And Generalization, such as answering questions like Can learned communication protocols transfer to unseen environments or new agents? And Robustness: How resilient is the communication strategy to message loss, dclays, or ad- versarial interference? Or for Emergent communication structure, an efficient benchmark should help researchers to answer Do agents develop structured and compositional languages, and how interpretable are they?
Comparisons among different multi-agent communication paradigms While there exist numerous algorithms for multi-agent communication ranging from explicit message-passing passing approaches to emergent language learning and ALM-based multi-agent cordination-ther is lack of standardized comparisons among them. To better understand their relative advantages and limitations, future research should estab- lish benchmarks that allow fair evaluations across different paradigms: (1). Standardized comparisons across communication strategies: Future benchmarks should compare traditional MARL communication approaches (e.g., differentiable communication via continuous signals) with emergent language- based com- munication and I.M.-augmented communication frameworks. By evaluating these approaches on the same set of tasks, researchers can better understand when structured communication is preferable over learned protocols. (2). Scalability of different communication methods: As the number of agents increases, how do different communication approaches scale? Some methods may perform well in small agent popula- tions but struggle with communication bottlenecks in larger teams. Benchmarks should include scalability evaluations where different communication architectures are tested under increasing numbers of agents and message complexity constraints. (3). Computational and learning efficiency: While some multi-agent communication methods achieve high performance, they may require significant computational resources for training. Future benchmarks should track training time, inference speed, and memory usage across differ- ent methods, providing insights into their practical fcasibility. (4). Adaptability to new agents and tasks: Benchmarks should evaluate how well communication strategies generalize to new agents or novel
117

---

Published in Transactions on Machine Learning Research (02/2026)
tasks. For example, if an agent trained in one environment is deployed in another with different teammates, how quickly can it adapt its communication protocol? Evaluating zero-shot and few-shot generalization capabilities would be crucial for developing flexible multi-agent systems.
Developing a unified benchmark suite for MA COMM To accelerate progress in multi-agent com- munication rescarch, a unified benchmark suite should be developed that provides: (1). A diverse set of environments: Benchmarks should cover cooperative, competitive, and mixed-motive multi-agent tasks, ensuring that communication strategies are evaluated across broad range of scenarios. (2). Extensive datasets for pretraining and evaluation: Similar to ImageNet in computer vision and GLUE in NLP, multi-agent communication research would benefit from large-scale datasets that facilitate pretraining and systematic evaluation. (3). A set of well-defined metrics: Future benchmarks should move beyond sim- ple reward-based evaluation and incorporate new performance metries that reflect communication efficiency, interpretability, robustness, and generalization. (4). Baseline implementations for fair comparisons: Providing open-source implementations of baseline communication methods would ensure reproducibility and encourage fair comparisons across different approaches.
8.3.4 Future Research on Human-Centric Multi-Agent Communication
How can multi-agent communication systems align with human expectations and interpretabil- ity? The effectiveness of multi-agent communication (MA COMM) is not solely determined by task per- formance but also by its interpretability, alignment with human communication norms, and ability to foster trust in human-agent collaboration. One fundamental research direction is to explore how multi-agent com- munication protocols can be designed to be both machine-efficien and numan-comprehensibl.. (1) Current emergent communication models often develop symbols or representations that are opaque to humans. En- suring that learned communication protocols exhibit structured, compositional, and generalizable properties similar to natural language is key challenge. A possible approach is to impose information-theore straints that favor structured communication while maintaining efficiency. Another direction is to explore how learned protocols can be mapped to existing human language structures without compromising perfor- mance. (2) Multi-agent communication should be interpretable not only in terms of the message content but also in how communication influences agent behavior. Formal methods for analyzing the causal effect of communication on decision-making remain underdeveloped. Fstablishing metries to evaluate whether agents correctly utilize received messages in a way that aligns with human intuition can be beneficial for interpretability and debugging.
con-
(3) The degree to which communication can be adapted for diverse user preferences is also an open question. Hurans exhibit variation in linguistic styles, levels of detail in com- munication, and implicit assumptions about shared knowledge. Future research should investigate adaptive communication strategies where agents dynamically adjust their communication style to align with individual user expectations. Such adaptability could be achieved through meta-learning techniques or reinforcement learning policies trained on diverse human interaction datasets.
How should multi-agent communication systems facilitate collaboration with humans? In many real-world applications, agents must. not only communicate with each other but also interact with humans as teammates, supervisors, or evaluators. Designing communication protocols that effectively integrate human feedback and decision-making is critical for advancing human-Al collaboration. (1) One challenge is deter- mining the optimal balance between autonomous agent communicatio and titman-in-the-lowe intervention. While some tasks require full autonomy, others benefit from human oversight, and communication models should be able to modulate information exchange accordingly. This raises the question of how to design communication-avarr policies that incorporate human corrections efficiently without excessive reliance on human intervention.
(2) Another open problem is learning communication strategies that generalize across different types of human users. Agents may need to adjust their communication level based on the expertise and role of the human collaborator. For instance, a novice user may require detailed explanations, while an expert user may prefer concise updates. A promising direction is to develop hierarchical communication frameworks where agents can reason about when and how to involve human input. (3) In settings where mul- tiple agents communicate with a human, ensuring that communication remains coherent and non-redundant is crucial. Unlike homogeneous multi-agent settings, human interactions require mechanisms for resolving conflicting information, synthesizing diverse perspectives, and presenting relevant insights in structured
118

---

Published in Transactions on Machine Learning Research (02/2026)
manner. Future work should investigate how multi-agent communication can be optimized for presenting actionable insights in an efficient and user-friendly manner.
How can human trust and safety be ensured in multi-agent communication? Trust in multi- agent communication systems is essential for adoption in high-stakes domains such as healthcare, autonomous driving, and critical infrastructure management. However, ensuring robustness and reliability in communi- cation remains an open challenge. (1) One primary concern is that communication protocols learned purely through reinforcement learning may not always prioritize safety or reliability. Messages exchanged between agents should not only be optimized for task efficiency but also be constrained to avoid potentially harmful or mislcading communication. Future research should explore formal safety constraints that ensure multi- agent messages adhere to ethical and regulatory guidelines. (2) Communication robustness under adversarial settings is another key research direction. In enviromments where agents operate in competitive or adver- sarial conditions, ensuring that communication remains truthful and resilient to manipulation is essential. Theoretical work on adversarial robustness in communication, such as establishing upper bounds on the probability of message tampering introducing cryptographic techniques for secure multi-agent commu- nication, is an important avenue of rescarch. (3) Beyond security concerns, communication reliability in dynamic and non-slationary environments is also a challenge. Agents must be able to detect and recover from missing or corrupted messages while ensuring that communication failures do not. lead to catastrophic decision errors. Future work should investigate celf-correcting communication mechanisms that allow agents to infer missing information reconstruct degraded messages in real time.
How can human feedback be effectively incorporated into multi-agent communication learning? Human-in-the-logg learning is promising approach for improving multi-agent communication by leveraging human expertise to refine communication strategies. However, incorporating human feedback efficiently and scalably remains an open challenge. (1) One direction is investigating how agents can learn effective communication through a combination of explicit and implicit feedback. While direct human corrections can provide useful supervision, implicit signals such as human engagement, response time, or sentiment analysis from text-based interactions could serve as valuable learning signals for communication adaptation. (2) Future work should explore techniques for efficiently integrating sparse human feedback into multi-agent learning frameworks. Since obtaining human annotations for large-scale multi-agent interactions is costly, techniques such as preference-based reinforcement learning or weak supervision could be useful for extracting meaningful guidance with minimal human effort. (3) Evaluating the effectiveness of human feedback on multi- agent communication learning remains an underexplored research problem. While traditional reinforcement learning relies reward functions, the impact of human feedback is more complex and may require new evaluation frameworks that capture the long-term benefits of hunan-informed communication strategies. Establishing benchmarks that explicitly test. human-guided communication adaptation would be a valuable contribution to this area.
How can multi-agent. communication be designed to enhance social interaction and fairness? As multi-agent systems are increasingly deployed in human-facing applications, it is important to consider fairness and social dynamics in communication. (1) One key question is how to ensure equitable information access in multi-agent interactions. In scenarios where multiple agents communicate with each other and with human users, disparities in information access can emerge, leading to biased decision-making. Research should focus on designing fairness fairness-aware aware communication policies that ensure all participants receive relevant information based on their role and decision maaiinnnmmaing nceds. (2) Another challenge is avoiding emergent behaviors that disadvantage certain agents or human users. For example, in competitive multi-agent envi- ronments, agents may develop deceptive communication strategies that provide short-term advantages but undermine trust and cooperation in the long run. Studying game-theoretic mechanisms for enforcing coop- erative communication norms could help mitigate such behaviors. (3) Finally, multi-agent communccation should support natural and socially appropriate interactions. This includes adapting communication tone, level of detail, and responsiveness based on the social context. Future research should explore computational models of social intelligence that enable agents adjust their communication style in human- manner, improving engagement and user experience.
119

---

Published in Transactions on Machine Learning Research (02/2026)
9 Conclusion
This survey presented a comprehensive overview of multi-agent communication through the unifying lens of the Five Ws of communication: who communicates with whom, what is communicated, when com- munication occurs, why it is beneficial, and how motivated and operationalized. Using this framework, we organized progress across three major paradigms: communication in MARL, EL, and LLsed multi-agent systems. We traced the evolution of MARL communication from early message- message-passing mech- anisms to learned, end-to-end protocols shaped by reward structure and control objectives, reviewed emergent language research that studies how communication arises through interaction with improved structure and interpretability, and surveyed LLM-powered multi-agent systems where natural language enables reasoning, planning, and collaboration in open-ended environments, including hybrid LLM MARL architectures that balance expressiveness, grounding, and control.
Beyond cataloging methods, this survey emphasized conceptual integration. By organizing diverse ap- proaches around shared communication questions, we clarified how communication design reflects under- lying assumptions about coordination, uncertainly, incentives, and agency. We further connected modern learning- bearning-based communication to foundational perspectives from philosophy, cognitive science, game the- ory, and classical AI, framing communication support collective decision-making. The added bridge sections explicitly illustrate how newer paradigmns emerge in response to the limitations of earlier ones, rather than replacing them outright.
action chosen to inffuence beliefs, align intentions, and
Looking ahead, multi-agent communication research faces persistent challenges in grounding, interpretability, generalization, scalability, and theoretical guarantees, especially as systems move toward open-ended, human- facing, and safety-critical deployments. Progress will require tighter integration of learning, language, and control, informed by both system-level design principles and formal reasoning about belief, knowledge, and interaction. As the first survey to unify MARL-based communication, EL, and LLM-driven multi-agent systems within a single, coherent. framework, we hope this work serves as a durable reference and a foundation for developing scalable, robust, and human-aligned communication in future multi- multi-agent systems.
References
Crewai.
Langchain. bttpe://ggthhaaaccoo///gggaaaaaa
Langgraph. http:///w..laacccaacccm//aaaaaga b.
Superagi. http:://ggtabbcco////rrraccertep
Sahar Abdelnabi, Amr Gomaa, Sarath Sivaprasad, Lea Schönherr, and Mario Fritz. Llm-deliberation: Eval- uating Ilms with interactive multi-agent negotiation games. 2023.
Sahar Abdelnabi, Amr Gomaa, Sarath Sivaprasad, Lea Schönherr, and Mario Fritz. Cooperation, com- petition, and maliciousness: Lim-stakeholderss interactive negotiation. Advances in Newral Information Processing Systems, 37:83548 83599, 2024.
Jeffrey L Adler and Victor Blue. A cooperative multi-agent transportation management and route guidance system. Transportation Research Part C: Emerging Technologies, 10(5-6) 454, 2002.
John P Agapiou, Alexander Sasha Vezhnevets, Edgar A Fuzzán, Jayd Matyas, Yiran Mao, Peter Sunchag, Raphael Köster, Udari Madhushani, Kavya Kopparapu, Ramona. Comanescu, et al. Melting pot 2.0. arXiv preprinl ar arXiv:2211.13//4 2022.
Akshat Agarwal, Swaminathan Gurumurthy, Vasu Sharma, Mike Lewis, and Katia Sycara. Community regu- dialog. In Proceedings of the 18th International Conference on Autonomous
larization of Agents and MultiAgent Systems, pp. 1042-1050, 2019a.
Akshat Agarwal, Sumit Kumar, and Katia Sycara. Learning transferable cooperative behavior in multi-agent teams. Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems, 2019b.
120

---

Published in Transactions on Machine Learning Research (02/2026)
Saaket Agashe, Yue Fan, and Xin Eric Wang. Evaluating multi-agent coordination abilities in large language models. arXiv preprint (XXx::2110003903 2023.
Saaket, Agashe, Yue Fan, Anthony Reyna, and Xin Eric Wang. Llm-coordination: evaluating and analyzing multi-agent coordination abilities in large language models. Findings of the Association for Computational Linguistics: NAACL 2025, 2025.
Junhyook Ahn and Luis Sentis. Nested mixture of experts: Cooperative and competitive learning of hybrid dynamical system. In Learning for Dynomics and Control, pp. 779 790. PMLR, 2021.
Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, al. Do as can, not i say; Grounding language in robotic affordances. arXiv preprint arXiv:2204.0161 01691, 2022.
Altera AL, Andrew Ahn, Nic Becker, Stephanic Carroll, Nico Christic, Manuel Cortes, Arda Demirci, Mclissa Du, Frankie Li, Shuying Luo, et al. Project sid: Many-agent simulations toward ai civilization. ar preprint axXi::2411.00144 2024.
et al. Andreas, Jacob. Measuring compositionality representation learning. Intervational Conference on Learning Representations. Vol. 375. Association for Computational Linguistics, 2019.
RM Aratchige and WMKS Ilmini. Lims working in harmony: survey on the technological aspects of building effective Ilm-based multi agent systems. arXiv preprint orxv:25004.011 019003 2025.
Robert L Arrington. Ludwig wittgenstein: Philosophical investigations. In Central Works of Philosophy v/, pp. 257-279. Routledge, 2016.
Michal Aucrsperger and Pavel Pecina. Defending compositionality in emergent languages. arXiv preprint arXiv:2206. 04751, 2022.
John Langshaw Austin. How to do things with words. Harvard university press, 1975.
Suma Bailis, Jane Friedhoff, and Feiyang Chen. Werewolf arena: case study in 1lm evaluation via social deduction. arXiv preprint 0rXiv:2407.13943 2024.
Bowen Baker, Ingmar Kanitscheider, Todor Markov, Yi Wu, Glenn Powell, Bob McGrew, and Igor Mordatch. Emergent tool use from multi-agent autocurricula. International Conference on Learning Representa- tions.
Lubomir Bakule and Martin Papik. Decentralized control and communication. Annual Reviews in Control, 36(1):1-10, 2012.
Razan Baltaji, Babak Hemmatian, and Lav R Varshney. Conformity, confabulation, and impersonation: Persona inconstancy in multi-agent 11m collaboration. arXiv preprint arXiv22455003862 2024.
Cristian-Paul Bara, Sky CH-Wang, and Joyce Chai. Mindcraft: Theory of mind modeling for siluated
dialogue in collaborative tasks. arXiv preprint arXiv:2109.06225, 2021.
Marco Baroni. Linguistic generalization and compositionality in modern artificial neural networks. Philo- sophical Transactions of the Royal Society B, 375(17911:20190307 2020.
John Batali. Computational simulations of the emergence of grammar. Approaches to the Evolution of Tanguage-Social and Cognâtive Bases-, 1998.
Ali Beikmohammadi. Learning to communccate through multi-agent reinforcement learning (marl): A systematic literature review. Preprints, April 2024. doi: 00.00944pprppht::202000......33. URL
Emily Bender and Alexander Koller. Climbing towards nlu: On meaning, form, and understanding in the age of data. In Proceedings of the 58th annual meeting of the association for computational linguistics, pp. 5198, 2020.
121

---

Published in Transactions on Machine Learning Research (02/2026)
Christopher Berner, Greg Brockman, Brooke Chan, Vicki Cheung, Przemyslaw Dębiak, Christy Dennison, David Farhi, Quirin Fischer, Shariq Hashme, Chris Hesse, et al. Dota 2 with large scale deep reinforcement learning. Xiv preprint arxiv: 291.. 2.06600 2019.
Daniel S Bernstein, Robert Givan, Neil Immerman, and Shlomo Zilberstein. The complexity of decentralized control of markov decision processes. Mathematics of operations research, 27(4):819 840, 2002.
Jan Blumenkamp and Amanda Prorok. The emergence of adversarial communication in multi-agent rein- forcement learning. In Conference on Robot Learning, pp. 1394-1414. PMLR, 2021.
Ben Bogin, Mor Geva, and Jonathan Berant. Emergence of communication in interactive world with consistent speakers. arXiv preprint orXic:1809.005// 2018.
Brendon Boldt and David Mortensen. A review of the applications of deep learning-basee based emergent com- munication. arXiv preprint ar (r77:2407.08330 2024.
Diane Bouchacourt and Marco Baroni. How agents see things: On visual representations an emergent lan- guage game. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 981 985, 2018.
Diane Bouchacourt and Marco Baroni. Miss tools and mr fruit: Emergent communication in agents learning about object affordances, 2019. URL
11871
Nicolo' Brandizzi. Toward more human-like communication: review of emergent communication re- search. IEEE Access, 11:142317-14221 2023.
Nicolo' Brandizzi, Davide Grossi, and Luca locchi. Rlupus: Cooperation through emergent communication in the werewolf social deduction game. Intelligenza Artificiale, 15(2):55-70, 2022.
Cynthia Breazeal, Cory D Kidd, Andrea Lockerd Thomaz, Guy Hoffman, and Matt Berlin. Effects of nonverbal communication on efficiency and robustness in human-robot teanwork. In 2005 IEFE/RS! international conference on intelligent robots and systems, pp. 708 713. IEEE, 2005.
Henry Brighton and Simon Kirby. Understanding linguistic evolution by visualizing the emergence of topo- graphic mappings. Artificial life, 12(2):229 242, 2006.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877 1901, 2020.
Christian Buck, Jannis Bulian, Massimiliano Ciaramita, Wojciech Gajewski, Andrea Gesmundo, Neil Houlsby, and Wei Wang. Analyzing language learned by an active question answering agent. arXiv preprinl 1
2018.
Kalesha Bullard, Franziska Meier, Douwe Kiela, Joelle Pineau, and Jakob Foerster. Exploring zero-shot emergent communication in embodied multi-agent populations. arXiv preprint arXiv:2010. 15896, 2020.
Kalesha Bullard, Douwe Kiela, Franziska Meier, Joelle Pineau, and Jakob Foerster. Quasi-oquivalence discovery for zero-shot emergent communication. arXiv preprint arXi::2103.0800 2021.
Lucian Busoniu, Robert Babuska, and Bart De Schutter. A comprehensive survey of multiagent reinforcement learning. IEEE Transactions on Systems, Man, and Cybernetics, Part (Applications and Reviews), 38 (2):156 172, 2008.
Salvatore Candido and Seth Hutchinson. Minimum uncertainty robot navigation using information-guie guided pomdp planning. In 2011 IEEE International Conference on Robotics and Automation, pp. 6102-6108. IEEE, 2011.
Lorenzo Canese, Gian Carlo Cardarilli, Luca Di Nunzio, Rocco Fazzolari, Daniele Giardino, Marco Re, and Sergio Spanò. Multi-agent reinforcement learning: A review of challenges and applications. Applied Sciences, [1(1)):4948, 2021.
122

---

Published in Transactions on Machine Learning Research (02/2026)
Angelo Cangelosi and Domenico Parisi. Simulating the evolution of language. Springer Science & Business Media, 2012.
Kris Cao, Angeliki Lazaridou, Mare Lanctot, Joel Z Leibo, Karl Tuyls, and Stephen Clark. Emergent communication through negotiation. arXie preprinl or arxiv:1804 NNi::8844009960 2018.
Yongean Cao, Wenwu Yu, Wei Ren, and Guanrong Chen. An overview of recent progress in the study of distributed multi-agent coordination. IEEE Transactions Industrial Informalics, 9(1):427-436; 2013. doi: 10.1109/TTI 211..11..22210000000
Boaz Carmeli, Ron Meir, and Yonatan Belinkov. Emergent quantized communication. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 11533 11541, 2023.
Micah Carroll, Rohin Shah, Mark K Ho, Tom Griffiths, Sanjit. Seshia, Pieter Abbeel, and Anca Dragan. On the utility of learning about humans for human-ai coordination. Advances in neural information processing systems, 32, 2019.
Thomas Carta, Clément Romac, Thomas Wolf, Sylvain Lamprier, Olivier Sigaud, and Pierre- Yves Oudeyer. Grounding large language models in interactive environments with online reinforccment learning. International Conference on Machine Learning, pp. 3676 3713. PMLR, 2023.
Cristiano Castelfranchi. The theory of social functions: challenges for computational social science and multi-agent. learning. Cognilive Systems Research, 2(1):5 38, 2001.
Mert Cemri, Mclissa Z Pan, Shuyi Yang, Lakshya A Agrawal, Bhavya Chopra, Rishabh Tiwari, Kurt Keutzer, Aditya Parameswaran, Dan Klein, Kannan Ramchandran, et al. Why do multi-agent 1lm systems fail? arXiv preprint arXiv:2503.1300 2025a.
Mert Cemri, Melissa Z Pan, Shuyi Yang, Lakshya A Agrawal, Bhavya Chopra, Rishabh Tiwari, Kurt Keutzer, Aditya Parameswaran, Dan Klein, Kannan Ramchandran, et al. Why do multi-agent Ilm systems fail? arXiv preprint ar Xiv:25500 13657, 2025b.
Rahma Chaabouni, Eugene Kharitonov, Emmanuel Dupoux, and Marco Baroni. Anti-efficient encoding in emergent communication. Advances in Neural Information Processing Systems, 32, 2019.
Rahma Chaabouni, Eugene Kharitonov, Diane Bouchacourt, Emmanuel Dupoux, and Marco Baroni. Com-
positionality and gencralization in emergent languages. In ACL 2020-8th annual meeting of the Association for Computational Lingwistics, 2020.
Rahma Chaabouni, Florian Strub, Florent Altché, Eugene Tarassov, Corentin Tallec, Elnaz Davoodi, Kory Wallace Mathewson, Olivier Tieleman, Angeliki Lazaridou, and Bilal Piot. Emergent communi- cation at scale. In International conference on learning representations, 2022.
Marwa Chafii, Salmane Naoumi, Reda Alami, Ebtesam Almazrouei, Mehdi Bennis, and Merouane Debbah. Emergent communication in multi-agent reinforcement learning for future wireless networks. IEEE Internet of Things Magazine, 6(4):18-2) 24, 2023.
Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu. Chateval: Towards better 1lm-based evaluators through multi-agent debate. arXiv preprint arXiv:2308. 07201, 2023.
Matthew Chang, Gunjan Chhablani, Alexander Clegg, Mikael Dallaire Cote, Ruta Desai, Michal Hlavac, Vladimir Karashchuk, Jacob Krantz, Roozbeh Mottaghi, Priyam Parashar, et al. Partur: A benchmark for planning and reasoning in embodied multi-agent tasks. arXiv preprint arXiv:241100000 2024.
Guangyao Chen, Siwei Dong, Yu Shu, Ge Zhang, Jaward Sesay, Börje F Karlsson, Jie Fu, and Yemin Shi. Autoagents: framework for autonatic agent generation. arXiv preprint orXiv:2309, 17288, 2023a.
Jiangjie Chen, Xintao Wang, Rui Xu, Siyu Yuan, Yikai Zhang, Wei Shi, Jian Xie, Shuang Li, Ruihan Yang, Tinghui Zhu, et al. From persona personalization: survey on role-playing language agents. arXiv preprint ar Xixi:2244. 18231, 2024a.
123

---

Published in Transactions on Machine Learning Research (02/2026)
Jingdi Chen, Tian Lan, and Carlee Joc-Wong Rgmcomm: Return gap minimization via discrete com- munications in multi-agent reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 17327-17336, 2024b.
Junzhe Chen, Xuming Hu, Shuodi Liu, Shiyu Huang, Wei-Wei Tu, Zhaofeng He, and Lijie Wen. Llmarena: Assessing capabilities of large language models in dynamic multi-agent environments. ar preprint 1xXi::242.114099 2024c.
Justin Cho Chen, Swarnadeep Saha, and Mohit Bansal. Reconcile: Round-table conference improves reasoning via consensus among diverse 1mms. arXiv preprint (rXii:2309.13000 2023b.
Pei Chen, Boran Han, and Shuai Zhang, Comm: Collaborative multi-agent, mulli-reasonnnng-pul prompting for complex problem solving. arXiv preprint orXiv:2444.17729, 2024d.
Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chen Qian, Chi-Min Chan, Yujia Qin, Yaxi Lu, Ruobing Xie, et al. Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors in agents. arXiv preprint or Xiv:23308. 10848, 2(4):5, 2023c.
Weize Chen, Ziming You, Ran Li, Yitong Guan, Chen Qian, Chenyang Zhao, Cheng Yang, Ruobing Xie, Zhiyuan Liu, and Maosong Sun. Internet of agents: Weaving a web of heterogeneous agents for collabo- rative intelligence. arXiv preprint arXiv:2407.00 07061, 2024e.
Weize Chen, Jiarui Yuan, Chen Qian, Cheng Yang, Zhiyuan Liu, and Maosong Sun. Optima: Optimizing effectiveness and efficiency for Ilm-based multi-agent system. arXiv preprint 14.::2410.0811 2024f.
Yongchao Chen, Jacob Arkin, Yang Zhang, Nicholas Roy, and Chuchu Fan. Scalable multi-robot collaboration with large language models: Centralized or decentralized systems? In 2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 4311 4317. IEEE, 2024g.
Ricson Cheng, Arpit Agarwal, and Katerina Fragkiadaki. Reinforcement learning of active vision for manip- ulating objects under occlusions. In Conference on Robot Learning, pp. 422-431. PMLR, 2018
Yuheng Cheng, Ceyao Zhang, Zhengwen Zhang, Xiangrui Meng, Sirui Hong, Wenhao Li, Zihao Wang, Zekai Wang, Feng Yin, Junhua Zhao, et al. Exploring large language model based intelligent agents: Definitions, methods, and prospects. arXiv preprint arXiv:2401.09 03428, 2024.
Edward Choi, Angeliki Lazaridou, and Nando de Freitas. Compositional obverter communication learning from raw visual input, 2018. URL btttps///axiit wttp:/www..rg/bbb///...000 02341
Jongeun Choi, Songhwai Oh, and Roberto Horowitz. Distributed learning and cooperative control for multi- agent systems. Automatica, 45(12):2802-2814 2009.
Noam Chomsky. Three models for the description of language. IRE Transactions on information theory, 2 (3):113 124, 1956.
Morten H Christiansen and Simon Kirby. Language evolution. OUP Oxford, 2003.
Tianshu Chu, Sandeep Chinchali, and Sachin Katti. Multi-agent reinforcement learning for networked system control. arXiv preprint orXiv:2004.001 011339, 2020.
Cédric Colas, Tristan Karch, Nicolas Lair, Jean-Michel Dussoux, Clément Moulin-Frier, Peter Dominey, and Pierre- Oudeyer. Language as cognitive tool to imagine goals in curiosity driven exploration. Advances in Newral Information Processing Systems, 33:3761-3774, 2020.
Alain Colmerauer. Metamorphosis grammars. In Natural language communication with computers, pp. 133 188. Springer, 2005.
Dylan Cope and Nandi Schoots. Learning to communicate with strangers via channel randomisation methods. In The 4th Workshop on Emergent Communication at the International Conference on Neural Information Processing (NeurIPS), 2020.
124

---

Published in Transactions on Machine Learning Research (02/2026)
Dylan Cope and Nandi Schoots. Learning to communicate with strangers via channel randomisation methods. arXiv preprint arXio:2104 09537, 2021.
Jorge Cortes, Sonia Martinez, Timur Karatas, and Francesco Bullo. Coverage control for mobile sensing networks. IEEE Transactions on robolics and Automation, 20(2):243-25) 2203, 2004.
Gautier Dagan, Dieuwke Hupkes, and Elia Bruni. Co-evolution of language and agents in referential games. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp. 2993-3004, 2021.
Mike D D'Arcy Tom Hope, Larry Birnbaum, and Doug Downey. Marg: Multi-agent review generation for scientific papers. arViv preprint ar XrXiv:2401.04885 2024.
Abhishek Das, Satwik Kottur, José MF Moura, Stefan Lee, and Dhruv Batra. Learning cooperative visual dialog agents with deep reinforcement learning. In Proceedings of the IEEE international conference computer vision, pp. 2951 2960, 2017.
Abhishek Das, Théophile Gervet, Joshua Romoff, Dhruv Batra, Devi Parikh, Michael Rabbat, and Joelle Pineau. Tarmac: Targeted multi-agent communication, 2018. URL http://wwwv..0rr/aa///111111..1.
Ayushman Das, Shu-Ching Chen, Mei-Ling Shyu, and Saad Sadiq. Enabling synergistic knowledge sharing and reasoning in large language models with collaborative multi-agents. In 2023 IEEE 9th International Conference on Collaboration and Internet Computing (CTC). pp. 92-98. IEFF, 2023.
Ankur Deka and Katia, Sycara. Natural emergence of heterogeneous strategies in artificially intelligent competitive teams. In International Conference on Swarm Intelligence, pp. 13-25. Springer, 2021.
Kevin Denamganai and James Alfred Walker. On (emergent) systematic generalisation and composi- tionality in visual referential games with straight-through gumbel-softmax estimator. arXiv preprint arXiv:2012 2020a.
Kevin Denamganai and James Alfred Walker. Referentialgym: A nomenclature and framework for language emergence & grounding in (visual) referential games. orXiv preprint ar 2xX::2012.09466, 2020b.
Kevin Denamganai, Sondess Missaoui, and James Alfred Walker. Visual referential games further the emer- gence of disentangled representations. arXiv preprint 2rX1v:2304.44511 2023.
Roberto Dessi, Diane Bouchacourt, Davide Crepaldi, and Marco Baroni. Focus on what's informative and ignore what's not: Communication strategies in referential game. arXio preprint 191101991..01892, 2019.
Ziluo Ding, Tiejun Huang, and Zongqing Lu. Learning individually inferred communicatin for multi-agent cooperation. Advances in Newral Information Processing Systems, 33:22069-22001 2020.
Yubo Dong, Xukun Zhu, Zhengzhe Pan, Linchao Zhu, and Yi Yang. Villageragent: A graph-based multi-agent framework for coordinating complex task dependencies in minecraft. arXiv preprint (ti::2206005792 2024.
Silin Du and Xiaowei Zhang. Helmsman of the masses? evaluate the opinion leadership of large language models in the werewolf game, orXiv preprint arX1v:2404.0160, 2024.
Yali Du, Bo Liu, Vincent Moens, Ziqi Liu, Zhicheng Ren, Jun Wang, Xu Chen, and Haifeng Zhang. Learning correlated communication topology in multi-agent reinforcement learning. In Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems, pp. 456-464, 2021.
Yilun Du, Shuang Li, Antonio Torralba, Joshua B Tenenbaum, and Igor Mordatch. Improving factuality and reasoning in language models through multiagent debate. arXiv preprint arXio:2305.1433 14325, 2023.
Zhuoyun Du, Chen Qian, Wei Liu, Zihao Xie, Yifei Wang, Yufan Dang, Weize Chen, and Cheng Yang. Multi-agent software development through cross-team collaboration. arXiv preprint arXiv:240600897 2024.
125

---

Published in Transactions on Machine Learning Research (02/2026)
Marina Dubova, Arsenii Kirillovich Moskvichev, and Rob Goldstone. Reinforcement communication learning in different social network structures. In Language in Reinforcement Learning Workshop at 1CML 2020.
Zane Durante, Qiuyuan Huang, Naoki Wake, Ran Gong, Jae Sung Park, Bidipta Sarkar, Rohan Taori, Yusuke Noda, Demetri Terzopoulos, Yejin Choi, al. Agent ai: Surveying the horizons of multimodal interaction. arXiv preprint arXiv:2401.0333 033668 2024.
Tom Eccles, Yoram Bachrach, Guy Lever, Angeliki Lazaridou, and Thore Graepel. Biases for emergent communication in multi-agent reinforcement learning. Advances in neural information processing systems, 32, 2019.
Andrew Estornell, Jean-François Ton, Yuanshun Yao, and Yang Liu. Acc-collab: An actor-critic approach to multi-agent Ilm collaboration. In The Thirteenth International Conference on Learning Representations, 2025.
Katrina Evtimova, Andrew Drozdov, Douwe Kiela, and Kyunghyun Cho. Emergent communication in a multi-modal, multi-step referential game. arXiv preprint XXx::1055.10000 2017.
Meta Fundamental Al Research Diplomacy Team (FAIR)), Anton Bakhtin, Noam Brown, Emily Dinan, Gabricle Farina, Colin Flaherty, Daniel Fried, Andrew Goff, Jonathan Gray, Hengyuan Hu, et al. Human- level play the game of diplomacy by combining language models with strategic reasoning. Science, 378 (6624):1067 1074, 2022.
Xueyang Feng, Yuan Chen, Yujia Qin, Yankai Lin, Xu Chen, Zhiyuan Liu, and Ji-Rong Wen. Large lan- guage model- mosel-based human-agent collaboration for complex task solving. arXiv preprint orXiv:2/02.190 12914. 2024.
Chrisantha Fernando, Daria Zenkova, Stanislav Nikolov, and Simon Osindero. From language games to drawing games. arXiv preprint arXiv:2010.02520 2020.
Nicole Fitzgerald and Jacopo Tagliabue. On the plurality of graphs. Journal of Logic and Computation, 32 (0):1129-114) 2022.
Jakob Foerster, Ioannis Alexandros Assael, Nando De Freitas, and Shimon Whiteson. Learning commu- nicate with deep multi-agent reinforcement learning. Advances in neural information processing systems, 29, 2016a.
Jakob Foerster, Francis Song, Edward Hughes, Neil Burch, lain Dunning, Shimon Whiteson, Matthew Botvinick, and Michael Bowling. Bayesian action decoder for deep multi-agent reinforcement learning. In International Conference on Machine Learning, pp. 1942 1951. PMLR, 2019.
Jakob N Foerster, Yannis M Assacl, Nando de Freitas, and Shimon Whiteson. Learning to communicate solve riddles with deep distributed recurrent g-rkts arXiv preprinl Xrx:2:162.. 02672, 2016b.
Adam Fourney, Gagan Bansal, Hussein Mozannar, Cheng Tan, Eduardo Salinas, Friederike Niedtner, Grace Proebsting, Griffin Bassman, Jack Gerrits, Jacob Alber, al. Magentic-one: generalist multi-agent system for solving complex tasks. arXiv preprint ar (tl::24110444000 2024.
Benjamin Freed, Rohan James, Guillaume Sartoretti, and Howie Choset. Sparse discrete communication learning for multi-agent cooperation through backpropagation. In 2020 IEEE/RSJ International Confer- ence on Intelligent Robots and Systems (IROS), pp. 7993-7998. IEEE, 2020a.
Benjamin Freed, Guillaume Sartorctti, and Howic Choset. Simultancous policy and discrete communication learning for multi-agent cooperation. IEEE Robolics and Aulomation Lellers, 5(2):2498 2505, 2020b.
Lukas Galke and Limor Raviv. Emergent communication and learning pressures in language models: language evolution perspective. arXiv preprint arXim=2403.14427, 2024.
Lukas Galke, Yoav Ram, and Limor Raviv. Emergent communication for understanding human language evolution: What's missing? Tn Emergent Communication Workshop at ICLR 2022.
126

---

Published in Transactions on Machine Learning Research (02/2026)
Alireza Ghafarollahi and Markus J Buchler. Sciagents: Automating scientific discovery through multi-agent intelligent graph reasoning. arXiv preprint arXiv:2409.0553 2024.
Ran Gong, Qiuyuan Huang, Xiaojian Ma, Hoi Vo, Zane Durante, Yusuke Noda, Zilong Zheng, Song Chag-Chun Zhu, Demetri Terzopoulos, Li Fei-Fei, et al. Mindagent: Emergent gaming interaction. arXiv preprint S1l::2309.09977 2023.
Laura Graesser, Kyunghyun Cho, and Douwe Kicla. Emergent linguistic phenomena in multi-agent commu- nication games. arXiv preprint arxiv:1901 1Xx:1000000706, 2019.
H Paul Grice. Meaning, philosophical review, 66 (3), 377-388, DOI, 10:2182440, 1957.
Sven Gronauer and Klaus Diepold. Multi-agent deep reinforcement learning: a survey. Artificial Intelligence Review, 55(2):895-944 2022.
Niko A Grupen, Daniel D Lee, and Bart Selman. Multi-agent curricula and emergent implicit signaling. arXiv preprint XrXi::206.. 1 1116, 2021.
Niko A. Grupen, Daniel D. Lee, and Bart Selman. Multi-agent curricula and emergent implicit signaling. In Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems, AAMAS 22, pp. 553 561, Richland, SC, 2022. International Foundation for Autonomous Agents and Multiagent Systems. ISBN 9781450392136.
Shixiang Gu, Ethan Holly, Timothy Lillicrap, and Sergey Levine. Deep reinforcement learning for robotic manipulation with asynchronous off- updates, 2016. URL http://aaxii..orrraa///////// hatpb///111.000660
Shengyue Guan, Haoyi Xiong, Jindong Wang. Jiang Bian, Bin Zhu, and Jian-guang Lou. Evaluating 1lm- based agents for multi-turn conversations: survey. arXiv preprind orXiv:2503.29456 2025.
Shangmin Guo. Emergence of numeric concepts in multi-agent autonomous communication. arXiv preprint wxXiv:1911.01000 2019.
Shangmin Guo, Yi Ren, Kory Wallace Mathewson, Simon Kirby, Stefano V Albrecht, and Kenny Smith. Expressivity of emergent languages is a trade-off between contextual complexity and unpredictability. In International Conference on Learning Representations.
Shangmin Guo, Yi Ren, Kory Mathewson, Simon Kirby, Stefano V Albrecht, and Kenny Smith. Expressivity of emergent language is a trade-off between contextual complexity and unpredictability. arXiv preprint arXiv:2106. 03982, 2021.
Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V Chawla, Olaf Wiest, and Xiangliang Zhang. Large language model based multi-agents: A survey of progress and challenges. arXiv preprint Xrx::22402.0 01680, 2024a.
Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V Chawla, Olaf Wiest, and Xiangliang Zhang. Targe language model based multi-agents: A survey of progress and challenges. arxiv preprint Xiv:22400..0 01680, 2024b.
Xudong Guo, Kaixuan Huang, Jiale Lin, Wenhui Fan, Natalia Vélez, Qingyun Wu, Huazheng Wang, Thomas L Griffiths, and Mengdi Wang. Embodied Ilm agents learn to cooperate in organized teams. arXiv preprint arXiv:2403. 12482, 2024c.
Abhinav Gupla, Mare Lanetot, and Angeliki Lazaridou. Dynamic population-based meta-learning for multi- agent communication with natural language. Advances in Newral Information Processing Systems, 34: 16899-16912, 2021.
Nikunj Gupta, G Srinivasaraghavan, Swarup Mohalik, Nishant Kumar, and Matthew E Taylor. Hammer: Multi-level coordination of reinforcement learning agents via learned messaging. Neural Computing and Applications, pp. 1-16, 2023.
127

---

Published in Transactions on Machine Learning Research (02/2026)
Shubham Gupta, Rishi Hazra, and Ambedkar Dukkipati. Networked multi-agent reinforcement learning with emergent communication. arXiv preprint arXiv:2004.02780 2020.
Thorsten Händler. Balancing autonomy and alignment: a multi-diimensional Laxonomy for autonomous llm-powered multi-agent architectures. arXiv preprint orXiv:2310.03659, 2023.
Matthew Hausknecht and Peter Stone. Deep recurrent q-learning for partially observable mdps. In 2015 aaai fall symposium series, 2015.
Serhii Havrylov and Ivan Titov. Emergence of language with multi-agent games: Learning to communicate with sequences of symbols. Advances in neural information processing systems, 30, 2017.
Pablo Hemaaddz-Leal,, Bilal Kartal, and Matthow Taylor. A survey and critique of multiagent deep reinforcement learning. Aulonomous Agents and Mulli- Agent. Systems, 33(6):750-77) 797, 2019.
Jaakko Hintikka. Knowledge and Belief: An Introduction to the Logic of the Two Notions. Cornell Universily Press, Ithaca, NY, 1962.
Yu-Chi Ho et al. Team decision theory and information structures optimal control problems part i. IEEE Transactions on Automatic control, 17(11:15-22) 1972.
Jerry R Hobbs, Mark E Stickel, Douglas E Appelt, and Paul Martin. Interpretation as abduction. Artificial intelligence, 63(-))) 1993.
Samuel Holt, Max Ruiz Luyten, and Mihacla van der Schaar. L2mac: Large language model automatic computer for exlensive code generation. In The Twelfth International Conference on Learuing Represen- tations.
Sirui Hong, Xiawu Zheng, Jonathan Chen, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, et al. Metagpt: Meta programming for multi-agent collaborative framework. ar preprint ar Xrv::22308. 00352, 2023.
Guangzhong HU, Yuanheng Zhu, Dongbin Zhao, Mengehen Zhao, and Jianye Hao. Event-triggered multi- agent reinforcement learning with communication under limited- batdwiddb constraint. arXiv preprint 2XXx:2010.04975 2020a.
Hengyuan Hu, Adam Lerer, Alex Peysakhovich, and Jakob Foerster. "other-play" for zero-shot coordination. International Conference on Machine Learning, pp. 4399-4410. PMLR, 2020b.
Shengran Hu, Cong Lu, and Jeff Clune. Automated design of agentic systems. arXiv:2408.08// 2024a.
arXiv preprint
Sihao Hu, Tiansheng Huang, Fatih Ilhan, Selim Tekin, Gaowen Liu, Ramana Kompella, and Ling Liu. A survey on large language model-based game agents. arXiv preprint arXi:22404 020099,0 2024b.
Yuwei Hu, Runlin Lei, Xinyi Huang, Zhewei Wei, and Yongchao Liu. Scalable and accurate graph reasoning with llm- based multi-agents. arXiv preprint arXiv:2410. 10.00330,,,00 2024c.
Wenyue Hua, Lizhou Fan, Lingyao Li, Kai Mei, Jianchao Ji, Yingqiang Ge, Libby Hemphill, and Yongfeng Zhang. War and peace (waragent): Large language model-based multi-agent simulation of world wars. arXiv preprint arXit:2311.17992 2023.
Kiran Ikram, Esther Mondragón, Eduardo Alonso, and M Garcia- Carcia-Ortiz. Hexajungle: a marl simulator to study the emergence of language. 2021.
Luca locchi et al. Emergent communication in human-machine games. In Emergent Communication Work- shop at ICLR 2022, 2022.
Yoichi Ishibashi and Yoshimasa Nishimura. Self-organized agents: A Ilm multi-agent framework toward ultra large-scale code generation and optimization. arXiv preprint arxi::2404 02183, 2024.
128

---

Published in Transactions on Machine Learning Research (02/2026)
Md Ashraful Islam, Mohammed Eunus Ali, and Md Rizwan Parvez. Mapcoder: Multi-agent code generation for competitive problem solving, arXiv preprint arXiv:2405.11 2024.
Shankar Kumar Jeyakumar, Alaa Alameer Ahmad, and Adrian Garret Gabriel. Advancing agentic systems: Dynamic task decomposition, tool integration and evaluation using novel metrics and dataset. In NeurIPS 2024 Workshop on Open- World Agents, 2024.
Dongfu Jiang, Xiang Ren, and Bill Yuchen Lin. Lim- blender: Ensembling large language models with pairwise ranking and generative fusion. arxiv preprint arXiv:236..00 Xix:2306.022611, 2023.
Jiechuan Jiang and Zongging Lu. Learning attentional communication for multi-agent cooperation. Advances in neural information processing systems, 31, 2018.
Jiechuan Jiang, Chen Dun, Tiejun Huang, and Zongging Lu. Graph convolutional reinforcement learning. arXiv preprint Xrxi 09202, 2018.
Xuanfa Jin, Ziyan Wang, Yali Du, Meng Fang, Haileng Zhang, and Jun Wang. Learning to discuss strategi- cally: A case study on one night ultimate werewolf. Advances in Neural Information Processing Systems, 37:77060-77097, 2024.
Shi Jinxin, Zhao Jiabao, Wang Yilei, Wu Xingjiao, Li Jiawen, and He Liang. Cgmi: Configurable general multi-agent interaction framework. arXiv preprint arXiv:2308.122 111000...200 12503, 2023.
Emilio Jorge, Mikael Kägebäck, Fredrik D Johansson, and Emil Gustavsson. Learning play guess who?
and inventing a grounded language as a consequence. arXiv preprint arXiv:1611. 002188 2016.
Zhao Kaiya, Michelangelo Naim, Jovana Kondic, Manuel Cortes, Jiaxin Ge, Shuying Luo, Guangyu Robert Yang, and Andrew Ahn. Lyfe agents: Generative agents for low-cost real-time social interactions. arXiv preprint arXiv:2310.00 02172, 2023.
Aleksandra Kalinowska, Elnaz Davoodi, Florian Strub, Kory Mathewson, Todd Murphey, and Patrick Pi- larski. Situated communication: A solution to over-communication between artificial agents. In Emergent Communication Workshop at ICLR 2022, 2022.
Yipeng Kang, Tonghan Wang, and Gerard de Melo. Incorporating pragmatic reasoning communication into emergent language. Advances in neural information processing systems, 33:10348-10359, 2020.
Shyam Sundar Kannan, Vishnunandan LN Venkatesh, and Byung-Cheol Min. Smart-llm: Smart multi-agent robot task planning using large language models. arXio preprint arXiv:2309. 10062, 2023.
Seth Karten, Mycal Tucker, Huao Li, Siva Kailas, Michael Lewis, and Katia Sycara. Interpretable learned emergent communication for human agent teams. IEEE Transactions on Cognilive and Developmental Systems, 15(4):1801-181) 2023.
Eugene Kharitonov and Marco Baroni. Emergent language generalization and acquisition speed are not tied to compositionality. arXiv preprint orXiv:2004.03420, 2020.
Eugene Kharitonov, Rahma Chaabouni, Diane Bouchacourt, and Marco Baroni. Egg: a toolkit for rescarch on emergence language games. arxio preprinl ur 1rxiv:1900 1900.00852. 2019.
Eugene Kharitonov, Rahma Chaabouni, Diane Bouchacourt, and Marco Baroni. Entropy minimization in emergent languages. In International Conference on Machine Learning, pp. 5220 5230. PMLR, 2020.
Diksha Khurana, Aditya Koli, Kiran Khattor, and Sukhdev Singh. Natural language processing: state of the art, current trends and challenges. Multimedia lools and applications, 82(3):3713 3744, 2023.
Ozsel Kiline and Giovanni Montana. Multi-agent deep reinforcement learning with extremely noisy obser- vations. arXiv preprint arXiv:1812.0090 00922, 2018
Daewoo Kim, Sangwoo Moon, David Hostallero, Wan Ju Kang, Taeyoung Lee, Kyunghwan Son, and Yung Yi. Learning to schedule communication in multi-agent reinforcement learning. arXiv preprint 1rXin:1902.0155: 2019a.
129

---

Published in Transactions on Machine Learning Research (02/2026)
Woojun Kim, Myungsik Cho, and Youngehul Sung. Message-dropout: An efficient training method for multi-agent deep reinforcement learning. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pp. 6079-6086, 2019b.
Woojun Kim, Jongeui Park, and Youngchul Sung- Communication in multi-agent. reinforcement learning: Intention sharing. In International conference learning representations, 2020.
Simon Kirby, Natural language from artificial life. Artificial life, 8(2):185 215, 2002.
Simon Kirby, Mike Dowman, and Thomas L Griffiths. Innateness and culture in the evolution of language. Proceedings of the National Academy of Sciences, 104(12):5211-5245, 2007.
Simon Kirby, Hannah Cornish, and Kenny Smith. Cumulative cultural evolution in the laboratory: An experimental approach to the origins of structure in human language. Proceedings of the Nutional Academy of Sciences, 105(31):
2008.
Ross A Knepper, Christoforos Mavrogiannis, Julia Proft, and Claire Liang. Implicit communication in a joint action. In Proceedings of the 2017 acm/ieee international conference humon-robot interaction, pp. 283-292, 2017.
Benjamin Kolb, Leon Lang, Henning Bartsch, Arwin Gansekoele, Raymond Koopmanschap, Leonardo Ro- mor, David Speck, Mathijs Mul, and Elia Bruni. Learning to request guidance in emergent language. Proceedings of the Beyond Vision and LANguage: inTEgrating Real-world &Nowledge (LANTERN), pp. 41-50, 2019.
Xiangyu Kong, Bo Xin, Fangchen Liu, and Yizhou Wang. Revisiting the master-slave architecture in multi- agent deep reinforcement learning. arXiv preprint or xrx: 1712. 2.03305, 2017.
Tomasz Korbak, Julian Zubek, Lukasz Kuciński, Piotr Miloš, and Joanna Raczasvek-Lconardi.. Develop- mentally motivated emergence of compositional communication via template transfer. arXio preprint arXiv:1910.06079 2019.
Tomasz Korbak, Julian Zubek, and Joanna Laczaszek-Leonardi.. Mcasuring non-trivial compositionality in emergent communication. arXiv preprint 1rXim:2010.15088, 2020.
Satwik Kottur, José Moura, Stefan Lee, and Dhruv Batra. Natural language does not emerge 'naturally 'in multi-agent dialog. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 2962-2967, 2017.
Landon Kraemer and Bikramjit Banerjee. Multi-agent reinforcement learning as a rehearsal for decentralized planning. Neurocomputing. 190:82-94 94, 2016.
Nikolaus Kriegeskorte, Maricke Mur, and Peter A Bandettini. Representational similarity analysis-connecting the branches of systems neuroscience. Frontiers in systems neuroscience, 2:249, 2008
Lukasz Kuciáski, Pawel Kotodziej, and Piotr Miloć. Emergence of compositional language in communication through noisy channel. In Language in Reinforcement Learning Workshop at ICML 2020, 2020.
Travis LaCroix. Biology and compositionalityy Empirical considerations for emergent-commmiattnn pro- tocols. arXiv preprint 191 1666, 2019.
Guillaume Lample and Devendra Singh Chaplot. Playing fps games with deep reinforcement learning, 2016.
URL
Yihuai Lan, Zhiqiang Hu, Lei Wang, Yang Wang, Deheng Ye, Peilin Zhao, Fe-Peng Lim, Hui Xiong, and Hao Wang. Llin-bsedd agent society investigation: Collaboration and confrontation in avalon gameplay. arXiv preprint Xrx:::2310.11 14985, 2023.
Ivano Lauriola, Alberto Lavelli, and Fabio Aiolli. An introduction deep learning in natural language processing: Models, techniques, and tools. Neurocomputing, 470:443 456, 2022.
130

---

Published in Transactions on Machine Learning Research (02/2026)
Angeliki Lazaridou and Marco Baroni. Emergent multi-agent communication in the deep learning era. arXiv preprint wrXYc:2006.0244 2020.
Angeliki Lazaridou, Alexander Peysakhovich, and Marco Baroni. Multi-agent cooperation and the emergence of (natural) language. arXiv preprint Xrv: 1rXi::1612.001 1et:.011208.2 2016.
Angeliki Lazaridou, Karl Moritz Hermann, Karl Tuyls, and Stephen Clark. Emergence of linguistic commu- nication from referential games with symbolic and pixel inpul. In International Conference on Learning Representations, 2018.
Angeliki Lazaridou, Anna Potapenko, and Olivier Tieleman. Multi-agent commmication meets natural language: Synergies between functional and structural language learning. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020.
Angeliki Lazaridou, Adhi Kuncoro, Elena Gribovskaya, Devang Agrawal, Adam Liska, Tayfun Terzi, Mai Gimenoz, Cyprien de Masson d' A'Atumme Tomas Kocisky, Scbastian Ruder, et al. Mind the gap: Assessing temporal generalization in neural language models. Advances in Neural Information Processing Systems, 34:29348 14429338-29363, 2021.
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521 (7553) :335333333---0 444, 2015.
Jason Lee, Kyunghyun Cho, Jason Weston, and Douwe Kiela. Emergent translation in multi-agent commu- nication. ar preprint arXiv: 1700. 0.06922, 2017.
Jason Lee, Kyunghyun Cho, and Douwe Kiela. Countering language drift via visual grounding. arXiv preprint arXio:1909.044: 2019.
Joel Leibo, Vinicius Zambaldi, Marc Lanctot, Janusz Marecki, and Thore Graepel. Multi-agent reinforce- ment learning in sequential social dilemmas. arXiv preprint orXiv:1702.03037 2017.
Oliver Lemon. Conversational grounding emergent communication data and divergence. In Emergent
Communécation Workshop at ICLR 2022, 2022.
Hector J Levesque. A logic of implicit and explicit belief. In AAAL, volume 84, pp. 198 202, 1984.
David Lewis. Convention: A philossppical study. John Wiley & Sons, 2008.
Cheng Li, Damien Teney, Linyi Yang, Qingsong Wen, Xing Xie, and Jindong Wang. Culturepark: Boosting cross-cultural understanding in large language models. arXiv preprint 2555 2024a.
Dapeng Li, Zhiwei Xu, Bin Zhang, and Guoliang Fan. From explicit communication to tacit cooperation: A novel paradigm for cooperative marl. arXiv preprint arXie:2334414466, 2023a.
Fushan Li and Michael Bowling. Case-of-teaching and language structure from emergent communication. Advances in neural information processing systems, 32, 2019.
Guohao Li, Hasan Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. Camel: Communicative agents for" mind" exploration of large language model society. Advances in Neural Information Processing Systems, 36:51991-5200S 2023b.
Huao Li, Hossein Nourkhiz Mahjoub, Behdad Chalaki, Vaishnav Tadiparthi, Kwonjoon Lee, Ehsan Moradi Pari, Charles Michael Lewis, and Katia P Sycara. Language grounded multi-agent reinforcement learning with human-interpretable communication. In The Thirlg-eighth Annual Conference on Neural Information Processing Systems.
Huao Li, Yu Chong, Simon Stepputtis, Joseph Campbell, Dana Hughes, Charles Lewis, and Katia Sycara. Theory of mind for multi-agent collaboration via large language models. In Proceedings of the 2023 Conference Empirical Methods in Natural Language Processing, pp. 180-192, 192, 2023c.
131

---

Published in Transactions on Machine Learning Research (02/2026)
Huao Li, Hossein Nourkhiz Mahjoub, Behdad Chalaki, Vaishnav Tadiparthi, Kwonjoon Lee, Ehsan Moradi Pari, Charles Lewis, and Katia Sycara, Language grounded multi-agent reinforcement learn- ing with auman-iteerpretable communication. Advances in Neural Information Processing Systems, 37: 87908 87933, 2024b.
Junkai Li, Yunghwei Lai, Weitao Li, Jingyi Ren, Meng Zhang, Xinhui Kang, Siyu Wang, Peng Li, Ya-Qin Zhang, Weizhi Ma, et al. Agent hospital: A simulacrum of hospital with evolvable medical agents. arxiv preprint ar (rXic2405.002957 2024c.
Junyou Li, Qin Zhang, Yangbin Yu, Qiang Fu, and Deheng Ye. More agents is all you need. arXiv preprint arXiv:2402.055 05120, 2024d.
Ling Li, Alcherio Martinoli, and Yaser S. Abu-Mostafa. Emergent specialization in swarm systems. In Pro- ceedings of the Third International Conference on Intelligent Data Engineering and Automated Learning, IDEAL '02, pp. 261 266, Berlin, Heidelberg, 2002. Springer- Verlag. ISBN 3540440259.
Sheng Li, Jayesh K. Gupta, Peter Morales, Ross Allen, and Mykel J. Kochenderfer. Deep implicit coordina- tion graphs for multi-agent reinforcement learning, 2020. URL https://arxiv.org/ wtwg///g/b//..ooo.a///////////oo 11438
Sheng Li, Yutai Zhou, Ross Allen, and Mykel J Kochenderfer. Learning emergent discrete message com- munication for cooperative reinforcement learning. In 2022 International Conference on Robotics and Automation (ICRA), pp. 5511-5517. IEEE, 2022.
Xinyi Li, Sai Wang, Siqi Zeng, Yu Wu, and Yi Yang. A survey on Ilm-based multi-agent systems: workflow, infrastructure, and challenges. Vicinagearth, 111):9, 2024c.
Yunxuan Li, Yibing Du, Jiageng Zhang, Le Hou, Peter Grabowski, Yeqing Li, and Eugene le. Improving multi-agent debate with sparse communication topology. arXiv preprint arXiv:2406. 11776, 2024f.
Paul Pu Liang, Jeffrey Chen, Ruslan Salakhutdinov, Louis-Philippe Morency, and Satwik Kottur. On emer- gent communication in competitive multi-agent teams. In Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems, pp. 735-743, 2020.
Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, et al. Holistic evaluation of language models. arXiv preprint ar (.Xi::2211091111 2022.
Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Shuming Shi, and Zhaopeng Tu. Encouraging divergent thinking large language models through multi-agent debate. arXiv preprint XrX:::2305.1 19118, 2023.
Jonathan Light, Min Cai, Sheng Shen, and Zinin Hu. Avalonbench: Evaluating 1lms playing the game of avalon. Xiv preprint ar arXic:2310.0000 2023.
Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning, 2015. URI. https: //arxiv.
baaxxiv.org/atee wttg/////////....//////// 02971
Toru Lin, Jacob Huh, Christopher Stauffer, Ser Nam Lim, and Phillip Isola. Learning to ground multi-agent communication with autocncoders. Advonces in Neural Information Processing Systems, 14115330-15245 2021.
Dorota Lipowska and Adam Lipowski. Emergence and evolution of language in multi-agent systems. Lingua, 272:103331, 2022.
Bang Liu, Xinfeng Li, Jiayi Zhang, Jinlin Wang, Tanjin He, Sirui Hong, Hongzhang Liu, Shaokun Zhang, Kaitao Song, Kunlun Zhu, al. Advances and challenges in foundation agents: From inspiied intelligence to evolutionary, collaborative, and safe systems. arXio preprint arXim:2504.01900 01135500.11990 2025a.
132

---

Published in Transactions on Machine Learning Research (02/2026)
Bang Liu, Xinfeng Li, Jiayi Zhang, Jinlin Wang, Tanjin He, Sirui Hong, Hongzhang Liu, Shaokun Zhang, Kaitao Song, Kunlun Zhu, et al. Advances and challenges in foundation agents: From brain-inspiin intelligence to evolutionary, collaborative, and safe systems. arXiv preprint arXim:2004.0199) 2025b.
Siqi Liu, Guy Lever, Josh Merel, Saran Tunyasuvunakool, Nicolas Heess, and Thore Graepel. Emergent coordination through competition. In International Conference on Learning Representations.
Wei Liu, Chenxi Wang, Yifei Wang, Zihao Xie, Rennai Qiu, Yufan Dang, Zhuoyun Du, Weize Chen, Cheng Yang, and Chen Qian. Autonomous agents for collaborative task under information asymmetry. arXiv preprint Xrv:22406. 14928, 2024a.
Yong Liu, Weixun Wang, Yujing Hu, Jianye Hao, Xingguo Chen, and Yang Gao. Multi-agent game abstrac- tion via graph attention neural network. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 7211-7218, 2020.
Zhiwei Liu, Weiran Yao, Jianguo Zhang, Le Xue, Shelby Heinocke, Rithesh Murthy, Yihao Feng, Zeyuan Chen, Juan Carlos Niebles, Devansh Arpil, et al. Bolaa: Benchmarking and orchestrating Ilm-augmenttd autonomous agents. arXiu preprint or ar:802908. 05960, 2023a.
Zijun Liu, Yanzhe Zhang, Peng Li, Yang Liu, and Diyi Yang. Dynamic Imm-agent network: An Imm-agent collaboration framework with agent team optimization. areee preprint arXiv:2310.021 2023b.
Zijun Liu, Yanzhe Zhang, Peng Li, Yang Liu, and Diyi Yang. A dynamic llm-powered agent network for task-oriented agent collaboration. In First Conference on Language Modeling, 2024b.
Yat Long Lo and Biswa Sengupta. Learning to ground decentralized multi-agent. communication with con- trastive learning. In Emergent Communication Workshop at ICLR 2022.
Ryan Lowe, Yi Wu, Aviv Tamar, Jean Harb, Pieter Abbeel, and Igor Mordatch. Multi-agent actor-critic for mixed cooperative- competttive environments, 2017. URL https://arxxv.org/ wwwpg/abb//700 02275
Ryan Lowe, Jakob Foerster, Y-Lan Boureau, Joelle Pineau, and Yann Dauphin. On the pitsalls of measuring emergent communication. In Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS '19, pp. 693-701. Richland, SC, 2019. International Foundation for Autonomous Agents and Multiagent Systemss. ISBN 9781450363099.
Ryan Lowe, Abhinav Gupta, Jakob Foerster, Douwe Kiela, and Joelle Pincau. On the interaction between supervision and self-play in emergent communication. International Conference Learning Representa- tions, 2020.
Diana Rodríguez Luna, Edoardo Maria Ponti, Dieuwke Hupkes, and Elia Bruni. Internal and external pres- sures on language emergence: least effort, object constancy and frequency. In Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 1128-4437, 2020.
Weiyu Ma, Qirui Mi, Yongeheng Zeng, Xue Yan, Yuqiao Wu, Runji Lin, Haifeng Zhang, and Jun Wang. Large language models play starcraft ii: Benchmarks and a chain of summarization approach. arXiv preprint 2rXi::2312.111 11865, 2023.
Kyle Mahowald, Anna A Ivanova, Idan A Blank, Nancy Kanwisher, Joshua B Tenenbaum, and Evelina Fedorenko. Dissociating language and thought in large language models. Trends in cognitive sciences, 28 (6):517-540, 2024.
Aleksandra Malysheva, Tegg Taekyong Sung, Chae-Bong Sohn, Daniel Kudenko, and Aleksei Shpilman. Deep multi-agent reinforcement learning with relevance graphs. arXiv preprint rxXiv:1811.1255 2018.
Zhao Mandi, Shreeya Jain, and Shuran Song. Roco: Dialectic multi-robot collaboration with large language models. In 2024 IEEE International Conference on Roboties and Automation (ICRA), pp. 286-299. IEEE, 2024.
Christopher Manning and Hinrich Schutze. Foundations of statistical natural language processing. MIT press, 1999.
133

---

Published in Transactions on Machine Learning Research (02/2026)
Hangyu Mao, Zhengchao Zhang, Zhen Xiao, Zhibo Gong, and Yan Ni. Learning agent communication under limited bandwidth by message pruning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 5142-5149, 2020.
Laetitia Matignon, Guillaume J Laurent, and Nadine Le Fort-Piat. Independent reinforcement learners in cooperative markov games: a survey regarding coordination problems. The Knowledge Engineering Review, 17-1) 2012.
Lingheng Meng, Daiwei Lin, Adam Francey, Rob Gorbet, Philip Becsley, and Dana Kulić. Learning to engage with interactive systems: A field study on deep reinforcement learning in public museumn, 2019. URL htttpp://axxvvorg/////
06764
William Merrill, Yoav Goldberg, Roy Schwartz, and Noah A Smith. Provable limitations of acquiring meaning from ungrounded form: What will future language models understand? Transactions of the Association for Computational Linguistics, 9:1047 9:1047-1060, 2021.
John-Jules Ch Meyer. Modal epistemic and doxastic logic. In Handbook of philosophical logic, pp. 1-38. Springer, 2003.
Grégoire Mialon, Clémentine Fourrier, Thomas Wolf, Yann LeCun, and Thomas Scialom. Gaia: a benchmark for general ai assistants. In The Twelfth International Conference on Learning Representations, 2023.
Daniela Mihai and Jonathon Hare. The emergence of visual semantics through communication games. ar preprint Xiv:22101 10253, 2021.
Reuth Mirsky, William Macke, Andy Wang, Harel Yedidsion, and Peter Stone. A penny for your thoughts: The value of communication in ad hoc teamwork. International Joint Conference on Artificial Intelligence, 2020.
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, loannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning, 2013. URL https://arxiiv http://awxvvv....aabaaa /bs// 5602.5602
Richard Montague. English as formal language. 1970.
Richard Montague and Richmond H Thomason. Formal philosophy: selected papers of richard montague. 1978.
Igor Mordatch and Pieter Abbeel. Emergence of grounded compositional language in multi-agent populations. In Proceedings of the AAAI conference artificial intelligence, volume 32, 2018.
Manuel Mosquera, Juan Sebastian Pinzon, Manuel Rios, Yesid Fonseca, Luis Felipe Giraldo, Nicanor Qui- jano, and Ruben Manrique. Can Ilm-augmented autonomous agents cooperate?, evaluation of their cooperative capabilities through melting pot. ar preprint (xXiv:2403.1188, 2024.
Manuel Mosquera, Juan Sebastian Pinzon, Yesid Fonseca, Manuel Rios, Nicanor Quijano, Luis Felipe Gi- raldo, and Ruben Manrique. Can IIm-augmented autonomous agents cooperate? an evaluation of their cooperative capabilities through melting pot. IEEE Transactions on Artificial Intelligence, 2025.
Clément Moulin- Frier and Pierre- Yves Oudeyer. Multi-agent reinforcement learning as computational tool for language evolution research: Historical context and future challenges. ar preprint arXiv:2002.08878 2020.
Jesse Mu and Noah Goodman. Emergent communication of generalizations. Advances in neural information processing systems, 34:17994-1888 18007, 2021.
Mathijs Mul, Diane Bouchacourt, and Elia Bruni. Mastering emergent language: learning to guide in simulated navigation. ar preprint 1rXii:1908.05..1 2019.
Nathalia Nascimento, Paulo Alencar, and Donald Cowan. Gpt-in-the-loop. Adaptive decision-makiin for multiagent systems. Xiu preprint or 2rxx:22308..003 10435, 2023a.
134

---

Published in Transactions on Machine Learning Research (02/2026)
Nathalia Nascimento, Paulo Alencar, and Donald Cowan. Self-adaptive large language model (1lm)-based multiagent systems. In 2023 IEEE International Conference on Autonomic Computing and Self- Organizing Systems Companion (ACSOS-C), pp. 104-109. IEFE, 2023b.
Thanh Thi Nguyen, Ngoc Duy Nguyen, and Saeid Nahavandi. Deep reinforcement learning for multiagent
systems: A review of challenges, solutions, and applications. IEEE transactions on cybernetics, 50(9): 3826-3830 2020.
Zepong Ning and Lihua Xic. A survey on multi-agent reinforcement learning and its application. Journal of Aulomation and Intelligence, 3(2):73 91, 2024.
Eleni Nisioti, Elías Masquil, Gautier Hamon, and Clément Moulin-Frier. Autotelic reinforcement learning in multi-agent environments, 2023. URL
Yaru Niu, Rohan R Paleja, and Matthew C Gombolay. Multi-agent graph-attention communication and teaming. In AAMAS, pp. 964-973, 2021.
Michael Noukhovitch, Travis LaCroix, Angeliki Lazaridou, and Aaron Courville. Emergent communica- tion under competition. In Proceedings of the 20th International Conference on Autonomous Agents and MulliAgent Systems, pp. 974 982, 2021.
Martin A Nowak and David C Krakauer. The evolution of language. Proceedings of the National Academy of Sciences, 96(14):8028 8033, 1999.
Frans A Oliehoek, Christopher Amato, et al. A concise introduction to decentralized POMDPs, volume 1. Springer, 2016.
Michael Oliphant and John Batali. Learning and the emergence of coordinated communication. Center for research on language newsletter, 11(1):1-46, 1997.
OpenAL. Openai swarm. httpp://githhb. http:///huub..oooppeaaaa.. chomm/ppeaa/swwrn
Ewa Orlowska. Kripke semantics for knowledge representation logics. Studia Logica, 49(2):255-277) 1990.
Afshin Oroajlooy and Davood Hajinezhad. review of cooperative multi-agent deep reinforcement learning. Applied Intelligence, 53(11): 13722, 2023.
Joni Pajarinen and Ville Kyrki. Robotic manipulation of multiple objects as a pomdp. Artificial Intelligence, 247:213-20 228, 2017.
Bo Pan, Jiaying Lu, Ke Wang, Li Zheng, Zhen Wen, Yingchaojie Feng, Minfeng Zhu, and Wei Chen. Agent- coord: Visually exploring coordination strategy for Ilm-bascn multi-agent collaboration. arXiv preprint arXiv:2404.11 119433 2024.
Joon Sung Park, Joseph O'Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology, pp. 1-22, 2023.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
Shivansh Patel, Saim Wani, Unnat Jain, Alexander G Schwing, Svetlana Lazebnik, Manolis Savva, and Angel X Chang. Interpretation of emergent communication in heterogeneous collaborative embodied agents. In Proceedings of the IEFE/CVF International Conference on Computer Vision, pp. 15953- 15963, 2021.
James Paulos, Steven W Chen, Daigo Shishika, and Vijay Kumar. Decentralization of multiagent policies by learning what to communicate. In 2019 International Conference on Roboties and Automation (ICRA), pp. 7990-7996. TEFE, 2019.
135

---

Published in Transactions on Machine Learning Research (02/2026)
Peng Peng, Ying Wen, Yaodong Yang. Quan Yuan, Zhenkun Tang, Haitao Long, and Jun Wang. Multiagent pntrrectionlly-coordintttt nets: Emergence of human-level coordination in learning to play starcraft combat games, 2017. URL
Fernando CN Pereira. and David HD Warren. Definite clause grammars for language analysis survey of the formalism and a comparison with augmented transition networks. Artificial intelligence, 13(3): 231 278, 1980.
Hugh Perkins. Neural networks can understand compositional functions that humans do not, in the context of emergent communication. arXiv preprint arXit:2103.0442 041000 2021a.
Hugh Perkins. Texrel: a green family of datasets for emergent communications on relations. ar preprinl orXiv:2105.19880 2021b.
Emanuele Pesce and Giovanni Montana. Improving coordination in small-scale multi-agent deep reinforce- ment learning through memory-driven communication. Machine Learning, 10077-000:17177771777 jan 2020a. htt.:///........./....0000-00--.
doi: 10.1007 00t7//119944011-05888809
URL
Emanuele Pesce and Giovanni Montana. Improving coordination in small-scale multi-agent deep reinforce- ment lcarning through memory-driven communication. Machine Learning, 109(9):1727-1777, 2020b.
Jannik Peters, Constantin Waubert de Puiseau, Hasan Tercan, Arya Gopikrishnan, Gustavo Adolpho Lueas De Carvalho, Christian Bitter, and Tobias Meisen. A survey on emergent language. arXiv preprint arXivv2409 02645. 2024.
Giorgio Piatti, Zhijing Jin, Max Ciehman-Weinne Bernhard Schölkopf, Mrinmaya Sachan, and Rada Mihal- cea. Cooperate or collapse: Emergence of sustainable cooperation in society of 1lm agents. Advonces in Newral Information Processing Systems, 37:111715 111759, 2024.
M. Pipattanasomporn, H. Feroze, and S. Rahman. Multi-agent systems in distributed smart grid: Design and implementation. In 2009 IEEE/PES Power Systems Conference and Exposition, pp. 1 8, 2009. doi: 4840007
David Premack and Guy Woodruff. Does the chimpanzee have theory of mind? Behavioral and Braón Sciences, 144):515-526, 1978.
Mikhail Prokopenko and Peter Wang. Relating the entropy of joint beliefs to multi-agent coordination. RoboCup 2002: Robot Soccer World Cup VI 6, pp. 367-374. 374. Springer, 2003.
Martin L Puterman. Markov decision processes: discrete stochastic dynamic programming. 2014.
Chen Qian, Xin Cong, Cheng Yang, Weize Chen, Yusheng Su, Juyuan Xu, Zhiyuan Liu, and Maosong Sun. Communicative agents for software development. arXiv preprint wxXi:230770099.3 6, 2023.
Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, et al. Chatdev: Communicative agents for software development. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Popers), pp. 15174 15186, 2024a.
Chen Qian, Zihao Xie, Yifei Wang. Wei Liu, Yufan Dang, Zhuoyun Du, Weize Chen, Cheng Yang, Zhiyuan Liu, and Maosong Sun. Scaling sogelllanuuge--ndll-bbeeee multi-agent collaboration. arXio preprinl arXiv:2406.0713 07155, 2024b.
Shuofei Qiao, Ningyu Zhang, Runnan Fang. Yujie Luo, Wangchunshu Zhou, Yuchen Fleanor Jiang, Chengfei Lv, and Huajun Chen. Autoact: Automatic agent learning from scratch for qa via self-planning. arXiv preprint arXie:2401.05266 2024.
Shuwen Qiu, Sirui Xie, Lifeng Fan, Tao Gao, Jungseock Joo, Song- Chun Zhu, and Yixin Zhu. Emergent graphical conventions in a visual commmication game. Advances in Neural Information Processing Sys- tems, 35:13119- 2022.
136

---

Published in Transactions on Machine Learning Research (02/2026)
Xipeng Qiu, Tianxiang Sun, Yige Xu, Yunfan Shao, Ning Dai, and Xuanjing Huang. Pre-trained models for natural language processing: A survey. Science China technological sciences, 63(10):1872-1897) 2020.
Chao Qu, Hui Li, Chang Liu, Junwu Xiong, Wei Chu, Weiqiang Wang, Yuan Qi, Le Song, et al. Intention propagation for multi-agent reinforcement learning. 2020.
Ahmed Hussain Qureshi, Yutaka Nakamura, Yuichiro Yoshikawa, and Hiroshi Ishiguro. Robot gains social intelligence through multimodal deep reinforcement learning, 2017. URL 07492
Murtaza Rangwala and Ryan Williams. Learning multi-agent. communication through structured attentive reasoning. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS'20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 9781713829546.
Sumodh Rasal and EJ Hauer. Navigating complexity: Orchostrated problem solving with multi-agent Ilms. arXiv preprinl Xrxi:2402. 16713, 2024.
Tabish Rashid, Mikayel Samvelyan, Christian Schroeder, Gregory Farquhar, Jakob Foerster, and Shimon Whiteson. Quix: Monotonic value function factorisation for deep multi-agent reinforcement learning. In International conference on machine learning, pp. 1295-4304 PMLR, 2018.
Yi Ren, Shangmin Guo, Matthieu Labeau, Shay B Cohen, and Simon Kirby. Compositional languages emerge in neural iterated learning model. arXiv preprint or 20Xi::2002.0333, 2020.
Cinjon Resnick, Abhinav Gupta, Jakob Foerster, Andrew M Dai, and Kyunghyun Cho. Capacity, bandwidth, and compositionality in emergent language learning. arXiv preprint 19X.:1910.11124, 2019.
Cinjon Resnick, Abhinav Gupta, Jakob Foerster, Andrew M Dai, and Kyunghyun Cho. Capacity, bandwidth, and compositionality in emergent language learning. In 19th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2020, pp. 1125 1133. International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS), 2020.
Mathicu Rita, Florian Strub, Jean-Bastien Grill, Olivier Pietquin, and Emmanuel Dupoux. On the role of population heterogeneity in emergent communication. In ICLP 2022- Tenth International Conference on Learning Representations, 2022.
Kai Ruan, Mowen Huang, Ji-Rong Wen, and Hao Sun. Benchmarking 1lms' swarm intelligence. arXiv preprint xrc:222505.0436 2025.
Stuart Russell, Peter Norvig, and Artificial Intelligence. A modern approach. Artificial Intelligence. Prentice- Hall, Egnlewood Cliffs, 25(27):79 80, 1995.
Heechang Ryu, Hayong Shin, and Jinkyoo Park. Cooperative and competitive biases for multi-agent rein- forcement learning. arXiv preprint arXiv:2101.06890, 2021.
Mikayel Samvelyan, Tabish Rashid, Christian Schroeder De Witt, Gregory Farquhar, Nantas Nardelli, Tim GJ Rudner, Chin-Man Hung, Philip HS Torr, Jakob Foerster, and Shimon Whiteson. The star- craft multi-agent challenge. arXiv preprint ar Xti:1990.040433 2019.
Pedro P Santos, Diogo S Carvalho, Miguel Vasco, Alberto Sardinha, Pedro A Santos, Ana Paiva, and Francisco S Melo. Centralized training with hybrid execution in multi-agent reinforcement learning, arxiv preprint (rXi::221000622 2022.
Samuel Schmidgall, Yusheng Su, Ze Wang, Ximeng Sun, Jialian Wu, Xiaodong Yu, Jiang Liu, Zicheng Liu, and Emad Barsoum. Agent laboratory: Using 1lm agents as research assistants. ai preprinl arXiv:2501. 0ate25000.,227,, 2025.
John Schulman, Sergey Levine, Philipp Moritz, Michael I. Jordan, and Pieter Abbeel. Trust region policy optimization, 2015. URL htttps://arra //arxiv///xvvii//// wtw//rr///////oooo http:///ai.....r//ass/55/0.2..
05477
John R Searle. Speech acts: An essay in the philosophy of language. Cambridge university press, 1969.
137

---

Published in Transactions on Machine Learning Research (02/2026)
Samuel Shaw, Emerson Wenzel, Alexis Walker, and Guillaume Sartoretti. Formic: Foraging via multiagent rl with implicit communicttinn. IEEE Robotics and Automation Letters, 7(2):4877-4888 2022.
Junjie Sheng, Xiangfeng Wang, Bo Jin, Junchi Yan, Wenhao Li, Tsung-Hui Chang, Jun Wang, and Hongyuan Zha. Learning structured communication for multi-agent reinforcement learning. Autonomous Agents and Multi-Agent Systems, 36(2):1-31, 2022.
Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Lan- guage agents with verbal reinforcement learning. Adoances in Neural Information Processing Systems, 36, 2024.
Alonso Silva. Large language models playing mixed strategy nash equilibrium games. In International Conference on Network Games, Artificial Intelligence, Control and Optimization, pp. 142-152. Springer, 2024.
David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, et al. Mastering chess and shogi by self-play with a general reinforcement learning algorithm. arXiv preprint 17Xi:1712.0110 2017.
David Simöes, Nuno Lau, and Luis Paulo Reis. Multi-agent actor contralized-crite with communication. Newrocomputing, 390:40-56, 2020.
Amanpreet Singh, Tushar Jain, and Sainbayar Sukhbaatar. Learning when to communicate at scale in multiagent cooperative and competitive tasks, 2018.
Brian Skyrms. Signals, evolution and the explanatory power of transient information. Philosophy of Science, 6973):407-428, 2002.
Oliver Slumbers, David Henry Mguni, Kun Shao, and Jun Wang. Leveraging large language models for optimised coordination in textual multi-agent reinforcement learning. 2023.
Kenny Smith, Simon Kirby, and Henry Brighton. Iterated learning: A framework for the emergence of language. Artificial life, 9(4):371-33 3763366,, 2003.
Luc Steels. A sell-organizing spatial vocabulary. Artificial life, 2(3):319 332, 1995.
Luc Steels. The synthetic modeling of language origins. Evolution of communication, 1(1):1-34 1997.
Shane Thrneerr-Treleeld, Xuhui Zhou, Zeyu Liu, and CM Downey. Emergent communication fine-tuning (ec-ft) for pretrained language models. In Emergent Commanication Workshop at. ICLR 2022, 2022.
Simon Stepputtis, Joseph Campbell, Yaqi Xie, Zhengyang Qi, Wenxin Sharon Zhang, Ruiyi Wang, Sanketh Rangreji, Michael Lewis, and Katia Sycara. Long-horizon dialogue understanding for role identification in the game of avalon with large language models. arXiv preprint orXiw:2311.088 1.05220, 2023.
Alessandro Suglia, Ioannis Konstas, and Oliver Lemon. Visually grounded language learning: a review of language games, datasets, tasks, and models. Journal of Artificial Intelligence Research, 79:173-239 2024.
Sainbayar Sukhbaatar, Rob Fergus, al. Learning multiagent communication with backpropagation. Ad- vances in neural information processing systems, 29, 2016.
Chuamneng Sun, Songjun Huang, and Dario Pompili. Llm-based multi-agent reinforcement learning: Current and future directions. arXiv preprint Xrv:::2405,111 11e.111000 11166, 2024a.
Haochen Sun, Shuwen Zhang, Lujie Niu, Lei Ren, Hao Xu, Hao Fu, Fangkun Zhao, Caixia Yuan, and Xiaojie Wang. Collab-overcooked: Benchmarking and evaluating large language models collaborative agents. arXiv preprint 2rXii:2502.20073 2025a.
Jingyun Sun, Chengxiao Dai, Zhongze Luo, Yangbo Chang, and Yang Li. Lawluo: A chinese law firm co-rum by Ilm agents. Xiv preprint ar arXiv:2407.1000 1 16252, 2024b.
138

---

Published in Transactions on Machine Learning Research (02/2026)
Lijun Sun, Yijum Yang. Qiqi Duan, Yuhui Shi, Chao Lyu, Yu-Cheng Chang, Chin-Tere Teng Lin, and Yang Shen. Multi-agent coordination across diverse applications: A survey, arXiv preprint arXim:2002.147/3 2025b.
Lijun Sun, Yijun Yang. Qiqi Duan, Yuhui Shi, Chao Lyu, Yu-Cheng Chang, Chin-Teng Teng Lin, and Yang Shen. Multi-agent coordination across diverse applications: A survey. arXiv preprint arXio:2502. 14748, 2025c.
Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018
Agnieszka Slowik, Abhinav Gupta, William L. Hamilton, Mateja Jamnik, Sean B. Holden, and Christopher Pal. Structural inductive biases in emergent communccation, 2021. URL httts:// org//b////0//
01335
Victor Talpaort, Ibrahim Sobh, Ravi Kiran, Patrick Mannion, Senthil Yogamani, Ahmad El-Sallab, and Patrick Perez. Exploring applications of deep reinforcement learning for real-world autonomous driving systems. Xin preprinl. 190::1901.11300 2019.
Weihao Tan, Wentao Zhang, Shanqi Liu, Longlao Zheng, Xinrun Wang, and Bo An. True knowledge comes from practice: Aligning large language models with embodied environments via reinforcement learning. In ICLR, 2024.
Xiangru Tang, Anni Zou, Zhuosheng Zhang, Ziming Li, Yilun Zhao, Xingyao Zhang, Arman Cohan, and Mark Gerstein. Medagents: Large language models as collaborators for zero-shot medical reasoning, arxiv preprint ar Xrx::22311. 105377 2023.
Alfred Tarski. The concept of truth in formalized languages. 1956.
Sekhar Tatikonda and Sanjoy Mitter. Control under comminccttin constraints. IEEE Transactions on automotic control, 49(7):1056-1000 2004.
Zheng Tian, Shihao Zou, lan Davies, Tim Warr, Lisheng Wu, Haitham Bou Ammar, and Jun Wang. Learning communicate implicitly by actions. Tn Proceedings of the AAAT conference on artificial intelligence, volume 34, pp. 7261 7268, 2020.
Michael Tomasello. The cultural origins of human cognition. Harvard university press, 2009.
Khanh-Tung Tran, Dung Dao, Minh-Duong Nguyen, Quoc-Viee Viet Pham, Barry O'Sullivan, and Hoang D Nguyen. Multi-agent collaboration mechanisms: A survey of 1lms. arXiv preprint arXiv:2501.0620 0662222....... 2025a.
Khanh-Tung Tran, Dung Dao, Minh-Duong Nguyen, Quoc-Viel Pham, Barry O'Sullivan, and Hoang
Nguyen. Multi-agent collaboration mechanisms: A survey of llms. arXiv preprint arXiv:2501.06:: 2025b.
Stavros Tripakis. Decentralized control of discrete-event systems with bounded or unbounded delay commu- nication. IEEE transactions on automatic control, 4989011449-1501, 2004.
Patara Trirat, Wonyong Jeong, and Sung Ju Hwang. Automl-agent: A multi-agent 1lm framework for full-pipeline automl. ar preprint Xiv: 10. 029529, 2024.
Anastasios Tsiamis, Christos K Verginis, Charalampos P Bechlioulis, and Kostas J Kyriakopoulos. Cooper- ative manipulation exploiting only implicit communication. In 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 864-869. IEFE, 2015.
Mycal Tucker, Roger P Levy, Julie Shah, and Noga Zaslavsky. Generalization and translatability in emer- gent communication via informational constraints. In NeurIPS 2022 Workshop Information-Then CiimattionThhorettii Theoretic Principles in Cognétive Systems.
Mycal Tucker, Huso Li, Siddharth Agrawal, Dana Hughes, Katia Sycara, Michael Lewis, and Julie A Shah. Emergent discrete communication in semantic spaces. Advances in Neural Information Processing Systems, 34:10574-10555 2021.
139

---

Published in Transactions on Machine Learning Research (02/2026)
Ryo Ueda, Taiga Ishii, Koki Washio, and Yusuke Miyao. Categorial grammar induction as compositionality measure for emergent languages in signaling games. 2022.
Paul Van Eecke and Katrien Beuls. Re-conceptualising the language game paradigm in the framework of multi-agent reinforcement learning. arXiv preprinl. arXiv:2004. 04722, 2020.
Hado van Hasselt, Arthur Guez, and David Silver. Deep reinforcement learning with double q-learning, 2015. http:/aass111.00000
URL
Astrid Vanneste, Simon Vanneste, Kevin Mets, Tom De Schepper, Siegfried Mercelis, Steven Latré, and Peter Hellinckx. An analysis of discretization methods for communication learning with multi-agent rein- forcement learning. arXiv preprint arXiv:2244.06665 2022a.
Simon Vanneste, Astrid Vanneste, Kevin Mets, Tom De Schepper, Ali Anwar, Siegfried Mercelis, Steven Latré, and Peter Hellinckx. Learning to communicate using counterfactual reasoning. In Adaptive and Learning Agents Workshop (ALA), collocated with AAMAS, 11-13 May, 2022, Auckland, New Zealand, 2022b.
Kyle Wagner, James A Reggia, Juan Uriagereka, and Gerald S Wilkinson. Progress in the simulation of emergent communication and language. Adaptive Behavior, 11(1):37 69, 2003.
Chao Wang, Jian Wang, Yuan Shen, and Xudong Zhang. Autonomous navigation of uavs in largo-scalc com- plex environments: A deep reinforcement learning approach. IEEE Transactions on Vehicular Technology, 68(3):2124 2136, 2019a.
Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. arXio preprint arXiv:2305.1828 16291, 2023a.
Jianhao Wang, Zhizhou Ren, Terry Liu, Yang Yu, and Chongjie Zhang, Qplex: Duplex dueling multi-agent q-lcarning. aiXiv preprint orXiw:2088.c 01062, 2020a.
Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et al. A survey large language model based autonomous agents. Frontiers of Computer Science, 18(6):186345, 2024a.
Qian Wang, Tianyu Wang, Qinbin Li, Jingsheng Liang, and Bingsheng He. Megaagent: A practical frame- work for autonomous cooperation in large-scale Ilm agent systems. ar preprint 0995555 2024b.
Qineng Wang, Zihao Wang, Ying Su, Hanghang Tong, and Yangqiu Song. Rethinking the bounds of Ilm reasoning: Are multi-agent discussions the key? arXiv preprint 18272 2024c.
Rundong Wang, Xu He, Runsheng Yu, Wei Qiu, Bo An, and Zinovi Rabinovich. Learning efficient multi- agent communication: An information bottleneck approach. In International Conference on Machine Learning, pp. 9908 9908-9918. PMLR, 2020b.
Shenzhi Wang, Chang Liu, Zilong Zheng, Siyuan Qi, Shuo Chen, Qisen Yang, Andrew Zhao, Chaofei Wang, Shiji Song, and Gao Huang. Avalon's game of thoughts: Battle against deception through recursive contemplation. arXiv preprint arXio:2310. 01320, 2023b.
Shiyong Wang, Jiafu Wan, Daqiang Zhang, Di Li, and Chunhua Zhang. Towards smart factory for indus- try 4.0: a self-organized multi-agent system with big data based feedback and coordination. Computer networks, 101:158-168, 168, 2016.
Tixian Wang, Amirhossein Taghvaei, and Prashant G Mehta. Q-learning for pomdp: An application to learning locomotion gaits. In 2019 IEEE 58th Conference on Decision and Control (CDC), pp. 2758- 2763. IEFF, 2019b.
Tonghan Wang, Jianhao Wang, Chongyi Zheng, and Chongjie Zhang. Learning nearly decomposable value functions via communication minimization. Xiv preprint aixi 1910. 1910.00666,, 2019c.
140

---

Published in Transactions on Machine Learning Research (02/2026)
Wei Wang, Dan Zhang, Tao Feng, Boyan Wang, and Jie Tang. Battleagentbench: A benchmark for evalu- ating cooperation and competition capabilities of language models in multi-agent systems. arXiv preprint arXiv:2408.0 15971, 2024d.
Yuqi Wang, Yu-Yao Zhang, Cheng-Lin Liu, Tieniu Tan, and Zhaoxiang Zhang. Emergence of machine language: towards symbolic intelligence with neural networks. National Science Review, 11(4):nwad317, 2024e.
Z Wang, S Mao, W Wu, T Ge, F Wei, and H Ji. Unleashing cognitive synergy in large language models: A task-solving agent through multi-persona seffcollaboration. arxiv 2023. arXiv preprint arXiv:2307.05300
Zhefan Wang, Yuanqing Yu, Wendi Zheng, Weizhi Ma, and Min Zhang, Macrec: multi-agent collaboration framework for recommendation. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retriccal, pp. 2760-2764 2024f.
Zhenhailong Wang, Shaoguang Mao, Wenshan Wu, Tao Ge, Furu Wei, and Heng Ji. Unleashing the emergent cognitive synergy large language models: A task-solving agent through multi-persona sclf-collaboration. arXiv preprinl Xrv::2337. 05300, 2023c.
Paul Watzlawick, Janct Beavin Bavclas, and Don D Jackson. Pragmatics of human communication: study of interactional pallerns, pathologies and parudores. WW Norton & Company, 2011.
Hans S Witsenhausen. A counterexample in stochastic optimum control. SIAM Journal on Control, 6(1): 131-147, 1968.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, pp. 38 38-45, 2020.
Cornelius Wolff, Julius Mayer, Elia Bruni, and Xenia Ohmer. Bidirectional emergent language in situated environments. arxiv preprint Xix::2448 14649, 2024.
Annie Wong, Thomas Bäck, Anna V Kononova, and Aske Plaat. Deep multiagent reinforcement learning: Challenges and directions. Artificial Intelligence Review, 56(():5003-500) 5056, 2023.
William Woods. Semantics and quantification in natural language question answering. In Advances in computers, volume 17, pp. 1-87. Elsevier, 1978.
Dekun Wu, Haochen Shi, Zhiyuan Sun, and Bang Liu. Deciphering digital detectives: Understanding Ilm behaviors and capabilities in multi-agent mystery games. arXiv preprint arXin:2312.0074 2023a.
Di Wu, Xian Wei, Guang Chen, Hao Shen, Xiangfeng Wang, Wenhao Li, and Bo Jin. Generative multi-agent collaboration in embodied ai: A systematic review. arXiv preprint arXi::2502. 115188 2025.
Minghao Wu, Jiahao Xu, Yulin Yuan, Gholamreza Haffari, Longyue Wang, Weihua Luo, and Kaifu Zhang, (perhaps) beyond human translation: Harnessing multi-agent collaboration for translating ultra-long lit- erary lexts. Xix preprint ar arXiv:2405. 11804, 2024a.
Qingyun Wu, Gagan Bansal, Jicyu Zhang, Yiran Wu, Shaokun Zhang, Erkang Zhu, Beibin Li, Li Jiang, Xiaoyun Zhang, and Chi Wang. Autogen: Enabling next-gen Ilm applications via multi-agent conversation framework. anXiv preprint arXiv:2308.00 08155, 2023b.
Tongshuang WU, Michael Terry, and Carrie Jun Cai. Ai chains: Transparent and controllable human-ai interaction by chaining large language model prompts. In Proceedings of the 2022 CHil conference on human factors in computing systems, pp. 1-22, 2022.
Yue Wu, So Yeon Min, Shrimai Prabhumoye, Yonatan Bisk, Russ R Salakhutdinov, Amos Azaria, Tom Mitchell, and Yuanzhi Li. Spring: Studying papers and reasoning to play games. Advances in Neural Information Processing Systems, 36, 2024b.
141

---

Published in Transactions on Machine Learning Research (02/2026)
Jiannan Xiang, Tianhua Tao, Yi Gu, Tianmin Shu, Zirui Wang, Zichao Yang, and Zhiting Hu. Language models meet world models: Embodied experiences enhance language models. Advances in neural infor- mation processing systems, 36:75392-7544 -54122 2023.
Ziyang Xiao, Dongxiang Zhang, Yangjun Wu, Lilin Xu, Yuan Jessica Wang, Xiongwei Han, Xiaojin Fu, Tao Zhong, Jia Zeng, Mingli Song, et al. Chain-of-experts: When 1mms meet complex operations research problems. In The twelfth international conference on learning representations, 2023.
Tianbao Xie, Fan Zhou, Zhoujun Cheng, Peng Shi, Luoxuan Weng, Yitao Liu, Toh Jing Hua, Junning Zhao, Qian Liu, Che Liu, et al. Openagents: An open platform for language agents the wild. arXiv preprint arXiv:2310.1000 10634, 2023.
Kai Xiong, Xiao Ding, Yixin Cao, Ting Liu, and Bing Qin. Examining inter-consistency of large language models collaboration: An in-depth analysis via debate. arXiv preprint arXiv:2305 15955, 2023.
Binfeng Xu, Xukun Liu, Hua Shen, Zeyu Han, Yuhan Li, Murong Yuc, Zhiyuan Pong, Yuchen Liu, Ziyu Yao, and Dongkuan Xu. Gentopia: collaborative platform for tool-augmented llms. orXiv preprinl arXiv:2308..m 200.000000 2023a.
Yuzhuang Xu, Shuo Wang, Peng Li, Fuwen Luo, Xiaolong Wang, Weidong Liu, and Yang Liu. Explor- ing large language models for communication games:: An empirical study on werewolf. orXiv preprint arXiv:2309. D.Xiv:2309.04658, 2023b.
Zelai Xu, Chao Yu, Fei Fang, Yu Wang, and Yi Wu. Language agents with reinforcement learning for strategic play in the werewolf game. arXiv preprint arXiv:2310.1800 2023c.
Hanqing Yang, Jingdi Chen, Marie Siew, Tania Lorido-Botran, and Carlee Joe- Noe-Wong Jlm-powered decen- tralized generative agents with adaptive hicrarchical knowledge graph for cooperative planning. arXiv preprint 2xXf:25502.05555 2025.
Shunyu Yao, Mo Yu, Yang Zhang, Karthik R Narasimhan, Joshua B Tencnbaum, and Chuang Gan. Linking emergent and natural languages via corpus transfer. In International Conference on Learning Represen- lations.
Shunyu Yao, Mo Yu, Yang Zhang, Karthik R Narasimhan, Joshua B Tenenbaum, and Chuang Gan. Linking emergent and natural languages via corpus transfer. arXiv preprint arXiv:2208.1332 2022.
Xie Yi, Zhanke Zhou, Chentao Cao, Qiyu Niu, Tongliang Liu, and Bo Han. From debate to equilibrium: Belief-driven multi-agent Ilm reasoning via bayesian nash equilibrium. arXiv preprint arXiv:2506. 08292, 2025.
Zihao Yi, Jiarui Ouyang, Yuwen Liu, Tianhao Liao, Zhe Xu, and Ying Shen. A survey on recent advances in Ilmed multi-turn dialogue systems. arXiv preprint arxiv:2402 18013, 2024.
Zhangyue Yin, Qiushi Sun, Cheng Chang, Qipeng Guo, Junqi Dai, Xuanjing Huang, and Xipeng Qiu. Exchange-of-thonght: Enhancing large language model capabilities through cross-model communication. arXiv preprint ar Xii::3312.01823 2023.
Chao Yu, Akash Velu, Eugene Vinitsky, Jiaxnan Gao, Yu Wang, Alexandre Bayen, and Yi Wu. The surprising effectiveness of ppo in cooperative multi-agent games. Advances in Neural Information Processing Systems, 35:24611-24624 2022a.
Dhara Yu, Jesse Mu, and Noah Goodman. Emergent covert signaling in adversarial reference games. In Emergent Communication Workshop at ICLR 2022, 2022b.
Xianhao Yu, Jiaqi Fu, Renjia Deng, and Wenjuan Han. Mineland: Simulating large-scale multi-agent inter- actions with limited multimodal senses and physical needs. arXiv preprint arXiv:2403. 19267, 2024.
Won Joon Yun, Byungju Lim, Soyi Jung, Young- Chai Ko, Jihong Park, Joongheon Kim, and Mehdi Ben- nis. Attention-basee based reinforcement learning for real-time uav semantic communication. In 2021 17th International Symposium Wireless Commmnication Systems (ISWCS), pp. 1 1-6. TEEE, 2021.
142

---

Published in Transactions on Machine Learning Research (02/2026)
Mohamed Salah Zaiem and Etienne Bennequin. Learning to communicate in multi-agent reinforcement learning: A review. arXiv preprint arXiv:1111.05438 2019.
Ceyao Zhang, Kaijie Yang, Siyi Hu, Zihao Wang, Guanghe Li, Yihang Sun, Cheng Zhang, Zhaowei Zhang, Anji Liu, Song-Chun Zhu, et al. Proagent: Building proactive cooperative with large language models. CoRR, 2023a.
Ceyao Zhang, Kaijie Yang, Siyi Hu, Zihao Wang, Guanghe Li, Yihang Sun, Cheng Zhang, Zhaowei Zhang, Anji Liu, Song-Chun Zhu, et al. Proagent: building proactive cooperalive agents with large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 2024a.
Fangyi Zhang, Jürgen Leitner, Michael Milford, Ben Uperoft, and Peter Corke. Towards vision-based deep reinforcement learning for robotic motion control, 2015. URL hattp::://arsiv.orr//bbbb///
Hongxin Zhang, Weihua Du, Jiaming Shan, Qinhong Zhou, Yilun Du, Joshua B Tenenbaum, Tianmin Shu, and Chuang Gan. Building cooperative embodied agents modularly with large language models. arXiv preprint ar Xiv:2200 02455, 2023b.
Jintian Zhang, Xin Xu, Ningyu Zhang, Ruibo Liu, Bryan Hooi, and Shumin Deng. Exploring collaboration mechanisms for 1lm agents: A social psychology view. arxiv preprinl arXiv:231000212,, 2023c.
Junjie Zhang, Yupeng Hou, Ruobing Xie, Wenqi Sun, Julian McAuley, Wayne Xin Zhao, Leyu Lin, and Ji- Rong Wen. Agentef: Collaborative learning with autonomous language agents for recommender systems. In Proceedings of the ACM Web Conference 2024, pp- 3679-3689, 2024b.
Sai Qian Zhang, Qi Zhang, and Jieyu Lin. Efficient communication in multi-agent reinforcement learning via variance based control. Advances in neural information processing systems, 32, 2019.
Sai Qian Zhang, Qi Zhang, and Jieyu Lin. Suceinct and robust multi-agent communication with temporal message control. Advances in neural information processing systems, 33:17271-17282, 2020.
Yusen Zhang, Ruoxi Sun, Yanfei Chen, Tomas Pfister, Rui Zhang, and Sercan Arik. Chain of agents: Large language models collaborating on long-context tasks. Advances in Neural Information Processing Systems, 17:132208-132237, 2024c.
Yi Zheng, Chongyang Ma, Kanle Shi, and Haibin Huang, Agents meet okr: An object and key results driven agent system with hierarchical solf-collaboration and solf-evaluaton. arXiv preprint PxXi:2311.16544 2023.
Changxi Zhu, Mehdi Dastani, and Shihan Wang. A survey of multi-agent reinforcement learning with communication. Xixi preprint arXi::2203.080 08975, 2022.
Changxi Zhu, Mehdi Dastani, and Shihan Wang. A survey of multi-agent deep reinforcement learning with communication. Autonomous Agents and Multi-Agent Systems, 38(1):4, 2024.
Kunlun Zhu, Hongyi Du, Zhaochen Hong, Xiaocheng Yang, Shuyi Guo, Zhe Wang, Zhenhailong Wang, Cheng Qian, Xiangru Tang, Heng Ji, et al. Multiagentbench: Evaluating the collaboration and competition of 1l agents. arXiv preprint
(tv:2509.01993 2025.
Quanyan Zhu. Reasoning and behavioral equilibria in Ilm-nash games: From mindsets to actions, 2025. URL
https://arxiv. www.orr......o.oo..e abs/ /0abs///55//2 2507 08208
Xizhou Zhu, Yuntao Chen, Hao Tian, Chenxin Tao, Weijie Su, Chenyu Yang, Gao Huang, Bin Li, Lewei Lu, Xiaogang Wang, et al. Ghost in the minccraft: Generally capable agents for open-world environments via large language models with text-based knowledge and memory. ar preprind ar arXin:2305.1 17144, 2023.
Mingchen Zhuge, Wenyi Wang, Louis Kirsch, Francesco Faccio, Dmitrii Khizbullin, and Jürgen Schmidhuber. Gptswarm: Language agents as optimizable graphs. In Forty-first International Conference on Machine Learning, 2024a.
Mingchen Zhuge, Wenyi Wang, Louis Kirsch, Francesco Faccio, Dmitrii Khizbullin, and Jürgen Schmidhuber. Language agents as optimizable graphs. Xixiv preprint 3rX1w:2402.16623, 2024b.
143


Reviews:
[{"review_id": "j790aiwXIR", "reviewer": "Reviewer_7M5j", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Summary: \n\nThe paper surveys multi-agent communication across three areas that are usually treated separately: communication in MARL, emergent communication (EL), and communication in LLM-based multi-agent systems. It presents a taxonomy inspired by “Five Ws” (who/whom, when, what, why) and uses it most systematically in the MARL section. It then reviews key methods, provides classification tables, and closes with cross-domain discussion of open challenges and future directions.\n\nContributions: \n\nThe main contribution is a unified survey that spans MARL-Comm, emergent language, and LLM-based agent communication (a combination not covered by existing surveys). The MARL portion offers a clear and well-structured taxonomy, helpful formalization, and comprehensive tables. The EL and LLM-Comm sections give accessible overviews of the literature.\n\nStrengths:\nThe MARL-Comm taxonomy (“what/whom, when, how”) is well-organized and accurate, with nice comparative tables. The paper is generally clear, detailed, and visually well-supported. It fills a gap, in my view.\n\nWeaknesses:\nThe “Five Ws” framing is not totally consistently applied in my view, especially “why,” which is not too developed as a true dimension. EL and LLM-Comm sections are not as in-depth as the MARL section and could have more focus on comparing existing works, rather than listing them. Some claims about LLM-based multi-agent systems are perhaps overly optimistic, and one communication formalization is a bit unclear (Section 6). The survey would benefit from explicit acknowledgment of LLM limitations in MARL settings and clearer connection to classical decentralized control literature.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The MARL-Comm section is accurate and technically solid; its formalization matches standard Dec-POMDP treatments and its categorization of methods is reliable. The EL and LLM-Comm sections are also factually correct but less analytic and perhaps makes optimistic claims about LLM capabilities. One communication formulation, in Section 6, is conceptually unclear due to circular dependencies (see that the message depends on the action and the action depends on the message). Overall clarity is good, the structure is consistent, and tables/figures are effective, though the “Five Ws” framing is only partially fulfilled and some repetition appears across sections.", "audience_yn": "Yes", "audience_explanation": "The paper is relevant. Communication in MARL is a core RL topic, emergent communication has become a standard tool in multi-agent RL research, and LLM-based agents increasingly interact with or guide RL components. A unified survey covering all three areas is timely and useful for researchers in and around the area.", "requested_changes": "I don't have strict requirements for changes.\n\nNice-to-haves:\n- Make the “Five Ws” framing a bit more explicit and consistent across the three main sections, especially clarifying “why communicate” as its own analytical axis;\n- Temper and contextualize claims involving LLM-based multi-agent systems, ensuring that limitations and failure modes are explicitly acknowledged;\n- Clarify the generic communication formalization in Section 6 to remove circular action–message dependence or explain its intended abstraction;\n- Add explicit links (discussion and references) to classical communication-constrained control and decentralized information theory.\n\nPlease also consider the following questions, whose answers could also appear in text:\n- What criteria guided which papers were included or excluded across the three domains?\n- How many total works were surveyed versus discussed in detail, and how were these proportions determined?\n- How do you envision integration between RL-based communication approaches and LLM-based multi-agent systems?\n- Can you articulate different categories of “why communicate,” and how each motivation affects communication design?\n\nA few typos, I believe:\n- “We begin…” mid-sentence;\n- “MARL-COMM” vs “MARL-Comm”;\n- Confusing phrase “agents’ numbers settings”;\n- “fixed set” instead of “fix set” and \"are\" instead of \"is\" in “communication symbols is.”.", "broader_impact_concerns": "Nothing to declare.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "P9F09CaygL", "reviewer": "Reviewer_CeNu", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents \"The Five Ws of Multi-Agent Communication: Who Talks to Whom, When, What, and Why,\" a comprehensive survey spanning the evolution from traditional Multi-Agent Reinforcement Learning (MARL) communication to emergent language and LLM-based multi-agent systems. The work is commendable in several respects: it provides extensive coverage of the multi-agent communication landscape, organizes a vast body of literature through the intuitive lens of the \"Five Ws\" framework, and distinguishes itself from existing surveys by bridging three distinct research paradigms—MARL-Comm, emergent language, and LLM-based multi-agent systems—under a unified conceptual umbrella.\nThe paper's taxonomies are thorough and well-structured, particularly in Section 5, which provides a detailed categorization of LLM-based multi-agent systems across communication topology, motivational settings, agent configurations, and communication structures. The extensive reference compilation (spanning hundreds of papers) demonstrates the authors' diligent effort to capture the field's breadth.  \n**However, I have several substantive concerns that affect the paper's coherence and depth:**\n1. Disjointed Integration Between MARL and LLM Sections: The transition from traditional MARL communication (Section 3) and emergent language (Section 4) to LLM-based approaches (Section 5) feels abrupt and insufficiently motivated. While the paper acknowledges these as distinct research paradigms, it lacks a compelling narrative arc that explains why and how the field evolved from one to another. The sections read more like parallel surveys concatenated together rather than chapters in a unified story. A stronger conceptual bridge is needed—perhaps discussing what limitations in MARL-Comm motivated emergent language research, and what gaps in both led to the adoption of LLMs.\n2. Insufficient Depth in Analysis: While the paper excels at categorization and enumeration, it falls short in providing critical analysis and insights. This is particularly evident in Section 5 on LLM-based multi-agent systems. For example: The communication topology subsection (5.2.1) lists five topologies (chain, decentralized, centralized, nested/hierarchical, and dynamic) with corresponding citations, but provides minimal comparative analysis of their trade-offs, applicability, or empirical performance.\nThe motivational settings subsection (5.2.2) categorizes systems as cooperative, competitive, or mixed-motive, but doesn't offer deep insights into how these different motivations fundamentally change communication strategies or what design principles emerge.\nThroughout Section 5, there is heavy reliance on citation stacking (e.g., listing 20+ papers for a single category) without synthesizing their key findings or identifying trends.\n\n3. Inadequate Treatment of Communication in LLM+RL Integration: Section 5.5, which discusses the integration of LLMs with MARL, is disappointingly brief given its centrality to the paper's thesis of bridging paradigms. More critically, this section largely neglects the communication mechanisms between agents in these hybrid systems. The examples provided (Cicero, Werewolf, FAMA, ACC-Collab) are described primarily in terms of their task domains rather than their communication architectures. Given that the paper's central organizing principle is the \"Five Ws of communication,\" the lack of systematic analysis of how agents communicate in these LLM+RL systems represents a significant gap. Questions like \"Do these systems use natural language, learned protocols, or hybrid approaches?\" and \"How do communication patterns differ from pure MARL or pure LLM systems?\" remain largely unaddressed.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The paper demonstrates strong scholarly rigor in its evidence and citations:\n\n1. The references are comprehensive and appear accurate, with proper attribution to foundational and recent works across all three research areas.  \n\n2. The categorizations are well-grounded in the literature. For instance, the taxonomy of LLM-based multi-agent systems in Section 5.2 effectively captures the diversity of approaches, with each category supported by numerous representative works.  \n\n3. The technical background sections (3.1 on MARL-Comm, 4.1 on emergent language, 5.1 on LLMs) provide accurate preliminaries that appropriately contextualize subsequent discussions.  \n\n4.The paper's tables (e.g., Table 10 on LLM Agent Communication Taxonomy, Table 11 on evaluation benchmarks) are informative and backed by proper citations.", "audience_yn": "Yes", "audience_explanation": "This survey addresses a timely and important topic that intersects multiple active research communities. The paper would be valuable to several audience segments:\n\n1. MARL researchers seeking to understand how communication protocols have evolved and how modern LLM-based approaches compare to traditional methods.  \n\n2. LLM researchers interested in multi-agent applications and wanting comprehensive context on how agents coordinate and communicate.  \n\nHowever, the paper's impact would be significantly strengthened by addressing the coherence issue mentioned earlier. Currently, readers from different subfields might find their respective sections useful while struggling to see the connections to other sections. A more integrated narrative would make the survey valuable not just as a reference collection, but as a conceptual contribution that helps researchers understand the field's evolution and identify future research directions at the intersections of these paradigms.", "requested_changes": "I recommend the following revisions to strengthen the paper:\n1. Add Bridging Content Between Major Sections. \nInsert a subsection at the end of Section 3 (MARL-Comm) that explicitly discusses the limitations that motivated emergent language research. What problems couldn't be solved with traditional MARL communication? Why did researchers turn to learning communication protocols end-to-end?\nSimilarly, add content at the end of Section 4 explaining why LLMs became attractive for multi-agent systems. What capabilities do LLMs provide that emergent language systems lack?\nConsider adding a high-level \"Evolution of Multi-Agent Communication\" subsection in the introduction that previews this narrative arc.\n\n2. Substantially Expand Section 5.5 on LLM+RL Integration. \nThis section is crucial for demonstrating how the field is synthesizing insights from both paradigms, yet it currently occupies only ~2 pages. Provide systematic analysis of communication mechanisms in hybrid systems: Do agents use natural language end-to-end? Are there learned message-passing layers beneath language? How do these systems handle the trade-off between language interpretability and communication efficiency? Discuss how the \"Five Ws\" apply differently in hybrid systems compared to pure MARL or pure LLM approaches. Include empirical comparisons where available (e.g., how does communication in FAMA compare to traditional MARL-Comm approaches in similar domains?).", "broader_impact_concerns": "No concerns. The paper is a survey that primarily organizes and synthesizes existing research. It does not introduce new methods or systems that would raise ethical concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ayypbMD5SJ", "reviewer": "Reviewer_ye4q", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors present a survey in the field of Multi-Agent Reinforcement Learning (MARL) with a particular focus on multi-agent communication (MA-Comm) and emergent language (MA-EL). Within the work the authors consider different aspects of agent communication under varying communication costs, agent motifs, goal alignments and communication constraints.\n\nThe first part of the survey is concerned with 'what', 'when' and 'how', comprising  discussion on limitations on the scalability of communication channels, centralized/decentralized communication, as well as the more general aspect of communication towards knowledge acquisition. A key focus of the presented survey is the consideration of emergent language within learning systems. The authors simultaneously focus on emergent language protocols in unregularized settings, as well as the alignment and leverage of communication through human-like natural language. For the former, aspects on the emergence of (symbolic) message sequences under difference scenarios are discussed. For the latter, use-cases of the application of large language models are comprehensively studied. Finally, the authors provide a discussion on open problems in current literature involving theoretical guarantees, benchmarks and challenges in human-computer interactions.\n\n\n\n**Strengths**\n\nThe individual sections consistently build on top of each other, starting with MARL and transitioning towards emergent- and natural language aspects. The paper is clearly written and the discussed concepts are presented in an easy to understand way. The chosen structure is intuitive and follows a common thread. Individual sections are accompanied by nicely designed figures which help communicating the discussed concepts on an intuitive level and are easy to grasp.\n\nWhile I do not posses a comprehensive overview of the field, the survey seems to accurately represent common concepts and ideas in MARL and agent communication. The authors seem to comprehensively cite and categorize recent developments in papers and methods. The work furthermore refers to existing surveys contrasting it with the aspects of emergent language and communication discussed in the paper.\n\n\n\n**Weaknesses**\n\n**Foundations of Agent Communication.** Given the long-standing line of work on multi-agent planning reaching back to the 1980s, it is striking that most cited works in the survey falls into the time-frame of 2010 onward. While the area undoubtedly gained traction under the past developments in general deep learning and LLMs, giving rise to a new understanding and application areas for agents, the authors ignore the foundations of the field. I would like to recommend adding a small discussion on the foundations of the the field. While not being well versed in the foundations of the field myself, I would kindly refer to, for example, Russel and Norvig \"Artificial Intelligence: A Modern Approach\", which contains a brief introduction on the matter, citing initial references.\n\n\n\n**Alignment with Title.** Although the aspects of 'Who', 'Whom' and 'Why' are implicitly covered within the other sections, the survey lacks an explicit account of those. Given that the particular question of 'Why' fundamentally justifies the need for communication in the first place, I would like to recommend (-also in order to meet expectations of the title. E.g., the word 'Why' only appears 5 times in the paper, 3 of those being the title, abstract and a referenced paper-), to give a more explicit discussion on the missing aspects in separate sections.\n\n\n\n**Broader Discussion.** The survey focuses strongly on the technical realizations of agents, but does not touch on the more general philosophical/mathematical considerations of agent communication. Considerations communication might be substantially shaped due to reasoning about an other agents' beliefs and their state of mind. These aspects are commonly considered a key skill for generally 'intelligent' actors, e.g. in cognitive science or general AI literature. Beyond specific realizations of agents, the authors might therefore want to discuss more general frameworks, such as Doxastic logic or more Kripke semantics that allow the formal modeling of such reasoning, and providing reasons for 'When', 'Why' and 'to Whom' to communicate.\n\n\n\n**Technical Detail.** Contrary to the previous point, the survey provides rather high-level descriptions on many mentioned concepts. Except for sections 3 and 6, concepts are primarily expressed in natural language, which might somewhat reduce the information that can be drawn directly from the paper. E.g., section 5.2.4 \"Structure of Communication\" could be complemented by a brief formalization of communication channels. Similarly, work concerned with mixed-motives or competitive settings (sec. 5.2.2 \"Motivational Settings\") is often concerned with Nash equilibria. While the aim of a survey is not to present excessively detailed formalism, I would at least like to discuss this topic with the authors.\n\n\n\n**Minor: Citations.** Given that the type of the presented work is a survey, the authors might want to put a stronger emphasis on providing up-to-date references. Upon checking some of the papers on the first page of references, I found several works cited as their arxiv versions, which where already published at conferences:\n\n* Agarwal, Akshat, et al. \"Community Regularization of Visually-Grounded Dialog.\" *Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems*. 2019.\n* Agarwal, Akshat, et al. \"Learning Transferable Cooperative Behavior in Multi-Agent Teams.\" *Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems*. 2020.\n* Agashe, Saaket, et al.  \"LLM-Coordination: Evaluating and Analyzing Multi-agent Coordination  Abilities in Large Language Models.\" *Findings of the Association for Computational Linguistics: NAACL 2025*. 2025.\n* Andreas, Jacob, et al. \"Measuring Compositionality in Representation Learning.\" *International Conference on Learning Representations*. Vol. 375. Association for Computational Linguistics, 2019.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The paper being a survey paper, no novel technical insights or claims are made. The paper comprehensively cites recent works. The perspective presented in the survey aligns with the common understanding of ml agent systems, covering recent developments in a well curated fashion.", "audience_yn": "Yes", "audience_explanation": "Autonomous agent systems are increasingly deployed in practice. Due to the rise in LLM capabilities, the field evolves rapidly as new environments can be targeted by these systems. The authors provide a well curated survey on the state of existing works that might help to navigate and categorize the ever increasing amount of papers.", "requested_changes": "The requested changes mainly concern the stated weaknesses above.\n\nGenerally, I would like to encourage the authors to more strongly consider foundational works in the field. Although, relevance might diminish as restrictions are lifted through the ability of modern systems to handle natural language, they commonly provide the theoretical and formal foundations of modern systems.\n\nSimilarly, I would like to encourage the authors to include perspectives beyond 'typical' computer science/ML view on agent systems. While these considerations rarely have an immediate impact on the field, they can provide a more general perspective of the field being studied.\n\nAs minor concerns, I would finally like to encourage the authors to, more explicitly, discuss the questions of 'Who', 'Whom' and 'Why' (possibly by implementing the previous changes) and possibly update their references.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)