G-Memory: Tracing Hierarchical Memory for Multi-Agent Systems
Authors: Guibin Zhang
Muxin Fu
Kun Wang
Guancheng Wan
Miao Yu
Shuicheng YAN
Venue: NeurIPS 2025 spotlight ()
Keywords: Multi-agent systems
agent memory
LLM agent
Primary area: deep_learning

Abstract: Large language model (LLM)-powered multi-agent systems (MAS) have demonstrated cognitive and execution capabilities that far exceed those of single LLM agents, yet their capacity for self-evolution remains hampered by underdeveloped memory architectures. Upon close inspection, we are alarmed to discover that prevailing MAS memory mechanisms (1) are overly simplistic, completely disregarding the nuanced inter-agent collaboration trajectories, and (2) lack cross-trial and agent-specific customization, in stark contrast to the expressive memory developed for single agents. To bridge this gap, we introduce G-Memory, a hierarchical, agentic memory system for MAS inspired by organizational memory theory, which manages the lengthy MAS interaction via a three-tier graph hierarchy: insight, query, and interaction graphs. Upon receiving a new user query, G-Memory performs bi-directional memory traversal to retrieve both \textit{high-level, generalizable insights} that enable the system to leverage cross-trial knowledge, and \textit{fine-grained, condensed interaction trajectories} that compactly encode prior collaboration experiences. Upon task execution, the entire hierarchy evolves by assimilating new collaborative trajectories, nurturing the progressive evolution of agent teams. Extensive experiments across five benchmarks, three LLM backbones, and three popular MAS frameworks demonstrate that G-Memory improves success rates in embodied action and accuracy in knowledge QA by up to $20.89\\%$ and $10.12\\%$, respectively, without any modifications to the original frameworks.

G-Memory: Tracing Hierarchical Memory for Multi-Agent Systems
Guibin Zhang" Muxin Fu Kun WangSt, Guancheng Wan', Miao Yu,, Shuicheng Yan
NUS, Tongji University, SNTU, 'WHU, *
Equal Contribution, Corresponding author
wang kun@ntu. edu.sg. yansc0comp nus edu.sg sg
Abstract
Large language model (LLM))powert)) multi-agent systems (MAS) have demon- strated cognitive and execution capabilities that far exceed those of single LLM agents, yet their capacity for self-evolution remains hampered by underdevel- oped memory architectures. Upon close inspection, we are alarmed to discover that prevailing MAS memory mechanisms (1) overly simplistic, completely disregarding the nuanced inter-agent collaboration trajectories, and (2) lack cross- trial and agent-specific customization, in stark contrast to the expressive memory developed for single agents. To bridge this gap, we introduce G-Memory, a erarchical, agentic memory system for MAS inspired by organizational memory theory [1], which manages the lengthy MAS interaction via a three-tier graph hierarchy: insight, query, and interaction graphs. Upon receiving new user query, G G-Memory performs bi-directional memory traversal to retrieve both high-level, generalizable insights that enable the system leverage cross-trial knowledge, and fine- graingd condensed interaction trajectories that compactly encode prior collaboration experiences. Upon task execution, the entire hierarchy evolves by assimilating new collaborative trajectories, nurturing the progressive evolution of agent teams. Extensive experiments across five benchmarks, three LLM backbones, and three popular MAS frameworks demonstrate that G-Memory improves suecess rates in embodied action and accuracy in knowledge QA by up to 20.89% and 10.12%, respectively, without any modifications to the original frameworks. Our codes are available at httph:://// ///it ggithuu..... com/ com/bbngreekyy// (eaeorr//Gemrrr.o
Introduction
As Large Language Models (LLMs) continue redefine the frontier of artificial intelligence, L.LM- driven agents have exhibited unprecedented prowess in perception [2, 3, 4, 5]. planning [6, 7, 8]. reasoning [9, 10], and action [11, 12], which have catalyzed remarkable progress across diverse downstream domains, including code generation [13, 14], data analysis [15], embodied tasks [16] and autonomous driving [3, 17, 18]. Building upon the impressive competencies of single agents, LLM- based Multi-Agent Systems (MAS) have been demonstrated to push the boundaries of single model capacity [19, 20, 21]. Similar to collective intelligence arising from human social collaboration [22, 23, 24], MAS orchestrates multiple agents [25, 26, 27]. whether through cooperation [28, 29, 30, 31] or competition [32, 33, 34], to transcend the cognitive and specialized limitations of solitary agents.
Gell-EEolling Agents. What especially characterizes LLM agents is their self-evolving capacity, i.e., the ability continuously adapt and improve through interactions with the environment, seen in prior works where such adaptability has led to two- to three-fold quantitative improvements [35]. The central driving force behind such self-evolving nature memory mechanism of agents [36, 37], which parallels human abilities to accumulate knowledge, process past experiences, and retrieve
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

OK my
D4-41
Quercy
cuoplic in gorden Exscoror BLSAAR
Token cost on ALFWorld benchmaakk
SIPE after actioe...
States
The ordor mon
Tokom cast
(6-3ed vakerd
Lawi wape
Compleado
Qeary
mrnmmrr tho in
207
of
Statts
Erecutioe
eraner Interaction Gruph
73
77
Insight Graph
Query Graph
Figure 1: (Left) We report the token cost of several single-agent and MAS baselines on ALFWorld benchmark; (Right) The overview of G-Memory's 's three-tier hierarchical memory architecture, en- compassing the insight graph, query graph and interaction (utterance) graph.
relevant information. Previous successful memory mechanism designs, including both inside-tria memory (i.e., context retained within solving one single query) and cross-trial memory (i.e., experi- ence accumulated across multiple tasks) [38], have empowered agents to excel in diverse applications such as personalized chat [36, 39, 40], recommendation [41], embodied action [42, 16], and social simulation [19, 43, 44], enabling them to evolve into experiential learners that effectively leverage past experiences and world knowledge.
Self-Evolving MAS. However, such self-evolving capacity remains largely absent in multi-agent systems. Most existing MAS are still constrained by manually defined workflows, such as the Standard Operating Procedures (SOP) in MetaGPT [21] and ChatDev [45], rely pre-defined communication topologies in MacNet [46] and AgentPrune [30]. More recent automated MASs, such GPTSwarm [47], ADAS [48], AFlow [49], and MaAS [50] have made to automatically optimize inter-agent topologies or prompts, which, nevertheless, ultimately yield giant and cumbersome MAS architectures, lacking the agility to self-adjust with accumulated collaboration experience.
The absence of the aforementioned self-evolving capacity is, in fact, rooted in the lack of memory mechanisms specifically tailored for MAS. One may challenge this claim from two perspectives: O Do existing MASs lack memory mechanisms altogether? Not entirely. Classical MAS frameworks such as MetaGPT, ChatDev, and Exxhaage--ofTThougee designs. However, these are often limited to inside-trial memory [5]], while cross-tric trial memory, if present, remains rudimentary- tyyiccll involving the transmission of overly condensed artifacts (e.g., final solutions execution results) [21, 45, 46), and failing to enable meaningful learning from collaborative experience. @ Why not directly transfer existing single-agent memory mechanisms to MAS? Unfortunately, such transfer is far from straightforward. The inherent nature of MAS, i.e., multi-turn orchestration across multiple agents [26, 27], leads substantially longer task-solving trajectories compared to single-agent settings (up to 10x more tokens, demonstrated by Figure 1 (Left)). This poses significant challenge traditional retrieval-base memory designs [36, 37, 16], as naive feeding of the entire long-context trajectory without proper abstraction from collaborative perspective offers little benefit. Given the aforementioned challenges, natural question arises:
Memory for MAS.
[51] incorporate memory-related
How can we design a memory mechanism capable of storing, retrieving, and managing the lengthy interaction history of multi-agent systems, such that agent leams can benefit from concise and instructive experience and insights?
The Present Work: G-Memory. In response the above question, we introduce Graph-bused
A Agentic Memory Mechanism for M.-s-d Multi-Agent Systems, dubbed G-Memory, which manages the complex and lengthy interaction history of MAS through a three-tier hicrarchical graph structure:
* Insight Graph, which abstracts generalizable insights from historical experience;
* Query Graph, which encodes meta-information of task queries and their connectivity;
* Interaction Graph, which stores fine-grained textual communication logs among agents.
Figure (Right) visualizes these structures, and their formal definitions are placed in Section 3. When a new query arrives, G-Memory efficiently retrieves relevant query records by leveraging the topology of the query graph, and then traverses upward (i.e., query->insight graph) to extract associated high- level insights and downward (i.e., query- graph) to identify core interaction subgraphs that are most pertinent to the task hand, thereby mitigating information overload. Based on the

---

retrieved memory, G-Memory offers actionable guidance to the MAS, e.g. division of labor, task decomposition, and lessons from past failures. Upon the completion of task, all three levels of the memory hierarchy are updated in an agentic manner, with newly distilled insights, enriched query records, detailed MAS trajectories, and their level of detailed associations. Through this refinement, G-Memory functions as a plug-and-play module that can be seamlessly embedded into mainstream MAS frameworks, empowering evolving inter-agent collaboration and collective intelligence.
Our contributions are summarized as follows:
Bottleneck Identification. We conduct thorough review of existing multi-agent systems and identify fundamental bottleneck in their self-evolving capabilities, which largely attributed to the oversimplified memory architectures.
@ Practical Solution. We propose G-Memory, hierarchical agentic memory architecture for MAS, which models complex and prolonged inter-agent collaboration through a three-tier structure comprising insight, query, and interaction graphs.
3 Experimental Evaluation. Extensive experiments across five benchmarks show that G-Memory is (I) perrformin. improving ofte--of-thh-an MAS by up to 20.89% and 10.12% on embodied action and knowledge QA tasks, respectively; and (II) resource-friendly, maintaining comparable or even lower token usage than mainstream memory designs.
2 Related Works
Single-Agent Memory. Memory serves as primary driving force for agents to accumulate experiences and explore the world through interactions with the environment [52, 53, 54, 55]. It plays a critical role in both task-solving and social simulation LLM agents, and this work primarily focuses the former. Early research on agent memory was confined to simple inside-trial memory, mainly addressing limitations posed by the LLM context window in chatbot applications, including Memory BannkryBan/// [36]. ChatDB [39], MemoChat [40], and MemGPT [37], which typically adopt retrieval- augmented generation (RAG)- similarity-
chunk retrieval. Subsequent developments have progressed toward more cognitively inspired memory architectures, including (1) memory scope extended to cross-trial memory like Expel. [42] and Synapse [56]; (2) application domains broadened to include computer control [56], embodied action [57], scientific discovery [58], coding and reasoning [59]; and (3) management techniques evolved from coarse- coarse-grained textual similarity toward more sophisticated abstraction and summarization of acquired knowledge and experiences [19], as seen in A-Mem [60], MemO [61] and MemInsight [62]. More discussions are in Appendix D.
Memory in Multi-agent System. However, the memory mechanisms tailored for MAS remain markedly underexplored. Some representative frameworks, such L.LM-Debate [20, 33] and Mixture-of-AAenn [63], omit memory components altogether. Others merely adopt simplistic inside- trial memory schemes [46, 51]. Even in frameworks that attempt cross-trial memory [45], the memory is merely compressed as the final outcome artifacts, overlooking the nuanced agent interactions. Collectively, there a pressing need for a principled memory architecture that can capture, organize, and retrieve the inherently intricate task-solving processes unique to MAS [38].
LLM-based Multi-Agent Systems. Our work focuses on sask-no MAS, which, unlike their
single-agent counterparts, often lack the capacity for continual evolution through interaction with the
environment [64, 65]. Early frameworks such as AutoGen [13], CAMEL [24], and Agent Verse [66] rely entirely on pre-defined workflows. More recent efforts [67, 68, 49, 48, 69, 31] introduce a degree of adaptivity by generating dynamic MAS in response to environmental feedback. However, such evolution is often one-shot: for example, AFlow [49] employs Monte Carlo Tree Search to construct a complex MAS tailored to a specific task domain, which yet lacks the capacity to evolve with increasing task exposure or transfer across domains [50, 70]. From this perspective, constructing MAS with genuine self-evolving capabilities remains an open and challenging research frontier.
3 Preliminary
In this section, we establish the notation and formalize key concepts of multi-agent systems and G-Memory's 's hierarchical memory architecture.
Multi-agent System Formalization. Consider multi-agent framework represented by directed graph G (V,E), where |V] = N is the number of agents and & C VxV defines their communication
3

---

channels. Each node C2 E V corresponds to an individual agent described by the quadruple:
C2 = (Base,, Role, Mem;, Plugin;):
(1)
where Base; denotes the underlying large language model instance, Role, specifies the agent's designated role or persona, Mem; encapsulates its memory state, including past interactions or external knowledge stores, and Plugin, the set of auxiliary tools (e.g., web-search engine).
Upon receiving a user query Q, the system evolves through T' synchronous communication epochs. At each epoch t, we derive a topological ordering an edge from Ty to Tk- then j < k; which guarantees that every agent processes its inputs only after all its predecessors have acted. For each agent C₁ in its output at iteration t is computed
[Ti..... AN] of the nodes such that if there is
(1) C. (PL) Q, (r) (C.)))
where: r/ denotes the response generated by C: (which may include reasoning steps, intermediate analyses, or final proposals), pf comprises global instructions (including each agent's R,), .' (C,) is the set of in in-neighbors of C₁ whose outputs serve contextual inputs. After all agents have acted, a global aggregation operator A fuses the collection of responses into an interim solution (!);
(+)
A(r (1)
Common implementations for A include majority voting schemes [47], hierarchical summarization via dedicated aggregator agents [13, 30), or simply adopting the final agent's 's output as the answer [46].
These epochs iterate for {1, (I,..., T) until either a preset limit is reached an early-stopping criterion is met [71], producing the final response a to the query Q.
Memory Architecture. Our proposed G-Memory orchestrates and manages the memory of multi- agent systems via the following three hierarchical graph structures:
[*] Interaction Graph (Utterance Graph). For query Q, let G (L(1), El denote its interaction trajectory, where (i) nodes it()) = {ur} represent atomic utterances, with each u, & (A,, m²) containing A; V (speaking agent), and m, (textual content), (ii) Edges eQ C cu x
2(() follow temporal relationships: (uj, ux) E e( = Uy is transmitted to and inspires uk-
[ Query Graph. The query graph, storing previously tackled queries and metadata, is as follows:
Gquery = (2, = (1Qi, Mi, ((0)) ( 1 )
where Q = {g3} is the node set, node d - (Q;, Vi,, (( is composed of the original query Qi task status V2, E (Failed, Resolved]. and its associated interaction graph c!Qu). The edges & S Q encode semantic relationships between queries. The query graph enables retrieval beyond coarse metrics such as embedding similarity, with its meticulous topology.
[*] Insight Graph. The highest-level insight graph is featured as follows:
Ginsight = (I, = (Kiss Ox 1- l.).
4.
where the node set I [(4} represents distilled insights, each node is composed of the insight content K3 and the set supporting queries 94 C Q. The edges E I x Q forming hyper-connections where (Im; (lm bn, 4j) indicates insight contextualizes in through query 4j-
4 G-Memory
This section outlines the management workflow of G-Memory, illustrated in Figure 2. Specifically, upon the arrival of a new query Q, G-Memory first conducts coarse-grained retrieval to identify pertinent trajectory records (D Section 4.1). It then performs bi-directional hierarchical memory traversal: upward to retrieve collective cognitive insights, and downward to distill concrete procedural trajectories Section 4.2). After the mngmere-augmented MAS completes the query execution, the hierarchical memory architecture is jointly updated based on environmental feedback, thereby achieving the institutionalization of group knowledge (> Section 4.3).

---

Symbols
Insight Graph
Distilled Insights
Operation
Nonomand
Piger (6,8)
Inrigh charly denn us and thenrs cz
Insight
Jesght Depple. ot the rorshs
Iraght mode Y
Qevrry
Temid precide nedrnd to the liet bution.....
Aggest (fiecatcc ne
upward traversal
Mlmorr
Augmentatiin
Query/Tak Q
Multi-agent System f
Output Solution
Update Insights
Reery beadir anc fimm Eractecp
Proplc Fashedted
ec film dinettur from respehive Tea
man puikly scoud caleet bas to
Similarity- havre Retrival
Pherotion
Alg Environment Feedhbakk
soard iifurultyy trcht Are both gena ardeidd?
Query Graph Kay (0.6g
Touen 5 3.45
downvogrd traversall
Update
Memory
Augmentatiin
Bicifouli -iiiniinn Your hak to fedi sutride Move to the
CEO open oqar fall Ewartr agent
Internctions
Trajectory Condensation
Collehs Experimmo
The hatory follace chci-
Topre nirfor hyyeouiyy--bbdbbbb adtrreel b7, 30 avid
Interaction
sired
(taltre Lessons
GOO
g
Graphs
Th sumearsing the real: bJ
22 4
Figure The overview of our proposed G-Memory.
4.1 Coarse-grained Memory Retrieval
As plug-in designed for seamless integration into mainstream MAS, G-Memory is triggered when the MAS G encounters a new user query Q. As emphasized in organizational memory theory [1], efficient knowledge retrieval typically begins with broadly relevant schemas prior to more fine-grained access. Following this principle, G-Memory first performs a coarse- coarse-grained similarity-based retrieval over the query graph Gquery to efficiently obtain sketched set of queries QS:
( (
((Q)) v(:))
Ø arg top-k 4.E0 st. |0
v(q:))
where v(-) maps queries into fixed-length embeddings using models such as MiniLM [72]. While Equation (4) retrieves semantically similar historical queries, the similarity may be only superficial or noisy. Therefore, G-Memory further enlarges the relevant set hop expansion on the query graph:
よ = Q U fQx E Q =Q, E Φ Qk E (Q,) (Q)))
(5)
where の is augmented with the 1-hop neighbors of @ on the query graph Gquery, and N and W-(-) denote the out-neighborhood and on-neighborhood of node Qj. respectively. However, is suboptimal to directly feed these relevant records as input akin to certain single-agent memory systems [40, 37]. On one hand, the excessive context length may overwhelm the LLM; on the other hand, agents in MAS play distinct roles and should be assigned specialized memory tailored to their functions. To address this, the next section introduces a bi-directional processing scheme in G G-Memory that operates over both abstract and fine- ine-grained memory levels.
4.2 Bi-directional Memory Traversal
Subsequent to identifying the expanded set of relevant query nodes Ộ within Gquery. G-Memory exe- cutes bi-directional memory traversal to furnish multi-granularity memory support. Specifically, G-Memory first performs an upward traversal (Qquery + Ginsight), retrieving insight nodes that may provide high-level guidance for the current task:
S (lle-((2²)) Ie-z (ux E 1 S4 ns, #
(6)
where IQQ-I - is a query-to-insight projector that identifies all the insight nodes whose supporting query sets intersect with the input query set, and the retrieved insights Is encapsulate distilled, generalized knowledge potentially relevant for orienting the MAS g's strategic approach to Q.
Beyond generalized insights, the fine-grained textual interaction history of the MAS is equally valuable, as it reveals the underlying reasoning patterns that led to successful or failed collabo- rations [67, 73, 74]. To utilize these concisely, in the downward traversal (Gquery. + Gitteacctonn.
5

---

G-Memory employs an encapsulates essential inter-agent collaboration:
graph sparsifier (LLM(-:)) to extract the core subgraph that
c inter
MMM =
(SLM( 1)1g E argtop-M RELM {h)
(7)
(1)) H
where R1,m(Q.4;) rates the relevancy of historical queries w.r.t Q. and the sparsifier ( inter Q) constructs a sparsified graph ( = (Q((2)) ((Q))) from the original ((0g) by identifying and retaining dialogue elements. Please refer Appendix C for their implementations.
Upon completing the bi-directional traversal, we obtain both generalizable insights (I*) and detailed collaborative trajectories (19 )nter)am)) G-Memory then proceeds to provide specialized memory support for each agent C E V within the MAS g.
Mem; <- (IS, [
IMMi Role;, Q): VC; (Base;, Role,, Mem;, Plugin;) E V,
(8)
where the operator ()::)) evaluates the utility and relevance of each insight i E I and sparsified interaction graph g(Q:) concerning the agent's 's specific role Role, and the task Q (see Appendix C). Based on this evaluation, & intializes each agent's 'sss internal memory state Mem; with filtered insights, interaction snippets, summaries thereof, cquipping it with pertinent historical context before participates in the subsequent reasoning epochs of the MAS. It is worth noting that G-Memory is invoked at the onset of solving query Q in our implementation. However, practitioners may flexibly configure more fine- mee-grained invocation strategies, such at the beginning of each MAS dialogue round or selectively for specific agents, based on their needs.
4.3 Hierarchy Memory Update
After completing memory augmentation for each agent, the system G is executed outlined in Sec- tion 3, yielding a final solution (2") and receiving environmental feedback, including execution status V; E [Failed, Resolved), token usage, and other performance metrics. Subsequently, G-Memory updates its hierarchical memory architecture incorporate this new query. At the interaction level,
G-Memory traces each agent's utterances to construct the interaction graph g( which then stored. At the query level, new query node is instantiated and added to the query graph Qquery:
Qnew f- (Q, Mr ((Q)), Neonn & @R
(U Pn). UEEIS
Enow {ggn, Ioew) In E Nooun, Gqeer + (QU {ghow } Eq Unew),
where edges are established between Qnew and (ii) the set Ω containing the top-M relevant historical queries identified in Equation (7), and (ii) the set of queries U.e 22 that support the insights S utilized for solving Q. Qquery denotes the updated query graph.
Finally, at the insight level, G-Memory integrates the learning from the completed query Q into the insight graph Ginsight (I, E.). First, possible new insights summarizing the experience generated and structurally linked via a summarization function I (.,.) (see prompt in Appendix C) as follows:
lnew = (J ( V), {9new }), E, new + {4k know; Inew)
Ug E I
(10)
G'nsight < (I (Inww) ) E, E, new)
where edges are added to connect the previously utilized insights which inspires the completion of Q in Equation (6). Afterward, the supporting query sets (92) for the utilized insights (IS) are updated to include Onew, reflecting their relevance to this successful (or failed) application:
m + (I Ina) U (FA, 2t U {gnew) 1) 144 = =m6,, S4k E Irec U (leew)
(11)
Ginsight +- (Irext & U. &, new),
where the final node set Teext incorporates the new insight and the updated versions of the utilized insights, and the resulting graph Gree thus encapsulates the integrated knowledge. This continuous update cycle across all hierarchical levels enables G-Memory learn and adaptively refine its collective memory based on ongoing experience.

---

Table I: Performance comparison with single/muuti-agent memory architectures five benchmarks.
The underlying LLM backbone is GPT-4o-mini. We highlight the best and second best results.
MAS
Memory
ALFWorld
SeiWorld
PDDL
HotpotQA
FEVER
Avg.
No-memory
17.60
54.49
23.53-0.00
28.57
57.13
48.27
Voyager
85.07
62.36,7.87
24.56,1.00
32.32
63.27
53.52
MemoryBank
12.96
13.11
20.41
33.6775.10
61.22pa..o
48.67
AutoGen
M
Generative
p5.76
61.19
25.53
31.63
60.20 (0.07
52.98
MetaGPT
81.34ys 13.75
61.91 (7.42
21.63 $1.00
32.67 F::10
62.67
52.04 (3/77
ChaiDev
79.85
50.96 ja.ca
16.65 jo.ss
24.49: 14.08
59.18-2.00
46.23
MacNet
76.55 1..8
ro.44
22.94 jo.50
28.36
60.87
48.83 10:58
0 G-Memory (Ours)
88.81
67.40
67.40-12.01
27.77-4.24
35.67,7.10
66.24
57.18
No-memory
56.72 10.00
55.38 10.00
11.62 y0.00
31.69 10.00
60.20 -0.00
43.12
Voyager
66.42
62.83 (7.45
32.64
62.24
47.85 y1.75
MemoryBannk
55.22|11.00
54.74,0.64
8.08 1a.64
29.59.2..
39.13
41.35 (1.77
DyLAN
Generative
67.91 (1..9 p11.19
64.16
13.87 72.25
29.29 ja
62.30 (2.10
47..51 44..51 p4.39
COLM 2024
MetaGPT-M
69.40 (12.68
62.37
y4.45
32.34
r0.20
47.75
ChatDev-M
46.27 10:45
53.35 12.03
10.75 jous7
22.45 p.22
58.33 gt.87
38.23 (1.89
MacNet-M
53.44 13.25
54.32 (1.06
12.11 1oo.4s
30.12,11.50
61.10
42.22 10.90
G-Memory (Ours)
70.90
65.64-118
11.64410.200
18.95 y7.03
34.69 (3.00
64.22
50.88 (7.76
No-memory
51.49 51.49 10.00
57.53 10.00
12.18
28.57 10.00
60.29 60.29
42,01 10.00
Voyager
61.94
64.53,7.00
14.06
y4.05
62.54
47.114
MemoryBank
$0.00
60.15
8.64
33.6775.10
61. 61.222oode
42.74
MacNet
ICLR 2025
Generative
62.69 (11.00
65.49
7.92
29.59
63.27
45.79
MetaGPT-M
63.70 (12.21
65.27 (7.74
16,03:3.85
31.00 (2.50
59.33 1o.s6
47.07 15.05
ChatDev-M
49.25 (2:24
56.58
13.51
29.00
59.18
41.50 (0.51
MacNet-M
53.44 11.05
56.14,1. (1..
13.59
27.89
59.20,1.cm
42.05 po.00
G G-Memory (Ours)
67.16
68.11-10.50
24.33 16
35.69 (7.18
64.44-4.3
51.95
5 Experiment
In this section, we conduct extensive experiments to answer: (RQ1) How does G-Memory perform compared to existing single/multi-agent memory architectures? (RQ2) Does G-Memory incur exces- sive resource overhead? (RQ3) How sensitive G-Memory to its key components and parameters?
5.1 Experiment Setup
Datasets and Benchmarks. To thoroughly evaluate the effectiveness of G-Memory, we adopt five widely-adopted benchmarks across three domains: (1) Knowledge reasoning, including Hot- PotQA [75] and FEVER [76]; (2) Embodied action, including ALFWorld [77] and SeiWorld [78]; (3) Game, namely PDDL [79]. Details on these benchmarks are in Appendix A.1.
Baselines. We select four representative single-agent memory baselines, including non-memory, Voyager [16], MemoryBank [36], and Generative Agents [19], as well as three multi-agent memory implementations from MetaGPT [21]. ChatDev [45], and MacNet [46], denoted as MetaGPT-M, ChatDev-M, and MacNet-M, respectively. Details are in Appendix A.2.
MAS and LLM Backbones. We select three representative multi-agent frameworks to integrate with G-Memory and the baselines, including AutoGen [13], DyLAN [71], and MacNet [46]. More details the MAS setups are placed in Appendix A.3. For instantiating these MAS frameworks, we adopt two open-source LLMs, Qwen-2 5- 5- 7b and Qwen-2 2. 5-15--146 14,, as well as one proprictary LLM, gpt-40-mini. The deployment of Qwen series is via local instantiation using Ollama!, and GPT models are accessed via OpenAl APIs
Parameter Configurations. We implement the embedding function v(-) in Equation (4) with ALL-MINILM--L6-V [80]. The number of the most relevant interaction graphs A in Equation (7) is set among (2. 3, 4, 5], and the number of relevant queries in Equation (4) is set among {1, 2). The detailed ablation study on prper-parammters is placed in Section 5.4.
5.2 Main Results (RQ1)
Tables 1, and comprehensively report the performance different memory architectures across three L.LM backbones and three MAS frameworks. We summarize the key observations follows:
'http://github com/ com//ollaee ollmma
7

---

PDDL + AutoGen
ALFWorld + Autooen
ALFWorld + DyLAN
(bdeaihh sakena)
Mexder
52
Gecendivv (a sakenaa
Cepecative
Token Cost (c+6)
Vioeger (l3--00 tobwae)
Token Cost Cost (c+6)
Mteooyylann (5
Token Cost (c++6
10.9
Voogerr
MMnore (Ours)
6.0
G-Venory 1
(5.0 Tiliees)
Meagari
(9.7e-06 toloas)
Mtggen Bank
+
54
#9
MetaGrr: (sodenci takene))
ChorDer
66
8.9
Chei (5.3e 106
Merrart
No DRCEAIO
Yo monnnry d-- hrlond)
77.
No.0
78
8o
Performance (%)
Performance (%)
Performance (%)
Figure 3: Cost analysis of G-Memory. We showcase the performance versus the overall system token cost when combined with different memory architectures.
Takeaway O: G-Memory consistently improves performance across all task domains and MAS frameworks. As shown in Table 2, when integrated with AutoGen and MacNet (powered by Qwen-2. 5b))), G-Memory surpasses the best-performing single-/multi-agent memory baselines by an average of 6.8% and 3.5%, respectively. With the more capable 2. 1b, the improvement is even more pronounced: in Table 3, G-Memory boosts MacNet's performance on ALFWorld from 58.2 28% to 79.10%, achieving a substantial 20.89% gain.
Multi-agent systems demand specialized memory designs. A thorough examination of existing baselines reveals a surprising insight: most memory mechanisms fail consistently benefit MAS settings. In Table 2, baselines such as Voyager and MemoryBank degrade AutoGen's s performance on PDDL by as much as 4.17% and 1.34%, respectively. We attribute this the inability of these methods to provide agent role-specific memory support, which is essential in the PDDL strategic game tasks, where effective division of labor is critical to success. Even MAS-oriented designs, such as ChatDev-M, result in 2.32% performance drop when applied to MacNet+SciWord We attribute this to ChatDev-M'ss narrow memory queries, which provides limited utility in embodied action environments. These findings highlight the necessity of G-Memory's core characteristics: role-specific memory cues, abstracted high-level insights, and trajectory condensation-al all of which are critical for effective memory in MAS.
Takeaway O:
only the execution results of past
5.3 Cost Analysis (RQ2)
To evaluate the efficiency of G-Memory in terms of token consumption, we visualize the performance versus token cost trade-off across various settings, as shown Figures 3 and 7. Our findings are:
Takeaway O: G-Memory achieves high-performing collective memory without excessive token consumption. As depicted in Figure 3, G-Memory consistently delivers the highest performance improvement (10.3%% 1 over no-memory setting on increase in token consumption (only 1.4 x 10%). In contrast, MetaGPT-M incurred additional 2.2 x 10t tokens for mere 4.07% gain. This clearly demonstrates the token-efficiency of G-Memory.
while maintaining a modest
5.4 Framework Analysis (RQ3)
Sensitivity Analysis. Regarding the hop expansion, as shown in Figure 4a, I-hop expansion consistently yields the best or near-best performance across tasks, with peak accuracies of 85.82% (ALFWorld), 55.20 24% (PDDL) in AutoGen. In contrast, 2-hop and 3-hop settings often degrade performance, e.g., PDDL drops to 49.799 79.9 (2-hop). This suggests that excessive hop expansion may introduce irrelevant insights during memory upward traversal, impairing task- ausk-specific reasoning. Similarly, Figure 4b shows that the optimal & is among {1, 2}. Larger i values (e.g., k=5) can significantly degrade the system performance, e.g., 7.71% % 4 on ALFWorld+Autode on FEVER+DYLAN, indicating that retrieving more queries may introduce task-irrelevant noise. Collectively, we employ 1-hop expansion and & E {1,2) throughout the experiments.
and 2.5% 4
Ablation Study. Figure 4c presents an ablation of G-Memory by isolating the impact of the high-
level insight module (IS in Equation (6)) and fine-grained interactions ( !!A!! in Equation (7)). As shown, removing either part leads to a consistent performance drop. When only fine-grained
(inter))
interactions are enabled, the average scores drop by 4.47% for AutoGen and 3.82% J for DyLAN
8

---

Parameter Autotien and DvLAN
Differentt Hops
Autofien and LAN
Performanec (%)
tlop
zhoy
=
day
MAS
Inter. Insi.
PDDL
FEVER
AutoGen
DyLAN
54.46
63.27
AutoGen
DyLAN
AutoGen
50.00
68.77
55.24
71,43
48.75
61.39
DyLAN
46.69
64.31
51.12
66.66
"11MMod EENRRAASON 700L. HOVE
Benchmarks
Benchmarks
(c) Ablation study two vari- ants of G-Memory.
(a) Sensitivity analysis #hop.
(h) Sensitivity analysis on parameter k.
Figure 4: (a) Sensitivity analysis of the hop expansion in Equation (5); (b) Sensitivity analysis of the number of selected queries in Equation (4); (c) We study two variants of G-Memory: merely
providing high-level insights (i.e., the insights I in Equation (6)) fine-graines interactions (i.e., the core trajectories in Equation (7)). All the experiments here are done with Qwen- 2. 5-14b.
ALFWorld AutoGen
HotpotQA DyLAN
PDDL MacNet
Query clath
AutoGen Team Salver agent
Query
DyLAN Team
TP2 S Query
MacNet Team
in countertop
Ground ogent
Question Are both Lygodium
Maxillria
@
on b2. ba is on b6, b3 on 57, on on
edge agent
genus of enchids?
b5,
High-level Insights
High-level Insights
High-level Insights
For
Era requine thru bicessble.
clean Tiley, cad reture there teei youted sternge
Avoid matakurll referning
Enaure blocke clear and the are-hee black because this provents indild uati escine the bloder are pizced cornectyy.
For
vanty thit the eearch resutts one ristekeely sted infor eutiis
Fam
After chearing then, esure it ploced
the
gesteeted shernge looatii iemedishely eaiid
Fine-grained Trajectory
Fine-grained Trajectory
Fine-grained Trajectory
Tack:
Ane Ruggaro Doacitto from Endyy, and Mexicon Alquaddr
Task:
The goo in esticty the foinnnng condiiees Compressed Traji
egg
Compressed Trujs
Compressed Traji
Snarch
Idamt by
Edentify
seurch
Unittck b2 Check bd unetask ba Check :3 and
Slick b2
Warning
from
enc b2 on
Figure 5: Case study of G-Memory.
compared to the full method. Conversely, enabling only insights leads to smaller drops of 3.95% and 3.39%. This indicates that while both components are contributive, interactions offer slightly greater impact, likely due to their preserving more fine-grained, dialogue-level contextual grounding.
5.5 Case Study
Figure 5 sillustrates concrete memory cues provided by G-Memory across diverse tasks. For example, in the ALFWorld+AAutoenn setting, given the task query "put a clean cloth in countertop". G-Memory successfully retrieves a highly analogous historical query, "put clean egg in microwave""-bor requiring the object be in clean state. Alongside this, G-Memory surfaces critical trajectory segment where the solver agent attempts to place the egg in the microwave before cleaning, prompting the ground agent to intervene. This collaborative trajectory offers actionable guidance for the current task. Morcover, the high-level insights retrieved by G-Memory prove equally valuable for task execution. In the context of HotpotQA's web search task, G-Memory retrieves insight warning against "mistakenly referring", which helps prevent agents from incorrectly answering based on similarly named individuals. Overall, G-Memory provides effective multi-level memory support across varied domains, including embodied action, knowledge reasoning, and game environments.
6 Conclusion & Limitation
In this paper, we conduct a thorough examination of existing memory architectures designed for multi-agent systems (MAS) and identify that their overly simplified designs fundamentally hinder the systems' capacity for self-evolution. To bridge this gap, we propose G-Memory, a hierarchical memory framework that organizes the complex and extended interaction trajectories of MAS into a three-tier graph hierarchy: the insight, query, and interaction graphs. G-Memory provides each agent with customized and hierarchical memory cues, ranging from abstract, generalizable insights

---

to tneegrained, task-critical collaborative segments, and dynamically evolves its knowledge base across episodes. Extensive experiments demonstrate that G-Memory can be seamlessly integrated into state-of-the-ar MAS frameworks, significantly enhancing their self-evolution capability, e.g., up to 20.89% improvement on embodied action tasks. Limitations: Although G-Memory has been evaluated across three domains and five benchmarks, further validation on more diverse tasks (e.g.- medical QA) would strengthen its soundness, which we leave for future work.
Acknowledgment
This research supported in part by NUS Start-up Grant A-0010106-00-00.
References
[1] James P Walsh and Gerardo Rivera Ungson. Organizational memory. Academy of management review, 16(1):57-91 1991.
[2] Danny Driess, Fei Xia, Mehdi SM Sajjadi, Corey Lynch, Aakanksha Chowdhery, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, et al. Palm-e: An embodied multimodal language model. 2023.
[3] Shihao Wang, Zhiding Yu, Xiaohui Jiang, Shiyi Lan, Min Shi, Nadine Chang, Jan Kautz, Ying Li, and Jose M Alvarez. Omnidrive: A holistic Ilm-agent framework for autonomous driving with 3d perception, reasoning and planning. arXiv preprint arXiv:2405.01533, 2024.
[4] Sipeng Zheng, Jiazheng Liu, Yicheng Feng, and Zongqing Lu. Steve-eye: Equipping Ilm-based embodied agents with visual perception in open worlds. arXiv preprint #rXiv:2310.13255. 2023.
[5] Yuxi Wei, Zi Wang, Yifan Lu, Chenxin Xu, Changxing Liu, Hao Zhao, Siheng Chen, and Yanfeng Wang. Editable scene simulation for autonomous driving via collaborative Ilm-agents. In Proceedings the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pages 15077-15087, 2024.
[6] Yuqi Zhu, Shuofei Qiao, Yixin Ou, Shumin Deng, Shiwei Lyu, Yue Shen, Lei Liang, Jinjie Gu, Huajun Chen, and Ningyu Zhang. Knowagent: Knowledge-augmented planning for Ilm-based agents. arXiv preprint arXiv:2403.03101 2024.
[7] Lutfi Eren Erdogan, Nicholas Lee, Sehoon Kim, Suhong Moon, Hiroki Furuta, Gopala Anu- manchipalli, Kurt Keutzer, and Amir Gholami. Plan-and-act Improving planning of agents for long-horizon tasks. arXiv preprint St0.::::0030005772 2025.
[8] Xu Huang, Weiwen Liu, Xiaolong Chen, Xingmei Wang. Hao Wang, Defu Lian, Yasheng Wang, Ruiming Tang, and Enhong Chen. Understanding the planning of Ilm agents: A survey, arXiv preprint arXiv:2402.02716. 2024.
[9] Pranav Putta, Edmund Mills, Naman Garg, Sumeet Motwani, Chelsea Finn, Divyansh Garg, and Rafael Rafailov. Agent q: Advanced reasoning and learning for autonomous ai agents, arXiv preprint arXiv:2408.0711 07199, 2024.
[10] Tula Masterman, Sandi Besen, Mason Sawtell, and Alex Chao. The landscape of emerging ai agent architectures for reasoning, planning, and tool calling: A survey. arXiv preprint arXiv:2404.1 1584, 2024.
[11] Manling Li, Shiyu Zhao, Qineng Wang, Kangrui Wang. Yu Zhou, Sanjana Srivastava, Cem Gokmen, Tony Lee, Erran Li Li, Ruohan Zhang, et al. Embodied agent interface: Benchmarking Ilms for embodied decision making. Advances in Neural Information Processing Systems, 111100428-100533 2024.
[12] Yijun Yang, Tianyi Zhou, Kanxue Li, Dapeng Tao, Lusong Li, Shen, Xiaodong He, Jing Jiang, and Yuhui Shi. Embodied multi-modal agent trained by an Ilm from a parallel textworld. In Proceedings the IEEF/CVF conference on computer vision and pattern recognition, pages 26275-26285, 2024.
10

---

[13] Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Shaokun Zhang, Erkang Zhu, Beibin Li, Li Jiang, Xiaoyun Zhang, and Chi Wang. Autogen: Enabling next-gen Ilm applications via multi-agent conversation framework, August 01, 2023 2023.
[14] Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Y. Wu, K. Li, Fuli Luo, Yingfei Xiong, and Wenfeng Liang. Deepseek-coder: When the large language model meets programming the rise of code intelligence, 2024.
[15] Sirui Hong, Yizhang Lin, Bang Liu, Bangbang Liu, Binhao Wu, Ceyao Zhang, Chenxing Wei, Danyang Li, Jiaqi Chen, Jiayi Zhang, et al. Data interpreter: An Ilm agent for data science. arXiv preprint arXiv:2402.1 18679, 2024.
[16] Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An Open-Ended Embodied Agent with Large Language Models. arXiv e-prints, page arXiv:2305.. 16291, May 2023.
[17] Long Chen, Oleg Sinavski, Jan Hünermann, Alice Karnsund, Andrew James Willmott, Danny Birch, Daniel Maund, and Jamie Shotton. Driving with Ilms: Fusing object-level vector modality for explainable autonomous driving. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 14093-14100 IEEE, 2024.
[18) Yuan Sun, Navid Salami Pargoo, Peter Jin, and Jorge Ortiz. Optimizing autonomous driving for safety: A numan--entric approach with llm-enhanced rlhf. In Companion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing, pages 76 (0-80, 2024.
[19] Joon Sung Park, Joseph C. O'Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. Generative agents: Interactive simulacra of human behavior, April 01, 2023 2023.
[20] Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch. Improving factuality and reasoning in language models through multiagent debate. CoRR, abs/2305.14325, 2023.
[21] Sirui Hong, Xiawu Zheng, Jonathan Chen, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, and Chenglin Wu. Metagpt: Meta programming for multi-agent collaborative framework, August 01, 2023 2023.
[22] Marvin Minsky. Society of mind. Simon and Schuster, 1988.
[23] Push Singh. Examining the society of mind. Comput. Artif. Intell., 22()):52 2003..
[24] Guohao Li, Hasan Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. CAMEL: communicative agents for "mind" exploration of large language model society. NeurIPS, 2023.
[25] Zhenhailong Wang. Shaoguang Mao, Wenshan Wu, Tao Ge, Furu Wei, and Heng Ji. Unleashing cognitive synergy in large language models: A task-solving agent through aulti-persona self- collaboration, July 01, 2023 2023. work in progress.
[26] Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V. Chawla, Olaf Wiest, and Xiangliang Zhang. Large language model based multi-agents: A survey of progress and challenges. CoRR, abs/2402.006880 2024.
[27] Pouya Pezeshkpour, Eser Kandogan, Nikita Bhutani, Sajjadur Rahman, Tom Mitchell, and Estevam Hruschka. Reasoning capacity in multi-agent systems: Limitations, challenges and human-centered solutions. CoRR, abs/2402.01 108, 2024.
[28] Giorgio Piatti, Zhijing Jin, Max Kleiman-Weiner, Bernhard Schölkopf, Mrinmaya Sachan, and Rada Mihalcea. Cooperate or collapse: Emergence of sustainability behaviors in a society of Ilm agents. arXiv preprint arXiv:2404.1 16698, 2024.
[29] Rafael Pina, Varuna De Silva, and Corentin Artaud. Discovering causality for efficient coopera- tion in multi-agent environments. CoRR, abs/2306. 11846, 2023.

---

[30] Guibin Zhang, Yanwei Yue, Zhixun Li, Sukwon Yun, Guancheng Wan, Kun Wang, Dawei Cheng, Jeffrey Xu Yu, and Tianlong Chen. Cut the crap: An economical communication pipeline for llm-based multi-agent systems. arXiv preprint arXiv:2410.02506, 2024.
[31] Yanwei Yue, Guibin Zhang, Boyang Liu, Guancheng Wan, Kun Wang, Dawei Cheng, and Yiyan Qi. Masrouter: Learning to route Ilms for multi-agent systems. arXiv preprint arXiv:2502. 1113, 2025.
[32] Qinlin Zhao, Jindong Wang, Yixuan Zhang, Yiqiao Jin, Kaijie Zhu, Hao Chen, and Xing Xie. Competeai: Understanding the competition behaviors in large language model- agents. arXiv preprint arXiv:2310. 175122 2023.
[33] Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yuju Yang, Zhaopeng Tu, and Shuming Shi. Encouraging divergent thinking in large language models through multi-agent debate. CoRR, 200s/3055.9118, 2023.
[34] Wei Wang, Dan Zhang, Tao Feng, Boyan Wang, and Jie Tang. Battleagentbench: benchmark for evaluating cooperation and competition capabilities of language models in multi-agent systems. arXiv preprint arXiv:2408. 15971, 2024.
[35] Chuanyang Zheng, Zhengying Liu, Enze Xie, Zhenguo Li, and Yu Li. Progressive-hin prompt- ing improves reasoning in large language models, April 01, 2023 2023. Tech Report.
[36] Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang. Memorybank: Enhancing large language models with long-term memory. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 19724-19731, 2024.
[37] Charles Packer, Vivian Fang, Shishir_G G Patil, Kevin Lin, Sarah Wooders, and Joseph_ Gonza- lez. Memgpt: Towards ilms operating systems. 2023.
[38] Zeyu Zhang, Xiaohe Bo, Chen Ma, Rui Li, Xu Chen, Quanyu Dai, Jieming Zhu, Zhenhua Dong. and Ji-Rong Wen. A survey on the memory mechanism of large language model based agents. arXiv preprint arXiv:2404. 13501, 2024.
[39] Chenxu Hu, Jie Fu, Chenzhuang Du. Simian Luo, Junbo Zhao, and Hang Zhao. Chatdb: Augmenting 11ms with databases their symbolic memory. arXiv preprint arXiv:2306.03901, 2023.
[40] Junru Lu, Siyu An, Mingbao Lin, Gabriele Pergola, Yulan He, Di Yin, Xing Sun, and Yunsheng Wu. Memochat: Tuning mlms to use memos for consistent long-range open-domain conversation. arXiv preprint arXiv:2308.082. 2023.
[41] Yancheng Wang, Ziyan Jiang, Zheng Chen, Fan Yang. Yingxue Zhou, Eunah Cho, Xing Fan, Xiaojiang Huang, Yanbin Lu, and Yingzhen Yang. Recmind: Large language model powered agent for recommendation. arXiv preprint arXiv:2308. 14296, 2023.
[42] Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. Expel: Llm agents are experiential learners. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 19632-19642, 2024.
[43] Yuan Li, Yixuan Zhang, and Lichao Sun. Metaagents: Simulating interactions of human behaviors for 11m- based task-oriented coordination via collaborative generative agents. arXiv preprint arXiv:2310006500, 2023.
[44] Chen Gao, Xiaochong Lan, Zhihong Lu, Jinzhu Mao, Jinghua Piao, Huandong Wang, Depeng Jin, and Yong Li. S3: Social-network simulation system with large language model-empowered agents, arXiv preprint arXiv:2307. 14984, 2023.
[45] Chen Qian, Xin Cong, Cheng Yang, Weize Chen, Yusheng Su, Juyuan Xu, Zhiyuan Liu, and Maosong Sun. Communicative agents for software development, July 01, 2023 2023. 25 pages, 9 figures, 2 tables.
[46] Chen Qian, Zihao Xie, Yifei Wang, Wei Liu, Yufan Dang, Zhuoyun Du Weize Chen, Cheng Yang, Zhiyuan Liu, and Maosong Sun. Scaling lage--langggemmdeeee model-baaed multi-agent collab- oration. arXiv preprint 200. 2024.
12

---

[47] Mingchen Zhuge, Wenyi Wang, Louis Kirsch, Francesco Faccio, Dmitrii Khizbullin, and Jürgen Schmidhuber. Gptswarm: Language agents as optimizable graphs. In Forty-first International Conference on Machine Learning, 2024.
[48] Shengran Hu, Cong Lu, and Jeff Clune. Automated design of agentic systems. arXiv preprint arXiv:2408.0445 2024.
[49]
Jiayi Zhang, Jinyu Xiang, Zhaoyang Yu, Fengwei Teng, Xionghui Chen, Jiaqi Chen, Mingchen Zhuge. Xin Cheng, Sirui Hong, Jinlin Wang. Bingnan Zheng, Bang Liu, Yuyu Luo, and Chenglin Automating Agentic Workflow Generation, October 2024, arXiv:2410.10722
Wu. AFlow:
[50] Guibin Zhang, Luyang Niu, Junfeng Fang, Kun Wang, Lei Bai, and Xiang Wang. Multi-agent architecture search via agentic supernet. arXiv preprint arXiv:2502.04180) 2025.
[51] Zhangyue Yin, Qiushi Sun, Cheng Chang. Qipeng Guo, Junqi Dai, Xuan-Jing Huang, and Xipeng Qiu. Exchange-of-thought Enhancing large language model capabilities through cross-model communication. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 15135-15153 2023.
[52] Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang. Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, and Ji- Ji-Rong Wen. A survey on large language model based autonomous agents. Front. Comput. Sci., 18, 2024.
[53] Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, Rui Zheng, Xiaoran Fan, Xiao Wang. Limao Xiong, Yuhao Zhou, Weiran Wang. Changhao Jiang, Yicheng Zou, Xiangyang Liu, Zhangyue Yin, Shihan Dou, Rongxiang Weng, Wensen Cheng, Qi Zhang, Wenjuan Qin, Yongyan Zheng, Xipeng Qiu, Xuanjing Huan, and Tao Gui. The rise and potential of large language model based agents: A survey, arxiv preprint, abs/2309.07866 2023.
[54] Chen Gao, Xiaochong Lan, Nian Li, Yuan Yuan, Jingtao Ding, Zhilun Zhou, Fengli Xu, and Yong Li. Large language models empowered agent-based modeling and simulation: A survey and perspectives. CoRR, abs/2312. 19970 2023.
[55] Xinyi Li, Sai Wang, Siqi Zeng, Yu Wu, and Yi Yang. A survey on Ilm-based multi-agent systems: workflow, infrastructure, and challenges. Vicinagearth, 1(1):9, 2024.
[56] Longtao Zheng, Rundong Wang, Xinrun Wang, and Bo An. Synapse: Trajeclory-as-eeemplar prompting with memory for computer control. arXiv preprint arXiv:2306.0786. 2023.
[57] Xizhou Zhu, Yuntao Chen, Hao Tian, Chenxin Tao, Weijie Su, Chenyu Yang, Gao Huang, Bin Li, Lewei Lu, Xiaogang Wang, et al. Ghost in the minecraft: Generally capable agents for open-world environments via large language models with text-based knowledge and memory. arXiv preprint arXiv:2305. 2023.
[58] Xiangru Tang, Tianyu Hu, Muyang Ye, Yanjun Shao, Xunjian Yin, Siru Ouyang, Wangchunshu Zhou, Pan Lu, Zhuosheng Zhang, Yilun Zhao, et al. Chemagent: Self-updating library in large language models improves chemical reasoning. arXiv preprint arXiv:2501.06599 2025.
[59] Noah Shinn, Beck Labash, and Ashwin Gopinath. Reflexion: an autonomous agent with dynamic memory and self-reflection. arXiv preprint, 0bs/2303.11366 2023.
[60] Wujiang Xu, Kai Mei, Hang Gao, Juntao Tan, Zujie Liang, and Yongfeng Zhang. A-mem: Agentic memory for Ilm agents. arXiv preprint arXiv:2502. 2025.
[61] Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. Mem0: Building production-ready ai agents with scalable long-term memory. arXiv preprint arXiv:2504.1 19413, 2025.
[62] Rana Salama, Jason Cai, Michelle Yuan, Anna Currey, Monica Sunkara, Yi Zhang, and Yassine Benajiba. Meminsight: Autonomous memory augmentation for Um agents. arXiv preprint 2Xiv:2503.21760, 2025.
13

---

[63] Junlin Wang, Jue Wang, Ben Athiwaratkun, Ce Zhang, and James Zou. Mixture-of-agents enhances large language model capabilities. arXiv preprint S04iv:2400004692, 2024.
[64] Wangchunshu Zhou, Yixin Ou, Shengwei Ding, Long Li, Jialong Wu, Tiannan Wang, Jiamin Chen, Shuai Wang, Xiaohua Xu, Ningyu Zhang. et al. Symbolic learning enables self-evolving agents. arXiv preprint arXiv:2406. 18532, 2024.
[65] Xuechen Liang, Meiling Tao, Yinghui Xia, Tianyu Shi, Jun Wang, and JingSong Yang. Self-evolving agents with reflective and memory-augmented abilities. arXiv preprint arXiv:2409.00S: 000702 2024.
[66] Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chen Qian, Chi-Min Chan, Yujia Qin, Yaxi Lu, Ruobing Xie, Zhiyuan Liu, Maosong Sun, and Jie Zhou. Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors in agents, 2023.
[67] Yue Hu, Yuzhu Cai, Yaxin Du, Xinyu Zhu, Xiangrui Liu, Zijie Yu, Yuchen Hou, Shuo Tang, and Siheng Chen. Self-evolving multi-agent collaboration networks for software development. arXiv preprint arXiv:2410.16996, 2024.
[68] Guibin Zhang, Yanwei Yuc, Xiangguo Sun, Guancheng Wan, Miao Yu, Junfeng Fang, Kun Wang. Tianlong Chen, and Dawei Cheng. G-designer: Architecting multi-agent communication topologies via graph neural networks. arXiv preprint arXiv:2410.11782 2024.
[69] Siyu Yuan, Kaitao Song, Jiangjie Chen, Xu Tan, Dongsheng Li, and Deqing Yang. Evoa- gent: Towards automatic multi-agent generation via evolutionary algorithms. arXiv preprint arXiv:2406. 14228, 2024.
[70] Guibin Zhang, Kaijie Chen, Guancheng Wan, Heng Chang, Hong Cheng, Kun Wang, Shuyue Hu, and Lei Bai. Evoflow: Evolving diverse agentic workflows on the fly. arXiv preprint arXiv:2502.07373, 2025.
[71] Zijun Liu, Yanzhe Zhang, Peng Li, Yang Liu, and Diyi Yang. Dynamic Ilm-agent network: An Ilm-agent collaboration framework with agent team optimization. CoRR, abs/2310.02177 2023.
[72] Kuansan Wang. Zhihong Shen, Chiyuan Huang, Chieh-Han Wu, Yuxiao Dong, and Anshul Kanakia. Microsoft academic graph: When experts are not enough. Quantitative Science Studies, 1(1):396-413, 2020.
[73] Wanjia Zhao, Mert Yuksekgonul, Shirley Wu, and James Zou. Sirius: Self-improving multi- agent systems via bootstrapped reasoning. arXiv preprint ar 2xXv:25002.04700 2025.
[74] Heng Zhou, Hejia Geng, Xiangyuan Xue, Zhenfei Yin, and Lei Bai. Reso: A reward-drive self- organizing Ilm-based multi-agent system for reasoning tasks. arXiv preprint arXiv:2503.02390 2025.
[75] Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W Cohen, Ruslan Salakhut- dinov, and Christopher D Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. arXiv preprint htt1:11009096000 2018.
[76] James Thore. Andreas Vlachos, Christos Christodouloppolls, and Arpit Mittal. Fever: large-scale dataset for fact extraction and verification. arXiv preprint arXiv:1803.0055 1803.05355 2018.
[77] Mohit Shridhar, Xingdi Yuan, Mlex-Ale Côté, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. Alfworld: Aligning text and embodied environments for interactive learning. aiv preprint arXiv:2010.03768. 2020.
[78] Ruoyao Wang. Peter Jansen, Marc-Alexandre Côté, and Prithviraj Ammanabrolu. Scienceworld: Is your agent smarter than 5th grader? arXiv preprint M0Xi::2203.005400 2022.
[79] Chang Ma, Junlei Zhang, Zhihao Zhu, Cheng Yang, Yujiu Yang, Yaohui Jin, Zhenzhong Lan, Lingpeng Kong, and Junxian He. Agentboard: An analytical evaluation board of multi-turn I1m agents. arXiv preprint arXiv:2401.13178 2024.
[80] Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou. Minilm: Deep self-attention distillation for aank-agi compression of pre-trained transformers. Advances in Neural Information Processing Systems, 33:5776-5788, 2020.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our main claim that multi-agent systems require more dedicated memory architectures is both methodologically and empirically supported.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Please refer Section 6.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model sell-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
15

---

Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We have thoroughly provided the relevant information (see Appendix and section 5.1).
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer: [Yes]
Justification: We have provided the relevant codes with clear instructions in an anonymous link (See Abstract). All the datasets used are publicly available and clearly cited.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
ccpps.cc/puullc//gu guwdee/iee////e CCodeeubeesiaceecceee cembbmmsssiinnn ssssonoliccy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We have described the dataset setups and evaluation metrics in Appendix A. All the hyperparameters are described in Section 5.1. There no explicit dataset splits.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
The experimental results are the average of three runs to avoid the random
bias.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
17

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative emrr rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Please refer to Section 5.1.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: / /ttp///eurrpp.ee/ee//ee cc wwtpppcccc/eaaccccccccccccceceee d
Answer: [Yes]
Justification:
This paper conforms, in every respect, with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Please refer to Section 6.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
18

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper poses such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The information has been provided in Appendix A.I.
Guidelines:
existing assets. The authors should cite the original paper that produced the code package or dataset.
The answer NA means that the paper does
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/dataseta has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
19

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
20

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA means that the method development in this research does involve L.MSs as any important, original, or non-standard components.
Please refer to LLM policy http::///neuripe http://p.....ooooooccc//////2220 for what should or should not be described.
Impact Statement
G-Memory introduces structured, hierarchical memory architecture for multi-agent systems (MAS), enabling large language model (LLM)-based agents to store, recall, and reason over past experiences with enhanced task generalization and cooperation efficiency. The broader impacts of this work include advancing the development of scalable and adaptive collective intelligence, with potential applications in long-term robotic planning, real-world decision-making systems, and collaborative Al assistants. However, if the underlying language model is compromised or adversarially manipulated, the memory mechanisms could amplify incorrect reasoning. We urge responsible deployment of this architecture with appropriate safeguards, including continual validation, ad adversarial robustness checks, and alignment with human values.
A Experimental Details
A.1 Dataset Descriptions
In this section, we describe the datasets used in our experiments:
ALFWorld [77] (available at http://allwoorl.ihhbbbbbb io/. MIT license) is a text-
based embodied environment featuring household tasks, where agents navigate and interact with objects via natural language commands.
http//lleeaa/ciieeeeeeoooo Apache-2.0 license) is another text-based embodied environment designed for interac- tive science tasks, Agents must navigate rooms and conduct experiments, testing their ability to perform procedural reasoning and scientific exploration.
ScienceWorts World [78] (available https://gither
PDDL is game dataset from AgentBoard [79] (available at https://gitss com/ Akust-nlyy AgentBoard, Custom properties). comprising a variety strategic games where agents PDDL expressions to complete complex tasks.
HotpotQA [75] (available at https: http://ttttttaaaa/ooee github. 1o/, CC BY-SA 4.0 License)
is multi-hop question answering dataset with strong supervision supporting facts. It evaluates the agent's ability to retrieve and synthesize information, especially through web search tools, for explainable reasoning.
FEVER [76] (available at https://feves ai/dataset/fever .tmll, Creative Com- mons AtributionnSSaarAlliin ShareAlike License) is a mooleedgeintensinn dataset focused on fact verification. Agents must validate claims using web search APIs, making it benchmark for evidence-based reasoning.
Evaluation Metrics. We use exact match accuracy for FEVER and HotpotQA. For ScienceWorld and PDDL., we report the progress rate, and for ALrWorl,, we use the success rate as the evaluation metric.
21

---

A.2 Baseline Setup
In this section, we provide detailed descriptions of each baseline used in our comparison:
The Voyager memory is derived from the Voyager agent [16], where an embodied agent continuously interacts with the Minecraft environment and creates new artifacts. Memory serves as the core driver of the agent's evolution. As Voyager's memory design tailored for a single-agent setting, we adapt it to the multi-agent scenario by implementing agent-speeific history retrieval based on each agent's 's visible dialogue context. Other single- agent memory designs adapted in similar manner.
Voyager:
Memory Bank: MemoryBank [36] mimics anthropomorphic memory behaviors by selec- tively preserving and forgetting information. incorporates memory updating mechanism inspired by the Ebbinghaus Forgetting Curve, allowing the agent to reinforce or discard memory based on temporal decay and the relative importance of stored information.
This memory baseline is based on [19], which includes both raw observational memory and high-level reflective memory. The latter captures abstract thoughts generated by the agent through reflection, providing more structured and conceptualized representation of experience.
Generative:
The memory design originates from MetaGPT [21], focusing solely on inside-trial neemory-information stored internally during the resolution of single task by multiple agents.
MetaGPT-M:
This memory design is adapted from ChatDev [45], which incorporates both inside-trial and cross-trial memory. The inside-trial memory is passed from the central or initiating agent at the beginning of each round to provide guidance based on prior interactions. The cross-trial memory is relatively simple, storing past solutions to previous queries for future retrieval. However, in our task, it does not effectively manage the information-rich inter-agent collaboration.
ChatDev-M:
MacNet-M:
This memory design is adopted from MacNet [46]. where the inside-trial memory consists solely of the final answers generated in the previous round. All non-artifact dialogue contexts, i.e., the interaction trajectories among agents, entirely discarded.
A.3
Multi-agent System Setup
In this section, we detail the setups of our three adopted MAS frameworks, AutoGen, DyLAN and MacNet:
A.3.1 AutoGen
AutoGen [13] is a popular multi-agent orchestration framework, to coordinate interactions among specialized agents for mooblem-solving tasks. Specifically, utilize their A3 Decision Making structure, which is composed of: (1) a Solver Agent, responsible for generating solutions, initialized with the system prompt "You smart agent designed to solve problems.": (2) a Ground Truth Agent, which critically evaluates the solver's output and identifies potential errors based on a reference standard; and (3) an Executor Agent, tasked with translating validated solutions into executable commands. This modular design enables transparent, verifiable, and actionable multi- agent collaboration.
A.3.2 DyLAN
DyLAN [71] is a debate-style framework similar LLM-Debate, but incorporates a more efficient agent-wiss wise early stopping mechanism during multi-turn interactions. DyLAN utilizes an agent selection algorithm based on an unsupervised metric, namely the Agent Importance Score, which identifies the most contributive agents through preliminary trial tailored to the specific task. In our implementation of DyLAN, three agents engage in the debate, while an additional ranker agent evaluates their relative importance.
22

---

A.4 MacNet
MacNet [46] is a representative work that explores decentralized and scalable multi-agent systems. Its key feature lies in the absence of central agent; instead, it introduces edge agents, which are invoked between agent interactions to provide actionable instructions to the next agent based on the previous agent's 's outputs. In our implementation, we adopt the random graph topology from MacNet, shown to be robust across diverse scenarios, and employ five agents in addition to the edge agents.
B
Additional Experiment Results
B.1 RQ1 Results
Tables 2 and 3 present additional experimental results using Qwen-2. i.een-25-22 5-55 and 2wen-2.5-12 5-14b the LLM backbones. Appendix B.1 illustrates the success rate curves on ALFWorld as the number of trials increases, comparing different MAS frameworks combined with various memory architectures. As shown in Figures 6b and 6c, G-Memory consistently enables MAS frameworks to achieve success with fewer trials and leads to higher final performance ceilings.
0.9
0.9
0.9
Mo-merbory
No memory
No memory
ChaiderM
0.8
Chatlev-M
Chablev-M
Generative
Generative
Generative
0.7
Voyager
0,7
Vorager
0,7
Voyager
Memocylank Bank
MemoryBank
MemoryBank
Success Rate
MetaGPT-M
M-Meory
Success Rate
0.6
MetaGPT-M
G-Memory
Success Rate
MetaGPT-M
G-Memocy
0.5
0.4
0.4
0.3
0.3
0.3
0.1
0.1
0.1
0.0
10
20
30
0.0
10
20
0.0
10
20
30
Trajectory Length
Trajectory Length
Trajectory Length
(a) The performance trajectory of (b) The performance trajectory of (c) The performance trajectory of
AutoGen on ALFWorld.
DyLAN on ALFWorld.
MacNet on ALFWorld.
B.2
RQ2 Results
Figure 7 provides additional comparisons of token cost across various benchmarks and MAS frame- works when combined with different memory architectures. Overall, G-Memory incurs only a marginal or no increase in token cost compared to classical baselines such as Generative and MetaGPT-M, while consistently delivering the most significant performance improvements.
B.3 Latency Results
As shown in Table 4, G-Memory incurs only moderate inference overhead even when delivering the most significant performance gains. For instance, on AutoGen+ALFWorld it increases latency by merely 9% compared to the no-memory baseline; (2) in some cases, G-Memory even improves efficiency, e.g., on DyLAN+ALFWorld it leads to a 20% speed-up, as helpful memory cues enable the multi-agent system to reach correct actions more quickly and terminate the interaction earlier.
23

---

Table 2: Performance comparison with single/multi-agent memory architectures five benchmarks.
The underlying LLM backbone is Owen- 5- We highlight the best and second best results.
MAS
Memory
ALFWorld
SciWorld
PDDL
HotpoRQA
FEVER
Avg.
No- nemmmmor
37.31 10:00
p3.49
10.86,0.00
20.26
48.
48.1770.00
28.02-0.00
Vanilla LLM
Voyager
38. to.8s
24.1 10.02
12
19.12
49.68
28.65-0.63
MemoryBank
40.30 +2.00
21.64j1.88
14.36 14.36
+a.50
18.79 11.47
47.666n.61
28.55-0.63
Generative
39.16
16.10
11.37
23.48
52.50
30.52
No-memory
52.99
$0.99
30.27
16.17
33.33 33.33
58.74 74.000
38.30
Voyager
55.22 12.23
26.70,a.67
12.00
34.29 10.96
52.44 16.00
36.13 .2.17
MemoryBann Bank
53.37 po.38
27.33 10:94
14.83 11:34
32.67 10.98
59.45 fu.71
37.53 Jo.77
AutoCien
Generative
62.69 (9.60
y1.45
17.88 88
34.17 10.88
61.25 r2.61
41.49-3.19
MetaGITTMM
55.5222.55
(2.44
17
35.36 (2.00
63.33 p3..33y..00
40.74-2
ChatDev-M
46.27 ys
28.67
13.42 (9.75
31.11 ja..e
61.32
36.16,s
MacNet-M
83.18
31.10
16, 89 ro.72
34.29
58.43
38.78-0.48
G-Nemory (Ours)
67.91 14..22
34.89 6.00
-1.01
37.34 p4.01
64..44a..00
45.10-6.50
No-memory
41.34
29.8440.00
13.56y0.00
24.29
56.23
33.05
Voyager
51.49 -10.3
26.6663.18
10.62
26.23
55.39 10.84
34.08-1
MemoryBank
46.46
26.99
14.10
22.44 11.95
59.21
33.84
DyLAN
COLM 2024
Generative
47.52
(1.55
16.31
26.54
50.19 50.19.6.04 16.04
34.66 34.62-1.58
MetaGPT-M
42.54
30.93
14.47
19.33 14.00
57.22
32,90 1000_...33
ChatDev-M M
39.85 11.49
28.25
7.14 16.42
17.32 ys
50.67 15.00
28.65
MacNet-M
42.48
28.22
14.23
25,12 y0.83
55.34 (0.89)
33,08-o.40
G-Nemory (Ours)
$2.99 11.68
33.81 13.81 15.97
20.71 (7..5
29.33 (s.04)
63.67 (7.44
40.10.2.00
No-memory
44.03
28, 76
13.36y0.00
22.24
55.12
32,70
Voyager
47.01 12.98
28.88 fp.12
11.36_2.00
25.67 13.43
58.78 ra.00
34.34-1
MemoryBank
52.24 18.22
27.86.0.50
13.33
23.97
54.18 (8.94
34.32-1.60
MacNet
Generative
48.51 48.51
31.05
14.04fo.c8
24.49
56.08 10.90
34.83
ICLR 2025
MetaGiPTMM
52.99
29.87
16.58
25.51 43.27
53.88 (1.88_1.24
35.77
ChatDev-M M
44.78
26.44 j2.32
10.19
16.32
56.02
30.75 13.95
MacNet-M
43,55
30.11
12.91
21.77 jo.cz
50.71 p4:41
318. 11.81,00
G-Nemory (Ours)
54.48 10:45
32.23 15.17
17.48
27.53
59.14,-.00
38.17-5.47
B.4
Case Study
B.4.1 Case Study on Insight Graphs
Figure visualizes the high-level insights summarized by G-Memory on the ALFWorld benchmark across different MAS frameworks and LLM backbones. Given that ALFWorld naturally consists of diverse task categories, we further examine how insight nodes corresponding to different task types are interconnected. Overall, we observe dense intra-category connections among insights derived from similar tasks, while also noting the emergence of meaningful inter-category links, reflecting transferable patterns across task domains.
B.4.2 Case Study on Query Graphs
Figures 9 to 11 visualize the query graphs constructed by G-Memory on the ALFWorld, PDDL, and SciWorld benchmarks. Recall that directed edge between two query nodes indicates that the historical trajectory of one query offers useful guidance for the execution of another. We observe emergent clustering patterns, where groups of semantically similar queries form densely connected subgraphs, while sparser inter-cluster edges capture cross-task inspirations. These patterns demonstrate G-Memory's 's ability to effectively organize and relate collaborative experiences through structured memory reasoning.
24

---

Table 3: Performance comparison with single/multi-agent memory architectures five benchmarks. The underlying LLM backbone is Owen-2. 5-14b. results.
We highlight the best and second best
MAS
Memory
ALFWorld
SciWorld
PDDL
HotpotQA
FEVER
Avg.
No-memory
74.63p0.00
46.84-0.00
44.92p0.00
24.49.0.00
63.27 f0.00
50.83
Voyager
76.87
59.00
40.21
31.33-6..4
61.22 [2.05
55.73-4.00
MemoryRank
70.15
14.18-7.34
jo.54
32.65-a.10
64.29 11.02
52.16-1.88
AutoGen
Generative
74.63 10.00
10.57
54.46p9.54
33.21 m8.72
63.27 10.00
56.59
COLM 2024
MetaGPT-M
82.09
58.86
48.99
31,63-7.15
62.27 11.00
56.77-s.0
67.16,7.47
11.83
31.77-7.28
61.28 11.90
22008
MacNet-M
73.65 10.08
(1.70
45.90 45.94
26.72-a.28
64.69 11.62
50.63 (0.3000.000
G-Memary (Ours)
85.82/11.19
60.62 (1.77
15.24
34.61
ys.43
61.54-20.77
No-memory
76.12
53.24
41.83 41.83
30.61
63.34 10.00
53.03-0.00
Voyager
72.39.a.78
58,93
48 p0.54
30.71 30.71
65.31 (1.31p1.000 91.97
88.18-2.00
MemoryBank
76.87 10.75
57.92
39.65 ja.18
29.59, 1.00
63.25 (0.09
53.46-o.44
DyLAN
Generative
11.79
61.522a.28
46.60 46.69
31.33-0.72
61.39 11.95
55.77-2.74
COLM 2024
MetaGPT-M
79.10
61.29
49.75
28.61
64.1 your7
56.57-3.55
14.63
54.03-0.79
(4.44
30.67-0.00
11.00
53.20-o011
MacNet-M
72.77
52.22 _1.02
42.98 11.15
29.22
62.66 jouss
51.98 1.06
G-Menory (Ours)
61.34
64.68 (11.44
(1.12
34.63-u 0.00
66.66,a.32
59.69 1199-6..00
No-memory
58.21
52.21 ro.00
41.74
28.60
64.65 10.00
49.08-0.00
Voyager
63.43 (5.22
60.24 60.24.8.0
43.95
29.67-1.0r
62.24 pu:1
51.91-9.4
MemoryBank
62.21 14.00
55.52
18.26
26.53_2.0r (2.07
65. 10.57
49.55-o.47
MacNet
Generative
73.13
60.83
y4.00
30.53-1.90
65.31 10.96
54.76-s.ds
2025
MetaGPT-M
70.43-12.22
59.70-7.40
42.34
26.26,2.84
66.33 11.68
53.01 -3.98
ChaiDev-M
68.66
45,98
42.19
29.49
59.18
49.10
MacNet-Ne
60.45p2.24
51.14 51.14_1.07
39.22 19.52
28.77-0.17
62.42 ja.23
48.40 00,00
G-Memory (Ours)
61.74-9.5a
45.76
32.33 -2.33
70.33
57.85
Table Performance (%) and latency (s) comparison of different memory mechanisms on AutoGen and DyLAN frameworks, along with ALFWorld and SciWorld benchmarks.
AutoGen
DyLAN
Method
ALFWorld
SciWorld
ALFWorld
SeiWorld
Perf.
Lat.
Perf.
Lat.
Perf.
Lat.
Perf.
Lat.
No-memory
77.61
19204
54.59
16953
56.72
33520
55.38
32408
Voyager
85.07
21754
62.36
16650
66.42
29628
62,83
31633
MemoryBant
74.96
15492
53.11
10104
55.22
31813
54.74
32925
Generative
86.36
20682
61.19
16674
67.91
31010
64.16
34038
MetaGPT-M
81.34
16021
61.91
15853
69.40
22936
62.37
32049
ChatDev-M
79.85
22347
50.96
12904
46,27
29739
53.35
33111
MacNet-M
76.55
21089
55.44
16882
53,44
24991
54.32
34815
G-Memory (Ours)
88.81
21113
67.40
17326
70.90
26726
65.64
33447
C Prompt Set
Query Relevance Filtration
cack_relevency_eye systee__proteeeeeeeeeee between tuo pieces of text tes www.rrelllaacc.ssrrprree.e
."*You an agent designed score
given succaastul case you successfully complete the task. Then you will be ongoing task. Do not sunnarize these tuo canes, but rather evaluate hav relevant and helpful ongoing
"*"Yau
1-10.
Success Case:
(trajectery) Ongoing task: (query.scenario?
25

---

FEVER AutoGen
HotpotQA + AutoGen
Temrrttee
Vigagar Da Gegesce lakess)
GoMaan Gpand Csue tokanal
(L56 (kkmm)
Token Cost (e+6)
(177-00 (oben)
MouGPT
O-Kencry (Ouzg)
Token Cost (e++))
Metacit Bleoo tobrns
Genaraije (S Ligi (olieta)
14-3
Mattory Sunk
XemoryTankk (1semut
Chalkke fs ioken))
9.0
CharDer U.Sevol talons)
14.0
LLe d6 tdbons)
1.8
Noettemory Lugeant mitrn))
13.9
$4
66
Performance (%)
Performance (%)
PDDL + DyLAN
SeiWorld + AutoGen
16.5
Metsciy hybone)
(Bae=od Miken))
Token Cost Cost(e60) (e+6)
16.4
(Our)) Tr-geeu boitin))
Token Cost (e+6)
7.3
Gareration sokens))
Giperiive L.ge--nnlpknn
6.6
Vorager Vo-eeeoppoken
Memsc Bank
tolicts)
Viriager te 07 takend)
16.1
Chatlker (Kaginon lukens)
4.9
16.0
CleiDo
No-memeey
Memory/ennk
44
40
42
44
46
50
90
85
30
35
40
45
50
55
60
Performance (%)
Performance (%)
Figure 7: Cost analysis of G-Memory. We showcase the performance versus the overall system token cost when combined with different memory architectures.
Score: nnP
Graph Sparsifier
extract_true_tree Ser.cttro__rrr__yssteerrrrr "*"You are an agent skilled at extracting key points.
Given task and auccesaful execution trajectory, your job is to identify the steps steps. une
complete while filtering inportant
Note:
Strictly
original trajectory; absolutely steps trajectory should be added.
succeseful trajectory, attention to actions that correspond to "Nothing happens" observations, actions are likely incorract. Filter that each step the finest granularity.
may sone incorrect steps Pay
actions me.
You need
You should strictly follow the output format in the exanple.
*# Here is the task:
(task)
26

---

Teek
Selees diligine
140, H4.
(a) Insight graph gpt-4o-mini +Mac Ferrdl.FWorld.
(b) Insight graph spt-4o-mini +Dy LAN+ALFWorld.
pil dels
HAL Houutes plnc
Cas
AALue Sea
(c) Insight graph on Qwen-7b +Mac-
(d) Insight graph LAN+AL
on
Qwen 7b
+Dy-
FettALFWorrd.
Selee Delgiede
(c) Insight graph Qwen-14b +Auto- ALordd.
(f) Insight graph L.Nid
Qwen-14b +Dy-
Figure 8: Visualizations of insight graphs across different LLM backbones, MAS, and benchmarks.
*** Trajectory (trajectory)
NAN Output
...
The prompt below is partially adapted from [42]. We would like to express sincere gratitude for their valuable implementation.
27

---

Tasks Graph
500
Figure 9: Query graph optimized from ALFWorld dataset.
Tasks Gruph
Figure 10: Query graph optimized from SciWorld dataset.
Inisght Summarization Function
watessss_ssttt_prrrrrpo You aze an analysis-drive agent focused on learning from experience. You will be provided with:
n**
A failed trajectory and its autcone, aucceasful trajectory completing
task.
Your task is to analyze both trajectories and generate clear, actionable insights. failed trajectory nissed the
Your insights should highlight succeseful one addressed or avoided these pitfslls.
# Requirenents:
Al1 insighta muat derived directly from contrasting the two trajectories.
28

---

Tasks Graph
Figure 11: Query graph optimized from PDDL dataset.
not speculate or introduce stepa not supported by aucceasful exanple.
Focus s*concrete behavioral or strategic differenceg4* between the two Keep each insight concise and impactful.
Output Fornst:
inmodiately
No introduction explanation.
Use this exact format:
Ineight
Insight
Ineight ...
dearn_lessone user peerrppropp...m pronpt_ccupare
Successful trajectory
(true_traj)
## Failed trajectory trajectory (ralse_traj?
Your output:
learn aassons_aysto__progg. an analyais driven agent focuaed learning from auccess. provided with set of successful trajectories that completed similar task.
Your goal is analyze these successful examples and
clear, actionable insights that capture what contributed to their success. These insighta will agenta working siniler
guidance for
Requirements:
All insights be grounded patterne strategies observed the succeseful trajectories.
speculate introduce staps reflected prorided examples.
Focus on common behaviors, strategies, or decisions that consiatently led to positive
Keep each insight concise, specific, and impactful.
Output Fornst:
Start inmediately with a numberod list. introduction explanation.
Use this exact format:
Ineight
29

---

Insight
Ineight ...
tearn_lessen ptsppsrorrrrpppp..e ".. Successful trajactorys (true_traje)
Your output:
à merge rules prompt
wtt.rrrleeaattta.rroo.... ""*You are an agent skilled at summarizing and insighte kere previously
distilling insights. extracted from similar tasks. These insights contain redundancy or overlap.
given
Your job is to *energe and consolidate similar insighte*, and output refined version that seclear, actionable, and concise**.
All narged insighta rrmust be based strictly on the given inputs*t. You are not allosed nake up*s or infer any nev information.
The output should be easy to read and follov.
Output Format:
Start your reapanse diractly with the numbered 1ist, no preamble or explanations
Each insight abould be short santance.
Use the following fornat exactly:
Insight
Insight
Insight ...
nergo_ wwwgeeruleee.. pepprrule__ueeeee seaiee__see__pe prompt NO Here the current insights that need be merged: (current_rules)
## Please consolidate and zewrite then into ean more than (Limeted,number) insights*e.
As the summarizing agent, renove reduadancies, combine similar ideas, and ensure clarity.
output:
Customizing Memory for Agents
str
You are thoughtful and context-avare agent. You will provided with successfully ezecuted trajectory, specific agent terole*s, and of general insights** applicable across all roles.
Your
**parsenalized insights that are specifically tailored to the given role and ita trajectory. These personalized insights aligning with their unique background, responsibilities, and perspective.. an understanding
general insights**
help agent improve
performance by
sure your output actionable. rolevrelevant advice.
promotes
NOTE Your output strictly follow 1. Insight 1
Ineight
3. Insight
*** Trajectory (trajectory)
Agent 'a Role:
30

---

(role)
*a* General Insights:
(insights)
8## Your Output (Personalized Insights for This Role):
D Discussion with Related Works
In this section, further discuss the relationship between G-Memory and several recent agent mem- ory frameworks. For A-Mem [60], while both A-Mem and G-Memory aim to enhance the memory capabilities of L.M agents, they differ in two key aspects. First, A-Mem is tailored for single-agent scenarios, whereas G-Memory designed for processing MAS's lengthy and nuanced interaction trajectory. Second, A-Mem emphasizes atomic memory construction for chathot-style interactions, while G-Memory focuses on distilling reusable strategies from collaborative task execution, where fine-grained atomicity is neither required nor beneficial. For Memo [61], although also employs a graph-based structure, it remains within the chatbot paradigm. Its graph is closer to knowledge graph, where nodes represent factual entities and edges represent relations, fundamentally differing from G-Memory's centric memory graphs that encode trajectories, decisions, and coordination patterns across agents.
31


Reviews:
[{"review_id": "AncRE1xHOL", "reviewer": "Reviewer_Lxdg", "summary": "In this paper, the authors propose a three-layer graph memory, including insight graph, query graph, and interaction graph. The proposed G-memory enables agents to store, recall, and reason over past experiences more effectively.", "questions": "question 1: see weakness 1\n\nquestion 2: How about the performance of G-memory in more domains, such as on WebArena dataset or GAIA dataset? \n\nquestion 3: How about the latency of the G-memory; In the paper, the authors present the token cost, but the latency is not mentioned.", "limitations": "None", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths: \n\n1. The design of the three layers hierarchical memory framework is novel. It captures the high-level insights, intermediate queries, and interaction between agents. \n\n2. The authors conduct experiment across multiple domains, including knowledge reasoning, embodied action, game. Experimental results demonstrate the effectiveness of the proposed method.\n\n3. The authors also conduct a series of ablation study to validate the performance.\n\nWeakness:\n\n1.  Some notations are not very clear. Such as:\n\n      a. What is one epoch in MAS? In debate case, one round of agent speaking is one epoch; but in solely task solving case, what does one epoch mean?\n\n      b. In equation (5), $\\mathcal{N}^{+}$ is? Is it out-neighbor?\n\n2. In the experiment section, it is expected to adopt more tasks to validate the effectiveness of the proposed method.", "was_revised": true, "final_justification": "In this paper, the authors propose a novel three-layer graph memory. During the rebuttal, they clarify previously missing concepts, which has increased my confidence in my score. My main concern was the performance of the proposed memory system across different tasks. The authors addressed this by providing additional experiments that demonstrate its effectiveness across various tasks. Regarding my second concern about latency, the authors balance performance gains with inference overhead, showing that G-Memory outperforms existing methods. Therefore, I am raising both my score and my confidence level.", "rebuttal": "Thank you immensely for your time and efforts, as well as the helpful and constructive feedback! Here, we give point-by-point responses to your comments.\n\n\n\n---\n\n> `Weakness & Question 1`: Some notations are not very clear. What is one epoch in MAS? In debate case, one round of agent speaking is one epoch; but in solely task solving case, what does one epoch mean?\n\nThank you for pointing out this issue. We would like to respectfully clarify the following:\n\n* In general, an *epoch* refers to a single iteration of the agent interaction process, typically defined by one full cycle of decision making or action execution within the multi-agent workflow. More specifically:\n\n  * In frameworks such as **LLM-debate** and **DyLAN**, an epoch corresponds to one complete round of agent responses, as you rightly noted.\n  * In coding frameworks, such as  **MetaGPT** and **EvoMac**, an epoch denotes one round of code generation, which often concludes with a reviewer or tester evaluating the output and feeding back signals for the next round.\n  * In **AutoGen**, where a single round of interaction typically produces a final action, the number of epochs is naturally set to 1.\n\n* In Eq. (5), $\\mathcal{N}^+(Q_j)$ and $\\mathcal{N}^-(Q_j)$ denote the out-neighborhood and in-neighborhood of node $Q_j$, respectively.\n\nWe greatly appreciate your careful reading and will incorporate these clarifications into the revised manuscript.\n\n\n---\n\n> `Weakness & Question 2`: In the experiment section, it is expected to adopt more tasks to validate the effectiveness of the proposed method; ... such as on WebArena dataset or GAIA dataset?\n\nThank you for your valuable suggestion. To further demonstrate the generalizability and effectiveness of G-Memory across diverse task settings, we integrate G-Memory into the SmolAgent framework and evaluate its **performance, API cost, and latency** on the GAIA benchmark across multiple levels:\n\n\n|Level|Metric|Vanilla|Voyager|G-Memory|\n|-|-|-|-|-|\n|Overall|Perf.|39.80|36.71|44.80|\n|    |API Cost|$18.1|$25.8|$24.0|\n|    |Latency|8.7h|10.3h|11.4h|\n|Level 1|Perf.|52.80|50.60|56.60|\n|    |API Cost|$5.7|$7.3|$7.0|\n|    |Latency|1.8h|2.0h|2.1h|\n|Level 2|Perf.|34.60|42.30|44.18|\n|    |API Cost|$8.9|$11.3|$9.9|\n|    |Latency|4.9h|5.6h|5.7h|\n|Level 3|Perf.|16.70|16.70|23.07|\n|    |API Cost|$3.5|$7.2|$7.1|\n|    |Latency|2.0h|2.7h|3.6h|\n\nThese results suggest that G-Memory consistently improves agent performance across all task levels, with only marginal increases in latency and API cost. We will supplement the experiment into the revised manuscript to further clarify G-Memory’s scalability and practical utility.\n\n\n\n---\n\n> `Question 3`: How about the latency of the G-memory; In the paper, the authors present the token cost, but the latency is not mentioned.\n\nTo address your concern, we provide additional latency measurements below:\n\n|MAS|Memory|ALFWorld||SciWorld||\n|-|-|-|-|-|-|\n|||Peformance|Latency (s)|Performance|Latency (s)|\n|AutoGen|No-Memory|77.61|19204|54.59|16953|\n||Voyager|85.07|21754|62.36|16650|\n||MemoryBank|74.96|15492|53.11|10104|\n||Generative|86.36|20682|61.19|16674|\n||MetaGPT-M|81.34|16021|61.91|15853|\n||ChatDev-M|79.85|22347|50.96|12904|\n||MacNet-M|76.55|21089|55.44|16882|\n||**G-Memory (Ours)**|**88.81**|21113|**67.40**|17326|\n|DyLAN|No-Memory|56.72|33520|55.38|32408|\n||Voyager|66.42|29628|62.83|31633|\n||MemoryBank|55.22|31813|54.74|32925|\n||Generative|67.91|31010|64.16|34038|\n||MetaGPT-M|69.40|22936|62.37|32049|\n||ChatDev-M|46.27|29739|53.35|33111|\n||MacNet-M|53.44|24991|54.32|34815|\n||**G-Memory (Ours)**|**70.90**|26726|**65.64**|33447|\n\n As shown, (1) **G-Memory incurs only moderate inference overhead even when delivering the most significant performance gains**. For instance, on AutoGen+ALFWorld, it increases latency by merely 9% compared to the no-memory baseline; (2) **in some cases, G-Memory even improves efficiency**, e.g., on DyLAN+ALFWorld, it leads to a 20% speed-up, as helpful memory cues enable the multi-agent system to reach correct actions more quickly and terminate the interaction earlier."}, {"review_id": "jbGw9dGRSV", "reviewer": "Reviewer_PY4E", "summary": "This paper introduces G-Memory, a hierarchical memory system for Large Language Model powered multi-agent systems (MAS). The authors identify that existing MAS memory architectures are oversimplified, lacking nuanced inter-agent collaboration tracking and cross-trial customization, hindering self-evolution. G-Memory addresses this via a three-tier graph hierarchy: Insight Graph, Query Graph and interaction graph. Upon a new query, G-Memory performs bidirectional traversal (upward for insights, downward for relevant interaction subgraphs) to provide role-specific memory cues. After task execution, the graphs evolve by assimilating new experiences. Evaluated across five benchmarks and three MAS frameworks, G-Memory improves success rates significantly. The system operates as a plug-and-play module, requiring no framework modifications.", "questions": "1. How are the three-tier graphs initialized during the bootstrapping process?\n2. Are all new queries incorporated into the memory? If not, what criteria filter queries for updates?\n3. How is the correctness of insights verified, and does insight quality impact G-Memory's performance?\n4. What safeguards prevent adverse effects from noisy/false insights or action trajectories (e.g., hallucinated lessons)?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. Novel architecture: The three-tier graph hierarchy  innovatively captures multi-granularity memory needs, enabling both high-level reasoning  and fine-grained collaboration recall .\n2. Significant Performance Gains: G-Memory achieves state-of-the-art improvements across diverse tasks.\n3. Resource Efficiency: Despite its advanced capabilities, G-Memory incurs only marginal token overhead\n4. Seamless Integration: As a plug-and-play module, G-Memory integrates into mainstream MAS frameworks without structural modifications, demonstrating practical deployability.\n\nWeaknesses\n1. Cold-Start Graph Initialization: The paper lacks details on bootstrapping the three-tier graphs from scratch, which is critical for real-world deployment with no prior collaboration data.\n2. Ambiguous Graph Update Mechanisms: The update process for assimilating new experiences requires clarification:\n3. Safety and Robustness Gaps: Potential risks from retrieving misleading insights/interactions are underexplored, despite the impact statement noting amplification of incorrect reasoning.", "was_revised": true, "final_justification": "After carefully reading the rebuttal, I find that the authors have addressed most of my concerns. I believe the paper's novelty and quality meet the conference's acceptance criteria. Therefore, I rate this paper as 5 (accept).", "rebuttal": "We sincerely thank you for the thoughtful and constructive reviews of our manuscript! Based on your questions and recommendations, we give point-by-point responses to your comments and describe the revisions we made to address them.\n\n---\n\n> **`Weakness & Question 1`: Cold-Start Graph Initialization**\n\n\nThank you for this important question! We respectfully detail the cold-start process for each memory layer as follows:\n\n* **Query Graph**: Each node represents a task query, and a new node is added upon the completion of each task. Retrieval in this graph does not involve a cold-start process—each query retrieves similar past ones via Eq. (4). To reduce early-stage noise caused by sparse or weakly correlated data, we apply a hard similarity threshold; when no historical query exceeds this threshold, no retrieval occurs, effectively suppressing spurious early matches.\n* **Insight Graph**: To ensure meaningful insight-level retrieval, we delay G-Memory’s insight-based querying until 20 queries have been processed. Nevertheless, the accumulation of insight-level data begins from the very start.\n* **Interaction Graph**: This graph evolves in sync with the query graph. If relevant past queries are retained (i.e., not filtered out due to low similarity), the corresponding sparsified interaction graphs are available for retrieval. If none are deemed relevant, no retrieval is triggered.\n\nOverall, G-Memory is designed to be **robust to cold-start issues** and can efficiently operate in new domains with minimal prior data. We sincerely thank you again for your thoughtful feedback and will include these clarifications in the revised manuscript.\n\n\n---\n\n> **`Weakness 2`: Ambiguous Graph Update Mechanisms: The update process for assimilating new experiences requires clarification**\n\n\nTo further address your concern, we provide a more intuitive and detailed explanation of how each layer of the graph is updated:\n\n* **Query Graph**: As described on Line 214, each encountered query is directly added as a node in the query graph. Additionally, previous queries that contributed to solving the current one (e.g., by providing helpful insights) are linked via directed edges. This construction captures not only semantic similarity but also *temporal and causal relationships*, representing how past queries inspire future ones.\n* **Insight Graph**: Newly distilled insights from each incoming query are added to the insight graph. Edges are drawn from previously leveraged insights that contributed to solving the current query. Compared to query-level connections, this high-level topology provides a more precise mapping of transferable collaborative knowledge, enabling MAS to quickly locate relevant prior experience. G-Memory also performs periodic consolidation of the insight graph: First, recent queries are grouped into semantically related clusters $\\{\\mathcal{C}\\_l\\}$. For each cluster, all corresponding insights are retrieved from the graph: $\\mathcal{I}\\_l = \\Pi\\_{\\mathcal{Q} \\to \\mathcal{I}}(\\mathcal{C}\\_l)$, which is then synthesized into a set of more general insight, $\\iota_l^\\text{merged}$, using the function $\\mathcal{J}$ in Eq. (10) that merges their textual content and supporting query sets: $\\iota\\_l^\\text{merged} = ( \\mathcal{J}(\\{\\kappa\\_k \\mid (\\kappa\\_k, \\Omega\\_k) \\in \\mathcal{I}\\_l\\}), \\bigcup\\_{\\iota\\_k \\in \\mathcal{I}\\_l} \\Omega\\_k )$.\nFinally, the insight graph is atomically updated by replacing the original set of insights $\\mathcal{I}\\_l$ with $\\iota\\_l^\\text{merged}$. This is explicitly implemented in the `merge_insight` function of `/mas/memory/mas_memory/GMemory.py` in our provided codebase. This iterative refinement prevents knowledge fragmentation and ensures the long-term coherence of the system's memory.\n* **Interaction Graph**: The interaction history associated with each query is stored in the interaction graph for future reference.\n\n\n\n\n---\n\n> **`Weakness 3 & Question 4`: \nSafety and Robustness Gaps: Potential risks from retrieving misleading insights/interactions are underexplored, despite the impact statement noting amplification of incorrect reasoning.**\n\n\nThank you for your insightful observation! We would like to respectfully highlight that the current design of G-Memory includes **at least two mechanisms to mitigate the potential negative impact of misleading insights/interactions**:\n\n* The consolidation process of insight graph will filter the useless insights and emphasize the impactful ones, as described in our response to  `Weakness 2`.\n* As the query and insight graphs evolve over time, correct and helpful insights/interactions naturally lead to more successful task executions, thereby accumulating additional edge connections. This process positions them as denser and more central nodes in the graph, making them more likely to be retrieved, as illustrated by nodes 45 and 55 in Figure 10. In contrast, misleading elements tend to have limited impact and remain on the periphery of the graph (e.g., nodes 36 and 37 in Figure 10). This update dynamic implicitly prioritizes reliable knowledge while suppressing noise.\n\n\n\n\n---\n\n> **`Question 2`: Are all new queries incorporated into the memory? If not, what criteria filter queries for updates?**\n\n\nBoth the **query** and its associated **interaction** history are incorporated into the memory. However, the **insights** distilled from these interactions may be dynamically merged or pruned, as described in our response to  `Weakness 2`.\n\n\n---\n\n> **`Question 3`: \nHow is the correctness of insights verified, and does insight quality impact G-Memory's performance?**\n\nThank you for your thoughtful question! We agree that the quality of insights can indeed influence the performance of G-Memory. While G-Memory does not explicitly evaluate the correctness of each insight (which is often impractical, as even human-generated insights may appear correct in the short term but prove inaccurate over time), our graph update mechanism implicitly addresses this issue. Specifically, helpful and positively contributing insights naturally accumulate more edge connections and are thus positioned as **denser and more central nodes** in the graph, while misleading ones are gradually marginalized. Additionally, the consolidation process of the insight graph further prunes or merges low-quality or redundant insights."}, {"review_id": "wgapc6hUpH", "reviewer": "Reviewer_o31T", "summary": "This paper introduces G-Memory, a hierarchical memory system designed to enhance the self-evolution capabilities of multi-agent systems (MAS) by addressing the limitations of existing memory architectures. G-Memory organizes complex inter-agent interactions into a three-tier graph structure to enable efficient retrieval of high-level insights and fine-grained collaboration trajectories. The system improves task performance on different kinds of tasks. Experiments demonstrate its effectiveness as a plug-and-play module, facilitating adaptive and role-specific memory support for agents.", "questions": "See above weaknesses", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n1. The writing is clear and engaging. Even though the memory system is complex, the authors explain it in a way that’s easy to follow. The paper flows well, the key points stand out, and the figures help a lot. It’s a solid, well-structured read.  \n\n2. The three-tier memory design is intuitive. It doesn’t just store past interactions but organizes them in a way that makes sense. The insight, query, and interaction graphs help agents pull out both high-level lessons and fine-grained details when needed.  \n\n3. It improves performance without blowing up costs. A 20%+ boost in some tasks while keeping token usage reasonable is impressive. A lot of memory systems add overhead, but this one seems efficient.  \n\n4. The experiments cover all the bases. They test performance, break down costs, run ablation studies, and include case studies. It’s thorough and convincing.  \n\n**Weaknesses**\n\n1. What about the cold-start problem? To my understanding, the system needs some initial memory to work well, but the paper doesn’t explain how that’s built or how sensitive performance is to the order of tasks. If early tasks feed bad data into memory, does that mess things up later? Some analysis here should be done.  \n\n2. How exactly are token costs calculated? Is it just completion tokens, or does it include prompts? And does the size of memory affect this? The memory pool becomes larger gradually, what's the impact on the cost? At the initial stage, the cost could be few, but the cost might be higher later, though the argtop operators could make the cost insensitive to the total volume of the memory pool. The details should be introduced.\n\n3. Why skip the 0-hop baseline in Figure 4? The hop expansion is supposed to fix noisy similarity matches, but without testing a no-hop version, it’s hard to tell if expanding actually helps or just adds more noise. Also, the motivation for hop expansion in line 173 \"the similarity may be only superficial or noisy\" seems weird to me. By using hop expansion, more nodes could be considered, and maybe more noise is introduced.\n\n4. The ranking/sparsification part feels handwavy. Eq. 7 mentions it, and the appendix only includes some prompts. How exactly does the LLM decide what to keep or drop? What's the effectivenss? A clearer explanation would make it easier to replicate.  \n\nOverall, it’s a good paper to me, but a few more details, especially on initialization and implementation, would make it even better. If authors could address my concerns, I could consider further raise my score.", "was_revised": true, "final_justification": "The authors have addressed my concerns.", "rebuttal": "We would like to express our deepest respect for your meticulous review! We can genuinely sense that you have dedicated considerable time to thoroughly reviewing our manuscript and providing very specific insights and feedback. We must acknowledge that this is one of the most enlightening and helpful reviews we have received in recent years! In response to your efforts, we have carefully prepared a point-by-point reply:\n\n---\n\n> **`Weakness 1`: What about the cold-start problem?**\n\n\nThank you for this important question! We respectfully detail the cold-start process for each memory layer as follows:\n\n* **Query Graph**: Each node represents a task query, and a new node is added upon the completion of each task. Retrieval in this graph does not involve a cold-start process—each query retrieves similar past ones via Eq. (4). To reduce early-stage noise caused by sparse or weakly correlated data, we apply a hard similarity threshold; when no historical query exceeds this threshold, no retrieval occurs, effectively suppressing spurious early matches.\n* **Insight Graph**: To ensure meaningful insight-level retrieval, we delay G-Memory’s insight-based querying until 20 queries have been processed. Nevertheless, the accumulation of insight-level data begins from the very start. Nevertheless, G-Memory is not sensitive to the choice of cold-start threshold. We present a sensitivity analysis below, showing that varying the minimum delay (i.e., the number of queries before retrieval is enabled) has a limited impact on performance: \n\n*Table A. Sensitivity study of delay queries using GPT-4o-mini.*\n|Delay|5|10|20|30|\n|-|-|-|-|-|\n|ALFWorld + AutoGen|85.07|87.01|88.81|87.50|\n|ALFworld + DyLAN|69.40|70.90|70.90|71.14|\n|FEVER + AutoGen|65.48|65.48|66.24|65.27|\n|FEVER + DyLAN|62.24|63.69|64.22|66.38|\n* **Interaction Graph**: This graph evolves in sync with the query graph. If relevant past queries are retained (i.e., not filtered out due to low similarity), the corresponding sparsified interaction graphs are available for retrieval. If none are deemed relevant, no retrieval is triggered.\n\nOverall, G-Memory is designed to be **robust to cold-start issues** and can efficiently operate in new domains with minimal prior data. We sincerely thank you again for your thoughtful feedback and will include these clarifications in the revised manuscript.\n\n\n---\n\n> **`Weakness 2`: How exactly are token costs calculated? And does the size of memory affect this?**\n\n\nPlease allow us to respectfully respond to the questions one by one:\n\n**(1) How exactly are token costs calculated?**\n The token cost demonstrated in Figures 3  and 7 is the sum of both prompt token and completion token. \n \n**(2) Does the size of memory affect this? The memory pool becomes larger gradually, what's the impact on the cost?**\n\nWe would like to clarify that all major operations within G-Memory are designed to **scale efficiently** without significantly increasing token cost as the memory pool grows:\n\n \n|Memory Layer|Formula|Process|LLM invocation|Detail|\n|-|-|-|-|-|\n|Query-level|Eq. (4)|Retrieval|No|based on embedding similarity|\n||Eq. (9)|Update|No|-|\n|Interaction-level|Eq. (5) and (7)|Retrieval|Yes|The LLM receives $\\tilde{\\mathcal{Q}}^S$ as input, i.e., the $k$ queries retrieved by Eq. (4) and their 1-hop neighbors. Assuming an average node degree $\\overline{D}$, the total input size is approximately $k \\cdot \\overline{D}$, which does not increase significantly with the number of nodes.|\n||-|Update|No|\n|Insight-level|Eq. (6)|Retrieval|No\n||Eq. (10)|Update|Yes|This process takes the new query and its interaction graph as input. The input size remains roughly constant.\n\n\nIn summary, G-Memory can scale its memory pool without incurring excessive token cost, making it suitable for long-horizon multi-agent tasks.\n \n\n\n---\n\n> **`Weakness 3`: Why skip the 0-hop baseline in Figure 4?**\n\nTo address your concern, we have supplemented the sensitivity analysis in Figure 4(a) with additional results for 0-hop:\n\n*Table B. Hop expansion experiment using qwen-2.5-14b.*\n| Benchmark | MAS| 0hop | 1hop | 2hop | 3hop|\n|-|-|-|-|-|-|\n| ALFWorld|AutoGen| 81.34| 85.82 | 79.85 | 79.85 |\n| PDDL|AutoGen |46.58 |55.24 | 49.79 | 56.31 |\n| FEVER |AutoGen | 69.70 |71.43 | 72.45 | 71.71 |\n| ALFWorld| DyLAN | 75.30 |81.34 | 78.36 | 77.61 |\n| PDDL |DyLAN |48.54 |51.12 | 49.58 | 44.94 |\n| FEVER |DyLAN| 65.31 | 66.66 | 63.27 | 64.29 |\n\nThese results demonstrate a trade-off between the receptive field and noise as the hop size increases. With fewer hops (e.g., 0-hop), the retrieved queries may be too sparse or uninformative. Conversely, expanding to larger hops may introduce excessive noise. We observe that **1–2 hops strike a reasonable balance**.\n\nTherefore, we agree that “the similarity may be only superficial or noisy” applies both when the hop expansion is too limited or too broad. Thank you again for your insightful comment! We will clarify this point in Section 5.4 of the revised manuscript.\n\n---\n\n> **`Weakness 4`: The ranking/sparsification part feels handwavy.**\n\nTo address your concern, we will humbly (1) provide a more fine-grained explanation of the sparsifier, and (2) empirically validate its effectiveness.\n\n\n**（1）Detailed Process**\n\nThe $\\mathcal{S}\\_\\text{LLM}$ operator distills a verbose interaction graph $\\mathcal{G}\\_\\mathsf{inter}=(\\mathcal{U}, \\mathcal{E}\\_\\mathsf{u})$ into its essential reasoning path relative to a new query $Q$. This is performed in two main stages: **(1) Node Summarization**, where an LLM function condenses the textual content $m_i$ of each utterance node $u\\_i = (\\mathcal{A}\\_i, m\\_i)$, producing a summarized node set $\\mathcal{U}'$ while preserving the original graph structure. This yields an abstracted graph $\\mathcal{G}\\_\\text{sum} = (\\mathcal{U}', \\mathcal{E}_\\mathsf{u})$, where $\\mathcal{U}' = \\\\{ (\\mathcal{A}\\_i, \\text{LLM}\\_\\text{sum}(m\\_i)) \\mid (\\mathcal{A}\\_i, m\\_i) \\in \\mathcal{U} \\\\}$; **(2) Graph Extraction** \nSubsequently, the abstracted graph $\\mathcal{G}\\_\\text{sum}$ is then provided to $\\mathcal{S}\\_\\text{LLM}$ and produces $\\hat{\\mathcal{G}}\\_\\mathsf{inter}(\\hat{\\mathcal{U}}, \\hat{\\mathcal{E}}) = \\mathcal{S}\\_\\text{LLM}(\\mathcal{G}\\_\\text{sum}, Q)$.\n\n**(2) Empirical Evalution**\n\nWe further conducted an ablation study on this component, and the results are presented below:\n\n*Table C. Ablation study of sparsifier. The backbone LLM is gpt-4o-mini.*\n|MAS|Variant|ALFWorld|SciWorld|PDDL|\n|-|-|-|-|-|\n|AutoGen|w/ $\\mathcal{S}_\\text{LLM}$|88.81|67.40|27.77|\n||w/o $\\mathcal{S}_\\text{LLM}$|86.36|62.36|24.56|\n|DyLAN|w/ $\\mathcal{S}_\\text{LLM}$|70.90|65.64|18.95|\n||w/o $\\mathcal{S}_\\text{LLM}$|68.13|64.16|16.09|\n\nThese results confirm that, providing interaction memory in a more concise manner is conducive for multi-agent systems to better leverage previous experience."}, {"review_id": "3oFaNDruja", "reviewer": "Reviewer_adrQ", "summary": "This paper introduces G-Memory, a novel hierarchical and agentic memory architecture for Large Language Model (LLM)-driven Multi-Agent Systems (MAS). It aims to fill the gap of sophisticated memory mechanisms in MAS, which are currently limited compared to single-agent systems. G-Memory proposes a three-tier memory hierarchy: Insight Graph, Encodes high-level, generalizable lessons from past collaborative experiences. Query Graph, Stores task-specific meta information and tracks relationships between queries. Interaction Graph, Contains fine-grained, role-specific utterances and interactions for each query/task.Upon a new query, G-Memory performs efficient bi-directional retrieval, providing each agent with high-level insights and relevant historical trajectories tailored to their role. The system updates all memory levels after each trial, supporting self-evolution and improved performance. Empirical results show significant performance gains across various benchmarks, frameworks, and LLMs, without excessive resource overhead. The framework is modular and plug-in, compatible with different LLM-based MAS designs.", "questions": "- How does the memory graph scale with increasing agents/queries/tasks?\n   - Are there mechanisms for automatic pruning or compression of obsolete information?", "limitations": "The paper discusses generalizability and suggests future evaluation on more diverse domains.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths: \nThe authors clearly identify the shortcomings of existing MAS memory systems, contrasting them with the requirements of MAS. The hierarchical design is theoretically motivated, practically justified, and systematically implemented. The cross-agent, multi-level approach with role- and trial-adaptive retrieval is novel.\n\nWeaknesses: The paper admits that tasks are limited to five benchmarks, mainly QA, embodied action, and games. Generality to other domains is unproven. Some lower-level details, such as precise prompt engineering and insight validation, rely on in-LLM processing. The robustness of these steps is not deeply analyzed.", "was_revised": true, "final_justification": "Thanks to the author for the reply, I'll keep my score", "rebuttal": "We sincerely thank you for your careful comments and thorough understanding of our paper! Here we give point-by-point responses to your comments and describe the revisions we made to address them.\n\n\n\n---\n\n> **`Weakness 1`: The paper admits that tasks are limited to five benchmarks, mainly QA, embodied action, and games. Generality to other domains is unproven.**\n\nThank you for your valuable suggestion. To further demonstrate the generalizability and effectiveness of G-Memory across diverse task settings, we add the GAIA benchmark, which includes tasks like **file reading, web browsing, coding, and multimodal understanding**. We integrate G-Memory into the SmolAgent framework and evaluate its **performance, API cost, and latency**:\n\n\n|Level|Metric|Vanilla|Voyager|G-Memory|\n|-|-|-|-|-|\n|Overall|Perf.|39.80|36.71|44.80|\n|    |API Cost|$18.1|$25.8|$24.0|\n|    |Latency|8.7h|10.3h|11.4h|\n|Level 1|Perf.|52.80|50.60|56.60|\n|    |API Cost|$5.7|$7.3|$7.0|\n|    |Latency|1.8h|2.0h|2.7h|\n|Level 2|Perf.|34.60|42.30|44.18|\n|    |API Cost|$8.9|$11.3|$9.9|\n|    |Latency|4.9h|5.6h|6.1h|\n|Level 3|Perf.|16.70|16.70|23.07|\n|    |API Cost|$3.5|$7.2|$7.1|\n|    |Latency|2.0h|2.7h|2.6h|\n\nThese results suggest that G-Memory consistently improves agent performance across all task levels, with only marginal latency and API cost increase. We will incorporate this analysis into the revised manuscript to further clarify G-Memory’s scalability and practical utility.\n\n\n---\n\n> **`Weakness 2`: Some lower-level details, such as precise prompt engineering and insight validation, rely on in-LLM processing. The robustness of these steps is not deeply analyzed.**\n\nThank you for your insightful question! To address your concern, we elaborate on both the intuitive design and empirical effectiveness of our prompt engineering and insight validation mechanisms.\n\n**(1) prompt settings**\n\nOur work incorporates three main components of prompt engineering, which we detail below:\n\n* **LLM-based Graph Sparsifier (Eq. 7):**\n  The operator $\\mathcal{S}_\\text{LLM}$ is designed to distill a verbose interaction graph $\\mathcal{G}\\_\\mathsf{inter} = (\\mathcal{U}, \\mathcal{E}\\_\\mathsf{u})$ into a minimal yet sufficient reasoning path tailored to a new query $Q$. This is achieved in two stages: **(1) Node Summarization**, where an LLM function condenses the textual content $m\\_i$ of each utterance node $u\\_i = (\\mathcal{A}\\_i, m\\_i)$, producing a summarized node set $\\mathcal{U}'$ while preserving the original graph structure. This yields an abstracted graph $\\mathcal{G}\\_\\text{sum} = (\\mathcal{U}', \\mathcal{E}\\_\\mathsf{u})$, where $\\mathcal{U}' = \\\\{ (\\mathcal{A}\\_i, \\text{LLM}\\_\\text{sum}(m\\_i)) \\mid (\\mathcal{A}\\_i, m\\_i) \\in \\mathcal{U} \\\\}$; **(2) Graph Extraction** Subsequently, the abstracted graph $\\mathcal{G}\\_\\text{sum}$ is then provided to $\\mathcal{S}\\_\\text{LLM}$ and produces $\\hat{\\mathcal{G}}\\_\\mathsf{inter}(\\hat{\\mathcal{U}}, \\hat{\\mathcal{E}}) = \\mathcal{S}\\_\\text{LLM}(\\mathcal{G}\\_\\text{sum}, Q)$.  \n  To further validate the necessity of this component, we conduct an ablation study comparing systems with and without $\\mathcal{S}\\_\\text{LLM}$:\n  \n|MAS|Variant|ALFWorld|SciWorld|PDDL|\n|-|-|-|-|-|\n|AutoGen|w/ $\\mathcal{S}\\_\\text{LLM}$|88.81|67.40|27.77|\n||w/o $\\mathcal{S}\\_\\text{LLM}$|86.36|62.36|24.56|\n|DyLAN|w/ $\\mathcal{S}\\_\\text{LLM}$|70.90|65.64|18.95|\n||w/o $\\mathcal{S}\\_\\text{LLM}$|68.13|64.16|16.09|\n\nThese results empirically confirm that the LLM-based interaction summarization and filtering mechanism is both effective and essential.\n\n* **LLM-based Filtration (Eq. 7):**\n  As referenced in Eq. (7), the filtration process is guided by an LLM, and we provide the full implementation details in Appendix C for transparency and reproducibility.\n* **Insight Extraction (Eq. 10):**\n  This is detailed below:\n\n\n\n**(2) insight validation**\n\nG-Memory is equipped with at least two mechanisms to ensure that accurate and helpful insights are increasingly emphasized over time:\n\n- **G-Memory periodically performs consolidation over accumulated insights to merge redundant ones and discard irrelevant or obsolete content.** First, recent queries are grouped into semantically related clusters $\\{\\mathcal{C}\\_l\\}$. For each cluster, all corresponding insights are retrieved from the graph: $\\mathcal{I}\\_l = \\Pi_{\\mathcal{Q} \\to \\mathcal{I}}(\\mathcal{C}\\_l)$, which is then synthesized into a set of more general insight, $\\iota_l^\\text{merged}$, using the function $\\mathcal{J}$ in Eq. (10) that merges their textual content and supporting query sets: $\\iota\\_l^\\text{merged} = ( \\mathcal{J}(\\{\\kappa\\_k \\mid (\\kappa\\_k, \\Omega\\_k) \\in \\mathcal{I}\\_l\\}), \\bigcup\\_{\\iota\\_k \\in \\mathcal{I}\\_l} \\Omega\\_k )$. This consolidation ensures that the insight graph is regularly updated to maintain concise and representative high-level knowledge.\n- As the query and insight graphs evolve over time, correct and helpful insights/interactions naturally lead to more successful task executions, thereby accumulating additional edge connections. This process **positions them as denser and more central nodes in the graph**, making them more likely to be retrieved, as illustrated by nodes 45 and 55 in Figure 10. In contrast, misleading elements tend to have limited impact and remain on the periphery of the graph (e.g., nodes 36 and 37 in Figure 10). This update dynamically implicitly prioritizes reliable knowledge while suppressing noise.\n\n---\n\n> **`Question 1`: How does the memory graph scale with increasing agents/queries/tasks**\n\nWe would like to respectfully point out that all major operations within G-Memory are designed to **scale efficiently** without significantly increasing token cost **as the queries/tasks grow**:\n\n\n|Memory Layer|Formula|Process|LLM invocation|Detail|\n|-|-|-|-|-|\n|Query-level|Eq. (4)|Retrieval|No|based on embedding similarity|\n||Eq. (9)|Update|No|-|\n|Interaction-level|Eq. (5) and (7)|Retrieval|Yes|The LLM receives $\\tilde{\\mathcal{Q}}^S$ as input, i.e., the $k$ queries retrieved by Eq. (4) and their 1-hop neighbors. Assuming an average node degree $\\overline{D}$, the total input size is approximately $k \\cdot \\overline{D}$, which does not increase significantly with the number of nodes.|\n||-|Update|No|\n|Insight-level|Eq. (6)|Retrieval|No\n||Eq. (10)|Update|Yes|This process takes the new query and its interaction graph as input. The input size remains roughly constant.\n\n\nIn addition to being robust to the number of queries/tasks, **G-Memory is also largely insensitive to the number of agents**, as almost all processes and operators within G-Memory are **independent of the agent team size**. The only exception is Eq. (8), where the computational cost scales linearly with the number of agents. Nevertheless, this cost remains manageable in practice and does not significantly affect overall efficiency.\n\nIn summary, G-Memory can scale its memory pool without incurring excessive token cost, making it suitable for long-horizon multi-agent tasks.\n\n\n\n\n\n----\n\n> **`Question 2`: Are there mechanisms for automatic pruning or compression of obsolete information?**\n\n\nThank you for this insightful inquiry! Please refer to our response to `Weakness 2`, where we introduce the consolidation mechanism where insights are periodically reconstructed or merged."}]

Author rebuttal:

Meta-review: Summary: 

This paper introduces G-Memory, a hierarchical, agentic memory system for multi-agent systems (MAS) inspired by organizational memory theory. It manages the lengthy MAS interaction via a three-tier graph hierarchy: insight, query, and interaction graphs. The paper conducts extensive experiments with three LLM backbones and three MAS frameworks on multiple benchmarks, and shows the performance gains brought by G-Memory without changing the original MAS framework. 

Strengths: 

1. The proposed memory system is novel. Reviewers all appreciate the design and clear distinction from existing memory systems. 

2. The experiments are comprehensive by showing performance and cost and conducting ablation studies and case studies. They look thorough and convincing. The memory system can also be seamlessly integrated into multiple MAS frameworks as a plug-and-play module. 

3. The paper is very well-written with a clear structure and carefully curated figures/tables. 

Weaknesses:

Before the rebuttal, reviewers asked many clarification questions, including notations and definitions of epochs, graph initialization, and ambiguous graph update mechanism, to which the authors have provided answers during the rebuttal. 

On the experiment side, during the rebuttal period, the authors added new results on the GAIA benchmark, conducted more analysis on the latency of G-memory, sensitivity, and the scaling efficiency as the queries/tasks grow. 

Overall, all reviewers are satisfied with the responses and some have increased their scores. I would highly recommend accepting the paper, and also encourage the committee to consider a spotlight/oral award.
Decision: Accept (spotlight)