MindForge: Empowering Embodied Agents with Theory of Mind for Lifelong Cultural Learning
Authors: Mircea Tudor Lică
Ojas Shirekar
Baptiste Colle
Chirag Raman
Venue: NeurIPS 2025 poster ()
Keywords: lifelong learning
theory of mind
LLMs
cultural learning
in-context learning
Primary area: deep_learning

Abstract: Embodied agents powered by large language models (LLMs), such as Voyager, promise open-ended competence in worlds such as Minecraft. However, when powered by open-weight LLMs they still falter on elementary tasks after domain-specific fine-tuning. We propose MindForge, a generative-agent framework for cultural lifelong learning through explicit perspective taking. We introduce three key innovations: (1) a structured theory of mind representation linking percepts, beliefs, desires, and actions; (2) natural inter-agent communication; and (3) a multi-component memory system. Following the cultural learning framework, we test MindForge in both instructive and collaborative settings within Minecraft. In an instructive setting with GPT-4, MindForge agents powered by open-weight LLMs significantly outperform their Voyager counterparts in basic tasks yielding $3\times$ more tech-tree milestones and collecting $2.3\times$ more unique items than the Voyager baseline. Furthermore, in fully collaborative settings, we find that the performance of two underachieving agents improves with more communication rounds, echoing the Condorcet Jury Theorem. MindForge agents demonstrate sophisticated behaviors, including expert-novice knowledge transfer, collaborative problem solving, and adaptation to out-of-distribution tasks through accumulated cultural experiences.

MoneForge: Empowering Embodied Agents with Theory of Mind for Lifelong Cultural Learning
Mircea Licä"
Ojas Shirekar"
Delft University of Technology mlica@tudelft.
Delft University of Technology k. shirekar@tudelft.
Baptiste Colle
Chirag Raman
Delft University of Technology collebaptiste@gmail. com
Delft University of Technology c. a. raman@tudefft.. nl
Abstract
Embodied agents powered by large language models (LLMs), such Voyager, promise open-ended competence in worlds such as Minecraft. However, when powered by open-weight LLMs they still falter elementary tasks after domain- specific fine-tuning. We propose MINDFORGE, generative-agent framework for cultural lifelong learning through explicit perspective taking. We introduce three key innovations: (1) a structured theory of mind representation linking percepts, beliefs, desires, and actions; (2) natural inter-agent communication; and (3) a multi-component memory system. Following the cultural learning framework, we test MINDFORGE in both instructive and collaborative settings within Minecraft. In an instructive setting with GPT-4, MINDFORGE agents powered by open-weight LLMs significantly outperform their Voyager counterparts in basic tasks yielding 3x more tech-tree milestones and collecting 2.3x more unique items than the Voyager baseline. Furthermore, in fully collaborative settings, we find that the performance of two underachieving agents improves with more communication rounds, echoing the Condorcet Jury Theorem. MINDFORGE agents demonstrate sophisticated behaviors, including expert-novice knowledge transfer, collaborative problem solving, and adaptation to out-of-distribution tasks through accumulated cultural experiences.
Introduction
The development of generally capable agents marks significant shift in advancing artificial in- telligence, transitioning from assimilating data to generating novel knowledge through embodied interactions with open-ended environments (1-4. Classical approaches leveraging reinforcement
learning [ 6 and imitation learning 77 often struggle with generalization and exploration, as agents tend converge on repetitive behaviors in static environments (8). To address these limitations, researchers have sought to emulate human-like lifelong learning capabilities, developing systems that can continuously acquire, update, and transfer knowledge over extended periods g 10 The advent of large language models (LLMs) has accelerated this pursuit, enabling the development of agents such Voyager [ that can apply internet-scale knowledge continuously explore, plan, and acquire new skills in partially observable, open-ended environments such as Minecraft.
Despite their promise, argue that state-of-the-ar lifelong learning agents like Voyager face crucial limitation: they learn isolation, neglecting a fundamental aspect of human intelligence-the
Equal Contribution.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

WWogght MindF
30
Lair 2-3-8 Mepiges
Larra 2-3-0 Pinnp-ppp
Open-LLM Voyager
MindForge
Larr's
Histu do- drrige
Hew Inter tome
Wisdon Tool
Farm cosordyyy
pricoed
bertur de garn converstion oncoern))
Piorgti Coneiing Docle Comanaaio
(a)
(b)
Figure 1: MINDFORGE enables agents to (a) collaborate through structured theory-of-minn rep- resentations, which (b) leads to improved performance compared to Voyager when powered by open-weight LLMs, averaged across 3 runs.
social context. So central is the social context to our existence that the Social Intelligence Hypothesis posits that our cognitive capabilities evolved primarily to navigate the complexities of social life [2] This isolated learning becomes particularly problematic when coupled with these agents' reliance closed LLMs like GPT-4 Wang et al. [11] note that "VOYAGER requires the quantum leap in code generation quality from GPT-4 which GPT-3.5 and open-source LLMs cannol provide". Our experiments (Section 5) corroborate this observation, demonstrating that replacing GPT-4 with open LLMs renders the Voyager agent incapable of even the most rudimentary tasks, such as collecting dirt in Minecraft. We ask: can an open LLM --owered agent, which struggles with basic tasks, improve through collaboration with others?
To address this question, introduce the MINDFORGE framework, inspired by cultural learning, which contends that individuals internalize skills and strategies by observing others' perspectives, using Theory of Mind (ToM) to infer goals, beliefs, and intentions and thereby deepen social learning 14.18] Our framework reframes knowledge distillation [99 as a social, test-time process: agents refine their beliefs through multi-turn dialogue with expert peers, without gradient updates. This in- teractive distillation supports test-lime compute scaling 20). enabling agents to improve performance via communication, memory retrieval and perspective-taking By structuring communication across turns. MINDFORGE also contributes to multi-tur turn LLM agent research (21 [22) showing that open models can match GPT-4 through collaboration alone. Moreover, our analysis of weak-weak settings highlights a compute-periee -erputtteprfromncceee tradeoff at inference, linking collaboration to test-time compute considerations 20).
Figure Ta provides an overview of our agents in action. Concretely, MINDFORGE extends Voyager by incorporating three key improvements: (1) perspective-taking through causal ToM template (23 linking agent percepts, beliefs, desires, and actions; (2) natural language communication interface between agents; and (3) memory updates following the Soar cognitive architecture (24]. including semantic memory of task and environment-rrlated knowledge and episodic memory.
Empirically, we instantiate MINDFORGE agents in Instructed Learning (mixed-expertise) and Collaborative Learning mmacheddexpeeriee
settings within the cultural learning framework (15). MINDFORGE agents powered by open- weight LLMs outperform the original Voyager architecture on two seemingly trivial tasks by more than 40%. We identify Voyager's s specific failure cases: (1) false beliefs and (2) code generation, and show how our method overcomes them through communication and perspective-taking To assess lifelong learning, we run MINDFORGE on the Minecraft tech-tree and find that surpasses open-weight Voyager variants in both key tech-tree milestones and the number of unique items obtained. Section 5.5 further shows that when both agents begin with the same limited capabilities, single round of GPT-4 seeding followed by just seven turns of peer-to-peer dialogue lifts the population-level success rate from 62% to 79%, revealing a Condorcet-style miinds boost that stems from richer interaction rather than larger models. This generalization in costboollaborttive settings paves the way for agents that can continuously adapt through social interaction, reducing the reliance on large-scale computing resources for training and fine-tuning the underlying language models.
2

---

2 Related Work
Theory of Mind in Embodied Agents Early attempts to equip machines with mental-state rea- soning drew on deep learning 27 reinforcement learning [28]. and Bayesian models [29]. These systems were either confined to toy 2-D worlds [25 26 or failed to generalize to richer environments (27] The emergence of large language models (LLMs) reignited interest in ToM,
sparking evaluations of their putative capabilities [23 32 Although evidence so far suggests that LLMs lack genuine ToM, their breadth of knowledge has inspired new wave of prompt-based frameworks [3333]]: the first two focus on perspective-takinn
while the latter pair ground reasoning in explicit causal models. BIP-ALM remains the sole example that integrates such framework with
an LLM 37 38).
LLM-based Multi-Agent Systems LLMs have demonstrated overwhelming success in performing a variety single-agent tasks ranging from question answering and text generation to code synthesis and complex reasoning in embodied settings [1] [3 40). This success has started growing interest in
developing LLM-based multi-agent systems [21 22, (3),, where language- baasdd communication is central component. For instance, CoELA [1 introduce modular multi- -ggent framework that integrates memory, perception and execution, while COMBO [41] present a multi-agent cooperative framework, that deals with collaboration in partial egocentric views of the environment. However, these frameworks lack an explicit belief representation and benchmarks like MineCollab 44, designed for collaborative reasoning, find that current LLM agents are not suited for multi-agent collaboration in long-horizon embodied scenarios, a gap that MINDFORGE addresses by endowing agents with structured perspective taking capabilities.
reasoning. Recently,
Single-agent Frameworks Recent research on long-horizon tasks in single-agent systems differs architecturally from MINDFORGE'S unified collaborative learning framework. For example, Ex- PeL [45] and CLIN [46) emphasize experiential learning through vectorized trajectories or causal abstractions but lack explicit belief graphs and integration across episodic, semantic, and procedural memories. Frameworks like DEPS [7] and ADaPT [48] enable in-episode self-correction via itera- tive refinement, yet insights are often ephemeral, confined to the context window without persistent storage or structured models of others' beliefs. SSO [49] provides procedural memory through skill libraries but omits episodic/semantic layers and social reasoning for targeted teaching or belief revision. Similarly, Adam (50) structures world knowledge as causal graphs in open environments, while Optimus-1 builds hierarchical semantic graphs from experiences.
Background
Voyager Voyager agents consist of three main components: (1) automatic curriculum generator, (2) a skill library, and (3) an iterative prompting mechanism. The automatic curriculum proposes progressively complex tasks, driving continuous learning. The skill library stores and retrieves learned behaviors, facilitating knowledge accumulation and transfer. The iterative prompting mechanism generates and refines executable code based on environmental feedback, allowing for adaptive behavior. Voyager demonstrated significant capabilities in item discovery, technology progression, and skill generalization. This system established new benchmark in open-world Al agents, showcasing the potential for lifelong learning and adaptation in complex, dynamic environments. The Voyager framework serves base for our lifelong collaborative learning agents within Minecraft.
BogToM Causal Template The Big ToM causal template [23], provides a structured framework for representing Theory of Mind (ToM) scenarios as causal graphs. This template comprises key components including context, desire, percept, belief, causal event, and resulting actions. By representing ToM scenarios in this causal structure, the template allows for systematic manipulation of variables such as an agent's 's perceptions, beliefs, and actions in response to environmental changes. This approach enables the creation of diverse, well-controlled ToM scenarios, capturing the complex causal relationships between an agent's 's mental states and their behavior in social situations. The causal graph representation facilitates a more nuanced and systematic exploration of ToM reasoning, allowing for the examination of various inference patterns such as forward belief, forward action, and backward belief inferences in social cognitive tasks.
3

---

MindForge
Causal TOM Template
Memory
Voyager
Percapts
Beliels
Epioodc
Perceprs
Memory
Astions
Semantt
(Tanb)
Chince
Faacusion Modkuee
Destres
Rocedual
(98B))
Esccution
Dwatres
Sechhoprfene
(a) FINDFORGE Architecture
(b) Voyager Architecture
Figure 2: Cognitive architectures of MINDFORGE and Voyager respectively. The Voyager architecture has been re- framed within the scope of our cognitive architecture. MINDFORGE expands the original Voyager framework to enable communication through theory of mind together with additional memory subsystems to enhance the lifelong learning capabilities of the agent.
4 Method
To grant MINDFORGE agents the ability learn through collaborative social interactions, we intro- duce the following components for communication and perspective-taking: (i) an explicit structured
representation linking percepts, beliefs, desires, and actions (Section 4.1). (ii) perspective taking so that the agent can reason over self-beliefs and those about its collaborators (Section 4.2) (iii) com-
munication medium that the agents can use to share information in natural language (Section 4.3)
and (iv) the ability to utilize different types of memory subsystems (Section 4.44.
4.1 Structured Representation of Own Mental States: Belief's, Desires, and Intentions
For MINDFORGE agents to develop effective theory of mind capabilities, they must first possess structured representations of their own mental states. This explicit modeling of the self is crucial for recursive social reasoning: enabling agents not only to reason about their own beliefs, goals, perceptions, and how these might lead to their actions, but also to form similar mental model for other agents. This mirrors findings developmental psychology. where children first develop an awareness of their others [2 53). This is further supported by neuroscientific research that has also identified distinct neural mechanisms for self-cognition and social cognition [54 55).
thoughts, feelings, and goals before being able to infer the mental states of
We draw on the classical Belief-Desirg Desire-Intention (BDI) framework [56] and operationalize it us- ing the BigToM causal template (Section 3 (23) In MINDFORGE, beliefs represent the agent's current understanding of the world inferred from sensory perception, including interactions with other agents. Desires encode the agent's 's goals, both generated by automatic curriculum and those arising from collaborative interactions. Intentions correspond to concrete executable actions, implemented as generated code-and so, consistent with BigToM [X], we treat "intentions" and "actions" interchangeably in implementation. However, we note that this equivalence does not always hold conceptually: intentions refer to internal commitments plans, while actions observable behaviors that may diverge from intentions. For deeper treatment of this distinction, see [57].
Belief Structure. We categorize MINDFORGE agent's beliefs into four key categories, illustrated in Figure 3 i. perception-related beliefs, formed from direct sensory input via the Minecraft API; task-related beliefs, inferred from the agent's 's reflection on its upcoming objectives; iii. interaction- related beliefs, derived from chat messages during task execution; and iv. partner-related beliefs, which track others' mental states. Each belief type generated via separate LLM call, with prompts conditioned on current percepts, memory retrievals, and conversation history (see Appendix (F). This modular belief formation feeds into the causal ToM template Figure 2a which governs downstream action generation. Structured self-modeling thus serves as the foundation for cultural learning and perspective-taking.
4.2
Theory of Mind: Modeling Partner Perspectives
Building on the structured representation of their own mental states, MINDFORGE extend this modeling to others via Theory of Mind-the capacity to attribute beliefs, desires, and intentions to others. Fach MINDFORGE agent achieves this by maintaining partner-specifie BigToM graph within its own beliefs, allowing reason about the internal state of each conversational partner

---

Quostions How wood log Mincoroft? Ansewor You can mine oak log direetly with your hands clicking
Perception selats
am curmently in anowy beach biorne. indicated suroundaiigg
get
blocks sanditionn, dirt. gross, block, sooni, and cool_ooe.
Beliefs
Trens Bnlints
Interice Bolies
Portner
Interacsion
Anower located near trme, indicated by the precence of leoves and cok loga in vicinity.
Portner Bollefs
lasks collectng one word log breaking
Partner
fully grown
befiersy
oction cinteraction detailss Percuption beliefs: oballela aocut parceptas
(fcition)
Figure 3: Each agent maintains a set of internal beliefs that it can exploit as needed. In order to enable ToM capabilities agents also maintain belief over the beliefs, actions and desires of their collaborative counterparts.
(see Figure 3) These representations guide how an agent interprets and responds to incoming
communication.
During collaboration, an agent uses its internal model of its partner to simulate what the partner knows, believes, needs. The underlying LLM receives this partner model along with the conversation history to generate responses incorporating the collaborator's 's point of view. As interaction proceeds, the partner beliefs are continuously updated using received through the communication channel (Figure 6). This dynamic modeling of partner mental states endows MINDFORGE agents with one of the most crucial abilities within the cultural learning framework: reasoning about partner perspectives [4]1]] [5]. For instance, an expert agent can diagnose a novice's 's misconception and tailor its feedback accordingly, while novice ask more targeted questions with fewer digressions by simulating what the expert knows. See Appendix [.] for more details on how perspective-taking is integrated into the communication module, and in-game examples.
4.3 Communication Medium
To take advantage of representations of each other's mental models and collaborate effectively, agents need to be able to communicate with each other. We operationalize communication in natural language, integrating the MINDFORGE framework's 's communication system with Minecraft's 's native in in-game chat functionality. Specifically, once MINDFORGE agent fails to complete a task, it initiates a communication round with its partner where it leverages its BigToM representation to ask for help. This system enables agents exchange textual information and, crucially, to integrate feedback from collaborators directly into their belief systems. Concretely, we utilize the Mineflayer API (58] to extract and process chat data from the Minecraft environment. By leveraging the game's 's existing chat system, the framework ensures that collaborative interactions occur in a coprext-appropriate manner, closely mirroring human-to to--uman communication within Minecraft.
To handle the resulting conversation, we use a dedicated LLM instance within the MINDFORGE agent which is tasked with synthesizing and interpreting the conversation into the structured BigTOM BDI template, generating both interaction beliefs and partner beliefs (see prompts in Appendix F) This instance does not directly influence any other parts of the agent.
4.4 Memory Subsystems
MINDFORGE agent's memory consists of three subsystems drawing on research about human memory: (i) episodic, (ii) semantic, and (iii) procedural memory.
Episodic memory deals with pertinent information about the agent's past attempts at task and social interactions with other agents. Recollections of past episodes serve context for humans to interpret others' behaviors, leading to divergent future actions in the same situation in response to different recollections [59]. Humans also tend to summarize information from their social inter- actions, but the same time are capable of preferentially accessing episodes when the credibility
5

---

of certain summaries is low due to the lack of evidence [60]. Moreover, findings also suggest that there are processes in the brain for updating these summaries as new episodes transpire [60] 61]. Inspired by complementary learning systems theory, Larimar [62 enhances L.MMs with distributed episodic memory module, enabling dynamic knowledge updates without any gradient- learning. Furthermore, Optimus-1 integrates past experiences in multimodal memory storage mechanism and abstracted experience pool used for summarization of past experiences. For a MINDFORGE agent, the role of episodic memory is twofold. Firstly, it stores details about its partners revealed during previous conversations that can be exploited for perspective-taking in future communication. Secondly, store past actions that led to failure and retrieve them based on their similarity with the task at hand. Thus, a MINDFORGE agent can track both the progress of its partner and learn from past failures.
Semantic memory constitutes an individual's s store of knowledge about the world. The content of semantic memory is abstracted from the experience and is therefore said to hold summarized generalizations, likely without references to specific experiences [60] 61 63]. For a MINDFORGE agent, the semantic memory stores information related to the task and environment. It stores information relevant to completing a task; for instance, the prerequisites for being able to mine dirt in the game world. Such piece of information is not necessarily tied to one specific episode. Rather, it is generally useful information about the world and can be distilled from the episodic memory.
Finally, procedural memory stores the skills that the agent learns and is analogous to the skill library present in the original Voyager framework [.]. The skills stored in the procedural memory are reusable pieces of code that the agent can use for solving problems, allowing the agent to take on more complex tasks while avoiding re-learning elements previously used successfully. As illustrated in Figure 2a episodic and semantic memory stores influence the agent's 's beliefs, while procedural memory provides the routines that inform the agent's next action. Additional information about the memory subsystems can be found in Appendix B.2 where we provide implementation details, and an ablation for the episodic memory component, where we show that omitting it from MINDFORGE leads to a drop in success rate.
5 Experiments
5.1 Experimental Setup
Our MINDFORGE framework introduces three advances that let agents improve continuously through social exchange: (1) explicit theory-of-minn -fnd representations for reasoning about peers, (2) Natural- language dialogue for coordination, and (3) integrated semantic and episodic memory retain experience. We evaluate these additions in the same Minecraft world used by Voyager [11], built on MineDojo (64] with motor control via Mineflayer (58]. Component LLMs communicate through DSPy [65 66) and LangChain [67] for generation and embedding. In the base experimental setting, we follow Voyager [ and allow the agent to execute 4 sequential actions solve task. MINDFORGE agents that interact (see Section 5.3) interleave one communication round (via the in- game chat) between two environmental actions. A snapshot of this setup can be seen in Figure 4a For isolated tasks we measure the fraction of successes over 24 independent trials (one task per trial). In tech- tree experiments we follow Voyager's 's protocol, tracking milestone achievements and unique items collected. To ensure comparability, we preserve Voyager's ' original prompt structure and workflow throughout.
5.2 Voyager With Open-Weight LLMs
To establish non-interactive performance baselines, we replace GPT-4 [68] with three wpen-weight LLMs in the Voyager framework [Т: Mistral-7B (69] Mixtral-8x7B 770 and Llama 3.BB8B (7T), selected for their long context windows and manageable size. Testing focuses on two fundamental Minecraft tasks: collecting single blocks of dirt and wood, Results in Table reveal that while GPT-4 achieves perfect performance, open- open-weight LLMs struggle significantly even with these basic tasks without collaboration. This aligns with Wang et al. [II's observation that GPT-4's superior code generation capabilities enable it to obtain 5.7x x more unique items than other models, establishing clear baseline for measuring the impact of our collaborative approach.

---

Table I: Voyager [1] powered by open-weight LLMs (even fine-tuned for Minecraft) struggles
with basic tasks, Closed source models like GPT-4 exhibit perfect accuracy. A single MINDFORGE agent bridges the performance gap between Mistral-7B and GPT-4 on both wood and dirt collection, showing significant gains compared to other learning techniques such as PEFT.
Task
Mistral-7B
Mixtral-8x7B
Llama 3.1-8B
Mistral-7B (PEFT)
Mistral-7B
GPT-
(MindForge)
7%
27%
4%
7%
37.5% 33.3%
100%
Wood 4%
27%
7%
12.5%
100%
pet
rach aid plocass cooeentt
compars
Hous agant usa foacbact
your
Serong hipers bu do vauu
maeeiiingg 1o vati coneeeaeoon fly
apwr
for Help
Exscution emen
and propertes unddlt ding 3r)
(b)
Figure 4: Failure mode correction. (a) Communication with an expert agent fixes the code error. (b) A weaker agent's incorrect task-related belief (initially believing it is mining "grass blocks' and needs tools) is corrected by stronger agent through communication (e.g., stating 'dirt blocks' do not require tools), leading to an updated task understanding.
Primary Failure Cases While Wang al. 11 only discuss onde-generation failures, find two prominent failure cases for the open- weight Voyager setup: (1) false beliefs and (2) faulty code generation. Importantly, we discover that the critic-based self-improvement component in Voyager fails to recover from initial mistakes and subsequently fails to solve the task. As shown in Figure 4b, Mistral-7B powered Voyager fails dirt collection due to incorrect assumptions about tool requirements and lack of basic knowledge pertaining to Minecraft. Similarly, Figure 4a showcases how feedback from a stronger agent can solve code misconceptions about the Mineflayer API [58].
Fine- Tuning Does Not Help Parameter-efficien tuning often helps LLMs 72], yet adapting Mistral-7B with GPT-4 execution traces (reasoning + code) and Minecraft manuals (domain knowl- edge; Appendix (E) produced no benefit. In cultural-larninn terms [14 [5) this is imitation learning. but the tuned model still scores 7% on dirt collection (=0) and 12.5% on wood collection (+8.5) as shown in Table T] Post-hoc analysis shows the tuning makes outputs wordier, blurs decision- making, and erodes instruction-followe indicating that naively imitating unstructured traces is counter-productive.
5.3 Introducing Collaboration: MINDFORGE Agents
Having shown that PEFT-based imitation learning fails to improve agent capabilities, we evaluate our proposed MINDFORGE framework. We adopt an instructive learning approach within the cultural learning framework [[5]. where an expert agent guides a novice through communication and seppecttivetakinng This multi-agent collaboration effectively bridges the performance gap between open-weight LLMs and GPT-4, addressing the failure cases identified in Figure 4
Feasibility: Human (Expert) Collaboration Improves Performance To establish feasibility, we first conduct wizard-of-oz study where a human expert interacts with an open-weight LLM--oowere MINDFORGE agent through the in-game chat. As shown in Table 2 human expert collaboration achieves on average +57.85% increase in task completion rate for dirt collection over the non-
7

---

Performance Acroes Communiesiin Rounds
Table 2: Fraction of successful MINDFORGE agents across 24 individual trials; dirt and wood collection. Standard Voy-
Fraetton saa sussondded (16
Mi
ager [T] setup: 4 attempts per task, with communication round interleaved where appropriate. See also Figure and Figure 7
54
34
Setting
Task
Mistral-
Mixtral- 8c7B
Llama 3.1-8B
7B
3 Communication Rounds
w/o interactions
Dirt
37.5%
29.15%
12.5%
instructive setting. human expert
Dirt
+15.83
+57.85
+41..3
instructive setting. GPT-4 expert
Dirt
+16.16
+37.51
+4.1
Figure 5: Instructive Setting (GPT- 4); Dirt Collection. Performance scales with communication rounds. MINDFORGE w/o interactions as a base- line. Actions are interleaved with com- munication rounds. Dashed (--): ex- tra collaboration rounds. A points: cold- start (4 actions) without interactions.
instructive sctting. Llama 3. 117088 collaborative setting, no expert
Dirt
+8.33%
Dirt
-1.2
++0
8..5
wro interactions
Wood
33.3%
50%
7%
instructive setting. GPT-4 expert instructive sctting. Llama 3. 3.1-70B collaborative setting, no expert
Wood
+12.52
+20.8
+5.5
Wood
+8.36%
Wood
-12.5
-.15
+0
interactive baselines. This demonstrates both the viability and flexibility of our framework enabling productive human-agent interactions for collaborative tasks.
GPT-4 (Expert) Collaboration Maintains Performance Benefits We replace the human expert with GPT-4-powered MINDFORGE agent that collaborates with open-weight LLM agent. The expert agent infers its partner's s failure cases through prsspective-taking based on natural language communication in the game chat. Unlike human collaboration, agents cannot directly access each other's s internal beliefs and must rely on Theory of Mind through communication to model their counterpart's 's mental state. This perspective-taking enables the expert to accurately diagnose and address the weaker agent's 's misconeeptions. Notably, this setup generalizes the classic notion of knowledge distillation beyond logits model weights: here, distillation unfolds through social interaction, where the expert agent transmits structural knowledge, corrects false beliefs, and shapes the student's internal representations in real time. As shown in Table 2 this approach significantly improves performance compared to the non-interactive baselines, achieving on average a 16.1% increase in task completion rates for dirt collection and a 13% increase for wood collection tasks respectively. A sample conversation can be found in Appendix D
To elucidate the MINDFORGE framework's components, we conducted targeted abla- tions. The perspective-takinn completion by up to 13% as evidenced by comparisons across multiple communication rounds (see Table 7 Appendix B.I Our structured ToM representation outperforms its unstructured variant in complex Minecraft tasks (Table(8.. Additionally, ablating the episodic memory component reveals that retaining past failures enhances performance (Table Appendix B.2)
Ablations
module, enabling agents to reason over partners' belief's, boosts task
More Communication Rounds Improve Performance Since communication is central to our framework, we examine the benefits of multiple rounds per task trial, assessing communication and perspective-taking amid escalating opportumities. We pair a GPT-4 MINDFORGE expert with a Mixtral-8x7B MINDFORGE weak agent. Figure demonssratss communication's 's value through a sharp rise task completion rates with additional rounds, while Figure G]lluutrraes the expert's progressively accurate grasp of the partner's 's mental state. Further evidence in Appendix A. communication's importance instructive and collaborative settings, even for demanding Minecraft tasks ("Craft pickaxe" and "Mine Iron") involving MINDFORGE agents.
Using a less constrained communication protocol We introduce the MINDFORGE communica- tion protocol in Section 4.3. initiating agent conversation only after task failure. To demonstrate the framework's s flexibility, we test two MINDFORGE tasks with a dynamic communication setup, allowing the weak agent to choose whether to initiate communication when uncertain or attempt the task independently. Table 6 (Appendix A.3) reveals comparable task completion rates between the
8

---

Performance Communication Rounds
Frocton taas sucosssuoo sucoudded
Boliefs
Portrox So
Partna Selicta
ge
and Aooi
Heallh soad
Gricon Unsiovs
cemsiication Sounda
Communication Rounds (N)
Round
Round
Figure 6: Partner beliefs of the strong agent progressively Figure 7: Collaborative Setting (Mix-
improve as the number of communication rounds increase. tral); Dirt Collection. Blue: GPT-
Subsequently, these belief's are used to reason throughout 4-guided baseline. Dashed the conversation and take perspective.
(--): extra
collaboration rounds.
Table 3: We follow Voyager's methodology [[]] for reporting the Minecraft tech-tree results. The reported scores represent the number of prompting iterations until the milestone was accomplished, averaged over three trials. The fewer the iterations, the more efficient the method. N//A represents the agent being unable solve the task. Best performance in terms of iterations required and number of times milestone reached (out of 3 trials).
Method
Wooden
Voyager 2023
GPT-4
6: (3/8)
11:22 (3/3)
2117 (3/8)
ReAct 2023
GPT-4
N/A (9/5)
NIA (9/3)
N/A (9/a)
Refietion
2023
GPT-4
N/A (9/8)
N/A (%/a)
NIA (9/s)
GPT.4
92:172 (9/5)
92:722 (3/3)
1354:103 (9/9)
Mixtral 8x7B
N/A (%/s
NIA (%/²)
NIA (9/3)
(open-weight)
Voyager
Llama 3.1-8B
N/A (9/s)
N/A (9/3)
NIA (9/a)
Llama 3.1-70B
19 (5/3)
N/A (9/4)
NIA (9/3)
Mixtral 8x7B
86/117(9/8)
NA (%/4
NIA (%/)
MINDFORGE
Llama 3.1
12010 (1/3))
NIA (%/s)
N/A (%/)
(wla collaboration)
Llama 3.1-70B
50:117 (9/8)
113:24 (2/3)
N/A (%/3
Mixtral 8
51:220
N/A (%/2)
N/A (%/)
MINDFORGE
(mixod-expertise collaboration)
Llana 3.1-8B
100 +:30 (5/a)
NAA (%/a)
NIA (%/)
Llama 7-B
62:14 (3/3)
79=25()//4)
1 120+30 (3/3)
MINDFORGE
(mixed expertise collab w/ Llama 3.1)70B)
Mixtral 7x
65:122 (9/9)
N/A (%/4)
N/A (%/)
default and flexible protocols, underscoring MINDFORGE's ' independence from specific communica- tion designs. Qualitatively, weaker models like Mistral-7B consistently act confidently, never seeking help despite the option, while both models adeptly bypass assistance for tasks aligned with prior experience.
5.4 MINDFORGE Lifelong Learning
We evaluate MINDFORGE's lifelong learning capabilities against both GPT-4 and open weight Voyager [II] variants. The progression through increasingly complex technological achievements (from wooden to diamond tools) provides rigorous test-bed for assessing lifelong learning and collaboration abilities of MINDFORGE. We follow the methodology from Voyager and evaluate performance on three key milestones: (1) wooden tools, (2) stone tools and (3) iron tools.
In line with the findings in Table I hhighihggglightt the performance gap between GPT-4 powered Voyager and its open-weight variants, with only Llama 3. 3-BB 71 reaching the wooden tools milestone. On the other hand, MINDFORGE shows substantial improvement across the board both in terms of achieved milestones and efficiency (number of iterations). Notably, Ilama 3.1-70B [71] powered MINDFORGE achieves two additional tech-tree milestones, nearing the performance with the original GPT-4 powered Voyager [II]. We observe a steady increase in efficiency (number of iterations) as the size of the LLM increases: Llama 3.1-70B (7 powered MINDFORGE is 1.62x more efficient than its Llama 3.1. 1-88 counterpart for reaching the wooden tools milestone, suggesting

---

that larger models can better utilize the collaborative learning framework. Furthermore, to understand the impact of collaboration in mixed-expertise collaborative setting, we experiment with non- collaborative MINDFORGE ablation. Notably, MINDFORGE agents without collaboration require on average 15 more prompting iterations to reach the wooden tools milestone and 34 iterations to reach the stone tools milestone. Thus, the collaborative framework not only enhances task completion rates but also enables more efficient acquisition of complex skills. We further test the generalization capabilities of MINDFORGE agents in Appendix A.2 where we show how even after collaboration finishes, the agents can reuse the knowledge in out-ol-distribution tasks.
5.5 MINDFORGE Collaborative Learning Setting
While prior sections focused on instructive learning from stronger peers, we now consider a more challenging setup: collaboration between two MINDFORGE agents of matched capabilitties each powered by the same type of open-weight LLM. This setting reflects mutual cultural learning without access to an expert where the agents must bootstrap improvements solely through interaction, thereby mimicking certain qualities of emergence. Critically, we evaluate not individual trajectories, but performance: after N rounds of communication. what fraction of agent pairs from the total pool succeed in completing the task? As shown in Table 2 when agents begin from unprimed states, collaboration often leads to stagnation or even degradation which mirrors the "blind leading the blind" dynamic, where flawed mental models are mutually reinforced.
instead focus on population-leve
However, even minimal prior expert instruction-sun as single round of GPT----uided collab- oration-can prime agents for more effective peer interaction, even if they still fail when acting alone. As shown in Figure 7 we pair Mixtral-8x7B agents that each received N communication rounds of GPT-4 guided trial before interacting. We observe that the fraction of successful agents steadily increases with additional communication rounds between themselves (Mixtral agents). From 62% baseline population succeeding, we reach 79% of the population succeeding after seven turns, showing a 17% increment. This result reflects a form of test-time compute scaling, where richer interaction, not larger models, drives the emergent improvements across the group.
This also aligns with the Condorcet Jury Theorem 73 which predicts that collective decisions improve when individuals exceed a baseline level of competence. Here, we see distributed variant of this principle in action: once agents cross a minimal competence threshold, the communication induced bootstrapping can yield reliable gains at the population level.
6 Conclusion
The MINDFORGE framework demonstrates that cultural learning can overcome the limitations of open- gpen-weight LLMs in embodied environments. Integrating theory of mind, natural communication, and structured memory systems enables MINDFORGE agents to learn from both mixed and matched expertise peers and overcome fundamental failure modes in perception and code generation. Our experiments show that collaboration significantly improves performance on lifelong learning sce- narios, with agents collecting 2.3x more unique items and achieving 3x more tech-tree milestones when paired with expert agents. Crucially, in the collaborative setting, we show that the fraction of successful agents scales function of communication rounds; revealing emergent gains through perspective taking and interaction. These results suggest that social learning could reduce embodied Al's dependence on large proprictary models, opening new paths for democratized agent development aligned with Green Al principles 774
Acknowledgments and Disclosure of Funding
We thank OpenAl's Researcher Access Program for providing API credits that enabled more com- prehensive experiments. We are also grateful to Dr. Max van Duijn for his constructive critique and insights on Theory of Mind, to Dr. Jesse Krijthe for his thoughtful input on the Condorcet Jury Theorem, and to members of the Pattern Recognition Lab at TU Delft for their valuable feedback.
10

---

References
[1] Eric Kolve, Roozbeh Mottaghi, Daniel Gordon, Yuke Zhu, Abhinav Gupta, and Ali Farhadi. Ai2-thor: An interactive 3d environment for visual ai. arXiv preprint arXiv:1712.05474, 2017.
[2] Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub. Jia Liu, Vladlen Koltun, Jitendra Malik, et al. Habitat: A platform for embodied ai research. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9339,9347, 2019.
[3] Xavier Puig, Kevin Ra, Marko Boben, Jiaman Li, Tingwu Wang, Sanja Fidler, and Antonio Torralba. Virtualhome: Simulating houschold activities via programs. Proceedings of the IEEE Conference on Computer Vision and Pattenn Recognition, pages 8494-8502, 2018.
[4] Mohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk, Winson Han, Roozbeh Mot- taghi, Luke Zettlemoyer, and Dieter Fox. Alfred: A benchmark for interpreting grounded instructions for everyday tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Putlern Recognition, pages 10740-10749, 2020.
[5] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv: 1XXiv:777.006888 2017.
[6] Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse domains through world models. arXiv preprint arXiv:2301 00000.441000 104, 2023.
[7] Maryam Zare, Parham M Kebria, Abbas Khosravi, and Saeid Nahavandi. A survey of imitation learning: Algorithms, recent developments, and challenges. IEEE Transactions on Cybernetics, 2024.
[8] Karl Cobbe, Oleg Klimov, Chris Hesse, Taehoon Kim, and John Schulman. Quantifying generalization in reinforcement learning. In International conference on machine learning. pages 1282-1289, PMLR, 2019.
[9] German I Parisi, Ronald Kemker, Jose Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: A review. Neural Networks, 113:54-71, 2019.
[10] Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu A comprehensive survey of continual learning: Theory, method and application. arXiv preprint arXiv:2302.00453 2023.
[11] Guanzhi Wang, Yugi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open- open-ended embodied agent with large language models. arXiv preprint 23xv:2305.1629, 2023..
[12] Nicholas K Humphrey. The social function of intellect. Growing points in ethology, pages 303-317, 1976.
[13] Robin IM Dunbar. The social brain hypothesis. Evolutionary Anthropology: Issues, News, and Reviews, 6(5):178-199) 1998.
[14] Michael Tomasello, Ann Cale Kruger, and Hilary Horn Ratner. Cultural learning. Behavioral and brain sciences, 16(3):495-511, 1993.
[15] Michael Tomasello. Cultural learning redux. Child development, 87(3):643-66 6633, 2016.
[16] David Premack and Guy Woodruff. Does the chimpanzee have theory of mind? Behavioral and brain sciences, 14)::1155526 1978.
[17] Simon Baron-Cohen. The evolution of theory of mind. In Michael C Corballis, editor, The descent of mind: Psychological perspectives on hominid evolution (pp. volume 361, pages 261-277. Oxford University Press, xii, 1999. URL 1999-04113-01
[18] Marie Devaine, Guillaume Hollard, and Jean Daunizeau. The social bayesian brain: does mentalizing make difference when we learn? PLoS computational biology, 10(12)e1000392, 2014.
11

---

[19] Geoffrey Hinton. Distilling the knowledge in neural network. arXiv preprint arXiv:1503.0255: 2015.
[20] Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling Ilm test-time compute opti- mally can be more effective than scaling model parameters. arXiv preprint arX3v:2408.033314 2024.
[21] Hongxin Zhang, Weihua Du, Jiaming Shan, Qinhong Zhou, Yilun Du, Joshua B Tenenbaum, Tianmin Shu, and Chuang Gan. Building cooperative embodied agents modularly with large language models. arXiv preprint Sexiv:2307.024833 2023.
[22] Yilun Du, Shuang Li, Antonio Torralba, Joshua B Tenenbaum, and Igor Mordatch, Improv- ing factuality and reasoning in language models through multiagent debate. arXiv preprint arXiv:2305. 14325, 2023.
[23] Kanishk Gandhi, Jan- lan-Philipp Fränken, Tobias Gerstenberg, and Noah Goodman. Understanding social reasoning language models with language models. Advances in Neural Information Processing Systems, 36, 2024.
[24] John E Laird. The Soar cognitive architecture. MIT press, 2019.
[25] Neil Rabinowitz, Frank Perbet, Francis Song, Chiyuan Zhang, SM Ali Eslami, and Matthew Botvinick, Machine theory of mind. In International conference on machine learning, pages 4218-4221 PMLR, 2018.
[26] Yun-Shiuan Chuang, Hsin- Yi Hung. Edwinn Gamborino, Joshua Oon Soo Goh, Tsung- Ren Huang, Yu-Ling Chang, Su-Ling Yeh, and Li-Chen Fu. Using machine theory of mind to learn agent social network structures from observed interactive behaviors with targets. In 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pages 1019-1019. IEEE, 2020.
[27] Tianmin Shu, Abhishek Bhandwaldar, Chuang Gan, Kevin Smith, Shari Liu, Dan Gutfreund, Elizabeth Spelke, Joshua Tenenbaum, and Tomer Ullman. Agent: A benchmark for core psychological reasoning. In International conference on machine learning, pages 9614-9625.
PMLR, 2021.
[28] Fabio Grassiotto and Paula Dornhofer Paro Costa. Cogtom: A cognitive architecture implemen- tation of the theory of mind. In ICAART (2), pages 546-553, 2021.
[29] Chris Baker, Rebecca Saxe, and Joshua Tenenbaum. Bayesian theory of mind: Modeling joint belief-desire attribution. In Proceedings of the annual meeting of the cognitive science society, volume 33, 2011.
[30] Michal Kosinski. Theory of mind might have spontaneously emerged large language models. arXiv preprint 202.022302.000083, 2023.
[31] Sébastien Bubeck, Varun Chandrasekaran. Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303. 12712, 2023.
[32] Tomer Ullman. Large language models fail on trivial alterations to theory-of-mine tasks. arXiv preprint arXiv:2302.00839, 2023.
[33] Alex Wilf, Sihyun Shawn Lee, Paul Pu Liang, and Louis-Philippe Morency. Think twice: Perspective-taking improves large language models' theory-of-ming capabilities. arXiv preprint arXiv:2311.1022 2023.
[34] Pei Zhou, Aman Madaan, Srividya Pranavi Potharaju, Aditya Gupta, Kevin R McKee, Ari Holtzman, Jay Pujara, Xiang Ren, Swaroop Mishra, Aida Nematzadeh, et al. How far are large language models from agents with theory-of-mint arXiv preprint arXiv:2310.0305 2023.
[35] Melanie Sclar, Sachin Kumar, Peter West, Alane Suhr, Yejin Choi, and Yulia Tsvetkov. Minding language models' (lack of) theory of mind: A plug-and-play multi-character belief tracker. arXiv preprint a006v:2306. 00924, 2023,
12

---

[36] Shibo Hao, Yi Gu, Haodi Ma, Joshua Jiahua Hong, Zhen Wang, Daisy Zhe Wang, and Zhit- ing Hu. Reasoning with language model is planning with world model. arXiv preprint arXiv:2305. 14992, 2023.
[37] Chuanyang Jin, Yutong Wu, Jing Cao, Jiannan Xiang, Yen-Ling Kuo, Zhiting Hu, Tomer Ullman, Antonio Torralb,, Joshua Tenenbaum, and Tianmin Shu. Mmtom-get Multimodal theory of mind question answering. arXiv preprint arXiv:2401.08743 2024.
[38] Huao Li, Yu Quan Chong, Simon Stepputtis, Joseph Campbell, Dana Hughes, Michael Lewis, and Katia Sycara. Theory of mind for multi-agent collaboration via large language models. arXiv preprini arXiv:2310. 10701, 2023.
[39] Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, et al. Do as i can, not as say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.0169) 2022.
[40] Kevin Lin, Christopher Agia, Toki Migimatsu, Marco Pavone, and Jeannette Bohg. Text2motion: From natural language instructions to feasible plans. Autonomous Robots, 4745):1355-1365, 2023.
[41] Hongxin Zhang, Zeyuan Wang. Qiushi Lyu, Zheyuan Zhang, Sunli Chen, Tianmin Shu, Yilun Du, and Chuang Gan. Combo: Compositional world models for embodied multi-agent coopera- tion. arXiv preprint arXiv:2404. 10775, 2024.
[42] Yashar Talehirad and Amirhossein Nadiri. Multi-agent collaboration: Harnessing the power of intelligent Ilm agents. arXiv preprint vrXiv:2306.03314. 2023.
[43] Alexander Sasha Vezhnevets. John Agapiou, Avia Aharon, Ron Ziv, Jayd Matyas, Edgar Duéñez-Guzmán, William A Cunningham, Simon Osindero, Danny Karmon, and Joel Z Leibo. Generative agent-based modeling with actions grounded in physical, social, or digital space using concordia. arXiv preprint arXiv:2312.0366/0 2023.
[44] Isadora White, Kolby Nottingham, Ayush Maniar, Max Robinson, Hansen Lillemark, Mehul Maheshwari, Lianhui Qin, and Prithviraj Ammanabrolu. Collaborating action by action: A multi-agent I1m framework for embodied reasoning. arXiv preprint rrXiv:2504.17551 2025.
[45] Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. Expel: Llm agents are experiential learners. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 19632-19642, 2024.
[46] Bodhisattwa Prasad Majumder, Bhavana Dalvi Mishra, Peter Jansen, Oyvind Tafjord, Niket Tandon, Li Zhang, Chris Callison- Burch,, and Peter Clark. Clin: A continually learning language agent for rapid task adaptation and generalization. arXiv preprint arXiv:2310.10134 2023.
[47] Zihao Wang, Shaofei Cai, Guanzhou Chen, Anji Liu, Xiaojian Ma, and Yitao Liang. Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents. arXiv preprint arXiv:2302.01155 2023.
[48] Archiki Prasad, Alexander Koller, Mareike Hartmann, Peter Clark, Ashish Sabharwal, Mohit Bansal, and Tushar Khot. Adapt: As-needed decomposition and planning with language models. arXiv preprint arXiv:23 010.22211.07777 105572, 2023.
[49] Kolby Nottingham, Bodhisattwa Prasad Majumder, Bhavana Dalvi Mishra, Sameer Singh, Peter Clark, and Roy Fox. Skill set optimization: Reinforeing language model behavior via transferable skills. arXiv preprint 4rXiv:2402.03244, 2024.
[50] Shu Yu and Chaochao Lu. Adam: An embodied causal agent in open-world environments. arXiv preprint arXiv:2410.22114, 2024.
[51] Zaijing Li, Yuquan Xie, Rui Shao, Gongwei Chen, Dongmei Jiang, and Liqiang Nie. Optimus-1: Hybrid multimodal memory empowered agents excel in long-horizer tasks. arXiv preprint arXiv:2408.0 03615, 2024.
13

---

[52] H M Wellman and J D Woolley. From simple desires to ordinary beliefs: the early development of everyday psychology. Cognition, 35(3):245-22) 275, jun 1990. doi: 10.10160001100027709)) 0277 (90) 90024-E
90024-e. htRl:/tt::///xx doi org/
[53] Josef Perner. Understanding the Representational Mind. The MIT Press, 03 1993. ISBN 9780262316033, doi: 10.755 http///meeee//88800.......
httpssts:://oo org/10.
sitpress/6988 011.001100000 001.00001000
[54] Kevin N Ochsner, Rebecca D Ray, Jeffrey C Cooper, Elaine R Robertson, Sita Chopra, John D E Gabrieli, and James Gross. For better or for worse: neural systems supporting the cognitive
down- and up-regulation of negative emotion. Neuroimage, 23(2):483-499, oct 2004, doi: 10. 10.1016/j.neuroo www..nuuuooiee200e.... 200..00.000.0. URL 2004. 26. 030
230 1ttp://tx..ii..001111..1..11.11. wttp://jjeuuooooo neer neeroomage
[55] Saxe and N Kanwisher. People thinking about thinking people. the role of the temporo- parietal junction in "theory of mind". Neuroimage, 19(4):1835-188 1842, August 2003. doi: 001166111055-11110000222222
htt.ttttt:///bbaeee ncbi nlm. nih. gov/ 11/48939998/
[56] Anand S Rao, Michael P Georgeff, et al. Bdi agents: from theory to practice. In Iemas, volume 95, pages 312-319, 1995.
[57] Hayley Hung, Litian Li, Jord Molhoek, and Jing Zhou. The discontent with intent estimation in-the-wild: The case for unrealized intentions. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, pages 1-9, 2024.
[58] PrismarineJS. Mineflayer, 2021. URL URL
[59] Aaron M Bornstein, Mel W Khaw, Daphna Shohamy, and Nathaniel D Daw. Reminders of past choices bias decisions for reward in humans. Nature Communications, 8(1):15958, 2017.
[60] S Klein, L Cosmides, C Gangi, Betsy Jackson, J Tooby, and Kristi A Costabile. Evolution and episodic memory: An analysis and demonstration of social function of episodic recollection. Soc. Cogn., 27(2):283 319, 21 apr 2009, doi: 10. 10.1521OSOCO. 0002/////..22......... 27.2..28..... URL UtLhttp
//guilfordjouree ournals. com www.ccoocccocccccccccc /tt////o1111111//// 1521 2009 27 22........ 283
[61] Stanley B Klein, Judith Loftus, and John F Kihlstrom. Self-knowledge of an amnesic patient: Toward neuropsychology of personality and social psychology. b; Tulving, 1983:1989, 1993.
[62] Payel Das, Subhajit Chaudhury, Elliot Nelson, Igor Melnyk. Sarath Swaminathan, Sihui Dai, Aurélie Lozano, Georgios Kollias, Vijil Chenthamarakshan, Soham Dan, et al. Larimar: Large language models with episodic memory control. arXiv preprint arXiv:2403. 1901, 2024.
[63] Jeffrey Binder and Rutvik Desai. The neurobiology of semantic memory. Trends Cogn. Sci., 15(11):527-536 1 nov 2011. doi: 10. t1.11//.ies.2201110.000 URL http: doi 111/1/..016//..oar tics 2011 10.001
[64] Linxi Fan, Guanzhi Wang. Yunfan Jiang, Ajay Mandlekar, Yuncong Yang, Haoyi Zhu, Andrew Tang, De-An Huang. Yuke Zhu, and Anima Anandkumar. Minedojo: Building open-ended embodied agents with internet-scale knowledge. Advances in Neural Information Processing Systems, 35:18343-18865 2022.
[65] Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts. Dspy: Compiling declarative language model calls into self-improving pipelines. 2024.
[66] Omar Khattab, Keshav Santhanam, Xiang Lisa Li, David Hall, Percy Liang, Christopher Potts, and Matei Zaharia. Demonstrate-searchh predict:: Composing retrieval and language models for knowledge-intensive NIP. arXiv preprint 2.t.:22//..44222. 2022.
[67] LangChain. Langchain documentation, 2024. Accessed: 2024-11-13
URL https: www.chainnga..ooooo
14

---

[68] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, 1lge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08877, 2023.
[69]
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7b, 2023. URL https://arxia org/ argg/bbb//311.06022222
[70] Albert Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, al. Mixtral of experts. arXiv preprint arXiv:2401 2xXx:22401004088,, 2024.
[71] Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint urXiv:2407.21783, 2024.
[72] Zeyu Han, Chao Gao, Jinyang Liu, Sai Qian Zhang, et al. Parameter-efficient fine-tuning for large models: A comprehensive survey. arXiv preprint 14X6:02403.14668, 2024.
[73] David Austen-Smith and Jeffrey S Banks. Information aggregation, rationality, and the condorcet jury theorem. American political science review, 90(1):34-45 45, 1996.
[74] Roy Schwartz, Jesse Dodge, Noah A. Smith, and Oren Etzioni. Green AI, August 2019.
[75] Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2 09685, 2021.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We back up the claims made in the abstract and introduction in Section 5 where we run extensive experiments in Minecraft and show the benefits of the proposed framework. Furthermore, we offer qualitative results through various figures and text snippets the manuscript. These are available in the main paper as well in the appendix (e.g. Appendix (D.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [No]
Justification: We acknowledge the limitations of MINDFORGE and discuss them through- out the paper when necessary. That said we do not have a standalone Limitations section.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
16

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification:
The manuscript does not include any theoretical result.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Since our method relies on LLMs, we provide a full comprehensive list of the models used (via API), the underlying prompts used to query the L.LM as well as extensive details in Section 5 that allows for potential reproducibility. Importantly, our method uses significant scaffolding enable multi-agent communication in Minecraft. While we do not include code snippets in the manuscript, we provide the full code as part of the Supplementary Materials.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
17

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: We provide the full code repository used execute any experiment referenced in the manuscript in the Suppiementary Material. We anonymize the code and provide adequate instructions inside the README for reproducibility.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cot
(eallic/guiee/Cooe /wodeSSumissiiioool11..... httpa////iiaes/odduuuiiis.ssssss
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
puaiss.cc/uublicguuiddes Soa.eeeeee/oooeSeeeeeeeeeeeeeeee wetssuuumiisiiiinii... sww.iioiiciiocicooooooooo for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Since the method to interact with LLMs is through text prompts, we provide these artifacts together with how we set up the experiments. These details can be found in Section 5 The full details (code) are provided in the Supplementary Material.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
18

---

Justification: We report eтrоr bars where possible throughout this manuscript. Specifically, we provide enror bars for the Minecraft tech-tree experiments in Table Moreover, to ensure proper statistical significance, we perform large number of individual trials across all the experiments Section
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: While our paper is not based on direct compute resources like GPUs, we do acknowledge in Appendix the cloud providers used throughout the experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: We have followed the NeurIPS Code of Ethics while writing and working at this manuscript.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
19

---

The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification: While the MINDFORGE framework is designed to work with both humans and agents, we believe there is no foreseeable negative societal impact that can stem from this specific manuscript. However, we do mention that our framework might decrease the dependence on proprietary models and thus affect in positíve way global problems like climate change.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: Our paper does not pose such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
20

---

Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes)
Justification: We credit the LLM providers in Appendix both for crediting their work and helping the reader reproduce the results in the paper.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website). the copyright and terms of service of that source should provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thooee cow com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: Since MINDFORGE makes of existing open-weight LLMs, our paper does not release any new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The manuscript and research process does not involve crowdsourcing nor research with human subjects. While Table 2 showcases a feasibility study with a human expert, the human is one of the authors.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
21

---

According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based on the requirements of your country institution) were obtained?
Answer: [NA]
Justification:
The manuscript and research process does not involve crowdsourcing research with human subjects. While Table showccssss a feasibility study with a human expert, the human is one of the authors.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Answer: [Yes]
Justification: We not use L.MSs to formulate our research problem however L.LMs are being used for the agents that we are proposing.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy httt:p//eeerrrreeccc/oooeerrrrrp for what should or should not be described.
22

---

MineF:rge: Empowering Embodied Agents with Theory of Mind for Lifelong Cultural Learning Appendices
A Additional Experimental Results
A.1 Collaborative Experiments
We provide an extended set of experiments to reinforce our findings in Figure S and Figure 7 Specifically, we consider atomic tasks that are significantly harder than dirt and wood collection in order to demonstrate the generality of our framework: (1) mining an iron block and (2) crafting wooden pickaxe.
Table 4: Agent performance increases as the number of rounds grows in both collaborative and instructive settings. In the instructive setting, Llama 3. 3-1- the weak agent and GPT-4 as the strong agent. Similarly, collaborative experiments use Ilama 3.1-70B for both MINDFORGE agents.
Setting
Task
Comm. Round
Comm. Round I
Comm. Round
Comm. Round 3
Instructive
Mine Iron
41%
50%
54%
62%
Instructive
Craft Pickaxe
20%
33%
41%
45%
Collaborative
Craft a Pickaxe
20%
25%
33%
33%
As Table shows, extend communication and perspective-taking between MINDFORGE agents enable significant improvements on complex Minecraft tasks that require compositionality (41% + 62% and 20% + 45%). Similarly to Figure performance gains (20% > 33%) than instructive learning, due to reinforcing wrong beliefs and similar thinking patterns.
that purely collaborative setting results in smaller
A.2
Colt-Collaboration and OOD tasks
Do social interactions have long-term effects on agents and help them perform better on different tasks? We consider the impact of interactions in post-collaboration and out-of-distribution scenarios to assess MINDFORGE'S generalization and lifelong learning capabilities. Specifically, we define post- collaboration scenarios non-collaborative setting where the MINDFORGE agent tries to complete a task based on prior beliefs formed from a previous collaborative experience stored in the semantic memory. Moreover, consider scenarios where the agent is tasked with solving novel task with prior collaborative experience on an adjacent task in the Minecraft tech-tree. As Table 3 shows, collaboration between MINDFORGE agents together with semantic memory showcase average improvement of 8.33% for in-distribution tasks (dirt collection) together with an average improvement of 8. 78% in non-collaborative out-of-distribution scenarios (wood collection). These results showcase the importance of semantic memory in achieving continual learning through collaboration.
Table 5: Post-collaboration and out-of-distribution generalization of the MINDFORGE agent. We report the task completion rates across 24 individual trials.
Scenario
Task
Mistral-7B
Mixtral-8x7B
Llama 3.1-BB
No Collaboration
37.5%
29.15% 41.66%
12.3% 25%
Collaberation
37.5%
No Collaboration
Wood
33.3%
50%
7%
OOD post collahoration
Wood
41.66%
58.33%
16.66%
A.3 Alternative Communication Protocols
Table oJsows the performance of MINDFORGE agents when they are allowed to begin a conversation point compared to when they can only initiate a conversation upon failure.
23

---

Table Comparison between the default MINDFORGE communication protocol and a more flexible alternative where agents can initiate the conversation at any point. Both protocols achieve similar results across different model sizes.
Task Completion Rute
Model
Task
Setring
Round
Round
Round
Round
MindForge flexible communication MindForge
37%
45%
67%
Mine dirt
29%
42%
62%
67%
Mixtral-Sx7B
MindFForge llexible communication MindForge
83%
Mine dirt and wood
79%
83%
MindForge ficxible communication MindForge
37%
42%
45%
54%
Mine dirt
37%
42%
45%
54%
Mistral-7B
MindForge w/ flexible communication MindForge
41%
45%
50%
50%
dirt
41%
45%
50%
50%
B
Additional Ablations
B.1 Perspective Taking
To quantitatively assess the effect of perspective-taking, perform ablation study where attempt to solve a Minecraft task without perspective-taking during communication in an instructive learning setting. Taable/]showcasee how the absence of perspective-taking leads to performance, irrespective of how much agents communicate. Moreover, as the agents use more communication rounds when trying to solve task, taking perspective of the other's s agent situation increases task- completion rate.
Table 7: Perspective- ablation. We consider the Minecraft task of collecting dirt block. Improvement quantified the fraction agents that solve the task across 24 individual trials.
Task Completion Rate
Model Variant
Round
Round
Round
Round 3
MINDFORGE w/ perspective-taking MINDFORGE w/o terkpectiv--taking
29%
42%
61%
67%
29%
37%
50%
54%
Improvement
0%
+5%
+11%
+13%
We attribute the positive correlation between performance and usage of perspective-takinn during communication to the ability of the MINDFORGE agent to provide more relevant and directed advice. This observation is also supported by the substantial increase in performance in the first two rounds as presented in Table 7 where the teacher agent provides more insightful information it gets to know the other's s agent situation better.
Additionally, further ablate structured representation of Theory of Mind by considering unstructured perspective-taking 2-step prompt presented Think Twice (33). Table showcasse that structured ToM representation provides a meaningful advantage complex Minecraft tasks.
Table Structured perspective-takeee taking ablation with unstructured alternative. The unstructured ToM approach is inspired from Think Twice [33]. Using a structured belief-system results in higher performance on complex Minecraft tasks.
Task Completion Rate
Task
Model Variant
Round
Round
Round
Round
Craft Pickaxe
MINDFORGE
20%
33%
41%
45%
Craft a Pickaxe
MINDFORGE w/o structured ToM
20%
29%
33%
41%
MINDFORGE
MINDFORGE structured

---

B.2 Memory Components
MINDFORGE employs three types of memory subsystems, as discussed in Section These components ensure that the agent can reuse and recall relevant information about the task it is trying complete from past experiences and interactions with other agents. The importance of procedural memory with respect to task completion is already studied in [I]], concluding that reusing previous code snippets becomes increasingly important with the difficulty the tasks. To understand the effect of episodic memory in the MINDFORGE framework, we perform an ablation on the episodic memory component within two Minecraft tasks. use of the episodic memory component. While we do not consider the tech-tree experiments here, we expect that considering past failure episodes leads to increasingly better results.
showcases the drop in completion rate without the
Table 9: Ablation the episodic memory component in MINDFORGE single task. We report the task completion rates across 24 individual trials.
Scenario
Task
Mixtral-8x7B
Llama 3.1-818
MINDFORGE episodic memory
Dirt
29.15%
12.5%
MINDFORGE w/o episodic memory
Dirt
25%
16.66%
MINDFORGE w/ episodic memory
Wood
50%
7%
MINDFORGE w/o spisodic memory
Wood
45.83%
7%
C Implementation details
C.1 Perspective Taking
Perspective-taking represents a central part of our framework since allows MINDFORGE agents to take advantage of the internal causal structure of other agents in order to aid more efficient and grounded communication. In practice, perspective-taking corresponds to an L.M call where the prompt contains instructions and relevant information required to understand the internal state of other MINDFORGE agents during communication. Figure g showcases how perspective-taking integrates with the overall communication mechanism.
Communication Channel
Communication Channel
Other Agent: am trying to complete the task using a pickeos However, encountered
Self Agent: You do not require
tools to mine cirt. You
your hands instead.
Update internal representation
Take Perspective
Generate Response
Self Internal Process
Figure 8: Flow of MINDFORGE communication components used to generate a natural language response. The agents' internal process consists of two stages: (i) internal representation update and (ii) perspective-taking Leveraging both processes the agent can then generate a natural language response and send through the Minecraft chat.
Once the agent receives a message in the Minecraft chat, first LLM call executed where the agent updates the stateful internal representation about the agent that sent the message. This update process is shown in Figure Subsequently, the updated internal representation is used as context together with the rest of the conversation take perspective of the situation of the other MINDFORGE agent. Similar to the first stage, perspective-taking is implemented as a LLM call, with the following prompt:
25

---

Prompt for perspective-taking
You Minecraft agent named ((aae)) and you having conversation with another agent named [ [other_namas _aneename))))
Based on the current conversation and your knowledge about the other agent, { [[other_name] ) take the other agent's perspective edge state, and likely needs from [other name) 's perspective.
and describe your current understanding, knowl-
Here is the current conversation between you and { | name h:mee:///omeeee conversation
Here is your mental model of ((other_name) name_ ): (world_model)
Perspective Analysis:
Below you find example of perspective-taking from Minecraft gameplay:
Expert agent taking perspective of weak agent
Weak's current knowledge state includes understanding the biome they are in, the time of day, and the fact that there are dark oak logs nearby. Weak knows that wooden axe is necessary for mining logs, and that crafting this tool will be a required step before proceeding with the task. Weak's ss likely needs from Strong include further assistance in navigating the environment during the night to find a tree or waiting until the day.
C.2 Episodic Memory
We implement episodic memory as a Retrieval- Augmeente Generation (RAG) using LangChain [67] Specifically, we embed all the episodes where the agent failed complete the task using set--embeddingg-ddd---0e
An episode consists of the context used to generate the action, the code itself and the corresponding critic message. Subsequently. when the agent generates new action, the RAG is queried to retrieve the k=5 most relevant failure episodes with respect to the task at hand. Lastly, ensure we do not populate the context window with too many tokens, we generate a summary (using an LLM) of the retrieved episodes and feed this summary in the context used to generate the action. Below you can find the corresponding prompts for generating the episodic summary:
System message
You are a helpful assistant tasked with summarizing past experience episodes and pointing out the causes of failure. Create a concise summary.
Prompt message
Please summarize these episodes and why they failed: dclledddccombiee_epppdddee
C.3 Procedural Memory
The procedural memory has identical design and functionality with the skill library introduced in Voyager [T]. Once a task gets successfully completed, we store the corresponding piece of code such that we can later reference it in adjacent tasks. This becomes increasingly important tasks that require multiple steps, like crafting pickaxe: the agent needs to first create a crafting table, create wooden sticks and only then attempt craft the pickaxe. When we the MINDFORGE framework in an open-ended setting, the procedural memory allows the agent to continuously evolve and create an increasingly large collection of reusable skills.
26

---

C.4 Semantic Memory
As described in Section 4.4 semantic memory stores certain beliefs the agent has about the environ- ment, coming from experience with the environment and the other MINDFORGE agents. Whenever MINDFORGE attempts a new task, it creates a certain belief about how the task should be solved. For example, the agent might believe it requires tools to mine a block of dirt. This belief is then added to the context of the L.LM when the environment action generated. As highlighted Section [ one of the primary failure cases of Voyager are the false beliefs that come from this LLM generation. However, compared Voyager [1], leverage the communication with other MINDFORGE agents such that these beliefs get updated and refined. Below we showcase an example of false belief' correction when trying to collect a block of wood:
Initial belief inside the semantic memory
("How to mine 1 wood log in Minecraft?": "Answer: To mine wood log in Minecraft, you need use an axe." "}
Updated belief after communication
("How to mine 1 wood log in Minecraft?": "Answer: To mine wood log in Minecraft, you need to punch a tree with your bare hands.")
In addition task-related memory, the MINDFORGE agent maintains stateful of beliefs about the other agents in the environment that are constantly updated across episodes. An example of this presented in Figure 6 where the agent updates its representation based on the contents of the conversation. Thus, keeping track of various information and facts about other MINDFORGE agents acts semantic memory retrieval when agents communicate.
D Sample Conversation
Each communication round between two MINDFORGE agents comprises of six sending messages in a turn-based manner through the Minecraft build-in chat. Below you can find a sample conversation showcasing how the expert agent (Strong) helps the weak (Weak) agent.
27

---

Conversation about completing the task: "Mine 1 wood log"
Weak: Hey, can you help me with Mine wood log?
Strong: Absolutely, I can help you with that. Given your current location the dark forest biome and the presence of dark oak logs around you, you should be able to mine a log without any issues. Here's the code you can use: 'mine( 'dark ___o__ak_____ _ak__og') This command will make your agent mine a dark oak log, Just make sure you are close enough to the tree when executing this.
Weak:
Thank you for the helpful guidance, Strong! However, in my case, am currently in dark forest biome as well, but am having trouble finding tree during the night. Could you suggest a way to find a tree during the day so can mine the wood log? I also noticed that don't it have wooden axe in my inventory, so I may need to craft one before attempting to mine the log. I appreciate your assistance in this matter.
Strong: course, understand your predicament. Since it's currently night time, visibility might be low which makes it challenging to locate tree. You wait until daybreak for better visibility. Use the 'wait until day()* function to wait until it's daytime. For crafting wooden axe, you would need wood planks and 2 sticks. You obtain wood planks from a wood log by using the 'craft(" wooden fali_pllank_' wood wood_log)' command and sticks from 2 wooden planks using the "craft( sttck',, wooden you can craft a wooden axe using the wooden_plank) command. Once you have these, _axe,, wrooden_plank, stick) function.
Here's the code: <c.
wait_ until_day()
craft ( 'wooden_ppank', wood_log)
wooden_plari
craft craft('woodens wwwoden_axe""" gooden_plank, stick)
Once you have your and it's day time, you should be able easily locate and mine tree for the wood log.
Weak: Thank you, Strong. appreciate your help.
E
Fine-tuning setup
E.1
Execution Logs
For the first fine-tuning variant, we use execution logs generated by a Voyager agent powered by GPT- 4. Specifically, these logs contain input and output text information obtained from the components specific to the architecture, such as the actor or the critic. We find this to be a meaningful way transfer knowledge as well as reasoning capabilities from GPT-4 Mistral-7B. To perform the fine-tuning, we use LoRA 75 due to its low computational requirements.
E.2
Wiki Data and Documentation
The second fine-tuning variation consider builds upon the initial setup presented above and incorporates more structured information about Minecraft by including the Minecraft Wiki [64] and Minecraft API documentation. Similar to the first variant LoRA [75]
28

---

Prompts
Prompt for updating partner beliefs
You Minecraft agent.
You just had a conversation with another agent based on a task you are trying to solve.
Based the contents of the conversation and the previous beliefs,, you have create set of beliefs that represent your perception of the other agent.
Prompt for creating interaction belief's
You are Minecraft agent.
You just had conversation with another agent based on task you trying to solve.
Based the contents of the conversation and the previous beliefs, you have to create set of beliefs that that can help you complete the task.
Prompt for perspective-taking
You Minecraft agent named ((name) and you having conversation with another agent named ([other_name])
Based on the current conversation and your knowledge about the other agent, footherr_aame ), take the other agent's perspective assess and describe your current understanding, knowl- edge state, and likely needs from [other_name, name] ) 's perspective.
Here is the current conversation between you and [ Bactrr__amm_e__ | [ ccnversation))
Here is your mental model of [other name ): [oedd_
Perspective Analysis:
G LLMs
This section details the technical specifications of the Large Language Models (LLMs) and related infrastructure used to implement the MINDFORGE framework and conduct the experiments presented in this paper.
All LLM calls, unless explicitly stated otherwise, were executed via API calls external services. The exception to this is the fine-tuned model, which was ran and trained locally.
The specific models and their corresponding access methods outlined below:
Llama 3.1-8B-Instruee This model accessed through the Lambda Inference API This API provided reliable and efficient interface for interacting with the Llama 3. Inst-uust model, enabling seamless integration within the MINDFORGE framework.
GPT-4: Accessed via the OpenAl API. The OpenAl API offered access to the advanced capabilities of the GPT-4 model, crucial for establishing performance baselines and for the expert agent in collaborative settings.
Mistral-7B Instruct v0.2, Mixtral-8x7B Instruct and Llama 3.1-70B: These Mistral
models were accessed through the Together AI API The Together Al API provided
https://api. ai/moeel
29

---

platform for utilizing these open-weight models, allowing for a comparative analysis within the Voyager and MINDFORGE frameworks.
30


Reviews:
[{"review_id": "RHZgtSxyRc", "reviewer": "Reviewer_qoax", "summary": "The paper proposes MINDFORGE, a multi-module framework that augments open-weight LLM-based embodied agents with a causal Theory-of-Mind template, episodic + semantic + procedural memory, and multi-round natural-language communication. Knowledge is primarily distilled from a single expert agent. During inference, two MINDFORGE agents can collaborate, exchanging beliefs and action plans to refine their policies without gradient updates. Experiments are conducted in Minecraft, where the authors report higher success rates on two basic resource-collection tasks and deeper advancement along the tech tree relative to the open-weight Voyager baseline.", "questions": "1. Broaden the task set: It would be great include success-rate or reward curves on a wider range of MineDojo goals spanning different difficulties.\n2. Deeper ablation: Provide separate ablations that disable each memory subsystem (episodic, semantic, procedural) and the ToM causal graph to quantify individual contribution.\n3. Compare communication cost to expert cost: Multi-round peer communication may involve more cumulative LLM tokens than a single expert rollout. Please report token counts, latency for (a) expert-only, (b) distilled MINDFORGE collaborative.", "limitations": "Yes", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n1. Clear architectural decomposition: The paper articulates how ToM, memory subsystems, and communication interact, with helpful diagrams and prompt listings.\n2. Initial evidence that social interaction with an expert can offset smaller model capacity: Even with open-weight LLMs, the distilled agent surpasses Voyager on the reported metrics.\n\nWeakness:\n1. Limited novelty in the core learning mechanism: The main performance gains come from distilling reasoning produced by a stronger expert model; the distillation pipeline itself follows standard practice.\n2. Questionable effectiveness under purely collaborative use: When two identical MINDFORGE agents collaborate—without an expert—the paper reports a performance drop after communication, hinting that the proposed belief-exchange may introduce noise rather than useful signal.\n3. Narrow evaluation scope: Quantitative success-rate numbers are provided only for dirt and wood collection, two of the simplest Minecraft tasks, leaving the bulk of the MineDojo/Minecraft benchmark unexplored.\n4. Insufficient ablations for a complex system: MINDFORGE chains many hand-engineered modules (three memory types, BigToM belief graph, skill library, conversation scheduler). The importance of each piece is unclear.", "was_revised": true, "final_justification": "As illustrated in my review and response to the authors' rebuttal, I still find the tasks set and baselines not sufficient to support the authors' claim about the BDI framework. \n\nThe original tasks \"dirt\" and \"wood\" are two simplest tasks possible in Minecraft, but most of the experiments and ablation studies are based on them. Although the authors added \"craft pickaxe\" and \"mine iron\" in the rebuttal phase, the authors did not include the important corresponding ablation studies. I am also skeptical about the \"mine iron\" results, as it is supposed to be much harder than \"dirt\" and \"wood\", but the difficulty does not negatively affect the performance (dirt 67%, mine iron 62%). \n(I apologize that I overlooked these results at the beginning of the discussion, which did not leave the authors an opportunity to explain about this)\n\nThe authors could not provide a baseline framework that communicates with a stronger model for the fundamental tasks (non-life long settings), which leaves only the comparison between MindForge and Voyager (which does not support the communication). \n\nMy rating will remain borderline reject. As I am not an expert of multi-agent communication, I am not sure if the current baseline settings are common in the field. I will lower my confidence score.", "rebuttal": "Thank you for your review qoax! We believe several of your comments contain factual inaccuracies (such as experiments and ablations we have already included in the paper) and misinterpretations of our contributions, which we clarify below. \n\n1. > Broaden the task set. include success-rate or reward curves on a wider range of MineDojo goals spanning different difficulties.\n\nIn fact, our main experiment (Fig.1b and Table 3) demonstrates MindForge agents’ effectiveness in an **open-ended setting, for multiple tasks along the Minecraft tech-tree with increasing complexity**, corresponding to MineDojo goals spanning different difficulties. Specifically, **we do not restrict the set of tasks MindForge agents are performing during lifelong learning but rather let the curriculum continuously choose diverse tasks in the environment.** The success-rate and reward curves presented indicate MineForge agents effectiveness on these diverse tasks over the Voyager counterparts.\n\nWe specifically report detailed results later in the paper on the simplest Minecraft tasks (dirt, wood) since the base Voyager agents failed at even these simple tasks when powered by open-weight LLMs. \n\nNevertheless, for further clarity for more complex tasks up the tech-tree (crafting pickaxe, mining iron), we ran additional experiments quantifying the effect of communication on task success rate ( like in Figure 4)  with a MindForge agent (LLaMA 70B):\n\n| Task | Comm. Round 0 | Comm. Round 1 | Comm. Round 2 | Comm. Round 3 |\n| --- | --- | --- | --- | --- |\n| Craft a pickaxe | 20% | 33% | 41% | 45% |\n| Mine iron | 41% | 50% | 54% | 62% |\n\nIn the *instructive setting*, **MindForge agents improve task success rate by +25% for crafting a pickaxe and 21% for mining an iron ore.** Together with the existing results in Figure 4, Figure 1b) and Table 3, these demonstrate that MindForge agents can learn through natural language communication in lifelong learning settings, where tasks become increasingly hard. \n\n| Task | Comm. Round 0 | Comm. Round 1 | Comm. Round 2 | Comm. Round 3 |\n| --- | --- | --- | --- | --- |\n| Craft a pickaxe | 20% | 25% | 33% | 33% |\n\nIn the *collaborative setting* two MindForge agents, we see that in a medium complex task like crafting a pickaxe, **collaboration leads to an increase from $20$% to $33$%**.\n\n2. > Deeper ablations: Provide separate ablations that disable each memory subsystem (episodic, semantic, procedural) and the ToM causal graph to quantify individual contribution.\n\nIn fact, **we already provide ablations for our novel contributions (specific memory components and perspective-taking)** (L269-72):\n\n*Ablations on memory subsystems*\n\n- **episodic memory (Appendix C Table 6)**: MindForge agents perform worse when there is no episodic memory to remember the agent's failures.\n- **semantic memory (Appendix A Table 4)**: positive contribution; the MindForge agent uses semantic knowledge gained through prior collaboration to solve a task either in-distribution or out-of-distribution.\n- **procedural memory**: we do not repeat the ablation since Voyager [60] already performs this experiment and shows the dramatic decrease in lifelong learning performance when this component is removed.\n\n*Perspective Taking*\n\nIn Table 5, Appendix B **we have already performed an ablation of perspective-taking as a whole in the MindForge framework, which together with the structured representations yields a boost in task success as the number of communication rounds increases.** A visually intuitive example of how partner perspective evolves with communication is shown in Fig. 5.\n\nPlease note that performing an ablation on an individual component within the Belief-Desire-Intention framework by Bratman [R1] is not conceptually viable: a lifelong learning agent needs to have a goal (desire) and intention (action) to perform any task in the environment.\n\nNevertheless, **we have additionally performed an ablation comparing our structured representation of Theory of Mind in MindForge with unstructured perspective-taking through a simple 2-step prompt as in Think Twice [64].** Results below indicate that structured representations provide a meaningful advantage.\n\n| Craft a pickaxe | Comm. Round 0 | Comm. Round 1 | Comm. Round 2 | Comm. Round 3 |\n| --- | --- | --- | --- | --- |\n| **MindForge w/o structured ToM** | 20% | 29% | 33% | 41% |\n| **MindForge**  | 20% | 33% | 41% | 45% |\n\n| Mine iron | Comm. Round 0 | Comm. Round 1 | Comm. Round 2 | Comm. Round 3 |\n| --- | --- | --- | --- | --- |\n| **MindForge w/o structured ToM** | 41% | 50% | 54% | 58% |\n| **MindForge** | 41% | 50% | 54% | 62% |\n\n\n3. > Questionable effectiveness under purely collaborative use. The paper reports a performance drop after communication.\n\nPotential misunderstanding about the goal: **we are not looking to provide agents for 'use' towards the best improvement in the Minecraft tasks. Rather, we are investigating social interactions (perspective-taking specifically) as a means for for continual improvement of agents that fail in isolation.**\n\nIn Fig. 6, we show that in the collaborative setting with extended communication regime, MindForge agents actually improve their task success rate depending on the initial expertise. This is still a positive scientific result characterizing the conditions when collaboration with symmetric expertise can work over the instructive setting. As we discuss in Sec. 5.5, the “blind leading the blind” scenario where agents below a certain individual performance threshold can reinforce mistakes is well known within the Jury theorem [3], and our results should be interpreted within this context.  \n\nSolving Minecraft tasks requires two core capabilities: \n\n- i: factual knowledge about the game \n\n- ii: the ability to translate that knowledge into appropriate actions. \n\nIn a collaborative setting, two agents of similar level cannot obtain new factual knowledge (unless there is an external environment signal), yielding the question: can two agents of similar level cooperate to correct their code capabilities with the right facts, reducing hallucination?\n\nIntuitively, *both communication and belief systems are as powerful as the capabilities of the underlying models allow it.* While a purely collaborative scenario does not work out of the box with the set of open weights models, Figure 6 shows that collaborative MindForge agents can thrive given better initial expertise.\n\nWe show **both quantitatively (see Figure 4) as well as quantitatively (see Figure 5) that the proposed system allows for a gradual increase of task performance as more communication is allowed between agents.**\n\n4. > Limited novelty.. The main performance gains come from distilling reasoning produced by a stronger expert model; the distillation pipeline itself follows standard practice.\n\nIn fact, **our incorporation of the Cultural Learning [56] for multi-agent embodied interactions has never been attempted in literature to the best of our knowledge.**  We also tackle two modes of Cultural Learning throughout the manuscript: *instructive* and *collaborative* learning. \n\nFurther, **our mode of knowledge distillation doesn't follow standard practice since we employ structured theory of mind representations along with memory.** In current literature, there are two main approaches to distill knowledge between different models: knowledge distillation by matching the distribution of the expert [22] and generating task specific data with the teacher model and performing supervised fine-tuning [R2]. We specifically tackle the latter case as a baseline (see Table 1) and find that distillation through supervised fine tuning just marginally improves the performance of failing Voyager [60] agents on the simplest tasks inside Minecraft: collecting dirt and wood blocks.\n\nThe approach **we propose in MindForge uses the in-context learning abilities of the models to learn more efficiently in terms of computation, avoiding expensive and data intensive weight updates.** Additionally, owing to natural language communication, MindForge works in human-agent settings too. We have also tested this scenario (Table 2), where interacting with a human results in improved task success rate (Mixtral-8x7B goes from 29.15% to 87% task success).\n\n5. > Compare communication cost to expert cost: Multi-round peer communication may involve more cumulative LLM tokens than a single expert rollout. Please report token counts, latency...\n\nReporting the **absolute token count would not be a meaningful way** of measuring cost since token count is depended on the prompt, model variant, type of model (base, instruct, reasoning) and API provider, to which our framework is agnostic. Instead, **we consider the number of actions needed to complete an objective** as the go-to measure when it comes to performance and trade-off, which we have reported in Fig. 1b.\n\nPotential misinterpretation of goal: **we are not seeking to reduce cost compared to expert rollouts in an engineering sense, but rather to get rid of the reliance on large, pretrained experts in favor of in-deployment skill acquisition.**\n\nThis means that if we look at things like token counts, we must also account for the computational cost of pretraining the experts for a fair comparison overall. Instead, as we show in Fig. 1b, MindForge agents **take fewer number of actions to reach a tech-tree milestone** as well as **reach new milestones that where previously unattainable for the agent**.\n\n**References:**\n\n[R1] Bratman, M. E. (1987). Intention, Plans, and Practical Reason. \n\n[R2] Radford, Alec, et al. \"Improving language understanding by generative pre-training.\" (2018)"}, {"review_id": "0LnoSQRVCt", "reviewer": "Reviewer_ADK5", "summary": "The authors propose a framework for agent memory and decision making in complex environments.", "questions": "How do you believe your work compares to other belief and memory architectures such as ExpeL, CLIN, SSO, DEPS, ADAPT?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- demonstrates benefit of agent architecture improvements on smaller llms\n- runs multiple ablations on architecture to help identify key components\n- architecture performs well empirically on low parameter, high complexity situations\n\nWeaknesses:\n- main results shown in table 3 are difficult to parse\n  - reporting success rate and task length side by side confuses which results are better than others\n  - I believe that I should be comparing each mindforge result to the open weight voyager results with the corresponding model, but that's not clear when looking at the table\n- some figures are nearly unreadable\n  - figure 1\n  - figure 3\n  - figure 5\n- lack of mentioned related work in LLM memory and belief architectures\n  - Expel: Llm agents are experiential learners\n  - CLIN: A Continually Learning Language Agent for Rapid Task Adaptation and Generalization\n  - Skill Set Optimization: Reinforcing Language Model Behavior via Transferable Skills\n  - Describe, explain, plan and select: interactive planning with llms enables open-world multi-task agents\n  - Adapt: As-needed decomposition and planning with language models", "was_revised": true, "final_justification": "Assuming that the authors improve the presentation of their results and elaborate more on how their method compares to related work, I am happy to recommend this paper be accepted.", "rebuttal": "Thank you for your review! We appreciate your recognition that (i) our agent-centric design enables smalelr LLMs to punch above their weight, (ii) the systematic ablations isolate how components such as the Theory-of-Mind graph and memory each contribute to the overall lift, and (iii) the architecture proves its value on challenging, high-complexity settings where parameter count alone typically stalls progress. Your acknowledgment underscores the practical importance of making lightweight models more capable through principled architectural choices.\n\nTo clarify, MindForge occupies a distinctive point in the design space for *multi-agent interactions* because it unifies three capabilities seldom found together.\n\n1. **Explicit Theory-of-Mind graph.** Each agent carries a causal belief network with four disjoint buckets—task, perception, interaction, partner—and can inspect or update another agent’s beliefs during dialogue. This makes perspective-taking and mis-belief repair first-class operations rather than emergent side effects.\n\n2. **Tri-partitite long-term memory.** MindForge separates experience into *episodic* traces, *semantic* summaries, and *procedural* code skills, each with tailored retrieval. A reward-gated consolidation step promotes useful skills and corrects false beliefs across episodes, so knowledge compounds over time instead of remaining in context.\n\n3. **Native multi-agent collaboration.** Because both pillars are baked in, agents can teach, coordinate, and critique one another, not merely self-reflect. Collaboration is therefore a built-in driver of learning, not an after-thought.\n\nTo begin, all the methods you mention (EXPEL, CLIN, SSO, DEPS, ADAPT) **focus on single-agent setups**. Nevertheless, each provide valuable pieces: self-reflection, skill libraries, or causal abstractions but omits at least one of these three elements. The following are the concrete differences.\n\n* **EXPEL**  \n  Targets non-parametric *experiential* learning, yet its representational substrate differs sharply from MindForge’s. The only enduring artefacts are (i) a **vector database of full task trajectories** and (ii) a free-text **“insight list”** distilled in offline batches. At run-time the system simply retrieves the top-k similar trajectories and concatenates them—together with any matching insights—into the current prompt. There is **no explicit belief graph (social or otherwise)**: the chain-of-thought produced by ReAct/Reflexion is discarded once the episode ends, so the agent cannot query or revise another agent’s mental state. In MindForge’s terms, ExpeL instantiates just a *two-store* design (episodic-like vectors plus semantic-like insights), omitting (a) a procedural skill library, (b) reward-gated consolidation across memory types, and (c) any Theory-of-Mind reasoning. MindForge, by contrast, maintains all three stores *and* exposes belief-revision operations as first-class actions during dialogue, making collaboration a driver of learning rather than a post-hoc prompt augmentation. \n\n* **CLIN**  \n  CLIN stores experience as *sentence-level causal abstractions* such as “Action → Effect (may/should).” After every trial a dedicated “memory-generator” LLM rewrites this list, pruning to the most salient rules; a single Controller LLM then retrieves the top-k rules (keyword \\+ embedding match) to pick the next goal, yielding fast *within-task* adaptation. Yet the system remains strictly single-agent: (i) it builds **no explicit belief graph**, (ii) the causal rules are not promoted into any long-lived semantic or procedural store, and (iii) there is **no recursion over other minds**—the agent cannot inspect or repair a partner’s beliefs. Cross-task generalisation is limited to a lightweight “meta-memory” text summary, so CLIN lacks the multi-store memory and Theory-of-Mind mechanisms that drive MindForge’s collaborative learning.\n\n* **SSO**  \n  Skill-Set Optimisation (SSO) is best understood as the procedural-memory slice of MindForge: it keeps a library of short, reward-validated code trajectories and retrieves the top-k matches by embedding similarity at inference time, precisely mirroring our procedural store. Yet SSO goes no further—there are no episodic or semantic layers, so any experience that is not rewritten into a surviving skill vanishes when the context window resets; its consolidation is purely numeric, retaining a skill only while its running return stays above ε, which trims dead weight efficiently but cannot migrate corrected beliefs or factual insights across tasks; and because the skills are treated as opaque black boxes, the framework never models *why* a collaborator failed or *which* beliefs need updating, leaving it unable to perform perspective-taking or mis-belief repair. MindForge therefore subsumes SSO’s economical skill caching while augmenting it with episodic and semantic memories and, crucially, an explicit belief graph that makes social teaching and belief revision first-class operations.\n\n* **DEPS**  \n  DEPS excels at **in-episode self-repair**—each cycle it re-describes the world, explains any failure, replans, and selects the next step—yet this textual *Descriptor \\+ Explainer* remains an ad-hoc string that vanishes with the context window. Because it never crystallises into a graph-structured belief model or persists beyond the episode, DEPS lacks both long-term memory (episodic, semantic, or procedural) and any representation of a partner’s mind. MindForge, by contrast, preserves experience in multi-store memory and manipulates an explicit Theory-of-Mind graph, enabling agents not only to debug themselves but also to query, revise, and teach one another’s beliefs across tasks.\n\n* **ADAPT**  \n  ADAPT offers a neat “plan-on-failure” loop but remains strictly ephemeral: its executor relies on a fixed prompt of hand-written atomic skills, and each decomposition tree evaporates once the episode ends. Because beliefs live only as transient chain-of-thought text, the agent neither stores episodic traces nor forms a structured Theory-of-Mind, and successful sub-plans are never promoted to new skills. MindForge supplies precisely what ADAPT leaves on the table—an explicit BigToM belief graph for self- and other-modeling, tri-partitioned long-term memory that survives across tasks, and reward-gated consolidation that turns ad-hoc sub-tasks into reusable procedures—thereby shifting recovery from a one-off patch to a cumulative, collaborative learning process.\n\n**Summary**\nAcross the spectrum, MindForge is the only system that simultaneously (i) reasons over explicit causal beliefs about self and others, (ii) separates memory into episodic, semantic, and procedural components with tailored retrieval, and (iii) treats collaboration as a primary driver of belief revision. The other frameworks focus on one or two of these dimensions but do not combine all three, which is why we believe MindForge occupies a distinctive place in the current literature.\n\nPlease let us know if there are further clarifications or information you require that could lead to a increase in the score!"}, {"review_id": "QU8IGeSlvQ", "reviewer": "Reviewer_rCMn", "summary": "The authors propose MindForge, a method that extends an LLM-powered embodied agent with a structured representation inspired by the theory of mind. This representation enables agents to reason about the beliefs, goals, and capabilities of others. The study focuses on inter-agent communication in two scenarios: when agents have differing capabilities and when they are equally capable. The proposed method is evaluated in the Minecraft environment, with Voyager serving as the main baseline.", "questions": "- During the collaborative setting, does the strong agent (e.g., a human expert or GPT-4) actively interact with the environment? Or is it treated more like an oracle, where the weaker agent queries it without requiring the strong agent to update its own beliefs about the environment?\n\n- The agent's beliefs are represented using a {question: answer} format. How are these questions generated? What mechanism determines which aspects of the environment are relevant or worth querying about?\n\n- Have the authors tested scenarios involving more than two agents with similar capability levels? If so, how does the communication and belief sharing scale in multi-agent settings?\n\n- In the case where two agents of similar ability interact, do their beliefs converge over time, or does each agent maintain its own distinct belief system? For example, if there are two valid strategies to reach the same goal, have the authors observed cases where each agent have a different belief about the strategy to adopt?\n\n- Have the authors explored settings where GPT-4 interacts with only one of the two agents in a collaborative task? If so, does the uninformed agent benefit indirectly from its partner’s interaction with GPT-4—e.g., through shared communication or updated beliefs?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "**Strengths**:\n\n* The paper is well-constructed and clearly written, making the proposed ideas and methodology easy to follow.\n* The ablation studies are detailed and well-structured, providing valuable insights into the contribution of each component.\n\n**Weaknesses**:\n\n* Figure 6 is unclear; it is not specified which experimental condition corresponds to each colored line, making interpretation difficult. Clarifying the legend or labeling would improve readability.", "was_revised": true, "final_justification": "The reviewer is very satisfied with the responses provided and the additional experiments; the paper is technically and experimentally sound.", "rebuttal": "Thank you for the review and overall positive assessment of our paper, rCMn!  We’re grateful that you found the manuscript effectively organized and easy to follow, clear exposition was and is a primary goal, so your comment affirms that effort. We also appreciate your endorsement of the ablation suite: by isolating the Theory-of-Mind graph and the memory, we aimed to give readers a transparent view of where the gains truly arise. Your acknowledgment that these studies are both detailed and well-structured reinforces the value of that analytical depth.\n\n1. > Figure 6 is unclear.\n\nWe will fix this for the camera-ready version. We will add a legend of the starting population success for each colored line, currently this is denoted on the curve itself in the figure. \n\n2.  > Does the strong agent actively interact with the environment?\n\nYes, **the strong agent is actively interacting with the environment while helping the weaker agent.**  The complexity and open-endedness of environments like Minecraft require agents to get continuous environment feedback as a way to revalidate or correct their pre-training induced beliefs. *Even strong models like GPT-4 struggle to complete tasks in a zero-shot manner, often requiring additional environment signals, as shown in Voyager [60].* Besides stronger grounding in the environment, the lack of environment interaction would hinder the ability of the other agents to take perspective, potentially making communication less effective, as shown in Appendix B Table 5.\n\nMoreover, requiring both agents to interact with the environment offers interesting perspectives on how they co-evolve (e.g do their belief structure converge over time?). While we consider this out-of-scope for this submission, we believe it is interesting to experiment how two or more agents influence each other’s curriculum, which becomes even more meaningful in a purely collaborative setting.\n\n**Expert agent as an oracle**\n\nIt is important to note that, while the weak agent is indeed asking for help from the strong agent, **the strong agent is not an oracle**. While rare in our experiments, **the weak agent can influence the beliefs of the strong agent given the way our MindForge agents are constructed.**\n\n3. > The agent beliefs are represented using a {question:answer} format. How are these questions generated? What mechanism determines which aspects of the environment are relevant or worth querying about?\n\nTo clarify, we employ the **{question:answer}** format for task-related beliefs, while for perception beliefs and partner beliefs we use a list like format **{perception_beliefs: list}** and **{partner_beliefs:list}**.\n\nThe questions for the task-related beliefs are generated similar to the base Voyager [60] setup by the autocurriculum module. Once the curriculum generator (also an LLM) decides which task to tackle next (e.g collect wood block) it generates a question for the beliefs about solving the task (e.g. \"How do you collect a wood block?\"). This becomes the question corresponding to the **{question:answer}** format. Note that the answer to the question might change over time as a result of communication with another agent, which results in a belief correction. \n\nA concrete example of task beliefs in the {question:answer} format can be seen in Appendix C.3.\n\nThe auto-curriculum generator is also responsible for parsing the environmental state based on the agent sensory input and the items in the current inventory for deciding which task to pursue next based on resources available in proximity. This is also similar to the base Voyager setup.\n\n4. > Have the authors tested scenarios involving more than two agents with similar capabilities level?\n\nWhile we have not specifically tested this setting for this manuscript, we consider this as a promising future research direction. **The way our framework is structured allows for easy scalability in terms of number of agents.** This would however add a layer of complexity on the communication side, and given the principle investigation needed for the two-agent case, we didn't have sufficient space to do the 3-and-more-agents case adequate justice. \n\n5. > In case where two agents of similar ability interact, do their beliefs converge over time, or does each agent maintain its own belief system?\n\nThroughout interaction in a lifelong *collaborative* setting (which refers to the symmetric ability scenario), we do observe that in most cases their beliefs (especially task related beliefs) match after a certain number of communication rounds. Note that this is not always beneficial; as we discuss in Sec. 5.5, below a certain individual performance threshold, agents can also reinforce false beliefs (see the discussion on Jury Theorem in 5.5). However, we observe that when performance improves for specific initial competence starting points (Fig. 6), the false beliefs related to the tasks do get corrected and converge. We will add concrete qualitative examples to the Appendix in the camera ready version. \n\n6. > Have the authors explored settings where GPT-4 interacts with only one of the two agents in a collaborative task?\n\n**We run this experiment as part of this rebuttal, showing that indeed the uninformed agent benefits indirectly from its partner interaction with GPT-4.** Thanks for this suggestion! We will add this to the paper.\n\nIn this experimental setup, we let a Mixtral-8x7B agent collaborate in an instructive setting with GPT-4 as shown in Figure 4 and then pair it with an uninitiated (base) Mixtral-8x7B. As the table below shows, the performance of the base Mixtral improves by 17% (37% $\\rightarrow$ 54%) if its partner previously communicated with GPT-4.\n\n| Mine dirt | Task Success Rate |\n| --- | --- |\n| **Mixtral-8x7B (base)** | 37% |\n| **Base Mixtral-8x7B after communication with GPT-4** | 67% |\n| **Base Mixtral-8x7B collaborating with parter Mixtral-8x7B that has had GPT-4 interaction** | 54% |\n\nThis is an advantage of our structured belief representation as well as memory components, allowing agent to recall correct beliefs from memory and share them further with weaker agents."}, {"review_id": "1nzGcBknqv", "reviewer": "Reviewer_rsUZ", "summary": "MindForge introduces a framework for cultural lifelong learning in embodied agents through explicit perspective taking. It uses (1) a structured theory of mind representation linking percepts, beliefs, desires, and actions; (2) natural inter-agent communication; and (3) a multi-component memory system. In Minecraft experiments, MindForge agents significantly outperform their Voyager counterparts. In fully collaborative settings, even agents that fail in isolation succeed together, with performance scaling over multiple communication rounds. Through these mechanisms, agents exhibit expert–novice knowledge transfer, collaborative problem solving, and adaptation to out-of-distribution tasks.", "questions": "1. The current design requires a weaker agent to completely fail a task before initiating dialogue. Can the communication protocol be made more flexible? This would improve efficiency and allow better generalization across different tasks.\n2. Would the framework generalize beyond Minecraft, such as being evaluated in another embodied environment? Since the paper claims it is a general framework for diverse settings, this would demonstrate whether the structured ToM representations, memory subsystems, and communication mechanisms transfer effectively.\n3. Can the compute–performance tradeoff be more precisely quantified? This would provide clearer insights and enhance the framework’s practical utility.", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. MindForge equips agents with structured Belief-Desire-Intention (BDI) models that link percepts, beliefs, desires, and actions, enabling them to form and update mental models of both themselves and their collaborators. This framework allows agents to reason recursively about partner states and generate more targeted, context-appropriate advice during collaboration.\n2. MindForge agents lead to significant gains with open-weight LLMs in experiments, both in the Voyager setting and fully collaborative settings.\n3. The integration of episodic, semantic, and procedural memory subsystems enables MindForge agents to keep and leverage past experiences. Ablation studies show that removing episodic memory leads to drops in task completion rates, and semantic memory facilitates both post-collaboration performance improvements and out-of-distribution generalization.\n\nWeaknesses:\n1. MindForge functions more as an engineering framework than a typical research paper. While it touches on many concepts (lifelong learning, collaborative learning, cultural learning), its core research contributions, particularly how it differentiates from other collaboration frameworks, are not clearly presented.\n2. MindForge agents can initiate dialogue only in a hardcoded fashion—specifically, the weaker agent must fail a task before requesting help, limiting flexibility in dynamic environments and potentially missing opportunities for proactive collaboration.\n3. MindForge introduces additional computational overhead due to multi-round communication. However, the paper does not quantify how much slower or more resource-intensive (e.g., in terms of API calls) MindForge is compared to non-collaborative baselines, nor does it explore different strategies or parameter settings to balance efficiency and performance.", "was_revised": true, "final_justification": "The authors addressed most of my questions and concerns. Considering that the overall quality is generally acceptable and the novelty of the work is present but limited, I raised the score and recommend a rating of 4: Borderline Accept.", "rebuttal": "Thank you for the review rsUz, we appreciate your comments and suggestions!\n\n1. > The current design requires a weaker agent to fail a task before initiating dialogue.\n\nTo clarify, **there is no limiting factor in the design of the MindForge framework itself with respect to the communication protocol** as long as communication is done in natural language. Note that we only adopt this approach (where failure is required before initiating dialogue and then performing communication and task actions in an interleaved manner) **for our experiments, to be able to compare the effect of communication on the base behavior for task completion**. \n\n**Flexbile communication protocol**\n\nTo showcase that our MindForge framework is not tied to a specific communication protocol, **we perform additional experiments on two MindForge tasks where we allow for a more flexible communication setting.** Specifically, we consider the following setup: in an instructive setting, **we allow at each turn for the weak agent to decide if it wants to start communication ( in the case it is unsure how to solve the task to begin with ) or try the action itself**. \n\nIf the agents decides to try the action and does not feel the need to communicate, in case of failure to complete the task, we follow the standard protocol considered in the manuscript. Below you can see the results of this experiment together with some useful statistics.\n\n*Mixtral-8x7B*\n\n| Mine dirt | Comm. Round 0 | Comm. Round 1 | Comm. Round 2 | Comm. Round 3 |\n| --- | --- | --- | --- | --- |\n| **MindForge w/ flexible communication** | 37% | 45% | 67% | 67% |\n| **MindForge** | 29% | 42% | 62% | 67% |\n\n| Mine dirt and wood | Comm. Round 0 | Comm. Round 1 | Comm. Round 2 | Comm. Round 3 |\n| --- | --- | --- | --- | --- |\n| **MindForge w/ flexible communication** | 75% | 79% | 79% | 83% |\n| **MindForge** | 75% | 79% | 79% | 83% |\n\n*Mistral-7B*\n\n| Mine dirt | Comm. Round 0 | Comm. Round 1 | Comm. Round 2 | Comm. Round 3 |\n| --- | --- | --- | --- | --- |\n| **MindForge w/ flexible communication** | 37% | 42% | 45% | 54% |\n| **MindForge** | 37% | 42% | 45% | 54% |\n\n| Mine dirt and wood | Comm. Round 0 | Comm. Round 1 | Comm. Round 2 | Comm. Round 3 |\n| --- | --- | --- | --- | --- |\n| **MindForge w/ flexible communication** | 41% | 45% | 50% | 50% |\n| **MindForge** | 41% | 45% | 50% | 50% |\n\n*Observations in a flexible communication scenario:*\n\n- Mixtral-8x7B correctly assess that it needs help to mine dirt without needing to fail first, which results in an increase in task success\n- Mistral-7B is extremely confident in its ability to solve the task without help and *never* asks for help before failure even if this option is given.\n- When dealing with a task resembling something they solved in the past, both models correctly identify they do not require additional help as they should already possess the required knowledge in semantic and procedural memory.\n\n2. > Since the paper claims it is a general framework for diverse settings...mechanisms transfer effectively. Would the framework generalize beyond Minecraft?\n\nTo begin with a clarification, in terms of measuring generalization, **different Minecraft biomes already serve as diverse environments for testing, making the tech-tree experiments a significant reflection of continual learning capabilities, similar to Voyager’s experiments (Wang et al., 2023a).** \n\nThat said, our proposed innovations (perspective taking, natural communication) and the MindForge architecture itself are game agnostic. Generalization beyond Minecraft would only change the base prompt for the individual modules to adapt to the specific environment (or the real world). However, incorporating different games for experimentation are non-trivial on academic budgets; the experiments in Minecraft alone in the current paper cost around 600 GPU hours.\n\n3. > its core research contributions, particularly how it differentiates from other collaboration frameworks, are not clearly presented.\n\nTo clarify, the manuscript tackles the following research questions:\n\n- **(i)** can we bridge the capabilities between closed-source and open-weight models in embodied settings through a natural language interface that is also compatible with humans\n\n- **(ii)** similar to humans, can we use perspective-taking and theory of mind to enhance the communication between multiple agents\n\n- **(iii)** what type of memory subsystems are beneficial for embodied agents in a lifelong learning setting\n\nTo the best of our knowledge, we are the first to enable agents to take perspective-taking using a structured representation. Moreover, we consider a lifelong cultural learning setting where agents communicate and evolve across a curriculum of tasks inside Minecraft.\n\n*Contributions with respect to the Research Questions*\n\n - **(i)** **we show in Figure 1 b) how our MindForge agents an in instructive setting achieve new milestones compared to the Voyager baseline in a lifelong setting inside Minecraft, bringing the performance closer to closed source models like GPT-4.** Besides the immediate performance improvements, our **MindForge framework offers a potential solution to tackle false beliefs through communication in multi-agent settings.**\n\n- **(ii)** We show through an ablation in Table 5 Appendix B that using **perspective taking and the BigTOM representation results in more effective communication between agents.** The text snippet in Appendix B shows an example how perspective taking incentives agents to think from the perspective of the other agents, tailoring their advice to their needs.\n\n- **(iii)** we add 2 more memory components compared to Voyager: episodic and semantic memory. **We consider semantic memory an important component in MindForge as it represents a store for correct beliefs about Minecraft tasks, gathered through multi-agent interactions.** This component functions as a way to override the inherent false beliefs that stem from LLM pretraining. Moreover, **we show how episodic memory enables embodied agents to remember their mistakes and avoid making them again. **\n\n> MindForge functions more as an engineering framework than a typical research paper.\n\nThe majority of the scientific work in this foundational model powered embodied agents can be considered engineering frameworks [1] [2].\n\nMoreover, Reviewer ADK5 compares our belief and memory systems to a number of single-agent papers (ExpeL, CLIN, SSO, DEPS, ADAPT), highlighting that most of the papers in this space are engineered around foundational models. Please refer to our response for Reviewer ADK5 for a more in-depth breakdown.\n\n\nHowever, the goal of MindForge is not to propose an engineering framework to solve Minecraft, but rather to **provide a solution towards human-AI and multi-agent collaboration in cultural learning settings purely through natural language and in-context learning.** Minecraft simply provides us with the environmental substrate required to assess the efficacy while the core of MindForge remains generally applicable\n\n4. > Can the compute-performance tradeoff be more precisely quantified?\n\nFocusing on the compute-performance trade-off would erroneously consider MindForge as an engineering goal (solving the task efficiently) rather than the manuscript's stated scientific goal: to investigate the process of in-situ learning and skill acquisition in an agent that is otherwise incapable, using social collaboration as the mechanism.\n\nNonetheless, the effect of compute-performance tradeoff is best seen in the context of the Minecraft tech tree experiments as shown in Fig. 1 b) and Table 3.\n\n**MindForge agents manage to achieve more milestones than Voyager open-weight alternatives in the same number of actions**. Moreover it reaches certain milestones faster ( as counted by the number of actions) than the Voyager counterpart.\n\nThe efficiency of MindForge agents can be quantified in two separate ways: \n\n- **(1)** number of effective tokens generated by the model across communication, action, belief creation and perspective taking\n\n- **(2)** number of actions in the Minecraft environment taken by the agent.\n\nWe consider the latter as a more appropriate way of measuring the compute-performance trade off since it is agnostic to the underlying model, as opposed to counting the effective number of tokens which is heavily dependent on the prompting, type of model ( base, instruct, reasoning model ) and model provider training recipe.\n\nPlease refer to the last point mentioned in the rebuttal for Reviewer qoax.\n\n**References:**\n\n[1] Zhang, Hongxin, et al. \"COMBO: compositional world models for embodied multi-agent cooperation.\" arXiv preprint arXiv:2404.10775 (2024).\n[2] Sumers, Theodore, et al. \"Cognitive architectures for language agents.\" Transactions on Machine Learning Research (2023)."}]

Author rebuttal:

Meta-review: This paper proposes MindForge, a method that promotes lifelong cultural learning among LLM-based agents through Theory of Mind, specifically, perspective-taking. Compared to prior LLM-based agent methods, it mainly introduces a structured BDI representation. Evaluation on Minecraft shows that MindForge agents powered by open-weight LLMs significantly outperform a strong baseline, Voyager.

During the author-reviewer discussion, the most discussed issues were the fairness of comparison with Voyager and the novelty (e.g., in comparison to COELA). Based on the discussion, the converged opinion is that the main novelty lies in the BDI component. 

Given the rebuttal and the discussion, AC thinks that the contribution of this paper is significant enough for acceptance. Having said that, AC would urge the authors to revise the claims about the three innovations. Based on the converged conclusion after the discussions, the more accurate claim appears to be introducing BDI representations to enable lifelong cultural learning.
Decision: Accept (poster)