ADAM: An Embodied Causal Agent in Open-World Environments
Authors: Shu Yu
Chaochao Lu
Venue: ICLR 2025 Poster (2025)
Keywords: embodied agent
causality
large language model
interpretability
vision language navigation
cross-modal application
cross-modal information extraction
multimodality
Primary area: applications to robotics, autonomy, planning

Abstract: In open-world environments like Minecraft, existing agents face challenges in continuously learning structured knowledge, particularly causality. These challenges stem from the opacity inherent in black-box models and an excessive reliance on prior knowledge during training, which impair their interpretability and generalization capability. To this end, we introduce ADAM, An emboDied causal Agent in Minecraft, which can autonomously navigate the open world, perceive multimodal context, learn causal world knowledge, and tackle complex tasks through lifelong learning. ADAM is empowered by four key components: 1) an interaction module, enabling the agent to execute actions while recording the interaction processes; 2) a causal model module, tasked with constructing an ever-growing causal graph from scratch, which enhances interpretability and reduces reliance on prior knowledge; 3) a controller module, comprising a planner, an actor, and a memory pool, using the learned causal graph to accomplish tasks; 4) a perception module, powered by multimodal large language models, enabling ADAM to perceive like a human player. Extensive experiments show that ADAM constructs a nearly perfect causal graph from scratch, enabling efficient task decomposition and execution with strong interpretability. Notably, in the modified Minecraft game where no prior knowledge is available, ADAM excels with remarkable robustness and generalization capability. ADAM pioneers a novel paradigm that integrates causal methods and embodied agents synergistically. Our project page is at https://opencausalab.github.io/ADAM.

Published as a conference paper at ICLR 2025
ADAM: AN EMBODIED CAUSAL AGENT IN OPEN- WORLD ENVIRONMENTS
Shu Yu Chaochao Lu
Shanghai Artificial Intelligence Laboratory SSanggai Innovation Institute (yushu, luchaochao] @ jlab. org. cn
Fuudan University
ABSTRACT
In open-world environments like Minecraft, existing agents face challenges in continuously learning structured knowledge, particularly causality. These chal- lenges stem from the opacity inherent in black-box models and an excessive reliance on prior knowledge during training, which impair their interpretability and generalization capability. To this end, we introduce ADAM, An emboDied causal Agent in Minecraft, which autonomously navigate the open world, perceive multimodal context, learn causal world knowledge, and tackle complex tasks through lifelong learning. ADAM is empowered by four key components: 1) an interaction module, enabling the agent to execute actions while recording causal model module, tasked with constructing an ever-growing causal graph from scratch, which enhances interpretability and reduces reliance on prior knowledge; 3) a controller module, comprising plan- ner, an actor, and memory pool, using the learned causal graph to accomplish tasks; 4) perception module, powered by multimodal large language models, enabling ADAM to perceive like human player. Extensive experiments show that ADAM constructs a nearly perfect causal graph from scratch, enabling ef- ficient task decomposition and execution with strong interpretability. Notably, in the modified Minecraft game where no prior knowledge available, ADAM excels with remarkable robustness and generalization capability. ADAM pioneers novel paradigm that integrates causal methods and embodied agents synergistically. Our project page is at htttttttpp/
the interaction processes;
atthttpss://ppeeauusee http://////pcccccuuaeaaaa lab. git.titttth... t.h...tub.......... Ao/ADMM
INTRODUCTION
Embodied agents exploring open- world environments mark a critical frontier in artificial intelligence (AI) research (Cassell 2000 Xia et al. [0018) Savva al.) 2019). The ultimate goal is to build generally capable agents (GCAs) (Team et al. 2021) that can autonomously perform broad range of tasks through perception, learning, and interaction (Mnih al. (2055) Xi et al.) [023] Park al.) 2023). Minecraft (Nebel et al., 2016) is globally renowned 3D video game, where players need to master complex crafting recipes (e.g. planks + sticks / wood wood pickaxe 7) and gather resources (e.g., mining cobbl lettoe with wood_pi ckade ckaxe >), progressively unlocking new items in the technology tree. The substantial freedom and precise simulation of physical laws in Minecraft render it an exceptional platform for researching GCAs.
In Minecraft, two primary approaches for developing GCAs have been extensively explored: rein- forcement learning (RL)-based (Lin et al. 2021 Baker et al. 2022 Fan et al. 2022, Mao et al.
2022
Hafner et al. (202)) and large language model (LLM)-based (Wang al., 2023a Zhu al,
2023 Qin et al., 2023a| Nottingham et al. 2023) Wang et al. d)) Specifically. RL agents learn through interactions and updating their black-box model weights, which poses challenges for inter- pretability, efficiency, and generalization. On the other hand, LLM-based agents possess and rely rich prior knowledge of both virtual games and real world (Ouyang et al.) 2022 Wei et al. 2022a
Achiam al.) 2023). Their reliance omniscient data (e.g., GPS coordinates, voxel blocks, biome,
etc., which are not explicitly observable by the player) (Wang et al. 2023a| Zhu et al., 2023 Wang
et al. 2023d) presents challenges for generalization and human gameplay observation alignment.
CCorreponding author.

---

Published as a conference paper at ICLR 2025
Technology
Diamond Age
Iron Age
Stone Ago
Wood Ago
75
100
125
150
200
Steps
-4-
ADAH
VONAGER
Refesdon WI
VOWAGER Cuited
-I-
React
Acf cllon
(a)
(b)
(c)
Figure 1: (a) The technology tree for acquiring di amonds @ in the Minecraft game. ADAM precisely discover item dependencies from scratch. (b) Modified Minecraft technology tree, where the prior knowledge from the Internet Wiki does not align with the actual game dynamics. Red arrows denote removed dependencies, while blue arrows denote added dependencies. (c) In the game setting shown in (b), ADAM maintains the ability to learn the correct causal graph and successfully obtains di amonds @ whereas other methods can only acquire r raw-iron Go within the step limit, and A ADAM achieves a 4.6x speedup obtaining raw. ron G compared to the SOTA.
To address these issues, we propose ADAM, An emboDied causal Agent in Minecraft, which can autonomously navigate the open world, perceive multimodal context, learn causal world knowledge, and tackle complex tasks through lifelong learning. Specifically, as shown in Fig. 2 ADAM is
composed of four key modules: (1) Interaction module, which enables the agent to execute actions from the action space and processes the agent's ' observable information into formatted records. (2) Causal model module, which includes two causal discovery (CD) methods. LLM-based CD utilizes interaction records to make causal assumptions. Intervention-base -basdd CD refines these assumptions to derive causal subgraph. Multiple causal subgraphs are integrated into comprehensive causal graph (i.e., technology tree). (3) Controller module, which includes a planner, an actor, and memory pool. The planner can utilize the causal graph to perform task decomposition. The actor uses the decomposed subtasks for action choosing. The memory pool ensures the long-term context dependence. (4) Perception module, which driven by multimodal LLMs (MLLMs), enabling ADAM perceive its surroundings without relying omniscient data, thereby achieving human- like gameplay observation.
Extensive experiments demonstrate that ADAM achieves 2.2x speedup compared to the SOTA in the task of obtaining d amonds @ In scenarios where crafting recipes are modified (Fig. [ only ADAM maintains the ability to obtain di amonds c while other methods can only acquire raw @ within the step limit, and ADAM achieves a 4.6x speedup in obtaining raw-iron T compared the SOTA (Fig. [e). ADAM demonstrates strong interpretability through constructing nearly perfect technology tree (Fig. Th) from scratch, whereas other methods exhibit at least 30% errors or omissions. Meanwhile, in tasks requiring environmental perception, ADAM closely aligns with human gameplay observation without relying on omniscient metadata, while maintaining comparable performance methods that utilize such data.
Overall, our contributions are as follows:
(1) We introduce ADAM, an embodied causal agent that can autonomously navigate the open world, perceive multimodal context, learn causal world knowledge, and tackle complex tasks through lifelong learning.
(2) We address the limitations of existing embodied agents. Our ADAM demonstrates strong robustness without relying on prior knowledge or omniscient metadata used by other L.LM- based agents, while aligns human gameplay observation during exploration.
(3) We pioneer the integration of causal methods into open-world embodied agents, al- lowing the agent to organize the learned knowledge into a rigorous causal graph, thereby demonstrating excellent interpretability.
2

---

Published as a conference paper at ICLR 2025
Task rxineDi
Interaction
Causal Model
ninedoldire craf-Ironlloe
New lask
"craftPlenke"
(atartItun,
Recording
addedivem)
LLM-based
Causal Graph
craftSticke
Assumptions
Sampling
Itam Space
rerafuPlenker
Bynthesize
GradtStickee
Causal Relations
intervention-aased CD
Bampling
Perception
Controller
Planner
Adiion
Actor
Artionn
Memory
Inventory
Figure 2: Four key modules of ADAM. The interaction module executes actions in the environment according to the task and records the processes. The causal model module identifies the causal relationship between items and actions to construct ever-growing causal graph. The controller module implements task execution based on the learned causal graph. The perception module aligns the agent's gameplay observation more closely with human.
(4) We improve the CD performance by employing embodied agent-driven interventions, which enhances the accuracy and efficiency of CD compared to existing methods without interventions.
PRELIMINARIES
Causal graphical models (CGMs). A CGM represents the structure of causality within a system (Peters et 2017) by detailing the direct causal relationships among of variables XI,..., X,- characterized by a distribution over these variables and is associated with a directed acyclic graph (DAG), known causal graph. In this graph, each node corresponds to variable, and each directed edge signifies a direct causal relation from X; to Xj.
Causal discovery from interventions. Causal Discovery (CD) Spirtes et al. 2001 Pearl 2009
Peters al., 2017 Glymour al. 2019) fundamental process to infer causal relationships from data. The relationships are typically represented in the form of a causal graph. While observational data reveals correlations, interventions allow us to analyze causal dependencies between variables. Specifically, interventions alter the distribution of variables (e.g.. the initial system state) during experimental sampling, which serves gold standard for CD (Eberhardt Scheines (007). By observing their effects, we can identify causal relationships rather than merely correlations.
METHOD
Then, we give an overview of our ADAM framework (Sec. 3.2) Next, we detail the four modules of ADAM: interaction module (Sec. 3.3 causal model module (Sec. 3.4). controller module (Sec. 3.5). and
In this section, we begin with the basic notations and definitions in our work (Sec.
3.1g
perception module (Sec. 3.6b
3.1 NOTATIONS AND DEFINITIONS
We first introduce several key notations and definitions in our work. Sets denoted by uppercase letters, and their elements by lowercase letters.
Inventory: The set I of items possessed by the agent any step Initialization: At the initial- ization Minecraft instances, the initial inventory In can be specified by the agent. Action Space: The set A of actions whose names (e.g., gather (WoodLogg replaced by letters and invisible.

---

Published as a conference paper at ICLR 2025
Interaction Module
Initial Action Space [gatherNoodLog]
Sampling
Recording
Action Space
IgatberVoodLog. 22,
R= 10,0,1
gatherWoodLog craftPlank
Initial Observed Items
startIten:
Observed Items
Take Action gatherNoodLog
End Inventory
consunedItom addodItem log
]
Initial Inventory
Sampling Record N
gatherteediiig (R..... Rd}
Environment Step
Fall
CD
Succeed
The interaction module has two core functionalities: sampling and recording. Sam- pling involves executing actions in the environment, and recording involves processing and docu- menting the observable information. For instance, the initial action space is [gatheriee hhroWWoo whose name is not exposed ADAM and denoted as {a) here (Note that, the original notation [gatherWoodLog]] is retained in the figure for the illustrative purpose.). The initial observed item space Ø. After executing for one step, logs
Figure 3:
obtained. sampling can be represented as (a, Ø, H). where Ø is the initial inventory and } is the inventory after this step. The result is recorded as
(0, 2, {+)). where the first Ø is the initial inventory and the second indicates that no items are consumed, and
represents the items that are obtained. After sampling N times, data Da = [R,,..., Rx) } is provided to the causal model module for CD. If the causal rela- tion failed to be identified, resampling occurs; if successful, new actions like craftP 1 lanks are enabled by the acquisition of and the observed item space updated to
ADAM must independently discover the effects of these actions. Movement Space: The set M of basic movements whose names (e.g., moveForward moveBackwarde are visible ADAM. Step: The agent takes an action a in the environment. A step ends either when action completes or when execution times out. Observed Item Space: The set of items that ADAM has encoun- tered. Initially, S is empty. Environmental Factors: The set & of environment conditions (e.g. biome, surrounding block types). Task: Denoted by the tuple (Tgoat: E), a task accomplished at step t if goaa C I and the factors & are present within certain distance around the agent.
3.2 OVERVIEW
ADAM comprises four modules as depicted in Fig. 2 which can be extended to other open-world environments, as discussed in Appendix F. Given task (Ageat, E), the interaction module enables the agent to execute each action and records data D.. This data is then utilized by the causal model module, which employs LLM-based CD to make causal assumptions and intervention-baie CD to refine these assumptions into causal subgraphs. These subgraphs are integrated into causal graph (i.e., technology tree). Once the causal graph G contains all required items Tgost in the task, the controller module executes the task from an empty inventory, aided by visual descriptions from the perception module. Newly discovered items enable the execution of new unknown actions and the CD on these actions. This iterative process ensures the lifelong learning through continuous engagement and adaptation.
3.3 INTERACTION MODULE
The interaction module (Fig. 3 enables the agent execute actions for sampling and records ob- servable information. Initially, the action space A contains one element gatherWoodLog which the most basic action in Minecraft and a common setup for Minecraft agents (Wang et al., (2023)). As the agent acquires certain new items, new actions are enabled in A. For example, the action of mining becomes available only after the agent obtains mining tool (e.g., wooden pi ckaxe 2). By continuously collecting data on each action and coordinating with other modules, the nodes on the technology tree are progressively discovered by the agent.
Sampling. This involves initiating Minecraft instance with all observed items S as the initial inventory Ins and then executing action a to observe the agent's inventory I after this step. The quantity of each item in S and 7 is recorded. This sampling, represented (a, S, I1), is performed
4

---

Published as a conference paper at ICLR 2025
Prompt
Causal Assumption
Role Playing
Depicted as
You Minecraft game analysis assistant. You good inferring causel relstionships between ections and itemss.
Problem Setting
Our task inforthe caural relationship.
Inforence Result
single,
Cause: action Effect:
Letter Mapping
items involved in the game include:
al: log'
bb: planks
'el: traae
Action a serves as an trivial cause and is added to the cause
Few-shot Prompting
will give you some examples
Consumed items;; Added items:
Matching 'at: 'log'
Your inference: Because there item
'b:: 'llanks'
'ct: "craftiinggtabe
consumipiion. Redstone a higher-level mineral,,
my answeris:
Cause: Effect::
Couse; Effect-lk; m] ... x Number of shot
Causerfa, c]Efffec((bb
Data D₃
Initial items.
Consumed Xqerm::
Added items: Yg
LLM Raw Answer
Ininiiiall items:
Because logs comsumed and planks get this is "Crafting" task. In addition to raw materials, the "Crafting" task also requires crafting_table (tc).
Consumed ttems: Xg Added items: Y,
LLM inference
So my answer Causei(a, (iffectt:(b]
Your inference:
Figure 4: LLM-based CD performs causal reasoning under the guidance of the prompt. Role Playing assigns analysis assistant role to the LLM. Problem Setting provides the reasoning task. Letter Mapping maps the item names to letters for the accurate output. Few-shot Prompting provides examples for chain-of-thought (Wei al.) 2022b) reasoning. Data D, is presented in the same form as the few-shot examples. The output of I.M serves as the causal assumption.
N times, focusing on one specific action at a time. Due to the same configuration of these N samplings, parallelization is available and accelerates the exploration.
Recording. This involves processing the results of samplings and documenting them. For the sam- pling (a, I)), consumed items are denoted by X, which includes items with decreased quantities and items that are present in but absent 7, Conversely, obtained items are denoted by Y. which includes items with increased quantities and items that newly appear in 71- A single record is denoted as R XY) Such records on action a collectively form data D, = {RI.... Rx).
3.4 CAUSAL MODEL MODULE
The causal model module uses data Da infer causal relationships and constructs causal subgraphs for each action a. These subgraphs are then integrated into comprehensive causal graph.
A causal relationship is stable and repeatable dependence in any step. where the agent takes an action a and the acquisition of items E (effect items) relies on the items C' (cause items) the agent possessed before the step. Since the technology where no items are obtained do not contribute to the technology tree.
focuses
item acquisition, causal relations
In the causal model module, LLM-based CD makes assumptions on the causal relationships, which effectively reduces the number of item nodes that need to be confirmed in the causal subgraph and achieves acceleration. Then, intervention-base CD refines these assumptions and accurately constructs causal subgraphs. We also detail our optimization techniques employed in this module.
LLM-based CD. The input to LLMe- CD (Fig. 4 is the data D. containing N records, and the output is causal assumption which consists of cause items and effect items E. The prompt is designed with five components: (1) Role Playing, which assigns specific role to the L.M. In this
5

---

Published as a conference paper at ICLR 2025
context, the LLM serves causal analysis assistant, dedicated to extracting causal relationships from data. (2) Problem Setting, which provides the reasoning task and the fundamental concepts of the environment. We avoid introducing specific environmental knowledge for generalization. (3) Letter Mapping, which involves mapping item names to letters, a simplification that facilitates the formatted output. (4) Few-shot Examples, which involves providing the L.LM with several reasoning examples in chain-of-thouge (Wei et al. (202b)) style, including example questions, the reasoning processes, and the expected answering format. The examples are independent of the technology tree, thus preventing the introduction of prior knowledge, (5) Data Da. which consists of N records from the interaction module, and formatted consistently with the few-shot examples.
Intervention-based CD. Intervention method to experimentally verify causal relationships among variables. Intervention-batt CD (Fig. (5) can refine the causal assumptions and construct a
Intervention-bssed CD
Assumption
Updato
highly accurate causal graph.
removed
daj
=D)
da
=0)
causal subgraph
Before interventions, it has to be con- firmed that C' sufficient condition for
Intervention Sampling
Conntruct caural
E. If C has already lacked the necessary items to achieve E, then the vital edges are missing and the graph not be corrected by excluding redundant edges. Specifi- cally, by performing sampling (a, C, IT) as described in Sec. [33] E is consis- tently absent from /, the assumption is deemed incorrect, leading the LLM-based CD re-infer the assumption. If these inferences continue to fail, the interaction module will resample data Da.
Can not obtoi
Update
confirmed
Figure 5: Intervention-base CD verifies causal as- sumptions. An example of causal assumption that, under the action log and crafting.tabl
planks This assumption is denoted (a. Intervention- based CD will verify each item in }. When removing from the inventory and execut- ing action o, cannot be obtained, proving that is dependency
contribute to the acquisition
intervention-basee CD performs sam- plings (a, C\ (c), I1) for each item e C. For each item e E, if there is always & I, within a maximum number sam- plings, then is the cause of and the edge с + e is retained. If at least one sample includes e I1, then is not indis- pensable for e, and the edge e + e is re- moved. Intervention-based CD yields ac- curate causal subgraphs, which are subse- quently integrated into a complete causal graph.
is retained in the causal graph (represented in green). When remov- ing and executing action a. still can be obtained, which shows that is not a dependency of and this edge
and this edge
is removed from the causal graph (rep- resented in red). Intervention-bast based CD results in corrected causal subgraph. Multiple subgraphs can be combined into the technology tree in Minecraft. The actions are not shown in the technology tree for the sake of simplicity.
Optimization techniques. (1) Temporal Modeling (TM): Temporal information can help deter- mine causal direction. Events occurring later cannot be the cause of events occurring earlier. This eliminates some edges in the causal graph. (2) Subgruph Decomposition (SD): By focusing on the causal effects of individual actions, the causal model module processes manageable number of items (only those in each cause-effect pair) at time. This significantly reduces the complexity of the CD process.
3.5 CONTROLLER MODULE
During the execution of task (4gont, E), ADAM begins with an empty inventory (ie.. to 2) to ensure fair comparison with other methods. Once all items in /goal have been obtained in the causal graph, the controller module (Fig. () is responsible for executing the task. there are newly discovered items, they will be added to the observed item space S and pend for cycle of CD, thus achieving lifelong learning. The controller module comprises three components as described below.

---

Published as a conference paper at ICLR 2025
Controller Module
Planner
Actor
Graph
Formatted prompt
Image description scene featuring green forest with trees, mushraoms, a grassy area...
Action Space: [A, B ...]
Operations
action, namins B
Movement Spacer
material names [R, b.:] material namer_dict
Your Task: Smelting
[Forward, Backward ...]
irongainear grass
1 a: log In plankss ...
craft planks (2) craft sticks
Discovered causal graph:
Decompose
LLM
[[A], a]
[[0, b]
(n) smeltiron
Choose Action
Memory
Inventory
Action trajectories
[, B,
Newly discovered item
every
(0. llogl, llogl
Interaction Module
Figure 6: The controller module comprises three components. The Planner utilizes the reasoning capabilities of LLMs decompose the task. receives the current inventory and the learned causal graph as input. The Actor leverages LLMs to choose an action in the the action space A or movement m in the movement space M, and executes it in the environment. The Memory records the step information, including action trajectories and item changes every step.
Planner. The Planner utilizes LLMs to decompose the task with current inventory I, at step and the learned causal graph. Relying solely on the causal graph suboptimal actions may fail or have side effects. LLMs can fully utilize the inventory information and provide detailed thought process of the decomposition process, which is passed the Actor for action selection.
The Actor leverages LLMs choose action A movement E . execute. It receives the task decomposition from the Planner, the description of game image at this step, and the records from the Memory. In the task (Igeat E), the Actor prioritizes obtaining the items Igaal because the agent's surroundings & at each step f. change during this process, which may affect the acquisition of E.
Actor.
The Memory records observable information during task execution, including action trajectories and inventory changes at each step. It plays a crucial role in tracking long-term depen- dencies and facilitates robust task exccution.
Memory.
3.6 PERCEPTION MODULE
The perception module utilizes MILMs for environmental observation, enabling ADAM to perceive the world without relying on metadata such as the names of surrounding blocks, GPS coordinates, or biome names, which are typically invisible to human players. This module captures first-person screenshots between steps, which are processed by MLLMs to generate text descriptions. This text description is subsequently passed to the Actor in the controller module for action selection.
EXPERIMENTS
4.1 EXPERIMENTAL SETUP
In study, employ Mineflayer (PrismarineJSS 2023a). JavaScript-based framework provid- ing control APIs for the commercial Minecraft (version 1.19) The encapsulation of Mineflayer the implementation in VOYAGER WWng al. 2023a For visual processing, utilize prismarine-viewer (PrismarineJS 2023b), an API for rendering game scenes from the agent's per- spective. ADAM and our baselines all use GPT-4-turbo 4-pt-4-0125-prres -1--ppee)))) (Achiam et al. 2023) for LLM inference, with the temperature set to 0.3 based on our experiments in Appendix A For
visual description, we utilize LLaVA-v1.5-13BB [Liu 2024 our perception module.
https//wwww.ineerat.nttt

---

Published as a conference paper at ICLR 2025
Model
SHT)
Model
SHID
ADAM
2 = 2
CDHRL
10 10
ADAM w/ TM,SD
19 - 6
CDHRL w/ SD
6 2
Reflexion
24 $
9
React w/ TM,SD
5 + 2
AutoGPT
24 + 6
Reflexion w/ TM,SD
4
Empty Graph
32
AutoGPT w/ TM,SD
4
Table Structural Hamming Distance (SHD) between the learned causal graph and the target graph. For non-embodied agents without built-in interventions, even with our TM and SD (Section 3.4). the causal graph learned by these agents remains suboptimal. The CD used by CDHRL is based on SDI (Ke et al., 2019), which incorporates temporal modeling (TM) into its implementation, and still exhibits over 30% errors or omission, while ADAM can identify a nearly perfect causal graph.
4.2 BASELINES
In the absence of directly comparable works, select representative methods as baselines and fo- cus on their comparable aspects, including: (1) ReAct (Yao et al. 2023), which explicitly expresses the thought processes through
prompting (Wei et al. 2022b). (2) Reflexion (Shinn
et al. 2024), derived from ReAct, which can reflect on its exploration history. (3) AutoGPT (Sig- nificant Gravitas), which can autonomously decompose tasks and execute subtasks in ReAct-style. Baselines 1-3 can only perform text-based tasks and lack embodied components to interact with the environment. Hereafter, they are referred as non-embodied agents, and we have adapted them with our interaction module for embodied exploration. (4) VOYAGER (Wang el al. 2023a) is an LLM--ased embodied lifelong learning agent in Minecraft, featuring an automatic curriculum aim- ing to "discover as many diverse things as possible". We also add our benchmarking tasks (e.g., ob- taining c amonds ) to the curriculum for oriented explorations, denoted as VOYAGER-Guider (5) CDHRL (Peng et al. 2022) introduces an RI. agent that constructs hierarchical structures based on causal relationships. Given that RL agents have disparate action spaces and magnitudes of difference in episode length compared to LLM-based methods (10 ~ 10 in Dreamer (Hafner et al., 2023) and DEPS (Wang et al., 2023d) versus 10a ~ 102 in VOYAGER and our ADAM), focus our comparison the CD component of CDHRL. Detailed discussion of other Minecraft agents that are not directly comparable can be found in Appendix
4.3 MAIN RESULTS
Interpretability. We evaluate the interpretability of agents by assessing their ability construct causal graph. Structure Hamming Distance (SHD) (Zheng et al. 2024) can quantify the discrepancy between the learned causal graph and the target graph as presented in Tab. [ Despite applying
our TM and SD optimization (Section 3.4), non-embodied agents without built-in interventions fail to achieve the optimal accuracy. For CDHRL, we directly provide ADAM'S sampling data for its CD. CDHRL performs CD with all nodes in the causal graph, which hampers its performance as CDHRL. shows improved performance when integrated with our SD optimization. Nevertheless, these competitive methods exhibit at least 30% errors or omissions, whilst ADAM is capable of learning nearly perfect causal graph. VOYAGER does not organize knowledge in a causal graph.
Efficiency. Efficiency is evaluated in the original Minecraft as shown in Tab. 2 ADAM achieves a 2.2x speedup compared to the SOTA in the task of obtaining di amonds @ The design of parallel sampling (Section 3.3) significantly boosts ADAM's exploration efficiency. Due to the absence of intervention-bast CD, non-embodied agents unable to refine their causal assumptions, there- fore limiting their exploration speed and confining them to the lower levels the technology tree. Additionally, ADAM achieves higher success rate across most tasks compared to other methods.
Robustness. Robustness is evaluated in modified Minecraft where crafting recipes are altered. The result is shown in Fig. Te. In this scenario, there exists a misalignment between the LLM's prior knowledge and the actual game dynamics. ADAM successfully obtains diandss @ in the modified Minecraft. The baselines lack CD approach to learn and organize new causal knowledge beyond prior, and struggle with complex dependencies. The most advanced item that baseline agents
8

---

Published as a conference paper at ICLR 2025
Framework
Wooden Tool
Stone Tool
Iron Tool
Diamond
React w/ TM w/ SD
51 19(2/3)
96(1/3)
N/A (0/3)
N/A (0/3)
Reflexion w/ TM w/ SD
27(33)
122 + 56(2/3)
N/A (0/3)
N/A (0/3)
AutoGPT w/ TM w/ SD
49 20/2/(
103 45(2/3)
N/A (0/3)
NIA (0/3)
VOYAGER
2+333
10+3(3/3)
113 17
41(23)
VOYAGER Guided
7+3(333)
21
9(3/3)
75 20(2/3)
ADAM
23 + 5(3/3)
88+//333)
53 + 16(3/3)
68 + 21(333)
ADAM Parallel
12 1 2(3/3)
18 3(3/3)
29 1 5(3/3)
34 1 473/3)
Exploration steps in different tasks. Fewer steps indicates higher efficiency. Each method has three trials for maximum length of 200 steps. The success rate is depicted in the parentheses. ADAM achieves a 2.2x speed compared to the SOTA in the task of obtaining diamonds, with higher success rate.
Table 2:
VOYAGER 16 1 8(2/3)
VOYAGER w/o Meta
ADAM 21 + 16(2/3)
ADAM w/o MLLM
Find a river Gather log near river Smelting iron near grass
N/A (0/3)
N/A (0/3)
36(1/3)
N/A (0/3)
10 + (2/3)
N/A (0/3)
N/A (0/3)
N/A (0/3)
95
N/A (0/3)
Table 3: Performance of ADAM and VOYAGER in tasks requiring environmental factors E. Each method has three trials for a maximum length of 100 steps. The success rate is depicted in the paren- theses. VOYAGER's performance significantly declines when metadata is not accessible, whereas ADAM do not rely on metadata. MLLM contributes ADAM'S 's performance in this type of tasks.
manage to acquire is raw_i ron in obtaining raw-iron Ga. For more details, please refer to Tab. in the Appendix.
raw @. In terms of exploration speed, ADAM achieves 4.6x speedup
Lifelong learning. ADAM utilizes CD meth- ods to learn the effects of each action, obtain- ing causal subgraphs that contain new items. These new items make it possible to perform more unknown actions, thereby continually ex- panding the knowledge of the game world in bootstrapping manner and achieves lifelong learning. ADAM successfully learns a complex causal graph of all 41 actions we implement, as demonstrated in Fig. 7
Human gameplay aligmment. Avoiding the metadata demonstrates
7 7
use of ADAM'S alignment with human gameplay. We compare VOYAGER, fully text-based agent that relies on metadata We test three tasks that requires environmental factors E. The results are shown in Tab. 3 VOYAGER's performance significantly declines when metadata is not ac- cessible. ADAM performs well on these tasks relying solely on observable information. Ablation experiments demonstrate that MI.LMs contribute to ADAM'S performance on this type of task.
Figure 7: The causal graph learned in lifelong learning. ADAM successfully unlocks all 41 ac- tions we implement and discovers accurate causal relationships.
4.4 ABLATION STUDIES
Ablation of LLM-based CD Prior knowledge and inference capabilities are key factors in ab- lating LLM-based CD. Prior knowledge can be ablated in modified games and enhanced through fine-tuning LLMs on Minecraft knowledge datasets as detailed in Appendix B Inference capabili- ties can be ablated by replacing SOTA LLMs with smaller LLMs. Our result in Tab. shows that ADAM primarily utilizes the reasoning abilities of LLMs rather than relying prior knowledge.
Thhe information used by VOYAGER and ADAM is compared in Tab. 5 in the Appendix.

---

which are rep- Published as a conference paper at ICLR 2025
Model
R&E (success/all)
Model
R&E (success/all)
popt--turbo-previee
0.0 (35/35)
gpt-4
0.1 (35/35)
gpt-4-turho-preview
0.1 (35/35)
gpt-47
0.1 (35/35)
gpt-3.5-turbo
0.2 (34/35)
Llama-2-70B
0.6 (23/35)
gpt-3.5-turbo
0.3 (32/35)
Llama-2-70B
0.9 (16/35)
1llama---70B-innummee
(27/35)
-lama-2-13B-iineull
1.8 (5/35)
Llama-2-70B-finen "70ttfiinttned
0.9 (15/35)
N/A (0/35)
Table 4: Ablation of LLM-based CD. f means "in a modified environment". We record the aver- age number of redundan//eroor items (represented as R&E) in the causal assumption proposed by LLM-based CD, and the success rate after the intervention- based CD. LLMs with only strong prior knowledge but weak inference abilities cannot perform well.
Ablation of intervention-based CD. With- intervention-base CD, agents are forced to rely on exhaustive trials to learn the game knowledge, significantly impairing their effi- ciency and effectiveness. Our experimental re- sults are shown Fig. g Through inter- ventions, ADAM achieves up to 4.4 X speed and a higher accuracy compared to the ablated group.
115
Figure 8: Average number of steps and success rate learn the causal subgraphs of "Collecting" actions (e.g.,
5 RELATED WORK
resentative due higher noise in their sampling data. We perform up to 20 steps for each action, and the CD fails, is counted as 20. Inter- vention contributes up to 4.4x acceleration and higher success rate in the exploration.
Causality in Agents. The integration of causality (Pearl 2009; Peters et al., 2017;;
Schölkopf 2022 into agents is primarily aimed at enhancing the learning efficiency (Méndez- Molina et al., 2020 Seitzer al. 2021; Gasse et al. 2021 Sun et al. 2021: Peng et al. 2022). Peng et al. (002) propose CDHRL to build high- quality hierarchical structures complicated environments.
Meende-Mooling et al. (2020) employ the causal models to restrict the search space. Zeng et al. (023) distinguish agents with causality in two categories: ones relying on prior causal information and ones that learn causality by causal discovery algorithms (Spirtes et al. 2000, Sun et al. 2007 Zhang & Hyvärinen 2009 Zhang et al.) 2011 Peters al. [0144] Zhu et al. 2019). Our work aligns with the latter category and extends to wider range of scenarios where prior knowledge is unknown or not available.
LLMMLLM-Basee Agents.
Leveraging the generalization capabilities of LLMs (Brown et al.
2020
Touvron et al.) 2023a b) to empower agent systems with tools (Qin et al. 2023b Schick al.
2024 Shen et al. 2024) is an essential task (Xi et al., 2023, Wang et al. 2023b). Schick et al. (2024)
design& framework to allow LLM use external APIs complete tasks. Qin al. (2023b) build
related benchmarks to evaluate performance in such tasks. In tasks involving interaction with the environment, (Shinn al. (2024) enhance the agent's ability through language feedback signals. Weil et al.) (2022b) employ Thouught prompting method to optimize the reasoning capabilities of the LLM agent. However, the hallucination and interpretability challenges (Zhang et al., 2023) of I.LMs also accompany these systems. In this work, we prove that the perspective of causal architecture can reduce the reliance on priors and enhance the robustness of inferences.
CONCLUSION
In this work, we introduce ADAM, an embodied causal agent in open-world environments. ADAM innovatively incorporates CD with embodied exploration, significantly improving the accuracy of CD while enhancing the efficiency and interpretability embodied exploration. Without relying on prior knowledge, ADAM demonstrates strong robustness, and its multimodal perception closely aligns with human approaches. Our work sets a foundation for developing autonomous agents that can understand and manipulate environments in causal manner.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENT
We thank all the anonymous reviewers and area chair for their valuable feedback throughout the review process, We also appreciate Zhuangyan Zhang for his contribution to some of the artistic elements in Fig. This work supported by the Shanghai Artificial Intelligence Laboratory.
REPRODUCIBILITY STATEMENT
We provide detailed descriptions our experimental setup Scction 4.1 and we use the same OpenAl API (gpt-4-0125-previeww for all LLM-based methods ensure comparability and reproducibility. We have open-sourced the code of ADAM at httts:://githe com/
OpenCausaLab/ADDr
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023.
Bowen Baker, 1lge Akkaya, Peter Zhokov, Joost Huizinga, Jie Tang, Adrien Ecoffet, Brandon Houghton, Raul Sampedro, and Jeff Clune. Video pretraining (vpt): Learning to act by watching unlabeled online videos. Advances in Neural Information Processing Systems, 35:24639-24665 2022.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Justine Cassell. Embodied conversational interface agents. Communications of the ACM, 43(4):
70-78, 2000.
Frederick Eberhardt and Richard Scheines. Interventions and causal inference. Philosophy of sci- ence, 74(5):981-995 2007.
Linxi Fan, Guanzhi Wang. Yunfan Jiang, Ajay Mandlekar, Yuncong Yang. Haoyi Zhu, Andrew Tang, De-An Huang, Yuke Zhu, and Anima Anandkumar. Minedojo: Building open-ended embodied agents with internet-scale knowledge. Advances in Neural Information Processing Systems, 35: 18343-18362, 2022.
Maxime Gasse, Damien GRASSET, Guillaume Gaudron, and Pierre-Y Oudeyer. Causal rein- forcement learning using observational and interventional data. 2021.
Clark Glymour, Kun Zhang, and Peter Spirtes. Review of causal discovery methods based on graph- ical models. Frontiers genetics, 10:524, 2019.
William H Guss, Cayden Codel, Katja Hofmann, Brandon Houghton, Noboru Kuno, Stephanie Milani, Sharada Mohanty, Diego Perez Liebana, Ruslan Salakhutdinov, Nicholay Topin, et al. The minerl 2019 competition sample efficient reinforcement learning using human priors. arXiv preprint arXiv: 1904. 10079, 2019.
William H Guss, Mario Ynocente Castro, Sam Devlin, Brandon Houghton, Noboru Sean Kuno, Crissman Loomis, Stephanie Milani, Sharada Mohanty, Keisuke Nakata, Ruslan Salakhutdinov, et al. The minerl 2020 competition on sample efficient reinforcement learning using human priors. arXiv preprint arXiv:2101.11071, 2021.
Danijar Hafner. Benchmarking the spectrum of agent capabilities. In The Tenth International Con- ference on Learning Representationss ICLR 2022, Virtual Event, April 25-29, 2022, OpenRe- view.net, 2022. URL htRps:tt:: https:httss///peeeet httt::://openreviiie
het//oorum?id-1W0r 96F
Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse domains through world models. arXiv preprint arXiv:2301.044 2023.

---

Published as a conference paper at ICLR 2025
Anssi Kanervisto, Stephanie Milani, Karolis Ramanauskas, Nicholay Topin, Zichuan Lin, Junyou Li, Jianing Shi, Deheng Ye, Qiang Fu, Wei Yang, et al. Minerl diamond 2021 competition: Overview, results, and lessons learned. NeurlPS 2021 Competitions and Demonstrations Track, pp. 13-28, 2022.
Nan Rosemary Ke, Olexa Bilaniuk, Anirudh Goyal, Stefan Bauer, Hugo Larochelle, Chris Pal. and Yoshua Bengio. Learning neural causal models from unknown interventions. CoRR, abs/1910.01075, 2019. [RLLltttp///axivoorot httppp///aaaaii..oo.//////0100.0 http:////////.//0.......
Zaijing Li. Yuquan Xie, Rui Shao, Gongwei Chen, Dongmei Jiang, and Liqiang Nie. Optimus- I: Hybrid multimodal memory empowered agents excel in long-horizon tasks. CoRR, abs/2408. 03615,, 2024. doi: 10.48550/ARXXI.2448...88 036115 URL htt.thhtt////tc http:////oo..o//1..... org/10. 18550/arXiv
Zichuan Lin, Junyou Li, Jianing Shi, Deheng Ye, Qiang Fu, and Wei Yang. Juewu-me: Playing minecraft with sample-efficient hierarchical reinforcement learning. arXiv preprint arXiv:2112.04907, 2021.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024.
Hangyu Mao, Chao Wang, Xiaotian Hao, Yihuan Mao, Yiming Lu, Chengjie Wu, Jianye Hao, Dong Li, and Pingzhong Tang. Seihai: sample-efficien hierarchical ai for the minerl competition. In Distributed Artificial Intelligence: Third International Conference, DAI 2021, Shanghai, China, December 17-18, 2021, Proceedings 3, pp. 38-51. Springer, 2022.
Arquímides Molina, Ivan Feliciano-Avelin Eduardo F Morales, and Luis Enrique Sucar. Causal based q-learning. Res. Comput. Sci., 149(3):95-104, 2020.
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Mare G Belle- mare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. Human-level control through deep reinforcement learning. nature, 51 8(7540):529-533 2015.
Steve Nebel, Sascha Schneider, and Günter Daniel Rey. Mining learning and crafting scientific experiments: literature review on the use of minecraft in education and research. Journal of Educational Technology & Society, 19(2)355-336, 2016.
Kolby Nottingham, Prithviraj Ammanabrolu, Alane Suhr, Yejin Choi, Hannanch Hajishirzi, Sameer Singh, and Roy Fox. Do embodied agents dream of pixelated sheep: Embodied decision mak- ing using language guided world modelling. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), International Confer- ence on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pp. 2631 16311-2 PMLR, 2023. URL https::/proceediing wtt.iissmm..........mm
mlr
http://tttinnaaaaaaa.aa html
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to fol- low instructions with human feedback. Advances in neural information processing systems, 35:
27730-27744, 2022.
Joon Sung Park. Joseph O'Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, pp. 1-22, 2023.
Judea Pearl. Causality. Cambridge university press, 2009.
Shaohui Peng, Xing Hu, Rui Zhang, Ke Tang, Jiaming Guo, Qi Yi, Ruizhi Chen, Xis- han Zhang, Zidong Du, Ling Li, Qi Guo, and Yunji Chen. Causality- hierar- chical structure discovery for reinforcement learning. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural In- formation Processing Systems 35: Annual Conference on Neural Information Process- ing Systems 2022, NewrIPS 2022, New Orleans, LA, USA, November 28 December 9, 2022, 2022. URL //tptttp//ppppers..o. http:///..aaaap.eeeeeee///////// 0alpaaa0000000aaaaaaa00000-a0a-0
12

---

Published as a conference paper at ICLR 2025
Jonas Peters, Joris M Mooij, Dominik Janzing, and Bernhard Schölkopf. Causal discovery with continuous additive noise models. 2014.
Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Elements of causal inference: foundations and learning algorithms. The MIT Press, 2017.
PrismarineJS. Prismaringjs/minctaayer, 2023a.
URL https://github...
PrismarineJSSmmneffllae
PrismarineJS. Prismarhejsyprissarineevvwwe
2023b.
URL https: com/ http:///iiubbbom//trtaanntsssspp
Viewer.
Yiran Qin, Enshen Zhou, Qichang Liu, Zhenfei Yin, Lu Sheng, Ruimao Zhang, Yu Qiao, and Jing Shao. Mp5: A multi-modal open-ended embodied system in minecraft via active perception.
arXiv preprint 2rX14:2312.07772, 2023a.
Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. Toolllm: Facilitating large language models to master 16000+ real-world apis. arXiv preprint arXiv:2307.16788 2023b.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763. PMLR, 2021.
Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, Vladlen Koltun, Jitendra Malik, et al. Habitat: A platform for embodied ai research. In Proceedings of the IEEE/CVF international conference on computer vision, pp-
9339 9347, 2019.
Timo Schick, Jane Dwivedi- Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36, 2024.
Bernhard Schölkopf. Causality for machine learning. In Probabilistic and Causal Inference The Works of Judea Pearl, pp. 765-804. 2022.
Maximilian Seitzer, Bernhard Schölkopf, and Georg Martius. Causal influence detection for improv- ing efficiency in reinforcement learning, Advances in Neural Information Processing Systems, 34: 22905-22918. 2021.
Yongliang Shen, Kaitao Song. Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. Hugging- gpt: Solving tasks with chatgpt and its friends in hugging face, Advances Neural Information Processing Systems, 36, 2024.
Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36, 2024.
Significant Gravitas. AutoGPT. URL /tt.a//aa///ggaaaa..ooooaaaaaaaa AutoGPT
Peter Spirtes, Clark N Glymour, and Richard Scheines. Causation, prediction, and search. MIT press, 2000.
Peter Spirtes, Clark Glymour, and Richard Scheines. Causation, prediction, and search. MIT press, 2001.
Xiaohai Sun, Dominik Janzing, Berhaard Schölkopf, and Kenji Fukumizu. A kemel-based causal learning algorithm. In Proceedings of the 24th international conference on Machine learning, pp- 855-862, 2007.
13

---

Published as a conference paper at ICLR 2025
Yuewen Sun, Kun Zhang, and Changyin Sun. Model-based transfer reinforcement learning based on graphical model representations. IEEE Transactions on Neural Networks and Learning Systems, 2021.
Xiaojuan Tang, Jiaqi Li, Yitao Liang, Song-Chun Zhu, Muhan Zhang, and Zilong Zheng. Mars: Sit- uated inductive reasoning in an open-world enviromment. CoRR, abs/2410.08116,, 2024, doi: 10. 48550/ARXXV.2410.08// 126. [ntEps::://ooi orq/o........... 48540 http://wwwwv.........
Open Ended Learning Team, Adam Stooke, Anuj Mahajan, Catarina Barros, Charlie Deck, Jakob Bauer, Jakub Sygnowski, Maja Trebacz, Max Jaderberg, Michael Mathieu, et al. Open-ended learning leads to generally capable agents. arXiv preprini arXiv:2217.. 12808, 2021.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez, Ar- mand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language models. CoRR, abs/2302. 13971, 2023a. doi: 1044550ARXIVV230.11909 httts:://do..org// 41w.://...855///880 arvvv. 2302 13971
URL
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton-Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao. Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee. Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurélien Rodriguez, Robert Stojnic, Sergey Edunov. and Thomas Scialom. Llama 2: Open foundation and fine- tuned chat models. CoRR, abs/2307.009283 2023b. doi: 10.48500/RXXIV2337.02222 0928088 URL https::/doi.. 2022://..4455///11...... 2307 09288
Guanzhi Wang, Yugi Xie, Yunfan Jiang. Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. arXiv preprint ar 2rX11:2305.16291, 2023a.
Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et al. A survey on large language model based autonomous agents. arXiv preprini arXiv:2308.1 /432, 2023b.
Zihao Wang, Shaofei Cai, Anji Liu, Yonggang Jin, Jinbing Hou, Bowei Zhang, Haowei Lin, Zhaofeng He, Zilong Zheng, Yaodong Yang, Xiaojian Ma, and Yitao Liang. JARVIS-1: open- open-world world multi-tast agents with memory augmented multimodal language models. CoRR, doi: 00448550/AXXI..311.0555 1.05997. htlthtttp://dde org/10. 48550/arXiv.. 2311.05997
abs/2311.059997, 2023c.
Zihao Wang, Shaofei Cai, Anji Liu, Xiaojian Ma, and Yitao Liang. Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents. CoRR, abs/2302.01560, 2023d. doi: 10.48550/ARX11.20220011 //doi. 48550/ 2302. 560
URL hhtps:
Zihao Wang. Shaofei Cai, Zhancun Mu, Haowei Lin, Ceyao Zhang, Xuejie Liu, Qing Li, Anji Liu, Xiaojian Ma, and Yitao Liang. Omnijarvis: Unified open-world world instruction following agents. CoRR, abs/2407 001 14, 2024. doi: 10. 10.48550/ARXIV. 2407.00114 14. URL. httpt:://////tt wrw/o0.......oooooo 1.0..48555 48550/ 2207..00111414
-cttaaggeeeaiinnnnooooo language- action tokenization enables
Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yo- gatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. Emergent abilities of large language mod- els. Trans. Mach. Learn. Res., 2022, 2022a. URL https: httts:://openreview /wwp/evieeeeeeeep/////p net forum?
id=yzkSU5zdwD
14

---

Published as a conference paper at ICLR 2025
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural Informa- fion Processing Systems 2022, NeurlPS 2022, New Orleans, LA, USA, November 28 December pttes//pppr/222222
9, 2022, 2022b. URL httttttp://pppaee nips wwwp: cc/papez Conference. Abstracct AbstracttCooferenceeeee. html
Yue Wu, So Yeon Min, Shrimai Prabhumoye, Yonatan Bisk, Russ Salakhutdinov, Amos Azaria, Tom M Mitchell, and Yuanzhi Li. Spring: Studying papers and reasoning to play games. Advances in Neural Information Processing Systems, 36, 2024.
Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, et al. The rise and potential of large language model based agents: 2023.
A survey. arXiv preprint arXiv:2309.07480
Fei Xia, Amir R Zamir, Zhiyang He, Alexander Sax, Jitendra Malik, and Silvio Savarese. Gibson env: Real-world perception for embodied agents. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 9068-9079, 2018
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R. Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali. Rwanda, May 1-5, 2023. OpenRe- view.net, 2023. URL
net det v1uYUI-X
Haoqi Yuan, Chi Zhang, Hongcheng Wang, Feiyang Xie, Penglin Cai, Hao Dong, and Zongqing Lu. Plan4mc: Skill reinforcement learning and planning for open-world minecraft tasks. arXiv preprint arXiv:2.03. 2023.
Yan Zeng, Ruichu Cai, Fuchun Sun, Libo Huang, and Zhifeng Hao. A survey on causal reinforce- ment learning. arXiv preprint urXiv:2302.052099 2023.
Kun Zhang and Aapo Hyvärinen. On the identifiability of the post-nonlinear causal model. In Proceedings of the Twenty-Fifh Conference Uncertainty in Artificial Intelligence, pp. 647- 655, 2009.
Kun Zhang, Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Kernel-based conditional independence test and application in causal discovery. In Proceedings of the Twenty-Seventh Conference on Uncertainty in Artificial Intelligence, pp. 804-813, 2011.
Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, et al. Siren's s song in the ai ocean: a survey on hallucination in large language models. arXiv preprint arXiv:2309.012
2023.
Yujia Zheng, Biwei Huang, Wei Chen, Joseph Ramsey, Mingming Gong, Ruichu Cai, Shohei Shimizu, Peter Spirtes, and Kun Zhang. Causal-lear: Causal discovery in python. Journal of Machine Learning Research, 25(60):1-8, 2024.
Shengyu Zhu, Ignavier Ng, and Zhitang Chen. Causal discovery with reinforcement learning. In International Conference Learning Representations, 2019.
Xizhou Zhu, Yuntao Chen, Hao Tian, Chenxin Tao, Weijie Su, Chenyu Yang, Gao Huang, Bin Li, Lewei Lu, Xiaogang Wang, et al. Ghost in the minecraft: Generally capable agents for open-world enviroments via large language models with text-basee knowledge and memory. arXiv preprint arXiv:2305.1 1414, 2023.
15

---

Published as a conference paper at ICLR 2025
A LLMS' PRIOR KNOWLEDGE ON MINECRAFT
We utilize the Minecraft crafting recipes to construct an MC-QA dataset (introduced in Appendix B aiming evaluate the prior knowledge of various LLMs on Minecraft. We test and determine that 0.3 is the optimal temperature as shown in Fig. 9.. Then we test various LLMs on this dataset.
The GPT series (Ouyang al. 2022 Achiam a. 2023) show significantly stronger Minecraft
prior knowledge than other LLMs as shown in Fig. 9
Utilizing this dataset funetune I.LMs can improve their prior knowledge on Minecraft as shown in Fig. 99. On the other hand, by modifying the crafting recipes in Minecraft, we can make the SOTA LLMs (e.g., GPT series) have no prior of this modified environment. This setup enables us to distinctively analyze the roles of prior knowledge and inference capability as shown in Fig. 9d. which serves as the basis of our ablation study.
Strong Ieforance
7.-
2.64 0.63
Ooggos
0,55
modified Game:
original Game:
Modes
0.5
OPT-
CPT-4
2.35
Weak
Strong
222
03
LL-MAD-10
11 108 secim
44
6.10
9.2
LLaMac 238
LLate 708
CPC4 norto
Tomsenature
GP9
(8)
i
(c)
(d)
Figure 9: (a) LLaMA2 Touvron et al. 2023b) demonstrates optimal accuracy in answering crafting recipes at temperature of 0.3, measured as the ratio of correct answers to total questions. (b) Per- formance of open-source LLMs and GPT series models, showcasing their inherent prior knowledge of Minecraft, (c) Illustration of the improvement in performance for open-source LLMs fine-tuned with the MC-QA dataset. (d) Categorizing the LLMs into four types based on their prior knowledge of Minecraft recipes and inference capabilities.
B MC-QA DATASET
MC-QA Dataset
Input prompt:
the game Minecraft, to craft "diamond_sword" what materials needed?
(diamond, stick)
game Minecraft, craft "furnace", what materials needed? [cobblestone]
game Minecraft, craft "birch_ boat", what materials needed? [birch_boat, chestj
game Minecraft, craft Pacacia_chest_baeeee_ what materials needed?
Answer: [acacia_bor _ooat chest]
Figure 10: An example of the questions in MC-QA dataset.
Minecraft is open-world game with a high degree of freedom. Players freely gather materials (e.g., log ), mine ore (e.g., i oronoore ), craft tools (e.g., wooden_pickaxe 7), and more. Crafting recipes (e.g., planks the Minecraft game, and serve as the basis for players to climb the technology tree and obtain more advanced items in the game.
sticks - wood_picka. 7)) the main knowledge in
16

---

Published as a conference paper at ICLR 2025
Given this, LLMs' mastery of crafting recipes can reflect the strength of their prior knowledge in Minecraft. We utilize the crafting recipes in Minecraft (version 1.19) to create the MC-QA dataset. An example of the QA pairs in the dataset is shown in Fig. [T] The questions in this dataset ask for the crafting ingredients required to obtain higher-level items in the technology tree, and the answers the ingredient items. LLMs need to provide their answers in the specified format. The order of the items in the answer is not required. For each question, we provide 3 examples to help LLMs understand the QA task and the format of the answers. For situations where there are multiple ways to craft the same item, we take them all into account to avoid the model being biased toward fixed understanding of the game. The dataset contains 754 QA pairs on the knowledge of obtaining items in Minecraft.
C AGENT IN MINECRAFT
RL explorations in Minecraft agents focus on the efficient use of data (Baker et al. 2022 Fan et al.,
2022), hierarchical RL design (Lin et al. 2021
Mao et al. 2022). innovative architecture modeling (Hafner et al. 2023). ) etc.. Hafner et al. 2023 use world models to achieve general and scalable
RL without human data or curricula. Much work (Guss et al. 2019 2021 Kanervisto et al., 2022]
Fan et al. (022) has made different simplifications for the Minecraft environment to facilitate the RL.
agent systems. The MineDojo framework (Fan et al.] 2022) provides an internet-scale knowledge database and game environments for CLIP model (Radford et al. (021) and RL training. These efforts provide efficient optimization for agent sampling and interaction, but there is still gap when compared to commercial Minecraft games with complete game features like in VOYAGER (Wang
it al. 2023a) and our work.
The reasoning capabilities and rich prior knowledge of LLMs have contributed to much work on Minecraft agents Yuan et al., 2023) Zhu et al. 2023 Wang et al. [023a Qin et al. 2023a Notting-
ham et al. 2023 Wang et al. [2033d c). VOYAGER (Wang et al. (023a) and GITM (Zhu et a 2023) use LLMs' prior knowledge of Minecraft and environment feedback to complete exploration
tasks in a bext-based manner. Qin et al. (2023a) leverage MLLMs to introduce visual information as the contextual basis for action execution. These methods more or less rely on prior knowledge of Minecraft. Our ADAM shows effectiveness even when the game rules are modified.
There are also agents that operate in non-commereial Minecraft environments and utilize frame- level control, including PMan4MC (Yuan et al.] (023)), DEPS (Wang et al., (023d), JARVIS-1 (Wang let al., 2023c), OmniJARVIS (Wang
al. 2024), and Optimus-1 (Li al. 2024). They need approximately 10tt steps for a task, in contrast to the action-level control in our system, which involves around 102 steps for a task. Furthermore, these RL methods require training stages, whereas our system does not involve weight updates. Notably, JARVIS-1 incorporates crafting recipes as an integral part of the system, utilizing prior knowledge rather than learning from scratch. Given the lack of comparuble open-source MLLM-basee basedd Minecraft agent, comprehensive ablation studies on ADAM itself including the ablation of TM and SD (Section 3.4 Tab. I]. LLM-based CD (Tab. and intervention-baser CD (Fig. 8 establish robust base line for MT.LM agents in Minecraft.
Due to the complexity of the Minecraft environment, Crafter (Hafner 2022) simplifies the game space while maintaining the game rules of Minecraft. Crafter is a 2D game environment with craft- ing routes similar to Minecraft. Agents such as SPRING (Wu et al., 2024) and Mars (Tang et al.) 2024) have been developed on this environment.
In the original implementation of VOYAGER, the environment feedback provides detailed infor- mation such as crafting recipe errors (e.g., "I cannot make an iron chestplate because I need: 7 more iron ingots.") "). We retain this informative feedback in all our experiments with VOYAGER, even in environments with modified crafting recipes where the feedback may not align with the changes.
Tab. shows the environmental information used by ADAM and VOYAGER. This setup is used in our experiments at Sec. 4.3 VOYAGER does not have visual input and requires omniscient metadata (Meta) which is not explicitly exposed to human players, while ADAM utilizes visual input and other observable information to make decisions.
17

---

Published as a conference paper at ICLR 2025
VOYAGER
VOYAGER Meta
ADAM
ADAM w/o MLLMM
Observation
Environment Feedback,
Environment Feedback,
Pixels.
Inventory
Space
Inventory,
Inventory
Inventory
Meta
Action
Code
Code
Discrete
Discrete
Space
Table 5: Comparison of environmental information used by ADAM and VOYAGER. VOYAGER does not have visual input and needs omniscient metadata (Meta) which is not explicitly exposed to human players, while ADAM relies on the visual input and the information observable by human players to make decisions.
Action Type
Action-Item Dependency
Data Quality
Skill Level
Collecting
Simple
Noise
Low
Crafting
Complex
Clean
High
Smelting
Complex
Noise
High
Table 6: Three action types in experiment setting. "Smelting" actions have complex causal
subgraphs, often leading to omissions in LLM-based CD. "Collecting" actions have noisy sampling data, and the results the LLM- LLMM-based CD are often redundant. The "Crafting" actions have complex causal subgraphs and clean sampling data.
D IMPLEMENTATION DETAILS
D.1 PROMPT
The prompt for LLM-based CD is shown in Fig. 11. which composed of components: (1) Role Playing, which assigns specific role to the LLM; (2) Problem Setting, which provides specific details of the inference task; (3) Letter Mapping, which involves mapping item names to letters, simplification that facilitates the formatted output: (4) Few-shot Prompting, which involves provid- ing the LLM with several inference examples in chain-of-though. (Wei al. (0226) style; (5) Data Da, which is the data collected by the interaction module for inference.
D.2 ACTION SPACE AND MOVEMENT SPACE
We have implemented 41 discrete actions and 6 movements to ensure the agent can freely explore the Minecraft world through diverse range of their combinations. The actions can be divided into three categories: "Smelting", "Collecting", and "Crafting". "Smelting" actions have complex causal subgraphs, often leading to omissions in LLM-based CD. "Collecting" actions have noisy sampling data, and the results of the LLM-based CD are often redundant. The "Crafting" actions have complex causal subgraphs and clean sampling data.
The movement space corresponds to the low-level movement control visible to human players, in- cluding moving forward / backward, lowering / raising the agent's coordinates and turning left / right.
E ROBUSTNESS
Tab. 7 shows our experiment results in the modified Minecraft environment where the crafting recipes altered. ADAM maintain its performance as it is equipped with CD methods, whereas agents that rely on prior knowledge struggle to explore efficiently. The result demonstrates the robustness and generalization capabilities of our ADAM architecture.
18

---

considered additional obtained during collection and Published as a conference paper at ICLR 2025
You Minecraft game analysis assislant. Our lask infer the effect action and explore the causal relationship by analyzing the items consumed and generated before and after an action. An action is a single, atomic action consisting of
"Crafing" type actions,
combining raw materials
2. "Callecting" type actions, that collecting certain items. There may be some by products in this process. consuming
3. "Smelling" type needed.
materials, obtaining
it's possible that the action didn't have the expected effect, or that some additional items were collected, and we'll give multiple sampling records
robustness. Such racords all comespond the execution the action. Your answer will only refer to the alphabelical codes of these ltems.
The should be in formal (Cause; Effect)
example of the Observed Item Space, "m" and "'n are for few-shot learning and not included the technology tree avoid prior) Items involved in the game include
(Comment: The following
'log'
bb: "planks'
"crafting_tabl"" tabgg
d' 'stick'
waoden pickaxe'
7: 'obblessonee'
'stone_ siton__pokaxee
h:: 'raw, iron'
T: "umaace'
T: 'ron. Ingor
k: 'iron con__pickae'
'diamond"
Imi: 'redstone"
'redstone
log corresponds variety log log
planks correspond to variety of planks named xx_ xx_planks
1. Initial items: iron_ pickaxe, crafting table Consumed Items:; Added items: redstone, cobbiestone, cobbled, despslate
2. Initial items: iron, pickaxe, crafting table :onssmmed items: Added Items: cobblestone, redstone, andesite
3. Initial items: iron, plckkaeeee crafting, table :oonsumed Items: Added items: granite
4. Initial items: iron, pickaxe, crafting_tan table :Consumed items: ; Added items: cobblestone, cobbled cobbleddddepplaat
Initial
pickaxe, crafling_labe fable :Consumed
inference:
Because there is no item consumption, this 'Collecting" task. Redstone higher-level mineral and needs to be collected with iron_ pickaxe (K'). The will not be considered. So my answer is:
Cause: Effect=(k:
1. Initial Items: stick, redstone, orafting, table; Consumed items: stick, redstone; Added items: redstone_ torch
2. Initial items: stick, redstone, crafting_tabe table; Consumed items: /Added items:
Inilial slick, redsione, crafting Consumed redslone, Added redslone larch
4. Initial items: stick, redstone, crafting_tan table; Consumed items: stick, redisoon;; Added items: redstone, torch
Initial stick, redsione, crafting Consumed redslone, Added redslone Jarch
inference:
Because redstone and sticks are being used to craft redstone torches, this is "Crafting" task. In addition to raw materials, the "Crafling" bask also requires crafting_late lable (c'). Cause; Effect-(c, my n)
(Our Data Your inference:
Figure 11: The prompt for LLM-based CD. The contents in red will be replaced in the inference process.
F GENERALIZATION
The ADAM architecture is general framework for embodied agents operating in various open- world environments including Minecraft. When adapting ADAM to other application scenarios, some modifications may be necessary:
(1) The world knowledge in Minecraft the dependence between items and actions. Conse- quently, in this paper, items and actions designed causal graph nodes. When mi- grating to other environments, key elements related to the agent's task objectives can be similarly designed as causal graph nodes.
19

---

Published as a conference paper at ICLR 2025
Framework
Wooden Tool
Stone Tool
Iron Tool
Diamond
React w/ TM w/ SD
91 34(2333
139(1/3)
N/A (0/3)
N/A (0/3)
Reflexion w/ TM w/ SD
76 28(233
120 = 40(2/3)
N/A (0/3)
N/A (0/3)
AutoGPT w/ TM w/ SD
82 22(223)
124(1/3)
N/A (0/3)
N/A (0/3)
VOYAGER
95 = -3((2/3)
152 + (3(333)
N/A (0/3)
N/A (0/3)
VOYAGER Guided
176(1/3) 108 35(2/3)
N/A (0/3)
N/A (0/3)
ADAM
28 = 4(3/3)
52 14(3/3)
94 27(3/3)
109 - 34(2/3) 18(2/3)
ADAM Parallel
2(3/3)
31 7(333)
54 14(3/3)
Table Performance in the modified Minecraft game. Each method has three trials for maximum length of 200 steps. The success rate is depicted in the parentheses. ADAM can maintain its per- formance as it is equipped with CD methods, whereas agents that rely on prior knowledge struggle to explore efficiently. The result demonstrates the robustness and generalization capabilities of our ADAM architecture.
(2) In ADAM, the perception module utilizes vision-based MLLM and does not rely niscient metadata. This allows the module to adapt well to other visual tasks. If conditions permit (e.g., robot equipped with LiDAR), the perception module can provide more pre- cise information, potentially further improving performance.
(3) To model actions a finite set of causal graph nodes, it is necessary to discretize the continuous action space. The granularity of this discretization should determined based the specific environment.
20


Reviews:
[{"review_id": "2dffxqlToh", "reviewer": "Reviewer_GNQs", "summary": "This paper introduces ADAM, an autonomous agent for open-world environments like Minecraft that builds a causal graph from scratch to improve interpretability and performance without relying heavily on pretrained knowledge. Through a combination of interaction, causal reasoning, planning, and multimodal perception, ADAM outperforms existing agents in task success and adaptability, even in modified game settings. ADAM’s approach establishes a new standard for causal reasoning in embodied agents.", "questions": "See the weakness.\n\nAccording to the author's reply, they refuse to admit the unfair comparisons they made during the rebuttal stage. I even doubt whether the author carefully reviewed the responses from all reviewers. Therefore, I have decided to change the score to strong reject. \n\nI strongly recommend that the author carefully review the reviewer's comments and provide a serious response.", "limitations": "", "rating": 1, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1.\tThe figures in the paper are well-done and enhance clarity, making the content easier to understand.\n2.\tThe experiments conducted in Minecraft show a higher success rate than those achieved by Voyager.", "weaknesses": "1.\tThe paper appears to be hastily prepared, as it contains numerous typos and minor errors, such as inconsistencies between “Fig.” and “Figure” references and improper usage of quotation marks in Table 4’s caption. I recommend that the authors carefully review and correct these issues.\n2.\tThe major issue lies in the extensive use of pretrained language models that already incorporate substantial knowledge of Minecraft. Since language models may internally form a comprehensive causal graph of the game world, primarily in linguistic form, the proposed additional causal graph construction might be redundant. I suggest that the authors explore scenarios with completely altered world rules in Minecraft to test the validity of models like GPT in such modified environments, perhaps using a setting like “Mars.” Alternatively, they could consider using a language model entirely devoid of Minecraft knowledge, though this may be challenging to achieve.\n3.\tThe agent’s modular design is nearly identical to Voyager, with the primary addition being the causal graph. Experimentally, however, it does not show significant advantages over Voyager, as it does not complete tasks that Voyager was unable to.\n4.\tThe causal graph generated by the model ADAM is quite similar to the hybrid knowledge graph in memory described in [1]. The authors should clarify the differences.\n5.\tAdditionally, the current causal graph is entirely object-centric. In open-world Minecraft, there are many open-ended tasks, such as building and farming, which are not strictly object-centric. This limitation restricts ADAM’s generalization capability in open-ended tasks.\n6.\tSeveral relevant works are not cited, including:\n\n\t[1] Optimus-1: Hybrid Multimodal Memory Empowered Agents Excel in Long-Horizon Tasks\n \n\t[2] Mars: Situated Inductive Reasoning in an Open-World Environment, NeurIPS 2024\n \n\t[3] OmniJARVIS: Unified Vision-Language-Action Tokenization Enables Open-World Instruction Following Agents, NeurIPS 2024", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wBffbEMXrb", "reviewer": "Reviewer_Gehh", "summary": "The paper introduces an architecture for agents that play the game of Minecraft, based on combining Large Language Models with causal inference. Featuring different modules (e.g., planner, actor, perception), the method is based on inferring causal graphs related to the various crafting dependencies of Minecraft's technology tree, and on enabling an agent to use the knowledge about these dependencies for progressing in the game. The method is evaluated against similar LLM-based methods (albeit using a different action space).", "questions": "- Would the method generalize to other environments? If so, what are the assumptions and requirements for the application of the method to a new environment?\n- How does the method compare to approaches trained with reinforcement learning? Is it possible to train an agent with reinforcement learning on the same action space that ADAM uses?\n- What is the captioning performance of the perception module? What are its failure cases?", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "- The method seems to be the first approach that combines casual inference with LLM agents in code-based action spaces, which is a potentially very important direction for future research.\n- The method performs quite well for inferring causal graphs on Minecraft, and it seems to provide a way for agents to take advantage of those causal graphs.", "weaknesses": "- The presentation of the method is quite high-level and does not help the reader understand how the method actually works in practice. When the different \"modules\" are introduced, it is not clear a priori what they actually are. Are they just prompts and specifications to a GPT4 model? If so, it could be beneficial to show one of the prompts earlier, to guide the understanding of the rest of the paper.\n- The comparisons in the paper are unclear, due to the choice of a specific action space that is different from the one used in previous work. Indeed, while the paper mostly discusses the \"observation space\" difference compared to the setting usually employed in reinforcement learning papers, one crucial difference is the one in action space. For instance, Voyager works in an extremely more high-level action space compared to DreamerV3. This is not accurately depicted in the current version of the paper.\n- It could be surprising to observe that an off-the-shelf open model is accurately describe an observation to the level of providing enough information for an actor to take the optimal action, especially in an environment that is as visually rich as Minecraft. An ad-hoc evaluation of this specific capability would strengthen the paper.\n- The method seems to be highly Minecraft-specific, and the paper does not extensively discuss how it could be generalized to other domains.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "91QtQ1Q4l2", "reviewer": "Reviewer_b8Jp", "summary": "This paper introduces ADAM (An emboDied causal Agent in Minecraft) - an agent architecture that autonomously explores, learns causal world knowledge, and executes complex tasks in Minecraft from multimodal inputs. The system consists of four main components: interaction module, causal model module, controller module, and perception module. The interaction module samples actions and records observations. The causal model module infers causal relationships and constructs causal subgraphs for each action.\nThe key innovation is integrating causal discovery methods with embodied exploration, enabling the agent to learn accurate causal relations from scratch without relying on prior knowledge.", "questions": "See previous", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The incorporation of causal discovery methods in a modular framework is a novel in LLM-based embodied exploration, and it does not rely on privileged information unlike prior work\n2. The paper demonstrate strong empirical results with well-designed experiments, that led to significantly faster discovery of skills in Minecraft. Performance in modified environments where prior knowledge is invalid did not degrade performance too much demonstrates causal learning is indeed effective. Method also does not require meta-data.\n3. The experiments are solid with multiple baselines and includes comprehensive ablation studies as well as detailed analysis of failure cases", "weaknesses": "1. One concern is whether ADAM scales with more complex world and causal graph for intervention-based causal discovery (CD).\n2. Interestingly the paper proposes a multimodal agentic framework but all the baselines compared to are text-based frameworks. It would be good to have at least one multi-modal baseline, e.g. [1] as this is also cited by the authors.\n\n[1] Wang, Z., Cai, S., Liu, A., Jin, Y., Hou, J., Zhang, B., ... & Liang, Y. (2023). Jarvis-1: Open-world multi-task agents with memory-augmented multimodal language models. arXiv preprint arXiv:2311.05997.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WbUy3JOMM6", "reviewer": "Reviewer_yx6R", "summary": "The paper proposes a Minecraft agent called Adam that relies on a combination of (M)LLM inference and causal discovery. At the core of the method is a causal graph that represents the agent's expertise on the environment logic. The causal graph is proposed by an LLM and each relation in the graph is confirmed/disproved by environment interactions. The proposed method obtains diamonds faster and more reliably than prior methods. Moreover,", "questions": "- ad interpretability claim: how does the interpretability of the presented method relate to the interpretability of baseline methods, e.g., voyager? isn't the interpretability of the method compromised by the lack of interpretability of the used LLMs?\n- the proposed method is quite complex as it consists of 4 modules consisting of several submodules each and the presented ablation studies seem insufficient to rigorously justify such a complicated system; so how was the method designed? how much inspiration was drawn from prior work? \n- Adam is claimed to be a \"generalizable framework\"; why not back this claim with a complementary application in another environment? \n- what are possible limitations of the method? (the paper does not mention any)\n- what do the error bars in tables 1, 2, 3 mean?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 1, "presentation": 3, "contribution": 2, "strengths": "- the paper is well structured and clearly written\n- the combination of LLM-prompting with CD seems to be quite novel\n- the intervention-based refinement of the causal graph seems meaningful and practical", "weaknesses": "- unsupported claims: the authors claim their method has \"excellent interpretability\" and that their agent \"closely aligns with human gameplay;\" yet, I cannot find any empirical evidence supporting these claims. \n- a runtime/memory analysis (be it theoretical or empirical) is completely missing, i.e., it is not clear at which cost the claimed SOTA results come\n- the results presented in figure 1 are based on a modified causal graph claiming that this removes prior knowledge from LLMs; yet, it is unclear in how far this claim is true; it would be interesting to see an analysis akin to appendix A for the modified environment\n- the choice of the acronym Adam is at best unfortunate as it coincides with one of the most influential machine learning papers (https://arxiv.org/abs/1412.6980) and could be mistaken for an attempt to tap unwitting citations as both paper titles start with \"Adam: ...\"\n- insufficient reproducibility due to missing source code", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)