ADAM: An Embodied Causal Agent in Open-World Environments

## Abstract

Abstract In open-world environments like Minecraft, existing agents face challenges in continuously learning structured knowledge, particularly causality. These challenges stem from the opacity inherent in black-box models and an excessive reliance on prior knowledge during training, which impair their interpretability and generalization capability.
To this end, we introduce Adam , An emboDied causal Agent in Minecraft, that can autonomously navigate the open world, perceive multimodal contexts, learn causal world knowledge, and tackle complex tasks through lifelong learning. Adam is empowered by four key components: 1) an interaction module, enabling the agent to execute actions while documenting the interaction processes; 2) a causal model module, tasked with constructing an ever-growing causal graph from scratch, which enhances interpretability and diminishes reliance on prior knowledge; 3) a controller module, comprising a planner, an actor, and a memory pool, which uses the learned causal graph to accomplish tasks; 4) a perception module, powered by multimodal large language models, which enables Adam to perceive like a human player.
Extensive experiments show that Adam constructs an almost perfect causal graph from scratch, enabling efficient task decomposition and execution with strong interpretability. Notably, in our modified Minecraft games where no prior knowledge is available, Adam maintains its performance and shows remarkable robustness and generalization capability. Adam pioneers a novel paradigm that integrates causal methods and embodied agents in a synergistic manner. Our project page is at https://opencausalab.github.io/ADAM .

## 1 Introduction

Figure: Figure 1.1: (a) The technology tree for acquiring diamonds in the Minecraft game. Adam can precisely discover item dependencies from scratch. (b) Modified Minecraft technology tree, where the prior knowledge from the Internet or wiki does not align with the actual game dynamics. Red arrows denote removed dependencies, while blue arrows denote added dependencies. (c) In the game setting shown in (b), Adam maintains the ability to learn the correct causal graph and successfully obtains diamonds , whereas other methods can only acquire raw_iron within the step limit, and Adam achieves a 4.6$\times$ speedup in obtaining raw_iron compared to the SOTA.
Refer to caption: x1.png

Embodied agents exploring open-world environments mark a critical frontier in artificial intelligence (AI) research (Cassell, ; Xia et al., ; Savva et al., ). The ultimate goal is to build generally capable agents (GCAs) (Team et al., ) that can autonomously perform a broad range of tasks through perception, learning, and interaction (Mnih et al., ; Xi et al., ; Park et al., ).
Minecraft (Nebel et al., ), a globally renowned 3D video game, serves as a representative open-world environment for these agents. It offers a randomly generated world of massive blocks, where players need to master complex crafting recipes (*e.g.*, planks + sticks $\to$ wood_pickaxe ) and gather resources (*e.g.*, mining cobblestone with wood_pickaxe ), progressively unlocking new items in the technology tree.
The substantial freedom and precise simulation of physical laws in Minecraft render it an exceptional platform for researching GCAs.

In Minecraft, two primary approaches for developing GCAs have been extensively explored: reinforcement learning (RL)-based (Lin et al., ; Baker et al., ; Fan et al., ; Mao et al., ; Hafner et al., ) and large language model (LLM)-based (Wang et al., ; Zhu et al., ; Qin et al., ; Nottingham et al., ; Wang et al., , ). Specifically, RL agents learn through interactions and updating their black-box model weights, which poses challenges for interpretability, efficiency, and generalization. On the other hand, LLM-based agents possess and rely on rich prior knowledge of both virtual games and real worlds (Ouyang et al., ; Wei et al., ; Achiam et al., ). Their reliance on omniscient data (*e.g.*, GPS coordinates, voxel blocks, biome, *etc*, which are not explicitly observable by the player) (Wang et al., ; Zhu et al., ; Wang et al., ) presents challenges for generalization and human gameplay alignment.

To address these issues, we propose Adam, An emboDied causal Agent in Minecraft, that can autonomously navigate the open world, perceive multimodal contexts, learn causal world knowledge, and tackle complex tasks through lifelong learning.
Specifically, Adam is composed of four key modules as shown in Fig. 1.2:
(1) Interaction module, which enables the agent to execute actions from the action space and processes the agent’s observable information into formatted records.
(2) Causal model module, which includes two causal discovery (CD) methods. LLM-based CD utilizes interaction records to make causal assumptions. Intervention-based CD refines these assumptions to derive a causal subgraph. Multiple causal subgraphs are integrated into a comprehensive causal graph (*i.e.*, technology tree).
(3) Controller module, which includes a planner, an actor, and a memory pool. The planner can utilize the causal graph to perform task decomposition. The actor uses the subtasks for action choosing. The memory pool ensures the long-term context dependence.
(4) Perception module, which is driven by multimodal LLMs (MLLMs), enabling Adam to perceive its surroundings without relying on omniscient data, thereby achieving human-like gameplay.

Extensive experiments demonstrate that Adam achieves a 2.2$\times$ speedup compared to the SOTA in the task of obtaining diamonds . In scenarios where crafting recipes are modified (Fig. 1.1b), only Adam maintains the ability to obtain diamonds , while other methods can only acquire raw_iron within the step limit, and Adam achieves a 4.6$\times$ speedup in obtaining raw_iron compared to the SOTA (Fig. 1.1c). Adam demonstrates strong interpretability through constructing a nearly perfect technology tree (Fig. 1.1a) from scratch, whereas other methods exhibit at least 30% errors or omissions. Meanwhile, Adam closely aligns with human gameplay without relying on omniscient metadata, while maintaining comparable environmental perception performance to methods that utilize such data.

Overall, our contributions are as follows:

- (1)
We introduce Adam, an embodied causal agent that autonomously navigate the open world, perceive multimodal contexts, learn causal world knowledge, and tackle complex tasks through lifelong learning.
- (2)
We tackle the limitations of existing embodied agents. Our Adam demonstrates strong generalization capability without relying on prior knowledge or omniscient metadata, unlike other LLM-based agents, while exhibiting human-like exploration behavior.
- (3)
We pioneer the integration of causal methods into open-world embodied agents, allowing the agent to organize the learned knowledge in a rigorous causal graph, thereby demonstrating excellent interpretability and robustness.
- (4)
We improve the CD performance by employing embodied agent-driven interventions, which enhances the accuracy and efficiency of CD compared to existing methods without interventions.

Figure: Figure 1.2: Four key modules of Adam. The interaction module executes actions in the environment according to the task and records the processes. The causal model module identifies the causal relationship between items and actions to construct an ever-growing causal graph. The controller module implements task execution based on the learned causal graph. The perception module aligns the agent’s behavior more closely with human gameplay.
Refer to caption: x19.png

## 2 Preliminaries

#### Causal graphical models (CGMs).

A CGM represents the structure of causality within a system (Peters et al., ) by detailing the direct causal relationships among a set of variables $X_{1},\ldots,X_{n}$. It is characterized by a distribution over these variables and is associated with a directed acyclic graph (DAG), known as a causal graph. In this graph, each node corresponds to a variable, and each directed edge signifies a direct causal relation from $X_{i}$ to $X_{j}$.

#### Causal discovery from interventions.

Causal Discovery (CD) (Spirtes et al., ; Pearl, ; Peters et al., ; Glymour et al., ) is a fundamental process to infer causal relationship from data. The relationship is typically represented in the form of a causal graph. While observational data reveals correlations, interventions allow us to analyze causal dependencies between variables. Specifically, interventions alter the distribution of variables (*e.g.*, types of initial items) during experimental sampling, which serves as a gold standard for CD (Eberhardt & Scheines, ). By observing their effects, we can identify causal relationships rather than mere correlations.

## 3 Method

In this section, we begin with the basic notations and definitions in our work (Sec. 3.1). Then, we give an overview of our Adam framework (Sec. 3.2). Next, we detail the four modules of Adam: interaction module (Sec. 3.3), causal model module (Sec. 3.4), controller module (Sec. 3.5), and perception module (Sec. 3.6).

### 3.1 Notations and Definitions

We first introduce several key notations and definitions in our work. Sets are denoted by uppercase letters, and their elements by lowercase letters.

Inventory: The set $I_{t}$ of items possessed by the agent at any step $t$.
Initialization: At the initialization of Minecraft instances, the initial inventory $I_{0}$ can be specified by the agent.
Action Space: The set $A$ of actions whose names (*e.g.*, gatherWoodLog) are replaced by letters and invisible.
Adam must independently discover the effects of these actions.
Movement Space: The set $M$ of basic movements whose names (*e.g.*, moveForward, moveBackward) are visible to Adam.
Step: The agent takes an action $a$ in the environment. A step ends either when action completes or when execution times out.
Observed Item Space: The set $S$ of all items that Adam has encountered. Initially, $S$ is empty.
Environmental Factors: The set $\mathcal{E}$ of environment conditions (*e.g.*, biome, surrounding block types).
Task: Denoted by the tuple $(I_{\text{goal}},\mathcal{E})$, a task is accomplished at step $t$ if $I_{\text{goal}}\subseteq I_{t}$ and the factors $\mathcal{E}$ are present within a certain distance around the agent.

### 3.2 Overview

Adam comprises four modules as depicted in Fig. 1.2. Given a task $(I_{\text{goal}},\mathcal{E})$, the interaction module enables the agent to execute each action $a$ and records data $D_{a}$. This data is then utilized by the causal model module, which employs LLM-based CD to make causal assumptions and intervention-based CD to refine these assumptions into causal subgraphs. These subgraphs are integrated into a causal graph (*i.e.*, technology tree).
Once the causal graph $\mathcal{G}$ contains all required items $I_{\text{goal}}$ in the task, the controller module completes the task from an empty inventory, aided by visual descriptions from the perception module. Newly discovered items enable the execution of new unknown actions and the CD on these actions. This iterative process ensures the lifelong learning through continuous engagement and adaptation. Adam is a general framework that can be extended to other open-world environments, as discussed in Appendix F.

### 3.3 Interaction Module

Figure: Figure 3.1: The interaction module has two core functionalities: sampling and recording. Sampling involves executing actions in the environment, and recording involves processing and documenting the observable information. For instance, the initial action space is {gatherWoodLog}, whose name is not exposed to Adam and is denoted as $\{a\}$ here (Note that, the original notation {gatherWoodLog} is retained in the figure for the illustrative purpose.). The initial observed item space is $\varnothing$. After executing $a$ for one step, logs () are obtained. A sampling can be represented as $(a,\varnothing,\{\raisebox{-1.79993pt}{\includegraphics[height=9.0pt]{ materials/Oak_Log_29_JE5_BE3.pdf}}\})$, where $\varnothing$ is the initial inventory and {} is the inventory after this step. The result is recorded as $R=(\varnothing,\varnothing,\{\raisebox{-1.79993pt}{\includegraphics[height=9.0 pt]{materials/Oak_Log_29_JE5_BE3.pdf}}\})$, where the first $\varnothing$ is the initial inventory and the second $\varnothing$ indicates that no items are consumed, and $\{\raisebox{-1.79993pt}{\includegraphics[height=9.0pt]{materials/Oak_Log_29_JE 5_BE3.pdf}}\}$ represents the items that are obtained. After sampling $N$ times, data $D_{a}=\{R_{1},\ldots,R_{N}\}$ is provided to the causal model module for CD. If the causal relation failed to be identified, resampling on $a$ occurs; if successful, new actions like craftPlanks are enabled by the acquisition of , and the observed item space is updated to {}.
Refer to caption: x20.png

The interaction module (Fig. 3.1) enables the agent to execute actions for sampling and records observable information. Initially, the action space $A$ contains one element gatherWoodLog, which is the most basic action in Minecraft and a common setup for Minecraft agents (Wang et al., ). As the agent acquires certain new items, new actions are unlocked in $A$. For example, the action of mining becomes available only after the agent obtains a mining tool (*e.g.*, a wooden_pickaxe ). By continuously collecting data on each action and coordinating with other modules, the nodes on the technology tree are progressively discovered by the agent.

#### Sampling.

This involves initiating a Minecraft instance with all observed items $S$ as the initial inventory $I_{0}$, and then executing action $a$ to observe the agent’s inventory $I_{1}$ after this step. The quantity of each item in $S$ and $I_{1}$ is recorded. This sampling, represented as $(a,S,I_{1})$, is performed $N$ times, focusing on one specific action at a time. Due to the same configuration of these $N$ samplings, parallelization is available and accelerates the exploration.

#### Recording.

This involves processing the results of samplings and documenting them. For the sampling $(a,S,I_{1})$, consumed items are denoted by $X$, which includes items with decreased quantities and items that are present in $S$ but absent in $I_{1}$. Conversely, obtained items are denoted by $Y$, which includes items with increased quantities and items that newly appear in $I_{1}$. A single record is denoted as $R=(S,X,Y)$. Such N records on action $a$ collectively form data $D_{a}=\{R_{1},\ldots,R_{N}\}$.

### 3.4 Causal Model Module

The causal model module uses data $D_{a}$ to infer causal relationships and constructs causal subgraphs for each action $a$. These subgraphs are then integrated into a comprehensive causal graph.

A causal relationship is a stable and repeatable dependence in any step, where the agent takes an action $a$ and the acquisition of items $E$ (effect items) relies on the items $C$ (cause items) possessed by the agent before the step.
Since Adam focuses on item acquisition, causal relations where no items are obtained do not contribute to the technology tree.

In the causal model module, LLM-based CD makes assumptions on the causal relationships, which effectively reduces the number of item nodes that need to be confirmed in the causal subgraph and achieves acceleration. Then, intervention-based CD refines these assumptions and accurately constructs causal subgraphs. We also detail our optimization techniques employed in this module.

Figure: Figure 3.2: LLM-based CD performs causal reasoning under the guidance of the prompt. Role Playing assigns an analysis assistant role to the LLM. Problem Setting provides the reasoning task. Letter Mapping maps the item names to letters for the accurate output. Few-shot Prompting provides examples for chain-of-thought (Wei et al., ) reasoning. Data $D_{a}$ is presented in the same form as the few-shot examples. The output of LLM serves as the causal assumption.
Refer to caption: x36.png

#### LLM-based CD.

The input to LLM-based CD (Fig. 3.2) is the data $D_{a}$ containing $N$ records, and the output is a causal assumption, which consists of cause items $C$ and effect items $E$.
The prompt is designed with five components:
(1) Role Playing, which assigns a specific role to the LLM. In this context, the LLM serves as a causal analysis assistant, dedicated to extracting causal relationships from data.
(2) Problem Setting, which provides the reasoning task and the fundamental concepts of the environment. We avoid introducing specific environmental knowledge for generalization.
(3) Letter Mapping, which involves mapping item names to letters, a simplification that facilitates the formatted output.
(4) Few-shot Examples, which involves providing the LLM with several reasoning examples in a chain-of-thought (Wei et al., ) style, including example questions, the reasoning processes, and the expected answering format. The examples are independent of the technology tree, thus preventing the introduction of prior knowledge.
(5) Data $D_{a}$, which consists of $N$ records from the interaction module, and is formatted consistently with the few-shot examples.

#### Intervention-based CD.

Figure: Figure 3.3: Intervention-based CD verifies causal assumptions. An example of causal assumption is that, under the action $a$, log () and crafting_table () contribute to the acquisition of planks (). This assumption is denoted as ($a$, {, }, {}). Intervention-based CD will verify each item in {, }. When removing from the inventory and executing action $a$, cannot be obtained, proving that is a dependency of , and this edge ( $\rightarrow$ ) is retained in the causal graph (represented in green). When removing and executing action $a$, still can be obtained, which shows that is not a dependency of , and this edge ( $\rightarrow$ ) is removed from the causal graph (represented in red). Intervention-based CD results in a corrected causal subgraph. Multiple subgraphs can be combined into the technology tree in Minecraft. The actions are not shown in the technology tree for the sake of simplicity.
Refer to caption: x37.png

Intervention is a method to experimentally verify causal relationships among variables. Intervention-based CD (Fig. 3.3) can refine the causal assumptions and construct a highly accurate causal graph.

Before interventions, it has to be confirmed that $C$ is a sufficient condition for $E$. If $C$ has already lacked the necessary items to achieve $E$, then the vital edges are missing and the graph cannot be corrected by excluding redundant edges.
Specifically, by sampling $(a,C,I_{1})$ as described in section 3.3, if $E$ is consistently absent from $I_{1}$, the assumption is deemed incorrect, leading the LLM-based CD to re-infer the assumption. If these inferences continue to fail, the interaction module will resample data $D_{a}$.

Intervention-based CD performs sampling $(a,C\setminus\{c\},I_{1})$ for each item $c\in C$. For each item $e\in E$, if there is always $e\not\in I_{1}$ within a maximum number of samplings, then $c$ is the cause of $e$, and the edge $c\rightarrow e$ is retained. If at least one sample includes $e\in I_{1}$, then $c$ is not indispensable for $e$, and the edge $c\rightarrow e$ is removed. Intervention-based CD yields accurate causal subgraphs, which are subsequently integrated into a complete causal graph.

#### Optimization techniques.

(1) Temporal Modeling (TM): Temporal information can help determine causal direction. Events occurring later cannot be the cause of events occurring earlier. This eliminates some edges in the causal graph.
(2) Subgraph Decomposition (SD): By focusing on the causal effects of individual actions, the causal model module processes a manageable number of items (only those in each cause-effect pair) at a time. This significantly reduces the complexity of the CD process.

Figure: Figure 3.4: The controller module comprises three components. The Planner utilizes the reasoning capabilities of LLMs to decompose the task. It receives the current inventory and the learned causal graph as input. The Actor leverages LLMs to choose an action $a$ in the the action space $A$ or a movement $m$ in the movement space $M$, and execute it in the environment. The Memory records the step information, including action trajectories and item changes every step.
Refer to caption: x78.png

### 3.5 Controller Module

During the execution of task $(I_{\text{goal}},\mathcal{E})$, Adam starts from an empty inventory (*i.e.*, $I_{0}=\varnothing$) to ensure the fair comparison with other methods. After all items in $I_{\text{goal}}$ have been obtained in the causal graph, the controller module (Fig. 3.4) is responsible for executing the task. If there are newly discovered items, they will be added to the observed item space $S$ and pend for a new cycle of CD, thus achieving lifelong learning. The controller module comprises three components as described below.

#### Planner.

The Planner utilizes LLMs to decompose the task with current inventory $I_{t}$ at step $t$ and the learned causal graph. Relying solely on the causal graph is suboptimal as actions may fail or have side effects. LLMs can fully utilize the inventory information and provide detailed thought process of the decomposition process, which is passed to the Actor for action choosing.

#### Actor.

The Actor leverages LLMs to choose an action $a\in A$ or a movement $m\in M$ to execute. It receives the task decomposition from the Planner, the description of game image at this step, and the records from the Memory. In the task $(I_{\text{goal}},\mathcal{E})$, the Actor prioritizes obtaining the items $I_{\text{goal}}$, because this process may affect the agent’s surroundings. For instance, consider the task $(I_{\text{goal}}=\{\texttt{raw_iron}\ \raisebox{-1.79993pt}{\includegraphics[
height=9.0pt]{materials/Raw_Iron_JE3_BE2.pdf}}\},\mathcal{E}=\{\texttt{grass}
\ \raisebox{-1.79993pt}{\includegraphics[height=9.0pt]{materials/Grass.pdf}}\})$, this task is accomplished only when the agent possesses raw_iron and simultaneously stays within a certain distance of grass . Mining raw_iron typically requires digging underground, which can impact the search for grass on the surface. If Adam first searches for grass and then proceeds to dig for raw_iron , the grass may no longer be nearby, requiring additional effort to locate it again, potentially resulting in unnecessary exploration steps.

#### Memory.

The Memory records observable information during task execution, including action trajectories and inventory changes at each step. It plays a crucial role in tracking long-term dependencies and facilitates robust task execution.

Figure: Figure 3.5: The perception module utilizes MLLMs to convert the Minecraft game images into text description. This module captures first-person game screenshots of the agent between steps, and provides them to the MLLM for image description.
Refer to caption: x88.png

### 3.6 Perception Module

The perception module (Fig. 3.5) utilizes MLLMs for environmental observation, enabling Adam to perceive the world without relying on metadata such as the names of surrounding blocks, GPS coordinates, or the biome names, which are typically invisible to human players. This module captures first-person screenshots between steps, which are processed by MLLMs to generate descriptions. This text description is subsequently passed to the Actor in the controller module for action choosing.

## 4 Experiments

### 4.1 Experimental Setup

In our study, we employ Mineflayer (PrismarineJS, ), a JavaScript-based framework providing control APIs for the commercial Minecraft (version 1.19) (^*^**https://www.minecraft.net). The encapsulation of Mineflayer uses the implementation in VOYAGER (Wang et al., ).
For visual processing, we utilize prismarine-viewer (PrismarineJS, ), an API for rendering game scenes from the agent’s perspective.
Adam and our baselines all use GPT-4-turbo (gpt-4-0125-preview) (Achiam et al., ) for LLM inference, with the temperature set to 0.3 based on our experiments in Appendix B.
For visual description, we utilize LLaVA-v1.5-13B (Liu et al., ) in our perception module.

### 4.2 Baselines

In the absence of directly comparable work, we select representative methods as baselines and focus on their comparable aspects, including:
(1) ReAct (Yao et al., ), which explicitly expresses the thought processes through chain-of-thought prompting (Wei et al., ).
(2) Reflexion (Shinn et al., ), derived from ReAct, which can reflect on its exploration history.
(3) AutoGPT (), which can autonomously decompose tasks and execute subtasks in a ReAct-style.
Baselines 1–3 can only perform text-based tasks and lack embodied components to interact with the environment. Hereafter, they are referred to as non-embodied agents, and we have adapted them with our interaction module for embodied exploration.
(4) VOYAGER (Wang et al., ) is an LLM-based embodied lifelong learning agent in Minecraft, featuring an automatic curriculum aiming to “discover as many diverse things as possible”. We also add our benchmarking tasks (*e.g.*, obtaining diamonds ) to the curriculum for oriented explorations, denoted as VOYAGER-Guided.
(5) CDHRL (Peng et al., ) introduces an RL agent that constructs hierarchical structures based on causal relationships. Given that RL agents have disparate action spaces and magnitudes of difference in episode length compared to LLM-based methods ($10^{5}\sim 10^{8}$ in DreamerV3 (Hafner et al., ) and DEPS (Wang et al., ) versus $10^{1}\sim 10^{2}$ in VOYAGER and our Adam), we focus our comparison on the CD component of CDHRL. Detailed discussion of other Minecraft agents that are not directly comparable can be found in Appendix C.

**Table 4.1: Structural Hamming Distance (SHD) between the learned causal graph and the target graph. For non-embodied agents without built-in interventions, even with our TM and SD (Section 3.4), the causal graph learned by these agents remains suboptimal. The CD used by CDHRL is based on SDI (Ke et al., ), which incorporates temporal modeling (TM) into its implementation, and still exhibits over 30% errors or omission, while Adam can identify a nearly perfect causal graph.**
| Model | SHD | Model | SHD |
| --- | --- | --- | --- |
| Adam | $\mathbf{2\pm 2}$ | CDHRL | $10\pm 4$ |
| Adam w/o TM,SD | $19\pm 6$ | CDHRL w/ SD | $6\pm 2$ |
| Reflexion | $24\pm 9$ | React w/ TM,SD | $5\pm 2$ |
| AutoGPT | $24\pm 6$ | Reflexion w/ TM,SD | $4\pm 2$ |
| Empty Graph | $32$ | AutoGPT w/ TM,SD | $4\pm 2$ |

**Table 4.2: Exploration steps in different tasks. Fewer steps indicates higher efficiency. Each method has three trials for a maximum length of 200 steps. The success rate is depicted in the parentheses. Adam achieves a 2.2$\times$ speedup compared to the SOTA in the task of obtaining diamonds, with a higher success rate.**
| Framework | Wooden Tool | Stone Tool | Iron Tool | Diamond |
| --- | --- | --- | --- | --- |
| React w/ TM w/ SD | $51\pm 19(2/3)$ | $96(1/3)$ | N/A $(0/3)$ | N/A $(0/3)$ |
| Reflexion w/ TM w/ SD | $60\pm 27(3/3)$ | $122\pm 56(2/3)$ | N/A $(0/3)$ | N/A $(0/3)$ |
| AutoGPT w/ TM w/ SD | $49\pm 20(2/3)$ | $103\pm 45(2/3)$ | N/A $(0/3)$ | N/A $(0/3)$ |
| VOYAGER | $8\pm 2(3/3)$ | $\mathbf{10\pm 3(3/3)}$ | $27\pm 11(3/3)$ | $113\pm 41(2/3)$ |
| VOYAGER Guided | $\mathbf{7\pm 2(3/3)}$ | $11\pm 2(3/3)$ | $\mathbf{24\pm 9(3/3)}$ | $75\pm 20(2/3)$ |
| Adam | $23\pm 5(3/3)$ | $33\pm 8(3/3)$ | $53\pm 16(3/3)$ | $68\pm 21(3/3)$ |
| Adam Parallel | $12\pm 2(3/3)$ | $18\pm 3(3/3)$ | $29\pm 5(3/3)$ | $\mathbf{34\pm 7(3/3)}$ |

### 4.3 Main Results

#### Interpretability.

We evaluate the interpretability of agents by assessing their ability to construct a causal graph. Structure Hamming Distance (SHD) (Zheng et al., ) can quantify the discrepancy between the learned causal graph and the target graph as presented in Tab. 4.1. Despite applying our TM and SD optimization techniques (Section 3.4), non-embodied agents without built-in interventions fail to achieve the optimal accuracy. For CDHRL, we directly provide Adam’s sampling data for its CD. CDHRL performs CD with all nodes in the causal graph, which hampers its performance as CDHRL shows improved performance when integrated with our SD optimization. Nevertheless, these competitive methods exhibit at least 30% errors or omissions, whilst Adam is capable of learning a nearly perfect causal graph. VOYAGER does not organize knowledge in a causal graph.

#### Efficiency.

Efficiency is evaluated in the original Minecraft as shown in Tab. 4.2. Adam achieves a 2.2$\times$ speedup compared to the SOTA in the task of obtaining diamonds . Its design facilitates parallel sampling (Section 3.3), which significantly boosts exploration efficiency. While VOYAGER leverages prior knowledge from LLMs to excel in simple tasks, Adam independently discovers world knowledge from scratch, outperforming VOYAGER in complex tasks.
Due to the absence of intervention-based CD, non-embodied agents are unable to refine their causal assumptions, limiting their exploration speed and confining them to the lower levels of the technology tree.
Additionally, Adam achieves higher success rate across most tasks compared to other methods.

Figure: Figure 4.1: The causal graph learned in lifelong learning. Adam successfully unlocks all 41 actions we implement and discovers accurate causal relationships.
Refer to caption: x91.png

**Table 4.3: Performance of Adam and VOYAGER in tasks requiring environmental factors $\mathcal{E}$. Each method has three trials for a maximum length of 100 steps. The success rate is depicted in the parentheses. VOYAGER’s performance significantly declines when metadata is not accessible, whereas Adam do not rely on metadata. MLLM contributes Adam’s performance in this type of tasks.**
|  | VOYAGER | VOYAGER w/o Meta | Adam | Adam w/o MLLM |
| --- | --- | --- | --- | --- |
| Find a river | $\mathbf{16\pm 8(2/3)}$ | N/A $(0/3)$ | $21\pm 16\mathbf{(2/3)}$ | N/A $(0/3)$ |
| Gather log near river | $\mathbf{36}(1/3)$ | N/A $(0/3)$ | $40\pm 23\mathbf{(2/3)}$ | N/A $(0/3)$ |
| Smelting iron near grass | N/A $(0/3)$ | N/A $(0/3)$ | $\mathbf{95(1/3)}$ | N/A $(0/3)$ |

#### Robustness.

We assess the robustness of agents in a modified Minecraft environment where crafting recipes are altered. The result is shown in Fig. 1.1c. In this scenario, there exists a misalignment between the LLM’s prior knowledge and the actual game dynamics. Adam successfully obtains diamonds in the modified Minecraft.
The baselines lack a CD approach to learn and verify causal knowledge,
and struggle with complex dependencies. The most advanced item that baseline agents manage to acquire is raw_iron . In terms of exploration speed, Adam achieves a 4.6$\times$ speedup in obtaining raw_iron . For more details, please refer to Tab. E.1 in the Appendix.

#### Lifelong learning.

Adam utilizes CD methods to learn the effects of each action, obtaining causal subgraphs that include new items. These new items make it possible to perform more unknown actions, thereby continually expanding the knowledge of the game world in a bootstrapping manner and achieves lifelong learning. Adam successfully learns a complex causal graph of all 41 actions we implement, as demonstrated in Fig. 4.1.

#### Human gameplay alignment.

Avoiding the use of human-invisible metadata demonstrates Adam’s alignment with human gameplay. We compare VOYAGER, a fully text-based agent that relies on metadata(^†^††The information used by VOYAGER and Adam is compared in Tab. C.1 in the Appendix.). We test three tasks that requires environmental factors $\mathcal{E}$. The results are shown in Tab. 4.3. VOYAGER’s performance significantly declines when metadata is not accessible. Adam performs well on tasks with $\mathcal{E}$ relying solely on observable information. Ablation experiments demonstrate that MLLMs contribute to Adam’s performance on this type of tasks.

### 4.4 Ablation Studies

**Table 4.4: Ablation of LLM-based CD. $\dagger$ means “in a modified environment”. We record the average number of redundant/error items (represented as R&E) in the causal assumption proposed by LLM-based CD, and the success rate after the intervention-based CD. LLMs with only strong prior knowledge but weak inference abilities cannot perform well.**
| Model | R&E (success/all) | Model | R&E (success/all) |
| --- | --- | --- | --- |
| gpt-4-turbo-preview | 0.0 (35/35) | gpt-4 | 0.1 (35/35) |
| gpt-4-turbo-preview^$\dagger$ | 0.1 (35/35) | gpt-4^$\dagger$ | 0.1 (35/35) |
| gpt-3.5-turbo | 0.2 (34/35) | Llama-2-70B | 0.6 (23/35) |
| gpt-3.5-turbo^$\dagger$ | 0.3 (32/35) | Llama-2-70B^$\dagger$ | 0.9 (16/35) |
| Llama-2-70B-finetuned | 0.4 (27/35) | Llama-2-13B-finetuned | 1.8 (5/35) |
| Llama-2-70B-finetuned^$\dagger$ | 0.9 (15/35) | Llama-2-13B-finetuned^$\dagger$ | N/A (0/35) |

Figure: Figure 4.2: Average number of steps and success rate to learn the causal subgraphs of “Collecting” actions (*e.g.*, gatherIronOre), which are representative due to higher noise in their sampling data. We perform up to 20 steps for each action, and if the CD fails, it is counted as 20. Intervention contributes up to 4.4$\times$ acceleration and higher success rate in the exploration.
Refer to caption: x95.png

#### Ablation of LLM-based CD

Prior knowledge and inference capabilities are key factors in ablating LLM-based CD. Prior knowledge can be ablated in modified environments and enhanced through fine-tuning LLMs on MC-QA dataset we constructed(^‡^‡‡For the details of the MC-QA dataset, please refer to Appendix A.). Reasoning capabilities can be ablated by replacing SOTA LLMs with smaller LLMs. Our result in Tab. 4.4 shows that LLMs with strong reasoning abilities but limited prior knowledge (*e.g.*, gpt-4-turbo-preview in modified environments) perform well, while LLMs with extensive prior knowledge but weak reasoning abilities (*e.g.*, fine-tuned Llama-2-13B) perform suboptimally. This demonstrates that Adam primarily utilizes the reasoning abilities of LLMs rather than relying on prior knowledge.

#### Ablation of intervention-based CD.

Without intervention-based CD, agents are forced to rely on exhaustive trials to learn the game knowledge, significantly impairing their efficiency and effectiveness. Our experimental results are shown in Fig. 4.2. Through interventions, Adam achieves up to 4.4 $\times$ speedup and a higher accuracy compared to the ablated group.

## 5 Related Work

#### Causality in Agent.

The integration of causality (Pearl, ; Peters et al., ; Schölkopf, ) into agents is primarily aimed at enhancing the learning efficiency (Méndez-Molina et al., ; Seitzer et al., ; Gasse et al., ; Sun et al., ; Peng et al., ). Peng et al. () propose CDHRL to build high-quality hierarchical structures in complicated environments. Méndez-Molina et al. () employ the causal models to restrict the search space.
Zeng et al. () distinguish agents with causality in two categories: ones relying on prior causal information and ones learn causality by causal discovery algorithms (Spirtes et al., ; Sun et al., ; Zhang & Hyvärinen, ; Zhang et al., ; Peters et al., ; Zhu et al., ). Our work aligns with the latter category and extends to a wider range of scenarios where prior knowledge is unknown.

#### LLM/MLLM-Based Agent.

Leveraging the generalization capabilities of LLM (Brown et al., ; Touvron et al., , ) to empower agent systems with tools (Qin et al., ; Schick et al., ; Shen et al., ) is an essential task (Xi et al., ; Wang et al., ).
Schick et al. () design a framework to allow LLM to use external APIs to complete tasks. Qin et al. () build related benchmarks to evaluate performance in such tasks. In tasks involving interaction with the environment, Shinn et al. () enhance the agent’s ability through language feedback signals. Wei et al. () employ Chain-of-Thought prompting method to optimize the reasoning capabilities of the LLM agent. However, the hallucination and interpretability challenges (Zhang et al., ) of LLMs also accompany these systems. In this work, we prove that the perspective of causal architecture can reduce the reliance on priors and enhance the robustness of inferences.

## 6 Conclusion

In this work, we introduce Adam, an embodied causal agent in open-world environments. Adam innovatively incorporates CD with embodied exploration, significantly improving the accuracy of CD while enhancing the efficiency and interpretability of embodied exploration. Not relying on prior knowledge, Adam demonstrates strong robustness, and its multimodal perception aligns with human behavior. Our work sets a foundation for developing autonomous agents that can understand and manipulate environments in a causal manner.

## Appendix A MC-QA dataset

Minecraft is an open-world game with a high degree of freedom. Players can freely gather materials (*e.g.*, log ), mine ore (*e.g.*, iron_ore ), craft tools (*e.g.*, wooden_pickaxe ), and more.
Crafting recipes (*e.g.*, planks + sticks $\to$ wood_pickaxe ) are the main knowledge in the Minecraft game, and serve as the basis for players to climb the technology tree and obtain more advanced items.

Given this, LLMs’ mastery of crafting recipes can reflect the strength of their prior knowledge in the Minecraft game. We utilize the crafting recipes in Minecraft (version 1.19) to create the MC-QA dataset. An example of the QA pairs in the dataset is shown in Fig. .1. The questions in this dataset ask for the crafting ingredients required to obtain higher-level items in the technology tree, and the answers are the ingredient items. LLMs need to give their answers in the specified format. The order of the items in the answer is not required. For each question, we provide 3 examples to help LLMs understand the QA task and the format of the answers. For situations where there are multiple ways to craft the same item, we take them all into account to avoid the model being biased toward a fixed understanding of the game. The dataset contains 754 QA pairs on the knowledge of obtaining items in the Minecraft.

## Appendix B LLM’s prior on Minecraft.

We utilize the Minecraft crafting recipes to construct an MC-QA dataset (introduced in Appendix A), aiming to evaluate the prior knowledge of various LLMs on Minecraft.
We test and determine that $0.3$ is the optimal temperature as shown in Fig. B.1a. Then we test various LLMs on this dataset. The GPT series (Ouyang et al., ; Achiam et al., ) show significantly stronger Minecraft prior knowledge than other LLMs as shown in Fig. B.1b.

Utilizing this dataset to fine-tune LLMs can improve their prior knowledge on Minecraft as shown in Fig. B.1c. On the other hand, by modifying the crafting recipes in Minecraft, we can make the SOTA LLMs (*e.g.*, GPT series) have no prior of this modified environment. This setup enables us to distinctively analyze the roles of prior knowledge and inference capability as shown in Fig. B.1d, which serves as the basis of our ablation study.

Figure: Figure B.1: (a) LLaMA2 (Touvron et al., ) demonstrates optimal accuracy in answering crafting recipes at a temperature of 0.3, measured as the ratio of correct answers to total questions. (b) Performance of open-source LLMs and GPT series models, showcasing their inherent prior knowledge of Minecraft. (c) Illustration of the improvement in performance for open-source LLMs fine-tuned with the crafting recipe dataset. (d) Categorizing the LLMs into four types based on their prior knowledge of Minecraft recipes and inference capabilities.
Refer to caption: x103.png

## Appendix C Agent in Minecraft

RL explorations in Minecraft agents focus on the efficient use of data (Baker et al., ; Fan et al., ), hierarchical RL design (Lin et al., ; Mao et al., ), innovative architecture modeling (Hafner et al., ), *etc*. Hafner et al. () use world models to achieve a general and scalable RL without human data or curricula. Much work (Guss et al., , ; Kanervisto et al., ; Fan et al., ) has made different simplifications for the Minecraft environment to facilitate the RL agent systems. The MineDojo framework (Fan et al., ) provides an internet-scale knowledge database and game environments for CLIP model (Radford et al., ) and RL training. These efforts provide efficient optimization for agent sampling and interaction, but there is still a gap to commercial Minecraft games with complete game features like in VOYAGER (Wang et al., ) and our work.

The reasoning capabilities and rich prior knowledge of LLMs have contributed to much work on Minecraft agents (Yuan et al., ; Zhu et al., ; Wang et al., ; Qin et al., ; Nottingham et al., ; Wang et al., , ). VOYAGER (Wang et al., ) and GITM (Zhu et al., ) use LLMs’ prior knowledge of Minecraft and environment feedback to complete exploration tasks in a text-based manner. Qin et al. () leverage MLLMs to introduce visual information as the contextual basis for action execution. These methods more or less rely on prior knowledge of Minecraft. Our Adam shows effectiveness even when the game rules are modified.

There are also Minecraft agents that integrate LLMs with RL, including Plan4MC (Yuan et al., ), DEPS (Wang et al., ), and JARVIS-1 (Wang et al., ). These methods operate in non-commercial Minecraft environments and utilize frame-level control, with approximately $10^{4}$ steps for a task, in contrast to the action-level control in our system, which involves around $10^{2}$ steps for a task. Furthermore, these RL methods require training stages, whereas our system does not involve weight updates. Notably, JARVIS-1 incorporates crafting recipes as an integral part of the system, utilizing prior knowledge rather than learning from scratch.

In the original implementation of VOYAGER, the environment feedback provides detailed information such as crafting recipe errors (*e.g.*, "I cannot make an iron chestplate because I need: 7 more iron ingots."). We retain this informative feedback in all our experiments with VOYAGER, even in environments with modified crafting recipes where the feedback may not align with the changes.

Tab. C.1 shows the environmental information used by Adam and VOYAGER. This setup is used in our experiments at Sec. 4.3. VOYAGER does not have a visual input and needs omniscient metadata (Meta) which is not explicitly exposed to human players, while Adam utilizes visual input and other observable information to make decisions.

**Table C.1: Comparison of environmental information used by Adam and VOYAGER. VOYAGER does not have a visual input and needs to directly read the game information (Meta) which is not explicitly exposed to human players, while Adam relies on the visual input and the information observable by human players to make decisions.**
|  | VOYAGER | VOYAGER w/o Meta | Adam | Adam w/o MLLM |
| --- | --- | --- | --- | --- |
| Observation Space | Environment Feedback,<br>Inventory,<br>Meta | Environment Feedback,<br>Inventory | Pixels,<br>Inventory | Inventory |
| Action Space | Code | Code | Discrete | Discrete |

## Appendix D Implementation Details

### D.1 Prompt

The prompt for LLM-based CD is shown in Fig. D.1, which is composed of 5 components: (1) Role Playing, which assigns a specific role to the LLM; (2) Problem Setting, which provides specific details of the inference task; (3) Letter Mapping, which involves mapping item names to letters, a simplification that facilitates the formatted output; (4) Few-shot Prompting, which involves providing the LLM with several inference examples in chain-of-thought (Wei et al., ) style; (5) Data $D_{a}$, which is the data collected by the interaction module for inference.

Figure: Figure D.1: The prompt for LLM-based CD. The contents in red will be replaced in the inference process.
Refer to caption: x104.png

### D.2 Action Space and Movement Space

We have implemented 41 discrete actions and 6 movements to ensure the agent can freely explore the Minecraft world through a diverse range of combinations. The actions can be divided into three categories: “Smelting”, “Collecting”, and "Crafting". "Smelting" actions have complex causal subgraphs, often leading to omissions in LLM-based CD. "Collecting" actions have noisy sampling data, and the results of the LLM-based CD are often redundant. The "Crafting" actions have complex causal subgraphs and clean sampling data.

**Table D.1: Three action types in our experiment setting. "Smelting" actions have complex causal subgraphs, often leading to omissions in LLM-based CD. "Collecting" actions have noisy sampling data, and the results of the LLM-based CD are often redundant. The "Crafting" actions have complex causal subgraphs and clean sampling data.**
| Action Type | Action-Item Dependency | Data Quality | Skill Level |
| --- | --- | --- | --- |
| Collecting | Simple | Noise | Low |
| Crafting | Complex | Clean | High |
| Smelting | Complex | Noise | High |

The movement space corresponds to the low-level movement control visible to human players, including moving forward / backward, lowering / raising the agent’s coordinates and turning left / right.

## Appendix E Robustness

Tab. E.1 shows our experiment result in the modified Minecraft environment where the crafting recipes are altered. Adam can maintain its performance as it is equipped with CD methods, whereas agents that rely on prior knowledge struggle to explore efficiently. The result demonstrates the robustness and generalization capabilities of our Adam architecture.

**Table E.1: Performance in the modified Minecraft game. Each method has three trials for a maximum length of 200 steps. The success rate is depicted in the parentheses. Adam can maintain its performance as it is equipped with CD methods, whereas agents that rely on prior knowledge struggle to explore efficiently. The result demonstrates the robustness and generalization capabilities of our Adam architecture.**
| Framework | Wooden Tool | Stone Tool | Iron Tool | Diamond |
| --- | --- | --- | --- | --- |
| React w/ TM w/ SD | $91\pm 34(2/3)$ | $139(1/3)$ | N/A $(0/3)$ | N/A $(0/3)$ |
| Reflexion w/ TM w/ SD | $76\pm 28(2/3)$ | $120\pm 40(2/3)$ | N/A $(0/3)$ | N/A $(0/3)$ |
| AutoGPT w/ TM w/ SD | $82\pm 25(2/3)$ | $124(1/3)$ | N/A $(0/3)$ | N/A $(0/3)$ |
| VOYAGER | $95\pm 33(2/3)$ | $152\pm 43(2/3)$ | N/A $(0/3)$ | N/A $(0/3)$ |
| VOYAGER Guided | $108\pm 35(2/3)$ | $176(1/3)$ | N/A $(0/3)$ | N/A $(0/3)$ |
| Adam | $28\pm 4(3/3)$ | $52\pm 14(3/3)$ | $94\pm 27(3/3)$ | $109\pm 34(2/3)$ |
| Adam Parallel | $\mathbf{15\pm 2(3/3)}$ | $\mathbf{31\pm 7(3/3)}$ | $\mathbf{54\pm 14(3/3)}$ | $\mathbf{61\pm 18(2/3)}$ |

## Appendix F Generalization

The Adam architecture is a general framework for embodied agents operating in various open-world environments including Minecraft. When adapting Adam to other application scenarios, some modifications may be necessary:

- (1)
The world knowledge in Minecraft is the dependence between items and actions. Consequently, in this paper, items and actions are designed as causal graph nodes. When migrating to other environments, key elements related to the agent’s task objectives can be similarly designed as causal graph nodes.
- (2)
In Adam, the perception module utilizes a vision-based MLLM and does not rely on omniscient metadata. This allows the module to adapt well to other visual tasks. If conditions permit (*e.g.*, a robot equipped with LiDAR), the perception module can provide more precise information, potentially further improving performance.
- (3)
To model actions as a finite set of causal graph nodes, it is necessary to discretize the continuous action space. The granularity of this discretization should be determined based on the specific environment.

## Peer Reviews

confidence: 5
contribution: 2
initial_rating: 5
presentation: 3
questions: See the weakness.

According to the author's reply, they refuse to admit the unfair comparisons they made during the rebuttal stage. I even doubt whether the author carefully reviewed the responses from all reviewers. Therefore, I have decided to change the score to strong reject. 

I strongly recommend that the author carefully review the reviewer's comments and provide a serious response.
soundness: 2
strengths: 1.	The figures in the paper are well-done and enhance clarity, making the content easier to understand.
2.	The experiments conducted in Minecraft show a higher success rate than those achieved by Voyager.
summary: This paper introduces ADAM, an autonomous agent for open-world environments like Minecraft that builds a causal graph from scratch to improve interpretability and performance without relying heavily on pretrained knowledge. Through a combination of interaction, causal reasoning, planning, and multimodal perception, ADAM outperforms existing agents in task success and adaptability, even in modified game settings. ADAM’s approach establishes a new standard for causal reasoning in embodied agents.
weaknesses: 1.	The paper appears to be hastily prepared, as it contains numerous typos and minor errors, such as inconsistencies between “Fig.” and “Figure” references and improper usage of quotation marks in Table 4’s caption. I recommend that the authors carefully review and correct these issues.
2.	The major issue lies in the extensive use of pretrained language models that already incorporate substantial knowledge of Minecraft. Since language models may internally form a comprehensive causal graph of the game world, primarily in linguistic form, the proposed additional causal graph construction might be redundant. I suggest that the authors explore scenarios with completely altered world rules in Minecraft to test the validity of models like GPT in such modified environments, perhaps using a setting like “Mars.” Alternatively, they could consider using a language model entirely devoid of Minecraft knowledge, though this may be challenging to achieve.
3.	The agent’s modular design is nearly identical to Voyager, with the primary addition being the causal graph. Experimentally, however, it does not show significant advantages over Voyager, as it does not complete tasks that Voyager was unable to.
4.	The causal graph generated by the model ADAM is quite similar to the hybrid knowledge graph in memory described in [1]. The authors should clarify the differences.
5.	Additionally, the current causal graph is entirely object-centric. In open-world Minecraft, there are many open-ended tasks, such as building and farming, which are not strictly object-centric. This limitation restricts ADAM’s generalization capability in open-ended tasks.
6.	Several relevant works are not cited, including:

	[1] Optimus-1: Hybrid Multimodal Memory Empowered Agents Excel in Long-Horizon Tasks
 
	[2] Mars: Situated Inductive Reasoning in an Open-World Environment, NeurIPS 2024
 
	[3] OmniJARVIS: Unified Vision-Language-Action Tokenization Enables Open-World Instruction Following Agents, NeurIPS 2024

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 1
questions: - Would the method generalize to other environments? If so, what are the assumptions and requirements for the application of the method to a new environment?
- How does the method compare to approaches trained with reinforcement learning? Is it possible to train an agent with reinforcement learning on the same action space that ADAM uses?
- What is the captioning performance of the perception module? What are its failure cases?
soundness: 2
strengths: - The method seems to be the first approach that combines casual inference with LLM agents in code-based action spaces, which is a potentially very important direction for future research.
- The method performs quite well for inferring causal graphs on Minecraft, and it seems to provide a way for agents to take advantage of those causal graphs.
summary: The paper introduces an architecture for agents that play the game of Minecraft, based on combining Large Language Models with causal inference. Featuring different modules (e.g., planner, actor, perception), the method is based on inferring causal graphs related to the various crafting dependencies of Minecraft's technology tree, and on enabling an agent to use the knowledge about these dependencies for progressing in the game. The method is evaluated against similar LLM-based methods (albeit using a different action space).
weaknesses: - The presentation of the method is quite high-level and does not help the reader understand how the method actually works in practice. When the different "modules" are introduced, it is not clear a priori what they actually are. Are they just prompts and specifications to a GPT4 model? If so, it could be beneficial to show one of the prompts earlier, to guide the understanding of the rest of the paper.
- The comparisons in the paper are unclear, due to the choice of a specific action space that is different from the one used in previous work. Indeed, while the paper mostly discusses the "observation space" difference compared to the setting usually employed in reinforcement learning papers, one crucial difference is the one in action space. For instance, Voyager works in an extremely more high-level action space compared to DreamerV3. This is not accurately depicted in the current version of the paper.
- It could be surprising to observe that an off-the-shelf open model is accurately describe an observation to the level of providing enough information for an actor to take the optimal action, especially in an environment that is as visually rich as Minecraft. An ad-hoc evaluation of this specific capability would strengthen the paper.
- The method seems to be highly Minecraft-specific, and the paper does not extensively discuss how it could be generalized to other domains.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: See previous
soundness: 3
strengths: 1. The incorporation of causal discovery methods in a modular framework is a novel in LLM-based embodied exploration, and it does not rely on privileged information unlike prior work
2. The paper demonstrate strong empirical results with well-designed experiments, that led to significantly faster discovery of skills in Minecraft. Performance in modified environments where prior knowledge is invalid did not degrade performance too much demonstrates causal learning is indeed effective. Method also does not require meta-data.
3. The experiments are solid with multiple baselines and includes comprehensive ablation studies as well as detailed analysis of failure cases
summary: This paper introduces ADAM (An emboDied causal Agent in Minecraft) - an agent architecture that autonomously explores, learns causal world knowledge, and executes complex tasks in Minecraft from multimodal inputs. The system consists of four main components: interaction module, causal model module, controller module, and perception module. The interaction module samples actions and records observations. The causal model module infers causal relationships and constructs causal subgraphs for each action.
The key innovation is integrating causal discovery methods with embodied exploration, enabling the agent to learn accurate causal relations from scratch without relying on prior knowledge.
weaknesses: 1. One concern is whether ADAM scales with more complex world and causal graph for intervention-based causal discovery (CD).
2. Interestingly the paper proposes a multimodal agentic framework but all the baselines compared to are text-based frameworks. It would be good to have at least one multi-modal baseline, e.g. [1] as this is also cited by the authors.

[1] Wang, Z., Cai, S., Liu, A., Jin, Y., Hou, J., Zhang, B., ... & Liang, Y. (2023). Jarvis-1: Open-world multi-task agents with memory-augmented multimodal language models. arXiv preprint arXiv:2311.05997.

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 3
questions: - ad interpretability claim: how does the interpretability of the presented method relate to the interpretability of baseline methods, e.g., voyager? isn't the interpretability of the method compromised by the lack of interpretability of the used LLMs?
- the proposed method is quite complex as it consists of 4 modules consisting of several submodules each and the presented ablation studies seem insufficient to rigorously justify such a complicated system; so how was the method designed? how much inspiration was drawn from prior work? 
- Adam is claimed to be a "generalizable framework"; why not back this claim with a complementary application in another environment? 
- what are possible limitations of the method? (the paper does not mention any)
- what do the error bars in tables 1, 2, 3 mean?
soundness: 1
strengths: - the paper is well structured and clearly written
- the combination of LLM-prompting with CD seems to be quite novel
- the intervention-based refinement of the causal graph seems meaningful and practical
summary: The paper proposes a Minecraft agent called Adam that relies on a combination of (M)LLM inference and causal discovery. At the core of the method is a causal graph that represents the agent's expertise on the environment logic. The causal graph is proposed by an LLM and each relation in the graph is confirmed/disproved by environment interactions. The proposed method obtains diamonds faster and more reliably than prior methods. Moreover,
weaknesses: - unsupported claims: the authors claim their method has "excellent interpretability" and that their agent "closely aligns with human gameplay;" yet, I cannot find any empirical evidence supporting these claims. 
- a runtime/memory analysis (be it theoretical or empirical) is completely missing, i.e., it is not clear at which cost the claimed SOTA results come
- the results presented in figure 1 are based on a modified causal graph claiming that this removes prior knowledge from LLMs; yet, it is unclear in how far this claim is true; it would be interesting to see an analysis akin to appendix A for the modified environment
- the choice of the acronym Adam is at best unfortunate as it coincides with one of the most influential machine learning papers (https://arxiv.org/abs/1412.6980) and could be mistaken for an attempt to tap unwitting citations as both paper titles start with "Adam: ..."
- insufficient reproducibility due to missing source code

Meta-review: (a) The paper introduces ADAM, an embodied agent that learns causal relationships in the open-world environment of Minecraft. The key finding is that by combining causal discovery methods with LLM-powered embodied exploration, the agent can learn accurate causal graphs without relying on prior knowledge or metadata. The authors claim this leads to robust performance, even in modified environments where pre-trained knowledge about crafting recipes is invalid. The paper demonstrates this through experiments in both original and modified Minecraft environments, showcasing the agent's ability to efficiently learn a causal graph of item dependencies and use it to solve tasks like obtaining diamonds.

(b) Strengths:
- The paper introduces a novel approach by integrating causal discovery with embodied exploration using LLMs in Minecraft. 
- The agent's ability to perform well in a modified environment with altered crafting recipes demonstrates the robustness of the causal learning approach and its independence from pre-trained knowledge.
- The experiments provide strong evidence for the agent's efficiency and effectiveness compared to baseline methods.
- The use of a causal graph to represent the agent's learned knowledge enhances interpretability and provides insights into the decision-making process, addressing a common limitation of black-box LLM-based agents.

(c) Weaknesses:
- While the paper proposes a generalizable framework, the experiments are limited to Minecraft. Providing a complementary application in a different environment would strengthen the generalizability claim.
- The paper compares ADAM to text-based baselines, but lacks comparisons to existing multimodal agents, which would provide a more comprehensive evaluation of its performance. Even though in the case that there doesn't exist a comparable baseline, it would have make the paper stronger to create one.

(d) The decision is to accept the paper. Despite some limitations, the paper's novel combination of causal inference with LLM-powered embodied exploration, its strong empirical results demonstrating robust performance, and its contribution to the interpretability of LLM-based agents make it a valuable contribution to the field.

Decision: Accept (Poster)