WALL-E: World Alignment by NeuroSymbolic Learning improves World Model-based LLM Agents
Authors: Siyu Zhou
Tianyi Zhou
Yijun Yang
Guodong Long
Deheng Ye
Jing Jiang
Chengqi Zhang
Venue: NeurIPS 2025 poster ()
Keywords: world model
embodied agent
large language model
neurosymbolic learning
Primary area: deep_learning

Abstract: Can we build accurate world models out of large language models (LLMs)? How can world models benefit LLM agents? The gap between the prior knowledge of LLMs and the specified environment's dynamics usually bottlenecks LLMs' performance as world models. To bridge the gap, we propose a training-free "world alignment" that learns an environment's symbolic knowledge complementary to LLMs. The symbolic knowledge covers action rules, knowledge graphs, and scene graphs, which are extracted by LLMs from exploration trajectories and encoded into executable codes to regulate LLM agents' policies. We further propose an RL-free, model-based agent "WALL-E" through the model-predictive control (MPC) framework. Unlike classical MPC requiring costly optimization on the fly, we adopt an LLM agent as an efficient look-ahead optimizer of future steps' actions by interacting with the neurosymbolic world model. While the LLM agent's strong heuristics make it an efficient planner in MPC, the quality of its planned actions is also secured by the accurate predictions of the aligned world model. They together considerably improve learning efficiency in a new environment. On open-world challenges in Mars (Minecraft like) and ALFWorld (embodied indoor environments), WALL-E significantly outperforms existing methods, e.g., surpassing baselines in Mars by 16.1%–51.6% of success rate and by at least 61.7% in score. In ALFWorld, it achieves a new record 98% success rate after only 4 iterations.

WALL-E: World Alignment by NeuroSymbolic Learning improves World Model-based LLM Agents
Siyu Zhou', Tianyi Zhou², Yijun Yang', Guodong Long', Deheng Ye Jing Jiang', Chengqi Zhang Australian Al Institute, University of Technology Sydney
University of Maryland, College Park Tencent "onng Kong Polytechnic University Siyu 2astuu-n uts. edu. au, zhou@umiaco acs. umd.ede
Target Item:
Diamand
Neurosymbolic Learning
Success
Succes
T. 7.l diamoed with Loon pickase
End
"abject" beingg wined, the aoti fall,
moouladgge Sraph:
Hine plctase
ad
Scane Braph
Predicted Faluur
with aSona plotaan
Model Model-Predicte Predictive Control
Success
LLM Agent Planted actions
Mine stace MATh
claks
Symbolic Roceledge:
sppeopetate
'stoow pickam
Model-Predictivv Control
LLM Agent
Planced actiota:
LLM World Model
2
5. s.crett an 1ron
Sondlat
Fondration: esccess
Falas,
Kine stone with
Penniction
platase
plokase;
Dewcket
1. L.sccoss Zcus,
iron plckaas
Start
3. Mine Scon withh
wooden pickass:
Figure 1: WALL-E mining diamond on Mars. Step 1-2: The agent makes decisions via MPC with the initial unaligned world model, resulting in failed action for mining iron. Step 3: Leveraging previous trajectories and world model predictions, WALL-E learns symbolic knowledge, including action rules, knowledge graphs, and scene graphs. Step 4-5: The learned symbolic knowledge helps the world model make accurate predictions and correct the previous mistake. Step 6-7: The agent ad- justs its decision accordingly and replaces stone pickaxe with iron pickaxe toward completing the task.
Abstract
Can we build accurate world models out large language models (LLMs)? How can world models benefit LLM agents? The gap between the prior knowledge of LLMs and the specified environment's dynamics usually bottlenecks LLMs performance as world models. To bridge the gap, we propose a training- free "world alignment" that learns an environment's symbolic knowledge complementary to LLMs. The symbolic knowledge covers action rules, knowledge graphs, and scene graphs, which are extracted by LLMSs from exploration trajectories and encoded into executable codes to regulate LLM agents' policies. We further propose an RL-free, model-based agent "WALL-E" through the model-predictive control (MPC) framework. Unlike classical MPC requiring costly optimization on the fly, we adopt an LLM agent as an efficient look-ahead optimizer of future steps' actions by interacting with the neurosymbolic world model. While the LLM agent's strong heuristics make it an efficient planner MPC, the quality of its planned actions is also secured by the accurate predictions of the aligned
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

world model. They together considerably improve learning efficiency in a new environment. On open-world challenges in Mars (Minecraft like) and ALFWorld (embodied indoor environments), WALL-- significantly outperforms existing methods, e.g., surpassing baselines Mars by 16.1%-5. by at least 61.7% in score. In ALFWorld, it achieves a new record 98% success rate after only iterations. Our code is available here
of success rate and
1 Introduction
While large language models (LLMs) have been suc- cessfully applied complex reasoning, generation, and planning tasks, they are not sufficiently reliable to be deployed as an agent in specific open-wont world environments, e.g., games, VR/AR systems, medical care, education, autonomous driving, etc [ primary reason for the failures is the gap between the prior knowledge driven commonsense reasoning by LLMs and the specified environment's dynamics. The gap leads to incorrect predictions of future states, hallucinations, agents' decision-makin makinn process 4 Although the alignment of LLMs with human preferences has been widely studied as a major objective of LLM post- training. "world alignment" with environment's s dynamics has not been adequately investigated for LLM agents (H10). Moreover, many existing LLM agents are model-free: they directly generate and execute actions in real environments without being verified or optimized within a world model or simulator [4] [6 in advance. This leads to safety risks and suboptimality of planned trajectories.
Predic-Pree Control (MPC)
Planned Actions
Agent Model
World Model
LLM
LLM
Code Rules
Predictions
Predicted Traj.
violation of basic laws in LLM
NeuroSymbolic Learning
Reward. Feedback, and Observation
LLM
Hules
Code
Pruning
Action Rules
Scane Graph
1h 'lool is not sporopriate
plans
Take an Optimized Action
Knowledge Graph
Ghoop
dosert
comble
LLM
Program
Real Traj.
In this paper, show that the "world alignment" of an LLM can significantly improve its perfor- mance promising world model, which enables us to build more powerful embodied LLM agents in partially observable settings. To this end, in- troduce a training- recipe "World Alignment by neurosymboLic LEarning (WALL-E)", which compares and analyzes agent-explored trajectories and model-predicted ones extract symbolic knowl- edge that is enviromment-specinn and complementary to the LLM's prior knowledge, as illustrated Fig- ure 2 WALL-E's symbolic knowledge covers action rules, knowledge graphs, and scene graphs, which can be converted into executable code rules to turn a pretrained LLM into an accurate neurosymbolic world model (via function calling). It combines the strengths of both LLMs and symbolic representations in modeling environment dynamics, i.e., (I) the rich prior knowledge, probabilistic, and deductive reasoning capability of LLMs under uncer- tainty [17]: and (2) the formal constraints, deterministic transition rules, and environment-speccin structures enforced or encoded by symbolic knowledge. our studies, LLMs already cover the most common knowledge of dynamics but a few additional symbolic knowledge can significantly improve the world model predictions generated by LLMs.
Environment
Figure 2: Overview of WALL-E. The agent determines actions to take via MPC, which optimizes future steps' actions by interacting with neurosymbolic world model. The world model adopts an LLM whose predic- tions are aligned with environment dynamics through code rules converted from symbolic knowledge (action rules, knowledge/scene graph) learned via inductive reasoning from real trajectories and predicted trajectories.
Different types of symbolic knowledge play important roles in building more reliable and adaptive model-based LLM agents, especially in partially observable Markov decision processes (POMDPs). The action rules capture and enforce deterministic constraints in the decision-making process; the knowledge graph represents feasibility constraints and action prerequisites; while the scene graph provides global information complementing the partial observations of agents. In WALL-E,

---

leverage LLMs' inductive reasoning capabilities to abstract concise symbolic knowledge from explored trajectories for episodes. We further develop maximum coverage-baser pruning to maintain a compact set of code rules. In contrast, existing L.M agents usually learn the environment dynamics through expensive finetuning of LLMM policies via RI//mitatiinn learning, or rely heavily on task-specific modeling procedures, limiting scalability and flexibility. 4 5 18/24.
Unlike the mainstream model-free LLM agents, WALL-E's precise neurosymbolic world model enables us to create more reliable and versatile model-based LLM agents for challenging open-world tasks. Specifically, we propose novel model -reee--predictvve control (MPC) framework for LLM agents, in which an LLM agent conducts a look-ahead optimization (planning) of future steps' actions by interacting with the world model. For example, the agent queries the world model "What will happen if I take action 0, in observation o,?", and receives the prediction with feedback/suggestion according to the code rules, based on which the agent chooses to execute the plan or keep refining until it passes the world model's examination, i.e., not violating any constraints of the environments and leading the agent to preferred (predicted) states. Our LLM-based MPC framework overcomes the inefficiency of classical MPC that requires online k-step optimization, by exploiting the strong reasoning and instruction following capability of LLMs.
We evaluate WALL-E in challenging open-worlds such as Mars (Minecraft-like) and ALFWorld (embodied indoor environment) where the agents can explore freely and target complicated tasks. Our main contributions can be summarized as:
We address the "world alignment" challenge for LLM agents and propose novel training-free approach "WALL-E"" to align L.LMs with environment dynamics, resulting neurosymbolic world models.
We develop a novel L.LM-baed MPC framework for world model-based LLM agents.
WALL.-F achieves state-of-the-art performance in Mars and ALFWorld.
2
Related Work
Although the alignment of LLMs has been widely studied as a major objective of LLM post-training 28] their alignment with the dynamics of external environments, which we refer to as "world alignment", remains far less explored for LLM agents.
LLMs to build world models. Previous research has used LLMs to construct explicit world models for planning. For instance, Ada (21) integrate LLMs with hierarchical planners to acquire task- specific action abstractions and low-level controllers, thereby offering more accurate plans. Other work constructs domain models in PDDL and leverages external planners, with LLMs acting as an interface for generating. validating, and refining the models using human or automated feedback [22). WorldCoder [19 instead builds a Python program to represent the agent's 's evolving world knowledge. The program is iteratively refined through interaction, serving as both a symbolic simulator of state transitions and constraint -eatrdddt--uuded planner, achieving higher sample efficiency. While effective, these methods typically require building symbolic models from scratch and rely on aevironment-specinf design, which reduces scalability. In contrast, we treat the LLM itself as the world model and align it via training-free symbolic knowledge extraction, avoiding complex, anviromment-specffe modeling pipelines.
LLMs as world models. Another line of work uses the LLM itself as the world model. Some approaches fine-tune LLMs on embodied experience to enhance reasoning and planning in situated environments 233, while others incorporate action preconditions and effects during fine-tuning for more accurate outcome prediction [24). trompt-based approaches include LLM-MCTS (20]. RAP [8) and RAFA (29, which guide LLMs to plan trajectories or explore reasoning trees through carefully designed, aavironment-specti on heavy fine-tuning or manual prompt engineering. limiting flexibility and scalability. contrast, our method uses the inductive reasoning ability of LLMS to extract symbolic rules directly from exploration experiences. This process requires no handcrafted prompts, task-specific modeling, or fine-tuning, enabling rapid and scalable adaptation across diverse environments.
prompts. While effective, these approaches often depend
3

---

NeuroSymbolic Learning of Code Rules
Stage 1. Exploration of Trajectory Data
Stage Learning Symbolic Knowledge
Tiadl Faaebbr
1it (he Majar
Real Trajectory
LLM
Program
Action For action 'aratt', the player dons not have enough natariale... the actioon will fail 2. For action
Knowledge Graph
plins
Seoent
Prodicted Trajectory
"ooden" [1"id" wood)
"edges" (1, (2,3)
"sodea" : "'''d"": plains)
redgea" [1
Stage Code Rulle Set Pruning Maximum Set
3. Translating Symbolic Knowledge Code
Prompt: Implement the leazned rules lythes Eunction to evaluate observation the Knowledge Graph oz Boone Graph as extra arguments. Isput: [tasttua] sules, kngvl graph, acens geaph)
actico, optionally using
CodeRule_
Output: (code roles)
CodoRut
CodeRule 1
def Codehule Diobservation. action, knowledge graph) if "eatte"invettor"" ). get imaterial, quantity:
def Codekule Aobseevvttio. action, scane graph) if "Unexplozed" not in location_iten itens:
<><()
selectadidropped CodeRules O/ Transitions canicannot be predicted by world model
Regions covered by
Figure 3: NeuroSymbolic Learning of Code Rules. WALL-E iteratively refines the symbolic knowledge with the agent's actual trajectories in the environment and the world model's predicted trajectories. The NeuroSymbolic learning takes 4 stages: (1) comparing predicted and actual trajectories; (2) learning new symbolic knowledge from real trajectories; (4) translating symbolic knowledge code; and (4) Code rule set pruning via solving maximum coverage problem.
3 World Alignment by NeuroSymbolic Learning (WALL-E)
We consider a Partially Observable Markov Decision Process (POMDP) defined by p, T,7), where S is the set of hidden (unobservable) states, A the action space, O the space of textual observations available to the agent, including possibly partial descriptions of the environment. The reward function p S x A + I assigns scalar reward, T(s++1 St, 4z) defines transition dynamics, and E [0, 1] is the discount factor. Since the agent cannot observe the true state st. it must act based on partial observations o, E O.
M (S, O,,
3.1
NeuroSymbolic Learning of Code Rules
3.1.1 Stage 1: Exploration of Trajectory Data
To evaluate inconsistencies between the LLM world model and the real environment, compare the enviromment-generan (real) trajectory rraal = (greal = (ot, 0 and the prede-eded trajectory ppredicted (peedieted = (or, 014
Specifically, our world model first predicts whether transition will succeed or fail (binary clas- sification), and then predicts the next observation 0++1. This design choice is motivated by the fact that 0:+1 can be attributed to the success or failure of an action. For example, if the action "craft a table" succeeds, the number of tables increases and the corresponding materials decrease in Oc+1- In contrast, directly predicting the full observation 0++1 regardless of the action's 's consequence more challenging and susceptible to noise from minor environmental details (e.g., weather or creature presence). Thus, we perform consistency checks on 01+1 and ōt+1 based on transition success/failure only. We then classify all transitions in predicted into two sets: correctly predicted transitions goor E Deor and incorrectly predicted transitions gine E Dine
As shown in Step of Figure 3. the real trajectories 7 real are used to induce code rules that align the I.LM world model with the true environment (Sec. 3.2. (.12.3.1..2 3.1.3.. Dinconect then serves as a diagnostic set for pruning redundant or conflicting code rules via maximum coverage problem (Sec. 3.1.4).

---

3.1.2 Stage 2: Learning Symbolic Knowledge
Deterministic Transitions We use the LLM's inductive reasoning to extract action rules from real trajectories 7 where Treal kt denotes the past i: real transitions before timestep L. The parameter k defines finite context window for the trajectory. These rules capture deterministic constraints in the environment and help enforce correct decision-making (an example is shown in the stage 2 of Figure 3). For example, an action rule might specify; "For action make, if table is not in newr_ _objecs, the action will fail." Formally,
Extraction of Action Rules:
R₂ = fLLMuue
real (t-kett
U
where MLLle is implemented by prompting an LLM to generate action rules (see Appendix B. for detailed prompt), and R (r1, 12,,7,..... " (() denotes the extracted rule set.
Constraints in the POMDP We construct a knowledge graph Glnowkodge (V,E) to represent feasibility constraints and action prerequisites, where nodes V are entities (e.g.. materials, tools) and edges & denote symbolic relations. The graph is incrementally updated by merging new relations extracted from recent trajectories:
Knowledge Graph:
g ghnwwkedge
fIIME (T U 9
gknowledge
where fiLMig prompts the L.MM to extract relation triples {(u, v) (see Appendix B.3 for detailed prompt), (u, v,e) captures constraint on how an entity u relates to another entity under the label e, e.g.. "require", "consume". and "enable". For instance, the LLM observes that item A always fails to be created unless r units of B and y units of C are available, it induces the constraints (B, A, require z) and (C, A, require y).
from Tt
Scene Graph: Complementing Partial Observations To handle partial observability in POMDPs, we construct a Scene Graph (SG) to augment the agent's local observation with global context. Formally, G EV,E), where V represents observed entities (e.g., objects, locations), while & captures spatial relations such as "ownership", "in" "adjacency to"". For example, the environment contains multiple rooms, and each room includes certain items, SG can record the specific items located every room, complementing the agent's 's local observation with global information. At each time step t, we update the SG via:
Oscene (o))ne(o)) U 9
(3)
where /eeeee is a function that analyzes the agent's observation and generates sub-graph recording the spatial relationships among entities present in Ot.
3.1.3 Stage 3: Translating Symbolic Knowledge to Compact Code Rules
Next, given the rules R and the graphs gknowledge and guenee (which, unless otherwise specified, refer to Rr, g ghnowledge and geeene respectively, throughout the rest the paper), we prompt the L.M fLLMeode generate executable code rules below (see Appendix B.2 for the detailed prompt):
Rodde fLLMoode (R, ghoowkedge gucene)
An example rule rooode RRode is shown on the right (all examples in Ap- pendix D). Each rule returns: feedback (fard ) success flag (flag and sugges- tion (dug The flag indicates transition success, while feedback and suggestion guide corrective re- ce-planning These com- pact functions distill symbolic knowledge to enhance planning.
CodeRule, CodeRule_exsmpLe (state, action, kg/sg):
tool action get O). get ("toolaane") inventery state. get ("inventery", (,)
for res, ant in ge. ktogl ()).items (): inventory get (rea. 0) ant:
return f'Action failed: Not enough (rea)." I"Collect more (res)."
return "Succesal", True,
3.1.4 Stage 4: Code Rules Pruning
To ensure that the code rules derived from neuro-symbolic learning are correct, executable, and compact, we employ a multi-stage validation and pruning pipeline that integrates empirical testing with optimization.
5

---

As described earlier. our world model predicts the success or failure of environment transitions. Accordingly, each code rule evaluates transition feasibility the form: "if [state, action conditions] then [transition success/failure), otherwise [null]" (see code format in Section 3.1.3) A rule is triggered when its conditions are satisfied, returning non-null output. By comparing these predictions against collected transitions, we can systematically pruning rules through the following steps.
Each candidate rule is first tested against ground-truth transitions sampled from the real trajectory T If a rule misfires, producing outputs inconsistent with the true environment dynamics, it is immediately discarded. This ensures that only rules faithfully aligned with actual behavior remain. Next, they are checked for executability, each rule is run in a controlled environment, and any rule that raises a runtime or logical error is eliminated. Together, these two verification steps guarantee that only correct and executable rules remain in the pool.
After verification, optimize the retained rule set using a maximum coverage objective. The goal is to prioritize rules that best explain and correct incorrectly predicted transitions Dine, If a code rule ri corrects a mispredicted transition 6jmh we define that 0j is covered by o Thus, we frame code rule selection as a maximum coverage problem, aiming to find a compact subset R C Roole that maximally covers Din:
Rs arg max RCR
U yaur ER
Dcoo s.t., S1
where Deov is the subset of transitions covered by code rules, i.e., Dooe { E Dine gime covered by peode E Rosde). ). The parameter >> the limit of selected code rules, and we find that a large usually leads to better performance.
We solve this optimization problem using greedy algorithm (see Appendix EE. Through this process, we eliminate code rules covering only correct transitions, as they cannot address misalignments, and redundant code rules fully covered by more comprehensive ones (see Step 4 in Figure 3
The NeuroSymbolic learning process consists of four stages: (1) comparing predicted and real trajectories, (2) extracting symbolic knowledge, (3) translating this knowledge into executable code rules, and (4) pruning the rules via verification and a maximum coverage objective. This procedure, defined as NSLEARNING(), yields a compact and effective rule set:
Rende < NSLLARNINN( pred rell).
(6)
The resulting rules enhance the LLM world model's s alignment and improve decision-making quality.
3.2 Model-Predictive Control for World Model-based LLM Agents
As shown in Figure 2 at each MPC step, the agent queries the LLM world model fLLMWM asking, "What happens if take action as in observation o,?", the model generates the next observation 6,+1, textual feedback fb, detailed suggestion sug, and flag identifying action's 's success/failure, according to the agent's 's action in og, as described below.
(ōe+1, fb, sug, flag) fLL. (0t, dt)-
To align predictions with the true environment dynamics, the code rules Rode derived from explo- ration are applied. Importantly, these code rules do not replace the LLM world model. Instead, they serve as corrective constraints that mitigate LLM's residual misalignments, while the LLM itself already captures most of the environment dynamics and contributes essential prior knowledge for prediction. Specifically, the LLM's outputs are verified against the rule set Roode which can generate corrective feedback (Moode, oug slagsode) code) (as discussed in Section 3.1.3.. The final aligned output
is handled by
(b,41,fb, sug..f,..uuueee flag) = ALIGN(R fLLMMm o2, at).
(8)
which internally combines the outputs of fLIMWM and Reode mentioned above. Concretely,
f fLL. (Ot, fbode code oog .ola if flag #flag, fLLM az),
ALIGN-
otherwise,
In the conflicting case flag fflgg, the world model is re-invoked with goode sugeode flag code to revise its prediction in line with code rules. We avoid using Roode 's outputs directly, as they lack the

---

full context needed for the LLM agent to follow reliably (as shown in Section 3.1.3) 1. If flag = False, the agent uses the feedback and suggestions to revise its plan accordingly.
a+ LLL sug).
(10)
This refinement loop continues until the proposed action passes verification or the replanning limit is hit (as shown Algorithm The entire MPC process defined as:
ae 0c+1 < MPC R
(11)
Crucially, unlike conventional MPC methods that rely on computationally expensive action search methods such as cross entropy method (CEM) [30] and derivative-free optimization (DFO) [3]] WALL-E leverages the rich prior knowledge and reasoning capabilities of pretrained LLMs to implement highly efficient look-ahead optimizer (i.e., LLMAGENT(())) substantially reducing the search space and computational complexity, especially in complex or high-dimensional environments.
WALL-E alternately executes MPC and NeuroSymbolic learning. In each MPC phase, actions proposed by the LLM agent are predicted and validated by the world model using the code rules. Meanwhile, NeuroSymbolic learning continuously extracts and updates code rules from recent interactions, which are then integrated into the world model. The full procedure is in Algorithm 2
Algorithm 1 Model- Control (MPC)
Algorithm 2 WALL-E
1: Input: or, Rodde
1: 1* NSLEARNING() in Sec. 3.1 MPCO in Sec. 3.2
2: Initialize: fb + [I. sug < D. count + 0
2: Initialize Reode + 0, rell pred f D. t
3: repeat
4: - LLMAGENT(O,,.0.oOO
Og + ENV()
10)+
3: while
-TASKCOMPLETE(O)) -AGENTDIED(O), do
and
5: 0141,bb, Sug. flag
ALIGN(RSod, fLLMMM: Ot, at)
4: (t,, 11,0141 +- MPC(oe, Roode) 1eeq.
/eeq. (8%
o++1 + (NV(a:)
6: count f count +1
6: Treal APPEND((O,, ((or, ags oc+1))
7:
if flag then
7:
preed .APPEND((OO) des ōc+1))
8: break A*Acioo accepted*/ end if
8:
Reode +- NSLEARNING (7 (rel)) redd)
9:
10:
until count 2 REPLANLIMIT
t ++++1
11: Output: a,, ōt+1
10: end while
4 Experiments
4.1
Benchmarks
Mars (32 is an interactive benchmark for testing situated inductive reasoning-wwe -where agents must derive and apply new rules in unfamiliar contexts. Built on Crafter [33 Mars modifies
terrain, survival settings, and task dependencies to generate diverse wunter-commmosenss worlds. Agents must interact with these worlds, adapting to novel rules rather than relying on pre-existing knowledge. This makes Mars unique testbed for evaluating agent adaptability and reasoning in dynamic environments. See Appendix for details.
ALFWorld [34] is a virtual environment designed as a text- based simulation where agents perform tasks by interacting with simulated household environment. This benchmark includes six distinct task types, each requiring the agent to accomplish a high-level objective, such as placing a cooled lettuce countertop. See Appendix F for details.
4.2 Evaluation Metrics
Mars: (1) Reward (higher better): the sum sparse rewards given during an episode, including +1 for unlocking achievements, -0.1 1 for health points lost, and +00.1 0.1 for health points regained. The reward primarily reflects the number of achievements unlocked. (2) Score (higher is better):
single aggregated value derived from the geometric mean of success rates across achievements, weighing rare and difficult achievements more heavily to better reflect overall capability.
ALFWorld: (1) Success rate (higher is better): percentage of tasks the agent completes.
7

---

Table Comparison of WALL.-E with RL-bbased (PPO 335 & DreamerV3
and L.L.-based methods (ReAct [37], Reflexion [8 Skill Library (3). IfR [32 in Mars [2). L.LM-based methods and RL-based methods' results are averaged over trials and 20 trials, respectively (*-reported in previous work). RL trains a policy separately for each world and is supposed to be better than LLM-based. Reward are accumulated and reflects unlocked achievements. Score (%) is the weighted geometric mean of success rates, emphasizing report standat deviation across trials. The "Average" row excludes the Default world to focus on commerr-oommonsense settings, following the Mars evaluation protocol. The best results within the training-free (itreeLLLMM (LLM-based) methods are in bold. WALL-E is the SOTA among training- methods, demonstrating strong adaptability and planning in both familiar and counner-commonsennee settings that conflict with pretrained I.I. priors. DreamerV3, while achieving higher absolute scores, trained with million environment steps and included for reference only, highlighting the contrast in sample efficiency and the rapid adaptation capabilities of WALL-E.
and challenging achievements. Both metrics
Metries
Mod. Typ
RL--aased methods
LLM-based methods
PPO"
Dreamer V3
ReAct"
Reflexion"
Skill Library"
IRR
WALL-E
Default
19 1.4
11.5 1.6
1
6.0 1.7
8.0 2.1
90+23
9.5 2.1
Terrain
-0.1 0.6
2.2
7.4
6.4 3.0
9.5 2.9
8.15. 3.7
10.7
+ 2.6
Survival
0.6 0.5
8.6+4.1
6.4
4.6 3.9
7.9
77+33
13.8
Reward
Task. Dep.
2.1 +1+-1
8.8
5.0
3.2
1.5
5.6-2.. 2.9
6.4 6.4
Terr
0.0
711221
6.7
2.0
6.8
5.5227
Task.
4.8.1.0.
Sury. Task.
0.640..
9.64344
1.0
2.
3.3
All three
0.1
-0.4
0.5
Average
0.0
7.9
4.6
3.6
4.2
5.5
6.7
Default
1.+
14.2 1.3
8.0 1.5
5.3
0.9
8.3 1.3
13.0 + 2.1
20.3
+ 1.8
Terrin
0.1
13.0
1.6
7.6
7.41 1.6
11.9
3.4 3.4
11.8
2.9
27.8
1.7
Survival
0.2 0.0
10.8
2.8
8.0
5.5
9.7 2.20
11.0 3.7
50.8
Score (%)
Task. Dep.
0.6
12.1
4.
22405
1.5 0.6
6.9
9.3
2.0
Tem:
0.1
7.9+13
7.1 7.1
2.6
0.7
8.6 1.9
Task.
0.1
3.3
7.1
Task.
15.9 2.6
All three
0.6
4.0
1.0
0.2
0.2
0.6
2.2 1.6
Average
0.6
10.3
5.2
5.0
7.4
15.3
Table 2: Comparison of WALL-E and baselines on 134 testing tasks from the ALFWorld 34
100
*-reported in previous work. The highest success rate (%) for each task is highlighted
Success Rate (36)
in bold. WALL-E outperforms all other baselines.
Blous)
Method
Avg.
Fick
Clea
Success Rate (%
Heat
Cool
Examiine
Picktwo
walen
wnner
GPT-BUTLER
BUTLER*
[5]
26
62
51
21
78
30
12
Iteratonn Times
14
Figure 4: WALL-E baselines on 134 ALFWorld tasks. WALL-E achieves the highest success rate after only 4 iterations, surpassing all baselines.
ReAct 11
74
7g
54
85
83
51
AdaPiannee
30
91
100
100
89
100
97
47
Reflec von
86
92
81
90
RAFA
95
100
97
95
100
WAL.I-K (omes)
100
4.3 Main Results
WALL-E excels in planning and task-solving with rapid adaptation. As shown in Tables I and
2. WALL-E consistently outperforms baselines across diverse environments. In Mars. WALL-E
achieves reward improvements of 16. 1% to 51. 51.6% and boosts task scores by least 61.7%. While DreamerV3 reaches higher absolute rewards, it is trained for 1 million environment steps-far far exceeding WALL-- 's budget of just 5,600 steps (0.56% of DreamerV3's budget). Our focus not on absolute performance, but on enabling efficient few-shot adaptation and situated reasoning. DreamerV3 is included as reference to emphasize this contrast learning efficiency. In the "Survival" setting, our method shows high variance due cow attacks: performance drops sharply when many cows spawn nearby, but remains strong when few appear, leading to a large standard deviation. In ALFWorld, WALL-E achieves the highest success rate after only four iterations, significantly surpassing strong baselines like RAFA and AdaPlanner [40], as shown Figure
World types include Default (original Crafter setting with no modifications) and connterrcomonsenns worlds, which feature individual modifications (Terrain, Survival, Task Dependency) or combinations of two or all three modifications (Tem. Surv., Tem. Task., Surv. Task., All Thre)).

---

WA adapts effectively to environments that contradict the LLM's prior knowledge. As shown in Table I performance drops significantly (by 3.3%-97.1%% when the environment deviates from the L.M'' priors. However, WALL-- overcomes this by inductively learning symbolic knowledge that realign the world model with actual environment dynamics. This leads to improved predictions and decision-making, yielding at least a 21.8% reward gain and 51.6% score gain over other methods counter-commonsent settings. When environmental changes originate from a single source, WALL-E remains highly adaptable, improving reward and score by least 31% and 66%, respectively. However, when multiple factors change, the improvement diminishes- because rules are tailored specific dynamics and cannot fully capture more complex mechanics.
WALL-E significantly outperforms the SOTA baseline IfR. Figure 5 shows that WALL-E improves from a reward of 3.6 to 6.7 and from score of 4% to 15.3%, exceeding IfR by 17.9% and 51.6%, respectively. Two key innovations drive this gain: (1) NeuroSym- bolic learning fuses action rules, knowledge graphs, and scene graphs for richer environ- ment model, whereas ItR uses only inductively learned rules; (2) We use executable code rules, ensuring strict, deterministic enforcement, while IfR's natural-lange language rules, applied via prompt- ing, introduce variability when priors conflict with rules. This combination delivers more re- liable, effective planning.
14
6-
12
Reward
10
Iteration Times
Iteration Times
Figure 5: Comparison between WALL-E and IfR (the best baseline in Table IT over learning iter- ations Mars. WALL-E achieves clear ad- vantage over IfR in both learning efficiency and overall performance, due to the world alignment with diverse symbolic knowledge and code rules.
4.4 Effectiveness of NeuroSymbolic Learning
To evaluate the effectiveness of our proposed NeuroSymbolic Learning approach, we compare it with the skill library, a widely used agent learn-
100
WALL-EJous
Silll
Cover Rate [%)
ing method also employed by JARVIS-1 [39
60
Reward
and Voyager 43 In the Mars benchmark, this approach is further simplified to better adapt the environment, resulting in a framework consisting of reasoning module, a reflector, and the skill library [32]. In contrast, WALL-E comprises reasoning module, reflector, and a neurosymbolic learning module. The skill li- brary stores successful plans and utilizes them as in-context examples to align LLM agents with the environment's dynamics.
40-
20
Iteration
Iteration
Figure 6: Left: WALL-E code rules' cover rate (higher the better) over neurosymsbolic learning iterations in Mars. The cover rate measures the percentage of LLM failed predictions that can be corrected by our world model. Right: Comparison between WALL-E's world model and skill library when applied to LLM agents with reasoning and reflection (base agent). WALL-E- agent + world model: Skill Library- base agent skill library. WALL-E's neurosymbolic learning sig- nificantly improves world alignment and brings greater gains to LLM agents than skill library.
NeuroSymbolic Learning enables efficient "world alignment". To assess how well the learned symbolic knowledge corrects the LLM world model, we collect the transitions that the world model mispredicts and compute the cover rate rate-the fraction of these errors fixed by newly acquired knowledge (details in Appendix FF3). By analyzing both the neurosymbolic learning process and the agent's performance across iterations (Figure (), we observe that WALL-E con- sistently outperforms the skill library, and significant performance improvement as the quality of the symbolic knowledge library increases (i.e., as the cover rate rises). This result highlights that WALL-E 's ad advancements primarily stem from acquiring and leveraging new symbolic knowledge.

---

Table Ablation study of WALL-E with differ- ent symbolic knowledge (KNWL) types. KG Knowledge Graph, SC-Seene Graph. The row highlighted in grey represents the configuration and performance of WALLL-F. Learning di- verse types of symbolic knowledge is essential for effectively aligning the LLM world model with the environment.
Table Ablation study of WALL-E with dif- ferent configurations on Mars tasks. Symbolic- apply code rules. The row highlighted in grey rep- resents WALL--. Code rules translated from symbolic knowledge bring more improvement when applied to the world model, indicating the importance of world alignment.
Agent
World Model
Reward 1
Score (8)
Symbolic
LLM
3.6
4,0
Action Rules
KGISG
Scure
LLM
LLM
3.8
4.1
LL M-symbllic
5.5
7.4
5.2
6.7
15.3
98
LLM-symbolic
LLM+symbolic
6.3
13.1
4.5
Ablation Study
Combining multiple types of symbolic knowledge enhances world alignment We evaluated world alignment using different symbolic knowledge combinations. When action rules are present, they run executable code; otherwise, the system uses prompt learning alone. T that relying on a single type degrades performance: in the Mars environment, "rules only" cuts reward by 23.9% and score by 45.8%, while "KG+SG only" drops reward by 34.3% and score by 66.0%. These findings underscore the necessity of multiple symbolic knowledge types.
World models with symbolic knowledge is an effective structure. We examine the impact of learned symbolic knowledge and the world model by removing each component from WALL-E and observing changes performance (Table 4. Adding symbolic knowledge, whether in the agent or the world model, consistently boosts the success rate. Specifically, applying symbolic knowledge in the world model yields about a 46.3% improvement, while applying them in the agent yields about a 30.9% gain. This difference likely arises because learned symbolic knowledge heavily depend on observation information (see Appendix D). Moreover, a standalone world model without symbolic knowledge offers little performance benefit, underscoring that symbolic nowledge-drivee alignment with environment mechanism is key to WALL-E 's success.
Ablation code rules pruning stage. We ablate the code rule pruning stage by evaluat- ing performance both without pruning and under different code rule set limits Table 5 shows that pruming stage is assential-withon it, perfor- mance drops sharply due to noisy or conflicting rules. As fewer rules retained, performance degrades, confirming that pruning stage selects impactful, high-quality rules that align the LLM world model with environment dynamics.
Table 5: Ablation study on the code rule prun- ing stage in Mars. The column bold shows the configuration and performance of WALL-E. Pruning is essential for selecting high-quality, impactful rules that align the LLM world model with environment dynamics.
Metrics
w/o Pruning
w/ Pruning- Rule Set Limit
5
no limit (9)
Reward 1 Score (5)
1.5
4.3
577
6.2
6.5
6.7
1.6
8.1
95
128
14.5
15.3
Conclusion
We have demonstrated that LLMs can effectively serve as world models for agents when aligned with environment dynamics via neurosymbolic knowledge learning. Our neurosymbolic approach leverages code-based, gradient-free integrations of action rules, knowledge graphs, and scene graphs, thereby bridging the gap between the LLMs' prior knowledge and specific environments. By using model-based framework, our agent, WALL-E, achieves substantial improvements planning and task completion. In open-world environments such as Mars and ALFWorld, WALL-E outperforms Mars baselines by 16. 15..6% success rate and achieves a new record 98% success rate in ALFWorld after only four iterations. These results underscore that additional symbolic knowledge is essential to align L.LM predictions with environment dynamics, enabling high-performing model- based agents in complex settings.
10

---

References
[1] OpenAI, "GPT-4 technical report," arXiv preprint arXiv:2303.08774 2023.
[2] J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. lchter, F. Xia, E. H. Chi, Q. V. Le, and D. Zhou, "Chain-of-thought prompting elicits reasoning in large language models," in NeurIPS, 2022.
[3] Y. Liu, W. Chen, Y. Bai, J. Luo, X. Song, K. Jiang, Z. Li, G. Zhao, J. Lin, G. Li, al., "Aligning cyber space with physical world: A comprehensive survey on embodied ai," arXiv preprint arXiv:2407.0666 2024.
[4] Y. Mu, Q. Zhang, M. Hu, W. Wang. M. Ding, J. Jin, B. Wang. J. Dai, Qiao, and P. Luo, "Embodiedgpt: Vision-language pre-training via embodied chain of thought," arXiv preprint arXiv:2305.1500 2023.
[5] Y. Yang. T. Zhou, K. Li, D. Tao, L. Li, L. Shen, X. Hc, J. Jiang, and Y. Shi, "Embodied multi-modaa agent trained by an Ilm from parallel textworld," in CVPR, 2024.
[6] A. Das, S. Datta, G. Gkioxari, S. Lee, D. Parikh, and D. Batra, "Embodied question answering," in CVPR, 2018.
[7] X. Wu, T. Guan, D. Li, S. Huang, X. Liu, X. Wang, R. Xian, A. Shrivastava, F. Huang, J. L. Boyd-Graber et al. "Autohallusion: Automatic generation of hallucination benchmarks for vision-language models." arXiv preprint arXiv:2406. 24000099000000 2024.
[8] S. Hao, Y. Gu, H. Ma, J. J. Hong. Z. Wang. D. Z. Wang, and Z. Hu, "Reasoning with language model is plamning with world model," arXiv preprint arXiv:2305. /4992, 2023.
[9] R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn, "Direct preference optimiza- tion: Your language model is secretly a reward model," NeurIPS, 2024.
[10] Z. Gc, H. Huang, M. Zhou, J. Li, G. Wang, Tang, and Y. Zhuang, "Worldgpt: Empowering Ilm as multimodal world model," arXiv preprint 2xXi::204.18200 2024.
[11] S. Yao, Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao, "React: Synergizing reasoning and acting in language models," in ICLR, 2023.
[12] N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao, "Reflexion: Language agents with verbal reinforcement lcarning," NeurlPS, 2024.
[13] B. Zitkovich, T. Yu, S. Xu, P. Xu. T. Xiao, F. Xia, J. Wu, P. Wohlhart, S. Welker, A. Wahid, et al., "Rt-2: Vision aagion-lagguag--rtee action models transfer web knowledge to robotic control," in CoRL, 2023.
[14] Q Wu, G. Bansal, J. Zhang, Y. Wu, S. Zhang, E. Zhu, B. Li, L. Jiang, X. Zhang, and C. Wang, "Au- togen: Enabling next-gen Ilm applications via multi-agent conversation framework," arXiv preprint arXiv:2308.08 081555 2023.
[15] V. Micheli and F. Fleuret, "Language models are few-shot butlers," arXiv preprint arXiv:210.009972, 2021.
[16] A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Hausman, A. Herzog, J. Hsu, al., "Rt-1: Robotics transformer for real-world control at scale," arXiv preprint arXiv:2212.068) 2022.
[17] Z. Hu and Shu, "Language models, agent models, and world models: The law for machine reasoning and planning," arXiv preprint arXiv:2312.052. 2023.
[18] Gao, B. Sarkar, Xia, T. Xiao, J. Wu, B. Ichter, A. Majumdar, and D. Sadigh, "Physically grounded lision-language models for robotic manipulation." arXiv preprint N3V:2230900266.1 2023.
[19] H. Tang, D. Key, and K. Ellis, "Worldcoder, a model-based I1m agent: Building world models by writing code and interacting with the environment," arXiv preprint arXiv:2402. 222.: 2024.
[20] Z. Zhao, W. S. Lee, and D. Hsu, "Large language models as commonsense knowledge for large-scale task planning." NewrIPS, 2024.
[21] L. Wong, J. Mao, P. Sharma, Z. S. Siegel, J. Feng, N. Korneev, J. B. Tenenbaum, and J. Andreas, "Learning adaptive planning representations with natural language guidance," arXiv preprint arXiv:2312.0888 2023.
[22] L. Guan, K. Valmeekam, S. Sreedharan, and S. Kambhampati, "Leveraging pre-trained large language models to construct and utilize world models for model-based task planning." Neur/PS, 2023.
11

---

[23] J. Xiang, T. Tao, Y. Gu. T. Shu. Z. Wang, Z. Yang, and Z. Hu, "Language models meet world models: Embodied experiences enhance language models," " NewrlPS, 2024.
[24] K. Xie, I. Yang, J. Gunerli, and M. Riedl, "Making large language models into world models with precondition and effect knowledge." arXiv preprint arXiv:2409. 122277,8,, 2024.
[25] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, al., "Training language models to follow instructions with human feedhack," " NeurlPS, 2022.
[26] Y. Bai, A. Jones, K. Ndousse, A. Askell, A. Chen, N. DasSarma, D. Drain, S. Fort, D. Ganguli, T. Henighan, et al., "Training a helpful and harmless assistant with reinforcement learning from human feedback," arXiv preprint ar0:::2204.000000 05862, 2022.
[27] K. Yang, Liu, Q. Xie, Huang. T. Zhang, and Ananiadou, "Metaaligner: Towards generalizable multi-ohjective alignment of language models," arXiv preprint Arv: 17141, 2024.
[28] G. Seneque. L.-H. Ho. A. Kuperman, N. E. Saeedi, and J. Molendijk, "Abe align: Large language model alignment for safety & accuracy." arXiv preprint wxXi:2448.00000) 2024.
[29] Z. Liu, H. Hu, S. Zhang, H. Guo, S. Ke, B. Liu, and Z. Wang, "Reason for future, act for now: A principled framework for autonomous Ilm agents with provable sample efficiency." arXiv preprint arXiv:2309.1338, 17382, 2023.
[30] P.-T. De Boer, D. P. Kroese, S. Mannor, and R. Y. Rubinstein, "A tutorial on the cross-entropy method," Annals of operations research, vol. 134, pp. 19 67, 2005.
[31] A. R. Conn, K. Scheinberg, and L. N. Vicente, Introduction to derivative-free optimization. SIAM, 2009.
[32] X. Tang, J. Li, Y. Liang, S.cc. Zhu, M. Zhang, and Z. Zheng, "Mars: Situated inductive reasoning in an open- environment," arXiv preprint arXiv:2410.08126. 2024.
[33] D. Hafner, "Benchmarking the spectrum of agent capabilities," arXiv preprint arXiv:2109006780. 2021.
[34] M. Shridhar, X. Yuan, M.-A. Coté, Y. Bisk, A. Trischler, and M. Hausknecht, "Alfworld: Aligning text and embodied environments for interactive learning." " arXiv preprint arXiv:2010.03768, 2020.
[35] Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, "Proximal policy optimization algorithms," arXiv preprint 177,006777,
[36] D. Hafner, J. Pasukomis, J. Ba, and T. Lillicrap, "Mastering diverse domains through world models," arXiv preprint arXiv:2301.04004 2023.
[37] S. Yao, Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao, "React: Synergizing reasoning and acting in language models," arXiv preprint arXiv:2110.03629, 2022.
[38] N. Shinn, F. Cassano, B. Labash, A. Gopinath, K. Narasimhan, and S. Yao, "Reflexion: Language agents with verbal reinforcement teaming.(0023)) arXiv preprint cs.A1/2303 2023.
[39] Z. Wang. S. Cai, A. Liu, Y. Jin, Hou, B. Zhang, H. Lin, Z. He, Z. Zheng, Y. Yang, et al., "Jarvis- I: Open-world multi-task agents with arXiv:2311.0599 2023.
multimodal language models," arXiv preprint
[40] H. Sun, Y. Zhuang. L. Kong, B. Dai, and C. Zhang, "Adaplanner: Adaptive planning from feedback with language models," NeurlPS, 2024.
[41] Z. Wang, S. Cai, G. Chen, A. Liu, X. Ma. Y. Liang, and T. Crafiarvis, "Describe, explain, plan and select: interactive planning with large language models enables open-world multi-task agents," in NeurIPS, 2023.
[42] M. Shridhar, J. Thomason, D. Gordon, Y. Bisk, W. Han, R. Mottaghi, L. Zettlemoyer, and D. Fox. "ALFRED: A benchmark for interpreting grounded instructions for everyday tasks," in CVPR, 2020.
[43] G. Wang, Y. Xie, Y. Jiang, A. Mandlekar, C. Xiao, Y. Zhu, L. Fan, and A. Anandkumar, "Voyager: An open-ended embodied agent with large language models," arXiv preprint arXiv:2305.16291 2023.
[44] X. Zhu, Y. Chen, Tian, C. Tao, W. Su, C. Yang. G. Huang, B. Li, L. Lu, X. Wang, et al., "Ghost in the minecraft: Generally capable agents for open-world environments via large language models with text-based knowledge and memory," arXiv preprint arXiv: /7144, 2023.
[45] Y. Qin, F. Zhou, Q. Liu, 7. Yin, I.. Sheng, R. Zhang, Y. Qiao, and Shao, "Mp5: A multi-modal open-ended embodied system in minecraft via active perception," " CVPR, 2024.
[46] Z. Li, Y. Xie, R. Shao, G. Chen, D. Jiang, and L. Nie, "Optimus-1: 1: Hybrid multimodal memory empowered agents excel in horgzoo tasks," arXiv preprint arXiv:2408.03615 2024.
2024.
12

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The Abstract and Introduction clearly state our main contributions, which fully supported by the proposed method and experimental results.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
Limitations are discussed in Section G
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't It acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
13

---

Justification:
This paper does not include theoretical results, assumptions, or formal proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Sections and B provide sufficient details on setup, implementation, and prompts to reproduce the main results.
Guidelines:
The answer NA means that the paper does include experiments. If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
14

---

Answer:
[Yes]
Justification: An anonymous GitHub repository with code and reproduction instructions is linked in the Abstract.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to to reproduce the results. See the NeurIPS code and data submission guidelines (https
wttp:lliiggiie////ee/ http://guuudee/deeSSuuunn ssionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: All the training and test details are provided in Sections and F
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Table I reports mean and standard deviation across 9 trials for LLM-based methods and 20 trials for RL-based methods.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
15

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: See Section F
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics http:///wrrrp...cppoop...aaaaaaa
Answer: [Yes]
Justification:
This research adheres fully to the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- cration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
This work is foundational and focuses exclusively on improving the alignment of L.LM-based world models and the planning efficiency of embodied agents. It does directly address any societal applications or their consequences- it is tied specific deployments use cases. Therefore, there no discussion of potential harms or benefits in the paper.
Justification:
or negative- since
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
16

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper does not introduce models data with high risk of misuse; thus, safeguards are not applicable.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: See Appendix
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
17

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, batasets.papersstt paperswi
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors encouraged to reach out the asset's s creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We provide anonymized URL in the abstract linking to the released assets, which include code, and documentation for reproducibility.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve any crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
This work does not involve human subjects or crowdsourcing.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
18

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification: LLMs are central to our method they serve as the world model, induce symbolic knowledge, and generate executable rules. These roles are core to the originality and effectiveness of our approach.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non ntn-standard components.
Please refer to our LLM policy for what should or should be described.
19

---

A Detailed Related Work
Using LLMs to Build World Models. Many studies have leveraged LLMs to construct world models for planning. For example, Ada [2] translates natural language instructions into adaptable planning representations via LLMs, enabling flexible and context-aware world modeling. Combining pre-trained LLMs with task-specific planning modules has also been shown to improve task success rates by providing a more detailed understanding of the environment [22] Another approach, WorldCoder [9] exemplifies an L.M agent that constructs world models by generating and executing code to simulate various states and actions, refining its understanding iteratively.
These studies demonstrate the effectiveness of using LLMs to construct explicit world models from scratch to enhance planning and reasoning in complex environments. However, our key novelty lies in treating the LLM itself as the world model and aligning it with environment dynamics through training- free symbolic knowledge extraction. Rather than building a complete symbolic model from the ground up, we focus on identifying and correcting the misalignment between the base LLM's s prior knowledge and the specific environment. This allows small amount of complementary symbolic knowledge, combined with the L.M'' existing priors, to form an accurate and generalizable world model. In contrast, prior methods rely on complex, environment-specific modeling procedures, which limits their scalability and adaptability to high-complexity environments like Mars.
Using LLMs as World Models. Several studies have explored using LLMs directly as world models by leveraging their implicit knowledge. Some methods rely on fine-tuning align the LLM world model with the environment. For example, one approach fine-tunes LLMs using embodied experiences in simulation to improve reasoning and planning in embodied settings (23). Another incorporates action preconditions and effects during fine-tuning to enable accurate outcome prediction and action reasoning [24]
Other approaches align LLMs as world models through prompting. LLM-MCTS (20) prompts LLMs to serve both the policy and world model for large-scale task planning, integrating commonsense priors with guided search. RAP g uses prompting to guide LLMs in generating reasoning trees for structured decision making. RAFA 29) applies Bayesian adaptive Markoy Decision Process to prompt LLMs to predict future states for trajectory planning.
While these approaches demonstrate the potential of using LLMs as world models, they often require extensive fine-tuning or rely heavily on human-crafted prompts, making them labor-intensive and inflexible. Our approach leverages the LLM's inductive reasoning to automatically extract symbolic rules from exploration trajectories, drastically cutting human effort and computational overhead. This enables rapid, domain-agnostic adaptation across diverse environments.
A.1 Existing Agents in Minecraft
Although our method is evaluated in Mars, 2D environment inspired by Minecraft, we also discuss prior agent methods developed in the Minecraft domain due to their conceptual relevance.
DEPS 41 uses an LLM to generate, explain, and revise plans, but does NOT learn knowledge from previous tasks and generalize to new scenarios. Methods like GITM 44 MP5 45 JARVIS-1 39. and Voyager + use memory buffer or skill library to directly store successful trajectories and reuse them later. However, no inductive reasoning is applied to these trajectories to learn condensed, generalized principles. So their agents have to reason based on many raw trajectories, which can suffer from poor generalization, overfitting, and inefficiency when adapted to new scenarios/tasks. Optimus- 1 [46] partially addresses this issue by extracting knowledge graphs from historical trajectories, enabling more flexible knowledge transfer across tasks. However, it still relies heavily on logging trajectories and custom scripts tailored to a specific environment's 's data format for graph extraction, which cannot be easily generalized to new environments.
In contrast, our method does not store and reason directly based on raw trajectories. overcomes these limitations by inductive reasoning using LLM on existing trajectories, condensing them into a compact, principal, generalizable, and comprehensive set of symbolic knowledge (action rules, knowledge graphs, and scene graphs) that can be reused efficiently and generalized across tasks/envi- ronments. This abbstrrttttt of environment novel and critical to building an accurate world model. Another primary difference and novelty of WALL-E is to compile all symbolic knowledge
20

---

into executable code, which makes the planning verifiable with formal guarantees. This avoids possible hallucinations and mistakes of in-context learning by LLMs, which common for planning in prior work.
A.2 Existing Agents in ALFWorld
ReAct [1 Reflexion [2]. and AutoGen 14 leverage LLMs for reasoning, but they do NOT learn reusable knowledge from past tasks. AdaPlanner [40] introduces a skill library, but merely stores trajectories from previous successful tasks. Reasoning solely based these specific trajectories can mislead the agent and lead to suboptimal behavior new scenarios. RAFA (29) relies on an
LLM's prior knowledge for planning, which can be problematic when the environment dynamics and principles diverge from or conflict with LLM priors.
WALL-E differs from them in several key properties: Instead of simply recording past trajectories, WALL-E prompts the LLM to perform inductive reasoning over historical trajectories to extract compact and generalizable symbolic knowledge, discussed above. Unlike RAFA, whose model LLM solely relies on LLM prior, WALL-E applies verifiable neurosymbolic learning to align the LLM'ss predictions with real environment dynamics. This "world alignment" greatly enhances both prediction accuracy and long-horizon planning capabilities.
B Detailed Prompt
B.1
Learn Action Rules from Real Trajectories
Prompt for Learning Action Rules from Real Trajectories
You responsible for mining
rules from the given transitions, ensuring that these rules differ from the ones already provided. Focus on generating general and universal rules that are not tied to any specific item or tool. Your goal is to generalize across different objects, creating flexible rules that can applied broadly to diverse contexts situations.
I will give you array of transitions:
[
{
'state00:
"state feature 1": ('feature name": value, ...,,
Paction';
"name": "action name", "action feature 1": ("feature name": value, ...h,
paction_ result': { ""eedback": "the environment feedback", "success": "Whether the action is executed successfully,"" "suggestion": "..." }.
]
and an array rules:
[
"Rule 1: For action if if..., the action will fail; Checking Method: ...",
You should only respond in the format as described below: RESPONSE FORMAT:
21

---

{
"new :
"Rule .... For action Checking Method: :::²;
"Rule .... For action Checking Method:
Instructions:
Ensure the response can be parsed by Python 'json loads', e.g. trailing commas, Please use you knowledge in <ENV>, do inductive reasoning. You need to dig up as many rules as possible that satisfy all transitions.
single quotes**, etc.
Extract and utilize only the features that influence the outcome of the action.
Please generate general and universal rules; the rules should not reference any specific item or tool! You need to generalize across various items or tools. Generate only the rules under what conditions the action will fail.
While generating a rule, you also need to state how to check a transition satisfies this rule. Please be specific as to which and how 'features' need to be checked
B.2 Translate Action Rules to Code
Prompt for Translating Action Rules to Code
You are responsible for generating code rules by implementing the learned rules in Python. Your task is to write a function that takes the current state and action inputs, optionally incorporating both provide additional contertual information (extra input), evaluating these conditions, and returna a Boolean value based the specified rule. This function should effectively mirror the logic of the rules, enabling precise predictions for various state-action pairs.
knovledge graph and the scene graph
The function should be defined as follows:
" '' 'yython
def expected_rule_coott (state, action, koovletge_grppp/sceee.rpplp
# Your code here return feedback, success, suggestion
where
feedback: a string, give the action feedback based on success or not.
success: bool, whether the action is executed successfully, give 'True' 'False'. If the action type is the action type in the rule, count as success (e.g., success True).
s suggestion: string, if the 'action' fails, 'suggestion' would be given based on 'rule', 'state' and Paction',
Here are examples the state and action format:
<Input Format>
For example:
" "Instead of obtaining [item] from [collettiggreesuuree], players can acquire from Ialternative [illectingrreooucce
"To craft a [tool/object], players will need [ crafting_materiat and must [crafting..lattorm...
22

---

" "[crafting.naterial] needs to be gathered from [resource].",
sure to include such details to make the suggestions more engaging and relevant.
Knowledge Graph:
<Knowledge Graph>
Scene Graph: <Scene Graph
You should only respond not give example usage or anything else:
format as described below, and
RESPONSE FORMAT:
def
code (state, action,
Your code here
where "input format" please refer to Appendix C
B.3 Learn Knowledge Graph from Real Trajectories
Prompt for Learning Knowledge Graph from Real Trajectories
You are a helpful assistant with inductive reasoning. Given the history trajectory, including action and observation, you need to reflect on the action execution results and identify and feasibility constraints, that is, discover when an action or item creation requires the presence of certain materials, resources, or other items.
extract prerequisite
We define the Knowledge Graph as:
"V": "the set entities (e.g., items, materials, location- specific objects, or abstract concepta)",
"E": "the set of directed edges, each capturing relationship prerequisite among entities"
An edge takes the form: (u, label),
where and are entities V, and label indicates how relates to v (for example, 'requires', 'consumes', coolects' etc.).
I will give you array of transitions:
[
{ "aittal_state> : "...',
laction': ....?
"Whether the action is executed successfully, give 'True' or 'False' only"
}.
{
'inital_sten state :
'action's .....,
pection_result :
"Whether the action is executed successfully, give 'True'
or 'False' only"
}.
You should ONLY respond in the following format:
23

---

{
('ul Pentity_u, 'y':'entity_v', 'label :: :'rellatoon:""...
quantity':?.. ..... " ('u'; entttyyttyyyuu' 'y':'entity.v' 'label : "e.'rreaaionn:1 ":"...",
quantity'; :
example:
{'u:: wwwowoooooooooe "y taabl'', 'label :f'relation ); requires', 'quantity' ; None)},
('u':'table' "v"; 'wodd', 'label --'relation' consumes', quantity': (2233
C Environments' Observation Space and Action Space
The format of observation and action information is crucial for understanding the action rules have extracted, In this section, we provide an description of the observation and action space used in different environments.
C.1 Mars
Observation Space. We collect observation information directly from the observation space provided The specific structure illustrated the following example.
by Mars 32
Examples for Mars's Observation Space
obs
"position": "grass", "in_front"s "table", "visible_objects"
"type": "plant", "x": -1, "y":
"type": "grass", "x": 1, "y": 0
},
"type": "table", "x" 0, "y": -1
)....
}.
"near obbjartsb.ect.... "sand", "plant",
"grass",
"table"
J.
"status": {
"health", 8,
"food": 4,
"drink": 5,
"energy": 8
}.
" "inventory": "sapling": 2,
"wood_pickaxe"" 2, "wood_sword";
24

---

}
Action Space. We utilize the action space provided by Mars directly, as demonstrated below.
Mars's Action Space
mine (block_nanme amount) # mine amount blocks of the block_name. attack (creature, amount) attack move. Creature include zombie, skeleton, cow, etc. sleep (); # put the player to sleep.
amount creature that can
place the block. Note you need not craft table and furnace, you can place them directly. make (tool_name); craft a tool.
place (block_ jallock_amme;;
explore (direction, steps); # the player explore in the direction for steps.
C.2 ALFWorld
Observation Space. In the original ALFWorld setup, observation information is represented as natural language dialogue history. facilitate the neurosymbolic learning process, we developed scripts to transform this dialogue history into structured JSON format, shown in the following example.
Examples for ALFWorld's Observation Space
obs
"reachable_occtionnss
"cabinet 5"
"cabinet 4"
"cabinet 3"
"cabinet 2",
"cabinet 1",
"coffeemachine 1",
"countertop 2",
"countertop 1",
"diningtable 1",
"drawer 2",
"draver 1",
"fridge 1",
"garbagecan 1",
"microwave 1",
"shelf 3",
"shelf 2",
"shelf 1",
"sinkbasin
"stoveburner 4",
"stoveburner 3"
"stoveburner 2"
"stoveburner 1"
"toaster 1"
l.
"items_in_ii istems_in_locations {
"fridge 1": "lettuce 2",
"mug 2", "potato 3"
1. "microwave 1":
"item_in in hand "item_name" name": "cup 1", "status"; "normal"
25

---

}, ccurrent_position name":
"location "ercatio__name::: "microvave 1", "status": "open"
Action Space. We utilize the action space provided by the ALFWorld directly, as demonstrated below.
Action Space for ALFWorld
go to [location/object]: Move to specified location object. open [object]: Open a specified object like cabinet or draver. close [object]: Close an opened object.
take [object] from [location]: Pick up an item from specified location.
put [object] in/on [location]: Place item or specified location.
clean [object] with [location/tool]] Clean an object using a specific location or tool, like cleaning lettuce at the sink basin.
heat [object] with [tool]: Use appliance, such microwave, to heat an item.
cool [object] with [tool]: Use a cooling tool or appliance, such as fridge, to cool an item. tool, such desklamp.
[tool]: Activate
D Learned Rules
There are two points to note about the numbering of the action rules:
The reason for duplicates is that the numbering is based on actions, and different actions have their own separate sequences. For example: Rules for Craft: [Rule 1, Rule 2, Rule Rule 4, Rule 5...]; Rules for Mine: [Rule 1, Rule 2, Rule 3, Rule 4, Rule 5...]
The reason the sequence may appear unordered that some rules have been pruned (Section
3.1.3 Rule Set Pruning via Maximum Coverage). For instance, Rules for Craft where [Rule
1, Rule 2, Rule 4, Rule 5] has been removed, Rules for Mine where [Rule 1, Rule 3, Rule 4, Rule 5, Rule 6] has been removed, and the final rule set is Rules for Craft: [Rule 3, Rule 6] and Rules for Mine: [Rule 2, Rule 7].
Additionally, the feedback and suggestions returned by each code rule automatically generated by prompting the LLM with the corresponding rule. The detailed prompts used to generate these code rules can be found in Appendix B.2 These feedback and suggestions play erucial role in helping the agent refine and improve its planning process (Section 3.2).
D.1 Rules in Mars
Action Rules for Mars
Rule 6: For action 'make', if ?table? is not in 'near_objects"
the action will fail; Checking Method: Check if 'table' is in the 'nearoobjecte
list the initial state.
Rule 7: For action 'make', if the player does not have the required materials in the inventory to craft the specified tool the action will fail; Checking Method: Verify if the player 's inventory contains the required materials for the tool specified in
'make' action 'tool_name argument Rule 8: For action 'make', if the required resources for the tool
are not present in the inventory or if 'table' is not in near_objects', the action will fail; Checking Method; Verify if
26

---

the inventory contains the necessary resources for the tool that 'table' is present
being crafted and near_objects!
Rule 1: For action 'place' with block_name 'sapling's if 'in_front is 'table', the action will fail; Checking Method: Check if in_front' in the initial state is 'table'.
Rule For action 'place', if the player does not have the required materials
the inventory to place the specified item the action will fail; Checking Method: Verify if the player 's inventory contains the required materials for the item specified in the 'place' action's 'block_name' argument. Rule For action 'place' with block_name 'table', visible_objects. or 'near peerrobjects: already contain a 'table', the action should not be executed; Checking Method: Examine and 'near_objects/ arrays in the proceed
both the state for an object with type 'table'. If found, do with placing new table, and instead reuse the existing one. Rule 2: For action 'mine', if the block_name is 'iron' and the player does not have 'stone_pickaxe" or better, the action will fail; Checking Method: Verify if the 'block_name' is 'iron' and ensure the inventory includes a 'stone_pickaxe' or better. Rule 4: For action 'mine', if the player attempte to mine 'stone'
and does not have a 'wood_pickaze
or better in their inventory
the action will fail; Checking Method: Check if 'wood_pickaxe better tool is present in the inventory when attempting to mine 'stone'.
Rule 5: For action 'mine', if the block_mane name is 'plant', the action will fail; Checking Method: Check if 'block_name' in the action is 'plant'.
Code Rules for Mars
def CodeRule_ faalll____makee (state, action, knovledge.graph) Extract action details action_nane name = action. get aation.get(aaction ("action, name")
args = action. get ("args", (})
Check the action is 'maee'
if action_name name == "make": # Check if 'table' is in 'near_ objjcts
if "table" not in state. get ""near_objects'" []) feedback "Action failed: 'table' is nearby. success False
suggestion "Nove closer to ?table> to make the item.
return feedback, success, suggestion
# If the action is not 'make' or the condition is met feedback "Action executed successfully." auccess True suggestion ""
return feedback, success, suggestion
def
action, knovledge_graph)
# Check if the action is 'make'
if action action['acti aaattionnamme '] 'make': tool_name action (' tooll___mmm
# Use LLM_request to find out the required materials for the tool question f"What are the required materials to craft cool_name)?" response_formes "Provide list of materials and their quantities."
27

---

requirei_aterials = LeM_se (question
response_formar
Check if the player has the required materials in the inventory
inventory state (iiveenrooy) True
missing_naterials
for material, quantity in aequired_aaterall setrrialsiitmss ();
if geventory.getiieee (eatteria,, 0) quantity:
has_all_materiale False
blasing_materian append (f*(material); {quantity inventory. get (material, 0)} more needed")
if has las_all_material
feedback "Action executed successfully." success = True suggestion
else:
feedback "Action failed: enough materials to craft the tool."
success = False
suggestion f"To craft {tool_name) you need: {', w.j.on (nissing.tteriils))
else:
# If the action is not 'make', consider it successful feedback "Action executed successfully." success True suggestion ""
return feedback, success, suggestion
def @odeRule___make (state, action, knovledgo_graph)
Extract action details action_name action. nmmet"on_name")
arga = action..... gerloo..ee(((( ("args" (})
Check the action is 'make'
action_name "make":
return "Action executed successfully." ", True,
# Extract tool name tooe_name args.ge ("tool_name")
# Check if 'table' is in 'near_objects if 'table' not in state. get ("near_objects []) feedback "Action failed: 'table' required nearby make the tool."
suggestion "Nove closer to a 'table' and try again."
return feedback, False, suggestion
# Check the required resources reguirred_reourcae knovledge graph. get (tool_name, ( inventory = state. get (" ("inventory", ())
the inventory
for resource, amount in required,resources items () if inventory get (resource, 0) amount:
feedback f"Action failed: Not enough (resource) to make (tool_name)."
suggestion f"Collect more (resource) make tool_ toml_name]." return feedback, False, suggestion
# If all checks pass
28

---

return "Action executed successfully.", True,
def Coleeulee (state, action, knovledge-graph);
# Extract action details
action_name action .eroonggetfe ("actionn__aas name")
block_name - action.get geegggettaaaaaer ("args", ()) get ("block_name"
Initialize feedback, success, and suggestion feedback success True
suggestion
Rule 1: For action 'place' with block name 'sapling', if in_front' is 'table', the action will fail if action, name == "place" and alock_name == "sapling": if state. ("in_ffront") feedback "Action failed: Cannot place sapling in front of a table."
"table":
success False
suggestion "Try placing the sapling in front of different object, such as grass or dirt."
else:
feedback = "Action succeeded: Sapling placed successfully."
the action type is not 'place', count as success
else:
feedback "Action succeeded: No rules apply to this action type."
return feedback, success, suggestion
def CodeRule_2 2epeaule___placeeee (state, action, knovledge_graph).
# Check if the action is 'place' action iasionl'cctonn ['action_name '] 'place': block name
"ectoon[arge blmmkkkck_aam lamookk_mmme"
Check if the required material is in the inventory "aiven'invytooy! and state("inventon
if block_namd name []mbocck_name] > 0:
feedback = f"Successfully placed (block_nane)." success = True
suggestion
else:
feedback f"Failed to place (block,name)."
success = False
suggestion f"You need to have {block_name} in your inventory to place it. Consider collecting crafting it first."
else:
# If the action is not 'place', consider it successful feedback "Action executed successfully." True
suggestion
return feedback, success, suggestion
def SeceRule_3_place (state, action, knoxledge-graph):
# Extract action details action_name action get ("actioon_ name") block_name action gettongeetteeo ("args", () (").ge(("bloo ("block_nam
Check the action is 'place' and block_ name 'table' if action, name == "place" and block_ name "table":
29

---

# Check for existing 'table' in objjible.ojjects' near_objects'
visible_objects state get ceee"riibble.chjecttt"""" (])
near_ state get ("near_objectee" (])
Check if 'table' is in viseble_ table_ sable_in_visiblle visible visible_
(obj get ("type") "table" for obj
Check if 'table' is in near_objects
table_ine iable_in_near "table" in near objects
table is found in either, the action should not executed
if visible or hbll_in_neaae
feedback = "Action failed: A table is already present in the vicinity."
success False
suggestion "Reuse the existing table instead of placing new one."
return feedback, success, suggestion
the action is not 'place' or block name is not 'table', consider it successful
feedback "Action executed successfully." success True suggestion ""
return feedback, success, suggestion
def CodeRule_2_mine (state, action, knowledge_graph)
# Extract action details action_name action get ("action_name") block_name = action get ("args", (t) get ("block_name")
# Check if the action is 'mine' and the block_name is 'iron' action_name
"mine" and block_name # Check if the player has a "stone_pickaxe" or better in the inventory inventory state getnneenttory" ( eet.ssttnepiccaaeeeoee better any( tool inventory for tool ["stone, iron_pickaxe", "diamond_pickaxe"]
"iron":
if not pas_stone_pickaxe.u or_ttterr feedback = "Action failed: You need stone_pickaxe or better to mine iron." False
suggestion "Consider crafting or acquiring stone_ stone_pickaxe or better to mine iron." return feedback, success, suggestion
the action is 'mine' the block_ name 'iron', consider successful feedback "Action executed successfully." success True suggestion ""
return feedback, success, suggestion
def CodeRule_4_mine
action, knovledge_graph)
# Extract action details action_nama name action. get aetiion.et("actton... ("action_
block_name action. geaeeggetaaar # Check if the action is 'mine' and the block is 'stone if action_name == "mine" and blmck_name == "stone";
()) get "ibbocck_name))
30

---

# Check if 'xood_pickaxe better tool is in the inventory
inventory state has pickaee 'voo__iickaxe" in inventory
get get ("inventory", (
False
for tool in inventory:
if tool != 'xood_pickaze
response LLM_request if (f"is {tool} better than good_pickaze""
"nnly reply True False")
if response "True": better_tool.exist break
True
if has wwod pickaee or cetter_tool_exise feedback = "Action executed successfully." True suggestion ""
else:
feedback "Action failed: You need wood_pickaxe better tool to mine stone." success False
suggestion "Consider crafting or acquiring wood_pickare or a better tool to mine stone."
else:
feedback "Action executed successfully."
success True
suggestion
return feedback, success, suggestion
def CodeRule_5_mine (state, action, knovledge.graphh Extract action details action_name = action get ("action_name
block_name = action gelion.get((e ("args"" ()) get ("block_name")
Initialize feedback, success, and suggestion feedback auccess True auggestion =
Check the action is 'mine' and block_name 'plant' if action_ name == "mine" and block_name "plant": feedback "Action failed: You cannot mine plant." success = False suggestion "Consider mining other resources like 'tree' or 'stone' instead of 'plant'."
else:
feedback "Action executed successfully. "
return feedback, success, suggestion
D.2 Rules in ALFWorld
Action Rules for ALFWorld
Rule 1: For action 'take', if the item is not present in the specified location, the action will fail; Checking Method: Check if the 'obj' in 'action' is not listed under iocationn for the 'source' location in 'inital_stata
Rule 2: For action 'take', if the agent 's hand is already holding an item, the action will fail; Checking Method: Check if hand in 'inital_state' has non-null 'amem_name',
31

---

Rule 3: For action 'take', if the agent is not at the specified location, the action will fail; Checking Method: Check if current_position in 'inital_state does not match the 'source' location in 'action'.
Rule 2: For action 'open', if the target location is not in the list of reachable locations, the action will fail; Checking Method: Check if 'action arga. target deachable_locations:
in
Rule 7: For action 'open', if the target location is a fridge and the current position is not the same as the target location, the action will fail; Checking Method: Check if "action. args. 'fridge' and 'current_position" equal to Rule 10: For action 'open', if the target location is drawer and the current position is not the same the target location, the action will fail; Checking Method: Check if 'action args. target' starts with 'drawer' and 'current.position. location_name is not equal to 'action. arga. argge.ttetttt
target' 'action.args target'.
location_name" is not
Rule 3: For action 'go to', if the item specified by 'amem_name'
in 'item_in_hand' is None and the action's 'target' in the Scene Graph is not 'Unexplored' or does not contain the state's 'target_item in the list, the action will fail; Checking name is None, and validate if the 'target' in 'action' is either not labeled 'Unexplored' does not include the 'target_item
Method: Check if litem_in in fem_innhand, hand item_name'
in the Scene Graph.
Rule 2: For action 'heat',
the object to be heated is not in the tool, the action will fail; Checking Method: Check if the Stem_in_hand matches the object to be heated and is not placed in the tool location.
Rule For action 'put', if the
to put is not in hand, the action will fail; Checking Method: Check if 'item_name" 'item_in_hande of the "inital_state matches the 'obj" in action args.
Rule 4: For action 'put', if the current position is not the as the target location,
action will fail; Checking Method:
Check if 'location_name in 'current_position inital_state" matches the 'target in action args.
of the
Rule 2: For action 'use', if the object is not in the current location, the action will fail; Checking Method: Check if the object specified in the action stens_in_locationn state.
not present in the of the 'current_ppsitioor in the initial
Rule 4: For action clean, if the object be cleaned is not in hand, the action will fail; Checking Method: Check if the item_in_hand matches the object specified in the action.
Code Rules for ALFWorld
def CodeRule_1 1_ tatkeu take (state, action, acene_graph)
# Extract action details
action, name action. get aeneggeettnnneeee ("name")
action_args action get ("args", ())
# Check if the action is ttake"
if action_name == "take": obj action_args get (""bj""
source action_args get ("source""
# Check if the object is present in the specified source location
items_in_source state get ("items.in ttt ( get(
source, [])
if obj not in itens_in.source:
feedback f'Action failed: {obj} is not present in source}."
success False
32

---

suggestion f"Check the items in (source) and try taking available item." return feedback, success, suggestion # If the action is not 'take', consider it successful feedback "Action executed successfully." success True auggestion "" return feedback, success, suggestion
def
action, scene_graph);
# Extract the action name
action_name action. ("name") # Check the action is 'take' if action name == "take":
Check if the agent's hand is already holding item
item_in_hand state, get "ritem_in_hand ()) get item_name")
if item_ iin_in_har hom_in_handd
feedback "Action failed: The agent is already
holding item.
success False
suggestion "Put down the current item before taking another one."
return feedback, success, suggestion
# If the action is not take', consider it successful by default feedback "Action executed successfully." success True suggestion return feedback, success, suggestion
def ta if action n'mame = 'take':
action, acene_graph):
current_location statel' cattel/oureent.poittioo/. P
location_name ']
source_location action lI'
if current_location != source source_ location:
feedback = "Action failed: You are not the
specified location
take the item."
success False
suggestion f"Move to (source__ocation)) before attempting to take the item." return feedback, success, suggestion the action is not 'take', consider it successful by default
feedback "Action executed successfully."" auccess True suggestion "
return feedback, success, suggestion
def RodeRule___open((ee if action['name')
action, scene_graph);
'open'; target action action[Pargs [Parg if target not in state reachabee locations ']:
feedback = f"Action failed: The target location '{ target} is not reachable." success False
suggestion f"Try moving closer '(targt}) before attempting to open it." return feedback, success, suggestion # If the action is not oopen', consider it successful by default
feedback "Action executed successfully." auccess True suggestion =
33

---

the return feedback, success, suggestion
def 7odeRule_7_opent (state, action, scene-graph):
# Extract necessary information from state and action current_position
cuatel"current_postta (" ["location_name"" action_name action [" target action ["arga"] get ("target", "")
# Check if the action is 'open' and the target is a fridge if action_nes name == "open" and "fridge" in target:
Check if the current position is not the target location if current_poeition feedback "Action failed: You must be at the fridge to open it."
target:
False
suggestion f"Move to (target) before trying to open it." return feedback, success, suggestion
the action is not 'open' or the rule does not apply, consider successful
feedback "Action executed successfully." auccess True suggestion "
return feedback, success, suggestion
def CodeRule_10..m 10_ ooanllee___ppeessteteeeeeeeee (state, action, scene_graph):
# Extract necessary information from state and action current.position state caateelcurrent.pooiii action_name action ["name"" target_location = action ("aloon"arree" ["args"] get ("target", "")
""""locationnnaa
# Check if the action is 'open' and the target is drawer action_name "open" and target_ startswith( draver");
# Check if the current position is not the same as the target location
if current_position feedback "Action failed: You must be the drawer to open it." success False suggestion I"Move to (target_location)) before trying open it." return feedback, success, suggestion
carget_location
# If the action is not 'open' the rule does not apply, consider auccessful feedback "Action executed successfully." success True suggestion feedback, success, suggestion
def CodeRule_ 3_go_ to action, scene_graph) # Extract necessary information from state and action item_in inndand state ("itee_imi_nnaaaa ["item_aaam target_iten state state ["target_item"]] action_name action ["ammem]
# Check if the action is 'go_to' if action_name =- "go to";
target_location
action [" arargs" ["target"]
Check item in hand is None if item in hand is None
34

---

Check if the target location is not 'Unexplored and does contain the target item target_location in (rapeergapalllatiiiooii location_items scene ("rcaattp"""occtttoon===""] target_location
if "Unexplored" not in location_items and not any target_item in item for item in location_items)
# Search for a location containing the target item
aaaaiion uexploreed []
None
for location, items in scene_ graph [" ["locations
"]. items()
if any (target_iten in item for item in items):
location_ withnnwwthhtee waath_target_iiee item = location break # Stop searching once
location with the target item is found
"Unexplored" items: lexppooredlocations append (location)
Prepare suggestion string
if location_with.......ooo.. wittrion_.thhtttrgtttiteee
suggestion f"According to scene graph.
Please to the location containing the target item: { wtt.ionnttttttttttttttttt suggestion =
elif oiff ueexporre__ocatioon::
"According scene graph
"Please explore the following
unexplored locations:
jojn " (mexplorod
)
else:
suggestion "No valid locations found that contain the target item or are unexplored."
feedback AAction failed: There is no Carget_itenl in (target_location) success False return feedback, success, suggestion
the action is not 'go_to' or the conditions are not met, consider it successful feedback "Action executed successfully."" True suggestion ""
return feedback, success, suggestion
def CodeRule_2_heat (state, action, scene_graph):
# Extract necessary information from the state and action action_name action. get ("name") action_args objato_heat args.get("toon) tool action_ aeenaarrgs.e. # Check if the action is 'heat'
sesion.get(aaae arccon_argg.get(""0)
())
action_name
"heat":
# Check if the object to be heated is in hand and not in the tool location
35

---

item_in_hand = state. get ("item_in_has ...get item_name")
current_position state get ( ("current-ositionn, +)) get ("
docation_name!!
if stem_in_hand -- obj abj_to_heat and current_position !"
tool:
feedback f"Failed heat {obj heath. It must be placed in [tool} to be heated."
success = False
suggestion f"Place {obj_to_heat) in {tool} before
heating.
return feedback, success, suggestion
the action is not 'heat', consider it successful feedback "Action executed successfully." success True suggestion return feedback, success, suggestion
def CodeRule_2_put (state, action, scene_graph):
if action name - 'put':
obj_to_put action['args []['
stem_in_hamd state["item,ing
'] item_name ']
if obj_to_put != fhem_in_hande
feedback = f"Action failed: [obj_to_pust is not in hand
success False
suggestion f'Ensure you have {obj_to_put} in hand before attempting to put it." return feedback, success, suggestion
the action is not 'put', consider it successful by default
feedback "Action executed successfully." auccess True suggestion
return feedback, success, suggestion
def CodeRule_4_putt( (state, action, acene_graph):
if action['name" == 'put':
current_location state current.
location_name ']
target_location a
if current_location I= target_location.
feedback = "Action failed: You must be the target
location to put the item." success False
suggestion f"Move to (target_locatiin)) before attempting to put the item." return feedback, success, suggestion the action is not 'put', consider it successful by default
feedback "Action executed successfully." True suggestion "
return feedback, success, suggestion
def
action, scene_graph);
if action['name") [[
'use':
obj action action['args a'args current_location state s'atel' o' cuureett.ppsitioo [] location_name
Check if the object is in the current location state "aatee"iees..nnocaatiiio if obj locaaton, [): current
(
feedback = f'Action failed: {obj} is not in the current location fourrent__ocation).
36

---

success = False
suggestion fove the location where {obj} is present or bring {obj} to the current location. return feedback, success, suggestion # If the action is not 'use', consider it successful feedback "Action executed successfully." True
auggestion return feedback, success, suggestion
def CodeRule__.cleannstaaa Extract the action name and arguments action_name name action. get ("name") args action. get ("args", ())
action, scene_graph);
Check if the action is 'clean' action_name "clean":
# Get the object to be cleaned from the action arguments obj_to_clean action_args.get (otboo"
the item currently hand Stem_in_hand state. gett aattmmin_hande ()) get (" item_name")
Check if the object to be cleaned is in hand if obj_to_clean item_in
feedback = f'Action failed: {obj_to_clean) is not in hand
success False
suggestion f"Please ensure tobj_to_clean) hand before cleaning.
return feedback, success, auggestion
# If the action is not 'clen'' consider it successful feedback "Action executed successfully. success True auggestion return feedback, success, suggestion
37

---

Algorithm 3 Greedy Algorithm for the Maximum Coverage Problem in Eq. (5)
1: Input: Dine {diee 0h N Roode
(r1 2
r-0dc) ). Ozjz Indicator matrix where
dij = 1 if dime rsode otherwise dij
2: Initialize R 0, Deov
3: while Deov * DDine do
4: For each nule pçode Roode compute:
gain(r
|D
U 1OT E D agj = 131 D
5: Get the index of radde with the largest gain, i.e.,
+ arg max gode
6: if gain (riasle) = 0 then
7: Break (Terminate if no rodee cover any additional gine
8: end if'
9: Add pioue the selected rules set:
R + R2 U (1+e0 code
Update the covered set:
Doo0 Deo0 180h E Dine a.- = 1}
11: if | = - then
12: Break (Terminate if hit the limit
13:
end if
end while
15:
Output: Set of selected rules R
E Greedy Algorithm
We implement the following Algorithm 3 to solve the maximum coverage problem
F
Experiment Details
Environment Licenses. Both the Mars and ALFWorld environments are released under the MIT License, and our experiments fully comply with their respective licensing terms. All code, assets, and interaction logs used this work adhere the usage guidelines provided by the original environment authors.
Compute Resources. All experiments were conducted a local Ubuntu 20.04.6 workstation with an Intel i5- 13600KF CPU, 64GB RAM, and single NVIDIA RTX 4060 Ti 16GB GPU. Each experimental run (e.g., evaluation trial) typically completed within 5-10 minutes. The evaluation required less than 24 hours in total.
F.
Mars
Task Details. Mars is an open-world environment designed for situated inductive reasoning, where agents must actively interact with their surroundings, induce generalizable rules, and apply them to achieve specific goals. Unlike traditional environments that rely on pre-existing commonsense knowledge, Mars introduces counter-commonsense mechanisms by modifying terrain distributions, survival settings, and task dependencies.
The agent's goal unlock various achievements, such as:
Collecting Achievements: Collect Coal, Collect Diamond, Collect Drink, Collect Iron, Collect Sapling, Collect Stone, Collect Wood.
38

---

Crafting Achievements: Make Wooden Pickaxe, Make Stone Pickaxe, Make Iron Pickaxe, Make Wooden Sword, Make Stone Sword, Make Iron Sword.
Placing Achievements: Place Table, Place Furnace, Place Plant, Place Stone.
Survival and Combat Achievements: Kill Skeleton, Kill Zombie, Kill Cow, Eat Plant, Wake Up.
However, achieving these goals requires adaptive reasoning since the default assumptions about item acquisition and crafting may no longer hold. For example: Mars introduces modifications challenge agents' reliance on prior knowledge. These modifications fall into three categories: Terrain, Survival Settings, and Task Dependencies, which can be combined to create diverse worlds.
Terrain Modifications:
Distribution: Resources appear in unexpected locations (c.g.. diamonds in sand, coal near grass).
Effects: Terrain properties change (e.g., lava is safe, grass is harmful, mining stone gives wood).
Survival Setting Changes:
Entity Behavior: Cows may be aggressive, zombies passive, skeletons melee instead of bows.
Food and Health: Eating cows might reduce health, drinking lava could restore it.
.
Task Dependency Alterations:
Resource Collection: Mining may yield unexpected materials (e.g., trees drop iron, stone gives coal).
Crafting Changes: Items require different materials (e.g., tables need diamonds, pick- axes need iron).
Placement Rules: Placing objects may consume extra resources (e.g., tables require two diamonds).
By combining these modifications, Mars forces agents to leam dynamically, making pre-stored knowledge unreliable and requiring real-time adaptation.
Each episode in Mars generates unique 64 64 grid-based world where agents operate under partial observability (7 * grid view). The environment ensures task achievability by maintaining resource balance, enforcing supply constraints, and allowing procedural rule induction.
Baselines. We compare our method against ReAct [37], which interleaves reasoning and actions; extension of ReAct with self-refiection; Skill Library [32], adaptation of JARVIS-1 39 and Voyager 45) that stores successful plans for in-context learning; and IfR [32],
Reflexion 38
which extends Skill Library with induction-from from-reflection module that derives and stores game rules for adaptive pocision-maaing These methods' planning framework and components are shown in Table
Table 6: Comparison of baselines' planning framework with different components.
Method
Reasoning
Experience Reflector
Memory/Skill Library
Rules
Knowledge Graph
Scene Graph
ReAct
37
Reflexion 38
Skill Library IIR 32
WALL-E E (ours)
Method Setup. We use GPT-4 as the backend for our method. Following the Mars setup, we run learning episodes and report results for L.LM-based methods and RL-based baselines averaged over and 20 trials, respectively.
39

---

F.2 ALFWorld
Task Details. ALFWorld is a text-based household simulator aligned with the 3D ALFRED bench- mark [42]. Each episode gives the agent natural- language goal (e.g., "put the cooled lettuce on the countertop"), Agents issue high-level text commands such as "go to countertop 1," "take lettuce from countertop 1," or "cool lettuce I with fridge 1," and receive textual observations describing their egocentric view, nearby objects, inventory contents, and action results. The agent's internal state the running history of these observations.
ALFWorld includes six task families: Pick & Place, Pick & Examine, Cool & Place, Heat & Place, Clean & Place, Pick-Two & Place, mirroring the ALFRED dataset's diversity of object manipulations and state changes.
Method Setup. We use GPT-3.5- Instruct as the backbone model and perform neurosymbolic learning on the training set. Additionally, to enhance neurosymbolic learning, developed scripts to convert the natural language dialogue history and action information into a structured JSON format, as illustrated in Appendix C.2. Evaluation is conducted on all 134 FLFWorld test tasks, following the evaluation protocol used by prior ALFWorlld agents A.2
F.3 Experiment Design for Effectiveness of NeuroSymbolic Learning
We set the number of learning episodes to 5. After each episode, the model, equipped with latest learned code rules or skill library, is tested on the testing set. The cover rate quantifies the extent to which the code rules derived from the neurosymbolic learning process address the LLM's Mss failed predictions. Specifically, represents the probability that mispredicted transitions by the LLM are correctly handled by the learned code rules.
To assess the alignment between the LLM-based world model and the actual environment, first identify transitions where the LLM fails to make accurate predictions. This is achieved by utilizing an unaligned LLM world model to generate predictions for trajectories obtained from the test set. The discrepancies between the predicted observation 0z+-1 and the actual observation 0++1 are compiled into a of mispredicted transitions. These mispredictions highlight areas where the LLM world model does not align with the environment's 's dynamics.
Subsequently, the learned code rules at each iteration are evaluated against the mispredicted transitions dataset to determine their effectiveness in correcting these mispredictions. If a code rule successfully predicts the outcome of previously mispredicted transition, it demonstrates that the code rule effectively addresses the I.M''s failure in that instance. The cover rate then calculated as the ratio of correctly addressed mispredictions to the total number of mispredicted transitions:
Cover Rate Number of Mispredictions Addressed by Code Rules
(12)
Total Number of Mispredicted Transitions
A higher cover rate indicates that the neurosymbolic learning process effectively enhances the aligmment of the LLM world model with the environment, thereby improving the overall accuracy and reliability of the agent's 's planning.
G Limitation and Future Work
Our current neurosymbolic learning framework exhibits two limitations. First, it focuses on learning transition-level code rules that determine whether a transition succeeds or fails. While effective for aligning local dynamics, these code rules lack the expressiveness to capture more abstract, long- horizon planning constraints or temporally extended dependencies. Future work will explore inducing higher-level symbolic structures-udc more complex reasoning and structured planning.
hierarchical knowledge or temporal logic-thal that can support
Second, the framework assumes deterministic dynamics, meaning that each action in given state is expected to produce a single. predictable outcome. However, many embodied environments involve inherent stochasticity-fot for instance, actions in Mars may occasionally succeed or fail depending on contextual factors like time of day or agent health. Our current neurosymbolic learning framework does not account for such probabilistic variations, leading to potentially misleading failure classifica- tions. Extending the framework to support probabilistic transition modeling is a key direction for improving robustness.
40


Reviews:
[{"review_id": "tz29id1oNr", "reviewer": "Reviewer_8TpS", "summary": "This paper proposes a neural-symbolic framework for building world-model-based LLM agents. Specifically, the authors prompt the LLM to extract symbolic knowledge from collected trajectories by inferring deterministic rules and constructing knowledge graphs. Also, the LLM attempts to reconstruct complete observations using scene graphs. This symbolic knowledge is then summarized into executable code functions, which are deduplicated and pruned to maximize coverage of both successful and failed transitions. The resulting world model supports the agent’s decision-making via model predictive control, with inference and rule learning executed in alternating steps.\n\nExperiments conducted on Mars and ALFWorld demonstrate that the proposed method outperforms RL-based approaches like DreamerV3, as well as LLM-based rule-learning methods such as IfR.", "questions": "In addition to the questions raised above, I’m curious why Table 2 doesn’t include IfR.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "# Strengths\n\n* Compared to existing LLM-based rule learning methods, this work introduces several key innovations by integrating deterministic rules, knowledge graphs, and scenario graphs into executable code functions.\n\n* The experimental results are promising, and the ablation studies are thorough.\n\n* The paper is well-written and easy to follow.\n\n# Weaknesses\n\n* As noted in the limitations section, this work only considers rules for deterministic transitions (success or failure). However, real-world environments are typically dynamic and governed by probabilistic rules.\n\n* The author claims that the proposed code-based rules offer stricter enforcement than natural language rules in previous work. While it is true, code-based rules are also more brittle and less generalizable to unseen scenarios. For instance, the model learns that an agent cannot pick up an item unless the resource is on a \"table.\" When encountering a similar situation involving a \"desk,\" the rule fails to apply. But a natural language rule might still help the LLM generalize, thanks to commonsense knowledge that desks and tables serve similar functions.\n\n* If I undestand correctly, the collected rules are all about successful transition, which is indeed an important part of the world dynamics. However, a compact world model should capture a more general abstraction of transition dynamics, enabling the agent not only to predict whether an action will change the state but also to evaluate how likely the action is to achieve a goal (i.e., which action has the highest success rate).", "was_revised": true, "final_justification": "after reading the rebuttal and the other reviews, I’m still leaning towards accepting the paper because has reasonable novelty, good empirical results, and no major flaws. that said, I haven’t raised my score because I’m still not fully convinced that code-based abstraction is much more effective than natural language, even in complex, highly open domains. so, I’ll be keeping my original score.", "rebuttal": "Thank you for your careful assessment and valuable comments. We are pleased to respond to your concerns and further clarify the design and impact of our approach below.\n\n**1. Deterministic Rule Limitation vs. Probabilistic Dynamics:**\nOur rule learning focuses on the deterministic part of dynamics because: (1) it is the backbone of many environments and complementary to the stochastic part that may have been captured by LLMs; (2) it enables stable and verifiable modeling of dynamics; and (3) many seemingly probabilistic outcomes collapse to deterministic when conditioned on the reasoning results of LLM on trajectories. For example, crafting an iron pickaxe may seem uncertain until one realizes that a furnace is a necessary precondition.\n\nAs noted in §Limitation and Future Work, extending our framework to handle explicit probabilistic transitions by allowing light-weight LLM finetuning is an important direction for our future work.\n\n**2. Code-Based vs. Natural Language Rules:**\nCompared to natural language rules, code-based rules avoid possible ambiguity and lead to more precise and verifiable generalization. This is essential for stable and controllable decision-making. Moreover, they are also easier to be followed by LLMs. \n\nUnlike natural language rules, our approach leverages LLM-induced symbolic patterns—e.g., abstracting both \"table\" and \"desk\" as functionally equivalent—to generalize safely without ambiguity. In contrast, natural language rules may lead to vague interpretation and reasoning that cause unsafe or inconsistent behaviors—an unacceptable risk in decision-making systems.\n\n**3. Extending the Method to Capture More General Abstraction of Transition Dynamics:**\nWe start with binary success/failure rules because they provide compact, straightforward, and verifiable instructions that various LLMs can easily understand and follow without further finetuning. The effectiveness is reflected by the strong long-horizon results in Tables 1–2. In §Limitation and Future Work, we plan to extend the framework to richer types of abstractions for future work.\n\n**4. Missing IfR in ALFWorld Comparison (Table 2):**\nIfR [1] is not included in the ALFWorld Comparison (Table 2) because it was developed and evaluated specifically for Mars environment and has not been applied to any other environments. Direct comparison would be unfair and potentially misleading.\n\n[1]. Tang, et al. \"Mars: Situated inductive reasoning in an open-world environment.\" NeurIPS 2024."}, {"review_id": "h3d50D6nS5", "reviewer": "Reviewer_KNBZ", "summary": "The authors introduce WALL-E, an LLM agent that is augmented with symbolic knowledge (in the form of action rules, and knowledge/scene graphs). The LLM uses its existing knowledge to plan actions, which are run in an MPC loop using both LLM knowledge and code rules derived from the symbolic knowledge. The agent is tested in two challenging environments, where it achieves state-of-the-art (SotA) performance.", "questions": "The claim is that WALL-E's \"focus is not on absolute performance\", but do you have any comments on what is preventing it from reaching DreamerV3's scores? Is it merely an issue of environment interactions, or are there some limitations to the neurosymbolic framework?", "limitations": "Yes.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "This is a clearly-written paper that introduces a novel LLM agent framework that achieves SotA performance on challenging domains. Although there are many components to the neurosymbolic agent, these are explained well, and are explored in the experiments/ablations. The method achieves strong performance, and is sample-efficient. LLM agents may be able to reason about these environments in a zero-shot fashion, but the ability to override this via neurosymbolic learning is very useful for generalisation.\n\nAs the authors admit, one drawback is the inflexibility of the symbolic rule structure. One aspect of the paper's results/claims that is a little unclear is about whether WALL-E is truly SotA. The authors acknowledge that \"DreamerV3 achieves higher absolute rewards\" - but with significantly more environment interactions. Table 1 is supposed to highlight the best results in bold, but WALL-E's results are bolded despite being worse than DreamerV3's. The authors could do a better job of addressing the different evaluation regimes.", "was_revised": true, "final_justification": "I have read the other reviews and the authors' rebuttals and believe that this paper still deserves a score of 5. In particular, I do not share as many concerns about the nature of the system and common criticisms against neurosymbolic methods - the results are strong and it is up to future work to show simpler methods that can perform as well/extending this system to incorporate shortcomings such as probabilistic dynamics.", "rebuttal": "Thank you for your thoughtful review and constructive comments. We're pleased that you recognized the potential of our approach, and we address your questions and suggestions in detail below.\n\n**1. Inflexibility of Symbolic Rule Structure:**\nOur symbolic rule structure is highly flexible, as the rules are automatically derived by an LLM from previous experiences without relying on any domain-specific engineering. As noted in the main text (Lines 69–74, 100–103) and Appendix (Lines 455–459), WALL-E performs inductive reasoning by an LLM to extract symbolic rules directly from experiences—without handcrafted prompts, task-specific modeling, or any fine-tuning—enabling fast and scalable adaptation across diverse environments.\n\n**2. Clarify WALL-E's SotA Claim in Table 1 Given DreamerV3's Higher Score:**\nWALL-E is the **SOTA training-free method** that does not require any finetuning of LLM parameters. While DreamerV3 achieves higher absolute rewards, it is trained for 1 million environment steps. Our bolding in Table 1 reflects WALL-E's leading performance within the training-free (LLM-based) category. DreamerV3's results are included for reference only, to highlight the contrast in sample efficiency and adaptation speed. \nWe will revise Table 1's caption to make the evaluation and comparison setting clearer.\n\n**3. Why WALL-E Underperforms DreamerV3:**\nThis is because WALL-E does not finetune any model parameters and focus on high-level planning, while DreamerV3 fully finetunes a policy for 1M steps to produce low-level actions. These gaps in the training and modeling lead to their difference in performance. This gap also motivates hierarchical planning/control in recent works (LLM planner + trained controller) [1,2]. However, in our submission, we keep using the same setup as most LLM agent baselines for fair comparisons, where the LLM is not finetuned for specific tasks and takes the full control of the agent.\n\n[1]. Shi, et al. \"Hi robot: Open-ended instruction following with hierarchical vision-language-action models.\" ICML 2025.  \n[2]. Wang, et al. \"Describe, explain, plan and select: interactive planning with large language models enables open-world multi-task agents.\" NeurIPS 2023."}, {"review_id": "T4nLK5XwMt", "reviewer": "Reviewer_G6RY", "summary": "This paper proposes WALL-E, a training-free framework to achieve \"world alignment\" of LLM agents. \nWALL-E learns environment-specific symbolic knowledge complementary to LLM’s prior knowledge. \nThe symbolic knowledge covers action rules, knowledge graphs, and scene graphs, which are extracted by comparing agent-explored trajectories and model-predicted ones. The symbolic knowledge can be converted into executable codes to regulate LLM agents’ policies through an efficient look-ahead optimization process. Experiments show that WALL-E significantly outperforms existing methods in open-world challenges like Mars and ALFWorld.", "questions": "- The code pruning method is quite confusing. Can you provide a more intuitive explanation of this method?\n\n- How do you ensure that the extracted code rules are correct and executable?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "## Strengths\n- This paper proposes a training-free framework, WALL-E, to achieve world alignment of an LLM through a meticulously designed neurosymbolic learning approach.\n\n- The proposed method significantly outperforms existing baselines on challenging benchmarks like Mars and ALFWorld.\n\n## Weaknesses\n- The writing needs to be improved. There are too many symbols involved in describing this framework. It would be better if the number of symbols can be reduced or these symbols can be marked in corresponding positions of the presented figures for reference.\n\n- The code rule pruning method is hard to understand.\n\n- The whole framework is quite complex, with LLMs playing different roles in summarizing action rules, constructing the knowledge graph,  translating symbolic knowledge into code rules, and other mechanisms for scene graph construction and code rule set pruning.", "was_revised": true, "final_justification": "- This paper proposes a training-free framework, WALL-E, to achieve world alignment of an LLM through a meticulously designed neurosymbolic learning approach, though the method is a bit complex for understanding and implementation.\n\n- The proposed method significantly outperforms existing baselines on challenging benchmarks like Mars and ALFWorld.\n\n- The experiments are extensive and solid.", "rebuttal": "Thank you for taking the time to evaluate our paper and provide thoughtful feedback.\nWe value and appreciate your comments! In the following, we provide detailed responses and clarifications to each of your concerns.\n\n**1. Symbol Overload and Readability:**\nWe will remove non-essential symbols and annotate key ones directly in the figures to improve the clarity and readability.\n\n**2. Intuitive Explanation of Code Rule Pruning:**\nIntuitively, Code Rules Pruning is to select a small subset of extracted rules that can cover and correct most mispredictions by LLM only. We formulate it as a classic maximum coverage problem (Eq.5 on Page 5) that can be addressed by the greedy algorithm (Alg.3 on Page 30). Specifically, we first collect all transitions where the LLM-only's predictions failed ($\\mathcal{D}^{\\text{inc}}$) —these mark LLM's misalignments with the real environment. Then, at each step, we select the rule that can cover/correct the most remaining mispredicted transitions and add it to the subset, remove its covered transitions from future consideration, and repeat the above procedure until the number of selected rules reach the budget size $l$. This yields a compact yet high‐impact rule set that best aligns the LLM's priors with environment dynamics.\n\n**3. WALL-E's Design is Complex:**\nWALL-E's architecture is modular by design—it is principal and general. In WALL-E, the world model is an LLM augmented by a set of symbolic knowledge extracted from previous experiences. The agent planning is acheived by model-predictive control, in which an LLM agent interact with the world model for multiple rounds to determine the best action(s) to take in the following step(s).\n\nThe idea is to use LLMs for inductive reasoning of symbolic knowledge and proposing actions, while lightweight program modules encode symbolic knowledge and ensure transparent, verifiable execution of the rules. This follows a growing best practice in agentic AI: leveraging LLMs for generalization and abstraction, while using structured programs to ground and constrain LLMs' outputs, reducing hallucinations and uncertainty of LLM-only predictions. Compared to LLM-only or rigid rule-based systems, this neuro-symbolic approach offers greater robustness, efficiency, interpretability, and reliability, by combining the complementary strengths of LLMs and symbolic systems. Moreover, this is achieved without any parameter training or intensive human supervision.\n\n**4. Verification of Code Rule Correctness, Executability:**\nWe ensure the correctness by verifying each candidate code rule on ground-truth transitions from the real trajectory $\\tau^{\\text{real}}$ and discarding any rule that leads to wrong predictions. For executability, any rule that raises an execution error during any run is also removed. This filtering mechanism is simple but rigorous. We will further clarify this in \"Stage 4: Code Rules Pruning.\""}, {"review_id": "ONRRKnw4O6", "reviewer": "Reviewer_Cnsg", "summary": "In the environment of Mars and ALFWorld, this paper propose propose a training-free method that learns an environment’s symbolic knowledge complementary to large language models (LLMs). Moreover, this paper propose propose an RL-free, model-based agent “WALL-E” through the model-predictive control (MPC) framework. Experimental results show substantial improvements with reported increases in success rates and scores.", "questions": "1. When translating symbolic knowledge into executable code, how is the correctness of the generated code verified (syntax, logical consistency, and intended functionality)?\n2. How long does the symbolic knowledge learning phase take, and how many trajectories are required for effective learning? Additionally, what is the storage overhead associated with maintaining the symbolic knowledge base?", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "## Strength\n1. The paper provides a thorough description of the proposed method, making it easy to follow.\n2. The paper conducts experiments across diverse environments (Mars and ALFWorld), which validates the effectiveness and generalizability of the proposed method. In addition, the paper includes comprehensive ablation studies that help to isolate and assess the contributions of individual components.\n\n## Weakness\n1. The approach of using LLMs to summarize rules and knowledge is intuitive and has been explored in other domains. \n2. Translating symbolic knowledge into executable code limits the generalizability of the proposed approach. Although the paper demonstrates its effectiveness in a 2D environment, the reliance on API calling (i.e., interacting with the environment through executable code) diverges from human-like behavioral patterns. This interaction paradigm poses significant challenges for adaptation to 3D environments or real-world settings, thereby constraining the contribution of the work to virtual 2D agent scenarios.", "was_revised": true, "final_justification": "I have read the author's rebuttal. Most of my concerns have been addressed, and I will keep my score.", "rebuttal": "Thank you for your valuable and detailed review. We're glad you engaged deeply with our work, and we address your concerns point by point below.\n\n**1. Novelty Beyond LLM-Based Rule/Knowledge Summarization:**\nHere are the distinctive novelties of WALL-E: (1) a training-free world alignment framework that aligns pretrained LLMs with a specfic environment's dynamics, which leads to a precise neuro-symbolic world model; (2) an effective symbolic knowledge learning scheme (using LLM as an inductive reasoner); (3) a world model-grounded agent with efficient model-predictive control (MPC, using LLM as the optimizer). \n\nThese novelties are revolutionary to existing world modeling strategies as we only need to focus on the complementary symbolic and verifiable knowledge that cannot be covered by pretrained LLMs, which are much more efficient to learn than training a world model from scratch. In our method, we do not require any parameter training. We also replace the low-level optimization in classical MPC with high-level reasoning by LLMs under the symbolic constraints. This makes the planning more precise for long-horizon challenging tasks. \n\n**2. The Reliance of Code Rules on API Calling Limits Generalization Beyond 2D Environments:**\nExecutable code via API calls is a strength rather than a limitation—and it is increasingly becoming a popular design in recent 3D-embodied robotic systems, especially for hierarchical agents where high-level LLM planners call low-level controllers via API calls [1–4]. \n\nOur neurosymbolic MPC agent operates at a high level of abstraction (e.g., code rules execution), agnostic to the environment's dimensionality; the API layer serves to map these abstractions to environment-specific low-level controllers. This setting is important for maximizing the potential of pretrained LLMs as they are good at high-level inductive reasoning. \n\nAs a result, adapting our method from 2D to 3D or real-world scenarios only requires updating the API/controller bindings, without changing the neurosymbolic learning or planning mechanism.\n\n**3. Verification Code Rule Correctness (Syntax, Logic, Functionality):**\nWe verify the correctness by discarding any rule that contradicts ground-truth transitions from real trajectories $\\tau^{\\text{real}}$. For Syntax, we filter out any rule that raises an execution error during any run. For logical consistency and intended functionality, we do not manually inspect rules; instead, we rely on empirical validation: only rules that are correct, executable, and successfully cover/correct many LLM-mispredicted transitions are retained—these form the output of \"Stage 4. code rules pruning\". In practice, this pruning process consistently yields logically sound and functional rules (see the learned rules in Appendix D). We will further clarify this in \"Stage 4: Code Rules Pruning\" and the Appendix.\n\n**4. Efficiency and Storage of Symbolic Knowledge Learning:**\nSymbolic knowledge learning is fast (<1h30min in Mars; <30min in ALFWorld), data-efficient (5 trajectories for Mars; 30 for ALFWorld), and lightweight—requiring <50kB of storage per environment (JSON format).\nMars features high degrees of freedom and task complexity (≈240 steps/trajectory), so 5 trajectories reveal diverse transitions. In contrast, ALFWorld tasks are simpler and shorter (≈18 steps/trajectory), and categorized into different types, so we sample 5 trajectories per type across 6 types to ensure a broad coverage.\n\n[1]. Liang, et al. \"Code as Policies: Language Model Programs for Embodied Control.\" ICRA 2023.  \n[2]. Mao, et al. \"Robomatrix: A skill-centric hierarchical framework for scalable robot task planning and execution in open-world.\" arXiv 2024.  \n[3]. Belkhale, et al. \"RT-H: Action Hierarchies using Language.\" RSS 2024.  \n[4]. Shi, et al. \"Hi robot: Open-ended instruction following with hierarchical vision-language-action models.\" ICML 2025."}]

Author rebuttal:

Meta-review: This paper proposes a framework of LLMs that extract rules from
trajectory data, assembles them into a world model, and then uses that
world model in an MPC style to produce a policy.  It achieves SOTA
performance on a couple agent benchmarks, mars and alfworld.

The strength of the paper is its interesting and novel architecture
and the corresponding SOTA results it gets using that architecture.
Although it is complicated, the paper explains it clearly enough to
see the idea.

Additional weaknesses beyond the complexity are the potential
brittleness/inapplicability to stochastic systems.  I think the authors
fairly state that this is a starting point in an interesting direction and
those weaknesses can be tackled in future work and thus recommend it be
accepted.
Decision: Accept (poster)