One Life to Learn: Inferring Symbolic World Models for Stochastic Environments from Unguided Exploration
Authors: Zaid Khan
Archiki Prasad
Elias Stengel-Eskin
Jaemin Cho
Mohit Bansal
Venue: ICLR 2026 Poster (2026)
Keywords: world modeling
programmatic RL
probabilistic program
symbolic rule learning
intrinsically motivated and open-ended learning
Primary area: foundation or frontier models, including LLMs

Abstract: Symbolic world modeling is the task of inferring and representing the transitional dynamics of an environment as an executable program. Previous research
on symbolic world modeling has focused on simple, deterministic environments
with abundant data and human-provided guidance. We address the more realistic and challenging problem of learning a symbolic world model in a complex, stochastic environment with severe constraints: a limited interaction budget
where the agent has only “one life” to explore a hostile environment and no external guidance in the form of human-provided, environment-specific rewards or
goals. We introduce OneLife, a framework that models world dynamics through
conditionally-activated programmatic laws within a probabilistic programming
framework. Each law operates through a precondition-effect structure, allowing
it to remain silent on irrelevant aspects of the world state and predict only the attributes it directly governs. This creates a dynamic computation graph that routes
both inference and optimization only through relevant laws for each transition,
avoiding the scaling challenges that arise when all laws must contribute to predictions about a complex, hierarchical state space, and enabling accurate learning
of stochastic dynamics even when most rules are inactive at any given moment.
To evaluate our approach under these demanding constraints, we introduce a new
evaluation protocol that measures (a) state ranking, the ability to distinguish plausible future states from implausible ones, and (b) state fidelity, the ability to generate future states that closely resemble reality. We develop and evaluate our framework on Crafter-OO, our reimplementation of the popular Crafter environment
that exposes a structured, object-oriented symbolic state and and a pure transition function that operates on that state alone. OneLife can successfully learn
key environment dynamics from minimal, unguided interaction, outperforming a
strong baseline on 16 out of 23 scenarios tested. 
We also demonstrate
the world model’s utility for planning, where rollouts simulated within the world
model successfully identify superior strategies in multi-step goal-oriented tasks.
Our work establishes a foundation for autonomously constructing programmatic world models of unknown,
complex environments.

Published as a conference paper at ICLR 2026
ONE LIFE TO LEARN: INFERRING SYMBOLIC WORLD MODELS FOR STOCHASTIC ENVIRONMENTS FROM UNGUIDED EXPLORATION
Zaid Khan
Archiki Prasad
Elias Stengel-Eskin"
Jaemin Cho
Mohit Bansal
UNC Chapel Hill
UT Austin
JJhnns Hopkins
(zaidkhan, archiki, mbansal @cs. unc.edu, esteng@utexas, jaemin@jhu. edu
ABSTRACT
Symbolic world modeling is the task of representing an environment's transition dynamics executable program. Previous research symbolic world mod- eling has focused on simple, deterministic environments with abundant data and human-provided guidance. We treat symbolic world modeling problem of autonomous scientific discovery in a complex, dangerous, and stochastic environ- ment. An agent has "one life" explore the environment, and must infer the dynamics that govern it from observation alone, without external guidance in the form of environment- specific rewards, goals, or prior knowledge. Our framework, ONELIFE, decomposes the transition function into mixture of small, probabilis- tie laws, each with a precondition that determines when it applies and an effect that governs narrow slice of the state. Laws activate only when relevant, forming a computation graph specific to each transition. Gradients flow only through active laws, giving precise credit assignment over large, hierarchical state space. To support evaluation, we contribute Crafter-OO, a reimplementation of Crafter that exposes structured, object-oriented state and a pure transition function, together with an evaluation suite of 30+ scenarios and evaluation protocols for measuring (a) state ranking, the ability distinguish plausible future states from implau- sible ones, and (b) state fidelity, the ability to generate future states that closely resemble reality. ONELIFE learns key dynamies from single unguided episode, outperforming a strong baseline on 16 of 23 scenarios tested. We also demonstrate the world model's utility for planning, where rollouts simulated within the world model successfully identify superior strategies in multi-step goal-oriented tasks. Our work establishes a foundation for autonomously constructing programmatic world models of unknown, complex environments.
INTRODUCTION
World modeling is a critical task in artificial intelligence, providing an agent with functional un- derstanding of its environment's underlying dynamics. By learning world model, an agent can predict the outcomes of its actions without having to actually interact with the real world. One line of research world modeling aims to learn symbolic world models via program synthesis (i.e., representing worlds models with code) with a view towards building representations that are inter- pretable, editable, and verifiable by humans.
While such approaches have been successful in environments with a limited number of discoverable mechanics and low stochasticity (Piriyakulkij et al., 2025; Tang et al., 2024; Dainese et al., 2024) these assumptions are often violated in more complex environments. Examples of such environ- ments are popular open-world sandbox games (e.g. MineCraft, RuneScape) containing numerous, diverse mechanics spanning crafting, combat, and physics. These more realistic environments have irreducible stochasticity (c.g., outcomes of actions are subject to random chance), a lack of extrinsic rewards (e.g., players set their goals and there is well-defined criteria for "winning""), and

---

Published as a conference paper at ICLR 2026
OneLife
Benefits of OneLife
Learn from single episode with unguided exploration
Explorative Actions
MRR Win Rate Env. Dynamica Episode
World Learner
Single episode
Environment
OneLite (Ouns)
Baseline (PoE World)
Infer probabilistic distribution over stochastic environments
Observations (No rewards)
Exploration
See zombie
Belief over its next move
Symbolic World Lawe
...
Figure 1: ONELIFE synthesizes world laws from a single unguided (no enviromment-specifee re- wards goals) episode in hostile, stochastic environment. ONELIFE models the world as mixture of laws in code with a precondition-effect structure, each governing an aspect of the world, and in- fers parameters for the mixture that best explain the observed dynamics of the world. The resulting world model (WM) provides a probability distribution over attributes of an object-oriented world state, such as the posi tion of particular zombie. ONELIFE outperforms a strong baseline mod- eling 16/23 core game mechanics tested, measured by MRR (Mean Reciprocal Rank) of the true next state (Sec. 4) under the WM's likelihood. See Box B.3 for a synthesized zombie law.
a high cost of exploration (e.g. entering dangerous areas without preparation can result in death), making it crucial learn from minimal interaction. This leads to our central research question:
How can an agent reverse engineer the laws of a complex, dangerous stochastic world, given a limited interaction budget and without cenironmmet--specffi human-specified goals rewards?
We introduce framework for symbolic world modeling, ONELIFE, name that reflects focus on learning symbolic world model from a single episode with unguided exploration. As illustrated in Fig. (top-right), ONELIFE learns from just a single, ungaided run in the environment, contrast to previous work (Piriyakulkij et al., 2025; Tang et al., 2024; Dainese et al., 2024) that assumes ac- cess to large number of interactions as well as environment specific guidance provided by humans (e.g., goals/ / rewards designed for the environment). ONELIFE recovers a program that describes the environment's underlying transition dynamics p(s++1| St, at) which models the probability dis- tribution over next states 81+1 given a current state and action aN The agent performs this inference using only observations, without access to rewards or other domain-specific guidance. ONELIFE has two key components: a law synthesizer (Sec. 3.3) that proposes new laws and an inference algorithm (Sec. 3.4) that re-weights laws based on their predictive ability over observa- tions. Crucially, the inference algorithm is gradient-based and only updates the laws that alter the observed variables between current state 8, and predicted next state $t+1. allowing for efficient and targeted learning. These components work together in probabilistic programming mixture-of-laws approach (Sec. 3.2) that proposes and re-weights rules based on whether the preconditions for the laws be applicable are met and the effect of the predictions w.r.t. the observed environment tran- sitions. This approach enables our model to infer distributions over complex, stochastic events, as shown in the Fig. (bottom-right), where learned world model outputs a distribution over zom- bie's 's next move. Crucially, ONELIFE not only produces a distribution over states but learns from stochastic observations; the true movement of the zombie in Fig. also follows a distribution, which LIF. seeks to approximate.
To evaluate our approach, we first created a suitable testbed Crafter-OO - by re-engineering the complex Crafter (Hafner, 2022) environment to be a pure function T(s, a) - s' of a structured, text-based hierarchical object-oriented world state. In other words, all the information needed to compute the next state is represented in single structured, object-oriented representation, and there is a ground -tutt program for the transition function that computes the next environment state purely from the state representation, without any "hidden variables". This text-based, object-oriented rep- resentation is natively readable by LLMs and thus allows them to try reconstructing the transition function by writing code that programatically modifies the structured state. It allows for structured,
2

---

Published as a conference paper at ICLR 2026
object-oriented representation that is directly comprehensible to language models and enables sym- bolic reasoning over a world with rich entity interactions. We introduce a new evaluation protocol that uses two axes (Sec. 4): state ranking, the abilitt to distinguish valid outcomes from invalid ones according to the world's laws, and state fidelity, the ability to produce plausible future states for planning. Our experiments show that ONELIFE better captures the environment's dynamics compared to several baselines, including PoE-World (Piriyakulkij et al., 2025), showing improved ability to simulate future states given state and candidate action, and distinguish between likely and unlikely outcomes of an action. We further show that the learned model supports planning in imagination; by simulating rollouts of different policies entirely within the model, we can evaluate and distinguish between effective and ineffective strategies for multi-step goal-oriented tasks.
In summary, our contributions include:
ONELiFE, probabilistic symbolic world model that can learn from stochastic and hostile en- vironments with minimal interactions and without access human-defined rewards. ONELiFE outperforms prior work, learning a world model that better predicts true environment dynamics. Crafter-OO, a reimplementation of Crafter (Hafner, 2022) that exposes a structured, object- oriented symbolic state and and a pure transition function that operates on that state alone. This enables us to test ONELIFE in a complex, stochastic environment and lays the groundwork for future work in symbolic world modeling and programmatic reinforcement learning.
An evaluation suite for world modeling within Crafter / Crafter-OO with 30+ executable scenarios that test knowledge of all core mechanics in Crafter and pool of mutators that can programatically generate illegal distractor states to probe world model understanding alongside, new state fidelity and state ranking metries for evaluating world models in complex, stochastic, environments.
2 RELATED WORK
Symbolic World Models. Symbolic world models represent an environment's 's transition dynamics as executable code, producing interpretable, editable, and generalizable models from limited data. Prior work has used LLMs to synthesize single, monolithic program that functions a world model (Tang et al., 2024; Dainese et al., 2024). Piriyakulkij et al. (2025) introduced a compositional approach by representing the world model product of programmatic experts, enabling modeling of more complex dynamics. Other methods have synthesized programs for planning (Ahmed et al., 2025) or combined functional and automata synthesis to capture latent state dynamics (Das et al., 2023). LLMs have also been used to construct formal planning representations like PDDL from en- vironment interactions or text for symbolic planners (Guan et al., 2023; Deng et al., 2024). Our work differs from these methods in three aspects. First, we operate in a complex, open- world environment based on Crafter (Hafner, 2022) with stochasticity and many interacting mechanics, whereas prior work has operated in simpler, often deterministic domains (e.g., grid-worlds or Atari games). Sec- ond, we do not assume abundant interaction data: our agent learns from a limited budget obtained in single episode or life. Third, ONELIFE learns without external rewards or human spean-spe goals, framing the task as unguided reverse engineering of the environment's laws.
Programmatic Representations for Decision-Making. Program synthesis has been used to rep- resent other components of intelligent agents. Programmatic policies have been shown to offer greater interpretability and generalization compared to neural networks (Trivedi et al., 2021; Liang et al., 2022). LLMs have been used to generate programmatic reward functions from natural lan- guage instructions, enabling agents pursue complex, user-specified objectives (Ma et al., 2024; Yu et al., 2023; Klissarov et al., 2025). Programs have been used to build libraries of composable, temporally extended skills, allowing agents to solve long-horizon tasks by combining previously learned behaviors (Wang et al., 2025; Stengel-Fskin et al., 2024). These methods focus on repre- senting components of the agent's internal decision-makinn what it should value (rewards), or what it is capable of doing (skills). In contrast, our work learns a model of how the external world behaves; this task-agnostic model of environment dynamics is complementary policies, rewards, and skills, and supports planning and decision-maken for any downstream goals.
process: how it should act (policies),
World Modeling for Open-Ended Exploration and Discovery. Agents that explore and learn in complex, open-world world environments without extrinsic rewards typically learn non-symbolic, latent world models and use them to drive exploration through intrinsic motivation (Hafner et al., 2023;
3

---

Published as a conference paper at ICLR 2026
Micheli et al., 2023; Dedieu et al., 2025; Schwarzer et al., 2021). These agents plan using their world models to find novelty or surprise in their environments, discovering useful skills without task-specific supervision (Sckar et al., 2020). This connects to automated scientific discovery, which requires autonomously forming hypotheses and performing experiments to understand unknown systems (Jansen et al., 2024; Chen et al., 2025; Geng et al., 2025). New evaluation frameworks have been proposed to assess an agent's ability to rapidly induce world models in novel contexts (Ying et al., 2025; Vafa et al., 2024). Unlike methods that learn implicit, latent world models, our work learns an explicit, symbolic representation of the world's ' laws. We frame learning as reverse engineering a complex system's 's rules from unguided, limited interaction.
Relation to Domain Inference and State Tracking, ONELIFE tackles the challenge of domain inference (learning transition dynamics) rather than state tracking (inferring state from partial obser- vations) ( (Gordon et al., 1993). Classical domain inference methods (Cresswell et al., 2009; Zhuo & Kambhampati, 2013) often rely on deterministic PDDI. representations. We use a neurosymbolic, Python-based formalism because (I) standard PDDL cannot easily capture the stochastic dynamics of Crafter-OO, and (2) LLMs are much better at generating Python than Probabilistic PDDL.
3 OVERVIEW OF ONELIFE
Our framework, ONELIFE is designed to learn symbolic world models from a single, unguided episode of exploration. built on two key abstractions, programmatic representation of world dynamics as mixture of modular laws with learnable weights and an observable extractor that decouples the environment's state from the learning process. The framework consists: a world model as program (Sec. 3.2), a law synthesizer that proposes new laws using offline data from an unguided exploration policy (Sec. 3.3), an inference algorithm that re-weights laws based on observations (Sec. 3,4), and a forward simulation process that uses the learned model for predicting future states (Sec. 3.5).
We model the environment as having a pure, but potentially stochastic, transition function T S A + (SS) where (S)) is the space of probability distributions over the state space S. This functional view aligns with modern reinforcement learning environment frameworks (Freeman et al., 2021; Matthews et al., 2024) and physical models, where the future state of a system is a pure function of an explicit state and any interventions. (See Section H for discussion why we model a pure transition function probabilistically)
3.1 CRAFTER-OO: A TESTBED FOR SYMBOLIC WORLD MODELING
A common design assumption in previous work on symbolic world modeling (Tang et al., 2024; Piriyakulkij al., 2025; Dainese et al., 2024) is that we have access an object-oriented world state to use as input to the symbolic world model under construction. In practice, this state is only easily accessible for simple environments such as Minigrid "Chevalier-Boisverr Chhevalier-Boiseerr et al., 2018). Programmatic access to the state of more complex environments such as Atari games as used by Piriyakulkij et al. (2025) is only possible due to standalone develop- ment efforts such as OCAtari (Delfosse et al., 2024) which makes the internal object-oriented state of these environments accessible to researchers. The lack of an environment with an exposed, object- oriented state that is more complex than gridworlds or with mechanics more diverse than Atari games has thus far prevented evaluation and development of symbolic world modeling approaches for more complex environments. To close this gap, we implement Crafter-OO (Sec. C), which emulates the Crafter (Hafner, 2022) environment and action space (Tab. 3) by operating purely on an explicit, object-oriented game state (Listing 1). Additionally, contribute utilities for programmatically modifying the game state to create evaluation scenarios (Sec. E, Sec. 4.
al., 2023) or BabyAl
Our target environment Crafter-OO features significant stochasticity, diverse forms of mechanics, and active non-player characters. This includes elements such as hostile and friendly agents with di- verse, inherently random behaviors. Our framework is designed to infer the rules governing these in-
We describe the state in Python/JSON because we found substantially easier for LLMs to manipulate than PDDL. PDDL representations of our complex state became prohibitively large, increasing experimental costs. Furthermore, while Prohabilistic PDDI. can capture stochastic dynamics, it makes synthesis significantly more difficult, likely because Probabilistic PDDL. is much rarer in pre-training data than standard PDDL Python.
4

---

Published as a conference paper at ICLR 2026
Computation Graph
Data
8t
Observables
Conpusble
Parmm
player pes. player
edgn ario
PlayerMoveneee tLaw
Zondsd antivenne ttaw
player. player.invent
inventory
toryulda
zompie.pos.y
Gooden uIdEAAT
right
Loss
92
Observation E(sra))
0y log hatp://wwttttalllprp..e.. posyeerrpoo....... x 2.5/s,, ax)
(player pos.x 2.5, playen pos
+-9 log, deeaaeetiontaa (player poa. y 4/9,, o,)/2
+02 2- log chzantateentt (wwabiz
www.oiii.............oooooo..e.e pos 2, a.) 1/2
scmbse.pon
4.2 log, Pwwnseetttta (ww.bi..ppp.... poa.y 4/4,, oz
2
Figure 2: Illustration of the inference process. The active laws for each observable (defined by (5e, a)) determine the structure of the computation graph, i..., which laws and their corresponding parameters 0, are related to which observables. This structure in turn informs the parameter updates. Shown here is dataset with single transition instance, in which the player (P) moves right; the same time, a zombie (Z) independently moves left. this implicates two laws, PlayerMovementL and Zombi CombieMovementeo eMovementLaw, while not implicating the InventoryUpdatel.nn computation only a function of 0, and 02. Note we use here to denote the normalizing factor. Examples of synthesized laws can be seen in Sec.
As result, the loss
teractions from observation alone, without access to rewards or human-specified goals. For instance, in Fig. 2, the scenario contains "zombie" character chasing the player via stochastic movements. While one cannot perfectly predict the future position of zombie due to inherent randomness built into the environment, our world model is able capture this "chasing the player" behavior without any explicit supervision by predicting discrete distribution for the zombie. posi tion attributes.
3.2 ONELIFE: WORLD MODEL AS A MIXTURE OF LAWS
We consider environments with complex, structured state spaces S where the full state S may be hierarchical and contain a mixture of entity types and attributes. An agent interacts with the environment by taking an action a E A and observing transition from state s, to 8 as illustrated in Fig. 2. We model an environment's ' transition function composition of programmatic laws. law, I,, is a program defined by pair (c,, e;), where ca(s, a) - (true, false) and e, a) + (SS) is effect. The precondition determines whether the law is applicable to state-action pair (s,a). The effect function specifies a probability distribution over next states by defining distributions over the values of state attributes. For example, the PlayerMovementLas in Fig. 2 applies to state-action pairs with a player and a move action, and has an effect on the player position's 's (x)) observable. This precondition-effeer structure is inspired by classical planning and provides a natural way to specify the scope of each law, ensuring modularity (McDermott et al., 1998). During any given transition, multiple
precondition
laws may be applicable.
To create a tractable interface to compare states predicted by a world model and the true state of the environment, introduce an observable extractor, & S O. This function maps complex state into a vector of primitive-valued observables o E O. In the scenario sketched in Fig. 2, the next state 8141 can be complex, with additional entities and objects (e.g., trees, inventory items, etc.). Nevertheless, one can tractably compare states via observations, i.e., changes between and 8141 such as player. wasseer.positiie tion, player. inventory, zombie. combie.position etc. Note that any given law La only makes predictions about subset of all possible observables. For instance, in Fig. 2, the PlayerMovement.. only makes predictions about player. Mllayerpoostt posipioinnnoo rosition observables and does not predict the zombie. www.ipiiionnno observables.
Our world model can be viewed as a probabilistic program (van de Meent et al., 2021) that generates the next state's observables o' conditioned on the current state and action a. The set of laws {L,} defines the components of this program. Formally, the effect ( a) yields set conditional probability distributions (or.oroEo. where (o vls,a) is the distribution over possible values v for observable o E O, with v E supp(o) denoting a specific outcome in the discrete support of the observable. Our implementation currently covers categorical and discrete distributions. In principle,
5

---

Published as a conference paper at ICLR 2026
the framework extends to continuous distributions, the inference algorithm only requires the abil- ity to query the likelihood of an observed data point, For a given state-action pair (s. a), the set of active laws is I(s, a) {a | (s a) is true) (c.g., PlayerMovementLaw and ZombieMovementLacg in Fig. 2). The model assumes that all observables are conditionally independent given the current state and action. The predictive distribution for single observable is formed by combining the predictions from all active laws that have an opinion on it. Let (s,s,a) = {i E I(s, a) o E O} be the set of active laws relevant to observable o. The probability of observing an outcome v for this observable is given by a weighted productt of conditional probability distribution from each law, parameterized by A:
p(o v|s, 0) x II 00.( = a)s
(1)
iEZo(..a)
The complete predictive distribution over the next state s' is the product of the individual observable distributions:
p(s'|s, = II o5O
p(o s, a;
The learnable weights 0 perform model selection over the set of candidate laws. Because the synthe- sizer generates a large pool of atomic laws, including incorrect hypotheses, the optimization process drives the weights of invalid laws toward zero remove them from the model. Additionally, the weights enable multiple plausible laws to vote on the final predictive distribution. allowing the model to aggregate conflicting predictions.
Comparison to Prior product-of-Expert Formalisms. Although we use a product-of-expert structure like PoE-World (Piriyakulkij et al., 2025), the underlying representation and optimization differ fundamentally. Conceptually, PoE-World learns a superposition of experts where each pro- gram predicts the entire next state, This causes the posterior to be noisy in complex environments, as irrelevant experts contribute uniform predictions to attributes they do not model well. In contrast, ONELIFE factorizes the transition function into atomic laws that individually predict a minimal sub- ser of the next state (e.g., only the player's 's health, only specific map tile). This atomicity enables our optimization procedure (Sec. 3.4) to construct a dynamic computational graph for every tran- sition. By exploiting the precondition-effect structure, route gradients only to laws relevant to the specific observed transition. This avoids the "static graph" limitation of prior work and allows ONELIFE to scale to diverse object-oriented attributes (e.g., inventory items, map tiles, NPC states) beyond the simple physics variables (e.g., position, velocity) targeted by prior work.
3.3 ONELIFE: UNGUIDED ENVIRONMENT EXPLORATION AND LAW SYNTHESIS
The set of candidate laws L, is generated from unguided agent-environment interactions through a two-stage process. First, an autonomous exploration policy gathers corpus of interaction data. Second, a synthesizer proposes candidate laws that explain the state transitions observed in this data.
Exploration Policy. Previous work in symbolic world modeling often assumes access to curated offline datasets or utilizes online interaction guided by providdd goals or environment re- wards. In our unsupervised setting, such guidance is unavailable. Furthermore, in hostile envi- ronment such as Crafter-OO, a simple random policy fails to survive long enough to experience the diverse mechanics necessary for comprehensive world modeling. Therefore, we employ explo- ration policy driven by a large language model. The policy is not provided with specific knowledge of the environment; instead, is given the high-level objective discover as many underlying me- chanics as possible, treating exploration as a reverse-engineering task. We distinguish between gen- eral genre priors and cetrrooment-sppccfit dynamics. General genre priors are high-level concepts common to the class of open- world survival environments, such as the existence of hostile entities, the ability to collect resources, or the ability to craft tools. In contrast, environment-specinn dynam- ics refer to exact rules, such as "Zombies chase players" or "Wood is needed to make pickaxe." " The exploration policy is provided with the former to prevent aimless behavior typical of random policies, but it is strictly withheld from the latter. This mimics a realistic scenario where an agent enters a new environment possessing broad intuition about the genre, but must reverse-enge everse--ngginer the specific laws and mechanics of that unique world from scratch. We use the agent scaffolding from Balrog (Paglieri et al., 2025) to implement the agent. The agent's 's architecture maintains a rolling window of its recent state- iate-action history to provide context for decisions. The prompt (see Sec. G)

---

Published as a conference paper at ICLR 2026
also instructs the agent to maintain transient summary of its current understanding of the world's rules, refining its hypotheses as interacts with the environment.
Law Synthesizer. The synthesizer is an automated routine that queries Large Language Model (LLM) to explain observed state transitions. Our system operates by iterating through every transi- tion in the exploration dataset and performing a systematic comparison of the object-oriented state each timestep. This process automatically identifies the specific object attributes that have changed, such an entity's 's position or an inventory count, without requiring manual specification of what to track. For each identified change, the routine queries the LLM to output a Python class containing precondi tion and effect methods. The effect method is generated explicitly perform the ob- served attribute assignment on state object. This process yields atomic laws that govern minimal subsets of state attributes. For instance, a complex combat event is automatically decomposed into separate candidate laws where one explains the health decrease and another explains the enemy's 's movement. This modularity allows the subsequent inference stage (Sec. 3.4) to perform precise credit assignment by isolating correct mechanics from incorrect hypotheses. We provide examples of synthesized laws Sec. B.
Synthesis Differences From PoE-World. While Piriyakulkij et al. (2025) adopt specialized ap- proach utilizing a bank of over 30 synthesizers equipped with prompts tailored to pre-identified me- chanics, ONELIFE employs single synthesizer. This setup requires agent to identify mechanics on the fly and codify them without prior knowledge of the environment' rules. Consequently, our synthesizer consumes the entire game state in a senproa-purpose format (ISON) to write code for di- verse aspects of the world, including map tiles, entities, and player inventories, whereas PoE-World limited synthesis a specific set of physics-based attributes.
3.4 ONELIFE: INFERENCE ON LAW PARAMETERS
We learn the weight vector 0 by maximizing the log-likelihood of dataset of observed transitions D {St, at, St+ f=1 For clarity, we first define the loss for single transition (s, a, a')"; the total loss is the sum over all transitions in the dataset.
Based on the conditional independence of observables, the negative log- likelihood for single tran- sition decomposes into sum over each observable o E O:
L(O: s, s') = Σ log log
DEO
where v {s') is the ground truth value of observable extracted from the next state s'. The log- probability term is derived from the combined predictions of the active laws. Let (s, a) be the set of active laws that make a prediction for observable o. We first define the combined, unnormalized log-score for any potential value u as the weighted sum of log-scores from these laws. The weights Og are the only learnable parameters:
Lo(v|s,0;0)
Σ
0, log (1 u0l, a)
(EI,(..a)
Normalized log-pro bog-probability of observing the specific outcome oo is then given by the log-softmax function. Let supp(o) be the discrete support (set of all possible values) for observable
log p( (s,ss, a; 0) to (v,ls, a; 0) - -log Σ o<Supp())
exp (follss))
(5)
The optimization process leverages the dynamic computation graph induced by our law structure. For each transition and each observable, the loss gradient is calculated with respect to the weights Og only for the active laws E Io(*,, a.). This effectively routes credit for an outcome exclu- sively to the laws that made a prediction about it. This sparse, targeted update mechanism provides more precise credit assignment than methods that update a global set of weights based on aggregate outcomes. We use L-BEGS for optimization (Nocedal & Wright, 2006).
3.5 ONELIFE: FORWARD SIMULATION AND LIKELIHOOD
Forward simulation is the process of using the learned world model generatively to predict a future state 8441 given current state 8t and an action dt. By generating rollouts of future trajectories,
7

---

Published as a conference paper at ICLR 2026
(a) State Ranking
(b) State Fidelity
Rank of GT state out of N total states
GT Next State
Next State
GT Next State
Spi
s-1tes
Sp+1
trl
5 t-1
Stus
Mutation
ka: valk, k2: vain
R1: vald,
World Model
s brl
Edit Distance (ISON Patch)
N-1 Incorrect states
(distractors)
Rank GT state
1 (2 common/ 3 total) 0.33
Two evaluation metric categories described in Sec. 4. A world state of an environment usually has more than two keys (i.e. Crafter-O0's state (Section C.2) when populated has 100+ key- value pairs.) ) and often has nested values, but here we show a simplest case to explain the calculation of (normalized) edit distance. We create distractors for state ranking using mutators (Sec. D), which programatically modify the next state s' a transition (s, 0, function. For example, one of our mutators allows crafting action (e.g. making stone pickaxe) to succeed even when the prequisites for the crafting are not met.
Figure 3:
8') ) to be illegal under the true transition
an agent can evaluate action sequences against a specific goal or reward function without costly or irreversible real-world interaction.
The simulation of a single timestep from (8t; (x) involves a multi-step sampling and reconstruc- tion process. First, for each observable O, the model forms predictive probability distribution [t ac; 0). This distribution is constructed by identifying the set of active laws Io (st, a²) relevant to that observable and combining their predictions according to their learned weights 0,, as specified in Equation This distribution can be used to evaluate the likelihood of an observable conditioned on a) pair. Second, concrete outcome i can be sampled from this distribution for each observ- able: C, ~ p[o]&,, oz; 0). This the collection of sampled outcomes [CoFoEd is used to construct the full symbolic next state 8141. A reconstruction function, which mirrors the observable extraction process, assembles these values back into the environment's 's structured state representation.
4 EVALUATION PROTOCOLS AND METRICS
The evaluation of world models for a stochastic environment is non-trivial. An useful world model fulfills two criteria: (a) state ranking, the ability distinguish plausible future states from implau- sible ones, and (b) state fidelity, the ability to generate future states that closely resemble reality. Both are illustrated in Fig. 3.
State Ranking (Fig. 3 (a)). These metrics assess the model's ability to rank the true next state higher than the distractors. To create the distractor states, we use mutators, which are programmatic functions that apply semantically meaningful, rule- aelebbrea changes to the true next state. For example, a mutator could change a character's position to location they cannot physically reach. We include details mutators in Sec. D.
Rank @ 1 (R @ 1): A binary metric that measures whether the model correctly assigns the highest probability (rank 1) to the true next state among all candidates. Mean Reciprocal Rank (MRR): This metric averages the reciprocal rank of the correct answer across all test instances. A higher MRR indicates that the model consistently ranks the correct state higher. The formula is: MRR = N Ei A, where r; is the rank of the ground truth state for the i-th transition, with rank being the highest probability. We favor MRR over raw mean rank because its inverse scaling (1/r) heavily penalizes missing the top rank, reflecting the high cost of sampling invalid states during planning. Furthermore, unlike mean rank, MRR provides standardized score invariant to the candidate set size N, which varies in our setup depending on the number of applicable mutators.
State Fidelity (Fig. (b)). These measure the error between predicted and ground truth states.
8

---

Published as a conference paper at ICLR 2026
Table 1: Performance comparison of world modeling methods on the Crafter-OO environment, av- eraged over ten trials. We evaluate models on two criteria: state fidelity and state ranking All methods use the ONELIFE exploration policy and law synthesizer but differ in their parameter infer- ence method. ONELIFE shows significant improvements over the PoE- World inference algorithm and ONELIFE variant without parameter inference. The random baseline is shaded in gray. The
"No Inference" row ablates learnable law parameters 0 from our world model.
Law Synthesis
Law Param. Inference
State Ranking
State Fidelity
(Sec. 3.3)
(Sec. 3.4)
Rank @ 1t
MRR
Raw Edit Dist. 4
Norm. Edit Dist. J
Random World Model
8.5%
0.322
121.538
0.809
WorldCoder
0.0%
0.264
27.180
0.181
ONELIFE
WoB-Wolld
10.8%
0,351
10.634
0.071
ONELIFE
No Inference
13. 13.0%
0.429
8.540
0.057
ONELIFE
ONELIFE
18.7%
0,479
8.764
0.058
A over PoE-World
(+7.9%
(+0.128)
(-1.870)
(-0.013)
Raw Edit Distance:
The total number of atomic JSON Patch operations required to transform the
predicted state, 8(41, into the ground truth state, 8t+1- Normalized Edit Distance: The raw edit distance divided by the total number of elements in the state representation.
4.1
EVALUATION FRAMEWORK IMPLEMENTATION ON CRAFTER-OO
Evaluating a world model on random rollouts may not provide sufficient coverage of rare or im- portant events in environment. To ensure evaluation is comprehensive, we create evaluation trajectories from a suite of scenarios. Each scenario runs short, scripted policy from an initial state designed to reliably exercise specific game mechanic or achieve a particular goal, ensuring that our evaluation thoroughly covers the environment's 's dynamics. We generate a comprehensive eval- uation dataset by implementing scenarios that cover every achievement in the game's 's achievement tree, seen in Fig. 4. This ranges from basic actions like collecting wood complex, muep tasks like crafting an iron sword, ensuring all of the game's core mechanics are tested. More details on scenarios are provided in Sec. E. We generate distractors for each transition in the evaluation dataset using a bank of mutators which each produce a subtle, but illegal transformation of the game state in response to action. Some examples are causing incorrect item to be produced when taking crafting action, or allowing an item to be produced without the correct requirements, or illegal entity behavior such teleporting. Because mutators have specific preconditions (e.g., combat mutators only apply during combat), the number of applicable distractors varies per state. In our experi- ments, the total candidate set size (ground truth plus distractors) ranges from N = to N 11 per transition. Details on mutators and the evaluation are provided in Sec. D and Sec. F.
EXPERIMENTAL SETUP AND RESULTS
We conduct a series of experiments to evaluate ONELIFE. First, we quantitatively assess the model's predictive accuracy using our state ranking and fidelity metrics across a comprehensive suite of scenarios. Second, we test the model's ability to support planning in imagination. We use the model to perform simulated rollouts of different policies, evaluating whether can predict the outcomes of these plans well enough distinguish effective strategies from ineffective ones (Sec. A).
We compare ONELIFE against three baselines (fully detailed Section I): a Random World Model; PoE-World (Piriyakulkij al., 2025), the prior state-of-the-art symbolic framework that leams a product of experts; and WorldCoder (Tang et al., 2024), which differs from Wor-World by synthesizing a monolithic, deterministic program for the transition function.
5.1 RESULTS
State Fidelity and Ranking. ONELIFE learns a world model with significantly higher predictive judgment than baseline methods while maintaining competitive generative fidelity. Table compares our full method against baselines and key ablations across all evaluation metrics. ONELIFE's pri- mary advantage appears in the predictive judgment metrics. We achieve discriminative accuracy

---

Published as a conference paper at ICLR 2026
ONELIFE (Qurs)
PoE-World
ONELIFE (Ouns) Avg
..... PoE-World Avg
Mean Reciprocal Rank (1RRB)
2.0
Resource Collection
Tool Production
Health Maintenance
World Manipulation
Combat
Movem ent Physics
64
Colfect Coue
Contect Diamound
Cosked Drint
Dalece from
Coleet stone
Goleet Wond
Cartt from Pokeae
cart inn Senre
Craft Store Pocon
Cratt Weuthn Meckst
craft Winden Seveid
Eat cow
Plant
Wale Up
Pare Parnicc
Rlcce Plant
Place stone
Pacce Rase
Deteel staintin
Zomble Dutest
Come Movernett
Randion Movieteet
cas stind
Figure 4: Per-scenario state ranking performance of ONELIFE (Ours) versus PoE- PoE-World, measured by Mean Reciprocal Rank (MRR t). Scenarios are grouped by the core game mechanic they test. Horizontal lines show the average MRR across all scenarios in group for ONELIFE and PoE- World. ONFLIFF demonstrates a more accurate understanding of the environment's 's laws, achieving higher average MRR and outperforming the baseline on the majority of individual scenarios.
of 18.7% and MRR of 0.479, outperforming the PoE-World optimization baseline by 7.9 per- centage points and 0.128, respectively. While precisely generating a complex future state remains challenging. our model has learned an accurate understanding the environment's underlying laws. This enables it to assign high probability to valid transitions and low probability to invalid ones. The comparison to the "random world model" shows that (i) a high edit distance can quickly be amassed if the world models updates observables that are unchanged in the ground truth state, thus, reinforcing why such simulation challenging; (ii) optimizing for generative metries like state fi- delity alone does not yield a better world model to guide agent, e.g., while the PoE-world model (row 2 in Tab. 1) dramatically improves the state fidelity by reducing the edit distance a factor of 10, it only marginally improves the ability rank multiple states by x2% over random (Rank @ 1) reiterating the need for state ranking metrics. Removing the parameter inference step ("ONEL LIFE & None"") results in performance drop of 5.7% in Rank @ 1 and 0.05 in MRR, confirming that the weights are essential for distinguishing valid laws from incorrect ones.
Fine-grained Evaluation. Figure 4 breaks down Mean Reciprocal Rank performance across in- dividual scenarios spanning mechanics from resource collection to combat. ONELIFE consistently outperforms the ForlWorld baseline the majority (16/23) of scenarios. These improvements stem from a robust understanding of the environment's 's diverse rules rather than strong performance on only a few simple mechanics.
6 CONCLUSION
We address the problem of learning symbolic world model from limited, unguided interaction in a complex, stochastic environment. We introduced ONELIFE, a framework that represents world dynamics as a probabilistic mixture of modular, programmatic laws. Its core learning mechanism routes credit for observed state changes exclusively to the laws responsible for predicting them, en- abling effective learning even when many rules are inactive during a given transition. Evaluated on Crafter-OO, our variant of the complex Crafter environment with object-centric state. ONELIFE learns a world model with superior predictive judgment compared to a strong baseline, more aceu- rately distinguishing plausible future states from implausible ones. This improvement is consistent across a wide range of game mechanics. Our work provides a foundation for building agents that can autonomously reverse engineer the rules of unknown environment.
ACKNOWLEDGEMENTS
This work was supported by NSF-AI Engage Institute DRL-2112635, ARO Award W91 INN21102200
ONR Grant N00014-23-1-22566
DARPA ECOLE Program No.
HR001 12390060,
Capital One Research Award, Apple PhD Fellowship, and NDSEG PhD
10

---

Published as a conference paper at ICLR 2026
The views contained in this article are those of the authors and not of the funding agency.
Fellowship.
ETHICS STATEMENT
We do not foresee any ethical implications beyond standard ethical and safety considerations that apply Al research generally.
REPRODUCIBILITY STATEMENT
We have open-sourced Crafter-OO, ONELIFE, and the evaluation framework used in our work to aid reproducibility. All prompts and key details of the exploration policy, synthesis algorithm, and law parameter inference have been described.
REFERENCES
Zergham Ahmed, Joshua Tenenbaum, Christopher! J Bates, and Samuel Gershman. Synthesizing world models for bilevel planning. TMLR, 2025.
John Backus. Can programming be liberated from the von neumann style? functional style and its algebra of programs, Communications of the ACM, 2166)6613-641, 1978. doi: 10. 145355579 359579.
(Ed.) Bryan and M. (Ed.) Nottingham. JavaScript Object Notation (JSON) Patch. RFC 6902, IETF, April 2013. URL https: https::/ddatttracker ietf wtrg/doc/htmm/ffc6500
Yimeng Chen, Piotr Piekos, Mateusz Ostaszewski, Firas Laakom, and Juergen Schmidhuber. Phys- gym: Benchmarking Ilms in interactive physics discovery with controlled priors. arXiv preprint, 2025.
Maxime Chevalier-Boiiveee Dzmitry Bahdanau, Salem Lahlou, Lucas Willems, Chitwan Saharia, Thien Huu Nguyen, and Yoshua Bengio. Babyai: A platform to study the sample efficiency of grounded language learning. arXiv preprint arXiv:/810.08277 2018
Maxime Boissve,, Bolun Dai, Mark Towers, Rodrigo Perez-Vicente Lucas Willems, Salem Lahlou, Suman Pal, Pablo Samuel Castro, and Jordan Terry. Minigrid & miniworld: Mod- ular& & customizable reinforcement learning environments for goal-oriented tasks. In Advances in Neural Information Processing Systems 36, New Orleans, LA, USA December 2023.
Stephen N Cresswell, Thomas L. McCluskey, and Margaret West. Acquisition of object-centred domain models from planning examples. In Proceedings of the International Conference on Automated Planning and Scheduling (ICAPS), volume 19, pp. 338-341, 2009.
Nicola Dainese, Matteo Merler, Minttu Alakuijala, and Pekka Marttinen. Generating code world models with large language models guided by monte carlo tree search. In NeurIPS, 2024.
Ria Das, Joshua B. Tenenbaum, Anmando Solar-Lczama, and Zenna Tavares. Combining functional and automata synthesis to discover causal reactive programs. Proc. ACM Program. Lang., (POPL), January 2023. doi: 10.1145/357 1249. URL
Antoine Dedieu, Joseph Ortiz, Xinghua Lou, Carter Wendelken, Wolfgang Lehrach, J Swaroop Guntupalli, Miguel Lazaro-Grede Gredill,,,,, and Kevin Patrick Murphy. Improving transformer world models for data-efficient rl. In 2025.
Quentin Delfosse. Jannis Blüml, Bjarne Gregori, Sebastian Sztwiertnia, and Kristian Kersting. OCAtari: Object-centric Atari 2600 reinforcement learning environments. Reinforcement Learn- ing Journal, 1 14400-499 2024.
Boyuan Deng, Qiaochu Xu, Hao Li. Ling Li, Rui Xu, Yunlong Lu. Jie Lin, and Ziqi Zhang. Pddlego: Iterative planning in textual environments. arXiv preprint, 2024.
11

---

Published as a conference paper at ICLR 2026
C. Daniel Freeman, Erik Frey, Anton Raichuk, Sertan Girgin, Igor Mordatch, and Olivier Bachem. Brax a differentiable physics engine for large scale rigid body simulation, 2021. URL http: //gi teiithub.com //githuub.com/ggogll/bbaaa
Jiayi Geng, Howard Chen, Dilip Arumugam, and Thomas L Griffiths. Are large language models reliable ai scientists? assessing reverse-engineeriing of black-box systems. arXiv preprint, 2025.
Neil J Gordon, David Salmond, and Adrian FM Smith. Novel approach to nonlinearhnon-gaussian bayesian state estimation. IEE Proceedings F (Radar and Signal Processing), 140(2):107-113, 1993.
Lin Guan, Karthik Valmeekam, Sarath Sreedharan, and Subbarao Kambhampati. Leveraging pre- trained large language models to construct and utilize world models for model-based task plan- ning. In NeurlPS, 2023.
Danijar Hafner.
Benchmarking the spectrum of agent capabilities. In The Tenth International Con- ference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenRe- view.net, 2022. URL https::/
EoH.
Danijar Hafner, Jurgis Pašukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse domains through world models. arXiv preprint, 2023.
Peter Jansen, Marc-Alexandre Côté, Tushar Khot, Erin Bransom, Bhavana Dalvi Mishra, Bod- hisattwa Prasad Majumder, Oyvind Tafjord, and Peter Clark. Discoveryworld: A virtual envi- ronment for developing and evaluating automated scientific discovery agents. In NeurlPS, 2024.
Martin Klissarov, Mikael Henaff, Roberta Raileanu, Shagun Sodhani, Pascal Vincent, Amy Zhang, Pierre-Luc Bacon, Doina Precup, Marlos C Machado, and Pierluca D'Oro. Maestromotif: Skill design from artificial intelligence feedback. In ICLR, 2025.
Jacky Liang. Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. Code policies: Language model programs for embodied control. arXiv preprint, 2022.
Yecheng Jason Ma, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayara- man, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Eureka: Human-level reward design via coding large language models. In ICLR, 2024.
Michael Matthews, Michael Beukman, Benjamin Ellis, Mikayel Samvelyan, Matthew Jackson, Samuel Coward, and Jakob Foerster. Craftax: A lightning-fast benchmark for open-ended re- inforcement learning. In International Conference on Machine Learning (ICML), 2024.
Drew McDermott, Malik Ghallab, Adele Howe, Craig Knoblock, Ashwin Ram, Manuela Veloso, Daniel Weld, and David Wilkins. PDDL the planning domain definition language. Technical report, Yale Center for Computational Vision and Control, 1998.
Vincent Micheli, Eloi Alonso, and François Fleuret. Transformers are sample-efficient world mod- els. ICLR, 2023. Oral Presentation.
Jorge Nocedal and Stephen J. Wright. Numerical Optimization. Springer, second edition, 2006.
Davide Paglieri, Bartlomiej Cupial, Samuel Coward, Ulyana Piterbarg, Maciej Wolczyk, Akbir Khan, Eduardo Pignatelli, łukasz Kuciñski, Lerrel Pinto, Rob Fergus, et al. Balrog: Bench- marking agentic Ilm and vlm reasoning on games. In The Thirteenth International Conference on Learning Representations, 2025.
Wasu Top Piriyakulkij. Yichao Liang, Hao Tang, Adrian Weller, Marta Kryven, and Kevin El- lis. Poe-world: Compositional world modeling with products of programmatic experts. arXiv preprint, 2025.
Max Schwarzer. Nitarshan Rajkumar, Michael Noukhovitch, Ankesh Anand, Laurent Charlin, De- von Hjelm, Philip Bachman, and Aaron Courville. Pretraining representations for lata-efficient reinforcement learning. In NeurlPS, 2021.
12

---

Published as a conference paper at ICLR 2026
Ramanan Sekar, Oleh Rybkin, Kostas Danillidis, Pieter Abbeel, Danijar Hafner, and Deepak Pathak. Planning to explore via self-supervised world models. In ICML, 2020.
Elias Stengel-Esking Archiki Prasad, and Mohit Bansal. Regal: Refactoring programs to discover generalizable abstractions. In International Conference on Machine Learning, pp. 46605-46624. PMLR, 2024.
Hao Tang. Darren Key, and Kevin Ellis. Worldcoder, a model-based Ilm agent: Building world models by writing code and interacting with the environment. arXiv preprint, 2024.
Dweep Trivedi, Jesse Zhang, Shao-Hua Sun, and Joseph J Lim. Learning synthesize programs as interpretable and generalizable policies. NeurlPS, 2021.
Keyon Vafa, Joshua B Tenenbaum, Jacob D Andreas, Yoshua Bengio, and Brenden M Lake. Evalu- ating the world model implicit in generative model. NeurIPS, 2024.
Jan-Willem van de Meent, Brooks Paige, Hongseok Yang, and Frank Wood. An introduction to probabilistic programming, 2021. URL //tps://arxiv.. wwtp:///g/bb//////// 10756.
Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. TMLR, 2025.
Lance Ying, Katherine M Collins, Prafull Sharma, Cedric Colas, Kaiya Ivy Zhao, Adrian Weller, Zenna Tavares, Phillip Isola, Samuel Gershman, Jacob D Andreas, Thomas L Griffiths, Francois Chollet, Kelsey R Allen, and Joshua Tenenbaum. Assessing adaptive world models in machines with novel games. In ICLR, 2025.
Wenhao Yu, Nimrod Gileadi, Chuyuan Fu, et al. Language to rewards for robotic skill synthesis. In CoRL, 2023.
Hankz Hankui Zhuo and Subbarao Kambhampati. Action-model acquisition from noisy plan traces. In Proceedings of the International Joint Conference on Artificial Intelligence (ICCAI), pp. 2444- 2450, 2013.
A PLANNING AND MULTI-STEP SIMULATION WITH THE LEARNED WORLD MODEL
To assess the practical utility the learned world model, we evaluate its effectiveness in a planning context. Our protocol tests the model's ability distinguish between effective and ineffective plans through forward simulation. For a set of scenarios, we define a reward function and two distinct, programmatic policies (plans) to achieve goal within the scenario. Each plan is represented hierarchical policy (in code) that composes subroutines for navigation, interaction, and crafting.
We give example in box G.3 for the "Zombie Fighter" scenario. Each reward function is likewise written in code and calculates rewards from the rollout of a plan. We execute rollouts of both plans within our learned world model and, separately, within the ground-truth environment. The measure of success is whether the world model's 's simulation yields the same preference ranking over the two plans as the true environment, based on the final reward. This assesses if the model has captured the causal dynamics necessary for goal-directed reasoning.
Setup. We design three scenarios that test distinct aspects of the environment's mechanics: combat, tool-use and resource consumption. shown in Table 2. In the Zombie Fighter scenario, an agent with low health must defeat two zombies. The superior plan involves a multi-step process: pathfind- ing to locate and harvest trees, crafting a table and then a sword, and only then engaging in combat. The alternative is to fight immediately. The Stone Miner scenario tests the model's understanding of resource collection. The effective plan is to first harvest wood, craft pickaxe, pathfind to stone, and then mine. Attempting to mine stone directly ineffective. Finally, the Sword Maker scenario evaluates knowledge of resource consumption. The goal is to craft multiple swords. The efficient plan places a single crafting table and reuses it, whereas the inefficient plan wastes wood by placing new table for each sword. On average, plan requires &s 18 steps execute, with the longest plans
13

---

Published as a conference paper at ICLR 2026
Pathfind
Repeet wood
Place crafting table
9..
9.9
Initial State
Mine stone
Pathfind to stone
Craft pickaxe
9.9
3.9
Simulating Plans within Onel.ife's World Model
Pathlind to stome
Mine stone X
The effective plan first acquires wooden pickaxe, then tries to mine
The ineffective plan tries mine stone directly.
999
Figure 5: We show example of plan execution within ONELIFE's world model for the "Stone Miner" scenario. The task to mine stone, and can only be successfully completed if wooden pickaxe is obtained before attempting mine stone. We simulate two plans within the world model. The effective plan carries out a multi-step sequence gathering wood, crafting a wooden pickaxe, and then attempting mine. The ineffective plan attempts mine the learned by ONELIFE correctly simulates causal game mechanics that cause the effective plan to succeed and the ineffective plan fail. The frames generated by rendering the structured states constructed by ONELIFE's learned transition function.
directly. The world
taking 30 steps. Thus, simulating the results of these plans tests the ability of the world model to accurately model the consequences of long sequences of actions upon the world. We show an example plan execution imagination for the "Stone Miner" scenario in Fig.
Results. Table shows that across all three scenarios, our learned world model correctly predicts the more effective plan. The ranking plans generated by simulating rollouts in ONFLIFE matches the ranking from the ground-truth environment. For instance, in the Zombie Fighter scenario, the model correctly simulates that the multi-step plan crafting a sword leads to higher Damage Per Second, identifying it the superior strategy. This demonstrates that ONELIFE captures suffi- ciently accurate causal model of the world to support basic, goal-oriented planning.
B
LAW EXAMPLES
Below, give examples of various laws synthesized by ONELIFE. In box B.1 and box B.2, show examples of how ONELIFE has learned the hierarchical structure of OT/frafr-O/Crraterr' tech- tree. In this case, must mine stone before stone pickaxe be produced. These laws deterministic the sense that they define probability distributions that place mass 1.0 on a single outcome, consistent with the probabilistic framework defined in Sec. 3.2. In box B.3, give
14

---

Published as a conference paper at ICLR 2026
Table 2: Planning via forward simulation. Our learned world model is used to compare alternative plans in three scenarios. This is done by executing the plans in the world model, and measuring the reward obtained by each plan. In each case, ONELiFE produces the same ranking over plans as the ground-truth environment, demonstrating its ability to capture causally relevant dynamics for goal-directed decision-making and accurately simulate long action sequences 30 steps. Each
plan was executed 10 times.
Scemario
Plan Deseription
Reward Function
Arg. Steps
True Eay.
ONELiFE's WM
Reward
Preferred
Reward
Preferred
Zombic Fighter
Hurvest Wood -s Craft Table Craft Sword Fight Fight Immediatelly
Damige Per Sccond
33
17
2.0
200
1.67
Miner
lnmmediately
Wood
Pickase Mine
Collected
Sword Maker
Place New Table per Sword
Reuse Crafting Table all Swords
Swords Crafted
2.0
4.0
20
an example of law synthesized by ONELIFE for a stochastic mechanic, in this case, the chase behavior of zombies when they are within certain range of player. The idle skeleton law in box B.6 and moving skeleton law in box B.4 make conficting predictions; these are aggregated by the weight inference process in Sec. 3.4 to produce a distributional prediction that takes into account the predictive accuracy of both laws.
Box B.1| Mine Stone Law
class MineStoneLaw:
def _init__ssel))).
""IIniiiaiie with configurable parameters. pass
def pfeconditionnself,, current_stater
WorldState, action: str) -> bool:
"*"Return True if this law should apply to the given state and action." if action != "Do":
return False
10
11
target_material,, current_state current.state.geettarrttttie
12
if target_material == "stone":
Check if the player has any pickaxe
15
has_pickaxe =
16
current_state.pp playeer inventory. woceentrr.wwodd..oo... pica_pickaxe > or
current_state www.ssttte..llyer...mmmm inventory sttrryssssoonscc.e.seeee or current_state, player wwtpveettor..rronpiieeeee
19
20
return has_pickaxe
return False
24
def (seefctseelf,, current_stater WorldState, action: str) -> None:
25
"""Apply the law by modifying the world state. "un # Increment stone in inventory
current_state wlapentssttteeelaee...re. inventoryy...ooom stone DiscreteDistributin supports(current_starc player.la.e.e..ee inventory, stone 1]
28
29
30
# Replace the mined stone material with grass current_state. set_ set_faaiin_materiaae ("traaasssal"""sss""
32
15

---

Published as a conference paper at ICLR 2026
Box B.2] Craft Stone Pickaxe
class CraftStonePickaxe:
def
__nit__(self):
"*"Initialize with configurable parameters. "**
specific parameters needed this crafting recipe.
pass
def
preconditionsself,, current_state: WorldState, action: str) -> bool: "Return True this law should apply the given state and action. """ Check if the action "Make Stone Pickaxe"
10
if action = "Make Stone Pickaxe":
Check if player has required materials
has_wood current_state. player inwennonyy wood >= 1
13
has_stone current_state. slaae. player. return has_wood and has_stone
inventory. stone
14
15
return False
17
def effect(self, current_state WorldState, action: str) Apply the law by modifying the world state. ""* # Decrease wood by current_state. caantttsttte.playe.... inw.enerrvrttrrr...om wood = Discrretbiisrriuuttooopppppprr
18
19
current_stäte wwweer.laa.r.nn.ee .iveentorv..... wood 1])
Decreass ease stone by
22
current__saaee.plyee www.er.lpp.r...e.e player..ivventory. stone Discreteoiittibbbtionnupporr
current_stare state.player.
stonne stone # Increase stone_pickaxe by
13)
current_state. wlaye. inventoryy..... stone_poootooo _pickaee DiscreteDistribution
player.inventory.
supports(current_ste player
player inventory. stone_pickaxe + 1])
Box B.3 Zombie Chase
class ZombieeAggroMovement:
def _eef_______llffff_ff
"""Initialize with configurable parameters. "**
pass # No specific parameters are needed for this observed law.
def (f current_state; WorldState, action: str) >> bool: "*"Return True this should apply the given state action." ... # This law applies if there are any ZombieState entities within the player's
# update range, their movement is autonomous process. zombies_in_range ZombieState)
10
11
return (e(t//mbee_in_rrrrgeeee
def effect current_state: WorldState, action: str) None: "AAplly law by modifying the world state. "**
14
15
player_pos = surrent_state.player positin
16
# Retrieve all ZombieState objects that within the update range. This implicitly filters for zombies close enough active/ observable.
18
19
taabiiesttouuddaee = current_state htartt_stttteggt_ooeeeeeee ittpeee_____ttt_______eee ZombieState)
for zombie zombies_ to_update:
22
Calculate the differences in coordinates between the player and the zombie.
dx = player_pos. zombie coibbie.psiiiio.xx
16

---

Published as a conference paper at ICLR 2026
dy player_pos.y zombie. paabiieoostion.yy
25
26
Initialize new positions to current positions (no movement by
default)
new_x zombie. rombie.position. new_y zombie. poabieeposiiie tion.y
28
29
Prioritize movement along the X-axis if dx != 0:
32
one step towards the player along the X-axis. new_x zombie. ponbiii.osstion.x + (1 if dx 0 else -1)
33
34
elif dy != 0:
If X-axis is already aligned, move step towards the player along the Y-axis.
36
newLy combie.position
if dy else -1)
37
# Update the zombie's ' position in the state using DiscreteDistribution
zombie, position.x #iscreteeissrrbbtooooouuuprrrrre
40
positionnsitio DiscreteDistribution (s=pprtt==tppppeeee
Box B.4| Skeleton Movement
class SkelletonBanddrrooeeeee......
def (init__(self))
"""Initialize with configurable par parameters." pass
current_state: pfeconditionnself, "*"Return True if this law should apply to the given state and action. # This law applies generally to all skeletons, independent of player action for movement return True
WorldState, action: str) -> bool:
10
def (ffecc(self,, current_state: WorldState, action: str) >> None: """Apply the law by modifying the world state. skeletons [obj for obj current_ objetts SkeletonState)]
13
objects if isinstance
14
for skeleton in skeletons:
current_x skeleton.positee position.x current_y position.
17
18
19
Possible next X positions:
current_xx current_x + current_x
skeleton, skeleton.position.x DiscreteDsttrbbiiinssspppese current_x, current_x 1,
23
current_x -
])
Possible next Y positions: current_y,, current_y +
current_y
26
skeleton. position.y Discrettobsstribbbtioosuupppppt
27
current_y,
28
current_y + 1,
current_y -
])
17

---

Published as a conference paper at ICLR 2026
Box B.5] Health Regeneration Law
class Player/wennoyyleethheeeerrrooooo
def _init__seeff max_health: int 20, tcooeer_treesoold: float 1.0): "*"Initialize with configurable parameters for health regeneration. telf.max_health max_health
self.rrecoer__threshoo recover_threeholld
def precondition(self,, current_state: WorldState, action: str) -> bool:
Return True if the player's inventory health should regenerate. This law applies if the player is not at max health, has sufficient recover points, and is not sleeping.
10
13
player current_ssate.
14
15
# Check if player's current inventory health is less than the defined maximum
16
jaa_sppace_frr_heall player innyeerinveettry.hhee health self mex__eellth
17
18
# Check if player has sufficient recover points to enable regeneration ras_recoovr_poonts player. recover >= self. rwwooeer_threshheeeeeeeeeeoooo
recoer
Check player not currently sleeping not_sleeping = not player. player.sleeping sleeping
22
33
# This is passive regeneration effect, so the specific action taken (e. "Move North") # not a direct precondition, transition.
8.,
25
effect occurs during state
return has_space_for_healt and res_recooer_points and not_sleeping
28
effect(self, current_state: WorldState, action: str) None: **n
29
30
Apply the law by increasing the player's inventory health by 1.
Increment the player's inventory health by
33
surrent_state. wla.sstttttetet...te..emee www.:pl.iveeetttooo... inventory. health Discretelistrrbutionssspprrrrr current_state pwa.er............ playerr. health 1])
Box B.6| Skeleton Idle
class SkeletonIdleLaw:
def _silit__(seef):
"""Initialize with configurable parameters. pass
WorldState, action: str) -> bool: "*"Return True if this law should apply to the given state and action.
current_state: (sel,,
# This law applies if there are any skeletons in the world that aren tt otherwise engaged.
Since no changes were observed, we assume this their default passive behavior.
10
return True # Applies universally as default behavior for skeletons
def effect current_state: WorldState, action: str)
None:
13
"." Apply the law by modifying the world state. .** for skeleton in current.state.get.co
14
SkeletonState):
Based observation, skeletons remain unchanged.
18

---

Published as a conference paper at ICLR 2026
16
We predict their attributes will stay the same.
17
skeleton, health Diserteesssrruuitooosspprrrrrrrs heal skeleton. position. skeleton.position. x Diserrteosssrbbtttooosspprrrrree positioonn
18
:x])
19
skeleton.posiie
.y])
20
celleton.relaatn = obsertteessrrrruuirouuerrrrrrerd
C THE CRAFTER-OO ENVIRONMENT
This appendix details Crafter-OO, our reimplementation of the Crafter environment that exposes structured, object-oriented symbolic state and operates through pure transition function. We developed Crafter-OO as a testbed for symbolic world modeling approaches in a complex, stochastic domain.
Table The discrete action space of Crafter-OO. The action space identical to the original Crafter benchmark (Hafner, 2022).
Category
Actions
Movement
move_left, move.right moveuuu move_down
Interaction
do, sleep, noop
Placement
place_stone, place_table, place.furnace, place-plant
Crafting
make make_wood_ make_stone makeeioon.iiiiiiccacaa make.wood_ssord,
C.1 MOTIVATION AND DESIGN PRINCIPLESS
Symbolic world modeling benefits from environments where the complete state is accessible structured representation. Simple grid worlds provide this but lack complexity, while more complex environments typically require additional engineering to expose their internal state. More funda- mentally, existing testbeds for symbolic world modeling have focused on environments that are either deterministic or have limited stochasticity and narrow range of mechanics. Atari games, for instance, while complex in visual processing demands, have relatively predictable dynamics and constrained set of interactions compared to open- open-world world environments.
We developed Crafter-OO to address this gap. The environment features significant stochasticity in entity behaviors, diverse mechanics spanning resource collection to combat, and multi-step causal chains. Our design follows three principles:
1. Explicit Object-Oriented State:
The entire game state is captured in a single, hierarchical data model that serves as input and output for world models.
2. Functional Purity: The environment's dynamics are exposed as pure transition function, T(state, action) > next_state, with no hidden variables.
3. Programmatic Modification: The state representation can be precisely manipulated with code, enabling controlled experimental setups.
C.2 THE Wor WoState DATA MODEL.
The core of Crafter-OO is the WorldState data model, which captures the environment a single timestep. This model is defined using Pydantic for structure and validation. Its components include:
player: A PlayerState object containing position, inventory, health, and current action.
objects: A list of non-player entities (CowState, ZombieState, PlantState, etc.) with type
discrimination via a name field.
materialss A 2D array representing the terrain map.
19

---

Published as a conference paper at ICLR 2026
Global Properties: World-level attributes including daylight, size, and serialized random state.
Listing shows the structure of this model. This representation provides the interface between the environment and symbolic world models.
from typing import TypeAlias, Literal
# Basic Data Structures
class Position:
"""Represents 2D position (x, y) the game world."".
x: int
int
10
class Inventory:
11
r""Represents the player's inventory counts for each item type.
12
health: int
food: int
drink: int
15
energy: int
16
sapling: int
17
wood: int
stone: int
coal: int
20
iron: int
21
diamond: int
pood_pickase int
stone_pickaxe: int
24
iick_eec int
25
wood_sword: int
26
stone_sword; int
iron_sword: int
29
class Achievements:
30
"""Rereseents the player's unlocked achievements.
collect_coal: int
collect_diamond int
33
collect_drink: int
34
collect_iron: int
35
collect_sapling int
collect_stone: int
collect_wood: int
38
defeat_skeleton: int
39
defeat_zombie, int
40
eat_cow: int
eat_plant: int
42
make make_iron_pickaxe: int
43
make_iron_sworfe int
44
make_stone_pickaxe int
nake_stone_sword: int
kake_wood_pickake int
47
make_wood_swordd int
48
place_furnace: int
49
place_plant: int
place_stone int place_ table: int wake_up: int
52
53
Game World Entities
56
57
class BaseObject:
58
""Thhe base class for all dynamic objects in the game world,
entity_id: int
position: Position
20

---

Published as a conference paper at ICLR 2026
health: int removed: bool
62
63
64
class Player(BaseOjject)
"T" state of the player character.
Literal["player"] "player"
67
facing: Position action: str
68
sleeping: bool
inventory: Inventory
achievements: Achievements
72
thirst: float
73
hunger: float
fatigue: float
recover: float last_health: int
26
77
78
class (BssBoeebj::ct):
"""The state of cow.
Literal["cow" "cow"
82
class Zombie (BaseObject):
"T" state of zombie."
Limere][" "zombie" cooldown: int
85
86
87
class Skeleton (BaseObject):
"""The state of skeleton."
name: Literal["skeletonn "skeleton"
90
reload: int
91
92
class Arrow(BaseObjectt)
"""The state an projectile," ""*
94
name: Literal["arrow"]] "arrow"
95
facing: Position
96
class (Baee(Bas)))eect))
"""The state plant, which be eaten."" "".
99
name: Literal["plant"] "plant"
100
grown: int
ripe: bool
class Fence (Basebbja))
104
"" state a fence object. "o*
105
name: Literal["fence"] "fence"
union of all possible entity types in the world.
Entity: TypeAlias Player Cow Zombie I Skeleton | Arrow Plant Fence
109
World and Spatial Structures
113
MaterialT: TypeAlias str
114
class Chunk:
""Represents spatial region of the world for efficient updates. "** chunk_key: tuple[int, int, int, int] object_ids: list[int]
118
class WorldState:
"Reeressenss the complete, hierarchical state of the game world single
timestep, ***
122
# World dimensions and configuration size: tuple[int, int]
chunk_size: tuple[int, int]
21

---

Published as a conference paper at ICLR 2026
view:
tuple[int, int]
127
# World status
daylight: float
step_count: int
131
# The grid static materials (e.g., grass, stone, water) materials: List(List[MaterialT | None]]
132
# A list of all dynamic entities currently in the world. objects: list[Entity]]
136
137
# A direct reference to the player object for easy access.
player: Player
140
# Spatial partitioning data. chunks: list[Chunk]
141
# Internal simulation state setityydd_ccuner_sstt int
145
Baadiiize_radd__seee str event_bus: list[str]
146
Listing 1: Simplified structure of the WorldState data structure.
C.3 EXTRACTING STATE FROM CRAFTER'S 's GAME ENGINE
Declarative State
Export (produces 8t+1)
St
Reconstruction (reconstruct)
Imperative World Instance
Imperative Step world. step(a)
Figure 6: The functional cycle for state transition. A declarative state snapshot is reconstructed into a live, imperative world instance. The engine simulates a single step, and the resulting world is exported back into new declarative state snapshot for the next timestep. This ensures match Crafter's mechanics exactly.
The simulation state in the original engine is not a single data structure but is distributed across graph of live Python objects, each with its own internal state and complex inter-dependencies, such as non-player characters holding direct references to the player object. Furthermore, the engine's behavior relies implicit state, including the internal state of its pseudo-random number generator, which governs all stochastic events. Achieving a pure functional interface required developing a robust mechanism to first serialize this entire, complex state into self-contained, declarative repre- sentation and then perfectly reconstruct the live object graph from that representation for each step of the simulation.
22

---

Published as a conference paper at ICLR 2026
The state export process transforms the live simulation into serializable snapshot. This procedure performs a deep traversal of the game engine's internal state, capturing all information required to reproduce the exact game moment. This includes the grid of world materials, the positions of all entities, and the type-specific attributes of each entity, such as a zombie's attack cooldown or plant's growth progress. Crucially, the process also serializes the state of the engine's pseudo- random number generator, ensuring that the sequence of random numbers for subsequent stochastic events preserved. To maintain the spatial partitioning data used for efficient queries, the set of entities within each world chunk is recorded by storing their unique identifiers. The final output is complete, declarative data structure that represents the world a single point time, free from any live object references or other runtime-specific information.
State reconstruction reverses this process, rebuilding the live simulation from the declarative snap- shot. This is more complex than simply loading data. It involves re-in re-instantiating the entire graph of game objects and correctly re-establishing their inter-dependencies. A key complexity arises from object relationships; for instance, hostile entities require direct reference to the live player object to guide their behavior. resolve this, we employ a multi-pass reconstruction algorithm. First, entities with external dependencies, such the player, instantiated. Then, dependent entities are instantiated in a second pass, receiving references to the already-created objects they require. Once all objects created, the spatial partitioning system is rebuilt by mapping the stored entity identifiers back to the newly created live object instances. Finally, the deserialized state of the pseudo-random number generator is loaded, ensuring that the reconstructed world will produce the exact same stochastic outcomes as the original. The overall process is described in Box 1 and illustrated in Figure 1.
Box C.1| Pseudocode for the Functional Transition Cycle
function FunctionalTransitionn
(decLarative_sstaee. (dectoo // 1. Reconstruct the imperative world from the declarative state snapshot. <- ReconstructWor IdF ronstate
action_t):
// 2. Emulate a single step in the imperative engine.
player (pallallayrrpplayerr action_t) ApplyActionToPlayerrro
for object in world_ins corld_instance. waancc.gee_aaae get_all get_all_objects() sejject.uddater
// 3. Export the world state into declarative representation. declarative_state.tt
return declarative_state.1
function "nportttateerrmmnoo1(ooorrdddddd snapshot new DeclarativeState
snapshot.materialss materials <- htwpr/ddwwooodnisstnneeteteeeeee snapshot. rng_state Serializeeworld__instannn randoee.e.oooodmmdttto for object in world_instance sitttanceeeetaal_ojeetttt.t
Add0bjectState (snapshot, object. type, return snapshot
abject.id)
function ReconstructworrlFrrossteeaa (smapshott: world_instance new ImperativeWorld
world_instance hwr.l__inttaneemmttrrii_eeerr material_grid <- CopyGrid(snapsaie materials)
world_instance. wttpssttanneerrnnm..meemem
<- Deserialize (wapaaiittnsnnntstttootoggggsssi
// Multi-pass object instantiation to handle dependencies.
player_state state FinePaaeerSsateesnnssssooooooo player_object <- InstantiateObject( player_state. type, player_state atareatsttteeetibuuteee..
23

---

Published as a conference paper at ICLR 2026
adlobbeectoohrrlldwedddddiitttt player_object)
for object_state in if not is is_player (objett_oojctsstatt
// Pass player reference to dependent objects (e.g., Zombie). dependencies
{player: player_object)
new_ new_object <- InstantiateObjectt object_state. type, berecttssate.aatriiii dependencies
aadobiiectooworrlddvorrd__nsaaa new_object)
Reeulllsppaialmmeeeovoodl___ssi return world_instanee
C.4 THE FUNCTIONAL ENVIRONMENT INTERFACE
We provide a transition function that implements a stateless API for environment steps:
1. Input: WorldState object 8r
Reconstruct live game engine instance
3. Execute single update tick with given action Export resulting state as S1+1
5. Return new Worlde WorldState object
This ensures every transition is pure function of the explicit state, making the environment suitable for symbolic reasoning and program synthesis.
C.5 UTILITIES FOR PROGRAMMATIC STATE INTERACTION
key contribution of Crafter-OO is rich set utilities that enable programmatic interaction with the world state. These functions essential for purposes: first, they allow for the precise, producible setup of the evaluation scenarios discussed in Section E; second, they provide a high-level API that simplifies the authoring of programmatic world model laws. To provide clear overview of this toolkit, Table catalogues the key functions, which are grouped into three main categories: World Setup, Player State, and High-Level State Queries Modifications.
Table 4: A catalogue of key utilities for programmatic state manipulation in Crafter-OO. These functions provide the building blocks for creating controlled experimental scenarios and for writing concise, high-level world model laws.
Category
Function Signature (Simplified)
Description
tile naterial (pos, naterial)
Modifies terrain specific coordinate (e.g.. changes stone).
World Setup Utilities
renove object from world(obs)
www.ottject... wabject.to.wwrlee to..srdd((ess, pes, ...)
entity (e.g. Removes a specific entity instance from the world.
Zambic) world.
set (level))
Sets the glohal dnylight level, affecting visibiliry and mob spawning.
player position(pes)
player's exact y) Sets the player's facing direction (e.g. up, down, left, right).
set player_facing(de facing(direction)
Player State Utilities
set player inventory iten(iten, aty)
Sets the quantity a specific fic item the player's inventory.
Adjusts internal player
like health, hunger, energy.
get target.. terget.tile()
Returns material and any object at the tile the player facing. Retums all entities of specific type near the player.
Ligh-Level
State
get wbitebbjcccco. of type. updste range(cls)
Queries & Modifica-
(ojact(oo]], dir, walkablo)
Moves entity step the torget tile valid and unocrupied. player facing.
tions
set.faccingnnttriallll (nttrill
Changes material
These utilities are composed to construct the specific initial conditions for our evaluation scenarios. Listing 2 demonstrates how they work in concert to create test case for resource collection mechanic. World setup utilities are first used to clear an area and place a specific resource (coal). Then, player state utilities (wood pickaxe) in their inventory. This level of programmatic control, enabled by the functions detailed in Table 4, is what makes our targeted evaluation methodology possible.
used position the player correctly and provide the necessary tool
24

---

Published as a conference paper at ICLR 2026
def get_initial_star sattttii_ssttt_f___calllll caor_coal_ooleccinn(]]
# Create a base world and get references to the world and player objects
world reconstruct_worldd farlddffoo_sssee htt.oo/d...oosttttttttttaaaaaaaa player = find_ plad_
World Setup Utilities
# Clear 3x3 area around the player to grass
for x in range(4, 7):
for in range(4, 7):
(x, y), "grass")
12
# Place the target resource a specific location world..till..setttill.... wtw.etiillemmeeaaaaeeoeooaa (6, 5), "coal")
13
# --- Player State Utilities
16
# the player's starting position
17
player_utils. set..paaaer_pssiiioooioee
(5, 5))
18
# Make the player face the target resource playeerutill.set..lppeefffffff player_utils ceet_playe__fccinpllleee.
(1, 0))
21
22
# Add the required tool to the player's inventory "wood_pickaxe", 1)
25
25
# Convert the configured world back to serializable WorldState return sotort_woold_staeeeeeee
26
view=(9, 9))
Listing 2: Example of programmatic state manipulation create initial state for scenario.
World setup utilities create the environment, while player state utilities configure the agent.
D MUTATORS
Mutators are a core component of our evaluation framework, designed to test a world model's ability distinguish between plausible and implausible future states, described in Sec. A mutator is a deterministic function that takes a state-action pair (8t, az) and produces an alternative, incorrect next state št+1. These generated states, called distractors. represent violations of the environment' true dynamics. For example, distractor might show the agent crafting an item without the necessary resources or moving through a solid obstacle.
By creating a candidate set containing the true next state 8141 and several such distractors {5 } we construct discriminative task for the world model. model with robust understanding of the environment's laws should assign significantly higher probability to the true outcome than to any of the distractors. This allows us to quantitatively measure the model's predictive judgment using the state ranking metrics from Sec. 4.
All mutators adhere common interface, shown in Listing Each mutator implements a 'precon- dition' method that checks if the mutation is applicable to given state and action. If the precondition met, the 'effect' method called to generate the mutated state. This design allows for the cre- ation of targeted mutators that only apply under specific circumstances, leading more subtle and challenging distractors.
class Mutator:
**"A protocol for functions that generate distractor states.
def preconditionnselff, state: WorldState, action: Action) >> bool:
Returns True if the mutator can be applied to the given state-action pair, False otherwise.
**"
10
def __allllsselll state: WorldState, action: Action) -> WorldState:
25

---

Published as a conference paper at ICLR 2026
Applies mutation to copy of the state and returns the
14
modified state, representing illegal transition outcome.
15
16
Listing 3: The general interface for mutator. Each mutator is callable object with method to check for applicabillity
We have implemented a suite of mutators for the Crafter-OO environment, categorized by the type of game mechanic they target. Tab. 5 provides comprehensive list of these mutators and the specific rule violations they introduce.
Table 5: Catalogue of mutators implemented for the Crafter-OO environment.
Category
Mutator Name
Description of Rule Violation
Physics
lllegalkoeeeenttttttrr
when non-mooementt action is taken. Teleports non-plag player entities random distant locations.
Causes the player
Enti tyPasi tionMutator
PlayerHeal thHutator Enti titityyytathMutattor
Arbitrurily adds subtracts small amount of health from the player. Sets the health of non-player entities random. incorrect value.
Combat
Crafting
CrafttllegalIItehhutttoo
Produces different item than the specified by the crafting action.
Collection
oollectillealMssrrrialiaaarrr
Adds incorrect resource the player's inventory when collecting.
Placement
PlacetllogalIttooutaaatrr
Places a different object or tile the one specified by the action.
Player State
InventoryMutator
Randomizes all quantities the player's inventory.
Below we provide detailed descriptions and simplified implementations for three representative mu- tators from different categories.
ILLEGAL MOVEMENT MUTATOR
This mutator tests the model's understanding of which actions cause player movement. activates when the agent takes action that should not result in change of position, such as noop or do. The effect to move the player one step in a random direction, creating a state that would valid for movement action but is invalid for the action actually taken. Listing 4 shows its logic.
["noop", "do", "sleep", "make_wood "ooke__-od_picaaxe" DIRECTIONS [(O. 1), (1, 0), (0, -1), (-1, 0)]
WWw_MOVEEENT__AATIOO
class IllegalNovementhutttorr def precondi (sel,, state: WorldState, action: Action) -> bool: # This mutator applies only to actions that should not cause movement. return action in NON_MOVEMENTAACTIONS
def call_ state: WorldState, action: Action) >> WorldState: mutated_state = datte.mddel_coyyddeeeee
12
# Choose a random direction and update the player's position.
13
random_direction random. choiceeIIREETIONNS mutated_state.
staeed__sttteeppaeer.. player player.position position.x += random_direction!] random_direetton[1]
mutated_state. www.stttteeppel...eeeeeeeeee s.layer.position
16
17
return mutated_stan state
Listing 4: Simplified logic for the IillegalNovemeethuutaooo.
CRAFT ILLEGAL ITEM MUTATOR
This mutator targets the logic of crafting recipes. It checks if the agent is attempting to craft an item. If so, it alters the outcome by giving the player different, randomly selected craftable item. This tests whether the world model has correctly associated specific crafting actions with their unique outcomes. For example, if the action is make wood pickaxe, this mutator might instead add stone sword the player's inventory. Listing illustrates this process.
26

---

Published as a conference paper at ICLR 2026
CRAFTING_ACTIONS mnake_wood_piccare "make_stone_swon"
class Craftilllegllltemtutttor:::
def precondition(self, state: WorldState, action: Action) >> bool:
# This mutator applies only to crafting actions.
return action in CRAFTING
def call__ state: WorldState, action: Action) -> WorldState: mutated_state = state. ceteeeoeleoooyyddeppppeeee
Select different crafting action determine the illegal outcome, other_ cateer_ccratingaactiiii actions CEPTTINNGACTIOON (action)
12
13
illegal_action random. (twereeiitooeeeerraatiiinn
# Add the item corresponding the illegal action the inventory.
16
illegal_action mutated_staan stttdsstteeeptteeeeeeee swwttt.plaat.............. inventorvy hewventory.ston_sswwe. stone_sword +=
mrake_ston__swodd..
17
18
logic for other craftable items
return nutated_state
Listing 5: Simplified logic for the CraftillleglllIeMMutttorr
ENTITY HEALTH MUTATOR
This mutator introduces arbitrary changes to the health of non-player characters (NPCs), violating the rules of combat, regeneration, and damage. is "always on" mutator, meaning precondi- tion is always true, health can be dynamic property in any state. Its effect is to iterate through all non-player entities and set their health to random value that is not close to their current health. This prevents generating trivial changes that might occur naturally (e.g.. from regeneration) and creates more distinctively incorrect state. Listing shows the implementation.
class EntityyeealtMMuuttor:: thMutator:
def precondi (sel,, state: WorldState, action: Action) bool:
This mutator is always applicable. return True
def call__ state: mutated_state state. cateeeoddee_oocyyeeeeere
WorldState, action: Action) >> WorldState:
for entity in mutated_state. objects::
10
Skip the player entity.
if antity.entity_ia mutated_state http:ssttteepply.eeeem elayer.eentity_dd: continue
12
13
14
Generate new health value that is not the same as the current health, nor inmediately adjacent to it. mossible_heallt__allees (rangee((1))) Health is 0-10
17
excluded_values cnntty.healle entity. health 1, entity. health antity.health valid_new_valuee hist values excluded_
1)
18
19
if valid_ values: :
entity. health random. (nvil,chh val_uuu
22
23
return sutated_state
Listing 6: Simplified logic for the antityHeelthMutttteo
E SCENARIOS
An evaluation framework that relies on data from unguided exploration may not sufficiently cover all of an environment's mechanics, especially those that rare require specific preconditions.
27

---

Published as a conference paper at ICLR 2026
To ensure comprehensive and targeted assessment of world model's understanding, gener- ate evaluation data from a suite scenarios. Each scenario a short, programmatic interaction sequence designed isolate and test single game mechanic under controlled conditions. This ap- proach produces a dataset transitions that robustly covers the environment's dynamics, from basic resource collection complex combat encounters. The transitions generated by these scenarios form the basis for the evaluation metrics described in Sec. 4.
E.1 SCENARIO STRUCTURE AND EXECUTION
A scenario is defined by a common programmatic interface, as outlined in listing 7. It specifies an initial state, a scripted policy guide the agent's actions, and termination condition based on either achieving specific goal or reaching maximum number of steps. The execution of scenario, shown in listing 8, produces sequence of (state, action, next.state) transitions that serve as ground truth test cases for the world model.
class Scenario: @property
def wwm_scenarioosccaaro))) transitions = [
2
def name (self) -> str:
3
state ceenari.get_initaal.co. for in range (scenarii mat.to..aaxssssss
def iet_initial_state(sell. WorldState:
action scenario. potrrioo.oliyy(tatte)
next_state env. transitionsstatee action)
def pseicy state: WorldState) ~>
transitions.co append( (state, action, next_state))
Action:
state next_state
def poal_test(self, transitions: list)
if scenario. goal_test(transiiiinn
-> bool:
break
eproperty
return transitions
12
max_steps(self) -> int:
Listing transitions.
Execution loop for generating
Listing 7: Structure of an evaluation scenario.
E.2 IMPLEMENTED SCENARIOS
We developed over 40 scenarios for Crafter-OO, covering every game mechanic present in the original Crafter environment. These scenarios are categorized by the type of mechanic they test, as detailed in Tab. 6. For many mechanies, ant. The successful version sets up the preconditions for an action to succeed (e.g.. having enough resources craft an item), while the unsuccessful version deliberately violates a precondition. This allows us to test whether a world model understands not only what should happen, but also what should not happen.
include both "successful" and an "unsuccessful" vari-
F EVALUATION IMPLEMENTATION DETAILS
This section provides a procedural specification of our evaluation framework. We begin by defin- ing general-purpose interface that any world model must satisfy be evaluated. We then detail the computational steps that transform the raw outputs of model satisfying this interface into the final State Fidelity and State Ranking metrics presented in Sec. 4. The process relies the evalu- ation trajectories generated from Scenarios (Sec. E) and the distractor states generated by Mutators (Sec. D).
Our evaluation framework is designed to be model-agnostic, Any world model can be benchmarked, provided it adheres the simple, two-method interface shown in listing 9. This interface cleanly separates the two core capabilities required for our metrics: the ability to generate likely future state (for fidelity) and the ability to score given future state (for ranking).
class EvaluaatblleWor1dhodll (Protocol):
"*"A protocol for world models that can be evaluated by framework.
28

---

Published as a conference paper at ICLR 2026
Table 6: Complete list of evaluation scenarios used to world models in Crafter-OO.
Category
Scenario Name
Description
Movement
randomunovement
Tests basic player movement in the cardinal directions.
collect wood collect drink collect
Player faces tree and collects wood.
Player faces water and collects
Player collects stone with the required pickaxe.
rrsuccessful.coleett.go collect_coal
Player attempts collect stone without the required pickaxe.
Player collects coal with the required pickaxe.
unsuccessful.colled... lect_cccc
Player atlempts to collect coul without the required pickuxe.
Collection
collect_iron
Player collects iron with the required pickaxe.
unsuccessful collect iron collect diamond
Player attempts collect iron without the required pickaxe.
Player collects diamond with the required pickaxe.
unsuccess ful_col aall__ooleee wolllect_ddannee amond eat.plannt
Player attempts to collect diamond without the required pickaxe. Player eats a ripe plant to gain food.
unsuccessful.co _eatss_plleattaat
Player attempts to eat an unripe plant.
craft wooden pickaxe unsuccessful craft wooden pickaxe
Player crafts a wooden pickaxe
sufticient wood.
Player attempts craft without sufficient wood. Player crafts wooden sword with sufficient wood.
unsuccessful craft wooden sword craft. stone pickaxe
Player attempts to craft without sufficient wood.
Player crafts a stone pickaxe with required resources.
unsuccessful httuluulaaaattttstteeee pickaxe craft_stone_sword
Player attempts to craft without required resources.
Crafting
Player crafts
sword with required resources.
Player attempts to craft without required resources.
craft.iron.pi pickaxe
Player erafts an iron pickaxe with required resources.
unsuccess craft. iron pl ckaax craft
Player attempts to craft without required resources.
Player crafts iron sword with required resources.
sasuccessull.rrat..rooo
Player attempls craft without required
place_table
Player places crafting table with sufficient wood.
souccessful..lacc.eee.
Player allempis to place lable without suflicient wood.
place_stone
Player places stone with sufficient inventory.
unsuccessful place stone
Player attempts to place stone without sufficient inventory.
Placement
place_furnace
Player places furnace with sufficient stone.
unsuccessful
Player attempts to place furnace without sufficient stone.
place_plant unsuccessful placcepplan
Player places sapling on grass tile.
Player attempts to place sapling without one in inventory.
zombie defeat deeeet.
Player, equipped with sword, defcats a zombic.
Player defeats skeleton.
Combat
Player defcats
obtain food.
player_death
Player with low health is defeated hy zomhie.
NPC Behavior
now.novement wake_up
Tests the stochastic movement
several steps. Player goes to sleep and wakes up after their energy is restored.
def sample_next_statesseff current_state: WorldState, action: Action) >> WorldState
Generative function: Samples a single predicted next state s_hat_(t+1} from the model's 's posterior distribution P(s_[t+1} | s_t, a_t).
10
11
def evaluate_log probabbbbll
12
self, state: WorldState, action: Action, next_state: WorldState float:
14 .*.
15
Discriminative function: Computes log-probability specific next_state given the current state and action.
16
Listing 9: The interface any world model must implement to be compatible with our evaluation framework.
29

---

Published as a conference paper at ICLR 2026
F.1 STATE COMPARISON VIA CANONICAL REPRESENTATION
All metrics that involve comparing two world states, such as edit distance or checking for equality, require deterministic and canonical representation of the state. A direct object-to-object compari- son can be unreliable due to factors like in-memory object identifiers or the ordering of elements in lists. To address this, we serialize each WorldState object to canonical JSON format before any comparison is performed. This process. outlined in listing 10, ensures that two states are considered identical if and only if they represent the same game- world configuration.
def co_canonicaL_jsonsttacc WorldState) dict:
Serializes a WorldState object to deterministic JSON representation.
# Exclude nen-semantic non-deterministic fields from serialization. excluded fields { ("event_bus", "serialized_ sttrrrannddd_stteaaaa_____ serialized_state state state.
mode="json")
# Sort lists of objects by stable, unique key
order invariance.
10
# The player object is handled separately and removed from the main list. state[""bjects"]] obj for obj in stailiied_state"""bjeeeee if obj["name") != "player"
I1
12
seriaalzed_statee"ddpp (errrrreeeee"ssssttteeearaaalrrr obj: obj[ "eniityttyyyd"
15
16
# Chunks are also sorted to ensure map representation is stable.
17
if "chunks" in serialized_statee
seriillize__tttee""hhee "lotunkks].sss sort toeyeltttteeabbbb chunk: chunk["chunk,kee"
19
20
return serialized_stee
Listing 10: Canonical serialization of WorldState object.
F.2 STATE FIDELITY METRIC CALCULATION
The state fidelity metries measure the difference between world model's predicted next state and the ground truth. We use ISON Patch (Bryan & Nottingham, 2013), a standard for describing changes in JSON document, provide precise, interpretable measure of this difference. The calculation for single transition (st, at s1+1) proceeds as described in listing 11.
def salculate_stater wwweeiiitdeeewllleeeee
s_t, a_t, s_t_plus_1):
Computes and Normalized Edit Distance for world model's prediction.
# 1. Generate a predicted next state from the world model. s_hat_t_plus_1= world_model. seapeeell_ammlleeetsstttttessss a_t)
# Convert both true predicted next states canonical JSON. $son_true canonical__soon( (s_otss____u_s) json_predicted = lwaponiicallssosssattttc...
10
12
# Compute the JSON Patch from the predicted state the true state. patch
13
json_true)
14
15
# 4. Raw Edit Distance is the number of operations in the patch. raw_edit_distance = len(lis(ppatch)
17
18
# Normalized Edit Distance is the raw distance divided by the total number
19
# of elements in the true state, providing a scale-invariant measure. dotal_elemente count_elementssjsonteee edi tdittddstttcteeeeeeeeeeeeeee raw_edi t_ittdistancee / total elements if total_element else
22
23
return raw_edit wwweeddddttaeeeeee
Listing 11: Calculation of State Fidelity metrics for single transition.
30

---

Published as a conference paper at ICLR 2026
Example. Consider a transition where the player, position (x: 5. y 5) with health 9, takes the action move_right. The true next state, S4+1. has the player at (ær 6, = 5) with health 9. Suppose world model predicts state, $e+1, where the player correctly moves to (x = 6, 5) but their health incorrectly drops to 8.
The simplified canonical JSON representations for the player object in each state would bes
2
"player":
2
"player": {
"position": ("x": 6, "y": 5),
"posit position"; ("x": 6, "y": 5), "health": 8
"health":
}
}
Listing 12: Canonical JSON for the true next state.
Listing 13: Canonical JSON for the predicted next state.
The JSON Patch required to transform the predicted JSON into the true JSON is single replace ''path': "/player/healthh
operation: [f''''': 'replace'
'vvalee': 9}]. The Raw Edit Distance is the number of operations in this patch, which is 1. The Normalized Edit Distance would be this value divided by the total number of elements in the true state's full JSON representation.
F.3 STATE RANKING METRIC CALCULATION
State ranking metrics evaluate a model's 's ability to distinguish the true outcome of an action from set of plausible but incorrect alternatives. This process involves generating set of candidate states and using the world model to score them, as detailed in listing 14.
def satnlatte_stttt__annnnnggggge
model, s_t, a_t, s_t_plus_1, mutators, nun_distractors
): **.
Computes Rank@1 and Mean Reciprocal Rank for a world model.
# Generate of distractor states using the mutator bank. distractors []
applicable_mutaan mwmatutttorrssoooooo [m m mutators if m. preconditionns.tt a_t)]
random. (wwpssssfffelaapllaaaaa (wtpppppppllleeeeaaaaaaaaaaaaapp _mmbllorsutttos) # Ensure variety in distractors for mutator in applicable__mterr mutmtoostss
if (distraacts num_distractors:
11
break
12
distractors, (llstractorssaappeddmiiis (mpend( a_t))
13
# 2. Form the candidate set, including the ground truth and distractors.
15
candidate_sed [s_t_plus_1] distractors random. www..ssuufflleccnddddee.eee
16
Avoid biasing models that may sensitive order
17
# 3. Score each candidate state using the world model's log-probability function. scores []
20
s_candidate in candidate_set.
21
log_prob = world_model. www.deeweealeeteeeeeoemeaeeoaeee evwluate_log pegauaate_og_probbbiiiyyy... a_t, s_candidate) scores. apog
22
# Determine the rank of the true state.
25
# Ranks are 1-indexed, with rank being the highest score.
26
ranked_indices = sor (Lerrrnggee((ssorss))) key=lambda i: scores[i], reverse=True) true_state_index index candidate_set. index(s_t_plus__1 rank wtw.oofftru__ataeeee
ranked_ Lanked_indices index(true_state indee)
29
30
# 5. Calculate metrics from the rank.
rank_at_1 1.0 if rank_of cocoooftreeseee tttet_stttteeeeeeeee = 1 else 0.0
reciprocal_rank = 1.0 took_of_tuue_stteee
31

---

Published as a conference paper at ICLR 2026
return rank_at_1, reciprocal_rank
Listing 14: Calculation of State Ranking metrics for single transition.
Example. Continuing the previous example, the true state St+1 the player moving right. A mutator might generate distractor state Sdistrcttor where the player illegally teleports to (z = 20, y = 20). The candidate set becomes {8t+11, Sdistractor.. A good world model should assign much higher probability to the true outcome. For instance, it might yield log- prgbobbbbiliiies of log p($(+1
-0.7 and log (ddistrratorr ...) 15.4. Since -0.7 -1..4, the true state is ranked first. This yields Rank @ of 1.0 and a Mean Reciprocal Rank of 1/1 1.0 for this transition.
F.4 AGGREGATION ACROSS SCENARIOS
The final metries reported in Tab. 1 are aggregated from the per-transition results. To ensure that each distinct game mechanic contributes equally to the final score, we employ two-level aggregation strategy. First, we compute the mean metric values across all transitions within single scenario. Second, we compute the final reported metric by taking the mean of these per-scenario means. This prevents scenarios with more transitions (e.g., long movement sequence) from dominating the overall results compared to scenarios with fewer, more critical transitions (e.g.. single crafting action). listing 15 formalizes this entire pipeline.
def evaluate_worlde wwwoordddmeeeeoeeeeeeee (worddlwwwwdddddddeo scenarios, mutators, config):
wen
Runs the full evaluation pipeline and returns aggregated metrics.
wrr_scenario_metriie
# 1. Evaluate each scenario independently.
for scenario in scenarios: transitions uun_scenario((cemario)) See Sec. C.1 for run_scenario
10
11
scemario_results
12
for (s_t, a_t, s_t_plus_1) in transitions:
Calculate metrics for each transition in the scenario.
r_at_1, aalculate_state_rantte world_model, s_t, a.t, s_t_plus_1, mutators, config. nundaittr
15
16
17
raw_ed, norm_ed calculate_statefffee world_model, s_t, a_t, s_t_plus_1
19
20
scenario_resil
wttpriiorrssuts..pppp.. appennd "R@1": r_at_1, "MRR": mrr,
21
"RawEditDist": raw_ed, "NornEditDist": norm_ed })
24
25
# 2. First level of aggregation: average metrics within the scenario. if not ccenario_results: continue por_ssenarri_mmen sttoometrrrssssseaiiioooo (ameeee =
26
key: sum(res[key] for in scenario_results) Len(scenario_resulle for key in scenario_results(00
29
30
# Second level of aggregation: average the per-scenario
final_metrics
34
key: sum(merrics[key] for metrics in per_scenario_metricss values()) / len( per per_ssennrro_metrre for key in sistppprrscc wtt.eerssseerrriieeereesssaaaaaa
37
38
return final_metrics
Listing 15: Overall evaluation pipeline and metric aggregation.
32

---

Published as a conference paper at ICLR 2026
G SYNTHESIS AND EXPLORATION IMPLEMENTATION DETAILS
The process of generating candidate world laws is divided into two main stages: unguided explo- ration to collect a dataset of interactions, and law synthesis to propose programmatic laws from that dataset.
G.1 EXPLORATION POLICY
To gather the interaction dataset D = {st, 81481111 1) employ autonomous exploration policy driven by large language model. This policy operates without access to environment- specific rewards numan-provided goals. Instead, is given high-level instruction to explore the environment and discover as many its underlying mechanics as possible, treating the task as reverse-engineering problem. The full prompt provided to the exploration policy is detailed in box G.1.
Box G.I| Exploration Policy Prompt
You are an explorer in an unknown digital world. Your mission is to experience as many of the world's s hidden mechanics possible. Your recorded experiences will be analyzed later to create complete map of the world's s physical laws.
The laws of any world can be thought of as IF-THEN hypotheses: "IF (a specific situation occurs) AND (you take an ACTION), THEN (a certain outcome happens)
To succeed, you must trigger as many different "IF-THEN" scenarios as you can.
**What to Expect in the World:**
This world is complex and may be dangerous.
**Hostile Entities:** You may encounter creatures that are hostile and will attack you.
*xResource collection:** The world contains raw materials that can be gathered, though there may be preconditions tions for collection.
11 *xItem Production: You have the ability to craft useful items from raw materials, though there may be preconditions for production.
12 #*Combat:** You can engage in combat with the entities you encounter.
Your primary goal is discover the rules governing these activities.
is will need to explore the game world by moving around and interacting with the entities and materials in the world.
16 If an action has effect, you may not have fulfilled the preconditions for the action have effect.
17 Try out a variety of actions from each category: movement, interaction, placement production.
18 If an action seems to have no effect, you not have fulfilled the preconditions for the action to have an effect. Try acquire additional resources change something about the world and try again.
20 Before taking actions, set goals for yourself in an IF-THEN format, and let the results invalidate those actions.
If entity is hostile, you attempt defend yourself from it.
22 If an entity seems passive or beneficial, you can attempt to interact with it.
23 You will likely need progress through the "tech tree" specific order.
game in
This will require interleaving resource collection with placement of crafting stations and production of better tools.
25 In the meantime, you will need to survive hostile enemies find ways to heal from damage you' ve taken.
26 Some resources likely cannot be acquired without first producing tool to acquire them.
33

---

Published as a conference paper at ICLR 2026
27 Tools may require mix materials and crafting stations produce.
28
29 The following are the only valid actions you can take:
faction_strings
33
You will now receive observations from the world. Begin your exploration.
This LLM-based policy is crucial for gathering sufficiently diverse data in hostile environment like Crafter-OO. purely random policy survives for an average 100 steps before the agent per- ishes. In contrast. LLM-based policy navigates the environment for average of 400 steps. Despite this improvement, exploration remains a significant bottleneck. The policy often struggles progress through the environment's technology tree, frequently failing to discover the necessary preconditions for crafting advanced items. It also exhibits a tendency to forget previously learned information, which prevents from effectively building upon past successes within a single trajec- tory.
G.2 LAW SYNTHESIS FROM TRAJECTORIES
The law synthesis pipeline processes the trajectory data from the exploration phase generate set of candidate laws } The core idea is to identify state transitions where meaningful changes occur, and then prompt a large language model to propose atomic, programmatic laws that explain those specific changes. This process is outlined in Algorithm 17.
Change Detection for Tractable Synthesis. In environment with complex, structured state like Crafter-OO, changes between timesteps are often sparse and localized to specific sub- components. To make law synthesis tractable, we first isolate these localized changes to provide focused context for the synthesizer. This is achieved through set of detectors that monitor different aspects the world state. An aspect semantically-cohesive subset of the state, typically corre- sponding tep-level attribute (e.g.. 'player.inventory") (e.g., all StatbieState" objects). For each transition (st, a, $1+1), we check for changes across all aspects. If detector identifies change, synthesis task is created for that specific transition and aspect.
collection of entities of the same type
class ChangeDetector:
def aspect_name (self) >> str:
def (as_changessself, s_t: WorldState, s_t_plus_1: WorldState) ~> bool:
class Pltperttweetyyyllaeeeeecrcrrhhhe
def arsectt_ameeselll return def sesscchanessseell s_t, s_t_plus_1)
return s_t.player. inventory != s_t_plus_1 www.ll.ppll..mmmmm. inventory
10
class Zemmiiestteeeenngeeeeeecerrtttte
11
def aspect_nad name (self): return "zombies"
12
def jas_chhngeesselff s_t, s_t_plus_1): # Logic to compare zombie states between and s_t_plus_1
14
15
16 # A list of all detectors is used to check each transition
ALL _LL_EEECTTOSS
Player Twterreeeeencychaaeeeeeettoooot tewailesstttchangeeeetee # Other detectors for map tiles, cows, etc.
19
20
21
Listing 16: Simplified change detection logic. Each detector checks for changes in a specific part of the world state between 8, and 81+1-
This decomposition is not form environment-specific guidance but rather a generic mechanism derived directly from the structure of the state representation itself. The Crafter-OO environment
34

---

Published as a conference paper at ICLR 2026
exposes an object-oriented state, defined by a schema of classes and attributes. Our change detec- tors mirror this schema, creating one detector for each top-level attribute and for each object type. This approach provides structural inductive bias that the environment's causal mechanisms likely aligned with its object-oriented structure without embedding knowledge of the environ- ment's actual dynamics. The process could be fully automated for any environment that exposes typed, structured state; the detectors can be generated programmatically by reflecting on the state schema. This analogous how computer vision model might process distinct objects in scene separately; we partition the state space based on its given structure, but the rules governing the interactions between these partitions must still be learned from scratch.
Prompt Generation. For each transition transition-aspect aspect pair that triggers a synthesis task, we generate detailed prompt for the LLM. The goal is to provide all necessary context for the model to infer the underlying game mechanic. The prompt contains several key components:
1. The initial state and resulting state S1+11 serialized to structured format (JSON). The action a, that caused the transition.
3. A textual 'diff" that highlights the exact changes between 9, and 8t+1.
human-readable 2D ASCII rendering of the local environment around the player for both states, providing spatial context.
5. The name of the aspect (e.g.. "player inventory") that changed, which instructs the LLM to focus its analysis.
This structured presentation the transition allows the LLM to ground its reasoning in the specific, observed changes. The full prompt template is provided in box G.2.
Box G.2| Synthesis Prompt
#if Role
x*World Law Synthesizer** an expert analyzing game state transitions and extracting the underlying rules that govern virtual worlds. Your job is to observe how actions transform game states and codify these transformations into precise, executable laws that can model game mechanics, well try model aspects of the underlying transition dynamics functions.
#tf Task Description
Given world state, action taken, aspect of the state interested in modeling, and the resulting next world state (plus diff highlighting the changes), you must:
Identify how the aspect of the state we are interested in modeling changed
between the observations
Determine the underlying rules laws that caused these changes Implement laws as executable Python code using provided WorldState interface and DiscreteDistribution for predictions
**IMPORTANT: You should write MULTIPLE laws when observe multiple distinct changes ** Each law you write should be modular, minimalistic, focused single game mechanic, and capable being combined with other laws to model complex game behavior.
11
In particular, you should strive write laws that responsible for little of the state possible. any given transition, you may see many changes Each these changes could be caused by a different law. Think about what changes could be grouped together into a single law, and write separate laws for different types of changes.
14 Break up laws to each account for a single precondition
effect. For example, an entity moves, write a law for the movement of entities of that type. If a player takes particular action, write law for that action specifically.
35

---

Published as a conference paper at ICLR 2026
15 Certain attributes cannot have a DDiscreteDistribtiio applied to them. example, the "materials" field should just be modified directly, not wrapped in "DiscreteDistribution Alternatively, use "set_material or to modify the materials field. Either way, they cannot be wrapped a DiscreteDistributtinn
set
16
"DiscreteDistributiin class indicate probabilistic predictions, for example when trying to write a general law governing all entities of a type when you cannot reconcile all changes visible to that entity type into a deterministic Law.
17
NOT need to use imports. Everything you need can be coded wiohuut the use of imports, and all classes defined below are already imported.
18
## Aspect of the State
You will be given an aspect of the state you write should be focused on modeling changes to this aspect the state. 21 However, you can use _all_ of the state to help you write the laws, as the aspect of the state may be influenced by other aspects of the state.
interested madeling. The laws
For example, if told to focus Zombies, you should write laws that govern the behavior of Zombies. This behavior may be influenced by other parts of the state such as the player's actions or position.
23 If told to focus on the player, you should write laws that model how the player's state changes. Again, these effects may be influenced by the entities that the player is interacting with.
25 #i Guidelines for Writing Laws
26 Some laws may be dependent on an action being taken, particular state of the world, while others may always apply. For these, the grecondition can always be "True".
27 Make use of "adjacent_ prticcen__topplayeee to_peyyer' about interactions between the player and other entities.
get_target__tll to help you write laws
Do NOT use "entity_id" when writing laws. You should instead write laws that apply type of entity, e.g. "ZobbeeState" or "CowState".
When modifying attributes, use RELATIVE assignments rather than absolute assignments. For example, instead changing a entity's position via entity. position. x x iasssrteeisstbbuciiooopppprreeee
use "entity. tiyy.posiiionn
position. deltal)". The only exception to this when modifying the materials field.
30 Use helper functions "fet_object_of..tye...n iegeii__uddtetrrane"" and rather than writing your own iteration logic.
get_ bobjjcts_in_uudat__rree 31 You DO NOT need to use the "entity_id" attribute. Use "get_target_tile" to get the tile entity targeted by the player. Use adjacentt_oopllayer to check if entity is adjacent the player for interactions between the player and other entities.
32 Consider writing laws that make "soft" predictions. For example, if you see an entity moving but are unsure if it is general principle, can assign discrete distribution the entity's position represent your uncertainty httpre/ttsssrruuuoooouuuurrrreee
Example: contity.position...
position.x delta_a,
delta_b, ...])"
33 You can speculatively pose laws, but these should go last. Speculative laws are those that were not directly observed in the transition, but those that you believe might exist. For example, given that you have identified about certain crafting recipe, you can speculatively pase law about _other_ crafting recipes that you believe might exist.
36
37 ## Formatting Instructions
38 Structure your response exactly as follows. **You can write multiple laws by repeating the pattern below for each law:**
40
xml
41 <keyChanges>
List the specific, concrete changes that occurred between the observations:
36

---

Published as a conference paper at ICLR 2026
B What entities appeared, disappeared, or moved
44 What stats/values changed and by how much
45 What items were added/removed from inventory Any other measurable state differences </keyChanges>
48 <naturallanguagel.a>
49 Write a clear, concise description of the game rule that explains these changes: What triggers this law (the precondi tions) What the 1aw does (the effects/transsormationss
52 Any important parameters or variations
53 Give the 1aw a descriptive name
54 </naturallaaguaggelaaa
<lawCode> python
57 class YourLawNameHere:
58
def (seff, param1: type default_value, param2: type = default_value
):
"""Initialize with configurable parameters.
self.param param1
61
self.param2 param2
62
# Add any lookup tables or constants here
def tf current_state: WorldState, action: str) bool: this law should apply the given # Implement your precondition logic here
"*"Reuurn
action. .."
66
67
# Check action type, entity presence, player state, etc.
return False # Replace with actual logic
70
(seff, current_state WorldState, action: str) -> None: """Apply the law by modifying the world state.
71
# Implement the state transformation here
# Modify entities, player stats, inventory, etc.
74
Use ficcreteiistriuuuuioouuuottttttt
deterministic
predictions
75
# Example: current_state, htt.ttssttee..paee.....em health DiscreteDiittribtttinnuuupptt
new_health]
pass Replace with actual implementation
77
78
</lawCode>
79
<keyChanges>
[Changes for second law...]
82 </keyChanges>
83 <naturallanguagel.a
[Description of second law...]
</naturallanguagelaa>
86 <lawCode>
87 phot
88 class YourSeconddLawNameterrs
# [Implementation of second law...]
90
</lawCode>
92
**Critical Formatting Notes**:
95
**Write multiple laws when you observe multiple distinct changes** each law should focus on a single type of change
96
Use exactly these XML-style tags: "<keyChanges>", "<naturallanguagele>>>
lawCode>*
Close each tag properly: "</keyChanges>", <</naurrallangusgelaa>>> *</lawCodes"
9%
Python code inside triple backticks within "<lawCode>" section
99 Be precise and specific in the key changes use exact numbers and entity names from the observations
37

---

Published as a conference paper at ICLR 2026
100
Make the natural language law description clear enough that another programmer
could implement independently
101
Only output the code for the law, not the entire file. Assume the "WollSState" already defined.
class as well its components
Format your response well, with newlines between the tags and code blocks. **Each law should be completely self-containedkr repeat full structure for each law you write.
103
#T WorldState
world state Pydantic model that represents the complete game world state The world laws you write will operate on this state.
107
python
{{ lerrld_state__hhieeeee }}
110
111
112 # World Laws
Each world law must conform the following interface:
115
python
116 class WorldLaw:
def preconditionnself,, current_state: WorldState, action: str) -> bool: "*"Return True if this law should apply the given state and action.
120
def effect current_state: WorldState, action: str) <> None:
"""Apply the law by modifying the world state. Use batsrrtteiisrrbbbiiioouuorrtirit
deterministic
predictions
124
# Example: current_state. player. health Disereteliisrrbbutloossuppppp
new_health])
127
You may add any additional fields or methods to the class as needed.
128
## Di DiscreteDistribution Usage
when modi fying state values your law's "effect" method, you must wrap the values with DiscreteDistrrbution:
131
132
python
For deterministic predictions:
current_state. some. value Discrteelisriittttoouuppprrrttto
135
136
For stochastic predictions (if needed): current_state some_value
value2, value3])
140
The DiscreteDistribation class represents probabilistic predictions over discrete values. For deterministic laws, typically provide a single value in the support list. For stochastic laws, you provide multiple values support
represent the possible outcomes.
142 When accessing the materials field, pay attention to the 'MaterialT' type. Everything in the "materials" field is "MaterialT'. Do not use the emojis in the world map, they only there for your convenience.
143
144 # Your Turn
145 ## Aspect of the State
Focus modeling changes to the following aspect of the state:
{{ aspect_of_staat
149 ## Focused Changes for {[ aspect_of_state }}
{{ aspect_changes })
38

---

Published as a conference paper at ICLR 2026
152 ## View Legend
153 {{ view_legend }}
# State
156 json
157
{{ state }}
#iht Local View
161 {{ local_view }}
162
## Action
action taken was: "(( action }}"
Law Generation and Parsing, The generated prompt is sent to an LLM, which is instructed to return one more atomic laws that explain the observed changes for the specified aspect. An atomic law is simple, modular rule focused on single game mechanic. The LLM's response is formatted using XML-style tags to clearly delineate the key components each proposed law.
The expected format for single law is:
<naturallLaguuggeLaw.. htt.://aw...aaa.....utuuuaaaaana </naturallLnguagelaw
<lawCode>
python
class LawName: def
state, action) def effec state, action):
</lawcode>>
We parse this semi-structured text to extract the natural language description and the executable Python code for each proposed law. This is done by searching for the corresponding tags and extracting their content. The Python code is then loaded as candidate law for the subsequent parameter inference stage.
def hatrsiieeeeas__rorrreraaeeeee syccheesiieLlw___rrrrrrrrrrrrrrn treeeetrrreeetttyytrreettrty List[Transition)) list[Law]: candidate__awe
# Iterate over all transitions from the exploration data for transition in trajectory: s_t, action, s_t_plus_1 transition
1. Detect which aspects of the state have changed Leanged_aspects = []
for detector in ALL_DETECTORS:
11
detector detector.has__cangee. has_changes(s_t,, s_t_plus_1):
12
changed_appects.ppene(((( (ppend aspect_name
13
# 2. For each detected change, generate laws for aspect thanged_aspects.
15
16
2a. Render detailed prompt for the LLM prompt state=s_t,
17
action=action, next_state=s_t__p t__pusss_________,,_ sspect_of_state=aspee
20
21
22
2b. Query the LLM synthesize laws
39

---

Published as a conference paper at ICLR 2026
1lm_ renporesppse_eett call sell_1impprompt)
26
27
2c. Parse the response to extract structured laws parsed_ laws = parse_laws__ response candidate_laws www.waaww.eeeeeee....eme.e extend extend(parsed__awss
28
31
return candidate__i laws
Listing 17: High-level overview of the law synthesis pipeline.
Box G.3| Zombie Fighter Plan
def htt__ooodn_swwrd_pllel
state: WorldState,
transition_i Callable[[WorldState,, CrafterAction], WorldState],
num trees: int
-> WorldState:
frees__choppdd
pathfind_option PlayerPathFinddptiinn
lambda s: Find__cloeett.aatrrrll...o....e "tree")[1]
interact_ PlayerIneerackkajjcnnnttoooo lambda find_clloest.matrrialoooooo
"tree")
12
13
# Gather wood by iterating between pathfinding and interaction while trees_chopped < num_trees:
16
try:
17
action
18
except TerminationCondition
action interrct_opptio..... action
if action "do":
21
crees_chopped
22
state transttion_fnnstatte, action)
# Place crafting table and craft sword
25
state transition_fn(statee, "place_table")
26
state transition_fn(state, mmake_wood_sworde
27
return state
30
31
def zombies_plant
state: WorldState,
transition_fis [allablle[ CrafterAction], WorldState], zombie_ids: list[int],
34
35
max_steps_per__ombll int 10
36
-> WorldState:
for zombie_id in zombie_ide combat_option fondefrfiieeittttyooooooooododdd
39
40
for in range (maxxxxxttsss_per_zonnll
11
try:
action state caansstion_fnsstttttitt except TerminationCondittin action) break Zombie defeated
43
44
45
return state
48
49
50
def sword_ther www.then_zeee sahen_zombies_plan
state: WorldState,
40

---

Published as a conference paper at ICLR 2026
52
transition Callsale_[Worldstatee,, CrafterAction], WorldState],
53
zombie ids: list[int]
54
-> WorldState:
High-level plan: Craft weapon before engaging combat. Composes two sub-plans **"
37
complete strategy.
58
# Sub-plan 1:
Obtain weapon
state _ wtw.wodee_ssrrpppllasseeeeeeee transition_fn, num_trees=3)
61
62
# Sub-plan 2: Defeat enemies
63
state srffet_zoobbee_plann(ttt transition_fn, zombie_ids) ids)
return state
H PROBABILISTIC MODELING OF PURE FUNCTIONS
In this section, we clarify the distinction between the environment's pure functional interface and the choice learn probabilistic world model.
We use the term "pure function" in the formal computer science sense (referential transparency and absence of side effects) (Backus, 1978), following the design patterns of modern JAX-based environments like Brax (Freeman et al., 2021) and Craftax (Matthews et al., 2024). In Crafter- O0, the transition function explicitly takes the entire state (including the PRNG state and entity attributes) as input. This ensures referential transparency, since calling the function with the same inputs guarantees the exact same output, whereas standard environment would diverge due to hidden state mutations.
# Standard "Impure" Environment # Hidden state (e.g., rng, cooldowns) mutates inside obs1 env. mtv.stepacction)
obs2 env e stv.step(accion)
Result: obsl != obs2 (The hidden state changed between calls)
Crafter-00 "Pure" Function
state explicit; no side effects s1, rng1 transition(state, action, rng) 10 s2, rng2 transition(state, action, rng)
# Result: s1 == s2 (Identical inputs guarantee identical outputs)
Listing 18: Comparison of impure vs. pure environment interfaces.
Although the transition function is pure, meaning deterministic world model is theoretically pos- sible if the agent could perfectly model the evolution of the global PRNG state, such a model is difficult learn in practice. It requires overfitting the simulator's serial execution order, which is undesirable for several reasons.
PRNG Scheduling. In a simulation with shared global PRNG, the exact next state depends on the order in which the RNG is consumed. For example, if the simulator updates zombie then zombie.b, the RNG stream advances differently than if the update order were swapped. To learn deterministic model, the agent would have to perfectly replicate the simulator's 's internal scheduling logic rather than what commonly understood as "law of the environment."
Micro vs. Macroscopic Physics. We motivate this using the distinction in statistical mechanics between micro-state trajectories and macroscopic laws. While the motion of every gas molecule in classical physics is theoretically deterministic given precise initial conditions (the "micro-state"), attempting to model these trajectories is intractable and brittle, analogous to overfitting the simula- tor's execution trace. Instead, aim discover robust "macroscopic" physical laws (e.g., "zom- bies move randomly but chase the player when closer than 5 units"", which requires modeling the distribution of outcomes to capture the valid aleatoric uncertainty.
41

---

Published as a conference paper at ICLR 2026
Rank Distribution: Histogram
100
ONELIFE (Ours)
PoE-World
Frequency
20
Rank
Figure 7: Distribution raw ranks assigned by PoE- World vs ONELIFE's world models our evaluation. Generally, ONELIFE's world model is better assigning a high rank to the ground truth state.
Effect on Hypothesis Verification. This distinction determines how we validate our understand- ing of the world. Validating a deterministic model requires verifying the microscopic trajectory, since the predicted attribute value must exactly match the observed value. This brittle; law that is correct (carries out the same computations as the true transition function) may be rejected simply because the specific observed path the RNG probabilistic formulation allows to validate macroscopic laws. By evaluating the likelihood of the observation under the predicted distribution, we can confirm that law is distributionally correct without requiring the prediction match the specific, arbitrary path of the simulator's RNG.
a different outcome than predicted. contrast,
I BASELINES
Random World Model: A model that assigns uniform probability to all candidate states in the discriminative task. Its performance is equivalent to random guessing and serves as a sanity check for discriminative accuracy.
WorldCoder (Tang et al., 2024): A model-based agent that synthesizes a Python program for the transition function using an LLM. employs an iterative refinement strategy, prompting the LLM to debug the code when it contradicts observed data. Crucially, WorldCoder assumes the envi- evaluate how well monolithic. deterministic program synthesis approach copes with the stochastic dynamics of Crafter-OO.
ronment is deterministic; we include
(Piriyakulkij et al., 2025): state-of-the-are symbolic world model that scaled sym- bolic world modeling domains like Atari Both PoE-World and ONELIFE represent the tran- sition function as weighted product of programs, though the structure of the programs and in- ference algorithms differ. Because PoE-World law synthesis component is Atari-specific and relies on online interaction using numan-providee goals, we reimplement this baseline with our exploration policy and law synthesizer, noting that this makes it stronger baseline (without these changes, PoF-World's Atari-specific implementation would be fundamentally incompatible with Crafter's state).
VISUALIZING RANK DISTRIBUTIONS
42

---

Published as a conference paper at ICLR 2026
Rank Distribution n by Scenario (Violin Plots)
Crift bes Sward
Crift Stess Ridass
Crift Steee Sword
Coeft Woodes Pickees
Place
Place lant
Plecs
Mace Tabe
Resdem Mosorent
Make De
Zuebie Sehett
Milel
10s9
VOURE
Figure 8: Distribution of raw ranks assigned by PoE- World vs ONELIFE's world models our evaluation, broken down by scenario. Across most scenarios, ONELIFE's world model is better at choosing the ground-truth next state.
43


Reviews:
[{"review_id": "7dXekGu6gc", "reviewer": "Reviewer_CLdf", "summary": "This paper introduces ONELIFE, a framework for learning a symbolic world model from a single, unguided \"one life\" exploration in a complex and stochastic environment. The core of ONELIFE is to model the world's dynamics as a \"mixture of laws\", which are programmatic rules with preconditions and effects. The system uses a probabilistic programming approach to infer the importance of these laws, efficiently routing credit only to the rules that are relevant to observed changes in the state. To test this method, the authors also developed Crafter-OO, a new, complex testbed based on the Crafter environment, which exposes a fully symbolic, object-oriented state. The authors demonstrate that ONELIFE is superior to a baseline in its ability to rank plausible future states, suggesting it learns a more accurate model of the environment's rules.", "questions": "See Weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 4, "contribution": 3, "strengths": "- The introduction of Crafter-OO is a valuable contribution to the research community, offering a new, complex testbed for agents that must operate in dynamic environments with a mixture of underlying laws.\n- The paper tackles a challenging and important problem setting: learning a world's rules from minimal, unguided interaction in a hostile, stochastic environment.", "weaknesses": "- The setup, while realistic in its limited interaction budget, makes several unrealistic assumptions. It relies on full, symbolic observability of a structured state, which is rare. The \"one life\" constraint also feels artificial for many real-world tasks (e.g., robot manipulation) where mistakes are often recoverable.\n- The technical novelty of the method itself is somewhat unclear. Domain inference, state tracking, and programmatic models are not new. The paper could do a better job of positioning its technical novelty against this prior work, beyond just the Crafter-OO and \"one life\" setting.\n- The paper is unclear about the environment's specifics. For instance, it's not immediately clear why the object-oriented state isn't just described as a PDDL-based representation. A clear list or description of the agent's action space is also missing, making it hard to grasp the task's complexity.\n- The framework is only evaluated on Crafter-OO. While this is a new and complex environment, testing on only one domain makes it difficult to assess the generalizability of the ONELIFE framework.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "diJY17LY01", "reviewer": "Reviewer_NpK1", "summary": "This paper learns a symbolic/programmatic world model for Crafter. It uses LLM for exploration to collect data for training world models. It then asks LLMs to generate codes as world models given the collected data. The programs are modeled as a probabilistic composition of simpler programs/laws, taking advantage of the decompositionability of states to speed up program synthesis. It also builds a Crafter-OO environment together with some evaluation metrics.", "questions": "* What are the differences between the program synthesizers in this paper versus PoE-World, and why are the differences important?\n\n* Are there metrics involving long trajectories and goals? How good are the learned world models from that perspective?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The new Crafter-OO environment is interesting. It's more stochastic (like the random movements of zombies) than the previous environments used in the field. It's good to see that symbolic world models also work in the Crafter environment. \n\nThe paper is generally well-written.", "weaknesses": "It's hard to tell the differences between program synthesizers in this paper versus PoE-World, and why the differences are important.\n\nAlthough it is informative to check if the next states are predicted correctly, it would be much better to use metrics that involve long trajectories and goals, such as the success rates of solving a problem in Crafter using the learned world model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rten9JvkWR", "reviewer": "Reviewer_SHXj", "summary": "The paper presents an approach to learning symbolic world models from observations in an MDP.\n\nIn the proposed framework, a world model is a weighted set of “laws,” each law being a pair of a precondition and an “effect” function that modifies attributes of an input state given an input action. The laws are synthesized by prompting an LLM, and, given the laws, the weights are optimized by maximizing with gradient descent the likelihood of a set of transitions.\n\nThe authors conduct experiments on a custom environment that reimplements Crafter (a popular RL environment) but exposes the entire state in text form. The experiments are conducted in a controlled (but significantly limiting) setting and compare the proposed system to a recent baseline, showing some improvement.", "questions": "Questions:\n- Why are the weights necessary? My guess is that they serve to “soften” the preconditions (which are deterministic predicates), but there is no actual motivation for the inclusion of the weights in the manuscript.\n- What is the space of laws? In the Appendix it can be observed that you have a (custom?) library with which you specify the distributions of attributes for the next state, but it is never stated what space is covered with the library (e.g., what types of distributions can be represented).\n- What is the experimental setup for measuring the rank? My understanding is that 10 “candidate next states” are generated with a method that leverages domain knowledge to create plausible candidates (one of which is the actual ground truth next state), and then the rank of the ground truth next state among that set is measured. Is this correct?\n- Why use reciprocal rank instead of (raw) rank?\n- In line 281 you state, “The policy is not provided with specific knowledge of the environment,” but in Box F.1 it can be seen that the exploration policy does leverage domain knowledge (e.g., 1588: “you may encounter creatures that are hostile and will attack you”). This must be clarified.\n- Experiments are conducted per scenario, where scenarios. Does this mean that a full model of the environment is never synthesized? I.e., a model that captures all the behaviors that could happen.\n- The transition function of the environment is described as a “pure transition function.” Is this accurate? If so, why not synthesize a deterministic world model?\n- Have you evaluated the model predicting multiple next states in sequence? Good-enough performance in this is necessary for model-based planning and training. Based on your results (18% rank @ one), I would expect the model to perform poorly (it is still useful for the community to know that this kind of system does not perform well enough).\n- Why did you not include other systems as baselines (e.g., WorldCoder)? Your baseline is adequate, but a single data point to compare to is not ideal.\n- Is the core difference between the proposed system and PoE that the proposed system aims to synthesize “atomic” laws?\n\nSuggestions:\n1. You must correctly state that the exploration policy leverages domain knowledge to get meaningful trajectories (Box F.1). This is important because the quality of the trajectories influences the synthesis of the world model.\n2. I suggest fully reworking the description of the synthesizer (line 288), as it cannot be inferred at all that the synthesizer is a custom routine that calls an LLM to synthesize a law for each detected change in the state (Listing 17).\n3. I suggest you describe (with some level of precision) the set of laws that can be synthesized. In the Appendix it is shown to be Python programs that use a library to describe the distributions of each attribute of the state in the next state. This should be stated in the main text, as it informs the reader about the level of expressivity of the system and the types of programs that are being synthesized, both of which are important to understand the work.\n4. I suggest you provide a visualization of the ranks of the states (e.g., a scatter plot with a corresponding histogram). This would help the reader better understand the performance of the model (and baseline ideally).\n5. The laws are stochastic (as can be seen in, e.g., Box A.4), but in line 237 you use notation typically reserved for (deterministic) functions. I suggest making the definition of all parts of the world model mathematically precise.\n6. If the per-scenario experimental setup implies that a “full” world model (i.e., a model that covers the entire state space) is never synthesized, then this should be stated in the manuscript, as it helps the reader understand the scope of the experiments.\n7. Motivate the use of the law's weights. Ideally this is done with both a conceptual explanation and an ablation study.\n8. Adding another baseline would significantly improve the experimental section.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- Significance: the work leverages LLMs to synthesize “reactive” laws that are used to parametrize a world model. Even if I don't consider the proposed method's novelty one of its strengths, in my opinion it is of interest to the community to understand the performance (and limitations) of this kind of system, and I expect the results in this work to inform future research. Thus, **I consider significance to be one of the strengths of the paper.**\n- Quality: the design choices in this framework are sensible (even if at times the motivations are not clearly articulated). Furthermore, the method appears to be relatively well-documented in the appendix, so even if the results are in some sense negative (the synthesized world models appear to be far from perfect), the experiment results are informative. Thus, **with some changes (see suggestions), I would consider the quality of the work to be adequate for this venue.**", "weaknesses": "The paper is generally well-written, but at times the manuscript could be more precise. In particular:\n- Some aspects of the method are not described with enough detail (see suggestions 2, 3 and 5).\n- Some aspects of the experimental setup should be emphasized (see suggestions 1 and 6).\n- The results are a bit difficult to interpret due to the use of MRR (see suggestion 4).\n- Some of the design choices are not motivated (see suggestion 7).\n\nThe experimental setup has significant limitations:\n- The experiments only include one baseline (see suggestion 8).\n- Experiments are performed in a single domain. I do not consider this to be a critical flaw, but it does limit the significance of the experiments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)