One Life to Learn: Inferring Symbolic World Models for Stochastic Environments from Unguided Exploration

## Abstract

Abstract Symbolic world modeling is the task of inferring and representing the transitional dynamics of an environment as an executable program.
Previous research on symbolic world modeling has focused on largely deterministic environments with abundant interaction data, simple mechanics, and human-provided guidance.
We address the more realistic and challenging problem of learning a symbolic world model in a complex, stochastic environment with severe constraints: a limited interaction budget where the agent has only “one life”
to explore a hostile environment and no external guidance in the form of human-provided, environment-specific rewards or goals.
We introduce OneLife , a framework that models world dynamics through conditionally-activated programmatic laws within a probabilistic programming framework.
Each law operates through a precondition-effect structure, allowing it to remain silent on irrelevant aspects of the world state and predict only the attributes it directly governs.
This creates a dynamic computation graph that routes both inference and optimization only through relevant laws for each transition, avoiding the scaling challenges that arise when all laws must contribute to predictions about a complex, hierarchical state space, and enabling accurate learning of stochastic dynamics even when most rules are inactive at any given moment.
To evaluate our approach under these demanding constraints, we introduce a new evaluation protocol that measures
(a) state ranking, the ability to distinguish plausible future states from implausible ones,
and
(b) state fidelity, the ability to generate future states that closely resemble reality.
We develop and evaluate our framework on Crafter-OO, our reimplementation of the popular Crafter environment that exposes a structured, object-oriented symbolic state and a pure transition function that operates on that state alone. OneLife can successfully learn key environment dynamics from minimal, unguided interaction, outperforming a strong baseline on 16 out of 23 scenarios tested.
We also demonstrate the world model’s utility for planning, where rollouts simulated within the world model successfully identify superior strategies in goal-oriented tasks.
Our work establishes a foundation for autonomously constructing programmatic world models of unknown, complex environments.

## 1 Introduction

World modeling is a critical task in artificial intelligence, providing an agent with a functional understanding of its environment’s underlying dynamics.
By learning a world model, an agent can predict the outcomes of its actions without having to actually interact with the real world.
One line of research in world modeling aims to learn symbolic world models via program synthesis (i.e., representing worlds models with code) with a view towards building representations that are interpretable, editable, and verifiable by humans.

While such approaches have been successful in environments with a limited number of discoverable mechanics and low stochasticity (Piriyakulkij et al., 2025; Tang et al., 2024; Dainese et al., 2024) these assumptions are often violated in more complex environments. Examples of such environments are popular open-world sandbox games (e.g. MineCraft, RuneScape) containing numerous, diverse mechanics spanning crafting, combat, and physics.
These more realistic environments have irreducible stochasticity (e.g., outcomes of actions are subject to random chance, non-player characters taking unpredictable actions), a lack of extrinsic rewards (e.g., players set their own goals and there is no well-defined criteria for “winning”), and a high cost of exploration (e.g., entering dangerous areas without preparation can result in death), making it crucial to learn from minimal interaction.
This leads to our central research question:

Figure: Figure 1: OneLife synthesizes world laws from a single unguided (no environment-specific rewards / goals) episode in a hostile, stochastic environment. OneLife models the world as mixture of laws written in code with a precondition-effect structure, each governing an aspect of the world, and infers parameters for the mixture that best explain the observed dynamics of the world. The resulting world model (WM) provides a probability distribution over attributes of an object-oriented world state, such as the position of a particular zombie. OneLife outperforms a strong baseline in modeling $16/23$ core game mechanics tested, measured by MRR (Mean Reciprocal Rank) of the true next state ([Sec. 4](https://arxiv.org/html/2510.12088v1#S4)) under the WM’s likelihood. See [Appendix A](https://arxiv.org/html/2510.12088v1#A1) for a synthesized zombie law.
Refer to caption: x1.png

We introduce a framework for symbolic world modeling, OneLife,
a name that reflects our focus on learning a symbolic world model from a single episode with unguided exploration.
As illustrated in [Fig. 1](https://arxiv.org/html/2510.12088v1#S1.F1) (top-right), OneLife learns from just a single, unguided run in the environment, a contrast to previous work (Piriyakulkij et al., 2025; Tang et al., 2024; Dainese et al., 2024) that assumes access to a large number of interactions as well as environment specific guidance provided by humans (e.g., goals / rewards designed for the environment).
OneLife recovers a program that describes the environment’s underlying transition dynamics
$p(s_{t+1}|s_{t},a_{t})$
which models the probability distribution $p$ over next states $s_{t+1}$ given a current state $s_{t}$ and action $a_{t}$.
The agent performs this inference using only observations, without access to rewards or other domain-specific guidance.
OneLife has two key components:
a law synthesizer ([Sec. 3.3](https://arxiv.org/html/2510.12088v1#S3.SS3)) that proposes new laws and
an inference algorithm ([Sec. 3.4](https://arxiv.org/html/2510.12088v1#S3.SS4)) that re-weights laws based on their predictive ability over observations.
Crucially, the inference algorithm is gradient-based and only updates the laws that alter the observed variables between current state $s_{t}$ and predicted next state $s_{t+1}$, allowing for efficient and targeted learning.
These components work together in a probabilistic programming approach ([Sec. 3.2](https://arxiv.org/html/2510.12088v1#S3.SS2)) that proposes and re-weights rules based on whether the preconditions for the laws to be applicable are met and the effect of the predictions w.r.t. the observed environment transitions.
This approach enables our model to infer distributions over complex, stochastic events, as shown in the [Fig. 1](https://arxiv.org/html/2510.12088v1#S1.F1) (bottom-right), where a learned world model outputs a distribution over a zombie’s next move.
Crucially, OneLife not only produces a distribution over states but *learns from* stochastic observations; the true movement of the zombie in [Fig. 1](https://arxiv.org/html/2510.12088v1#S1.F1) also follows a distribution, which OneLife seeks to approximate.

To evaluate our approach, we first created a suitable testbed – Crafter-OO – by re-engineering the complex Crafter (Hafner, 2022) environment to be a pure function $T(s,a)\rightarrow s^{\prime}$ of a structured, text-based hierarchical object-oriented world state. In other words, all the information needed to compute the next state is represented in a single structured, object-oriented representation, and there is a ground-truth program for the transition function that computes the next environment state purely from the state representation, without any “hidden variables”.
This text-based, object-oriented representation is natively readable by LLMs and thus allows them to try reconstructing the transition function by writing code that programatically modifies the structured state.

We introduce a new evaluation protocol that uses two axes ([Sec. 4](https://arxiv.org/html/2510.12088v1#S4)):
state ranking, the ability to distinguish valid outcomes from invalid ones according to the world’s laws, and
state fidelity, the ability to produce plausible future states for planning.
Our experiments show that OneLife better captures the environment’s dynamics compared to several baselines, including PoE-World (Piriyakulkij et al., 2025), showing improved ability to simulate future states given a state and candidate action, and to distinguish between likely and unlikely outcomes of an action.
We further show that the learned model supports planning in imagination; by simulating rollouts of different policies entirely within the model, we can evaluate and distinguish between effective and ineffective strategies for goal-oriented tasks.

In summary, our contributions include:

- •
OneLife, a probabilistic symbolic world model that can learn from stochastic and hostile environments with minimal interactions and without access to human-defined rewards. OneLife outperforms prior work, learning a world model that better predicts true environment dynamics.
- •
Crafter-OO, a reimplementation of Crafter (Hafner, 2022) that exposes a structured, object-oriented symbolic state and a pure transition function that operates on that state alone. This enables us to test OneLife in a complex, stochastic environment and lays the groundwork for future work in symbolic world modeling and programmatic reinforcement learning.
- •
An evaluation suite for world modeling within Crafter / Crafter-OO  with 30+ executable scenarios that test knowledge of all core mechanics in Crafter and a pool of mutators that can programatically generate illegal distractor states to probe world model understanding alongside, new state fidelity and state ranking metrics for evaluating world models in complex, stochastic, environments.

## 2 Related Work

Symbolic World Models.  Symbolic world models represent an environment’s transition dynamics as executable code, producing interpretable, editable, and generalizable models from limited data.
Prior work has used LLMs to synthesize a single, monolithic program that functions as a world model (Tang et al., 2024; Dainese et al., 2024).
Piriyakulkij et al. (2025) introduced a compositional approach by representing the world model as a product of programmatic experts, enabling modeling of more complex dynamics. Other methods have synthesized programs for planning (Ahmed et al., 2025) or combined functional and automata synthesis to capture latent state dynamics (Das et al., 2023). LLMs have also been used to construct formal planning representations like PDDL from environment interactions or text for symbolic planners (Guan et al., 2023; Deng et al., 2024). Our work differs from these methods in three aspects. First, we operate in a complex, open-world environment based on Crafter (Hafner, 2022) with stochasticity and many interacting mechanics, whereas prior work has operated in simpler, often deterministic domains (e.g., grid-worlds or Atari games).
Second, we do not assume abundant interaction data: our agent learns from a limited budget obtained in a single episode – or life.
Third, OneLife learns without external rewards or human-specified goals, framing the task as unguided reverse engineering of the environment’s laws.

Programmatic Representations for Decision-Making.  Program synthesis has been used to represent other components of intelligent agents. Programmatic policies have been shown to offer greater interpretability and generalization compared to neural networks (Trivedi et al., 2021; Liang et al., 2022). LLMs have been used to generate programmatic reward functions from natural language instructions, enabling agents to pursue complex, user-specified objectives (Ma et al., 2024; Yu et al., 2023; Klissarov et al., 2025). Programs have been used to build libraries of composable, temporally extended skills, allowing agents to solve long-horizon tasks by combining previously learned behaviors (Wang et al., 2025; Stengel-Eskin et al., 2024). These methods focus on representing components of the agent’s internal decision-making process: how it should act (policies), what it should value (rewards), or what it is capable of doing (skills). In contrast, our work learns a model of how the external world behaves; this task-agnostic model of environment dynamics is complementary to policies, rewards, and skills, and supports planning and decision-making for any downstream goals.

World Modeling for Open-Ended Exploration and Discovery.  Agents that explore and learn in complex, open-world environments without extrinsic rewards typically learn non-symbolic, latent world models and use them to drive exploration through intrinsic motivation (Hafner et al., 2023; Micheli et al., 2023; Dedieu et al., 2025; Schwarzer et al., 2021). These agents plan using their world models to find novelty or surprise in their environments, discovering useful skills without task-specific supervision (Sekar et al., 2020). This connects to automated scientific discovery, which requires autonomously forming hypotheses and performing experiments to understand unknown systems (Jansen et al., 2024; Chen et al., 2025; Geng et al., 2025). New evaluation frameworks have been proposed to assess an agent’s ability to rapidly induce world models in novel contexts (Ying et al., 2025; Vafa et al., 2024). Unlike methods that learn implicit, latent world models, our work learns an explicit, symbolic representation of the world’s laws. We frame learning as reverse engineering a complex system’s rules from unguided, limited interaction.

## 3 Overview of OneLife

Our framework, OneLife is designed to learn symbolic world models from a single, unguided episode of exploration.
It is built on two key abstractions, a programmatic representation of world dynamics as a mixture of modular laws with learnable weights and an observable extractor that decouples the environment’s state from the learning process.
The framework consists:
a a world model as a program ([Sec. 3.2](https://arxiv.org/html/2510.12088v1#S3.SS2)),
a law synthesizer that proposes new laws using offline data from an unguided exploration policy ([Sec. 3.3](https://arxiv.org/html/2510.12088v1#S3.SS3)),
an inference algorithm that re-weights laws based on observations ([Sec. 3.4](https://arxiv.org/html/2510.12088v1#S3.SS4)),
and a forward simulation process that uses the learned model for predicting future states ([Sec. 3.5](https://arxiv.org/html/2510.12088v1#S3.SS5)).

We model the environment as having a pure, but potentially stochastic, transition function $T:\mathcal{S}\times\mathcal{A}\to\Delta(\mathcal{S})$, where $\Delta(\mathcal{S})$ is the space of probability distributions over the state space $\mathcal{S}$.
This functional view aligns with modern reinforcement learning environment frameworks (Freeman et al., 2021; Matthews et al., 2024) and physical models, where the future state of a system is a pure function of an explicit state and any interventions.

### 3.1 Crafter-OO: A Testbed for Symbolic World Modeling

A common design assumption in previous work on symbolic world modeling (Tang et al., 2024; Piriyakulkij et al., 2025; Dainese et al., 2024) is that we have access to an object-oriented world state to use as input to the symbolic world model under construction.
In practice, this state is only easily accessible for simple environments such as Minigrid (Chevalier-Boisvert et al., 2023) or BabyAI (Chevalier-Boisvert et al., 2018).
Programmatic access to the state of more complex environments such as Atari games as used by Piriyakulkij et al. (2025) is only possible due to standalone development efforts such as OCAtari (Delfosse et al., 2024) which makes the internal object-oriented state of these environments accessible to researchers.
The lack of an environment with an exposed, object-oriented state that is more complex than gridworlds or with mechanics more diverse than Atari games has thus far prevented evaluation and development of symbolic world modeling approaches for more complex environments.
To close this gap, we implement Crafter-OO ([Appendix B](https://arxiv.org/html/2510.12088v1#A2)), which emulates the Crafter (Hafner, 2022) environment by operating purely on an explicit, object-oriented game ([Section B.2](https://arxiv.org/html/2510.12088v1#A2.SS2)).
Additionally, we contribute utilities for programmatically modifying the game state to create evaluation scenarios ([Appendix D](https://arxiv.org/html/2510.12088v1#A4), [Sec. 4.1](https://arxiv.org/html/2510.12088v1#S4.SS1)).

Our target environment Crafter-OO features significant stochasticity, diverse forms of mechanics, and active non-player characters.
This includes elements such as hostile and friendly agents with diverse, inherently random behaviors. Our framework is designed to infer the rules governing these interactions from observation alone, without access to rewards or human-specified goals. For instance, in [Fig. 2](https://arxiv.org/html/2510.12088v1#S3.F2), the scenario contains a “zombie” character chasing the player via stochastic movements. While one cannot perfectly predict the future position of a zombie due to inherent randomness built into the environment, our world model is able to capture this “chasing the player” behavior without any explicit supervision by predicting a discrete distribution for the zombie.position attributes.

Figure: Figure 2: Illustration of the inference process. The active laws for each observable (defined by $\mathcal{I}_{k}(s_{t},a)$) determine the structure of the computation graph, i.e., which laws and their corresponding parameters $\theta_{i}$ are related to which observables. This structure in turn informs the parameter updates. Shown here is a dataset with a single transition instance, in which the player (P) moves right; at the same time, a zombie (Z) independently moves left. this implicates two laws, PlayerMovementLaw and ZombieMovementLaw, while not implicating the InventoryUpdateLaw. As a result, the loss computation is only a function of $\theta_{1}$ and $\theta_{2}$. Note we use $Z$ here to denote the normalizing factor. Examples of synthesized laws can be seen in [Appendix A](https://arxiv.org/html/2510.12088v1#A1).
Refer to caption: x2.png

### 3.2 OneLife : World Model as a Mixture of Laws

We consider environments with complex, structured state spaces $\mathcal{S}$ where the full state $s\in\mathcal{S}$ may be hierarchical and contain a mixture of entity types and attributes.
An agent interacts with the environment by taking an action $a\in\mathcal{A}$ and observing a transition from state $s_{t}$ to $s_{t+1}$, as illustrated in [Fig. 2](https://arxiv.org/html/2510.12088v1#S3.F2).
We model an environment’s transition function as a composition of programmatic laws. A law, $L_{i}$, is a program defined by a pair $(c_{i},e_{i})$, where $c_{i}(s,a)\to\{\text{true},\text{false}\}$ is a precondition and $e_{i}(s,a)\to s^{\prime}$ is an effect.
The precondition determines whether the law is applicable to a state-action pair $(s,a)$. The effect function makes a prediction by modifying attributes on a copy of the state.
For example, the PlayerMovementLaw in [Fig. 2](https://arxiv.org/html/2510.12088v1#S3.F2) applies to state-action pairs with a player and a move action, and has an effect on the player position’s $(x)$ observable.
This precondition-effect structure is inspired by classical planning and provides a natural way to specify the scope of each law, ensuring modularity (McDermott et al., 1998).
During any given transition, multiple or no laws may be applicable.

To create a tractable interface to compare states predicted by a world model and the true state of the environment, we introduce an observable extractor, $\mathcal{E}:\mathcal{S}\to\mathcal{O}$.
This function maps a complex state $s$ into a vector of primitive-valued observables $o\in\mathcal{O}$.
In the scenario sketched in [Fig. 2](https://arxiv.org/html/2510.12088v1#S3.F2), the next state $s_{t+1}$ can be complex, with additional entities and objects (e.g., trees, inventory items, etc.).
Nevertheless, one can tractably compare states via observations, i.e., *changes* between $s_{t}$ and $s_{t+1}$ such as player.position, player.inventory, zombie.position, etc.
Note that any given law $L_{i}$ only makes predictions about a subset of all possible observables.
For instance, in [Fig. 2](https://arxiv.org/html/2510.12088v1#S3.F2), the PlayerMovementLaw only makes predictions about player.position observables and *does not predict* the zombie.position observables.

Our world model can be viewed as a probabilistic program (van de Meent et al., 2021) that generates the next state’s observables $o^{\prime}$ conditioned on the current state $s$ and action $a$.
The set of laws $\{L_{i}\}$ defines the components of this program. The effect $e_{i}$ of each law specifies a set of conditional probability distributions $\phi_{i,o}(o=v|s,a)$ for an observable $o$, where $v$ denotes a specific outcome in the discrete support of the observable $\mathrm{supp}(o)$. For a given state-action pair $(s,a)$, the set of active laws is $\mathcal{I}(s,a)=\{i\mid c_{i}(s,a)\text{ is true}\}$ (e.g., PlayerMovementLaw and ZombieMovementLaw in [Fig. 2](https://arxiv.org/html/2510.12088v1#S3.F2)).
The model assumes that all observables are *conditionally independent* given the current state and action. The predictive distribution for a single observable $o$ is formed by combining the predictions from all active laws that have an opinion on it. Let $\mathcal{I}_{o}(s,a)=\{i\in\mathcal{I}(s,a)\mid o\in\mathcal{O}\}$ be the set of active laws relevant to observable $o$. The probability of observing an outcome $v$ for this observable is given by a weighted-product of conditional probability distribution from each law, parameterized by $\bm{\theta}$:

$$ $p(o=v|s,a;\bm{\theta})\propto\prod_{i\in\mathcal{I}_{o}(s,a)}\phi_{i}(o=v|s,a)^{\theta_{i}}$ (1) $$

The complete predictive distribution over the next state $s^{\prime}$ is the product of the individual observable distributions:

$$ $p(s^{\prime}|s,a;\bm{\theta})=\prod_{o\in\mathcal{O}}p(o|s,a;\bm{\theta})$ (2) $$

### 3.3 OneLife : Unguided Environment Exploration and Law Synthesis

The set of candidate laws ${L_{i}}$ is generated from unguided agent-environment interactions through a two-stage process. First, an autonomous exploration policy gathers a corpus of interaction data. Second, a synthesizer proposes candidate laws that explain the state transitions observed in this data.

#### Exploration Policy.

Previous work in symbolic world modeling often assumes access to curated offline datasets or utilizes online interaction guided by human-provided goals or environment rewards. In our unsupervised setting, such guidance is unavailable. Furthermore, in a hostile environment such as Crafter-OO, a simple random policy fails to survive long enough to experience the diverse mechanics necessary for comprehensive world modeling. Therefore, we employ an exploration policy driven by a large language model. The policy is not provided with specific knowledge of the environment; instead, it is given the high-level objective to discover as many underlying mechanics as possible, treating exploration as a reverse-engineering task. We use the agent scaffolding from Balrog (Paglieri et al., 2025) to implement the agent. The agent’s architecture maintains a rolling window of its recent state-action history to provide context for decisions. The prompt (see [Appendix F](https://arxiv.org/html/2510.12088v1#A6)) also instructs the agent to maintain a transient summary of its current understanding of the world’s rules, refining its hypotheses as it interacts with the environment.

#### Law Synthesizer.

The synthesizer’s task is to propose laws explaining the experienced transitions. Prior approaches (Piriyakulkij et al., 2025) have often relied on a large suite of hand-designed synthesizers, each tailored to specific types of interactions. This method embeds significant domain knowledge, which runs contrary to our goal of unsupervised discovery. We instead adopt a more general approach where a synthesizer is prompted to propose a large set of simple, atomic laws for each observed transition. An atomic law is one that describes a change to a minimal number of state attributes. For instance, a complex combat event involving the player and a zombie – resulting in changes to both entities’ positions and health – is not modeled by a single monolithic law. Our synthesizer decomposes the event into multiple atomic laws: one for the player’s health decrease, another for the zombie’s movement, and so on. This decomposition into fine-grained, modular laws allows the subsequent weight-fitting stage [Sec. 3.4](https://arxiv.org/html/2510.12088v1#S3.SS4) to perform more precise credit assignment, isolating and down-weighting incorrect hypotheses without discarding entire complex rules that may be partially correct.
We provide examples of laws in [Appendix A](https://arxiv.org/html/2510.12088v1#A1).

### 3.4 OneLife : Inference on Law Parameters

We learn the weight vector $\bm{\theta}$ by maximizing the log-likelihood of a dataset of observed transitions $\mathcal{D}=\{(s_{t},a_{t},s_{t+1})\}_{t=1}^{N}$. For clarity, we first define the loss for a single transition $(s,a,s^{\prime})$; the total loss is the sum over all transitions in the dataset.

Based on the conditional independence of observables, the negative log-likelihood for a single transition decomposes into a sum over each observable $o\in\mathcal{O}$:

$$ $\mathcal{L}(\bm{\theta};s,a,s^{\prime})=-\sum_{o\in\mathcal{O}}\log p(v_{o}^{*}|s,a;\bm{\theta})$ (3) $$

where $v_{o}^{*}=\mathcal{E}(s^{\prime})_{o}$ is the ground truth value of observable $o$ extracted from the next state $s^{\prime}$.
The log-probability term is derived from the combined predictions of the active laws. Let $\mathcal{I}_{o}(s,a)$ be the set of active laws that make a prediction for observable $o$. We first define the combined, unnormalized log-score for any potential value $v$ as the weighted sum of log-scores from these laws. The weights $\theta_{i}$ are the only learnable parameters:

$$ $\ell_{o}(v|s,a;\bm{\theta})=\sum_{i\in\mathcal{I}_{o}(s,a)}\theta_{i}\cdot\phi_{i,o}(v|s,a)$ (4) $$

Normalized log-probability of observing the specific outcome $v_{o}^{*}$ is then given by the log-softmax function. Let $\textrm{supp}(o)$ be the discrete support (set of all possible values) for observable $o$:

$$ $\log p(v_{o}^{*}|s,a;\bm{\theta})=\ell_{o}(v_{o}^{*}|s,a;\bm{\theta})-\log\sum_{v\in\textrm{supp}(o)}\exp\left(\ell_{o}(v|s,a;\bm{\theta})\right)$ (5) $$

The optimization process leverages the dynamic computation graph induced by our law structure. For each transition and each observable, the loss gradient is calculated with respect to the weights $\theta_{i}$ only for the active laws $i\in\mathcal{I}_{o}(s_{t},a_{t})$. This effectively routes credit for an outcome exclusively to the laws that made a prediction about it. This sparse, targeted update mechanism provides more precise credit assignment than methods that update a global set of weights based on aggregate outcomes. We use L-BFGS for optimization (Nocedal & Wright, 2006).

### 3.5 OneLife : Forward Simulation and Likelihood

Forward simulation is the process of using the learned world model generatively to predict a future state $\hat{s}_{t+1}$ given a current state $s_{t}$ and an action $a_{t}$. By generating rollouts of future trajectories, an agent can evaluate action sequences against a specific goal or reward function without costly or irreversible real-world interaction.

The simulation of a single timestep from $(s_{t},a_{t})$ involves a multi-step sampling and reconstruction process. First, for each observable $o\in\mathcal{O}$, the model forms a predictive probability distribution $p(o|s_{t},a_{t};\bm{\theta})$. This distribution is constructed by identifying the set of active laws $\mathcal{I}_{o}(s_{t},a_{t})$ relevant to that observable and combining their predictions according to their learned weights $\theta_{i}$, as specified in Equation [1](https://arxiv.org/html/2510.12088v1#S3.E1).
This distribution can be used to evaluate the likelihood of an observable conditioned on $(s,a)$ pair.
Second, a concrete outcome $\hat{v}_{o}$ can be sampled from this distribution for each observable: $\hat{v}_{o}\sim p(o|s_{t},a_{t};\bm{\theta})$. This the collection of sampled outcomes $\{\hat{v}_{o}\}_{o\in\mathcal{O}}$ is used to construct the full symbolic next state $\hat{s}_{t+1}$. A reconstruction function, which mirrors the observable extraction process, assembles these values back into the environment’s structured state representation.

## 4 Evaluation Protocols and Metrics

Figure: Figure 3: Two evaluation metric categories described in [Sec. 4](https://arxiv.org/html/2510.12088v1#S4). A world state of an environment usually has more than two keys (i.e. Crafter-OO’s state ([Section B.2](https://arxiv.org/html/2510.12088v1#A2.SS2)) when populated has 100+ key-value pairs,) and often has nested values, but here we show a simplest case to explain the calculation of (normalized) edit distance. We create distractors for state ranking using mutators ([Appendix C](https://arxiv.org/html/2510.12088v1#A3)), which programatically modify the next state $s^{\prime}$ in a transition $(s,a,s^{\prime})$ to be illegal under the true transition function. For example, one of our mutators allows a crafting action (e.g. making a stone pickaxe) to succeed even when the prequisites for the crafting are not met.
Refer to caption: x3.png

The evaluation of world models for a stochastic environment is non-trivial. An useful world model fulfills two criteria:
(a) state ranking, the ability to distinguish plausible future states from implausible ones,
and
(b) state fidelity, the ability to generate future states that closely resemble reality.
Both are illustrated in [Fig. 3](https://arxiv.org/html/2510.12088v1#S4.F3).

#### State Ranking ( Fig. 3 (a)).

These metrics assess the model’s ability to rank the true next state higher than the distractors.
To create the distractor states, we use mutators, which are programmatic functions that apply semantically meaningful, rule-breaking changes to the true next state. For example, a mutator could change a character’s position to a location they cannot physically reach.
We include details on mutators in [Appendix C](https://arxiv.org/html/2510.12088v1#A3).

- •
Rank @ 1 (R@1): A binary metric that measures whether the model correctly assigns the highest probability (rank 1) to the true next state among all candidates.
- •
Mean Reciprocal Rank (MRR): This metric averages the reciprocal rank of the correct answer across all test instances. A higher MRR indicates that the model consistently ranks the correct state higher. The formula is:
$\text{MRR}=\frac{1}{N}\sum_{i=1}^{N}\frac{1}{r_{i}}$,
where $r_{i}$ is the rank of the ground truth state for the $i$-th transition, with rank 1 being the highest probability.

State Fidelity ([Fig. 3](https://arxiv.org/html/2510.12088v1#S4.F3) (b)).
These measure the error between predicted and ground truth states.

- •
Raw Edit Distance: The total number of atomic JSON Patch operations required to transform the predicted state, $s^{\prime}_{t+1}$, into the ground truth state, $s_{t+1}$.
- •
Normalized Edit Distance: The raw edit distance divided by the total number of elements in the state representation.

### 4.1 Evaluation Framework Implementation on Crafter-OO

Evaluating a world model on random rollouts may not provide sufficient coverage of rare or important events in an environment. To ensure our evaluation is comprehensive, we create evaluation trajectories from a suite of scenarios. Each scenario runs short, scripted policy from an initial state designed to reliably exercise a specific game mechanic or achieve a particular goal, ensuring that our evaluation thoroughly covers the environment’s dynamics.
Our scenarios cover every achievement in the achievement tree of Crafter-OO/Crafter, and can be seen in [Fig. 4](https://arxiv.org/html/2510.12088v1#S5.F4).
We generate a comprehensive evaluation dataset by implementing scenarios that cover every achievement in the game’s achievement tree. This ranges from basic actions like collecting wood to complex, multi-step tasks like crafting an iron sword, ensuring all of the game’s core mechanics are tested.
More details on scenarios are provided in [Appendix D](https://arxiv.org/html/2510.12088v1#A4).
We generate distractors for each transition in the evaluation dataset using a bank of 8 mutators which each produce a subtle, but illegal transformation of the game state in response to an action.
Some examples are causing an incorrect item to be produced when taking a crafting action, or allowing an item to be produced without the correct requirements, or illegal entity behavior such as teleporting.
More details on mutators are provided in [Appendix C](https://arxiv.org/html/2510.12088v1#A3) and general implementation details are in [Appendix E](https://arxiv.org/html/2510.12088v1#A5).

**Table 1: Performance comparison of world modeling methods on the Crafter-OO environment, averaged over ten trials. We evaluate models on two criteria: state fidelity and state ranking All methods use the OneLife exploration policy and law synthesizer but differ in their parameter inference method. OneLife shows significant improvements over the PoE-World inference algorithm and OneLife variant without parameter inference. The random baseline is shaded in gray.**
| Law Synthesis | Law Param. Inference | State Ranking | State Fidelity |  |  |
| --- | --- | --- | --- | --- | --- |
| ([Sec. 3.3](https://arxiv.org/html/2510.12088v1#S3.SS3)) | ([Sec. 3.4](https://arxiv.org/html/2510.12088v1#S3.SS4)) | Rank @ 1 $\uparrow$ | MRR $\uparrow$ | Raw Edit Dist. $\downarrow$ | Norm. Edit Dist. $\downarrow$ |
| Random World Model | $8.5\%$ | 0.322 | 121.538 | 0.809 |  |
| OneLife | PoE-World | $10.8\%$ | 0.351 | 10.634 | 0.071 |
| OneLife | None | $13.0\%$ | 0.429 | 8.540 | 0.057 |
| OneLife | OneLife | 18.7% | 0.479 | 8.764 | 0.058 |
| $\Delta$ over PoE-World | (+7.9%) | (+0.128) | (-1.870) | (-0.013) |  |

## 5 Experimental Setup and Results

We conduct a series of experiments to evaluate OneLife. First, we quantitatively assess the model’s predictive accuracy using our state ranking and fidelity metrics across a comprehensive suite of scenarios. Second, we test the model’s ability to support planning in imagination. We use the model to perform simulated rollouts of different policies, evaluating whether it can predict the outcomes of these plans well enough to distinguish effective strategies from ineffective ones.

### 5.1 Baseline Models

To contextualize the results of our proposed model, we compare against several two baselines:

- •
Random World Model: A model that assigns a uniform probability to all candidate states in the discriminative task. Its performance is equivalent to random guessing and serves as a sanity check for discriminative accuracy.
- •
PoE-World (Piriyakulkij et al., 2025): A state-of-the-art symbolic world model that scaled symbolic world modeling to domains like Atari. Both PoE-World and OneLife represent the transition function as a weighted product of programs, though the structure of the programs and inference algorithms differ. Because PoE-World’s law synthesis component is Atari-specific and relies on online interaction using human-provided goals, we reimplement this baseline with our exploration policy and law synthesizer, noting that this makes it a stronger baseline (without these changes, PoE-World’s Atari-specific implementation would be fundamentally incompatible with Crafter’s state).

### 5.2 Results

Figure: Figure 4: Per-scenario state ranking performance of OneLife (Ours) versus PoE-World, measured by Mean Reciprocal Rank (MRR $\uparrow$). Scenarios are grouped by the core game mechanic they test. Horizontal lines show the average MRR across all scenarios in a group for OneLife and PoE-World. OneLife demonstrates a more accurate understanding of the environment’s laws, achieving a higher average MRR and outperforming the baseline on the majority of individual scenarios.
Refer to caption: x4.png

**Table 2: Planning via forward simulation. Our learned world model is used to compare alternative plans in three scenarios. This is done by executing the plans in the world model, and measuring the reward obtained by each plan. In each case, OneLife produces the same ranking over plans as the ground-truth environment, demonstrating its ability to capture causally relevant dynamics for goal-directed decision-making and accurately simulate long action sequences of $>30$ steps. Each plan was executed 10 times.**
| Scenario | Plan Description | Reward Function | Avg. Steps | True Env. | OneLife’s WM |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Reward | Preferred | Reward | Preferred |  |  |  |  |
| Zombie Fighter | Harvest Wood $\rightarrow$ Craft Table $\rightarrow$ Craft Sword $\rightarrow$ Fight | Damage Per Second | 33 | 2.0 | ✓ | 2.03 | ✓ |
| Fight Immediately | 17 | 1.0 |  | 1.67 |  |  |  |
| Stone Miner | Harvest Wood $\rightarrow$ Craft Table $\rightarrow$ Craft Pickaxe $\rightarrow$ Mine | Stone Collected | 31 | 3.0 | ✓ | 3.0 | ✓ |
| Mine Immediately | 13 | 0.0 |  | 0.0 |  |  |  |
| Sword Maker | Reuse Crafting Table for all Swords | Swords Crafted | 5 | 4.0 | ✓ | 4.0 | ✓ |
| Place New Table per Sword | 10 | 2.0 |  | 2.0 |  |  |  |

State Fidelity and Ranking.  OneLife learns a world model with significantly higher predictive judgment than baseline methods while maintaining competitive generative fidelity. Table [1](https://arxiv.org/html/2510.12088v1#S4.T1) compares our full method against baselines and key ablations across all evaluation metrics.
OneLife’s primary advantage appears in the predictive judgment metrics. We achieve a discriminative accuracy of 18.7% and an MRR of 0.479, outperforming the PoE-World optimization baseline by 7.9 percentage points and 0.128, respectively. While precisely generating a complex future state remains challenging, our model has learned an accurate understanding of the environment’s underlying laws. This enables it to assign high probability to valid transitions and low probability to invalid ones. The comparison to the “random world model” shows that (i) a high edit distance can quickly be amassed if the world models updates observables that are unchanged in the ground truth state, thus, reinforcing why such simulation is challenging; (ii) optimizing for generative metrics like state fidelity alone does not yield a better world model to guide an agent, e.g., while the PoE-world model (row 2 in [Tab. 1](https://arxiv.org/html/2510.12088v1#S4.T1)) dramatically improves the state fidelity by reducing the edit distance *a factor of 10*, it only marginally improves the ability to rank multiple states by $\approx$2% over random (Rank@1) – reiterating the need for state ranking metrics.

Fine-grained Evaluation.  Figure [4](https://arxiv.org/html/2510.12088v1#S5.F4) breaks down Mean Reciprocal Rank performance across individual scenarios spanning mechanics from resource collection to combat. OneLife consistently outperforms the PoE-World baseline on the majority ($16/23$) of scenarios. These improvements stem from a robust understanding of the environment’s diverse rules rather than strong performance on only a few simple mechanics.

Figure: Figure 5: We show an example of plan execution within OneLife’s world model for the “Stone Miner” scenario. The task is to mine stone, and can only be successfully completed if a wooden pickaxe is obtained before attempting to mine stone. We simulate two plans within the world model. The effective plan carries out a multi-step sequence of gathering wood, crafting a wooden pickaxe, and then attempting to mine. The ineffective plan attempts to mine the stone directly. The world learned by OneLife correctly simulates causal game mechanics that cause the effective plan to succeed and the ineffective plan to fail. The frames are generated by rendering the structured states constructed by OneLife’s learned transition function.
Refer to caption: x5.png

### 5.3 Planning with the Learned World Model

To assess the practical utility of the learned world model, we evaluate its effectiveness in a planning context. Our protocol tests the model’s ability to distinguish between effective and ineffective plans through forward simulation. For a set of scenarios, we define a reward function and two distinct, programmatic policies (plans) to achieve a goal within the scenario. Each plan is represented as a hierarchical policy (in code) that composes subroutines for navigation, interaction, and crafting.

We give an example in [Sec. F.2](https://arxiv.org/html/2510.12088v1#A6.SS2.SSS0.Px3) for the “Zombie Fighter” scenario.
Each reward function is likewise written in code and calculates rewards from the rollout of a plan.
We execute rollouts of both plans within our learned world model and, separately, within the ground-truth environment. The measure of success is whether the world model’s simulation yields the same preference ranking over the two plans as the true environment, based on the final reward. This assesses if the model has captured the causal dynamics necessary for goal-directed reasoning.

Setup. We design three scenarios that test distinct aspects of the environment’s mechanics: combat, tool-use and resource consumption, as shown in [Table 2](https://arxiv.org/html/2510.12088v1#S5.T2). In the Zombie Fighter scenario, an agent with low health must defeat two zombies. The superior plan involves a multi-step process: pathfinding to locate and harvest trees, crafting a table and then a sword, and only then engaging in combat. The alternative is to fight immediately.
The Stone Miner scenario tests the model’s understanding of resource collection. The effective plan is to first harvest wood, craft a pickaxe, pathfind to a stone, and then mine. Attempting to mine stone directly is ineffective.
Finally, the Sword Maker scenario evaluates knowledge of resource consumption. The goal is to craft multiple swords. The efficient plan places a single crafting table and reuses it, whereas the inefficient plan wastes wood by placing a new table for each sword.
On average, a plan requires $\approx\!18$ steps to execute, with the longest plans taking $>\!30$ steps.
Thus, simulating the results of these plans tests the ability of the world model to accurately model the consequences of long sequences of actions upon the world.
We show an example of plan execution in imagination for the “Stone Miner” scenario in [Fig. 5](https://arxiv.org/html/2510.12088v1#S5.F5).

Results. [Table 2](https://arxiv.org/html/2510.12088v1#S5.T2) shows that across all three scenarios, our learned world model correctly predicts the more effective plan. The ranking of plans generated by simulating rollouts in OneLife matches the ranking from the ground-truth environment. For instance, in the Zombie Fighter scenario, the model correctly simulates that the multi-step plan of crafting a sword leads to higher Damage Per Second, identifying it as the superior strategy. This demonstrates that OneLife captures a sufficiently accurate causal model of the world to support basic, goal-oriented planning.

## 6 Conclusion

We address the problem of learning a symbolic world model from limited, unguided interaction in a complex, stochastic environment. We introduced OneLife, a framework that represents world dynamics as a probabilistic mixture of modular, programmatic laws. Its core learning mechanism routes credit for observed state changes exclusively to the laws responsible for predicting them, enabling effective learning even when many rules are inactive during a given transition. Evaluated on Crafter-OO, our variant of the complex Crafter environment with object-centric state, OneLife learns a world model with superior predictive judgment compared to a strong baseline, more accurately distinguishing plausible future states from implausible ones. This improvement is consistent across a wide range of game mechanics. Our work provides a foundation for building agents that can autonomously reverse engineer the rules of an unknown environment.

## Acknowledgements

This work was supported by NSF-AI Engage Institute DRL-2112635, ARO Award W911NF2110220, ONR Grant N00014-23-1-2356, DARPA ECOLE Program No. HR00112390060, Capital One Research Award, Apple PhD Fellowship, Bloomberg PhD Fellowship, and NDSEG PhD Fellowship. The views contained in this article are those of the authors and not of the funding agency.

## References

- Ahmed et al. (2025)
Zergham Ahmed, Joshua B Tenenbaum, Christopher J Bates, and Samuel J Gershman.
Synthesizing world models for bilevel planning.
*TMLR*, 2025.
- Bryan & Nottingham (2013)
P. (Ed.) Bryan and M. (Ed.) Nottingham.
JavaScript Object Notation (JSON) Patch.
RFC 6902, IETF, April 2013.
URL [https://datatracker.ietf.org/doc/html/rfc6902](https://datatracker.ietf.org/doc/html/rfc6902).
- Chen et al. (2025)
Yimeng Chen, Piotr Piekos, Mateusz Ostaszewski, Firas Laakom, and Juergen Schmidhuber.
Physgym: Benchmarking llms in interactive physics discovery with controlled priors.
*arXiv preprint*, 2025.
- Chevalier-Boisvert et al. (2018)
Maxime Chevalier-Boisvert, Dzmitry Bahdanau, Salem Lahlou, Lucas Willems, Chitwan Saharia, Thien Huu Nguyen, and Yoshua Bengio.
Babyai: A platform to study the sample efficiency of grounded language learning.
*arXiv preprint arXiv:1810.08272*, 2018.
- Chevalier-Boisvert et al. (2023)
Maxime Chevalier-Boisvert, Bolun Dai, Mark Towers, Rodrigo Perez-Vicente, Lucas Willems, Salem Lahlou, Suman Pal, Pablo Samuel Castro, and Jordan Terry.
Minigrid & miniworld: Modular & customizable reinforcement learning environments for goal-oriented tasks.
In *Advances in Neural Information Processing Systems 36, New Orleans, LA, USA*, December 2023.
- Dainese et al. (2024)
Nicola Dainese, Matteo Merler, Minttu Alakuijala, and Pekka Marttinen.
Generating code world models with large language models guided by monte carlo tree search.
In *NeurIPS*, 2024.
- Das et al. (2023)
Ria Das, Joshua B. Tenenbaum, Armando Solar-Lezama, and Zenna Tavares.
Combining functional and automata synthesis to discover causal reactive programs.
*Proc. ACM Program. Lang.*, 7(POPL), January 2023.
doi: 10.1145/3571249.
URL [https://doi.org/10.1145/3571249](https://doi.org/10.1145/3571249).
- Dedieu et al. (2025)
Antoine Dedieu, Joseph Ortiz, Xinghua Lou, Carter Wendelken, Wolfgang Lehrach, J Swaroop Guntupalli, Miguel Lazaro-Gredilla, and Kevin Patrick Murphy.
Improving transformer world models for data-efficient rl.
In *ICML*, 2025.
- Delfosse et al. (2024)
Quentin Delfosse, Jannis Blüml, Bjarne Gregori, Sebastian Sztwiertnia, and Kristian Kersting.
OCAtari: Object-centric Atari 2600 reinforcement learning environments.
*Reinforcement Learning Journal*, 1:400–449, 2024.
- Deng et al. (2024)
Boyuan Deng, Qiaochu Xu, Hao Li, Ling Li, Rui Xu, Yunlong Lu, Jie Lin, and Ziqi Zhang.
Pddlego: Iterative planning in textual environments.
*arXiv preprint*, 2024.
- Freeman et al. (2021)
C. Daniel Freeman, Erik Frey, Anton Raichuk, Sertan Girgin, Igor Mordatch, and Olivier Bachem.
Brax - a differentiable physics engine for large scale rigid body simulation, 2021.
URL [http://github.com/google/brax](http://github.com/google/brax).
- Geng et al. (2025)
Jiayi Geng, Howard Chen, Dilip Arumugam, and Thomas L Griffiths.
Are large language models reliable ai scientists? assessing reverse-engineering of black-box systems.
*arXiv preprint*, 2025.
- Guan et al. (2023)
Lin Guan, Karthik Valmeekam, Sarath Sreedharan, and Subbarao Kambhampati.
Leveraging pre-trained large language models to construct and utilize world models for model-based task planning.
In *NeurIPS*, 2023.
- Hafner (2022)
Danijar Hafner.
Benchmarking the spectrum of agent capabilities.
In *The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022*. OpenReview.net, 2022.
URL [https://openreview.net/forum?id=1W0z96MFEoH](https://openreview.net/forum?id=1W0z96MFEoH).
- Hafner et al. (2023)
Danijar Hafner, Jurgis Pašukonis, Jimmy Ba, and Timothy Lillicrap.
Mastering diverse domains through world models.
*arXiv preprint*, 2023.
- Jansen et al. (2024)
Peter Jansen, Marc-Alexandre Côté, Tushar Khot, Erin Bransom, Bhavana Dalvi Mishra, Bodhisattwa Prasad Majumder, Oyvind Tafjord, and Peter Clark.
Discoveryworld: A virtual environment for developing and evaluating automated scientific discovery agents.
In *NeurIPS*, 2024.
- Klissarov et al. (2025)
Martin Klissarov, Mikael Henaff, Roberta Raileanu, Shagun Sodhani, Pascal Vincent, Amy Zhang, Pierre-Luc Bacon, Doina Precup, Marlos C Machado, and Pierluca D’Oro.
Maestromotif: Skill design from artificial intelligence feedback.
In *ICLR*, 2025.
- Liang et al. (2022)
Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng.
Code as policies: Language model programs for embodied control.
*arXiv preprint*, 2022.
- Ma et al. (2024)
Yecheng Jason Ma, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi Fan, and Anima Anandkumar.
Eureka: Human-level reward design via coding large language models.
In *ICLR*, 2024.
- Matthews et al. (2024)
Michael Matthews, Michael Beukman, Benjamin Ellis, Mikayel Samvelyan, Matthew Jackson, Samuel Coward, and Jakob Foerster.
Craftax: A lightning-fast benchmark for open-ended reinforcement learning.
In *International Conference on Machine Learning (ICML)*, 2024.
- McDermott et al. (1998)
Drew McDermott, Malik Ghallab, Adele Howe, Craig Knoblock, Ashwin Ram, Manuela Veloso, Daniel Weld, and David Wilkins.
PDDL – the planning domain definition language.
Technical report, Yale Center for Computational Vision and Control, 1998.
- Micheli et al. (2023)
Vincent Micheli, Eloi Alonso, and François Fleuret.
Transformers are sample-efficient world models.
In *ICLR*, 2023.
Oral Presentation.
- Nocedal & Wright (2006)
Jorge Nocedal and Stephen J. Wright.
*Numerical Optimization*.
Springer, second edition, 2006.
- Paglieri et al. (2025)
Davide Paglieri, Bartłomiej Cupiał, Samuel Coward, Ulyana Piterbarg, Maciej Wolczyk, Akbir Khan, Eduardo Pignatelli, Łukasz Kuciński, Lerrel Pinto, Rob Fergus, et al.
Balrog: Benchmarking agentic llm and vlm reasoning on games.
In *The Thirteenth International Conference on Learning Representations*, 2025.
- Piriyakulkij et al. (2025)
Wasu Top Piriyakulkij, Yichao Liang, Hao Tang, Adrian Weller, Marta Kryven, and Kevin Ellis.
Poe-world: Compositional world modeling with products of programmatic experts.
*arXiv preprint*, 2025.
- Schwarzer et al. (2021)
Max Schwarzer, Nitarshan Rajkumar, Michael Noukhovitch, Ankesh Anand, Laurent Charlin, Devon Hjelm, Philip Bachman, and Aaron Courville.
Pretraining representations for data-efficient reinforcement learning.
In *NeurIPS*, 2021.
- Sekar et al. (2020)
Ramanan Sekar, Oleh Rybkin, Kostas Daniilidis, Pieter Abbeel, Danijar Hafner, and Deepak Pathak.
Planning to explore via self-supervised world models.
In *ICML*, 2020.
- Stengel-Eskin et al. (2024)
Elias Stengel-Eskin, Archiki Prasad, and Mohit Bansal.
Regal: Refactoring programs to discover generalizable abstractions.
In *International Conference on Machine Learning*, pp. 46605–46624. PMLR, 2024.
- Tang et al. (2024)
Hao Tang, Darren Key, and Kevin Ellis.
Worldcoder, a model-based llm agent: Building world models by writing code and interacting with the environment.
*arXiv preprint*, 2024.
- Trivedi et al. (2021)
Dweep Trivedi, Jesse Zhang, Shao-Hua Sun, and Joseph J Lim.
Learning to synthesize programs as interpretable and generalizable policies.
In *NeurIPS*, 2021.
- Vafa et al. (2024)
Keyon Vafa, Joshua B Tenenbaum, Jacob D Andreas, Yoshua Bengio, and Brenden M Lake.
Evaluating the world model implicit in a generative model.
In *NeurIPS*, 2024.
- van de Meent et al. (2021)
Jan-Willem van de Meent, Brooks Paige, Hongseok Yang, and Frank Wood.
An introduction to probabilistic programming, 2021.
URL [https://arxiv.org/abs/1809.10756](https://arxiv.org/abs/1809.10756).
- Wang et al. (2025)
Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar.
Voyager: An open-ended embodied agent with large language models.
*TMLR*, 2025.
- Ying et al. (2025)
Lance Ying, Katherine M Collins, Prafull Sharma, Cedric Colas, Kaiya Ivy Zhao, Adrian Weller, Zenna Tavares, Phillip Isola, Samuel J Gershman, Jacob D Andreas, Thomas L Griffiths, Francois Chollet, Kelsey R Allen, and Joshua B Tenenbaum.
Assessing adaptive world models in machines with novel games.
In *ICLR*, 2025.
- Yu et al. (2023)
Wenhao Yu, Nimrod Gileadi, Chuyuan Fu, et al.
Language to rewards for robotic skill synthesis.
In *CoRL*, 2023.

## Appendix A Law Examples

Below, we give examples of various laws synthesized by OneLife.
In [Appendix A](https://arxiv.org/html/2510.12088v1#A1) and [Appendix A](https://arxiv.org/html/2510.12088v1#A1), we show examples of how OneLife has learned the hierarchical structure of Crafter-OO/Crafter’s tech-tree. In this case, one must mine stone before a stone pickaxe can be produced.
These laws are deterministic.
In [Appendix A](https://arxiv.org/html/2510.12088v1#A1), we give an example of a law synthesized by OneLife for a stochastic mechanic, in this case, the chase behavior of zombies when they are within a certain range of a player.
The idle skeleton law in [Appendix A](https://arxiv.org/html/2510.12088v1#A1) and moving skeleton law in [Appendix A](https://arxiv.org/html/2510.12088v1#A1) make conflicting predictions; these are aggregated by the weight inference process in [Sec. 3.4](https://arxiv.org/html/2510.12088v1#S3.SS4) to produce a distributional prediction that takes into account the predictive accuracy of both laws.

## Appendix B The Crafter-OO Environment

This appendix details Crafter-OO, our reimplementation of the Crafter environment that exposes a structured, object-oriented symbolic state and operates through a pure transition function. We developed Crafter-OO as a testbed for symbolic world modeling approaches in a complex, stochastic domain.

### B.1 Motivation and Design Principles

Symbolic world modeling benefits from environments where the complete state is accessible as a structured representation. Simple grid worlds provide this but lack complexity, while more complex environments typically require additional engineering to expose their internal state. More fundamentally, existing testbeds for symbolic world modeling have focused on environments that are either deterministic or have limited stochasticity and a narrow range of mechanics. Atari games, for instance, while complex in visual processing demands, have relatively predictable dynamics and a constrained set of interactions compared to open-world environments.

We developed Crafter-OO to address this gap. The environment features significant stochasticity in entity behaviors, diverse mechanics spanning resource collection to combat, and multi-step causal chains. Our design follows three principles:

- 1.
Explicit Object-Oriented State: The entire game state is captured in a single, hierarchical data model that serves as input and output for world models.
- 2.
Functional Purity: The environment’s dynamics are exposed as a pure transition function, $T(\text{state},\text{action})\rightarrow\text{next_state}$, with no hidden variables.
- 3.
Programmatic Modification: The state representation can be precisely manipulated with code, enabling controlled experimental setups.

### B.2 The WorldState Data Model

The core of Crafter-OO is the WorldState data model, which captures the environment at a single timestep. This model is defined using Pydantic for structure and validation. Its components include:

- •
player: A PlayerState object containing position, inventory, health, and current action.
- •
objects: A list of non-player entities (CowState, ZombieState, PlantState, etc.) with type discrimination via a name field.
- •
materials: A 2D array representing the terrain map.
- •
Global Properties: World-level attributes including daylight, size, and serialized random state.

LABEL:lst:worldstate_structure shows the structure of this model. This representation provides the interface between the environment and symbolic world models.

Figure: Listing 1: Simplified structure of the WorldState data structure.

### B.3 Extracting State from Crafter’s Game Engine

Figure: Figure 6: The functional cycle for state transition. A declarative state snapshot is reconstructed into a live, imperative world instance. The engine simulates a single step, and the resulting world is exported back into a new declarative state snapshot for the next timestep. This ensures we match Crafter’s mechanics exactly.

The simulation state in the original engine is not a single data structure but is distributed across a graph of live Python objects, each with its own internal state and complex inter-dependencies, such as non-player characters holding direct references to the player object.
Furthermore, the engine’s behavior relies on implicit state, including the internal state of its pseudo-random number generator, which governs all stochastic events.
Achieving a pure functional interface required developing a robust mechanism to first serialize this entire, complex state into a self-contained, declarative representation and then perfectly reconstruct the live object graph from that representation for each step of the simulation.

The state export process transforms the live simulation into a serializable snapshot. This procedure performs a deep traversal of the game engine’s internal state, capturing all information required to reproduce the exact game moment. This includes the grid of world materials, the positions of all entities, and the type-specific attributes of each entity, such as a zombie’s attack cooldown or a plant’s growth progress. Crucially, the process also serializes the state of the engine’s pseudo-random number generator, ensuring that the sequence of random numbers for subsequent stochastic events is preserved. To maintain the spatial partitioning data used for efficient queries, the set of entities within each world chunk is recorded by storing their unique identifiers. The final output is a complete, declarative data structure that represents the world at a single point in time, free from any live object references or other runtime-specific information.

State reconstruction reverses this process, rebuilding the live simulation from the declarative snapshot. This is more complex than simply loading data.
It involves re-instantiating the entire graph of game objects and correctly re-establishing their inter-dependencies. A key complexity arises from object relationships; for instance, hostile entities require a direct reference to the live player object to guide their behavior. To resolve this, we employ a multi-pass reconstruction algorithm. First, entities with no external dependencies, such as the player, are instantiated. Then, dependent entities are instantiated in a second pass, receiving references to the already-created objects they require. Once all objects are created, the spatial partitioning system is rebuilt by mapping the stored entity identifiers back to the newly created live object instances. Finally, the deserialized state of the pseudo-random number generator is loaded, ensuring that the reconstructed world will produce the exact same stochastic outcomes as the original. The overall process is described in Box 1 and illustrated in Figure 1.

### B.4 The Functional Environment Interface

We provide a transition function that implements a stateless API for environment steps:

- 1.
Input: WorldState object $s_{t}$
- 2.
Reconstruct live game engine instance
- 3.
Execute single update tick with given action
- 4.
Export resulting state as $s_{t+1}$
- 5.
Return new WorldState object

This ensures every transition is a pure function of the explicit state, making the environment suitable for symbolic reasoning and program synthesis.

### B.5 Utilities for Programmatic State Interaction

A key contribution of Crafter-OO is a rich set of utilities that enable programmatic interaction with the world state. These functions are essential for two purposes: first, they allow for the precise, reproducible setup of the evaluation scenarios discussed in [Appendix D](https://arxiv.org/html/2510.12088v1#A4); second, they provide a high-level API that simplifies the authoring of programmatic world model laws. To provide a clear overview of this toolkit, [Table 3](https://arxiv.org/html/2510.12088v1#A2.T3) catalogues the key functions, which are grouped into three main categories: World Setup, Player State, and High-Level State Queries & Modifications.

**Table 3: A catalogue of key utilities for programmatic state manipulation in Crafter-OO. These functions provide the building blocks for creating controlled experimental scenarios and for writing concise, high-level world model laws.**
| Category | Function Signature (Simplified) | Description |
| --- | --- | --- |
| World Setup Utilities | set_tile_material(pos, material) | Modifies the terrain at a specific coordinate (e.g., changes grass to stone). |
| add_object_to_world(cls, pos, ...) | Adds an entity instance (e.g., a Cow or Zombie) to the world. |  |
| remove_object_from_world(obj) | Removes a specific entity instance from the world. |  |
| set_daylight(level) | Sets the global daylight level, affecting visibility and mob spawning. |  |
| Player State Utilities | set_player_position(pos) | Sets the player’s exact (x, y) coordinates. |
| set_player_facing(direction) | Sets the player’s facing direction (e.g., up, down, left, right). |  |
| set_player_inventory_item(item, qty) | Sets the quantity of a specific item in the player’s inventory. |  |
| set_player_internal_stat(stat, val) | Adjusts internal player stats like health, hunger, or energy. |  |
| High-Level State Queries & Modifications | get_target_tile() | Returns the material and any object at the tile the player is facing. |
| get_object_of_type_in_update_range(cls) | Returns all entities of a specific type near the player. |  |
| move_object(obj, dir, walkable) | Moves an entity one step if the target tile is valid and unoccupied. |  |
| set_facing_material(material) | Changes the material of the tile the player is facing. |  |

These utilities are composed to construct the specific initial conditions for our evaluation scenarios. LABEL:lst:scenario_helper_example_revised demonstrates how they work in concert to create a test case for a resource collection mechanic. World setup utilities are first used to clear an area and place a specific resource (coal). Then, player state utilities are used to position the player correctly and provide the necessary tool (wood_pickaxe) in their inventory. This level of programmatic control, enabled by the functions detailed in [Table 3](https://arxiv.org/html/2510.12088v1#A2.T3), is what makes our targeted evaluation methodology possible.

Figure: Listing 2: Example of programmatic state manipulation to create an initial state for a scenario. World setup utilities create the environment, while player state utilities configure the agent.

## Appendix C Mutators

Mutators are a core component of our evaluation framework, designed to test a world model’s ability to distinguish between plausible and implausible future states, as described in [Sec. 4](https://arxiv.org/html/2510.12088v1#S4). A mutator is a deterministic function that takes a state-action pair $(s_{t},a_{t})$ and produces an alternative, incorrect next state $\tilde{s}_{t+1}$. These generated states, called distractors, represent violations of the environment’s true dynamics. For example, a distractor might show the agent crafting an item without the necessary resources or moving through a solid obstacle.

By creating a candidate set containing the true next state $s_{t+1}$ and several such distractors $\{\tilde{s}_{t+1}\}$, we construct a discriminative task for the world model. A model with a robust understanding of the environment’s laws should assign a significantly higher probability to the true outcome than to any of the distractors. This allows us to quantitatively measure the model’s predictive judgment using the state ranking metrics from [Sec. 4](https://arxiv.org/html/2510.12088v1#S4).

All mutators adhere to a common interface, shown in Listing LABEL:lst:mutator_interface. Each mutator implements a ‘precondition‘ method that checks if the mutation is applicable to a given state and action. If the precondition is met, the ‘effect‘ method is called to generate the mutated state. This design allows for the creation of targeted mutators that only apply under specific circumstances, leading to more subtle and challenging distractors.

Figure: Listing 3: The general interface for a mutator. Each mutator is a callable object with a method to check for applicability.

We have implemented a suite of mutators for the Crafter-OO environment, categorized by the type of game mechanic they target. [Tab. 4](https://arxiv.org/html/2510.12088v1#A3.T4) provides a comprehensive list of these mutators and the specific rule violations they introduce.

**Table 4: Catalogue of mutators implemented for the Crafter-OO environment.**
| Category | Mutator Name | Description of Rule Violation |
| --- | --- | --- |
| Physics | IllegalMovementMutator | Causes the player to move when a non-movement action is taken. |
| EntityPositionMutator | Teleports non-player entities to random distant locations. |  |
| Combat | PlayerHealthMutator | Arbitrarily adds or subtracts a small amount of health from the player. |
| EntityHealthMutator | Sets the health of non-player entities to a random, incorrect value. |  |
| Crafting | CraftIllegalItemMutator | Produces a different item than the one specified by the crafting action. |
| Collection | CollectIllegalMaterialMutator | Adds an incorrect resource to the player’s inventory when collecting. |
| Placement | PlaceIllegalItemMutator | Places a different object or tile than the one specified by the action. |
| Player State | InventoryMutator | Randomizes all quantities in the player’s inventory. |

Below we provide detailed descriptions and simplified implementations for three representative mutators from different categories.

### Illegal Movement Mutator

This mutator tests the model’s understanding of which actions cause player movement. It activates when the agent takes an action that should not result in a change of position, such as noop or do. The effect is to move the player one step in a random direction, creating a state that would be valid for a movement action but is invalid for the action actually taken. Listing LABEL:lst:illegal_movement shows its logic.

Figure: Listing 4: Simplified logic for the IllegalMovementMutator.

### Craft Illegal Item Mutator

This mutator targets the logic of crafting recipes. It checks if the agent is attempting to craft an item. If so, it alters the outcome by giving the player a different, randomly selected craftable item. This tests whether the world model has correctly associated specific crafting actions with their unique outcomes. For example, if the action is make_wood_pickaxe, this mutator might instead add a stone_sword to the player’s inventory. Listing LABEL:lst:craft_illegal illustrates this process.

Figure: Listing 5: Simplified logic for the CraftIllegalItemMutator.

### Entity Health Mutator

This mutator introduces arbitrary changes to the health of non-player characters (NPCs), violating the rules of combat, regeneration, and damage. It is an ”always on” mutator, meaning its precondition is always true, as health can be a dynamic property in any state. Its effect is to iterate through all non-player entities and set their health to a random value that is not close to their current health. This prevents generating trivial changes that might occur naturally (e.g., from regeneration) and creates a more distinctively incorrect state. Listing LABEL:lst:entity_health shows the implementation.

Figure: Listing 6: Simplified logic for the EntityHealthMutator.

## Appendix D Scenarios

An evaluation framework that relies on data from unguided exploration may not sufficiently cover all of an environment’s mechanics, especially those that are rare or require specific preconditions. To ensure a comprehensive and targeted assessment of a world model’s understanding, we generate evaluation data from a suite of scenarios. Each scenario is a short, programmatic interaction sequence designed to isolate and test a single game mechanic under controlled conditions. This approach produces a dataset of transitions that robustly covers the environment’s dynamics, from basic resource collection to complex combat encounters. The transitions generated by these scenarios form the basis for the evaluation metrics described in [Sec. 4](https://arxiv.org/html/2510.12088v1#S4).

### D.1 Scenario Structure and Execution

A scenario is defined by a common programmatic interface, as outlined in LABEL:lst:scenario_structure. It specifies an initial state, a scripted policy to guide the agent’s actions, and a termination condition based on either achieving a specific goal or reaching a maximum number of steps. The execution of a scenario, shown in LABEL:lst:scenario_runner, produces a sequence of (state, action, next_state) transitions that serve as ground truth test cases for the world model.

Figure: Listing 7: Structure of an evaluation scenario.

Figure: Listing 8: Execution loop for generating transitions.

### D.2 Implemented Scenarios

We developed over 40 scenarios for Crafter-OO, covering every core game mechanic present in the original Crafter environment. These scenarios are categorized by the type of mechanic they test, as detailed in [Tab. 5](https://arxiv.org/html/2510.12088v1#A4.T5). For many mechanics, we include both a ”successful” and an ”unsuccessful” variant. The successful version sets up the preconditions for an action to succeed (e.g., having enough resources to craft an item), while the unsuccessful version deliberately violates a precondition. This allows us to test whether a world model understands not only what should happen, but also what should not happen.

**Table 5: Complete list of evaluation scenarios used to test world models in Crafter-OO.**
| Category | Scenario Name | Description |
| --- | --- | --- |
| Movement | random_movement | Tests basic player movement in the cardinal directions. |
| Collection | collect_wood | Player faces a tree and collects wood. |
| collect_drink | Player faces water and collects it. |  |
| collect_stone | Player collects stone with the required pickaxe. |  |
| unsuccessful_collect_stone | Player attempts to collect stone without the required pickaxe. |  |
| collect_coal | Player collects coal with the required pickaxe. |  |
| unsuccessful_collect_coal | Player attempts to collect coal without the required pickaxe. |  |
| collect_iron | Player collects iron with the required pickaxe. |  |
| unsuccessful_collect_iron | Player attempts to collect iron without the required pickaxe. |  |
| collect_diamond | Player collects diamond with the required pickaxe. |  |
| unsuccessful_collect_diamond | Player attempts to collect diamond without the required pickaxe. |  |
| eat_plant | Player eats a ripe plant to gain food. |  |
| unsuccessful_eat_plant | Player attempts to eat an unripe plant. |  |
| Crafting | craft_wooden_pickaxe | Player crafts a wooden pickaxe with sufficient wood. |
| unsuccessful_craft_wooden_pickaxe | Player attempts to craft without sufficient wood. |  |
| craft_wooden_sword | Player crafts a wooden sword with sufficient wood. |  |
| unsuccessful_craft_wooden_sword | Player attempts to craft without sufficient wood. |  |
| craft_stone_pickaxe | Player crafts a stone pickaxe with required resources. |  |
| unsuccessful_craft_stone_pickaxe | Player attempts to craft without required resources. |  |
| craft_stone_sword | Player crafts a stone sword with required resources. |  |
| unsuccessful_craft_stone_sword | Player attempts to craft without required resources. |  |
| craft_iron_pickaxe | Player crafts an iron pickaxe with required resources. |  |
| unsuccessful_craft_iron_pickaxe | Player attempts to craft without required resources. |  |
| craft_iron_sword | Player crafts an iron sword with required resources. |  |
| unsuccessful_craft_iron_sword | Player attempts to craft without required resources. |  |
| Placement | place_table | Player places a crafting table with sufficient wood. |
| unsuccessful_place_table | Player attempts to place a table without sufficient wood. |  |
| place_stone | Player places stone with sufficient inventory. |  |
| unsuccessful_place_stone | Player attempts to place stone without sufficient inventory. |  |
| place_furnace | Player places a furnace with sufficient stone. |  |
| unsuccessful_place_furnace | Player attempts to place a furnace without sufficient stone. |  |
| place_plant | Player places a sapling on a grass tile. |  |
| unsuccessful_place_plant | Player attempts to place a sapling without one in inventory. |  |
| Combat | zombie_defeat | Player, equipped with a sword, defeats a zombie. |
| defeat_skeleton | Player defeats a skeleton. |  |
| eat_cow | Player defeats a cow to obtain food. |  |
| player_death | Player with low health is defeated by a zombie. |  |
| NPC Behavior | cow_movement | Tests the stochastic movement of a cow over several steps. |
| wake_up | Player goes to sleep and wakes up after their energy is restored. |  |

## Appendix E Evaluation Implementation Details

This section provides a procedural specification of our evaluation framework. We begin by defining a general-purpose interface that any world model must satisfy to be evaluated. We then detail the computational steps that transform the raw outputs of a model satisfying this interface into the final State Fidelity and State Ranking metrics presented in [Sec. 4](https://arxiv.org/html/2510.12088v1#S4). The process relies on the evaluation trajectories generated from Scenarios ([Appendix D](https://arxiv.org/html/2510.12088v1#A4)) and the distractor states generated by Mutators ([Appendix C](https://arxiv.org/html/2510.12088v1#A3)).

Our evaluation framework is designed to be model-agnostic. Any world model can be benchmarked, provided it adheres to the simple, two-method interface shown in LABEL:lst:evaluatable_interface. This interface cleanly separates the two core capabilities required for our metrics: the ability to generate a likely future state (for fidelity) and the ability to score a given future state (for ranking).

Figure: Listing 9: The interface any world model must implement to be compatible with our evaluation framework.

### E.1 State Comparison via Canonical Representation

All metrics that involve comparing two world states, such as edit distance or checking for equality, require a deterministic and canonical representation of the state. A direct object-to-object comparison can be unreliable due to factors like in-memory object identifiers or the ordering of elements in lists. To address this, we serialize each WorldState object to a canonical JSON format before any comparison is performed. This process, outlined in LABEL:alg:canonicalization, ensures that two states are considered identical if and only if they represent the same game-world configuration.

Figure: Listing 10: Canonical serialization of a WorldState object.

### E.2 State Fidelity Metric Calculation

The state fidelity metrics measure the difference between a world model’s predicted next state and the ground truth. We use JSON Patch (Bryan & Nottingham, 2013), a standard for describing changes in a JSON document, to provide a precise, interpretable measure of this difference. The calculation for a single transition $(s_{t},a_{t},s_{t+1})$ proceeds as described in LABEL:alg:fidelity.

Figure: Listing 11: Calculation of State Fidelity metrics for a single transition.

Example.  Consider a transition where the player, at position $(x=5,y=5)$ with $health=9$, takes the action move_right. The true next state, $s_{t+1}$, has the player at $(x=6,y=5)$ with $health=9$. Suppose a world model predicts a state, $\hat{s}_{t+1}$, where the player correctly moves to $(x=6,y=5)$ but their health incorrectly drops to $8$.

The simplified canonical JSON representations for the player object in each state would be:
[⬇](data:text/plain;base64,ewogICJwbGF5ZXIiOiB7CiAgICAicG9zaXRpb24iOiB7IngiOiA2LCAieSI6IDV9LAogICAgImhlYWx0aCI6IDkKICB9Cn0=)
1{
2 "player": {
3 "position": {"x": 6, "y": 5},
4 "health": 9
5 }
6}
Listing 12: Canonical JSON for the true next state.
[⬇](data:text/plain;base64,ewogICJwbGF5ZXIiOiB7CiAgICAicG9zaXRpb24iOiB7IngiOiA2LCAieSI6IDV9LAogICAgImhlYWx0aCI6IDgKICB9Cn0=)
1{
2 "player": {
3 "position": {"x": 6, "y": 5},
4 "health": 8
5 }
6}
Listing 13: Canonical JSON for the predicted next state.

The JSON Patch required to transform the predicted JSON into the true JSON is a single replace operation:
[{‘‘op’’: ‘‘replace’’, ‘‘path’’: ‘‘/player/health’’, ‘‘value’’: 9}].
The Raw Edit Distance is the number of operations in this patch, which is 1. The Normalized Edit Distance would be this value divided by the total number of elements in the true state’s full JSON representation.

### E.3 State Ranking Metric Calculation

State ranking metrics evaluate a model’s ability to distinguish the true outcome of an action from a set of plausible but incorrect alternatives. This process involves generating a set of candidate states and using the world model to score them, as detailed in LABEL:alg:ranking.

Figure: Listing 14: Calculation of State Ranking metrics for a single transition.

Example.  Continuing the previous example, the true state $s_{t+1}$ is the player moving right. A mutator might generate a distractor state $s_{\text{distractor}}$ where the player illegally teleports to $(x=20,y=20)$. The candidate set becomes $\{s_{t+1},s_{\text{distractor}}\}$. A good world model should assign a much higher probability to the true outcome. For instance, it might yield log-probabilities of $\log p(s_{t+1}|\dots)=-0.7$ and $\log p(s_{\text{distractor}}|\dots)=-15.4$. Since $-0.7>-15.4$, the true state is ranked first. This yields a Rank@1 of 1.0 and a Mean Reciprocal Rank of $1/1=1.0$ for this transition.

### E.4 Aggregation Across Scenarios

The final metrics reported in [Tab. 1](https://arxiv.org/html/2510.12088v1#S4.T1) are aggregated from the per-transition results. To ensure that each distinct game mechanic contributes equally to the final score, we employ a two-level aggregation strategy. First, we compute the mean metric values across all transitions within a single scenario. Second, we compute the final reported metric by taking the mean of these per-scenario means. This prevents scenarios with more transitions (e.g., a long movement sequence) from dominating the overall results compared to scenarios with fewer, more critical transitions (e.g., a single crafting action). LABEL:alg:aggregation formalizes this entire pipeline.

Figure: Listing 15: Overall evaluation pipeline and metric aggregation.

## Appendix F Synthesis and Exploration Implementation Details

The process of generating candidate world laws is divided into two main stages: unguided exploration to collect a dataset of interactions, and law synthesis to propose programmatic laws from that dataset.

### F.1 Exploration Policy

To gather the interaction dataset $\mathcal{D}=\{(s_{t},a_{t},s_{t+1})\}_{t=1}^{N}$, we employ an autonomous exploration policy driven by a large language model. This policy operates without access to environment-specific rewards or human-provided goals. Instead, it is given a high-level instruction to explore the environment and discover as many of its underlying mechanics as possible, treating the task as a reverse-engineering problem. The full prompt provided to the exploration policy is detailed in [Sec. F.1](https://arxiv.org/html/2510.12088v1#A6.SS1).

This LLM-based policy is crucial for gathering sufficiently diverse data in a hostile environment like Crafter-OO. A purely random policy survives for an average of 100 steps before the agent perishes. In contrast, our LLM-based policy navigates the environment for an average of 400 steps. Despite this improvement, exploration remains a significant bottleneck. The policy often struggles to progress through the environment’s technology tree, frequently failing to discover the necessary preconditions for crafting advanced items. It also exhibits a tendency to forget previously learned information, which prevents it from effectively building upon past successes within a single trajectory.

### F.2 Law Synthesis from Trajectories

The law synthesis pipeline processes the trajectory data from the exploration phase to generate a set of candidate laws $\{L_{i}\}$. The core idea is to identify state transitions where meaningful changes occur, and then prompt a large language model to propose atomic, programmatic laws that explain those specific changes. This process is outlined in Algorithm LABEL:alg:synthesis_pipeline.

#### Change Detection for Tractable Synthesis.

In an environment with a complex, structured state like Crafter-OO, changes between timesteps are often sparse and localized to specific sub-components. To make law synthesis tractable, we first isolate these localized changes to provide a focused context for the synthesizer. This is achieved through a set of detectors that monitor different aspects of the world state. An aspect is a semantically-cohesive subset of the state, typically corresponding to a top-level attribute (e.g., ‘player.inventory‘) or a collection of entities of the same type (e.g., all ‘ZombieState‘ objects). For each transition $(s_{t},a_{t},s_{t+1})$, we check for changes across all aspects. If a detector identifies a change, a synthesis task is created for that specific transition and aspect.

Figure: Listing 16: Simplified change detection logic. Each detector checks for changes in a specific part of the world state between $s_{t}$ and $s_{t+1}$.

This decomposition is not a form of environment-specific guidance but rather a generic mechanism derived directly from the structure of the state representation itself. The Crafter-OO environment exposes an object-oriented state, defined by a schema of classes and attributes. Our change detectors mirror this schema, creating one detector for each top-level attribute and for each object type. This approach provides a structural inductive bias—that the environment’s causal mechanisms are likely aligned with its object-oriented structure—without embedding knowledge of the environment’s actual dynamics. The process could be fully automated for any environment that exposes a typed, structured state; the detectors can be generated programmatically by reflecting on the state schema. This is analogous to how a computer vision model might process distinct objects in a scene separately; we partition the state space based on its given structure, but the rules governing the interactions between these partitions must still be learned from scratch.

#### Prompt Generation.

For each transition-aspect pair that triggers a synthesis task, we generate a detailed prompt for the LLM. The goal is to provide all necessary context for the model to infer the underlying game mechanic. The prompt contains several key components:

- 1.
The initial state $s_{t}$ and resulting state $s_{t+1}$, serialized to a structured format (JSON).
- 2.
The action $a_{t}$ that caused the transition.
- 3.
A textual ‘diff‘ that highlights the exact changes between $s_{t}$ and $s_{t+1}$.
- 4.
A human-readable 2D ASCII rendering of the local environment around the player for both states, providing spatial context.
- 5.
The name of the aspect (e.g., “player_inventory”) that changed, which instructs the LLM to focus its analysis.

This structured presentation of the transition allows the LLM to ground its reasoning in the specific, observed changes. The full prompt template is provided in [Sec. F.2](https://arxiv.org/html/2510.12088v1#A6.SS2.SSS0.Px2).

#### Law Generation and Parsing.

The generated prompt is sent to an LLM, which is instructed to return one or more atomic laws that explain the observed changes for the specified aspect. An atomic law is a simple, modular rule focused on a single game mechanic. The LLM’s response is formatted using XML-style tags to clearly delineate the key components of each proposed law.

The expected format for a single law is:

We parse this semi-structured text to extract the natural language description and the executable Python code for each proposed law. This is done by searching for the corresponding tags and extracting their content. The Python code is then loaded as a candidate law for the subsequent parameter inference stage.

Figure: Listing 17: High-level overview of the law synthesis pipeline.

## Peer Reviews

confidence: 4
contribution: 3
initial_rating: 4
presentation: 4
questions: See Weaknesses.
soundness: 2
strengths: - The introduction of Crafter-OO is a valuable contribution to the research community, offering a new, complex testbed for agents that must operate in dynamic environments with a mixture of underlying laws.
- The paper tackles a challenging and important problem setting: learning a world's rules from minimal, unguided interaction in a hostile, stochastic environment.
summary: This paper introduces ONELIFE, a framework for learning a symbolic world model from a single, unguided "one life" exploration in a complex and stochastic environment. The core of ONELIFE is to model the world's dynamics as a "mixture of laws", which are programmatic rules with preconditions and effects. The system uses a probabilistic programming approach to infer the importance of these laws, efficiently routing credit only to the rules that are relevant to observed changes in the state. To test this method, the authors also developed Crafter-OO, a new, complex testbed based on the Crafter environment, which exposes a fully symbolic, object-oriented state. The authors demonstrate that ONELIFE is superior to a baseline in its ability to rank plausible future states, suggesting it learns a more accurate model of the environment's rules.
weaknesses: - The setup, while realistic in its limited interaction budget, makes several unrealistic assumptions. It relies on full, symbolic observability of a structured state, which is rare. The "one life" constraint also feels artificial for many real-world tasks (e.g., robot manipulation) where mistakes are often recoverable.
- The technical novelty of the method itself is somewhat unclear. Domain inference, state tracking, and programmatic models are not new. The paper could do a better job of positioning its technical novelty against this prior work, beyond just the Crafter-OO and "one life" setting.
- The paper is unclear about the environment's specifics. For instance, it's not immediately clear why the object-oriented state isn't just described as a PDDL-based representation. A clear list or description of the agent's action space is also missing, making it hard to grasp the task's complexity.
- The framework is only evaluated on Crafter-OO. While this is a new and complex environment, testing on only one domain makes it difficult to assess the generalizability of the ONELIFE framework.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: * What are the differences between the program synthesizers in this paper versus PoE-World, and why are the differences important?

* Are there metrics involving long trajectories and goals? How good are the learned world models from that perspective?
soundness: 3
strengths: The new Crafter-OO environment is interesting. It's more stochastic (like the random movements of zombies) than the previous environments used in the field. It's good to see that symbolic world models also work in the Crafter environment. 

The paper is generally well-written.
summary: This paper learns a symbolic/programmatic world model for Crafter. It uses LLM for exploration to collect data for training world models. It then asks LLMs to generate codes as world models given the collected data. The programs are modeled as a probabilistic composition of simpler programs/laws, taking advantage of the decompositionability of states to speed up program synthesis. It also builds a Crafter-OO environment together with some evaluation metrics.
weaknesses: It's hard to tell the differences between program synthesizers in this paper versus PoE-World, and why the differences are important.

Although it is informative to check if the next states are predicted correctly, it would be much better to use metrics that involve long trajectories and goals, such as the success rates of solving a problem in Crafter using the learned world model.

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 2
questions: Questions:
- Why are the weights necessary? My guess is that they serve to “soften” the preconditions (which are deterministic predicates), but there is no actual motivation for the inclusion of the weights in the manuscript.
- What is the space of laws? In the Appendix it can be observed that you have a (custom?) library with which you specify the distributions of attributes for the next state, but it is never stated what space is covered with the library (e.g., what types of distributions can be represented).
- What is the experimental setup for measuring the rank? My understanding is that 10 “candidate next states” are generated with a method that leverages domain knowledge to create plausible candidates (one of which is the actual ground truth next state), and then the rank of the ground truth next state among that set is measured. Is this correct?
- Why use reciprocal rank instead of (raw) rank?
- In line 281 you state, “The policy is not provided with specific knowledge of the environment,” but in Box F.1 it can be seen that the exploration policy does leverage domain knowledge (e.g., 1588: “you may encounter creatures that are hostile and will attack you”). This must be clarified.
- Experiments are conducted per scenario, where scenarios. Does this mean that a full model of the environment is never synthesized? I.e., a model that captures all the behaviors that could happen.
- The transition function of the environment is described as a “pure transition function.” Is this accurate? If so, why not synthesize a deterministic world model?
- Have you evaluated the model predicting multiple next states in sequence? Good-enough performance in this is necessary for model-based planning and training. Based on your results (18% rank @ one), I would expect the model to perform poorly (it is still useful for the community to know that this kind of system does not perform well enough).
- Why did you not include other systems as baselines (e.g., WorldCoder)? Your baseline is adequate, but a single data point to compare to is not ideal.
- Is the core difference between the proposed system and PoE that the proposed system aims to synthesize “atomic” laws?

Suggestions:
1. You must correctly state that the exploration policy leverages domain knowledge to get meaningful trajectories (Box F.1). This is important because the quality of the trajectories influences the synthesis of the world model.
2. I suggest fully reworking the description of the synthesizer (line 288), as it cannot be inferred at all that the synthesizer is a custom routine that calls an LLM to synthesize a law for each detected change in the state (Listing 17).
3. I suggest you describe (with some level of precision) the set of laws that can be synthesized. In the Appendix it is shown to be Python programs that use a library to describe the distributions of each attribute of the state in the next state. This should be stated in the main text, as it informs the reader about the level of expressivity of the system and the types of programs that are being synthesized, both of which are important to understand the work.
4. I suggest you provide a visualization of the ranks of the states (e.g., a scatter plot with a corresponding histogram). This would help the reader better understand the performance of the model (and baseline ideally).
5. The laws are stochastic (as can be seen in, e.g., Box A.4), but in line 237 you use notation typically reserved for (deterministic) functions. I suggest making the definition of all parts of the world model mathematically precise.
6. If the per-scenario experimental setup implies that a “full” world model (i.e., a model that covers the entire state space) is never synthesized, then this should be stated in the manuscript, as it helps the reader understand the scope of the experiments.
7. Motivate the use of the law's weights. Ideally this is done with both a conceptual explanation and an ablation study.
8. Adding another baseline would significantly improve the experimental section.
soundness: 3
strengths: - Significance: the work leverages LLMs to synthesize “reactive” laws that are used to parametrize a world model. Even if I don't consider the proposed method's novelty one of its strengths, in my opinion it is of interest to the community to understand the performance (and limitations) of this kind of system, and I expect the results in this work to inform future research. Thus, **I consider significance to be one of the strengths of the paper.**
- Quality: the design choices in this framework are sensible (even if at times the motivations are not clearly articulated). Furthermore, the method appears to be relatively well-documented in the appendix, so even if the results are in some sense negative (the synthesized world models appear to be far from perfect), the experiment results are informative. Thus, **with some changes (see suggestions), I would consider the quality of the work to be adequate for this venue.**
summary: The paper presents an approach to learning symbolic world models from observations in an MDP.

In the proposed framework, a world model is a weighted set of “laws,” each law being a pair of a precondition and an “effect” function that modifies attributes of an input state given an input action. The laws are synthesized by prompting an LLM, and, given the laws, the weights are optimized by maximizing with gradient descent the likelihood of a set of transitions.

The authors conduct experiments on a custom environment that reimplements Crafter (a popular RL environment) but exposes the entire state in text form. The experiments are conducted in a controlled (but significantly limiting) setting and compare the proposed system to a recent baseline, showing some improvement.
weaknesses: The paper is generally well-written, but at times the manuscript could be more precise. In particular:
- Some aspects of the method are not described with enough detail (see suggestions 2, 3 and 5).
- Some aspects of the experimental setup should be emphasized (see suggestions 1 and 6).
- The results are a bit difficult to interpret due to the use of MRR (see suggestion 4).
- Some of the design choices are not motivated (see suggestion 7).

The experimental setup has significant limitations:
- The experiments only include one baseline (see suggestion 8).
- Experiments are performed in a single domain. I do not consider this to be a critical flaw, but it does limit the significance of the experiments.


Decision: Accept (Poster)