Multiverse Mechanica: A Testbed for Learning Game Mechanics via Counterfactual Worlds
Authors: Robert Ness
Ricardo Cannizzaro
Yunshu Wu
Lars Kunze
Venue: ICLR 2026 Poster (2026)
Keywords: Causal generative modelling
Game generative AI models
Counterfactual reasoning
Primary area: causal reasoning
TL;DR: We introduce Multiverse Mechanica, a causal testbed game that generates consistency-guaranteed counterfactual data to evaluate whether generative models can learn underlying game mechanics—not just reproduce visuals.

Abstract: We study how generative world models trained on video games can go beyond mere reproduction of gameplay visuals to learning game mechanics—the modular rules that causally govern gameplay. We introduce a formalization of the concept of game mechanics that operationalizes mechanic-learning as a causal counterfactual inference task and uses the causal consistency principle to address the challenge of generating gameplay with world models that do not violate game rules. We present Multiverse Mechanica, a playable video game testbed that implements a set of ground truth game mechanics based on our causal formalism. The game natively emits training data, where each training example is paired with a set of causal DAGs that encode causality, consistency, and counterfactual dependence specific to the mechanic that is in play—these provide additional artifacts that could be leveraged in mechanic-learning experiments. We provide a proof-of-concept that demonstrates fine-tuning a pre-trained model that targets mechanic learning. Multiverse Mechanica is a testbed that provides a reproducible, low-cost path for studying and comparing methods that aim to learn game mechanics—not just pixels.

Published as a conference paper at ICLR 2025
MULTIVERSE MECHANICA: A TESTBED FOR LEARN- ING GAME MECHANICS VIA COUNTERFACTUAL WORLDS
Robert Osazuwa Ness*1
Ricardo Cannizzaro
Yunshu Wu UC Riverside ywu380@ucr.edi
Lars Kunze UWE Bristol
Microsoft Research robertness @ microsoft.com
University of Oxford
ricardo@ Cecardorrbots..oo ox.ac.uk
lars kunze @ ww..acc.uk
ABSTRACT
We study how generative world models trained on video games can go beyond mere reproduction of gameplay visuals to learning game mechanics-the mod- ular rules that causally govern gameplay. We introduce formalization the concept of game mechanics that operationalizes mechanic-leaminng as causal counterfactual inference task and uses the causal consistency principle to address the challenge of generating gameplay with world models that do not violate game rules. We present Multiverse Mechanica, a playable video game testbed that im- plements set of ground truth game mechanics based The game natively emits training data, where each training example is paired with set of causal DAGs that encode causality, consistency, and counterfactual depen- dence specific to the mechanic that is in play- play-these provide additional artifacts that could be leveraged in mechanic-leamnnn leamnig experiments. We provide a proof- of of-concept that demonstrates fine- the-ttning a pre-trained model that targets mechanic learning. Multiverse Mechanica is testhed that provides reproducible, low-cost path for studying and comparing methods that aim to learn game mechanics- not just pixels.
causal formalism.
INTRODUCTION
Interactive world models have recently gained attention for their potential to simulate or extend video game experiences (Bruce et al. 2024 Parker-Holded & Fruchter 2025 Decart et al. [024 He al. 2025 Che et al.] 2025). SOTA models, typically leveraging deep autoregressive transformer archi- tectures, are trained on large datasets containing sequences of visual frames, user inputs, internal virtual states produced by a graphics engine. These models of video games (game world models) can generate gameplay sequences that are visually similar original gameplay; impressive feat considering modern video games often have cinematic levels of visual complexity.
A key motivation for focus video games is the generation of novel gameplay experiences (Gin- gerson et al., 2024). To this end, the ability to produce high-fidelity visuals is necessity, as the novel experiences must look the part. But addition looking good, the game world model must generate gameplay that is consistent with the game's mechanics (Gingerson et al.) 2024). In simple terms, if the generated gameplay violates game rules or logic, it breaks the gaming experience and, therefore, is not useful, regardless of how good it looks.
Authors of game world models often claim to have learned game's mechanics rules governing gameplay-through post-hoc observations of generated gameplay from the trained model, which vi- sually demonstrates the mechanics in play. For example, in reference to their World Models frame- work, the authors Ha & Schmidhuber (2018) claim that "by learning only from raw image data collected from random episodes, [their model] learns how to simulate the essential aspects of the game, such as the game logic, enemy behavior, physics ...", Kim et al. (2020) claim their GameGAN
"Conesponding author
1 See the Appendix for author contribution details.
I Work conducted durigg an internship at Microsoft Research.
Also affiliated with the Department of Engineering Seience, University of Oxford, UK.

---

Published as a conference paper at ICLR 2025
S
S₂
S3
Takes
Figure I: Clips from consistent contrasts (3.1.2) sampled at their respective impact frames: each column contains two contrasting gameplay clips where differences solely attributable to the shield mechanic. Left, middle, and right correspond to parallel world statements S,, S2, S3.
model learned the collision and power pellet mechanics of PAC-MAN. Parker-Holder & Fruchter 2025 claimed that consistency was an "emergent" property of Genie 3.
There is a problem with such claims. An a posteriori observation that game-world model has learned mechanic suggests that, with SOTA architectures, it is sometimes possible to learn some mechanics or, more precisely, to learn representations that are sufficient to generate gameplay consistent with those mechanics.
However, this does not tell us a priori whether it is possible to learn a representation of mechanic sufficient to reproduce core element of gameplay. To use game world model to reliably generate gameplay experiences in practical settings, we need this knowledge before expending on training and deploying model.
Part of the difficulty is a lack of any formalized notion of what means to learn a game mechanic. could determine if mechanic is statistically identifiable from train-
With formal specification, ing data, and we could evaluate claims that a model has learned a mechanic. Moreover, we could define exactly what it is that we want identify. The model needs learn a representation of mechanic sufficient to reliably reproduce it in generated gameplay, without going as far as reverse- engineering the mechanic's 's source code; formal specification would tell us the level of detail required. Lacking this, we cannot know when we have disentangled the mechanic's representation from learning other representations that drive the game's 's visuals.
In this paper, we address this problem with the following contributions. Firstly, we provide causal formalization of the concept of game mechanics using causal graphical modeling theory. We use this formalization to demonstrate data and inductive biases (in the form of causal graphs) that enable learning a game mechanic. We introduce Multiverse Mechanica, a playable game for use as a testbed for evaluating the learning of game mechanics. The game mechanics are implemented with our causal graphical formalization, providing a ground truth for evaluating mechanic learning. provides causal graphical representations of the mechanic directly to the user for use in evaluation or to supervise mechanie-learning Multiverse Mechanica visually simple enough to facilitate inexpensive experimentation, while it enables generalization by using mechanics that typical of fantasy combat games. Finally, we provide a proof-of-concep: for fine-tuning a generative model on specific game mechanism that leverages these causal representations in its objective function.
BACKGROUND AND RELATED WORK
Defining Game Mechanics. Building on prior definitions (o et al. 2021). ), we define a game
mechanic modular subser (Björk & Holopainen, 2004 Schaul 2013 Thielscher 2011 Zook & Ried] 2019) of the game rules triggered by specific player/agent interactions (Lund gren & Bjork
2003
Fullerton et al. 2004), producing transitions in game state (Järvinen 2008 Fabricatore 2007
2

---

Published as a conference paper at ICLR 2025
Level
Example Statement
Deseription
1) Observation
"A warrior with light weapon might equip shield, block an incoming attack, and take no damage."
prediction drawn from the distribution of observed gameplay; no variable is set by fiat.
2) Intervention
"If beavy-weapon warrior were given a shield, they would block an incoming attack and take no damage."
Shield is forced on, overriding its natural dependence on weapon type.
3) Counterfactual
"Given a shieldless warrior took damage from an attack, if they had instead had shield and blocked, they would have taken no damage-with all else the scene unchanged."
A specific observed episode is revisited under a contrary toccttt fact change, with unaffected variables held consistent.
Table 1: The three meisaalherrrrhhy levels illustrated with the shield mechanic. Each row asks progressively stronger causal claim about the same gameplay scenario.
including transitions that affect gameplay visuals (Hunicke et al. 2004). These subsets entail causal relations with preconditions and effects, representable as logic, finite-state machines, behavior trees,
or transition functions (Zook Riedl, 2014] Thielscher] 2011; Schaul 2013: Dormans) 2012 Zook
& Riedl 2019). We formalize this detinition in Section 3.2
Causal Framing. We adopt the causal hierarchy, which describes three progressively stronger levels of causal reasoning: (level-1) observation, (level-2) intervention, and (level-3) counterfactual (Bareinboim et al. 2022). A level-1 observational statement predicts what might happen based on the distribution of observed gameplay, P(V); no variable is set by fiat. A level-2 interventional statement asks what would happen if a variable were forced to a value- value-possibly one that overrides its natural maases-targeting P(V xz)). A level-3 counterfactual statement revisits specific ob- served outcome and asks what would have happened under a contrary-to-fact change, while holding all unaffected variables are held consistent between the observed world and the hypothetical one; it targets joint quantities such as (Yx=z, Yx=x) that encode "all-else-equal" comparisons. Here, the term "world" refers a hypothetical scenario, and we contrast worlds under different hypothet- ical conditions such as X = versus X = x'. Table iillutraate these three levels with concrete examples from the shield mechanic. Further background on the causal hierarchy is provided in Appendix E. including introduction to the above notation in Table
We use causal DAGs and structural causal models (SCMs) (Pearl 2009) to model mechanics as mechanisms (rules) (Bongers et al.) 2018). For focusing on a mechanic's variables, we reference marginalized DAGs (mDAGs) that preserve causal and interventional semantics while marginaliz- ing others (Evans) 2016). The causal consistency principle states that variables not downstream of an intervention retain the same value across worlds. (Pearl, 2010; Shpitser & Pearl 2012); coun- terfactual graphs capture this cross world consistency compactly (single nodes for shared variables; world-indexed nodes for affected ones) (Shpitser & Pearl 2012). Construction details appear in Appendix E.5 additional background is in Appendix E
Learning game mechanics. Empirical results highlight the challenges of learning game mechan- ics. A study performed by Gingerson et al. (2024) highlights the continuing challenge of gener- ating consistent gameplay with SOTA architectures evvn when generations look plausible, they frequently break the rules of the mechanic. Chen al. (2025) report similar failures in spatial and numerical consistency, necessitating explicit corrective modules. More broadly, empirical studies of video prediction models show that while they excel in-distribution, they often rely on based mimicry and fail under distribution shift, violating simple physical principles (Kang et al. 2024 Riochet et al. 2021).
If we view a mechanic a latent generative factor, then unsupervised learning of mechanics is provably impossible from video observations of gameplay alone (Locatello et al.] 2019) without strong inductive biases (Mitchell 1980 Wolpert 1996). Prior work in this area shows that causal representations are at best only partially identifiable from observational data without intervention data or strong causal inductive biases (Spirtes et al. 2000, Bareinboim & Pearl, 2016 Schölkopf ct al. 202I). Our work builds on prior work that employs these causal approaches to learning latent generative factors. But our knowledge, our work is the first to apply this type of causal analysis to
3

---

Published as a conference paper at ICLR 2025
Descriptions Causal Logic
Formal Counterfactual Notation
Consistent Contrast Sample Data
S1 All else equal, if the opponent has light weapon, they may equip shield; if heavy. they cannot.
S: P(Sw=1 = 1, Sw= 0)
D1= (w), C, (Sw=1, Bw=1, Dw=1, Vw=1),
2
(Sw=o, Bw=o, Dw Vw==))
S2 All else equal, if the opponent shield, they may block; no shield, they cannot.
S2 P(Bs=1 1, Bs=0 =0)
= {w2, C, W,
2 62
BBs=1 Ds=1, Vs=1):
Bs==, Ds=0, Vs=0)
Sy Given shield, if a block succeeds then damage; if fails, damage occurs.
S P(D =0, Da- =11
D== = D C, W, S (Da-1, VB-1). (DB=0, VB=0)]
S = 1)
Table 2: The shield mechanic described as in natural language causal logic (column 1), which are then formalized with counterfactual notation (column 2), where strictly positive probabilities (e; > 0) indicate bounded uncertainty due to other causal factors in the system. Column shows consistent-contrust tuples (each row shares seed w;).
the problem of learning game mechanics during training, and generating consistent gameplay from a trained model.
Datasets, testbeds and environments. Existing testbeds, datasets, and environments for world models and video prediction largely emphasize intuitive reasoning about real-world Newtonian physics rather than explicitly defined game mechanics. For instance, IntPhys (Riochet et al. 2018) probes intuitive physics by testing whether models respect basic object permanence and motion, while Physion (Bear al. 2021) provides simulated videos of collisions and stability events to eval- uate physical prediction. However, game mechanics can encompass non-realistic "physics," such as spell casting and passing through portals. Multiverse Mechanica focuses on a broader set of game mechanics, and contributes a playable generator that emits data and artifacts that target learning of a formally defined ground-true -tunn set of mechanics.
3 FORMALIZING AND LEARNING GAME MECHANIC
In this section, demonstrate the formalization of game mechanic as well how would learn that mechanic from data. Then, Section 3.2 we provide a general mathematical description of this approach.
3.1 ILLUSTRATING EXAMPLE
The Shield Mechanic Consider scene from a stylized 90's fantasy turn- turn-based combat game, where an archer battles a warrior. Like many games in this genre, there is a shield mechanic, as shown in Figure where the warrior may raise a shield to block incoming attacks.
3.1.1 FORMALIZING THE SHIELD MECHANIC
How might we describe this shield mechanic in formal causal terms?
Step I: Describe the Mechanic with Causal Logic. We start by completely describing the shield mechanic using series of causal hypothetical statements of the form "Given preconditions W, all else equal, if X, then Y." Specifically, we focus on level-3 multiverse logic statements that employ conjunctions of conflicting conditions. Table column 1 shows three statements, S1, S2, and Sg. that fully describe the shield mechanic.
The columns of Figure I correspond to S1. S2, and S3. We could instead use level-2 interventional statements, which are normally preferred because they are generally testable with experimental data. But the level-3 parallel world statements provide equal": that outcomes unaffected by the conditions must remain consistent across the clauses. As will below, can that constraint to operationalize consistency in game generation. Secondly, we can leverage the gaming setting's rare opportunity to generate level-3 data to validate level-3 statements.
additional constraint in the phrase "all else
4

---

Published as a conference paper at ICLR 2025
controller (C)
opponent attack
player has shield (S)
score
player weapon class
player does block (B)
scene (V)
player weapon is light (W)
player is hurt (D)
player action
player is immune to attack
Figure 2: Causal DAG marginalized focus on the nodes specific the shield mechanic and their latent confounders (gray nodes)
Step 2: Rewrite Counterfactual Expressions. We rewrite S1, S2, and S3 mathemat- ical expressions using counterfactual notation, capturing the contrasts more compactly (see Table 4 for an introduction to probabilistic counterfactual notation). For simplicity. let W denote the weapon type, S indicate whether a shield is equipped, B indicate whether the shield is used to block, and D indicate whether damage occurs. We treat these as binary variables for clarity, without loss of generality. Let W = 1 and W 0 denote light and heavy weapon, respectively. S, B, and D, let mean True and 0 mean False.
We use counterfactual notation to denote variables under the influence of intervention, such that Y under an intervention that sets X to is written as Yx x=-r- We can formalize the parallel world statements S1. S2, and Sş shown in column 2 of Table 2 where e;; Vi (1, 3) denotes strictly positive probabilities (e, > 0), indicating bounded uncertainty due to other causal factors in the system.
With this, our shield mechanic is described in formal mathematical terms.
Step 3: Represent Mechanic with Causal Graphs Let V denote full clip of gameplay. An outcome, denoted v, a sequence of frames. Let C' denote the controller input from a player at the start of the player's turn. Let G denote the full causal DAG for single turn in the battle (see the full graph in Figure 6 in Appendix FF. Let us assume we have access to this DAG, or that we could create it using knowledge of the game structure, analyzing causal dependence in the game's code (Winskel, 1986 Aho et al. 2006). or by applying causal discovery methods (Glymour et al. 2019) The variables implicated in our description of the shield mechanic are Z (C, W,S, B, D. The causal DAG G is quite large, so we derive the mDAG G.M that zooms in on Z (Figure by marginalizing out the variables not in Z (see Appendix |E.3 for description of the algorithm). Next, we can combine the mDAG with each counterfactual expression in the mechanic's description to construct the counterfactual graphs Figure
The counterfactual graphs in Figure encoode a representation of causal ceaistency-vrraablle that are not downstream of interventions and thus are consistent across worlds are unique, while incon- sistent variables have nodes indexed by each world. Thus, the counterfactual graphs are representa- tions of the shield mechanic that explicitly describe what should remain consistent when generating gameplay depicting the shield mechanic.
3.1.2 GENERATING SHIELD MECHANIC DATA
We can generate level-3 parallel world data consistent with S1, S2, and S3 by creating parallel runs with identical initial conditions and random seeds. We intervene separately in each run, producing clips of parallel virtual worlds that differ only in their respective interventions. We call the tuple of these clips, combined with the outcomes of other mechanic-related variables under these interventions and their shared initial condition/seed, consistent contrasts. Figure Iillustrates gameplay clips from the contrasts.
Let Wi. W2, and W3 represent distinct sets of random seeds and initial conditions. Let C' represent the controller input from the player. Let Vx represent video clip of gameplay under intervention
5

---

Published as a conference paper at ICLR 2025
S1
S2
Sa
20
Ger Acaaen
Seftee ahaak
A
Clou
Book
Bodea
Shic
Figure 3: Counterfactual graphs for the shield mechanic. frames(f)* (rrrmmes((*chanels(33 (
The video variables Vx mar have shape
that sets X to I. Reading consistency from the graphs in Figure 3 we see Ws=1 = Ws=o = W, WB=1- WB=o- W. and SB S S. Let D1, D2, and D3 represent samples of consistent contrasts for S1, S2, and S3 respectively, as shown in column 3.
3.1.3 LEARNING THE SHIELD MECHANIC FROM DATA
Let P1 = Sw P2 = P(Bs=1. 1, Bs=o), and P3 = DD S) denote the distributions constrained by S1, S2, and S3. We can estimate these distributions through repeated sampling of consistent contrasts D1, D2, and D3, then averaging over the sampling distributions to obtain the sampling distributions PI. P1, and P3 (see Appendix [E66). In each case P. converges almost surely to Pt. This provides a precise operationalization of what means for a generative model to learn the shield mechanic: learning constraints (S,, S3} distributions (Pi, P2, P3) or modeling (Pi, } directly.
However, in the canonical case of training game world models, assume that only the controller inputs and the video outputs observed during training. Here, the inference of P; becomes task of unsupervised learning of a latent vector D (C, Vx=z, Vx=r') using {C, Vx=e. Vx=z²) } as features, where Vx= Vx=z' a vector of shape 2 frame height * frame width RGB channels * number of frames. Without further assumptions, disentangling the components of 'D \ {C, Vx=x, Vx=x²) is generally infeasible. However, the counterfactual graphs in Figure already disentangle these variables for us. Using these, the problem reduces to training latent variable model.
3.2 FORMAL FRAMEWORK FOR GAME MECHANICS
We assume a causal DAG G
EV,E) for a single step gameplay with V (C,, C.4,, X V1, V4+1), and typical edges X, + C. X, X C2 + X X, + Ve, X411 - V--1 For a given mechanic, we restrict to the variable subset M = (C,, XM V4+1, X.M S X,, X.M, C Xt+1, and form the marginalized DAG GM by marginalizing variables outside M while preserving interventional semantics (Appendix E.3
We work on a probability space with sample space Q. Each variable Z E M is a measurable mapping Z : + Xz, and each counterfactaal Zx=r= Ω > Xzz, where x E Xx is an intervention
value in the state space of X. The ground-truth SCM, consistent with GM induces the family
F(M) { P(V+ Zx=x D) X, E M, I Xx, E' E'E (
where (M) denotes the o-algebra generated by the variables in M (in practice, can be any measurable predicate on M, e.g., 5=1 for "has shield").
We formalize a mechanic as the tuple (GM,, (M), where M = = (Si,..., S&} and each constraint > & binds counterfactuals of variables in M under inter-
S, P(A my Yx=2, y ventions on X M, with 2y Xx, E E oMM), and e, (0, (allowing non noneetteriiiistic relations due to factors outside M). For each S, we denote the targeted distribution by P. (c.g., Pi P(Sw=1, Sw=o) in the shield example).
E

---

Published as a conference paper at ICLR 2025
Data and Estimation. A consistent-contrast dataset for S, of size N is
D = = (VX=s, ..... Vx=xm, (Wn)) Wh E S2 n= .....N
optionally restricted to win satisfying E. Let p(N) be the empirical distribution induced by D. N Under i..d. sampling of seeds wi. p(N) - P. For each S, (and associated P(), we construct a counterfactual graph G. In partially observed settings (video + controller only), G specifies which latent variables are shared across worlds and which differ, reducing estimation to a well-posed latent variable problem aligned with the counterfactual graph's structure.
4 MULTIVERSE MECHANICA: A PLAYABLE TESTBED FOR LEARNING MECHANICS
We introduce Multiverse Mechanica, a fantasy-style battle game designed as a testbed for learning game mechanics. Unlike static datasets, Multiverse Mechanica is playable game that emits the artifacts required to study and evaluate whether models capture the game's mechanics not just gameplay visuals. Its design integrates three innovations:
(i) native support for level-3 parallel- world contrasts with consistency under the same w; (ii) per-mechanic mDAGs G.M, parallel world and counterfactual graphs, and specifications of MM; and (iii) explicit visual grounding, where stance and scene variables are rendered into pixels. The full codebase for Multiverse Mechanica is publicly available to support reproducible research on mechanic learning
4.1 GAME OVERVIEW
Each episode consists of pre-battle setup (character and equipment selection, random assignment of elemental buffs (e.g., fire, ice) followed by turn-based combat. The player occupies the left side of the screen, and the enemy occupies the right. On the player's turn to attack, liming-based interaction yields an attack score; the enemy's 's turn samples an analogous attack score. Outcomes depend on weapons, defenses, the attack score, and buff's. See Appendix for additional details.
4.2 IMPLEMENTED MECHANICS (V1.0)
Version 1.0 of Multiverse Mechanica includes the following mechanics, each with associated G.M and parallel-world data sufficient to estimate M. The shield mechanic focuses on equip- ping and blocking with shield, as discussed in Section 3.1 In the elemental immunity me- chanic, "elemental" attributes (e.g.. fire and ice) govern immunity and vulnerability to attacks. The weapon range mechanic governs melee vs. ranged combat. The spell-casting mechanic gov- ems five submechanics that allow players to give themselves an advantage in battle (e.g., gain increased attack power, dodge ability), or their opponent disadvantage (e.g., disarm them or lower their defense) -projccciiiojeeelsesss. self-levitation, enemy-levitation, self-trans transformatiion, and enemy- transformation. See Appendix F for detailed descriptions, including causal formalizations, DAGs, and illustrations.
4.3 DESIGN DECISIONS
Mechanic-Specific Game Systems. Each mechanic is implemented with a unique instance of game system (Nystrom, 2014; Gregory 2018), independent of the others. This ensures parity between the mechanic and the code logic. Full game system details are given in Appendix G.I.I
Impact Frames and Visual Conventions. We designed the game such that each turn contains an impact frame-the most visually and mechanically expressive phase of an interaction (e.g., the precise moment when an attack lands). Impact frames are not defined by fixed time-points; instead, they are triggered by specific game state configurations. See Appendix G.2 for details.
Simple yet Information Dense Visuals.
facilitate rapid, inexpensive experimentation, we focus on the ability to run experiments with episodes that have a minimal number of frames. To this
httpp::/ http:/ggttutboooo http:////ttttt..oo.....aaammmaaa htt.://w......oooooooaooaeeee.ee

---

Published as a conference paper at ICLR 2025
end, simple art style that aligns with the representational biases of pretrained vision models (lucil 2024; Zhang et al. 2023). A complete tttack-reactie animation plays after player input at turn start, permitting cheaper architectures that model multi-framm action sequences instead of one frame per action. Animation conventions emphasize dynamic information, such as speed lines ("zip ribbons") depict fast motion, trajectory lines for projectiles, curved swipes for melee attacks, and burst lines and explosion visual effects for collisions or blocked strikes (McCloud) [2020) Eisner 2008 Cohn 2013). In Section 5 we highlight this ability by limiting our analysis single time- point snapshots at the impact frame, chosen as the impact frame of the clip. The images in Figure] are all sampled at their respective impact frames.
4.4 DATA GENERATION
Multiverse Mechanica is not dataset but generator. To generate data, an automated agent repeat- edly plays the game to produce clips. Users can select number N of generations. The generation process can randomly generate N clip examples, constituting level-1 data. The user can also specify interventions on specific game state variables and generate N clip examples where those interven- tions are applied, constituting level-2 data. Finally, the user can specify interventions and assign them to multiple game instances with a shared "w" (same random seed and initial conditions) and generate N consistent contrasts (tuples of clips), constituting level-3 data. Each mechanic has pre- sets for level-2 and level-3 generation. Each clip is 512x512 MP4 video averaging 4 seconds at 50 FPS. Each generated example is a tuple consisting of a clip, controller inputs, game-state variable outcomes, and a random seed for reproducibility. See Appendix H for additional details related to data generation.
Summary. Multiverse Mechanica provides compact yet expressive testbed for studying whether generative models can capture mechanics. Its design couples causal structure with visual grounding, leverages art and animation conventions for low-cost training, and enables reproducible creation of parallel-world contrasts.
PROOF-OF-CONCEPT: LEARNING A MECHANIC WITH MULTIVERSE FINE-TUNING
In this section, we provide a proof-of-concept that illustrates how the proposed causal graphical approach to identifying and learning game mechanics can motivate a new world model training strategy. Given realistic player action policies, real-world game engines could be reconfigured into simulators that generate consistent contrasts depicting the mechanics they implement. We could then treat the visual information of what remains consistent across the pair as supervision in a con- trastive learning objective. Using this objective, we could fine-tune a world model to improve its causal consistency; method we call multiverse alignment. The initial training phase learns rough representation of the mechanic, then multiverse fine-tuning refines that representation into one ca- pable of producing causally consistent depictions of the mechanic. We demonstrate this idea using consistent contrasts generated with Multiverse Mechanica. which generates consistent contrasts by design.
There is no established metric for causal consistency of full video roll-outs. Practice is to com- pare event-aligned frames- same event label, possibly different timestamps-witth image similarity (PSNR, SSIM) (Wang et al., 2004 Wolf et al. 2009). This is orthogonal to within-vides temporal generation. We thus evaluate our proof-of- of-ccceept by applying these metrics multiverse fine- tuning of pre-trained conditional image diffusion model targeting event-aligned frames.
Concretely, we generated contrastive pairs spanning the full set of mechanics (i.e., shields, elemental immunity, weapon range, and spell-related submechanics), as discussed in Section 4.2 Each pair consists of two video roll-outs that depict a complete interaction: an attacker executes an attack animation, and the reactor produces the corresponding response. For every roll-out, the simulator also records a dictionary of outcome values for the game-state variables that determine the events shown in the video. For each contrastive pair, extract the impact frame (Section 4.3) from each roll-out, yielding a paired set of event-aligned images. We then fine-tune the conditional image diffusion model OpenJourney-v4 (PromptH-ero) 2022) on these impact-frame pairs. To condition
8

---

Published as a conference paper at ICLR 2025
B
Figure 4: Counterfactual samples from the multiverse-aligned diffusion model (shield mechanic). Non-mechanic content preserved across interventions. (A) Held-out factual image: wizard attacks warrior. (B) V==1,B=0* shield present, no block. (C) B== shield and block active (note: minor sword aliasing artifact).
the model, convert the associated game-state dictionary into textual description that is provided as input.
Conceptually, consider image Vx== snapshot of simulated roll-out from the game with mechanic-related state settings X = x (as affected by controller input). The simulation has a random seed w, a random draw whose value proxies for the effect that variables, which are in or affected by X, have on Vx=a- A consistent contrast is paired set of images (Vx=ro>> Vx=z,) ) that share random seed w. The shared seed enforces causal consistency; the only differences between Vx==, and Vx==,) ) are due to different interventions on X.
We can view the conditional image diffusion model as a model of how the simulation culminates in snapshot Vx=-- With timesteps E [0, 7), Zr is the noisy latent that initializes the denoising process, and the conditioning variable used to guide that process. Starting from completely ran- dom ZT, one denoising step conditioned on X yields -1,1, Iterative denoising from (counting down to 0) culminates in Zo,x=av which then decoded into Vx=a.
Multiverse alignment is motivated by viewing Zr an analog of w: both determine what varies in the image given X. To reflect the shared w in a consistent contrast, we constrain the Zr in each trajectory to agree across the pair; the objective modifies standard diffusion training to enforce this.
Conditioned on controller input c, the denoiser eœ(-) iteratively transforms the shared seed w into clean latent Zo,x= that decodes into the frame Vx=x,. The multiverse alignment loss has two components:
L == A C2, d1 A2 2 0, X +2
C,: seed ceed-consistency loss. schedule estimate ūj (deterministic inversion; Appendix 1.10.5). The loss enforces agreement between abducted seeds:
Abducta traces No.x and cj backward via ca(-) through the noise
L1 (7,,,x=a- Co Z0,x=z1;1) (1) Abducto Zo, X Co) Abducto (o, (1) ll
Remark. We use deterministic diffusion inversion to estimate w and penalize differences between abducted seeds.
Let C {l,..., , T} be subset of high-noise timesteps. For each t S, the denoiser predicts noise ca (2, Cy). We align predictions across the contrast:
L2t structure-aligmment loss.
L2
Σ IES
en (2, co) e0 mnd 11
Remark: Aligning high-noise steps enforces global identity; mechanic-specife differences emerge in later steps.
Evaluation Results. Figure shows qualitative counterfactual generations: the model preserves non-mechanic-related content while toggling the targeted shield mechanic. From an input "factual" image e with = z we abduct the exogenous seed w, then generate a counterfactual image with X = x' while keeping fixed, sampling from P(Vx=r' X x, V v). This provides visual evidence that the fine-tuned model has learned a representation that can generate causally consistent aspects of the mechanic, not merely pixels.

---

Published as a conference paper at ICLR 2025
To quantify the contribution of each loss component, conduct ablation study over four configurations-staae standard diffusion fine-tuning (baseline). L2 only, L1 only, and the full loss C1+t2-evaluated against engine-rendered ground-trute counterfactuals (Table Reconstruction metrics compare inverted images to their inputs; counterfactual (CF) metrics compare generated counterfactuals to ground truth; and Exo MSE measures |100 W1 111 the squared distance between abducted seeds of a consistent-contrast pair.
The results divide by the presence of Li. Without it, Exo MSE remains high (+s 0.79); adding La alone is indistinguishable from the baseline. Including Li reduces Exo MSE by nearly an order of magnitude (== 0.09), enforcing tight seed alignment at cost of some pixel-level fidelity. The dtructrrealignnee term L2 complements L1: adding it improves reconstruction SSIM from 0.275 to 0.429 and counterfactual SSIM from 0.129 to 0.196 without degrading seed alignment. CLIP- based evaluation corroborates this: CF Transfer CLIP is substantially higher with the full loss ( 24 vs. 17 for L1 alone) (Hessel et al.] 2021 Radford et al.) 202T), and image-level CLIP scores (image-text: 24.42, image-image: 0.898) confirm that the procedure preserves input-side fidelity.
Table Ablation over loss components evaluated against engine-rendered ground-trun truth counterfac- tuals. A horizontal rule separates configurations without Li (top) from those with it (bottom).
Reconstruction
Counterfactual
Configuration
PSNR1
SSIM
PSNR1
SSIM
Exo MSE]
Diffusion only
19.54
0.776
13.58
0.358
0.793
w/o L1
19.42
0.769
13.55
0.355
0.795
w/o C2 13.35
0.275
11.01
0.129
0.087
Full (CI +22
14.02
0.429
11.04
0.196
0.092
SCALABILITY & LIMITATIONS
Multiverse Mechanica provides a simplified setting for experiments: a 2D turn-based domain. The logic of the causal formalization generalizes principle to continuously interactive 3D environ- ments, continuous-valued interventions, and temporal dynamics. However, further work is needed to analyze how to formalize more complex mechanics that arise such settings. Furthermore, development of new metrics for evaluating causal consistency across entire roll-outs remains open challenge.
The proof-of-concee presented here is limited. For simplicity, focused modeling single impact frames rather than full video roll-outs; we did this with an image model rather than a standard world model architecture. The seed-consistency loss L1 requires tracking gradients through every step of the DDIM inversion trajectory, making it substantially more expensive than standard diffusion training.
The same causal-consisenccy formulation extends to other controllable simulators and VFX pipelines; we discuss pathways for transferring these ideas to more realistic settings in Appendix L Multiverse Mechanica serves as best-case benchmark with explicit contrasts, whose insights can inform models trained under weaker supervision.
7 CONCLUSION
We formalized game mechanics causal counterfactual inference and introduced Multiverse Me- chanica, playable testbed that emits parallel-world contrasts and per-mechanic causal graphs. Building on this foundation, we proposed multiverse-alinnment objective and demonstrated proof-of-concept fine-tuning that learns targeted mechanics. Together, these components provide reproducible path to assessing whether world models learn mechanics-not just pixels.
ACKNOWLEDGMENTTS
This research was supported in part by the Australian Defence Science and Technology Group. Lars Kunze's RAi UK Enterprise Fellowship supported by EPSRC [grant number EP//0009800/1].
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Alfred V. Aho, Monica S. Lam, Ravi Sethi, and Jeffrey D. Ullman. Compilers: Principles, Tech- niques, and Tools. Meddsonn-Weslyee Boston, MA, 2nd edition, 2006. ISBN 97803214868155
Elias Bareinboim and Judea Pearl. Causal inference and the data-fusion problem. Proceedings of the National Academy of Sciences, 1 113(27):345-7352 2016. doi: 10. 1073//pas.1110000
Elias Bareinboim, Juan D. Correa, Duligur Ibeling, and Thomas Icard. On pearl's hierarchy and the foundations causal inference. In Hector Geffner, Rina Dechter, and Joseph Y. Halpern (eds.), Probabilistic and Causal Inference: The Works of Judea Pearl, pp. 507-556. Association for Computing Machinery, New York, NY, USA, 2022. ISBN 9781 9781450395861. doi: 10. 10.1145/ 350171433001377.
Daniel M Bear, Elias Wang, Damian Mrowea, Felix Binder, Hsiao- Yu Fish Tung. RT Pramod, Cameron Holdaway, Sirui Tao, Kevin Smith, Fan- Yun Sun, et al. Physion: Evaluating physical prediction from vision humans and machines. arXiv preprint arXiv:2106.08200 2021.
Eli Bingham, Jonathan P Chen, Martin Jankowiak, Fritz Obermeyer, Neeraj Pradhan, Theofanis Karaletsos, Rohit Singh, Paul Szerlip, Paul Horsfall, and Noah D Goodman. Pyro: Deep universal probabilistic programming. Journal of machine learning research, 20(28):1-6, 2019.
Staffan Björk and Jussi Holopainen. Patterns in Game Design. Charles River Media, Hingham, MA, 2004. ISBN 1-58450-354-8.
Stephan Bongers, Tineke Blom, and Joris M Mooij. Causal modeling of dynamical systems. arXiv preprint 1803 08784, 2018.
Jake Bruce, Michael D Dennis, Ashley Edwards, Jack Parker-Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, et al. Genie: Generative inter- active environments. In Forty-first International Conference Machine Learning. 2024.
Haoxuan Che, Xuanhua He, Quande Liu, Cheng Jin, and Hao Chen. Gamegen-x: Interactive open- world game video generation. In International Conference on Learning Representations, 2025. URL httptttss::/pppeeeeieieee
n forum? wtw.rrumidd-aatttr33 8VG8tpP2NN Zhe
Z. Chen, T. Xu, Y. Zhang, et al. Model game: On numerical and spatial consistency for gen- erative games. arXiv preprint arXiv:2503.2117. 2025. URI. htt.ttts///exii..rrrre... http///axxiv..../a...s/. abs/ 2503. 21172
Neil Cohn. The Visual Language of Comics: Introduction to the Structure and Cognition of Sequen- tial Images. Bloomsbury Advances in Semiotics. Bloomsbury Academic, London, 2013. ISBN 9781 9781472542175.
Decart, Julian Quevedo, Quinn McIntyre, Spruce Campbell, Xinlei Chen, and Robert Wachen. Oa- sis: universe in a transformer. bttps::/oassismoose git.ll.itthubbb... io/] 2024. Accessed: 2025-09-24.
Joris Dormans. Engineering Emergence: Applied Theory for Game Design. PhD thesis, University of Amsterdam, 2012. URL https:://printss www.::llw..............eo w.t.../w..../////errrtt////1111.
03-2012-12.55 .dd
Will Eisner. Comics & Sequential Art. W. W. Norton & Company, New York, revised edition edition, 2008. ISBN 978039333 1264. First published 1985; expanded edition 1990; revised edition 2008.
Robin J Evans. Graphs for margins of bayesian networks. Scandinavian Journal of Statistics, 43(3): 625-648 2016.
Carlo Fabricatore. Gameplay and game mechanics: A key to quality videogames. In Proc. OECD Expert Meeting Videogames and Education, 2007. URI. httpt::tp///....... nnttps://eprintst hud ac.
uk/ ht///d//ppp h.d/pprinn//0097..
Tracy Fullerton, Chris Swain, and Steven Hoffman. Game design workshop: Designing, prototyp- ing, & playtesting games. CRC Press, 2004.
11

---

Published as a conference paper at ICLR 2025
J. Gingerson, S. Amershi, et al. World and human action models towards gameplay ideation. Nature, 2024. URL technical report also available.
http://ww...uu.....mmm..m.mm nature. com/ con//atticlle///an222222 Microsoft Research
Clark Glymour, Kun Zhang, and Peter Spirtes. Review of causal discovery methods based on graph- ical models. Frontiers in genetics, 10:524, 2019.
Jason Gregory. Game Engine Architecture. A K Peters/CRC Press, Boca Raton, FI., 3 edition, 2018.
ISBN
David Ha and Jürgen Schmidhuber. World models. arXiv preprint arXiv:1803. 2018. URL
Xianglong He. Chunli Peng. Zexiang Liu, Boyang Wang. Yifan Zhang, Cui, Fei Kang, Biao Jiang, Mengyin An, Yangyang Ren, Baixin Xu, Hao-Xiang Guo, Kaixiong Gong, Cyrus Wu, Wei Li, Xuchen Song. Yang Liu, Eric Li, and Yahui Zhou. Matrix Matrix-game 2.0: An open-source, real-time, and streaming interactive world model. arXiv preprint arXiv:2508. 3009, 2025.
Mariya Hendriksen, Tabish Rashid, David Bignell, Raluca Georgescu, Abdelhak Lemkhenter, Katja Hofmann, Sam Devlin, and Sarah Parisot. Adapting vision- vinion-language models for evaluating world models. XrXiv preprint arXiv:2506.17967 2025.
Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation metric for image captioning. In Proceedings of the 2021 Conference on Empirical Methods Natural Language Processing (EMNLP), pp. 7514-7528, 2021. doi:
10.18653/v 1/202111111111222 1etl02eeenllpmmeee 59.
Paul W. Holland. Statistics and causal inference. Journal of the American Statistical Association, 81(396))945-960, 1986. doi: 10.2307/2289064.
Charles Tapley Hoyt, Craig Bakker, Richard J. Callahan, Joseph Cottam, August George, Ben- jamin M. Gyori, Haley M. Hummel, Nathaniel Merrill, Sara Mohammad Taheri, Pruthvi Prakash Navada, Marc-Antoine Parent, Adam Rupe, Olga Vitek, and Jeremy Zucker. Causal Iden- tification with Yo. 2025. URL https:://doll wtg//oo.oogg...oo 1.0...85500 48550/ 2508. 03167 arXiv:2508.03167.
Robin Hunicke, Mare LeBlane, and Robert Zubek. proach to game design and game research.
MDA: formal ap- AAAI
In
Proc.
Workshop
Challenges in Game AI, 2004.
actps://aaai http://aaaa..oo/pppperr.. and--inn-nd-aa------eaa wsc4-04-00: aormaa approach-to-gaanne htg.aaa-------eeeeeeeeeeaaaaaea-
URL
Aki Järvinen. Games without Frontiers: Theories and Methods for Game Studies and Design. PhD thesis, University of Tampere, 2008 URL http::tttt/tttttt /
metu edu.. wttp:t////uugniflee...
Dissertation.
D. Kang, Y. Wu, et al. How far is video generation from world models? 1901 2024. |ttppstss///axxivv.ooo wttp:/iiww..rg//////////0 19014
arXiv preprint
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. Advances in neural information processing systems, 35:26565-26577, 2022.
Seung Wook Kim, Yuhao Zhou, Jonah Philion, Antonio Torralba, and Sanja Fidler. Learning to simulate dynamic environments with gamegan. Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020. URL
thecvf.
com/ cttt//ootee__ceee wwwp_//2P22220 201FR22220/pppeeeee Btae//iinn_aeLaaee Learning wwt.Siimmlle__e___ ate DatbmicDnmmc___
Envi ronments Environnents s s_tmments__tthhhhh SachhhaaeeaaaaCC _C1.aa_____________ 201PP_20000__221 ______aaee___________________ www.ppppppp.... .dff]
Priscilla Lo, David Thue, and Elin Carstensdottir. What is a game mechanic? In International
Conference Entertainment Computing, pp. 336-347. Springer, 2021.
12

---

Published as a conference paper at ICLR 2025
F. Locatello, S. Bauer, M. Lucic, G. Raetsch, S. Gelly, B. Schölkopf, and O. Bachem. Chal- lenging common assumptions in the unsupervised learning disentangled representations. In Proceedings of the 36th International Conference Machine Learning (ICML), 2019. URL 123399
lucil. Chibi model. httpp:/s/dakaaaaaacc///////////a
2024. Community submission on Civitai. Accessed: 2025-09-24.
Sus Lundgren and Staffan Bjork. Game mechanics: Describing computer-augmmetted games in terms of interaction. In Proceedings of TIDSE, volume 3, 2003.
Scott McCloud. Understanding Comics: The Invisible Art. Harper Perennial, reprint edition edition, 2020. ISBN 9780060976255.
M. Mitchell. The Need for Biases in Learning Generalizations. Technical Report CBM-TR-1177 Rutgers University, 1980.
Miguel Monteiro, Fabio De Sousa Ribeiro, Nick Pawlowski, Daniel C Castro, and Ben Glocker. Measuring axiomatic soundness of counterfactual image models. arXiv preprint arXiv:2303.01274. 2023.
Robert Nystrom. Game Programming Patterns. Genever Benning, 2014. ISBN 9780990582908.
Jack Parker-Holder and Shlomi Fruchter. Genie 3: new frontier for world models. Google DeepMind Blog, aug 2025. URL httpp:: //de/mindde.d goog e/di di scooerrbloo genie- a- frontier- for worldworld-oo world-models/ Accessed 24 September 2025.
Judea Pearl. Causality. Cambridge university press, 2009.
Judea Pearl. On the consistency rule in causal inference: axiom, definition, assumption, or theorem?
Epidemiology. 21(6))872-875 2010.
PromptHero. Openjourney v4.
openjourrey-v4 2022. Diffusion model hosted on Hugging Face.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and llya Sutskever. Learning transferable visual models from natural language supervision. Proceedings of the 38th International Conference on Machine Learning (ICML), volume 139, pp. 8748-8763, PMLR, 2021. http: http:::///cocccenggs.. mlr press wwwss/vweseee htts///1111//raffo2....... htt.oordllaaa..llllll..mmm
Riochet, H. Le Borgne, E. Ricci, al. Intphys A benchmark for physical consistency in video prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021. URL httttts:///axiii/ org/ abs abs/ 06227
Ronan Riochet, Mario Ynocente Castro, Mathieu Bernard, Adam Lerer, Rob Fergus, Véronique Izard, and Emmanuel Dupoux. Intphys: A framework and benchmark for visual intuitive physics reasoning. arXiv preprint 18Xiv:1803.076// 2018.
Tom Schaul. video game description language for model-based or interactive learning. In Proceedings of the IEEE Conference Computational Intelligence in Games, 2013. URL https://scct
s site44. www.coo//oooo.o hotp:///aab...tttt.oo////gggllll
B. Schölkopf, F. Locatello, S. Bauer, N. R. Ke, N. Kalchbrenner, A. Goyal, and Y. Bengio. Towards causal representation learning. Proceedings of the IEEE, 109(5):612-634, 2021. doi: 10.1109/ JPROC.2021.305899
Ilya Shpitser and Judea Pearl. What counterfactuals can be tested. arXiv preprint arXiv:1206.5294, 2012.
Spirtes, C. Glymour, and R. Scheines. Causation, Prediction, and Search. MIT Press, 2nd edition, 2000. ISBN 9780262194402.
13

---

Published as a conference paper at ICLR 2025
Michael Thielscher. The general game playing description language is universal. In Proceedings of the 22nd International Joint Conference on Artificial Intelligence (ICAI)) 2011. URL hetps:
/ www. jeaii htwpaaorrgorroceigggeeeee httpiings/////Paereeeee
Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):
600-612, 2004. doi: 10. 10.1119//111 10.11099TP.2003..0 819661.
Glynn Winskel. Event structures. In Advanced course on Petri nets, pp. 325-392. Springer, 1986.
Stephen Wolf, Margaret Pinson, al. Reference algorithm for computing peak signal noise ratio (psnr) of a video sequence with a constant delay. Technical Report NTIA/ITS / COM-9 C.6, National Telecommunications and Information Administration (NTIA) ITS, 2009. Proposed reference implementation and calibration procedures for PSNR.
D. H. Wolpert. The lack of a priori distinctions between learning algorithms. Neural Computation, 8(7):1441-390,, 1996. doi: 10.1162/ncco.19% 1996.8.7.1341.
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 3813-3824. IEEE, 2023. ISBN 9798350307184, doi: 10.1 10.1109/CCV51100.222. 00355.
Alexander Zook and Mark Riedl. Automatic game design via mechanic generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 28, 2014.
Alexander Zook and Mark O. Riedl. Automatic game design via mechanic generation. arXiv preprint arXiv:1908.01 04420, 2019. htllhttss://aaatt wrt. httpg//bs///......
/ttp:////aaaaa.tr///////0000000 01420
A USE OF LARGE LANGUAGE MODELS (LLMs)
GPT-5 used for editing and polishing during the writing of this paper. The LLM used to make any technical or scientific contributions to the paper writing process; for example: writing the literature review background sections, creating citations, analyzing data.
B ETHICS STATEMENT
There are no ethics concerns raised by this paper.
C REPRODUCIBILITY STATEMENT
To support full reproducibility of this work, make the following resources publicly available:
The complete implementation of Multiverse Mechanica, research testhed for eval- uating learning of game mechanics, including the game engine, causal-graph con- struction utilities, parallel-world data generation pipeline, and proof-of-concept train- ing scripts, is publicly available at hhttps://githhii http://ww.....cc.ccedrcaaaaaaaaa multiverse mechanica
The proof-of-concept dataset used Section 5 including mechanism- speeiffc graphical artifacts and level-3 worlllel-world contrasts, is accessible via links provided in the Multi- verse Mechanica repository.
C.I PROOF-OF-CONCER DATASET
Here, we describe the dataset used for the proof-of-concept training presented in Section 5 used to learn the shield gameplay mechanic.
14

---

Published as a conference paper at ICLR 2025
Figure 5: Impact frame generated by Multiverse Mechanica for world in a parallel world tuple.
C.I.1 GENERATION
We generate level-3 dataset, as described in Section 4.4 We automatically derive parallel world interventions based on the full game DAG, targeting the variables relevant the parallel world contrast statements for the shield mechanic. The set of derived interventions define multiple parallel world tuples, enumerated such that we sufficiently cover the support of joint distribution of the mDAG induced by the query variables associated with the level-3 parallel world statements for the mechanic.
We assign these interventions multiple game instances with shared "w" (same random seed and initial conditions) and generate N = 1000 consistent contrasts, constituting level-3 data. Since we training text and images (i.e., not gameplay clips), for each we generate only subset of training artifacts consisting of these two modalities: 1) the impact frame as a 512x512 PNG image, and the game-state variable outcomes (converted into caption in the pre-processing method, outlined below).
C.1.2 PRE-PROCESSING
To align our artifact modalities with the text-to-image latent diffusion architecture used in our proof- of-concept, we must perform a pre-processing step to convert from the game-state variable out- comes dictionary of variable-name// captioning process for each parallel world tuple, by which plain-text captions are deterministically constructed based on the values of the game-state variables.
pairs-to
caption. We implement a templated
Consider, for example, the following impact frame image and game-state variable artifacts generated by Multiverse Mechanica for one the worlds in a parallel world tuple.
For this world, Multiverse Mechanica has generated the impact frame image shown in Figure 5 and the following game-play state:
smmplay_stast {
"background'; : 'forest', plaaerr action'; : 'melee attack',
"player class'; 'warrior', "player does block/ False, 'player element's : rice',
15

---

Published as a conference paper at ICLR 2025
'player has shield': False, ppaayer is hurt'; False, plaayer is immune to attack'; False, 'player woapon': ssoott sword', 'player weapon class' 'light', 'player weapon element': rice', 'player weapon is light' True, "paayer weapon range'; 'melee', opponent action': 'detend', opponent class' 'warrior', opponent does block's True, opponent element': : 'noee', opponent has shield': True, opponent is hurt' False, opponent is immune to attack'; False, opponent weapon': 'short sword', opponent weapon class'; 'light', opponcnt weapon element': 'fire', opponent weapon is light' Truc, opponent weapon range': 'melee',
Given this game-play state, our pre-processing step constructs the following caption:
"A 1-on-I battle between two warriors. The warrior the left has ice element buff. The warrior on the left's weapon is a short sword with a ice buff. The warrior the left is launching melee attack. The warrior on the right's weapon is short sword with a fire buff and they have a shield. The warrior on the right is blocking with their shield."
The captions generated by the pre-processing step is combined with the generated image to thus create training data suitable for text-to-image latent diffusion architecture used our proof-of- concept.
D AUTHOR CONTRIBUTIONS
We follow the CRediT taxonomy to describe author contributions. Robert Osazuwa Ness led the project, including conceptualization, methodology, and formal analysis, and wrote the original manuscript draft; he also supervised the work, coordinated the project, and led revisions. Ricardo Cannizzaro developed the Multiverse Mechanica codebase and testbed over an extended period, including environment design, combining causal abstractions with game logic, evaluation/bench- marking tooling, dataset generation pipelines, and supporting analysis scripts; he also contributed to manuscript revisions and the reviewer response. Yunshu Wu developed and implemented the proof-of-concept multiverse fine-tuning approach (Multiverse Alignment), including the LI seed consistency loss and 1.2 structural-alignment loss formulations, experimental codebase, metric selection and implementation, and experiments/ablationss she also contributed text and edits to the manuscript, reviewer feedback, and appendix. Lars Kunze provided supervisory support and manuscript feedback.
E BRIEF PRIMER ON CORE CAUSAL CONCEPTS
E. THE CAUSAL HIERARCHY
The causal hierarchy describes three levels of statements that employ causal logic: (level-1) obser- vation, (level-2) intervention, and (level-3) counterfactual (Bareinboim et al.) 2022). predictive statement, such as "The scene would generate in this way." is a level-1 observational statement. A level-2 interventional statement targets outcomes under hypothetical interventions, such as "If an enemy were placed here (e.g., in place it would not naturally appear), then the scene would gen- erate in this way." A level-3 counterfactual statement considers observed outcomes and how those outcomes might have been different under hypothetical interventions. For example: "Given there
16

---

Published as a conference paper at ICLR 2025
Variable / Operator
Description
P(X)
The probability distribution for random variable X
P(X Z=z)
As above, but conditioned observation Z=z
do(X=x)
An intervention taken to fix variable X to value x. Interrupts sampling from its conditional probability distribution given parents in the causal DAG: P(X P())) Probability distribution for where an intervention is taken to fix variable value x (i.e.. do(X=x)) (optionally conditioned on observation Z=z)
P(Yx=r Z=z)
S,
The 1h contrast statement, in causal terms and counterfactual notation, respectively
D.
The 1th consistent contrastt tuple, defining two worlds sharing a common seed
X1, M,C,V
Game data at frame t: game state Xt. player controller input C. and video frame (image) vector Vi.
Table 4: Key probability distributions, variables and operators used in the adopted formalism.
was no enemy and the scene generated that way, if an enemy were placed here, the scene would generate this way."
Formally, level-k statements describe events in the sample space of a level-k distribution (Barein- boim et al. (022). Level-k data can be viewed as samples from such a distribution, and with i..dd sampling, the empirical distribution converges to the true sampling distribution. Ordinary gameplay logs are level-1 data; data where variables are artificially fixed before sampling (as experiments) is level-2 data. In most settings, level-3 data does not exist due to the fundamental problem of causal inference- it is impossible to observe outcomes for the same effect variables across worlds (Hol- land 1986). In this work, we leverage the video game setting's 's ability to observe level-3 data across virtual worlds, i.e., multiple instances of a game run with shared initial conditions.
Models also align with this hierarchy. A causal DAG is level-2 model: generative model, it encodes the family of interventional distributions over the DAG's variables. A structural causal model (SCM) is a level-3 model: it additionally encodes the family of counter- factual distributions over the DAG's variables (Pearl 2009). Level-k data is sufficient to identify level-k models, but the causal hierarchy theorem states that level-k statements cannot, in general, be inferred from data below level k (Bareinboim et al. 2022).
when combined with
Thus, we follow the three-level hierarchy (Bareinboim et al. 2022): do do(X= and (L3) coun-
(L.1) observational ques-
tions about P(V); (L.2) interventional questions about P(V terfactual (parallel-world) questions about joint outcomes under conflicting interventions, e.g., (Yx=z,Yx=x)) Level-3 quantities encode "all-else-equal" comparisons across worlds.
E.2 SCMS AND RULES
An SCM M (U, V. F. P(U)) specifies exogenous factors U, endogenous variables V. structural assignments F, and distribution over U (Pearl 2009). In our context, F plays the role of executable rules; see Bongers et al. (2018) for SCMs as formal rule systems.
E.3 MARGINALIZATION OF CAUSAL DAGS
We implement Evans (016) approach to DAG marginalization. Given a causal DAG and set of nodes to marginalize out of the DAG, Evans (2016) creates a marginalized DAG with hyper-edges that represent the footprint of latent common causes. Directed edges encode parent-child relation- ships among observed variables as usual. Hyper-edges encompassing set of observed nodes, indi- cate that all nodes in the set share an unobserved exogenous influence. For example, hyper-edge touching nodes X, Y, Z represents the fact that there is some unobserved variable that acts as a com- mon cause X,Y, Z simultaneously. The mDAG maintains the causal interpretation, intervention model, and implications to conditional independence as the full DAG.
While mDAGs are convenient for reasoning about marginalized structures, we require fully explicit DAG representation for generating parallel-word and counterfactual graphs, as well as for using standard graph serialization. We therefore modify the algorithm such that, for each hyperedge,
17

---

Published as a conference paper at ICLR 2025
d-separating sets of shared ancestors that entail the hyperedge explicitly added back into the model. This implements the expansion the canonical mDAG by making the latent causes explicit expansion discussed in Evans (2166).
E.4 MULTIVERSE REASONING.
With multiverse counterfactual reasoning, we envision one world where observed outcomes oc- curred, and separate "parallel" worlds where hypothetical interventions lead outcomes that differ from the observed outcomes Shpitser & Pearl (201)). For example, consider the statement "Given there was no enemy and the scene generated that way, if an enemy were placed here, the scene would generate in this way". With this statement, we can envision two parallel worlds, one where there was no enemy and the scene generated that way, and one where there was an enemy and the scene generated this way. Level-3 data, therefore, is data tuple representing outcomes across par- allel worlds. The "fundamental problem of causal inference" (Holland, 1986) is that level-3 data unobservable- in real world settings, is impossible to observe potential outcomes for the same variable across parallel worlds. A key insight of our work that virtual world settings, the data exist by creating parallel instances of the same world with the same initial conditions.
E.5 COUNTERFACTUAL GRAPHS AND CONSISTENCY
Counterfactual graphs are based on parallel world graphs. The parallel world graph clones a causal DAG across parallel worlds and uses graph surgery to represent hypothetical conditions in certain worlds (Shpitser & Pearl 2012). Variables that are not descendants of the intervention share single node across worlds (consistency); variables affected by the intervention are duplicated and indexed by world. This graph encodes which quantities must remain identical across worlds and which may differ, providing a compact template for supervision and evaluation. The counterfactual graph collapses nodes that must be consistent across worlds into single nodes, creating graph that (unlike the parallel world graph) encodes conditional independence across parallel worlds. In this work, formalize the concept of game mechanics such that, for given mechanie,
generate set of parallel world graphs and counterfactual graphs that explicitly encode its structure and which variables should remain consistent while the mechanic is in play. These can be used in training alongside the level-3 gameplay data.
E.6 ESTIMATING COUNTERFACTUAN DISTRIBUTIONS
Sw P2 P(Bs-I, P(DB=1, D S) denote the distributions constrained by S1, S2. and Sz. We can estimate these distributions with repeated sampling consistent contrasts D,, D2. and Dz, then averaging over sampling distributions to obtain sampling distributions Pt. P.. and Ps.
In Section 3.1.3 we let P1
and P3
P:: Ejeo, Eos Eos_, W
P(Sw Bw Dw-1, V Sw o, Bw Dw Vw-o)
6 W-3
. P2: Ejeo,1 Ewdds
P(W, Bs=1, D8=1, Vs=1, Bs=o, Ds=o, Vs-o)
Ps: Ejeo,1 Em,a,ee, P(W.S, DB=1. VB=1, DB=0, VB=v)
In each case P; converges almost surely to P.. We describe each game mechanic in terms of:
F MECHANICS IMPLEMENTED IN MULTIVERSE MECHANICA v1.0
In this section, we describe the ground truth causal structure and mechanics Multiverse Mechanica.
level-2 interventional statements: A set of causal hypothetical statements of the form "Given preconditions W. all else equal, if X, then Y
A set of causal Markov kernels encompassing the relevant conditional probability distribu- tions in the causal DAG.
18

---

Published as a conference paper at ICLR 2025
player class
opponent class
opponent weapon class
player weapon class
controller (C)
player weapon light (M)
player element
opponent
opponent weapon light
opponent weapon range
player character
player has shield (S)
opponent has shield
player weapon element
opponent weapon element
player weapon
player action
player does block (B)
opponent action
player is immune attack
player attack
opponent attack score
opponent does black
opponent immune attack
player weapon range
opponent charactor
player hurt (D)
opponent is hurt
player etance
background
opponent stance
opponent weapon
Figure Full causal DAG turn in Multiverse Mechanica
level-3 parallel world statements: set of counterfactual statements of the form "Given preconditions W. all else equal, if X, then Y AND if X' then Y
A set of counterfactual probability expressions defining the induced counterfactual outcome probabilities.
Particularly, in the case the level-3 parallel world statements, we can enumerate counterfactual worlds in which make change and the target outcome variable(s) change, with guarantees consistency, with respect to the game mechanic.
We formulate the counterfactual cases by taking interventions change variables (bold), from the factual case (Case 0). Downstream changes are shown in italics.
Full Causal DAG Figure 6 shows the full causal DAG of turn in Multiverse Mechanica v1.0.
F.1
SHIELD MECHANIC
The warrior character can equip a shield they can use to block incoming attacks, if they have a free hand and perform the "defend" action.
F.1.1 LEVEL-2 INTERVENTIONAL DEFINITION
We can completely define this mechanic by set of contrasting statements:
a player has a small weapon, they may hold shield, otherwise they cannot.
If player has shield, they may block incoming attacks, otherwise they cannot.
If a player blocks an incoming attack, they avoid taking damage, otherwise they may take damage.
Involved variables:
plaaer action [idle, melee attack, defend]
19

---

Published as a conference paper at ICLR 2025
(a) Factual:
The warrior is equipped with a one- handed short-sword and shield, and blocks the ar- row with a shield.
(b) Counterfactual: The warrior is equipped with a two-handed long-sword, and no shield, and is unable to block the attack.
Figure 7: Example counterfactual contrast statement for the shield mechanic.
opponent weapon class: [small, large]
opponent has shield: (True, False]
opponent action: lidle, melee attack, defend]
opponent is hurt [True, False]
Latent variables:
opponent weapon is heavy [one-handed, two-handed]
F.1.2 CAUSAL MARKOV KERNELS
In model, block is of the actions. Thus, reason about the shield game mechanic in terms of the player/opponent action being sampled as block.
The causal Markov kernel for the opponent action variable encompasses the shield game mechanic:
Pooponeent action opponent has shield, opponent weapon).
To reason about the opponent blocking, consider the probability that the action variable is block:
Pooponeent action block opponent has shield, opponent weapon).
To reason about the opponent getting hurt, we consider the probability that the opponent gets hurt, given the opponent action and player action:
P(opponent is hurt opponent action, opponent has shield, opponent weapon).
An important detail take note of is that the causal DAG does have variable to represent the character's outcome (e.g. getting hit, blocking, dodging) only has a notion of final stance. However, the stance variable is currently unused after sampling; the battle video game handles the underlying outcome logic and renders the scene.
F.1.3 LEVEL-3 COUNTERFACTUAL PARALLEL WORLD STATEMENTS
We can rewrite the level-2 interventional statements as the following level-3 parallel world state- ments:
20

---

Published as a conference paper at ICLR 2025
1. Given character had heavy weapon and they did not have shield; if the character would have had a light weapon instead, then they could have equipped shield.
2. Given character had had shield and did not block; if the character had shield instead, then they could have blocked.
3. Given a character had a shield but did not block and took damage; the character had blocked instead, then they would have avoided taking damage from the attack.
F.1.4 COUNTERFACTUAN PROBABILITY EXPRESSIONS
We can formulate these natural- language statements in mathematical notation.
Notation.
Let:
WC weapon class,
HS has shield,
B = does block,
= is hurt.
Then we have:
P(HSWC=Ligh = True WC = Heavy, HS = False) > 0,
P(BHS=TTee True HS False, B = False) 0,
P(HB=True False HS True, B False, H True)
F.2 ELEMENTAL. IMMUNITY MECHANIC
Under the elemental immunity game mechanic, players may have elemental attributes; either fire or ice. Weapons may also have elemental attributes. the attacking player weapon element and opponent element the same (and non-none), the opponent is granted elemental immunity, and thus avoids taking damage from the attack, since they are already imbued with the element the attacker is using attack them.
The elemental attributes of characters are indicated by variations from their base appearance:
none: Characters appear in their standard appearances
fire: Characters appear with orange shading and/or red outline
ice: Characters appear with light blue shading and/or blue outline
The elemental attributes of weapons are similarly indicated by variations from their base appearance:
none:
Weapons appear in their standard appearances
fire: Weapons appear with orange shading and/or red outline. Melee weapon slash visual effects appear with red shading. Ranged weapon projectiles (e.g., arrows, spell attacks) and explosions appear with red shading.
ice: Weapons appear with light blue and/or blue outline. Melee weapon slash visual effects appear with blue shading. Ranged weapon projectiles and explosions appear with blue shading.
F.2. LEVEL-2 INTERVENTIONAL DEFINITION
We can completely define this mechanic by a set of contrasting statements:
If a player has an elemental attribute, they may wield a weapon with the same elemental attribute but they cannot wield one of opposing element in the fire-io ice dichotomy. For example, a player with a fire-elemental attribute may wield a fire-elemental weapon (or non-elemental weapon); they cannot wield an ice-elemental weapon.
21

---

Published as a conference paper at ICLR 2025
(a) Factual: The fire wizard is immune to the war- rior's 's melce attack with a fire-type short sword.
(b) Counterfactual: The wizard takes damage from the warrior's 's melee attack with an ice-type short sword.
Figure 8: Example counterfactual contrast statement for the elemental immunity mechanic.
If player does not have an elemental attribute, they may wield any elemental-type weapon; otherwise the rule above applies.
If a player hit by incoming attack from weapon with the same elemental attribute, they are immunc and thus avoid taking damage; otherwise they may take damage.
Involved variables:
player element [none, fire, ice]
player weapon element: [none, fire, ice]
opponent element : [none, fire, ice]
opponent is immune to attack: : [True, False]
opponent is hurt [True, False]
Latent variables: None.
F.2.2 CAUSAL MARKOV KERNELS
The element of the player and the opponent are both independent variables:
Pplaayr element),
Pooponnent element).
The causal Markov kernel for the player weapon elemental is as follows:
P (pllayr weapon element player element).
To reason about the opponent being immune to the attack, we consider the conditional probability given the opponent's element and player's weapon element:
P (opponeent is immune to attack player weapon element, opponent element).
To reason about the opponent getting hurt, consider the probability that the opponent gets hurt,
given the opponent being immune:
P(opponent is hurt
opponent is immune attack).
22

---

Published as a conference paper at ICLR 2025
F.2.3 LEVEL-3 COUNTERFACTUAL PARALLEL WORLD STATEMENTS
We can rewrite the level-2 interventional statements as the following level-3 parallel world state- ments:
1. Given a character had fire elemental attribute and they had fire elemental weapon; if the character would have had an ice elemental attribute instead, then they could have had an ice- or none-elemental weapon but not a fire elemental.
2. Given character had fire elemental attribute and they had fire elemental weapon; if the character would have had no elemental attribute instead, then they could have had any elemental non-elemental type weapon.
3. Given character had ice elemental attribute and was hit with attack from an ice elemental weapon, and thus was immune and did not take damage; if the character had fire elemental attribute instead, then they would have taken damage from the attack.
F.2.4 COUNTERFACTUAL PROBABILITY EXPRESSIONS
We can formulate these natural language statements in mathematical notation.
Notation.
Let:
PE = player element,
PW = player weapon element,
OE opponent element,
[ = is immune to attack,
H is hurt.
Then we have:
Statement 1.
P(PWEPrntee Fire
PE Fire, PWE E Fire) 0,
P(PW EPE=Ie Ice
PE Fire, PWE E Fire)
P(PWErE-ke None
PE = Fire, PW = Fire) > 0.
Statement 2.
P(PWEPE-Nme EpE-None = Fire PE = Fire, PW E = Fire) > 0,
P(PW EpE=Non: Ice PE Fire,
PWE Fire) > 0,
P(PW EPe-None
None PF Fire, PW Fire) 0.
Statement
P(HoE-mme True PWE E Ice, OF Ice) 1.
F.3 WEAPON RANGE MECHANIC
Under the weapon range game mechanic, weapon classes have range attributes; either melee or ranged. Each weapon class is either melee ranged type. For example, the light sword, the heavy sword, and the dagger are melee weapons; while the bow, the staff, and the throwing knife are ranged weapons. The weapon range attribute affects primarily visual elements of the battle scene:
1. Stance: character attacking with melee weapon will be depicted in the snapshot phys- ically swinging the melee weapon at the opponent's 's location, while a character attacking with ranged weapon will be depicted shooting/hhrowingcaasng from their position.
23

---

Published as a conference paper at ICLR 2025
2. Scene: character attacking with melee weapon will be shown in the game scene (video) first approaching the opponent's position before physically swinging the melee weapon, while a character attacking with ranged weapon will be depicted shooting/hrooing/- casting from their position and the emitted projectile will be shown traveling towards the opponent from left to right (possibly paraholic trajectory, if affected by gravity, e.g- the arrow shot from the bow).
(a) Factual: The assassin performs a melee attack on a wizard with sword.
The assassin performs a ranged attack on a wizard with throwing knife.
(b) Counterfactual:
Figure 9: Example counterfactual contrast statement for the weapon range mechanic.
F.3.1 LEVEL-2 INTERVENTIONAL DEFINITION
We can completely define this mechanic by set of contrasting statements:
If a weapon is one of the weapon classes light sword, heavy sword, or dagger, it melee weapon; otherwise if it is bow, staff, or throwing knife it is ranged weapon.
If a player wields a melee weapon, they will be depicted the snapshot as performing a melee attack at the opponent's location; otherwise they will be depicted as performing ranged attack from their own location.
If player wields melee weapon, they will be shown in the game scene (video) first approaching the opponent's position before physically swinging the melee weapon; other- wise they will be depicted thooting//rrowingccsing from their position and the emitted projectile will be shown traveling towards the opponent from left to right.
Involved variables:
player weapon class: [light sword, heavy sword, dagger, bow, staff, throwing knife]
player weapon range : [melee, ranged]
player stance pixels the rendered image snapshot
scene pixels in the rendered gameplay video
Latent variables: None.
F.3.2
CAUSAL MARKOV KERNELS
The causal Markov kernel for the player weapon range is as follows:
P (plaaer weapon range player weapon class).
24

---

Published as a conference paper at ICLR 2025
To reason about the player stance rendered in the image snapshot, consider the conditional prob- ability given the player weapon and player weapon range:
P( P(stance| player weapon, player weapon range).
Similarly, reason about the scene rendered in the gameplay video, consider the conditional probability given the player weapon, player weapon range, and player stance:
P(scene player weapon, player weapon range, stance).
F.3.3 LEVEL-3 COUNTERFACTUAL PARALLEL WORLD STATEMENTS
We can rewrite the level-2 interventional statements the following level-3 parallel world state- ments:
1. Given character had a light sword and they had melee weapon; if the character would have had a bow instead, then they would have had a ranged weapon.
2. Given a character had a melee weapon and they were depicted in the snapshot as performing a melee attack at the opponent's 's location; if the character would have had ranged weapon instead, then they would have been depicted as performing ranged attack from their own location.
3. Given character had a melee weapon and they were shown in the game scene video first approaching the opponent's 's position before physically swinging the melee weapon; the character would have had ranged weapon instead, then they would have been depicted shooting/throoinn/cssting from their position and the emitted projectile would have been shown traveling towards the opponent from left to right.
F.3.4 COUNTERFACTUAL PROBABILITY EXPRESSIONS
We can formulate these natural language statements in mathematical notation.
Notation.
Let:
PWC = player weapon class,
PW player weapon range,
PS player stance,
S scene.
Then we have:
Statement
PPW RpWC=bow ranged| PWC C light sword, PW melee)
Statement 2.
p PS Remanged depicted performing ranged attack
PWR melce, PS depicted performing melce attack = 1,
(I)
P PSPw Reranged depicted performing melee attack
PW R = melee, PS = depicted performing melee attack) = 0.
(2)
25

---

Published as a conference paper at ICLR 2025
Statement 3.
P (SS ft=ranged shown performing ranged attack
PWR - melee, PS shown approaching the opponent and performing melee attack)
1,
(3)
P P(Srw Rtranged shown performing melee attack
P = melee, PS shown approaching the opponent and performing melee attack
(4)
F.4 SPELL-CASTING MECHANIC
Under the spell-casting mechanic, the wizard character may perform of five spells:
1. Spawn Magic Projectile Spell to Perform Ranged Attack
2. Summon Cloud Platform Spell to Dodge Attack
3. Self-Transform Spell to Increase Melee Strength
4. Opponent Transform Spell to Lower Enemy Defense
5. Levitation Spell to Disarm Opponent
None of these spell actions can be mitigated by the enemy. Even in the case of the self-transform spell, the sheer size of the transformed wizard (into a golem) renders any block action by the oppo- nent useless. Each spell action choice affects the stance and scene variables in the rendered game.
F.4.1 LEVEL-2 INTERVENTIONAL DEFINITION.
All spells share the same base level-2 contrasting statements:
If the player character is a wizard, they may wield a magical staff, otherwise they cannot.
If the character is equipped with magical staff, they may cast spells, otherwise they can- not.
If the character casts spell, they gain particular offensive defensive benefit help them the battle, otherwise they do not.
F,4.2 CAUSAL MARKOV KERNELS
Notation.
Let:
PC player character
PW player weapon
PA player action
PF player form
PS = player stance
OC opponent character
OW opponent weapon
0A4 opponent action (e.g., defend)
OF opponent form
OS opponent stance
B = opponent successfully blocks (semantic variable, only true if the defend action actu- ally succeeds)
opponent dodges
26

---

Published as a conference paper at ICLR 2025
H opponent is hurt
Stance and Scene the rendered variables
Latent variables:
0IA = opponent incoming attack indicator
Shared enabling kernels.
P(PW = staff PC = wizard) >
P(PA E {spell actions} PW # staff) 0
P(PA] PW staff, state) exogenous (agent policy)
Shared state rendering kernels.
PS, PW,
P(Stance PF, PF, 0S,
P(Scene Stance, PF, PS, OF, OS, PA,
Shared interaction/outtcome kernels.
P(B PFÇOA)
P(D PS,OIA)
P(H B, PA)
F.4.3 LEVEL-3 COUNTERFACTUAL PARALLEL WORLD STATEMENTS.
All spells also share a common set of counterfactual statements:
1. Given character was not a wizard and therefore could not wield magical staff; if the character had been wizard instead, then they could have wielded magical staff.
2. Given character was not equipped with magical staff and therefore could not cast spells; if the character had been equipped with staff instead, then they could have cast spells.
3. Given character did not cast spell and therefore received no benefit in the battle; if the character had cast spell instead, then they would have gained the corresponding offensive or defensive benefit.
F.4.4 COUNTERFACTUAL PROBABILITY EXPRESSIONS (SHARED).
Not wizard > Wizard = Staff access
P(PWPC-wiadd staff PC wizard, PW staff) 0.
No staff + Staff Spell casting possible
P(PAAW=san (spell actions] PW * staff, PA & {spell actions)) 0.
F.5 SPAWN MAGIC PROJECTILE SPELL TO PERFORM RANGED ATTACK
The wizard can cast spell to conjure and launch magical projectile directly at the opponent. This action is a ranged attack, and its visual and mechanical dynamics follow the same principles as other ranged weapons described in Section
27

---

Published as a conference paper at ICLR 2025
F.5.1 LEVEL-2 INTERVENTIONAL DEFINITION.
If the wizard casts the spawn projectile spell, a magical projectile is launched towards the opponent (i.c., OIA True), otherwise no projectile is spawned (OIA = False).
If magical projectile strikes the opponent and is not successfully blocked or dodged, the opponent is hurt, otherwise they are not.
Involved variables:
PA player action
OA opponent action (e.g., defend, dodge)
PS player stance
opponent successfully blocks
opponent dodges
11 opponent is hurt
Latent variables:
OIA = opponent incoming attack indicator
F.5.2
CAUSAL. MARKOV KERNELS.
P(OTA PA) P(B PF,OA)
P(D PS,O1A)
P(H D,D,PA)
F.5.3 LEVEL-3 COUNTERFACTUAL PARALLEL WORLD STATEMENTS.
1. Given the wizard idled and there was no incoming attack; if the wizard had cast the spawn projectile spell instead, then the opponent would have faced an incoming attack (O1A True).
2. Given the wizard cast the spawn projectile spell and the projectile hit (i.e., successful block or dodge) and the opponent was hurt; if the opponent had successfully blocked or dodged instead, then they would not have been hurt.
F.5.4 COUNTERFACTUAL PROBABILITY EXPRESSIONS.
Idle + Projectile spell = Incoming attack present
P(OLAPA=poojecle spell True PA= idle, O1A False) 1.
Projectile hits + Successful block + Not hurr
P False PA = projectile spell, 01A = True, B = False, H = True) = 1.
Projectile hits + Successful dodge = Not hurt
P(Hp-Tme False P.A projectile spell, OLA True, D False, True) =
F.5.5 SUMMON CLOUD PLATFORM SPELL TO DODGE ATTACK
The wizard has a spell they can use to summon a levitating cloud platform and use it to raise them- selves up above the battlefield to dodge incoming enemy attacks.
28

---

Published as a conference paper at ICLR 2025
level-2 Interventional Definition.
If the wizard casts the cloud platform spell, platform is summoned; the wizard moves onto and levitated upwards above the battlefield, otherwise they remain on the ground.
If the wizard in elevated position platform during incoming attack, they cessfully dodge and avoid damage, otherwise if they are on the ground they remain vulner- able and may take damage.
Involved variables:
PA = player action
OA = opponent action
OS opponent stance (grounded, elevated)
opponent is hurt
Latent variables: None.
Causal Markoy Kernels.
P(OS PA) P(D OS,OOA) P(H D).
level-3 Counterfactual Parallel World Statements.
1. Given the wizard performed the idle action and remained the ground; if the wizard had cast the cloud platform spell instead, then they would have been elevated.
2. Given the wizard remained on the ground during an incoming attack and was hurt; the wizard had cast the cloud platform spell instead, then they would have been elevated and dodged the attack, and thus not been hurt.
Counterfactual Probability Expressions.
Idle + Cloud Platform Elevated stance
P(OSPA- A-cloud platform elevated PA idle, OS grounded) 1.
Grounded, hurt during incoming allack Cloud Platform + Dodge, not huri
PHooscolaase False PA = idle, OS grounded, O1A = True, H = True) = 1.
F.5.6 SELF-TRANSFORM SPELL TO INCREASE MELEE STRENGTH
The wizard can cast spell to transform themselves into large golem. In this form, their melee attacks are enormously strengthened and cannot blocked by opponents.
level-2 Interventional Definition.
If the wizard casts the self-transform spell, they transform into golem, otherwise they remain in their normal form.
If the wizard is transformed into golem, their melee attack cannot be blocked due to sheer size and strength, otherwise can.
If the wizard's 's golem-form melee attack is not successfully blocked, the opponent will be hurt, otherwise they will be.
Involved variables:
29

---

Published as a conference paper at ICLR 2025
(a) Factual: The wizard struck by the archer's 's arrow.
The wizard summons magical cloud platform to gain an elevated position and dodge the archer's 's arrow.
(b) Counterfactual:
Figure 10: Example counterfactual contrast statement for the summon cloud platform spell me- chanic.
PA player action
OA opponent action (e.g., defend)
= opponent successfully blocks
opponent is hurt
Latent variables:
PF player form (normal, golem)
Causal Markoy Kernels.
P(PF PA) P(B PF, P(H B,
level-3 Counterfactual Parallel World Statements.
Given the wizard performed the idle action and their form remained unchanged: if the wizard had cast the self-transe tiff-traasfomm spell instead, then they would have transformed into mighty golem.
2. Given the wizard cast a projectile spell and it blocked by the opponent; if the wizard had cast the self- oofftrransfomm spell instead, then they would have transformed into golem and the melee attack would not have been blocked by the opponent.
3. Given the wizard cast a projectile spell and it was blocked by the opponent and thus they were not hurt; if the wizard had cast self-transform spell instead, then they would have transformed into golem and the melee attack would have bypassed the block and hurt the opponent.
Counterfactual Probability Expressions.
Idle + Self-fransfomm- = Gcolemform
P(PFrA trnsform golem PA idle, PF normal) 1.
30

---

Published as a conference paper at ICLR 2025
Magic projectile attack blocked Self- Teef-Transfomm Block attempt failed
P(Bpr-golem True PA magic projectile attack, PF normal, ОA = defend, B True) 0.
Magic projectile attack blocked, hurt Self-Transform Hurt despite block action
P(HPF=golens True PA magic projectile attack, PF normal, OA defend, False) 1.
(a) Factual: The wizard casts magic projectile spell but the warrior blocks it.
(b) Counterfactual: The wizard casts spell to trans- form itself into large golem perform powerful melee attack.
Figure 11: Example counterfactual contrast statement for the self-transform spell mechanic.
F.5.7 OPPONENT TRANSFORM SPELL LOWER ENEMY DEFENSE
The wizard can cast spell to transform an opponent into a weak or harmless creature (i.e., a snail), disarming them and preventing them from defending against incoming attacks.
level-2 Interventional Definition.
If the wizard casts no spell, the opponent retains their normal form and can block or defend as usual.
If the wizard casts the opponent transform spell, the opponent is transformed (e.g. into snail), otherwise they remain in their normal form.
If the opponent transformed, they are disarmed and cannot block; otherwise they may block.
Involved variables:
PA = player action
OA opponent action
B opponent successfully blocks
opponent is hurt
Latent variables:
OF opponent form (normal, transformed)
31

---

Published as a conference paper at ICLR 2025
Causal Markov Kernels.
P(OF) PA), P(H OF,, B).
level-3 Counterfactual Parallel World Statements.
1. Given the wizard cast projectile spell and the opponent remained in their normal form; if the wizard had cast the opponent transform spell instead, then the opponent would have been transformed into snail.
2. Given the wizard cast a projectile spell and it was blocked by the opponent, and thus the opponent was not hurt; the wizard had cast the opponent transform spell instead, then the opponent would have been unable to block and would have been hurt by a follow-up attack.
Counterfactual Probability Expressions.
Idle projectile spell Opponent Transform = Opponent form changes
P(OFr A Amoppument transfomm transformed
PA F opponent transform, OF = =normal) =1
Projectile spell blocked, not hurt - Opponent Transform = Block disabled (opponent disarmed)
P(Borstantomed True PA magic projectile attack, OF normal, True) 0.
(a) Factual: The wizard magic projectile spell warrior blocks
(b) Counterfactual: The wizard spell trans- form the warrior into a snail disarm them.
Figure 12:
Example counterfactual contrast statement for the opponent transform spell mechanic.
F.5.8 LEVITATION SPELL TO DISARM OPPONENT
The wizard can cast spell that lifts the opponent into the air, leaving them unable to defend or block effectively, and rendering them disarmed.
level-2 Interventional Definition.
If the wizard casts no spell, the opponent remains grounded and may block normally.
If the wizard casts the levitation spell, the opponent is lifted into the air, otherwise they remain grounded.
32

---

Published as a conference paper at ICLR 2025
If the opponent is levitated, they cannot block and are effectively disarmed; otherwise they may block.
Involved variables:
PA = player action
OA opponent action
OS opponent stance (grounded, levitating)
opponent successfully blocks
H opponent is hurt
Latent variables: None.
Causal Markov Kernels.
P(OS | PA), P(H OS, B)
level-3 Counterfactual Parallel World Statements.
1. Given the wizard cast projectile spell and the opponent remained grounded; if the wizard had cast the levitation spell instead, then the opponent would have been levitated.
2. Given the wizard cast a projectile spell and was blocked by the opponent, and thus the opponent was not hurt; if the wizard had cast the levitation spell instead, then the opponent would have been unable to block and left vulnerable.
Counterfactual Probability Expressions.
Idle or projectile spell + Levitation = Opponent stance changes
P(OSPA=leviatio levitating
PA A levitation, OS = grounded) = 1.
Projectile spell blocked, not hurt - Levitation Block disabled (opponent disarmed)
P(Bos=tevitatin True PA = magic projectile attack, OS = grounded, True) = 0.
G GAME DESIGN DECISIONS
To ensure that the datasets produced by our simulation respect the causal assumptions of model, we designed the game architecture with consistency guarantees as a primary objective. This ap- pendix documents how these guarantees were realized practice, focusing on two complementary aspects: (i) the modular, system-basee based design of game mechanics that mirrors the structure of the causal model, and (ii) the temporal alignment of captured frames ensure semantic consistency in dynamic interactions.
G.1 BRIDGING GAME MECHANICS AND CAUSAL MECHANISMS WITH SYSTEM-BARE DESIGN
At the foundation of our data generation pipeline lies design principle: game mechanics must be implemented in a way that respects and preserves the structure of the causal model they in- stantiate. To achieve this, adopted fully modular ture (Nystrom 2014; Gregory 2018), which enforces locality causal mechanisms and supports reproducibility across
(ECS) architec-
33

---

Published as a conference paper at ICLR 2025
(a) Factual: The wizard casts magic projectile spell hut the warrior blocks it.
(b) Counterfactual: The wizard casts levitation spell to lift the warrior off the ground, disarming them.
Figure 13: Example counterfactual contrast statement for the levitation spell mechanic.
G.1.1 ENTITIES, COMPONENTS, AND SYSTEMS AS CAUSAL UNITS.
In our implementation, entities represent the units of analysis (e.g., characters, projectiles, plat- forms), components represent their attributes and state (e.g., position, animation phase, shield pres- ence), and systems encapsulate the transformation rules that govern state evolution (e.g., combat resolution, kinematics, physics, or screenshot scheduling). This separation guarantees that cach causal mechanism is expressed locally, without entanglement with unrelated processes.
Each system implements a distinet causal mechanism: for example, the GameCombattSyster maps action states of attacker and defender into outcomes such hit, block, immune, while the SameehhyscssSye em governs the motion of projectiles according to deterministic physical rules. Systems generally designed operate frame-by-frang using component data inputs, but they may also maintain local state when required (for example, the GameAn iatt: t. onSyst stytte em manages per-entity priority queue of animation requests). This design ensures that each causal transformation encapsulated and modular, while still supporting the persistent state needed for realistic simulation.
G.1.2 ALIGNMENT WITH CAUSAL GRAPH STRUCTURE
The ECS architecture chosen deliberately to reflect the modular structure of causal model. Nodes in the causal graph correspond to entity attributes (e.g.. weapon class, stance, elemental immunity), while edges correspond the update dependencies realized through system logic.
For example, the transition of defender into a blocking or hurt state depends on multiple up- stream components: the presence of shield component, the character's action selection com- ponent (indicating whether the chosen action is to block), and downstream trigger flags such as needs. to.block or is_hurt. The shield and action selection components establish the poten- tial for blocking, while the trigger flags are set based on situational context (e.g., proximity of an incoming attack). Only when both preconditions and triggers align does the GameCombat em update the defender's 's state to blocking; otherwise, the state transitions to hurt.
This design directly encodes the causal mechanism:
shield component + action selection - tneeds.to is.hurt]
outcome (block or hurt).
By structuring dependencies in this way, the system preserves the logic of the causal graph within the mechanics the game engine. This mapping ensures that system update rules correspond closely to the assignment functions of the causal model.
34

---

Published as a conference paper at ICLR 2025
G.1.3 LOCALITY AND MODULARITY FOR CONSISTENCY
By localizing mechanics in dedicated systems, the architecture prevents hidden confound- ing across game features. For instance, animation timing is managed exclusively by the famerriimttonnSysters GamePhysicsSystem, This guarantees that modifying one mechanism (e.g., projectile grav- ity) does not inadvertently alter another (e.g., collision detection or blocking). Such modularity enforces a form of "causal isolation," allowing dataset generation to reflect the true structure of the designed model.
while collision and trajectory updates are confined to the
G.1.4 REPRODUCIBILITY AND CONNECTION TO INTERVENTIONS
The ECS structure also guarantees reproducibility: since each system applies deterministic update rules to the current component state, identical initial conditions yield identical traces. Importantly, the systems themselves do not support interventions in the sense of directly overriding assignment functions during simulation. Instead, interventions are handled at the model level: sampled val- ues from the causal model are passed into the simulation as inputs that determine which branches executed (e.g., sampled values specifying whether character does block). The behavior tree then orchestrates the scene by triggering the appropriate system up- dates, while each system executes its assignment function deterministically given the requested state changes. In this way, the game engine acts as a faithful executor of causal mechanisms, while the intervention logic is confined to the sampling layer above.
of the Game Beeehvviorrree
Through this design, the simulation environment operates a direct computational analogue of the causal model, where each mechanism is encapsulated in a corresponding system. This guaran- tees that generated training data inherits the same modularity and independence properties as the underlying causal graph, thereby supporting consistency-guaratteed counterfactual analysis.
G.2 IMPACT FRAMES: DEFINING SEMANTIC CONSISTENCY IN DYNAMIC CAUSAL MODEL TRACES VIA Point of Maximum Action CONCEPT
While the system-based architecture guarantees local causal consistency at the level of game logic, temporal alignment must also be addressed to preserve counterfactual consistency in dynamic inter- actions. To this end, our system generates gameplay clips of contrasting player turns, each designed to include canonical impact frame: the instant attack connects, shield block occurs, projec- tile visibly misses. Ensuring that these per-turn impact frames correspond to semantically equivalent points in the causal process is critical; otherwise, contrasts risk reflecting phase misalignment rather than true causal differences. We therefore formalize alignment using the Point of Maximun Action (PoMA) principle, which anchors impact frames the most visually and mechanically expressive phase of the interaction.
G.2.1 CONCEPTUAL FRAMING: TEMPORAL ALIGNMENT FOR COUNTERFACTUAL COMPARISONS
In static structural causal models (SCMs), counterfactuals are evaluated at a single time index, and semantic alignment across factual and counterfactual worlds is immediate. In dynamic SCMs, the meaning of an event depends on when it occurs relative to the unfolding process. For example, melee strike might connect later than a projectile impact due to differences action duration. frames are extracted at fixed indices, we risk capturing non-equivalent phases of these interactions (e.g., an attack wind-up one turn versus a point of contact in another). we extract training artifacts (e.g., impact frames) at fixed indices, we risk capturing non-equivalent phases of these interactions (such as a wind-up in one run versus a point contact in another). This undermines the validity of counterfactual comparisons by introducing differences that are artifacts of temporal phasing rather than consequences of the intervention.
We therefore treat each player turn temporal causal trace and align counterfactual observa- tions to the most informative temporal locus of the relevant event class. We formalize this with the principle of the Point of Maximum Action (PoMA).
35

---

Published as a conference paper at ICLR 2025
G.2.2 POINT OF MAXIMUM ACTION (PoMA).
Let A(Sp) R20 score how action-intense the counterfactual state Sp with respect the target event class (e.g., impact, block). The PoMA alignment selects
'PMMAA = arg max A(Sy). N'TT
PoMA frames are then extracted at fToMA, aligning the impact frame to the point of maximum expressivity. This guarantees that contrasts correspond to the same semantic phase of the interaction, regardless of variation in action duration.
G.2.3 FOUR ALIGNMENT METHODS: BRIEF SUMMARY WITH PROS AND CONS
We summarize four practical approaches for temporal alignment of dynamic counterfactuals. Each provides a distinct trade-off between simplicity, robustness, and semantic fidelity.
1) Constant Time Interval. Rule. Evaluate the counterfactual variable the same nominal time as the factual: 1' = f. Pros. Conceptually simple; trivial to implement; deterministic. Cons. Fails when action durations differ; risks capturing non-equivalent phases (e.g., mid-swing impact); unsuitable for dynamic interactions where event timing adapts to interventions.
Rule. Evaluate once the counterfactual dynamics have reached steady state or absorbing condition (e.g.. || S < or a domain predicate holds). Pros. Appropriate for tasks where long-run properties matter; robust transient phasing differences; aligmment invariant to small shifts in sequence length. Cons. Inapplicable to inherently transient events (c.g., impacts); some episodes may not converge; equilibrium may erase the very distinctions needed to analyze acute causal effects.
2) Equilibrium-Bast
3) Point Maximum Action (PoMA). Rule, Align to the counterfactual time of peak action intensity for the event class:
COOA = arg A(S)).
Pros. Directly targets the most salient phase of the interaction; robust to differences in sequence length; naturally accommodates variable-duration actions by abstracting to their peak. Cons. Re- quires well-defined intensity score A; be non-trivial for abstract multi-agent interactions; may require smoothing when peaks are brief or noisy.
4) Semantic Consistency. Rule. Align by maximizing semantic similarity between factual and counterfactual states. Pros. General and flexible in principle; useful settings where semantic descriptors are available. Cons. Not used in implementation. It requires an additional similarity metric and embedding design, which introduces complexity and potential bias.
In practice, adopt the PoMA approach, extending it with event-specific scoring functions and event-defined windows to handle variable-duration interactions. Constant Time and Equilibrium serve primarily conceptual baselines, while semantic similarity was considered but not imple- mented.
G.2.4 IMPLEMENTATION IN OUR GAME: EVENT-BASER WINDOWS AND WEIGHTED SCORING
To instantiate these principles in a reproducible data pipeline, our engine implements an event-driven GameSCrrenshotSyyseen that schedules captures at semantically aligned moments:
1. Event Detection. The simulation raises structured events for interactions of interest (c.g., me melee.Be lele-iimpact petojeccileliie shield.block Each event associated with the participating entities and their current states.
2. Event-Based Scoring Windows. For each interaction type, define start event and a stop event that bound scoring window (e.g., swing.start - impact, or projectile.com + collision These windows managed directly in the be- havior tree, ensuring that scoring only occurs during the semantically relevant phase of the interaction.
36

---

Published as a conference paper at ICLR 2025
3. Weighted Scoring. Within each window, frames scored according to event-specific weights. For example, projectile impact events may be given higher weight than projectile flight, and shield contact may be prioritized over shield raise. The capture frame is then chosen as
E arg max A(S)),
E'Ewindow
with deterministic tie- tie-brekking for reproducibility.
By anchoring artifact capture event-defined scoring windows and applying weighted intensity scoring, our pipeline produces semantically aligned visual data across simulations, despite natu- ral variability in action durations. This guarantees that counterfactual comparisons reflect genuine causal differences, rather than artifacts of capturing frames at arbitrary, non-equivalent time indices.
G.3 SUMMARY
Taken together, these design choices preserve consistency both structural and temporal levels. The ECS architecture ensures local mechanics map cleanly to modular causal mechanisms, while the sereenshot system anchors each player turn contrast canonical impact frame selected via PoMA. By aligning contrasts at the most expressive phase of interaction, the system guarantees that observed differences reflect genuine causal effects rather than timing artifacts, producing impact frames that are both causally and semantically consistent.
H SPECIFICATIONS OF GENERATED VIDEO CLIPS
Generated video clips have following technical specifications:
H.1 RESOLUTION AND FORMAT
All video clips are rendered at a resolution of 512 x 512 pixels in a square aspect ratio. Videos are encoded in MP4 format using the H.264 codec with yuv420p pixel format ensure broad compati- bility across video players and analysis frameworks.
H.2 FRAME RATE, DURATION, AND TIMING
Videos are captured at 50 frames per second (FPS) using fixed-interval delta time method to ensure consistent temporal sampling across all generated clips. This approach decouples game simulation time from wall-clock time, enabling reproducible frame timing essential for dataset generation and comparative analysis.
Video clip duration is variable and event-driven, spanning the complete battle sequence from initial- ization completion. Clip length determined by the termination of both player behavior trees, typically ranging from several seconds to longer sequences depending the complexity of actions performed (c.g., melce attacks, spell casting, projectile interactions, defensive maneuvers).
H.3 DATASET ORGANIZATION
The system generates consistent contrasts, enabling direct comparison between observed battle out- comes and alternative scenarios under modified conditions. Each video file follows the naming con- vention baxxxexxxxxxx battle identifier. Accompanying JSON metadata files provide technical details including encoding parameters, frame timing information, and battle configuration data.
mp4, where XXXXXX represents a zero pro-ppdded 6-digit
H.4 IMPLEMENTATION
Video generation utilizes the imageio library with PyAV backend for efficient encoding. The ren- dering pipeline captures frame sequences from the game's 's screenshot buffer system, which also maintains temporal consistency in impact frames through the behavior tree execution framework.
37

---

Published as a conference paper at ICLR 2025
H.5
VARIABLES GENERATED
In addition to clips, each generated example includes:
Controller inputs C
Full or partial state X,
Mechanie-specific CL-DAG G.M
Set of mechanic-specifie parallel world DAGs
Set of mechanic-speciffc counterfactual DAGs
Seed identifiers
I SUGGESTED METRICS
Task
Description
Metric
Fully observed me- chanic inference
Infer P; using clips, game state variables, and controller inputs
DKu P. Pl.)
Partial (canonical) mechanic inference
Infer P₁ using clips and controller inputs, assuming game state is unobserved
DKL (1..P.
Generate consistent contrasts
Generate multiple consistent contrast examples for each d, associated with the mechanic, and validate that they visually accurate and consistent
Human
VLM
valida-
tion
Counterfactual ab- duction
Generate {ux may "X=z)) } from game, obtain úx E[Vx==' Vx=z uX=z] using the model, then com- pare 0x=x² UX=='
Human
VLM valida-
tion
Table 5: Tasks that can be used to evaluate mechanic learning with Multiverse Mechanica.
3 summarrzes the four tasks for evaluating mechanic-leamingg with Multiverse Mechanica. We expand here how each metric is operationalized and its relation to prior work.
1.1 MECHANIC INFERENCE.
The fully- and partially- observed inference tasks require estimating the distributions P. implied by mechanic-specifie constraints M and comparing them against the ground truth distributions P.. We adopt KL-divergence the primary quantitative measure, explicitly reporting DKL(P: P.) for
each mechanic. This aligns with classical practice in distributional evaluation, where divergence to the ground truth provides a direct measure of whether the model has captured the stochastic relations induced by the mechanic.
I.2 CONSISTENCY IN CONTRAST GENERATION.
The contrast-generation and counterfactual-abdiice tasks require evaluating whether pairs of clips (Ux=z, UX==() ) are consistent, i.e. differing only in outcomes attributable the intervention variable X while holding non-descendant factors fixed. Prior work has typically relied on human evaluation: for example, Gingerson al. (2024) collected large-scale human judgments of whether generated gameplay videos adhered to intended mechanics. Recent work has investigated whether VLMs can serve as automated evaluators model generations. Hendriksen et al. (2025) show that pre- trained VLMs can be adapted for evaluating world model rollouts, e,g. scoring whether predicted target outcomes. However, to our knowledge, no prior work
videos match textual descriptions has specifically used VLMs to evaluate consistency across parallel-world contrasts, defined by causal counterfactual principles. This remains an open direction, and Multiverse Mechanica pro- vides level-3 world data where such evaluations be systematically explored.
38

---

Published as a conference paper at ICLR 2025
Work by Monteiro et al. (2023) introduced composition metrics that effectively evaluate consistency in image-based counterfactuals by checking whether attributes remain unchanged under controlled edits. These metrics capture aspects of counterfactual faithfulness that parallel our notion of consis- tency. However, they have not yet been extended to video data, nor applied to generative modeling of game mechanics. We view Multiverse Mechanica platform to develop such extensions, allowing both human and VLM-based evaluation methods to be compared side-by-side.
1.3 SUMMARY.
In short, our metrics combine classical distributional divergences for inference tasks with human- or VLM-based consistency checks for contrastive generation tasks. This dual approach ensures that models are evaluated not only on reproducing distributions of state variables but also on capturing the causal consistency of gameplay dynamics under controlled interventions.
J THEORETICAL AND IMPLEMENTATION DETAILS FOR PROOF-OF-CONCEPT
J.1 BACKGROUND ON DIFFUSION MODELS AND REVERSE-SAMPLLNG
Denoising diffusion probabilistic models (DDPMs) define forward Markov chain that gradually adds Gaussian noise to an image and a learned reverse process that denoises it step by step. Given data sample To ((x)), the forward process produces 22 by directly adding noise to the data: g( N (x 1 (1 () for t 1,..., T. Here 0 < Oz, < are predefined vari- ances. In the reverse generation, one starts from pure noise TT ~ N(0 f) and iteratively denoises to To using a model 60 trained to predict the injected noise. At each step, the model predicts e such that 7 can be estimated by removing noise: e.g., 24-1 Vo. (xr (1 -aa) (with additional variance for stochastic sampling). In practice, one also use the continuous-tinn formulation and solve a reverse stochastic differential equation or its deterministic counterpart (as in DDIM), yield- ing a mapping from an initial noise u directly an image x G(u. c). Importantly, this exogenous noise u acts as the stochastic latent that accounts for random variation in generated images. Using a deterministic sampler (e.g..setting 7) = 0 in DDIM), one obtains a one-to-one mapping between and the output x, and can invert given image to its corresponding for particular conditioning c.
We will leverage this invertibility extract the latent noise u, from the original image x and the latent noise from the counterfactual image Ler. where denotes the factual image generated under the original conditioning, Her represents the counterfactual image generated under modified conditioning, Un is the inverted latent noise corresponding the factual image z, and us is the inverted latent noise corresponding to the counterfactual image Tet-
J.2 BACKGROUND ON CAUSAL COUNTERFACTUALS IN IMAGE GENERATION
In causal terms, e is a cause (exg.,textual description or a set of discrete random variables) and u is an unobserved exogenous variable accounting for randomness. A counterfactual image aims to answer: "What would the image look like if we change e from CA to CB. while keeping all other latent factors the same?" The classical procedure for generating such counterfactuals the three-step abduction- petion-prediction (I) Abduction: infer the exogenous noise u. that produced the factual image x under Ca (this Ua captures the instance-specific variations of x); (2) Action: intervene by setting the prompt to cg (while keeping u, fixed); (3) Prediction: generate the new image 전5t fo (ua, CB). This procedure, if the model perfectly captures the true causal mechanism. would change only the aspects of the image directly affected by and leave other details intact (satisfying causal consistency that no nonddescendant features of e change). In practice, directly using un with a new prompt ep can produce reasonable edited image, but it may fail produce artifacts if eg demands alterations that conflict with the original latent factors.
view the generative model as structural causal model == fo uu c). where
By contrast, many non-SCM image editing approaches do not explicitly enforce the same latent noise. For example, one might simply prompt the model with cg and generate a new sample (dif- ferent u), apply heuristics like latent interpolation. attention refocusing, mask-based noising of only certain regions. These approaches can produce plausible results, but often lack guarantees that only the intended changes theur-the model might inadvertently change unrelated details because
39

---

Published as a conference paper at ICLR 2025
the random draw other generation conditions differ. Our goal is to incorporate causal prin- ciples into the diffusion editing process to maximize counterfactual faithfulness (only c-dependent changes) while still allowing the model flexibility to implement the edit realistically.
J.3 CONTRASTIVE TRAINING VIA ALIGNMENT LOSSES
In this section, we provide the necessary background to help understand our method.
J.4 NOTATION
We first define the notation used in our counterfactual editing framework. Let denote the original (factual) image, and let Tef denote the counterfactual image aim to generate (the edited image after an intervention). We model image generation via diffusion model as r = GG,, c). where is initial exogenous noise (drawn from Gaussian prior, typically u ~ (N( [)) and is the conditioning (in our case, a text prompt). We use CA for the original prompt and св for the counterfactual prompt. Using inversion), we can obtain Va as the noise that generates x under CA. and similarly us as the noise corresponding Far under cg- We denote by Ia the (noisy) latent image at diffusion timestep when evolving toward x (with zi = x and zT Ua for the forward noising process), and likewise Fer, for the counterfactual trajectory. The diffusion model's denoiser is denoted eø ( C, f), which predicts the added noise at step t for latent Ia and conditioning c. For brevity, we write c(T,, c, when A is clear from context. Finally, L1, L2, Luxi, and Cuub will denote different loss terms introduced below.
inversion technique (e.g., reverse ODE deterministic DDIM
J.5 METHOD L1 - CONSISTENCY ALIGNMENT
Our first new loss function enforces consistency alignment between the factual and counterfactual generations. We obtain the noise ua and u, corresponding to x and der respectively (via the inversion process described above). The consistency alignment loss is then defined as,
L = llu. - 11.12,
H. T (I.,
(5)
where HT-0 represents the inversion function that maps from image space back to noise space timestep T.
Given the diffusion model fo(u,e), we have:
x fo (u CA) Jet folus, CB)
(6)
The inversion takes the image back to the noise, which yields:
Ua H Ca, ca) up H ( cB)
(8)
The C is added to the training loss as a regularization term to enforce exogenous invariance. In the ideal case where ug taa the counterfactual generation becomes:
Jef fe CB)
(10)
which ensures that all variations between and Fet are attributed solely to the conditioning change CA + cB, while preserving the exogenous factors encoded in u,,
This loss directly penalizes any differences between the underlying noise vectors of the original and edited image. The motivation to ensure that æ and Æe share the same source variation, so that much of the scene's 's random details possible remain unchanged. This approach extracts cditing-related information from the seed, enabling differences to be more expressed by the condi- tioning rather than random variations. The primary change affects the reverse mapping/generator's decomposition conditions, belonging to "seed-level" invariance.
Intuitively, if uh and us are identical, the only differences between žet and z will come from the changed conditioning en vs CΛ. In the ideal case, L₁ means we are generating the counterfac- tual with the exact same "random seed" the factual image (the pure SCM counterfactual). This
40

---

Published as a conference paper at ICLR 2025
encourages maximal consisteney: the background, lighting, style, and other incidental attributes should stay the same unless the new prompt explicitly demands their change. Enforcing C provides strong alignment that leads stable edits. It prevents the edited image from drifting in appearance or composition: the counterfactual will tend to have the same objects and layout as the original, only differing in the aspects dictated by the prompt change. This is beneficial for preserving iden- tity (eeg.,the same person's 's face before and after an edit) and ensuring only the intended attributes change.
However, this strict constraint can also have a few limitations. If the counterfactual prompt cn is significantly different from сa, using exactly the same noise Ue might overly constrain the genera- tion, resulting in artifacts or an incomplete edit. The model might struggle to reconcile uaa (which was optimal for the original content) with the new prompt, leading implausible images or failure to fully achieve the desired change.
J.6 METHOD 2: L2 STRUCTURE PRESERVATION AT HIGH-NOISE
As a more relaxed alternative, we propose to align the diffusion model's behavior for the two im- ages the high-noise stages of generation, rather than forcing the initial noises to be identical. Concretely, let S be set of diffusion time steps focusing on the high-noise region (e.g..the latter half or last third of the diffusion schedule, when Te is still highly noisy). We define the structure preservation loss L2 as:
C2 = Σ Elle(TE,,CA. - co( CB, f122.
(II)
LES
Le measures the disparity between the model's denoising predictions for the factual versus counter- factual image trajectories, but only at very noisy states (where x; is mostly noise). By penalizing this difference, we encourage the denoiser's reaction to the two inputs to be the same in the early stages of generation. This effectively steers Zet, to evolve in a similar direction as ze while the image is still coarse and noisy, ensuring the two generation processes start out aligned in terms of global structure. Importantly, L2 does not enforce that the latent noises I, themselves are exactly equal, only that the predicted noise residuals (or equivalently, the score vectors) are similar. This distinction makes L2 a partial relaxation of the Lı constraint. nudges the counterfactual to have a similar high-level appearance without locking in all the exact stochastic details.
When using L2, the model is free to adjust un as needed, but it will still preserve large-scale aspects of ua. For example, if depicts particular scene layout, Le will bias zet to keep that layout, since early denoising steps (which shape the overall composition) will be similar for both. As t gets smaller (less noise), Fat can gradually diverge more realize the new content eg specifies. This approach maintains structure and identity better than an unconstrained edit, while granting more flexibility than L1 for the model to incorporate the new prompt. In essence, La focuses on aligning the coarse-grained features (which are determined in high-noise stages) and lets the fine details emerge freely.
One might consider applying a spatial mask so that structure preservation is enforced only on cer- tain regions (for instance, only aligning the background areas that should remain unchanged), In our approach, we generally did not require an explicit mask for L2. Since L2 operates on high-noise (low-detail) states, it inherently affects global structure more than specific fine features. We found that a well-balanced Le encourages overall consistency without needing per-pixel restrictions- the model naturally preserves unedited parts of the image. However, if a particular application demands strict locality (c.g., editing only small region while leaving everything else exactly as is), a mask could be introduced to further ensure no influence of L2 on the region to be changed (or conversely, to focus L2 only the region to preserve). In summary, L2 already provides soft, global consis- tency constraint, and masking is an optional refinement rather than a necessity in most cases.
J.7
AND ITS DIFFUSION EMULATION
J.7.1 SCM SETUP.
Let M = (U, P(U)) be a structural causal model with exogenous variables U, endogenous variables V. structural assignments F, and exogenous distribution P(U) (Pearl 2009). We single
41

---

Published as a conference paper at ICLR 2025
out: (i) the mechanic variables X S V that will intervene on; (ii) the controller inpur C; and (iii) the visual variable whose realization is impact frame snapshot v. Throughout, we explicitly treat generation seed a realization of the SCM exogenous variables, i..,, P(U), and we treat all other shared rendering conditions as part of w (while is held fixed explicitly).
J.7.2 CODUCTION-ATTOON PREDICTION (AAP). -ATTIAAOPOOOOIOOTOOOOOOO
Given a factual observation Up obtained under (X =zo C=c), the AAP recipe for the two-world case proceeds as:
(Abduction) Infer P(U V==0,X=ay C=c), choose representative w.
(Action) Form the intervened model Mx==, while keeping C'=e and U= fixed.
(Prediction) Evaluate the counterfactual (A).
Equivalently, in distributional form,
ú1 P(Vi Vo=to, Xo=Io, C=c),
which is shorthand for propagating @ P(U Vo= Xo= C=c) through Mx=z
Algorithm Abduction ietenttiionAttiio Prediction (two-world case)
Require: SCM .M (U, V, F, P(U)), factual (Vo, do, c). target
Abduction: Infer +- MAP/mean/sample from P(U V= X = C=c)
2: Action: Construct MM==xx == hold C=c and U:=@
3: Prediction: Compute f + Vx=x, (4)
4: return 01
J.8 DIFFUSION- BASED EMULATION OF AAP
In our latent diffusion setting, we emulate abduution-action-prediic (AAP) by identifying the SCM exogenous variables with the model's initial latent noise:
U
+ w ~ (O, I).
Abduction (DDIM inversion). estimate from factual impact frame Vx=ao under (ro,, c) via deterministic sampler inversion (DDIM, 7=0). Let Zt.x denote its noisy latents across t E [0, At each step, the denoiser (Z t, co) predicts the noise, and inversion equa- tions (Appendix 5.10.5) are used to propagate forward in the schedule, yielding estimate Abducts (Zo,X=no: Cp).
Action. Hold fixed and change the mechanic state from Tu T1.
Prediction (deterministic reverse). Initialize the trajectory with Zr.x=zı := e and run the de- terministic reverse process conditioned on (Z1, c) to obtain a counterfactual latent Zo,X=x,> then decode the counterfactual impact frame Vx=m--
Algorithm 2 Diffusion-based AAP via DDIM (170)
Require: Diffusion model (co, scheduler, decoder), factual (Vx =x= To, c). target æ1
1: Abduction: +- Abduct, (Zo. Co)
// DDIM inversion in latent space
Action:
Keep fixed, set X x
3: Prediction: ZT.X=m, == W; for t
:....., 1: 2,-1x==, + DDIMStep (2,
Vx=e, ddecoder(Zo.x=an
4: return Vx=r:
J.8.1 CONNECTION TO OUR TRAINING LOSSES.
The AAP framing clarifies the roles of our loss components. (i) Exogenous alignment L, encourages shared-seed seed invariance by driving
Abducto (70,
Abducto Aoducta(Zo,xxaz) (Zo 01),
42

---

Published as a conference paper at ICLR 2025
for elements of the same contrast that share the true This ensures the abduction step produces sistent seeds across worlds. (ii) Structure preservation Le aligns denoiser outputs co ((x,,,, oy) at high-noise steps E S, enforcing agreement coarse global structure during the early reverse process. This mirrors AAP's assumption that exogenous factors (w) are held fixed while only X changes.
J.8.2 SCM + DIFFUSION MAPPING (TWO-WOR) TWOOWORLD CASE).
SCM concept
Diffusion instantiation
Exogenous U
Initial latent noise seed No
Abduction P(U Vx==o;
To,
++ DDIM (7=0) inverse sampling å = Abducto (Z0 co)
Action X = 21
+ Condition sampling process on (21,c)
Prediction Vx= Vx=z, (w)
+ Deterministic sampling to Zo.x=z, then decode V Vx=mi
Causal consistency
Shared seed W; early-step structure preservation (C2)
Abduction via DDIM inversion yields an estimate whose fidelity depends the schedule and conditioning; see Appendix J.10.6 for caveats and tuning guidance.
(oppendix..10.661
J.9
ADDITIONAL REGULARIZERS
Beyond the core losses L, and L2, our framework can incorporate additional terms to improve consistency and fidelity:
J.9.1 SUBSPACE CONSISTENCY Loss.
We can encourage the factual and counterfactual images remain close in certain intermediate rep- resentations of the diffusion model. For example, one may align hidden latents or cross-attention maps at corresponding diffusion steps. By penalizing differences model's multi-head attention maps for background tokens, or feature maps in a particular UNet layer), we enforce that the internal generation pathways for and Ver stay similar. This helps pre- serve layout and identity at semantic level, complementing the pixel-space alignment enforced by L1LL2. Formally, if (z) denotes some feature (such as a latent embedding or attention tensor) computed during the denoising of x: at step I,
these subspaces (e.g., the
define loss T ΣieT Fe( (x)er) || for some chosen set of layers or timesteps T. This subspace consistency loss encourages the edited image differ only minimally in features unrelated the intervention.
Text Consistency Loss. To ensure the edited image indeed reflects the counterfactual prompt cg, we include a text-image consistency term. We rely on a pretrained image-text similarity model (such as CLIP) to measure alignment between Fer and the deseription cg- Let sim(2.s CB) ) be similarity score (higher means the image matches the prompt better). We define a loss Lueat = -sim (Tef, CB) (or equivalently 1 sim, depending on the normalization) maximizes the agreement between the generated image and the desired attributes. This ensures that while preserving content, we do not under-shoot the edit: the new image should clearly exhibit the prompted change. The text consistency loss guides the generation to remain faithful to the user's s request, especially when L, or La are pulling towards the original image. It helps avoid the outcome where the edit is so conservative that the difference between and is hard to discern.
that minimizing Liett
J.10
LOss COMBINATION
Our full counterfactual editing objective combines these components in a weighted sum:
Lioial \1 AaL2 + Aa + Aq
(12)
where A tunable weights that control the influence of each loss term. In practice, choose these weights to balance identity preservation against effective editing. Typical values and trade-offs follows:
43

---

Published as a conference paper at ICLR 2025
A1 (Consistency Alignment): This is often kept relatively small (e.g., A1 in the range 0 to 0.5) unless the edit is very minor. A small A nudges the initial noise vectors closer without forcing identity completely. Increasing A₁ leads to more literal counterfactuals (very high consistency with the original image's details), but if set too high it may prevent the new attributes from appearing strongly. There trade-off between maintaining hackground/identiyy (higher X favors this) and allowing change (lower Y1 gives more freedom).
22 (Structure Preservation): We usually give L2 moderate high weight (on the order of 1.0) as it is the principal mechanism to preserve structure. X2 in range roughly 0,5 to 2.0 works well. A larger X2 tightly constrains the high-level layout and style to match the original, which is good for identity preservation; however, if X2 is excessively large, it can act almost as strictly as Ir, potentially impeding necessary changes. Reducing X2 allows the counterfactual generation to deviate more in composition if needed, but too low X2 might result in unwanted alterations in background or other objects.
This weight should be high enough to ensure the edit actually happens (especially for subtle changes), but not so high that overrides the preservation losses. In practice Xa is often set around 0.5 to 1.0 (assuming similarity is scaled to comparable range) so that the image aligns with the prompt without artifacts. A3 is set too low, the edit might be too conservative (the model might simply regenerate the original image to satisfy L2/L2). If λ is too high, the model may introduce exaggerated or incorrect features to satisfy the prompt, possibly compromising identity or visual quality.
a (Text Consisteney):
X (Subspace Consistency): If used, this is typically a small auxiliary weight (e.g., 0.1). Since Luub operates internal features, it can strongly bind the generation if overweighted. A modest Å helps reinforce structural consistency without conflicting with the primary losses. Tuning X involves checking that indeed improves preservation of details like face identity or scene layout, without, for example, freezing the image in an early-state that ignores the new prompt. In some cases, we might set \ (i.e., not use this term) if we find L and L2 are sufficient; when used, it serves as an extra regularizer.
In summary, L1 and L2 provide spectrum between strict and loose alignment, Às drives the fidelity to the requested counterfactual change, and A can bolster consistency on feature level. We recom- mend starting with a balanced combination (for instance, À = 0.2, X2 = 1.0, X3 = 0.5, 2 0.1).
J.10.1 DIFFUSION BACKBONE.
We use pretrained Stable Diffusion variant with a deterministic sampler which makes the map- ping between exogenous noise and image v approximately invertible. This enables abduction of from an image and consistent reuse across parallel worlds. Classifier-free guidance is applied with scale 7.5, and the scheduler uses T 50 steps.
J.10.2 CONDITIONING.
In the main text we describe conditioning directly on game state variables (e.g., shield, weapon type, block outcome). For implementation, these variables and outcomes converted into natural- language captions for compatibility with CLIP text encoders. This is a nuisance parameterization: the underlying conditioning remains the game state variables.
J.10.3 TRAINING SETUP.
We fine-tune the network backbone (UNet) only, keeping the VAE and text encoder frozen. Batch size is 4, training runs for 50 epochs, with a cosine learning rate schedule, weight decay 0.01, and gradient clipping. Images are peak-action snapshots extracted at canonical times in each episode to minimize temporal ambiguity.
J.10.4 ALIGNMENT LOSSES.
We mainly use two loss functions for finetuning:
specifically DDIM also the deterministic samplers in Karras al. 2022
44

---

Published as a conference paper at ICLR 2025
(a) Factual: 1-on-1 battle. Fae type archer, ele- ment is fire, weapon bow, weapon element is none, has shield is false, action is shoot. Foe 2 type is war- rior, element fire, weapon is long sword, weapon element none, has shield is false, action is idle. Foe turn is for 1.
(b) Counterfactual: A 1-on-1 battle. Foe type is warrior, element is fire, weapon is long sword, weapon element is none. has shield is false, action is idle. Foe type is warrior, element is fire, weapon is long sword, weapon element is none, has shield false, action idle. Foe turn foe 1.
Figure 14: Example counterfactual image pairs generated from finetuned text-to-image diffusion model.
L1: consistency alignment. After inverting both factual and counterfactual images to latent noise (Ma, us), we penalize I| uoll? enforcing invariance of exogenous factors. Note that this loss function is very expensive due needs to inverse two sampling paths for each counterfactual data pair. In practice, only apply Li one data point per batch of training data.
Lz: structure preservation high noisy region (large SNRs). At early diffusion timesteps, we penalize discrepancies between denoiser predictions for factual vs. counterfactual tra- jectories, encouraging global structural consistency.
Both terms be weighted with coefficients Åι, A2-
J.10.5 SEED ABDUCTION WITH DETERMINISTIC DIFFUSION.
Our multiverse alignment objective requires that consistent contrasts share the same exogenous noise w. In deterministic samplers (c.g., DDIM), this means that two parallel reverse processes (2,,x=ma) >o= and (Z,,=zz)i= can be initialized with the same w, ensuring non-descendant con- tent is consistent.
Given observed impact frame Vx=r with controller input c, let Zo.x= denote its clean latent before decoding and Zt,x=z the noisy latent at step t. At each step, the denoiser cØ(-) predicts noise 1, c), which is then used to trace the trajectory backward through the noise schedule. Iterating these updates recovers an estimate i Abducta (V C).
<o(,,
Applying this inversion procedure both members of contrast yields Wa,, and wr, which ideally coincide. The seed-consistency loss L1 penalizes their distance, providing a concrete operational- ization of the causal consistency principle within deterministic diffusion.
J.10.6 CAVEATS.
Deterministic inversion. We DDIM with r 0 for Abducte. yielding approximately bijec- tive mapping between seed and latent trajectory under fixed conditioning and schedule. In practice, invertibility approximate and sensitive to: (i) the precise noise schedule; (ii) classifier-free guid-
45

---

Published as a conference paper at ICLR 2025
settings; and (iii) conditioning (2, c). Hence w should be treated consistent estimate rather than ground -ound-truthha latent.
J.10.7 CHOOSING THE HIGH-NOISE SET S FOR 22.
We select S as either (i) the last k steps of the schedule (empirically 1 2), or (ii) all with (NR(t) I r for a threshold r. A weighted variant uses wy x SNR(t) and
C Σ we 1|co(=o,1, c) - co(21,1, t,,
Under mild conditions, aligning score predictions at high-noise is connected alignment in data space via Stein's ss identity.
J.10.8 SCOPE.
This is a feasibility study. We claim no pixel-level counterfactual identification and provide only qualitative illustrations. Future work may extend this approach video sequences and com- plex mechanics.
K SOFTWARE DEPENDENCIES
K.1
GAME ENGINE.
The game itself is implemented in Pygame, lightweight Python library for 2D graphics and in- teraction. We chose Pygame because it enables rapid prototyping of turn-based combat mechanics, Trame-accurate rendering impact frames, and reproducible control of random seeds, all within Python environment that integrates smoothly with machine learning workflows.
K.2 CAUSAL MODELING.
To formalize and simulate the causal generative process underlying gameplay, we use the Pyro probabilistic programming library (Bingham et al. 2019). Pyro provides the primitives required to implement SCMs consistent with the game's s causal DAG, including stochastic functions for exoge- nous variables, deterministic assignments for endogenous variables, and intervention operators. This allows us to align the game engine's execution trace with an explicit causal model, and to sample parallel-world contrasts in principled manner.
K.3 REPRODUCIBILITY.
Both components are integrated in unified Python codebase, ensuring that gameplay, causal mod- eling, and data generation can run deterministically from a single seed.
K.4 GRAPH LIBRARIES.
We generate mDAGs, world graphs and counterfactual graphs using the YO library (Hoyt et al. 2025).
K.5
GRAPH SERIALIZATION.
All graphs are serialized as directed graphs in JSON using a node-link format. Nodes are repre- sented as JSON objects with keys for the node identifier and attributes, while edges are represented as objects with source and target identifiers and an edge type field. In the case of mDAGs and parallel world graphs, exogenous nodes are marked with the attribute "exogenous": For example, above example an mDAG with observed nodes {A, B, C), one directed edge A + B, and one hyper-edge [B, C}, is converted to CL-DAG and serialized as follows:
true.
"nodes": ":
46

---

Published as a conference paper at ICLR 2025
("id": "A",,
("id": "B"),
("id": "c"),
("id": (B,C)) "exogenous": true)
"links":
("source"; "A", "target": "3", "type": "directed"),
("source"; "N_(B,C)", "target": "B", "type": "directed"),
("source"; "N](B,C)", "target"; "C", "type": "directed")
]
L GENERALIZATION BEYOND GAMES
Our connss-conssttenyy formulation applies wherever structured state, deterministic updates, and seed-controlled stochasticity are wailable-innluddng physics simulators and VFX pipelines built on graphics engines. In partially controlled environments, one could generate a limited number of explicit contrasts and design objectives that approximate seed consistency and structure alignment when only noisy or implicit multiverse structure is available. However, transferring these insights to more complex real-wer world domains will require significant future work: realistic simulators involve higher-dimensional state and stochastic rendering that may not align cleanly across worlds, and real-world video lacks explicit multiverse structure entirely.
47


Reviews:
[{"review_id": "P4I9hdiBNq", "reviewer": "Reviewer_vVfQ", "summary": "The paper proposes a causal framework for making generative models mechanically consistent (that is, able to change only the causal effects of an intervention while keeping all else fixed). To this end, the authors introduce Multiverse Mechanica, a controllable game environment that generates counterfactual video pairs: parallel worlds sharing the same exogenous randomness (non-descendant variables) but differing in a single game mechanic (e.g., shield on/off). Using single impact frames from these videos, they finetune a pretrained T2I diffusion model (OpenJourney-v4) with a new Multiverse Alignment Loss that enforces (a) shared exogenous seed consistency and (b) early-stage structural alignment. Experiments show that the finetuned model preserves global scene structure while correctly toggling mechanic-specific effects, demonstrating a step toward causally consistent generative modeling.", "questions": "See the above weaknesses.\n\nBesides, I have one question (or suggestion): do you think your framework could be applied beyond the game domain? It seems the same causal-consistency formulation could extend to other areas such as physics simulation or movie generation. If so, adding a short paragraph discussing these broader applications or potential impacts could help widen the paper’s relevance and readership.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- The paper introduces a causally principled definition of mechanical consistency via counterfactual constraint. Unlike previous works which focused on visual realism, it provides a framework of grounding generative modeling in causal reasoning.\n\n- The Multiverse Mechanica environment provides a controllable, reproducible setting with groundtruth causal graphs and consistent counterfactual pairs. This is valuable for studying causal generalization in generative models. (Although the code and dataset release is remained to be seen.)\n\n- The proposed Multiverse Alignment Loss (seed consistency & structure alignment) is intuitive and conceptually sound. It aligns neatly with the diffusion process’s structure (e.g., early steps for global layout; late steps for fine details).\n\n- In their proof-of-concept experiment, qualitative and quantitative results show that the finetuned diffusion model preserves global scene structure while correctly toggling mechanic-dependent effects, supporting the causal-consistency claim.", "weaknesses": "- The approach is tested only on _single-frame_ impact images, not full temporal sequences. This prevents evaluation of whether the proposed causal consistency generalizes across time, although the authors claim their framework should, in principle, extend to video models.\n\n- All results focus on a single mechanic example (\"shield on/off\") and one pretrained model (OpenJourney-v4). It remains unclear whether the framework scales to multiple simultaneous mechanics or to other base models. Again, this should work in principle, but empirical evidence is still lacking.\n\n- In terms of empirical validation, the paper also lacks ablation and baseline comparisons. Since the new loss formulation is one of the main contributions, it would be highly beneficial to show what happens when either the seed-consistency (L1) or structure-alignment (L2) term is removed, and to compare against simpler alternatives (for example, standard fine-tuning or contrastive alignment).\n\n- Finally, the evaluation metrics are indirect. While CLIP, PSNR, and SSIM provide useful proxies, they do not directly capture causal correctness. The paper could propose or adopt explicit causal evaluation metrics to better quantify causal consistency.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hxmr0Ld7JC", "reviewer": "Reviewer_LygS", "summary": "The authors present a new game environment designed to test the ability for pixel-based game world models to accurately learn game mechanics. The environment is designed to automatically produce causal graphs that represent specific game mechanics to allow for ground-truth checking against model predictions. The authors train a proof-of-concept model in their proposed environment and argue that it points the way towards richer training of other world models.", "questions": "- How should we interpret the quantitative results in Table 3? More specifically, what might an unsuccessful model and an oracle model look like?\n- Line 161: is this meant to say the rows of Table 1 instead of columns?\n- For S3 in Table 1: why is it necessary to condition S = 1? Are there ways for a player to block aside from having a shield, and does blocking have different behavior in that context?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "I think the motivation of this paper is quite compelling. The introduction does a good job both laying out existing perspectives on the nature of pixel-based world models and arguing for them to be made more rigorous and formal. In general, the introduction does a good job of laying out why the paper's contributions would be both novel and timely. The Multiverse Mechanica game, while not currently available, does also seem like a useful contribution to other researchers interested in studying world models. While the technical foundations presented in the paper are at times a bit confusing (see below), they also point towards the authors' deep understanding of the  subject matter.", "weaknesses": "I think the paper suffers somewhat in terms of clarity. I admit that this may be due to my own unfamiliarity with the relevant mathematical underpinnings, but I found the technical descriptions in Section 3 fairly difficult to follow and the accompanying figures too busy to easily parse. Later sections use a variety of evaluation methods without clearly defining them (e.g. exogenous distance and transfer CLIP), which makes it difficult to clearly understand the implications of the proof-of-concept experiment. In particular, I wonder about the claim that “the model successfully generates semantically meaningful counterfactuals” -- this seems like something that would benefit greatly from one or more concrete example outputs. In the absence of any baseline models against which to compare or a dedicated user study to evaluate the outputs, it’s difficult to say what the model has accomplished.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YnsKHPk0mF", "reviewer": "Reviewer_WHRQ", "summary": "The authors tackle the relevant but understudied problem of determining whether learnable models can correctly learn game mechanics. To do so, the authors contribute Multiverse Mechanica, a video game containing simple visuals, but reasonably complex game mechanics, which could serve as a benchmark for game mechanics learning. Defining characteristics of Multiverse Mechanica include its logic formalism for precisely describing game mechanics, and its ability to mark \"impact frames\", a sparse set of video frames containing full information on game mechanics which provide a convenient way to perform cheap experimentation. A latent diffusion model is trained on data produced by the synthetic environment, leveraging the established logical formalism to improve model understanding of game mechanics, but evaluation makes it difficult to determine whether this had positive impact on the model. The paper is well written and explains Multiverse Mechanics in great detail. Overall Multiverse Mechanica has potential to serve as a benchmark for game mechanics learning and the logical formalism introduced in the work could spark new ideas in the field, though it remains unclear if employment of such logical formalism will become applicable to more complex models and scenarios of practical interest.", "questions": "I'd like authors to clarify the aspects highlighted in the weaknesses section with particular regards to the concerns regarding clearer definition of experimental setting, definition of metrics, and creation of a baseline against which to compare the proposed method's performance.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The paper tackles the relevant but not yet well studied problem of studying the degree to which generative models are capable of understanding game mechanics. In the rise of powerful video generation models investigation of this hard-to-achieve capability can become an interesting research direction.\n- The authors contribute a videogame that was carefully designed to probe understanding of relatively complex game mechanics under constrained training budgets. The game is reasonably complex from the perspective of the contained mechanics, requiring reasoning on spell types, elemental affinities, weapon classes and equipment to make decision on taken damages. At the same time, its visual simplicity and ability to produce \"impact frames\" make it convenient for small scale ablations. This can constitute a good benchmark or tool for ablations for future works.\n- The paper seems polished and detailed. Writing is clear and the appendix with its 45 total pages describes in great detail the proposed system and logic framework.", "weaknesses": "- The logic formalism employed in the paper may be unfamiliar to a large portion of readers. The authors make every effort to introduce the formalism, but the paper might not easily accessible to the broad generative models audience.\n- The employed logic formalism appears difficult to define even for a relatively simple game as the one created by the authors. A concern is whether the proposed formalism and methodology could eventually scale to modeling the more complex game mechanics encountered in mainstream video game titles. If such possibility is not foreseen, the impact of the method could be limited. I could see a use for the formalism in evaluating in a rigorous way video sequences for the purpose of determining if a model has learned the described mechanic correctly, but I don't see an easy way of scaling training as performed in Sec 5 on a more complex scenario.\n- The proof of concept model shown in Sec. 5 is built based on the assumption that contrasts are known, a condition that is satisfied for the proposed synthetic data generator. For example, it proposes a seed consistency loss aligning noise for samples across contrasts and a structure alignment loss. These components would be difficult to leverage in the context of large scale video generation methods, which would be the ultimate models on which to perform evaluation for game mechanics understanding, but that would require large amounts of data for which contrasts would be very difficult to derive.\n- The experimental setting with respect to employed metrics and experimental setting is unclear, making it difficult to interpret results. I suggest authors to describe each of the employed metrics and its computation procedure.\n- Interpretation of the experimental results is difficult because of the lack of baselines or ablations. The paper does not validate whether any of the introduced components in Sec 5 improves model performance with respect to straightforward diffusion model training.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FOeyYAVXLo", "reviewer": "Reviewer_4gTF", "summary": "The paper employs the formalism of causal learning to formulate the problem of learning game mechanics from visual observations. The paper proposes a framework to describe how interventions on (discrete) game states alter the corresponding image representations under different data generation conditions. This includes an example domain to generate data, including the relevant causal modeling information. To evalute the approach a diffusion model is trained to predict the visual outcomes of causal interventions generated using the framework.", "questions": "# questions\n- How are causal hierarchy level-1, level-2, and level-3 defined?\n- How does the experiment demonstrate the need for level-3 data? (claims in lines 178-182)\n- How does the experiment demonstrate generalization? (claims in lines 92-93)\n\n\n# suggestions\n- The following experiments may be a natural progression to test the claim that the level-2 and level-3 data are required for learning game mechanics. Given the established criteria of how well the model predicts outcomes given information on the intervention:\n\t- As a base experiment, remove all visual information (or map it as a single pixel indicating each state) and demonstrate the core claims. This setting decouples model representational learning capacity from causal learning. Then demonstrate that the model trained with equivalent amounts of level-1, level-2, and level-3 data shows the expected pattern of succeeding in tests only with level-3 and not the prior levels. Perhaps include a small scaling test for samples required.\n\t\t- The outcomes can easily be validated automatically without human evaluation. This bypasses the problems of assessment and directly tests the core claims.\n\t- Add the current scenario and evaluate on the above metrics.\n\t- Add the ability to generate scenarios that violate the assumptions required, like (fully or partially) hiding visual representation of the relevant causal variables.\n\t- Add generation of full videos and require learning on those to support the claim in the formalism around video input.\n\t- Add procedural variation of assets and placements to the framework to test generalization.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 1, "presentation": 2, "contribution": 1, "strengths": "# originality\nProvides more formal rigor on defining the task of learning game mechanics from observations compared to prior efforts. The causal learning model here is novel relative to prior formalizations.\n\n\n# quality\nExplicitly seeks to create a data generator for model training that matches the formalism.\n\n\n# clarity\nIntroduces the case study domain with a worked example (Table 1).\n\n\n# significance\nOpens the path to investigating causal learning for game mechanics based on discrete states. This could be of interest to studies of world models and their learning and limitations. Also for learning world models for game playing agents from demonstration videos.", "weaknesses": "# quality\nThe experiments seem mismatched to the objective of formalizing causal learning. The model starts by framing the problem as learning from video, then reduces that to a few input frames. Unfortunately, the evaluation metrics do not touch on the central question of whether the generated data was neccessary or sufficient for causal learning. Under questions I offer some suggestions on what experiments could help directly test this core idea. With development I believe these would be a strong launching point for the ideas to see further development and extension.\n\n# clarity\nThe notation often was confusing and cluttered, rather than clarifying on the formalism. Perhaps it would help to have some listing of the symbols used and their definitions for easy reader reference.\n\nThe causal hierarchy being adopted should be introduced as background to readers. It's a core part of the framework, but never directly explained.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)