Learning Interactive World Model for Object-Centric Reinforcement Learning
Authors: Fan Feng
Phillip Lippe
Sara Magliacane
Venue: NeurIPS 2025 poster ()
Keywords: World Model
Object-Centric RL
Primary area: reinforcement_learning
TL;DR: We provide a two-level factorization to learn structured latent states with interactions for object-centric RL.

Abstract: Agents that understand objects and their interactions can learn policies that are more robust and transferable. However, most object-centric RL methods factor state by individual objects while leaving interactions implicit. We introduce the Factored Interactive Object-Centric World Model (FIOC-WM), a unified framework that learns structured representations of both objects and their interactions within a world model. FIOC-WM captures environment dynamics with disentangled and modular representations of object interactions, improving sample efficiency and generalization for policy learning. Concretely, FIOC-WM first learns object-centric latents and an interaction structure directly from pixels, leveraging pre-trained vision encoders. The learned world model then decomposes tasks into composable interaction primitives, and a hierarchical policy is trained on top: a high level selects the type and order of interactions, while a low level executes them. On simulated robotic and embodied-AI benchmarks, FIOC-WM improves policy-learning sample efficiency and generalization over world-model baselines, indicating that explicit, modular interaction learning is crucial for robust control.

Learning Interactive World Model for Object-Centric
Reinforcement Learning
Fan Fengl: 1.2
Phillip Lippe Sara Magliacane
University of California San Diego Mohamed bin Zayed University of Artificial Intelligence
University of Amsterdam
(ffeng1017,
ippe, sara magl iacane @gma i 1. com
Abstract
Agents that understand objects and their interactions can learn policies that more robust and transferable. However, most object-centric RL methods factor state by individual objects while leaving interactions implicit. We introduce the Factored Interactive Object-Centric World Model (FIOC-WM), a unified framework that learns structured representations of both objects and their interactions within a world model. FIOC- WM captures environment dynamics with disentangled and modular representations of object interactions. improving sample efficiency and generalization for policy learning. Concretely, FIOC- WM first learns object-centric latents and an interaction structure directly from pixels, leveraging pre-tras pre-trained vision encoders. The learned world model then decomposes tasks into composable interaction primitives, and hierarchical policy trained on top: a high level selects the type and order of interactions, while low level executes them. On simulated robotic and embodied-Al benchmarks, FIOC-WM improves policy-learning sample efficiency and generalization over world-model baselines, indicating that explicit, modular interaction learning crucial for robust control?.
Introduction
World models aim to learn state abstractions and action coniiincoddiiooned dynamics that capture the evolution of high-ddmeensonnl observations, along with auxiliary information (e.g.. rewards, skills), for decision-making tasks [1-5). Recent advances have demonstrated their effectiveness in downstream applications, such as robotics [2, 6-10) and autonomous driving [11-14].
One of the central challenges in world model is to extract Iow- now-dimensional, structured latent repre- sentations from nigh-dimensional observations, which often display high complexity and variability across both semantic and dynamic aspects. On the dynamics side, latent spaces often contain underly- ing structures [15, 16). Prior work imposes structural priors learn compact latents that encourage disentanglement and capture relational or compositional patterns [17-23]. On the semantics side, pre-trained visual features are leveraged to better encode rich content and improve fidelity [9. 24-31J. Collectively, these approaches learn compressed, structured representations of high-dimensions perceptual data to support downstream decision making. However, remains unclear what extent such compression and structure are necessary and sufficient for down-streaming policy learning.
In this work, we study which types and degrees of decomposition structure make latent representations effective for efficient and generalizable policy learning. Real-world settings exhibit substantial variability in both visual appearance and dynamic interactions, often involving multiple objects with diverse attributes. It therefore natural to reason in terms of objects, their interactions, and
Now at Google Deepmind Project page: https: //sttce///ttt google. wttp://.o..o..eeeeeeoee.eooetott htwp:///wwwwww/wwmm..m.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Stage 1: Offline Model Learning
Stage2: Online Pollcy Learning
Interaction Policy
Update x². x
Environment
Stove
Lights
Cabinets
State Factorization
Pre-trained Visual Encoder
s' (d',
9g(5,103
Interaction Graphs
P,U, sp) PalS11 a,
x rl,,,s,, GP)
Figure 1: The overall pipeline, including offline model learning (left) and online policy learning (right) phases. The illustrative examples are from the Pranka-kitchen environment [40].
the attributes that induce these interactions. To this end, we propose the Factored Interactive Object-Centric World Model FHOC-WM) which learns a two-level factorization: an object-level representation with explicit interactions, and an attribute-leve representation for each object. This factorization then exploited for down-streaming planning and control.
At the object level, we consider both the decomposition of scenes into independently evolving objects and the modeling of their interactions. Modeling the interactions among objects is crucial for effective policy learning as real-world dynamics are heavily influenced by rich interactions among objects, such as collisions, containment, stacking, and physical forces like friction or gravity, which collectively determine the evolution of the environment [32-34].
At the attribute level, each object be further factorized into attributes based their temporal behavior, e.g., if they are static (e.g., color, shape) or dynamic (e.g.. position, velocity) over time. This factorization provides principled inductive bias reduce redundancy and highlight the minimal sufficient components needed for planning and control. Importantly, this also supplements the accurate object-level interaction modeling the interaction can be further factorized: for each object, only the dynamic part (e.g.. position, velocities) will be changed during interactions with others. By incorporating both object-level and attribute-level factorization, we can precisely model the dynamics of all objects, including their interactions.
This structured modeling enables accurate prediction of system behavior and allows the learned interaction models to serve as efficient surrogates for decision making. Building on recent hierarchical RL with object-centric subgoals [35-37], we instantiate subgoals object interactions, allowing complex tasks to be decomposed into sequences of interaction primitives and thereby enabling more efficient planning and control.
FIOC-WM jointly factorizes the static attributes and dynamic variables of each object in the environ- ment, as well as their interactions with each other and the agent. After learning the FIOC-WA -M, can then leverage its interaction models to learn an interaction-centire policy. This enables efficient solutions for long-horizon policy learning. Inspired by recent work [26, 27]. we pre-trained visual embeddings [38, 39] as surrogates for raw high-dimensional observations, facilitating the learning of semantically meaningful latents. FIOC-WM can recover interactions and learn the factorized states within the latent representations derived from these visual embeddings. The learned interactions then used to train policy designed induce the desired interactions between objects. These offline-learned policies are subsequently employed as composable modules for long-horizon tasks. diverse set of robotic control and embodied AI benchmarks, demonstrat- ing enhanced world model capability and more efficient downstream policy learning by employing the appropriate factorization and leveraging as sub-tasks.
We evaluate FIOC-WM
2 Factored Interactive Object-centric POMDP
We focus on Partially Observable Markov Decision Process (POMDP) [41] and consider an environment in which objects interact with each other, and in which there are global latent factors that can affect or modulate these interactions. We denote the state at timestep t s, and assume can be

---

factored across N objects. Moreover, we assume that the state of each object i can be represented as {di c'}, where d; represents the dynamic, vaneing variables (e.g., position, velocity) and c represents the constant, time-invariant properties such as color, mass, and friction, some of which can affect the dynamics of the object.
We represent interactions between objects with a sequence of fimevaryying graphs G [GI,.... Gr where each edge in a graph G2 captures an interaction between two objects at time This models that at each timestep, dif- ferent objects might interact. We also assume that these graphs are sparse, meaning that at each timestep there are only a subset of objects interacting.
0
O
For each object, we define self-transition function f which represents the evolution of the object dynamics with- interactions. In the ceff-rransitonn function the constant properties influence the evolution of its dynamic variables over time, but not viceversa. When two objects and j interact, an object can only affect the dynamic variables of the other object through the interaction transition function funter- More formally, we model that the state transition for object follows the form: di di+1 fsetr(d (di, c', aa,e))))
d
F12
a
[+
Σ JEN.(1)
finteedli, di d',ce', 61),
where Ni( denotes the set of objects interacting with object i time and 6, and 6, indicate the latent noise variables that model the stochasticity of the system.
An example of FIOC- POMDP, where we only show the reward for + for clarity. Gray nodes are ob- served variables, while white nodes are latent variables. Each orange box repre- sents the state of an object (in this case, objects and j). Red solid edges are the state transition per object, and dashed edges are the interactions among objects.
Figure 2:
We assume that also the observations o factored across the N objects and that the generating process for obser- vation of object i time is of g(s,, et), where ej latent i.i.d. random noise that represents the stochasticity in the observations. Finally, as in standard settings, the reward function is a function of the global state and the action, i.e., F2 h(s,, ar).
We call model that satisfies all of these assumptions a Factored Interactive Object-centric POMDP (HIOC-POMDP). Fig. 2 depicts example of FIOC-POMDP.
3 Learning the FIOC World Models
The overall framework (Fig. 1) consists of two stages: (1) offline model learning (Fig. 3) and (2) hierarchical policy learning. In offline model learning, we learn world model for FIOC-POMDP as two-level factorization of the latent space, at the object and attribute levels, and model latent dynamics based on object interactions. Leveraging the learned interactions, we train an inverse dynamics model to map the states of two separate objects to the states where they interact effectively, which we use as an interaction policy. In hierarchical policy learning, hierarchical policy is trained. The high-level policy selects a sequence of target interaction graphs, while the low-level interaction policy trained in the first stage executes them by inducing the corresponding interaction graph the environment.
3.1 Stage Offline Model Learning
We encode the observations using object-centric representation learning built on top of pre-trained models such as DINO-v2 [38] and R3M [39], which have been empirically shown to provide high-quality image understanding capabilities [42-44, 27, 45] and facilitate robotic manipulation tasks [25, 26, 31]. Building on the empirical and theoretical work regarding the recoverability latent features from supervised pre-trained models [46], we assume that these embeddings provide sufficient features and information for world models. This includes supporting the dynamics and reward models, as well as capturing action-related features effectively. Then, similarly to Zadaianchuk
3

---

Object-centric Network
Dynamics Network
apa+1
0
O+1 Observations
O+N
Object- factored Latents
Pre-trained Vision Network & Slot Attention
Attribute Encoder Interaction Network
Observation Decoder
Reward Decoder
O Ö1+1 Ö
$
Object-centric Embeddings
Attribute-factordd Latents
Fp Frt1
O
Figure 3: The pipeline of Offline Model Learning (Stage jointly learns the observation function, state factorization, dynamics model, and reward model. Although Fig. 2 includes low-level policy learning as part of Stage I, for clarity, we defer the discussion of low-level policy learning to Stage 2.
[47], we use slot attention [48] to cluster the object-centric representation on top of the embeddings. The slot attention outputs set of slot representations, which we use the factored observation {ô .i..... ,
corresponding to the factored raw observation {o', .o....., ......."" o J. To map these factored observations to factored states {s' s² .......... s?....... J. we train variational auto- encoder (VAE) [49] with the encoder (s (s' and the decoder pv (ō' s'), where s' the latent state corresponding to the observation óf, and shared parameters are used across all slots.
To encourage structured representations, we learn to factorize the latent state into static and dynamic components, denoted by and d, respectively. Two separate encoders, fc(s) and fa(s), are used to extract static and dynamic features from observations. We assume that static features remain invariant over time, while dynamic features evolve. To enforce this, we regularize the output of fc(s) remain temporally consistent for each of the N object slots:
Sstatic ΣΣ
(e(((41) fc(s))
t=1
where 7' is the number of time steps. To ensure that different objects encode distinct static attributes, we use a contrastive loss [50] that separates static features across slots:
T-1 L
(f))
ΣΣ log
(fc(s)), fa(st)) + EjEN (f((s)).
where 1' is different time step and N denotes a set negative slots j * from the same scene, g is the distance measurement of the representation,
cosine similarity here.
For the dynamic features, leverage their temporal evolution model latent state transitions, as only d varies over time. We adopt the variational inference framework [49] to learn the encoder fa(s), parameterized by GRU [51], which captures the dynamics of each object slot. The prior over the dynamic state de is factorized across the N object slots as: P. (d-1,at-1) d.-1, a, Gr), where G, denotes the interaction graph representing the relational structure among objects at time f. In other words, G, captures the pairwise interactions between objects at time step I, where each edge indicates whether an interaction exists between a pair of objects. Concretely, this is represented as binary adjacency matrix of size N x N, where N is the number of objects.
II (d}
The posterior over S, is conditioned on the current the visual embeddings ô, and the hidden state h, = GRU(s:-1 h,-1), as: (s; 6,, hy). Then we use an observation decoder to reconstruct observations: Po (ō, st), with the reconstruction loss:
Crecon Σ
| o decoded

---

where od is sampled from Po(: st). To capture temporal consistency, we also predict the next-step observation:
ppdd
|0041
0+1
decoded
(4)
We encourage alignment between the posterior and the prior using KI. divergence:
T
LKI.
KL (go(se oz, he) Pa
Si-1; 1,- G+))
(5)
Similarly, we apply reward decoder (Te Sc, as) based on the learned latent states and actions. The reward loss is as follows:
T life rell..
Lrew Σ
(6)
To learn the interaction graph Gt, we use the current estimated latent states S; as input. We introduce surrogate latent variable u, that parameterizes the distribution over interaction graphs. This captures the underlying interactions that may vary over time.
Specifically, for each object pair (i.j) at time we encode their latent states s and sy using GRU encoder to obtain pairwise embedding:
u = fenc.o. (s)....(sssss
(7)
The transition of u, is modeled as: P. S.) = (St). where fu is a parameterized function that captures the dependencies among the current latent states sya We consider two approaches for learning the state transition distribution p.: (i) learning variational masks, following [52, 53]; and (ii) applying conditional independence testing, following [54]. The detailed loss functions are provided in Appendix C.2.
3.2
Stage 2: Online Hierarchical Policy Learning
In this section, we describe how we use the learned interactive world model for object-centric RL, particularly for long-horizon task learning. Our framework built on the recent work that models the object interactions as skills [37]. The key intuition is that long-horizon tasks can be decomposed into a sequence of interactions.
Our approach first focuses on learning low-level policy capable of invoking the desired interactions. Based the learned interactive world model, we can accurately predict the dynamics of interactions and the regimes governing these interactions. This enables the agent to learn the policy by leveraging the predicted interactions to learn the inverse mapping from interactions to actions. We learn the low-level policy n' by employing model predictive control (MPC) [55, 56, 5] or proximal policy optimization (PPO) [57], where initial and target interaction of two objects are provided. At time step f, we given the target interaction graph at future steps from high-level policy, denoted as G and the low-level policy is ra st, G.).. Given the learned transition models p. and Pu. we use s" and u to infer the target states s and sy Using these inferred target states, we apply MPC or PPO to generate a sequence of actions that transitions the system from t to t + k while minimizing the discrepancy between the predicted and target states. We learn the low-level policy during world model learning (Stage 1), and then fine-tune it with online data during Stage 2, where the policy is updated each time new interaction data becomes available.
We then learn a schedule of interactions for the model to handle long-horizon tasks by optimizing the task reward. We learn the iraain-of-nteeractoon for the high-level policy T² S G, which selects the interaction graph G based on the input state S. This implies that the action space corresponds to graph selection, but this space can grow exponentially with the number of objects. To address this, following previous works on skill discovery with object interaction [58, 37], we impose constraints by limiting the number of objects considered at each time step. Following the graph selection policy introduced in [37], at any given time, we focus on a fixed subset of objects (smaller than 2), leveraging a diversity reward Fdiv as a surrogate to make the selection process diverse. We define Taiv vviiiel where |Gvisteal is the number of graphs that have been visited in the past transitions. Then the high-level policy n (G st) is updated with both the task reward Ftask and this diversity reward Talv-
5

---

3.2.1 Practical Implementation
We assume that each state s, is associated with an interaction graph G", and the final task corresponds to reaching desired target graph G The high-level policy n" selects a sequence of intermediate subgoal graphs that gradually transform G' into G9, where each subgoal graph differs from the previous one in only a single interaction. For example, in task such moving kettle from the counter to the stove, the graph transitions involve first enabling an interaction between the arm and the kettle, followed by an interaction between the kettle and the stovetop.
To make the subgoal selection both tractable and structured, we do not sample directly from the full space of possible object interactions. Instead, at each decision point, we first identify small subset of objects (typically one or two objects) as primary candidates for initiating interaction changes. These candidates define the anchor object(s) i, and we then select target object j conditioned on to form the proposed subgoal interaction ji, This scheme reduces the combinatorial action space and leads to more localized graph transitions. Note that the selected subset does not constrain the interaction to only occur between these objects; rather, it defines a focused region of the graph for subgoal exploration.
4 Related Work
Our framework aims to uncover interactive and factored object-based based representations of environments, so it is closely related to factored RI., particularly object-centric RL. Factored RL models the envi- ronment in terms of Factored Markov Decision Process [59], where the state of the Markov decision process is factored in state components and sparse relationships exist among state components, actions, and rewards. This factorization enables efficient policy solutions [60, 61]. A specific type of factorization, which we also adopt, is object-centric reinforcement learning [62]. where states or observations are grouped into object-centric clusters. In object-centric RL, actions typically target only subset of objects, and rewards are often associated with the states of specific objects or object subsets. This facilitates more structured and efficient decision-make making.
Recent works on object-centric RL can be broadly categorized into two major directions: (i) learning object-centric representation and (ii) modeling the object relations and policy architectures for compositional generalization. For the first line of research, approaches focus on using object-centric representation learning techniques [48, 63 63-65] to extract meaningful object-level features from raw observations. These methods then learn object-centric policies directly from object-centric representations [66, 67, 62). The second line of work develops object-centric policies by modeling object relations and policy structures, incorporating inductive biases in the state transition and policy networks. Methods include the use of graph neural networks [68]. linear relational networks [69], self- self-attention, and deep sets [70]. The learned object-centric states and relational structures are then used achieve compositional generalization in reinforcement learning [71, 18, 33, 72, 22, 73]. Our work combines ideas from both directions, especially related to the series of works [33, 71, 22, 67], which learn factored state attributes, providing a more fine- ane-grained representation than object-centric factorizations, and also model the interactions among objects to achieve compositional generalization. However, we go beyond object-centric policies by learning an interaction interaction-centrie centric policy.
Our framework, which uses low-level and high-level policy for decomposing complex tasks into interaction learning, is similar to hierarchical reinforcement learning (HRL). HRL typically consists of a high-level policy (often referred to as an option [74], sub-skills, or sub-goals in the literature) and a low-level policy, enabling the efficient learning of complex RL tasks [75]. Within the scope of HRL, the most close to our work is the line of research that focuses on learning coal-conditioned hicrarchical policies or hierarchical skill discovery. Zadaianchuk et al. [66] propose the goal-conditioned hierar- chical policies with learning object-based hierarchical goals. Hierarchical skill discovery focuses on decomposing complex tasks into object-wiss or injeectinteraccion-- based components. For instance, Wang et al. [37] use conditional independence testing to identify sub-goals, while Chuck et al. [35, 73] and Hu et al. [76] use Granger causality or counterfactual reasoning to uncover hierarchical structures. Our work is also built upon those works in using interactions as sub-skills [36. 37], but we learn interaction models jointly with observations and dynamies within the world model directly from high-dimensional inputs.

---

[a]
(b)
(c)
Figure 4: Visualization of evaluated benchmarks: (a). Sprites-Worid: World; (b). OpenAl-Gym Fetch; (c). Franka Kitchen; (d). i-Gibson; and (e). Libero. A larger version is in Table A4.
5 Experiments
To evaluate the effectiveness of our proposed interactive world model and policy learning framework, we aim to address the following questions: (i) How accurately does the model learns the state disentanglement and interaction models? (ii) How well does it perform in long-horizon task learning?
and (iii) How well does the framework achieve compositional generalization?
To answer these questions, we evaluate our method on a range of simulated control, robotic ma- nipulation, and embodied AI benchmarks, including Sprites World [77], OpenAl-Gym Fetch [78], iGibson [79], and Libero [80]. We consider both reinforcement learning and imitation learning tasks.
Baselines. For online RL or imitation learning, we compare against established baselines including DreamerV3 [4], TD-MPC2 [81] and the object-centric model-free method EIT [67]. For offline RL, we compare with DINO- MWOO-MMPP [27], which also leverages DINO-based pretraining for downstream planning.
Benchmarks. We consider long-horizon tasks that require completing several sub-skills to achieve the overall objective. OpenAl Gym Fetch [78] is simulated environment featuring a Fetch robotic arm capable of manipulating cubes and switches. The tasks involve completing sub-tasks that require pushing or switching a varying number of objects. Franka-kitchen [40] is an environment where the 7-DoF Franka Emika Panda arm performs tasks in a kitchen. We consider several sequential sub-tasks, such as turning on the microwave, moving the kettle, turning on the stove, and turning on the light. i-Gibson [82] is a simulated environment with Fetch robot operating in everyday household tasks with rich objects and interactions. Similarly to [37], we consider the tasks with the peach object. Libero [80] is benchmark for lifelong robot learning and imitation learning in houschold and tabletop environments. We focus on randomly selected tasks within libero-goal.
5.1 Evaluation Metrics.
In addition to evaluating policy learning and planning performance, we assess the effectiveness. of world model learning by examining three key aspects: observation and dynamics modeling, interaction learning, and disentanglement quality. Specifically, for all methods (excluding those evaluated under nSHD), we adopt variational masks to infer the interaction structures. For downstream control, we apply MPC for Gym-Fetch and Tranka-Kitchen and use PPO for LIBERO and iGibson.
Observation and Dynamics Modeling We measure the predictive quality of future observations using the Learned Perceptual Image Patch Similarity (LPIPS) metric [83], which evaluates perceptual similarity between predicted and ground-trufe -tuth image patches.
Interaction Learning We evaluate the ability of our model to learn interactions through two approaches: (i) Variational mask learning, where the state encodes adjacency matrices latent variables. Each edge is sampled from either a differentiable approximation of a categorical distri- bution [84, 85, 52] or discrete codebook [53]. (ii) Conditional independence testing, where we test for the existence of interaction using parametric models to predict dynamics [54]. We com- pare our approach with baselines that not explicitly model dynamic structure, but instead rely on post hoc analysis based on attention weights to infer interactions, in local causal discovery methods [86, 87]. We use normalized Structured Hamming Distance (nSHD). Further details are provided in Appendix C.2.
7

---

Environment
Dreamer-
TD-MPC2
EIT
DINO-WM
FIOC
Fetch
0.042
0,039
0.026
0,009
0.007
Kitchen
0.102
0.123
0.096
0.035
0.038
Libero
0.089
0.061
0.040
0.035
0.027
Table 1: Comparison of world models on LPIPS metrics on Fetch, Kitchen, and Libero.
(a)
(b)
FIOC-DING
DING-WM
OCADINO DONG
0.4
MSB
0.3
0.2
0.1
0.2
position
velocity
color
shape
0.0
Kitchen
Fetch: 4-Push
Fetch: 4-5witch Fetch: 3-Push 3-Switch
Figure 5: (a) Evaluation of state factorization in Sprites-world. We report MSE from linear probing assess the quality of the learned representations against ground-truth attributes. Offline RL performance (success rate) comparison with DINO-WM, including FIOC-DINO and FIOC-R3M.
Disentanglement Quality SpritesWorke [77], we perform linear probing analysis by training a linear regression layer top of the learned representations to predict ground-truth static and dynamic factors. Static factors include object color and shape (encoded as one-hot vectors), while dynamic factors consist of object positions and velocities.
Policy Learning We consider both the policy performance on single-task and the generalization task. For generalization tasks, consider three types of generalization: (1) Artribute Generalization: we evaluate for zero-shot generalization on new composition object attributes; (2) Object Attribute Composition: train models domains with specific combinations of object attributes (e.g., color, shape, or material) and test them on domains with unseen attribute combinations; and Skill Composition Generalization:
train models on tasks with simple combinations of skills and test them on tasks requiring new combinations of skills, For all tasks, we use the average success rate as the evaluation metric.
5.2 Results on Learning World Models.
As evaluation the learned dynamics and observations, Table reports the LPIPS metric (Full Results are in Table A3). Compared to the baselines, our method achieves comparable or better reconstruction performance, particularly on the Fetch and Libero environments, where object interactions and dynamics are complex. Full results are in Appendix D.1.
We report also results the accuracy of the learned interactions, quantified by the normalized Structured Hamming Distance (SHD) between the inferred interaction structures g ( (cI,... Gr) and the ground truth structures. Fig. 6(a) presents the results on attribute and compositional general- ization. For each bar, the shaded areas represent the performance of single-task learning with the same number of objects. The gap between the top of each bar and the top of the corresponding shaded area quantifies the performance drop when generalizing novel scenarios (i.e., empirical generalization gap). These results demonstrate that FIOC consistently outperforms attention-basee methods across all cases, verifying the importance of explicitly modeling the interaction structures and their changes using regime variables. And importantly. FIOC demonstrates superior generalization compared to attention-based methods, as shown by the smaller empirical generalization gap. Among the three versions of FIOC, all achieve strong attribute-level and compositional generalization. Notably, the variational masks with categorical distributions perform best, particularly in scenarios with large number of objects. Full results are in Appendix D.I.

---

(b)
Compoctional
Nomalbed STD
Strrree Etste
0.
305
w/ Vadiekkal Maria (Cai]
1S01 Variational Maada (Cokktook)))
Enstronnenn stepe (1X)
7100
Drsoomn V2
7D-MUU
Figure 6: (a) Evaluation of learned interaction graphs with Normalized SHD for attribute and compositional generalization on Sprites- World with multiple objects. The shaded areas show results with the same number of objects for single-task learning. Lower values mean better performance. (b) Learning curves single-task learning for i-Gibson (left) and Libero (right).
Fig. 5(a) reports the linear probing MSE of the learned static and dynamic representations, с and d, against ground-truth attributes in the Sprites-world environment. We evaluate both the DINO-v2 raw input features and the object-centric DINO features (obtained from our first-stage learning without disentanglement). Our method achieves the best factorization of attributes: and effectively capture useful representations for dynamic features (i.e., position & velocity) and static features (i.e., color shape), respectively. Notably, the object-centric DINO generally outperforms vanilla DINO on dynamic features. However, object-centric clustering tends to degrade static attribute representations such as color and shape. Our disentanglement module addresses this limitation by improving the representation of static attributes within each object.
Envs
FIOC
Dreamer- V3
EIT
TD-MPC2
Push & Switch
0.91
0.90+ 0.07
0.924 0.04
0.95
Attri. Gen.
i-Gibson
0.79 0.13
0.62+ 0.16
0.70 0.14
0.65= 0.15
Libero
0.76# 0.14
0.59 0.18
0.73 + 0.12
0.69 0.18
Push & Switch
0.86 :: 0.10
0.81 0,12
0.83 0.02
0.79 :: 0.08
Comp. Gen.
Libero
0.70 0.09
0.58 1: 0.12
0.65 L 0.08
0.63 0.14
Push & Switch
0.8 0.81+ 0.06
0.66+t 0.10
0.73 0.08
0.65= 0.13
Skill Gen.
Franka Kitchen
0. 77.33.. 0.06
0.59 0.09
0.65 0.18
0.62 0.08
Table 2: Policy learning (success rate) of world model in Gym Fetch, Franka Kitchen, i-Gibson, and Libero tasks.
5.3 Results on Policy Learning.
Fig. 6(b) presents the learning curves (sampled every 100 time steps) on the i-Gibson and Libero tasks. The results indicate that world models incorporating object interactions, such as FIOC and EIT, achieve faster convergence compared to state-of-the-ar methods like Dreamer-V V3 and TD-MPC2. FIOC not only converges faster than EIT but also achieves a higher final success rate on Libero.
Fbg.5(b) presents the offline RL results, comparing our method with DINO-WM [27], along with two variants of FIOC that use DINO-v2 [38] and R3M [39] as pre-trained visual embeddings. The results demonstrate that our approach achieves superior performance in both single-task learning and generalization, highlighting the advantages of the proposed two-level factorization on top of pre-trained visual features and the use of hierarchical policy.
Table presents the results of policy learning on single tasks, as well as those in the context of attribute, compositional, and skill generalization. The results indicate that FIOC performs comparably or better than other baselines in single-task learning scenarios and consistently outperforms them in all generalization tasks. Among the baselines, EIT achieves the second-best performance across generalization tasks. Detailed task settings are provided in Appendix The full results are in Table A2 and Fig, A3 in the appendix.

---

5.4 Ablation Studies
To evaluate the effectiveness of dif- ferent components in both offline world model learning and online pol- icy learning, we conduct series of ab- lation studies on the following aspects. For the world model part, we con- sider cases: Without state factoriza- tion: The state is not factorized into static and dynamic components. In- stead, the state transition P, is learned directly on the original s obtained from the DINO embeddings. With- out interaction modeling: Instead of modeling dynamic interactions, we as- sume a fully connected graph for all time steps and learn the dynamics us- ing this dense graph; and Using ran- dom actions in offline learning: In- stead of using pre-trained policies, we train the model with random actions
Success Rate
Ablations
Single Task
Comp. Gen.
FIOC
0.81
0.70
w/o Factorization w/o Interaction w/ random actions
0.77(J 0.04)
0.64(J 0.06)
0.63(J 0.18)
0.52( 0.18)
0.64(J 0.17)
0.48() 0.22)
w/o hierarchical policy w/o pre-trained n' w/o diversity
0.58(J 0.23)
0.42(J 0.28)
0.69(J 0.12)
0.59(J 0.11)
0.62(1 0.19)
0.50(1 0.20)
Table 3: Ablation studies on Libero, evaluating the impact of removing specific components from the world model and policy learning. The bold entries indicate the ones with the largest performance drop. Light yellow and green areas represent the world model and policy learning components.
in the offline learning phase. For the policy learning stage, we consider the cases: chical policy: Policy learning performed directly on low-level actions without a high-level policy governing the sequence of interactions. Without pre-trained low-level policy: The low-level policy n' not trained during the offline phase but learned from scratch in the online phase. Without diversity term: The diversity term in high-level policy learning is disabled.
Without hierar-
The results in Table 3 show that for world models, interaction modeling and using the pre-trained policies in offline learning are the most critical components, as their removals lead to the most significant drop in policy learning performance. For policy learning, the hierarchical policy plays the most essential role. Other components, such as state factorization, utilizing pre-trained policies instead of random actions in the offline learning phase, pre-training the low-level policy, and incorporating diversity, also contribute to improving the policy learning performance.
6 Conclusions and Discussion
We study which types and degrees of decomposition make latent representations effective for efficient and generalizable policy learning. To this end, we introduce the Factored Interactive Object-Centric World Model (FIOC-WMP WM), which learns a two-level factorization: an object-level representation with explicit interactions and attribute-level representation for each object. FIOC-WM learns these decomposed structures directly from observations and leverages the resulting composable interaction primitives enhance planning and policy learning via hierarchical RL approach. The framework exhibits strong compositional generalization across attributes, objects, and skills, demonstrating that explicit, object-centric interaction decomposition is key inductive bias for robust control.
Limitations and Future Works FIOC-WM still relies on a pretrained object-centric model for object discovery, and its interaction models primarily generalize to object categories. Addressing these limitations and extending the framework to real-world robotic settings is part of the future work, potentially leveraging recent advances in robot-learning foundation models (88-97].
Acknowledgment
We would like to thank the anonymous reviewers for their helpful comments and suggestions during the review process. We also acknowledge the computational support provided by the IVI servers at the University of Amsterdam and the University HPC centers at City University of Hong Kong.
10

---

References
[1] David Ha and Jürgen Schmidhuber. Recurrent world models facilitate policy evolution. Advances in neural information processing systems (NerulPS), 31, 2018.
Danijar Hafner, Timothy Lillierap, Jimmy Ba, and Mohammad Norouzi. Dream control: Learning behaviors by latent imagination. In International Conference on Learning Represen- tations (ICLR), 2020. URL https: openreview net forum idd-iid-- S110TC4tDS.
[3] Danijar Hafner, Timothy Lillicraa, Mohammad Norouzi, and Jimmy Ba. Mastering atari with discrete world models. In International Conference on Learning Representations (ICIR), 2021. URL. hwwps:ttp//......... wt/poppeevvvviiieeee net. forum ? d=0oabby7bbbu
[4] Danijar Hafner, Jurgis Pasukonis. Jimmy Ba, and Timothy Lillicrap. Mastering diverse domains through world models. arXiv preprint urXiv:2301.04004 2023.
[5] Nicklas Hansen, Xiaolong Wang, and Hao Su. Temporal difference learning for model predictive control. arXiv preprint arXiv:2203.04990 2022.
[6] Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, and James Davidson. Learning latent dynamics for planning from pixels. In International Conference on Machine Learning (ICML), pages 2555-2565. PMLR, 2019.
[7] Philipp Wu, Alejandro Escontrela, Danijar Hafner, Pieter Abbeel, and Ken Goldberg. Day- dreamer: World models for physical robot learning. In Conference on Robot Learning (CoRL), pages 2226 2240. PMLR, 2023.
[8] Siyuan Zhou, Yilun Du, Jiaben Chen, Yandong Li, Dit-Yan Yeung, and Chuang Gan. Ro- bodreamer: Learning compositional world models for robot imagination. arXiv preprint arXiv:2404. 12377, 2024.
[9] Sherry Yang, Yilun Du, Seyed Kamyar Seyed Ghasemipour, Jonathan Tompson, Leslie Pack Kaelbling, Dale Schuurmans. and Pieter Abbeel. Learning interactive real-world simulators. The Twelfth International Conference on Learning Representations, 2024. URL https:
// /oppneeview net forrm/ id FFyTEEgg.U
[10] Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chattopadhyay, Yongxin Chen, Yin Cui, Yifan Ding, et al. Cosmos world foundation model 2025.
platform for physical arXiv preprint arXiv:2501.033:
[11] Anthony Hu, Lloyd Russell, Hudson Yeo, Zak Murez, George Fedoseev, Alex Kendall, Jamie Shotton, and Gianluca Corrado. Gaia-1: A generative world model for autonomous driving. arXiv preprint arXiv:2309. 17080, 2023.
[12] Xiaofeng Wang, Zheng Zhu, Guan Huang, Xinze Chen, Jiagang Zhu, and Jiwen Lu. Drive- dreamer: Towards real- dorvd-drive world models for autonomous driving. In European Conference on Computer Vision, pages 55-72. Springer, 2024.
[13] Lloyd Russell, Anthony Hu, Lorenzo Bertoni, George Fedoseev, Jamie Shotton, Elahe Arani, and Gianluca Corrado. Gaia-2: controllable multi generative world model for au- tonomous driving, arXiv preprint arXiv:2503.200 200223, 2025.
[14] Amir Bar, Gaoyue Zhou, Danny Tran, Trevor Darrell, and Yann LeCun. Navigation world models. arXiv preprint arXiv:2412.03572. 2024.
[15] Aditya Mohan, Amy Zhang, and Marius Lindauer. Structure in deep reinforcement learning:
A survey and open problems. Journal of Artificial Intelligence Research, 79:1167-1236, 2024.
[16] Klemen Kotar, Wanhee Lee, Rahul Venkatesh, Honglin Chen, Daniel Bear, Jared Watrous, Simon Kim, Khai Loong Aw, Lilian Naing Chen, Stefan Stojanov, et al. World modeling with probabilistic structure integration. arXiv preprint arXiv:2509.097:: 2025.
[17] Thomas Kipf, Elise Van der Pol, and Max Welling. Contrastive learning of structured world models. arXiv preprint 191.27911.

---

[18] Linfeng Zhao, Lingzhi Kong, Robin Walters, and Lawson LS Wong. Toward compositional generalization in object-oriented world modeling. In International Conference on Machine Learning (ICML.), pages 26841-26866 PMLR, 2022.
[19] Tongzhou Wang, Simon Du, Antonio Torralba, Phillip Isola, Amy Zhang, and Yuandong Tian. Denoised mdps: Learning world models better than the world itself. In International Conference on Machine Learning, pages 22591-22612 PMLR, 2022.
[20] Atharva Sehgal, Arya Grayeli, Jennifer J. Sun, and Swarat Chaudhuri. Neurosymbolic ground- ing for compositional world models. In The Twelfth International Conference on Learning Representations, 2024. URL https:t////..... oppnrevieeeiveeee/ee/eeeee net/ forum? i d 4Nh.
[21] Russell Mendonca, Shikhar Bahl, and Deepak Pathak. Structured world models from human videos. 2023.
[22] Fan Feng and Sara Magliacane. Learning dynamic attribute-factore world models for efficient multi-object reinforcement learning. Advances in Neural Information Processing Systems (NeurIPS), 36, 2023.
[23] Junyeob Baek, Yi-Fu Wu, Gautam Singh, and Sungjin Ahn. Dreamweaver: Learning composi- tional world models from pixels. arXiv preprint arXiv:2501.14174, 2025.
[24] Simone Parisi, Aravind Rajeswaran, Senthil Purushwalkam, and Abhinav Gupta. The unsur- prising effectiveness of pre-trained vision models for control. In international conference machine learning, pages 17359-17371. PMLR, 2022.
[25] Kaylee Burns, 7ach Witzel, Jubayer Ibn Hamid, Tianhe Yu, Chelsea Finn, and Karol Hausman. What makes pre-trained visual representations successful for robust manipulation? arXiv preprint 2rXiv:2312.12444 2023.
[26] Zichen Cui, Hengkai Pan, Aadhithya lyer, Siddhant Haldar, and Lerrel Pinto. Dynamo: In-domain dynamics pretraining for visuo-motor control. Advances in Neural Information Processing Systems, 37:33933-33961 2024.
[27] Gaoyue Zhou, Hengkai Pan, Yann LeCun, and Lerrel Pinto. Dino-wm: World models on pre-trained visual features enable zero-shot planning. arXiv preprint arXiv:2411.049885 2024.
[28] Yucen Wang, Rui Yu, Shenghua Wan, Le Gan, and De-Chuan Zhan. Founder: foundation models in world models for open-ended embodied decision making. Forty- second International Conference Machine Learning, 2025.
Grounding
[29] Calvin Luo, Zilai Zeng, Yilun Du, and Chen Sun. Solving new tasks by adapting internet video knowledge. In The Thirteenth International Conference on Learning Representations, 2025. URL https: wpw.revieeeeeeeeeeeeeeeeeeeoeewe net forum? (o
[30] Mido Assran, Adrien Bardes, David Fan, Quentin Garrido, Russell Howes, Matthew Muckley, Ammar Rizvi, Claire Roberts, Koustuv Sinha, Artem Zholus, et al. V-jepa 2: Self-supervised video models enable understanding, prediction and planning. arXiv preprint w7Xvv25006.09955, 2025.
[31] Federico Baldassarre, Mare Szafraniec, Basile Terver, Vasil Khalidov, Francisco Massa, Yann LeCun, Patrick Labatut, Maximilian Seitzer, and Piotr Bojanowski. Back to the features: Dino as a foundation for video world models. arXiv preprint arXiv:2507. 19468, 2025.
[32] Daniel Bear, Elias Wang, Damian Mrowca, Felix Jedidja Binder, Hsiao Yu Tung, RT Pramod, Cameron Holdaway, Sirui Tao, Kevin A Smith, Fan- Yun Sun, et al. Physion: Evaluating physical prediction from vision in humans and machines. In Advances in Neural Information Processing Systems (NeurIPS), 2021.
[33] Akihiro Nakano, Masahiro Suzuki, and Yutaka Matsuo. Interaction-based disentanglement of entities for object-centric world models. In The Eleventh International Conference on Learning Representations (ICLR), 2023. URL https: //ap::///opt /ttp//ppeeevvee/ee/eee ew net / forum?
id=JQo2VowqCzz.
12

---

[34] Shiqian Li, Kewen Wu, Chi Zhang, and Yixin Zhu. 1-phyre: Interactive physical reasoning. In The Twelfth International Conference on Learning Representations (ICLR), 2024.
[35] Caleb Chuck, Kevin Black, Aditya Arjun, Yuke Zhu, and Scott Niekum. Granger-causal hierarchical skill discovery. arXiv e-prints, pages arXiv 2306, 2023.
[36] Caleb Chuck, Kevin Black, Aditya Arjun, Yuke Zhu, and Scott Niekum. Granger causal interaction skill chains. Transactions on Machine Learning Research, 2024, ISSN 2835-8856. URL https: / /pwnr//eeeeeeeeeeeee/oeoo net/
met/forumt forum? -eterid-iiAae 1d11A2d2youun...
[37] Zizhao Wang, Jiaheng Hu, Caleb Chuck, Stephen Chen, Roberto Martin-Martin, Amy Zhang, Scott Niekum, and Peter Stone. Skild: Unsupervised skill discovery guided by factor inter- actions. In The Thirty-eighth Annual Conference on Neural Information Processing Systems (NeurIPS), 2024.
[38] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V. Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, Mido Assran, Nicolas Ballas, Wojeiech Galuba, Russell Howes, Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Herve Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski. DINOv2: Learning robust visual features without supervision. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https wpw.reveee.eeeeeeeeeeeeee.eoeeee net wttp///oummmmmme a68Stt 6zFt. Featured Certification.
[39] Suraj Nair, Aravind Rajeswaran, Vikash Kumar, Chelsea Finn, and Abhinav Gupta. R3m: A universal visual representation for robot manipulation. arXiv preprint arXlv:2203.12601 2022.
[40] Abhishek Gupta, Vikash Kumar, Corey Lynch, Sergey Levine, and Karol Hausman. Relay policy learning: Solving long-horizon tasks via imitation and reinforcement learning. In Proceedings of the Conference on Robot Learning (CoRL). PMLR, 2020.
[41] Leslie Pack Kaelbling, Michael L Littman, and Anthony R Cassandra. Planning and acting in partially observable stochastic domains. Artificial intelligence, 101(1-2):99-134 1998.
[42] Norman Di Palo and Edward Johns. Dinobot: Robot manipulation via retrieval and alignment with vision foundation models. In JEEE International Conference on Robotics and Automation (ICRA), 2024.
[43] Yingdong Hu, Renhao Wang, Li Erran Li, and Yang Gao. For pre-trained vision models in motor control, not all policy learning methods are created equal. In International Conference Machine Learning (ICML), pages 13628- 13651. PMLR, 2023.
[44] Wei-Di Chang, Francois Hogan, David Meger, and Gregory Dudek. Generalizable imitation learning through pre-trained representations. arXiv preprint arXiv:2311.09350, 2023.
[45] Xingyu Lin, John So, Sashwat Mahalingam, Fangchen Liu, and Pieter Abbeel. Spawnnet: Learning generalizable visuomotor skills from pre-trained network. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 4781-4787. IEEE, 2024.
[46] Patrik Reizinger, Alice Bizeul, Attila Juhos, Julia E Vogt, Randall Balestriero, Wieland Brendel, and David Klindt. Cross-entropy is all you need to invert the data generating process. In The Thirteenth International Conference on Learning Representations, 2025. URL https: https/////peeeeeeeeeeemmme /ernteeur////reeee di-rhiooxII....
[47] Andrii Zadaianchuk, Maximilian Seitzer, and Georg Martius. Object-centric learning for real-world videos by predicting temporal feature similarities. Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https: wttpr//iepeeeveeeeeee eww.et/
forum? cerummid=t1i forum?dd-t
[48] Francesco Locatello, Dirk Weissenborn, Thomas Unterthiner, Aravindh Mahendran, Georg Heigold, Jakob Uszkoreit, Alexey Dosovitskiy, and Thomas Kipf. Object-centric learning with slot attention. Advances in Neural Information Processing Systems (NeurIPS), 33:
11525-11538. 2020.
13

---

[49] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. The International Conference on Learning Representations (ICLR), 2014.
[50] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXiv: 1807.03748, 2018.
[51] Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares. Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder- decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1724-1734, 2014.
[52] Sindy Löwe, David Madras, Richard Zemel, and Max Welling. Amortized causal discovery: Learning to infer causal graphs from time-series data. In Conference on Causal Learning and Reasoning, pages 509-525. PMLR, 2022.
[53] Inwoo Hwang, Yunhyeok Kwak, Subyung Choi, Byoung- Tak Zhang, and Sanghack Lee. Fine- grained causal dynamics learning with quantization for improving robustness in reinforcement learning. In Pruceedings of the 41th International Conference on Machine Learning (ICML), 2024.
[54] Zizhao Wang, Xuesu Xiao, Zifan Xu, Yuke Zhu, and Peter Stone. Causal dynamics learning for task-independent state abstraction. International Conference on Machine Learning (ICML., 2022.
[55] Carlos Garcia, David M Prett, and Manfred Morari. Model predictive control: Theory and practice-a a survey, Automatica, 25(3):335-348, 1989.
[56] KS Holkar and Laxman M Waghmare. An overview of model predictive control. International Journal of control and automation, 3(4):47-63, 2010.
[57] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv: 1707006777 2017.
[58] Zizhao Wang, Jiaheng Hu, Peter Stone, and Roberto local dependencies. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https: opwnreview.eeeeee////e net/ forum?ide 11p
ELDEN: Exploration via
[59] Carlos Guestrin, Daphne Koller, Ronald Parr, and Shobha Venkataraman. Efficient solution algorithms for factored mdps. Journal of Artificial Intelligence Research, 19:399-468, 2003.
[60] Karina Valdivia Delgado, Scott Sanner, and Leliane Nunes De Barros. Efficient solutions to factored mdps with imprecise transition probabilities. Artificial Intelligence, 175(9-10): 1498-1527, 2011.
[61] Ian Osband and Benjamin Van Roy. Near-optimal reinforcement learning in factored mdps. Advances in Neural Information Processing Systems (NIPS), 27, 2014.
[62] Jaesik Yoon, Yi-Fu Wu, Heechul Bae, and Sungjin Ahn. An investigation into pre-training object-centric representations for reinforcement learning. In International Conference on Machine Learning (ICML), pages 40147-40174. PMLR, 2023.
[63] Ziyi Wu, Jingyu Hu, Wuyue Lu, Igor Gilitschenski, and Animesh Garg. Slotdiffusion: Object- centric generative modeling with diffusion models. Advances in Neural Information Processing Systems, 36:50932-50958 2023.
[64] Jindong Jiang, Fei Deng, Gautam Singh, and Sungjin Ahn. Object-centric slot diffusion. Advances in Neural Information Processing Systems (NeurIPS), 36, 2023.
[65] Jindong Jiang, Fei Deng, Gautam Singh, Minseung Lee, and Sungjin Ahn. Slot state space models. In The Tiirty-eighth Annual Conference on Neural Information Processing Systems (NeurIPS), 2024. URL https: wwwpoppwwwwwweeetteeeee wwwp:eeet/ttdatateeeeee 4XNJW.
14

---

[66] Andrii Zadaianchuk, Maximilian Seitzer, and Georg Martius. Self-supervised visual re- inforcement learning with object-centric representations. In International Conference on Learning Represeniations (ICIR), 2021. URL https: /ttpss//ppppeeveeeeeeee ew net/ forum?
id=xppImxcbow1.
[67] Dan Haramati, Tal Daniel, and Aviv Tamar. Entity-centric reinforcement learning for object ma- nipulation from pixels. The Twelfth International Conference on Learning Representations (ICLR), 2024. URL https::////... opwpreveeeeeee.ee////ep net/ forum? id= 2 wdI.
[68] Richard Li, Allan Jabri, Trevor Darrell, and Pulkit Agrawal. Towards practical multi-object manipulation using relational reinforcement learning. In 2020 IEEE international conference on robotics and automation (ICRA), pages 4051 IEEE, 2020.
[69] Davide Mambelli, Frederik Träuble, Stefan Bauer, Bernhard Schölkopf, and Francesco Lo- catello. Compositional multi-object reinforcement learning with linear relation networks. In ICLR2022 Workshop on the Elements of Reasoning: Objects, Structure and Causality, 2022. URL htwps:ttp..... opw.revieeee...ee//e/ee net forum? BwtpiiddHhHHFSSShPF-
[70] Allan Zhou, Vikash Kumar, Chelsea Finn, and Aravind Rajeswaran. Policy architectures for compositional generalization in control. In ICML Workshop on Spurious Correlations, Invariance and Stability, 2022.
[71] Michael Chang, Alyssa Li Dayan, Franziska Meier, Thomas L. Griffiths, Sergey Levine, and Amy Zhang. Hierarchical abstraction for combinatorial generalization in object rearrangement. In The Eleventh International Conference on Learning Representations (ICI.R), 2023. URL.
https: : openreview net. Eorum? d= FGGGVGG3333999
[72] Zhongwei Yu, Jingqing Ruan, and Dengpeng Xing. Learning causal dynamics models in object-oriented environments. In International Conference on Machine Learning, pages 57597-57638. PMLR, 2024.
[73] Caleb Chuck, Fan Feng, Carl Qi, Chang Shi, Siddhant Agarwal, Amy Zhang, and Scott Niekum. Null counterfactual factor interactions for goal-conditioned reinforcement learning. arXiv preprint TrXiv:2505.03172 2025.
[74] Richard S Sutton, Doina Precup, and Satinder Singh. Between mdps and semi-mdps: A framework for temporal abstraction in reinforcement learning. Artificial intelligence, 112(1-2): 181-211, 1999.
[75] Shubham Pateria, Budhitama Subagdja, Ah-hwee Tan, and Chai Quek. Hierarchical rein- forcement learning: A comprehensive survey. ACM Computing Surveys (CSUR), 1-35) 2021.
[76] Xing Hu, Rui Zhang, Ke Tang, Jiaming Guo, Qi Yi, Ruizhi Chen, Zidong Du. Ling Li, Qi Guo, Yunji Chen, et al. Sausality-driver hierarchical structure discovery for reinforcement learning. Advances in Neural Information Processing Systems (NeurIPS), 35:20064 20076, 2022.
[77] Nicholas Watters, Loic Matthey, Matko Bosnjak, Christopher P Burgess, and Alexander Lerchner, Cobra: Data-efficient model-based rl through unsupervised object discovery and curiosity-driven exploration. arXiv preprint arXiv: arXiv:1905. aeee:090599.555 2019.
[78] Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojeiech Zaremba. Openai gym. arXiv preprint arXiv:1606.01555 011.6000011440 2016.
[79] Chengshu Li, Fei Xia, Roberto Martin, Michael Lingelbach, Sanjana Srivastava, Bokui Shen, Kent Elliott Vainio, Cem Gokmen, Gokul Dharan, Tanish Jain, Andrey Kurenkov, Karen Liu, Hyowon Gweon, Jiajun Wu, Li Fei-Fei, and Silvio Savarese. igibson 2.0: Object-centric simulation for robot learning of everyday household tasks. Aleksandra Faust, David Hsu, and Gerhard Neumann, editors, Proceedings of the 5th Conference on Robot Learning, volume 164 of Proceedings of Machine Learning Research, pages 455 PMLR, 08-11 Nov 2022. URL https:/ / pttpp/eeeoegg...ccoo/ mlr. wwwpppewssseeeeee wwt.:smww.........ee...e..eeetee /aaessvvlee 64 1i22b. html.
15

---

[80] Bo Liu, Yifeng Zhu, Chongkai Gao. Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. Libero: Benchmarking knowledge transfer for lifelong robot learning. Advances in Neural Information Processing Systems (NeurIPS), 36, 2023.
[81] Nicklas Hansen, Hao Su, and Xiaolong Wang. TD-MPC2:
Scalable, robust world models for continuous control. In The Twelfth International Conference on Learning Representations (ICIR), 2024. URL https: / hopppppeevviieeeeeeeeeeeeeeeeeee net/ forum? d=OxhhCstDDDD.
[82] Chengshu Li. Fei Xia, Roberto Martin-Martinn Michael Lingelbach, Sanjana Srivastava, Bokui Shen, Kent Elliott Vainio, Cem Gokmen, Gokul Dharan, Tanish Jain, Andrey Kurenkov, Karen Liu, Hyowon Gweon, Jiajun Wu, Li Fei-Fei, and Silvio Savarese. igibson 2.0: Object- centric simulation for robot learning of everyday household tasks. In Proceedings of the Sth Conference on Robot Learning (CoRL), volume 164, pages 455 465. PMLR, 2022.
[83] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unrea- sonable effectiveness of deep features as perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 386-595, 2018.
[84] Thomas Kipf, Ethan Fetaya, relational inference for interacting systems. In International Conference on Machine Learning (ICML), pages 2688-2697. PMLR, 2018.
Wang, Max Welling, and Richard Zemel. Neural
[85] Colin Graber and Alexander G Schwing. Dynamic neural relational inference. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8513-8522, 2020.
[86] Silviu Pitis, Elliot Creager, and Animesh Garg. Counterfactual data augmentation using locally factored dynamics. Advances Neural Information Processing Systems (NeurIPS), 33: 3976-3990, 2020.
[87] Silviu Pitis, Elliot Creager, Ajay Mandlekar, and Animesh Garg. Mocoda: Model-based counterfactual data augmentation. Advances in Neural Information Processing Systems (NeurIPS), 35:18143-188
2022.
[88] Yilun Du, Mengjiao Yang. Pete Florence, Fei Xia, Ayzaan Wahid, Brian Ichter, Pierre Sermanet, Tianhe Yu, Pieter Abbeel, Joshua B Tenenbaum, al. Video language planning. arXiv preprint arXiv:2310. 10625, 2023.
[89] Yilun Du, Sherry Yang. Bo Dai, Hanjun Dai, Ofir Nachum, Josh Tenenbaum, Dale Schuurmans, and Pieter Abbeel. Learning universal policies via text- ext-guided video generation. Advances in neural information processing systems, 36:9156-911 9172, 2023.
[90] Mengjiao Yang. Yilun Du Kamyar Ghasemipour, Jonathan Tompson, Dale Schuurmans, and Pieter Abbeel. Learning interactive real-world simulators. arXiv preprint a3Xiv:2310.06114, 2023.
[91] Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, et al. Octo: An open source generalist robot policy. arXiv preprint arXiv:2405.12213 2024.
[92] Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Openvla: An open-source vision-language-attinn model. arXiv preprint arXiv:2406.09246. 2024.
[93] Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et pi_0: 0: A iissonllanguage-action flow model for general robot control. arXiv preprini 2rXiv:2410.24164 2024.
[94] Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, et al. =_(0.5]: vision- language-action model with open-world generalization. arXiv preprint arXiv:2504. 16054, 2025.
16

---

[95] Yifan Zhong, Fengshuo Bai, Shaofei Cai, Xuchuan Huang, Zhang Chen, Xiaowei Zhang, Yuanfei Wang, Shaoyang Guo. Tianrui Guan, Ka Nam Lui, al. A survey on vision-language- action models: An action tokenization perspective. arXiv preprint arXiv:2507.01995, 2025.
[96]
Francesco Capuano, Caroline Pascal, Adil Zouitine, Thomas Wolf, and Michel Aractingi. Robot learning: A tutorial. arXiv preprint arXiv:2510.1244 2025.
[97] Kento Kawaharazuka, Jihoon Oh, Jun Yamada, Ingmar Posner, and Yuke Zhu. Vision-language- action models for robotics: A review towards real-world applications. IEEE Access, 2025.
[98] Cheol-Hui Min, Jinseok Bae, Junho Lee, and Young Min Kim. Gatsbi: Generative agent- centric spatio-temporal object interaction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3074-3083, 2021.
[99] Stefano Ferraro, Pietro Mazzaglia, Tim Verbelen, and Bart Dhoedt. Focus: Object-centric world models for robotic manipulation. Frontiers in Neurorobotics, 19:1585386, 2025.
[100] Avinash Kori, Ben Glocker, Bernhard Schölkopf, and Francesco Locatello. Unifying causal and object-centric representation learning allows causal composition. In ICLR 2025 Workshop
World Models: Understanding, Modelling and Scaling.
[101] Zhihong Deng, Jing Jiang, Guodong Long, and Chengqi Zhang. Causal reinforcement learning: A survey. arXiv preprint arXiv:2307.01-522 2023.
[102] Maximilian Seitzer, Bernhard Schölkopf, and Georg Martius. Causal influence detection for improving efficiency in reinforcement learning. Advances in Neural Information Processing Systems (NeurIPS), 94:22905-22918 2021.
[103] Sumedh Sontakke, Arash Mehrjou, Laurent Itti, and Bernhard Schölkopf. Causal curiosity: RI agents discovering self- self-super experiments for causal representation learning. In International conference on machine learning, pages 9848-9858, PMLR, 2021.
[104] Hongye Cao, Fan Feng. Tianpei Yang, Jing Huo, and Yang Gao. Causal information prioritiza- tion for efficient reinforcement learning. arXiv preprint arXiv:2.00.. 10097, 2025.
[105] Hongye Cao. Fan Feng, Meng Fang, Shaokang Dong, Tianpei Yang, Jing Huo, and Yang Gao. Towards empowerment gain through causal structure learning in badel-based rl. arXiv preprint arXiv:2502.10077 2025.
[106] Zizhao Wang, Jiaheng Hu, Peter Stone, and Roberto Martin-Martin. Elden: exploration via local dependencies. Advances in Neural Information Processing Systems (NeurIPS), 36, 2023.
[107] Núria Armengol Urpí, Marco Bagatella, Marin Vlastelica, and Georg Martius. Causal action influence aware counterfactual data augmentation. In Forty-first International Conference on Machine Learning (ICML), 2024.
[108] Haohong Lin, Wenhao Ding, Jian Chen, Laixi Shi, Jiacheng Zhu, Bo Li, and Ding Zhao. Because: Bilinear causal representation for generalizable offline model-based reinforcement learning. In The Thirtt-eighth Annual Conference Neural Information Processing Systems (NeurIPS), 2024.
[109] Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterizzation with gumbel-softmax. In International Conference Learning Representations (ICLR), 2017. URL https:// openrevi wwe.vviewwrrr
ew. net. forummi dtt.rmmmm.rrrk.338e3e.m
[110] Homanga Bharadhwaj, Kevin Xie, and Florian Shkurti. Model-predictive control via cross- entropy and gradient-basee based optimization. In Learning for Dynamics and Control, pages 277-286. PMLR, 2020.
[111] Xun Zheng, Bryon Aragam, Pradeep K Ravikumar, and Eric P Xing. Dags with no tears: Continuous optimization for structure learning, Advances in neural information processing systems, 31, 2018
[112] Phillip Lippe, Taco Cohen, and Efstratios Gavves. Efficient neural causal discovery without acyclicity constraints. arXiv preprint arXiv:2107.10483, 1021.
17

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Yes, we verify the claims of our model in the empirical results on set of simulated robotics and control tasks.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We provide limitation discussions, especially on the usage of specific pre- trained models and the validation only on simulated benchmarks, but not real robots.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
18

---

Answer: [NA]
Justification:
Not theoretical paper.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: Yes, will provide all essential details in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
19

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [No]
Justification:
The code and data will be publicly available after acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please the NeurlPS code and data submission guidelines tbttps://nippi cc/
public guides CodeSubmi ssionPolli oolicy for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / /tnpps.ccppuiii wttpccccpppccccccccccc..c...cc.a ceciiii/guidss//ee / Code Summiuubmiii ss cesioooolli aolicyy)). for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Yes, we provide all details in the appendix, including the simulation setup, hyperparameters, and architecture design.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:All runs are with either or 10 random seeds (with error bars shown in the learning curve figures).
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
20

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
We provide the compute details in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https:/ http:::////uriss..e/
Answer: [Yes]
Justification: This research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:Thie paper presents work at reinforcement learning and world models. While our research has potential societal implications, such as applications in robotics that could misused, we do not identify any specific risks directly arising from our work that require explicit highlighting.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
21

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: There are no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best. faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Original methods are properly cited, and used environments, simulators, and tool are cited.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
22

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, papersw sws thoode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
No new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
23

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: This research does not involve LLMs any important, original, or non- standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https: htt////eeip..p/eeeeee/ee http://iis.cc/ooeeeeeeeeeeeeeeee cc/Confeeeneccceeecceeeeea 2025/ LLM) for what should or should not be described.

---

Appendix
Overview
Extended Related Works
B.I Factored and Object-centric Reinforcement Learning
B.2 Hierarchical Reinforcement Learning
C Overall Framework
C.I Graphical Representation of State Transitions
C.2 Learning the Interaction Models
C,3 Online Policy Learning
6
D Full Results
D.1 Full Results World Modeling
D.2 Policy Learning
D.3 Full Ablations
Network Architectures and syyper-parameters
10
E.1
World Models
10
E.2 Policy Learning
11
E.3 Computes and Training Time
12
Task Details
12
A Overview
In this appendix, we provide supplementary details, extended discussions, and full results for FIOC- WM. Specifically, Section presents in-depth discussion of related work, including factored and object-centric reinforcement learning, hierarchical RL, and causality-inspired RL, all of which are relevant to our approach. Section C offers detailed analysis of world model learning, focusing on the two-level factorization of state transitions (Section C.I), interaction modeling (Section C.2), and policy learning (Section C.3). Sections D, E, and F cover the experimental results, network architectures, and task specifications.
B Extended Related Works
B.1 Factored and Object-centric Reinforcement Learning
Our paper takes inspiration from multiple factorization frameworks in RI.. Factored RI is based on the model of Factored Markov Decision Process (MDP) [59], where structural relationships exist

---

among states, actions, and rewards. This factorization enables efficient policy solutions by leveraging these structural relationships [60, 61].
specific type of factorization, which we also adopt, is object-based, referred to as object-centric reinforcement learning (object-centric RI. [62], where states or observations are grouped into object- centric clusters. In object-centric RL, actions typically target only subset of objects, and rewards are often associated with the states of specific objects or object subsets. This object-wise factorization facilitates more structured and efficient decision-making.
Recent works on object-centric RL can be broadly categorized into two major directions: (i) learning object-centric representation and (ii) modeling the object relations and policy architectures for com- positional generalization. For the first line, approaches focus using object-centric representation learning techniques [48, 63-65] to extract meaningful object-level features from raw observations. These methods, based object-centric representations, focus on policy learning that leverages the object-centric encodings to learn object-centric policies either directly from object-centric repre- sentations [66, 98, 22, 99]. Haramati et al. [67] further Consider the object interactions and learn the policy that has compositional generalization with the model-free framework, learning directly from images. The second line of work develops object-centric policies modeling object relations and policy structures, incorporating inductive biases in the state transition and policy networks. Methods include the use of graph neural networks [68], linear relational networks [69], self-attention, and deep sets [70]. and then they leverage the modeled object-centric states and relational structures to achieve compositional generalization in reinforcement learning [71, 18, 33, 22, 73, 22]. Haramati et al. [67] further leverages object interactions to learn a policy with compositional generalization, using a model-free framework that learns directly from images. Our work combines ideas from both directions, which learn state factorization from the object-centric level cobject-based factorization) and state level (dynamics and static factorization), and model the interactions among objects to achieve compositional generalization. Different from [22, 100J, which explores a more fine-grained object-centric and attribute-level factorization, our approach demonstrates that a simpler dynamic- static factorization of objects is already sufficient for effective world model and policy learning, striking a balance between minimality and expressiveness. Additionally, we go beyond object-centric policies by learning interaction-centric policy that leverages the learned interaction model to facilitate fong-horizon task learning.
B.2 Hierarchical Reinforcement Learning
Our work, using lowelevel and high-level policy for decomposing complex tasks into interaction learning, relevant to hierarchical reinforcement learning (HRL). HRL typically consists of high- level policy (often referred to as the option framework [74], sub-skills, or sub-goals in the literature) and lowelevel policy, enabling the efficient learning of complex RI. tasks (see recent survey [75]).
Within the scope of HRL, the most relevant to our work is the line of research that focuses on learning coal-conditioned hierarchical policies hierarchical skill discovery. Within the scope of HRL, the closest to our work is the line of research that focuses on learning goal-conditioned hierarchical policies or hierarchical skill discovery. Zadaianchuk hierarchical policies with learning object-based hierarchical goals. Hierarchical skill discovery focuses on decomposing complex tasks into object-wise or iojeectinteractton--
[66] proposes the conditiond
components. For instance, Wang et al. [37] use conditional independence testing to identify sub-goals, while Chuck et al. [35] and [76] use Granger causality and causal models to uncover hierarchical structures. Our work shares similarities with [35, 37], particularly in using interactions as sub-skills. However, our work is on learning interaction models jointly with observations and dynamics within the world model directly from high-dimensional inputs, providing more general and unified framework.
B.2.1 Causality-inspired Reinforcement Learning
Closely related to factored RL, causality- based RL aims to learn and leverage causal structures in Markov Decision Processes (MDPs) [101]. Building causal structures within MDPs or world models can enable more efficient exploitation (attattionn021003::0 102-105) or policy learning [54, 106). Additionally, learned causal structures can facilitate counterfactual reasoning, providing benefits such counterfactual data augmentation to improve RL. efficiency [86, 87, 107, 108). Similarly, our work builds an interactive world model that aligns with this line of research, utilizing factored and causal structures in dynamic
3

---

Object-centric Network
Dynamics Network
4p. 9p+1
1rl Observations
I+N
Object factored Latents
Pre-trained Vision Network & Slot Attention
Attribute Encoder Interaction Network
Observation Decoder
Reward Decoder
0,1 Ö7
x/
4
Object-centric Embeddings
suribute-ffcttrre Latents
Fp TH1
Ô
Figure Al: The pipeline of Offline Model Learning (Stage 1) jointly learns the observation function, state factorization, dynamics model, and reward model. Although Fig. 2 includes low-level policy learning as part of Stage I, for clarity, we defer the discussion of low-level policy learning to Stage 2.
models to enhance generalization. Specifically, focus achieving compositional generalization at the levels of objects and their attributes.
The most relevant work to ours is SKILD [37]. which also employs interaction- baseed hierarchical policies. However, there are several key differences. First, we aim develop a general framework that incorporates state factorization, latent interaction-relevant states, and multiple approaches for learning interactions, including directly from pixel observations. In contrast, SKILD primarily focuses on state-based settings and learns interactions using conditional independence testing. Second, while SKILD designed for unsupervised RL and skill discovery. our work focuses on general RL settings, although consider extending it to unsupervised RL in future work. Despite these differences in scope and objectives, we acknowledge the contributions of SKILD, particularly in policy learning, and directly adopt certain components such diversity measurement parameters and settings.
C Overall Framework
Fig. Al illustrates the overall framework for learning FIOC-WN, In this section, we provide more supplementary details on this two-level factorization and interaction learning.
C.1 Graphical Representation of State Transitions
Fig.A2 provides an illustrative example that complements the graphical model in Fig.2 of the main paper, showing state transitions under dynamic graph structures (dashed red edges). We learn two-level factorization of object-level and attribute-level representations, as detailed in Section 3.1. Here, we further motivate the choice this two-level structure. First, decomposing scene into individual objects reduces model complexity, as many objects move independently in most scenarios. By factoring dynamics and static attributes, the model can focus on learning the evolution dynamic properties (e.g.. position, velocity), while separately accounting for how static attributes (e.g., shape, mass) influence those dynamics. Second, to model interactions precisely and compactly, we focus on dynamic features being influenced by the attributes of interacting objects. For example, when two balls collide, it is primarily their dynamic attributes (e.g., velocity) that change, influenced by the full set of features (e.g., mass, shape, velocity) of the other object. This targeted interaction modeling allows the world model to be more precise and interpretable.
As result, the learned world model benefits from this minimal yet sufficient factorization, which also enables more effective policy learning. The effectiveness of this design is further supported by ablation results shown in Table 3 in the main paper.

---

d'
d'
d
H+1
a
a
e
e
d
d'
I+2
Figure A2: The detailed structure (example) of the state transitions with dynamic graph structure.
C.2 Learning the Interaction Models
C.2.1 Learning with Variational Masks
We consider two cases, where the masks are sampled from the approximated categorical distribution or from codebook. In both cases, we have the ELBO
maakk Eg (Gls) [log pe., (s G)] KL 19s.. (G [p()]]
(A1)
The encoders parameterized by graph neural networks, following the settings in neural relational inference [84, 85] and amortized causal discovery frameworks [52]. P(G) is the prior distribution of the graph structures. Specifically,, for the encoder fene, Ou. for each pair of nodes i and j. compute:
Uij Seno,ee ss',
(A2)
For the categorical distribution one, we have:
G ~ Softmax (u.j + g/T),
(A3)
where 7 is the temperature parameter and is Gumbel-distributed noise [109].
For the latent codebook, consider the learned latent embedding, while maintaining codebook prototype vector set e = (ei.... e₁ where represents the number of different graph types (which
can be interpreted interaction patterns). Following [53], apply quantization step to discretize u:
where argmin ||u -e,l2- jE(K]
(A4)
After quantization, we retrieve the corresponding interaction graph by decoding the selected codebook entry into an adjacency matrix G:
G (e2)
(A5)
All additional constraints and optimization techniques are directly adopted from [53].
Foor clarity, in this section, we omit state factorization indices, as well as object and time indices, whenever they are not essential for computation or when they generally apply to factored variables across all objects and time steps.
5

---

C.2.2 Learning with Conditional Independence Testing
Following [54], we use conditional independence testing to identify object interactions. Specifically, for each pair of objects í and j. we compute the conditional mutual information (CMI) of the parameterized dynamics at each time step t:
Ps (s/+11 a4, 81
CML =
E Be.ae.5
log
Ps (6/41 {ac, sc
where represents the state transition dynamics used in our framework. During testing, we use the same parameterized transition model. Instead, after computing CMI, G encoded the adjacent matrix in IR xN directly determining the interaction structure G. Since we employ an object-wise factorization, the number of required CMI tests remains manageable.
C.3 Online Policy Learning
In policy learning tasks, we only use the variational masks for learning the regime variables.
Low-level Policy For the low-level policy that invokes interactions, we consider two approaches: model predictive control (MPC) and RL policies.
At time step f, we are given the target interaction graph at t + kt denoted as G = G Given the learned transition model Pa. target states, we apply MPC to generate a sequence of actions that transitions the system from to t + & while minimizing the discrepancy between the predicted and target states.
s' and u infer the target states sy and sy. Using these inferred
Following Bharadhwajj et al. [1]0], Zhou et al. [27], we employ the cross-entropy method (CEM) for optimization. Specifically, we minimize the mean squared error (MSE) between the predicted state & & and the target state Sy. given an action sequence ,,,...,ar............ a and the learned transition dynamics ps:
LMPC
||s|+| - syl1
(A6)
At each iteration, sample population of action sequences from Gaussian distribution and use the MSE loss to update the mean and covariance of the distribution via stochastic gradient descent.
For RL policies, we train the policy nT (a; ss Sg. Ug) using PPO [57], similarly to the setting in [37].
High-level Policy For high-level policy, we use the diversity measurement, for each object i, we have this intrinsic motivation sample the j that has not been interacted. Hence, for object set S= (s', s², .,....,, s' at each time step f, we fix one object s; and sample a subset S, S S based on the diversity reward rdiv introduced that prioritizes diversity and avoids already-interacee We learn this policy via PPO, using the same way as those in SKILD [106] but with the additional task reward.
objects.
D Full Results
D.1 Full Results on World Modeling
Interaction Learning Table Al gives the full results on the interaction learning. We use Structural Hamming Distance (SHD) verify the effectiveness of capturing interactions. SHD is standard metric widely adopted in the relational inference and causal discovery literature [111, 112, 53]. Results show that the variational method with a categorical distribution the prior achieves the best performance, while the variational approach with codebook latents performs second-bes across domains. Conditional independence testing (CIT) also yields comparable results to these two methods. In contrast, relying solely attention-based mechanisms is not robust across all settings, indicating that directly inferring interactions from the attention matrix insufficient. For precise interaction modeling, the relational inference modules necessary.

---

Algorithm FIOC-WM: Offline World Model and Online Policy Learning (Simplified)
Require: Offline dataset D = {(o,, 4,, rx)]; pre- pre-trained visual encoder Ppee inference model Jøi transition model Ps; reward model pr; interaction model Pu; high-level policy x²; interaction (low-level) policy x² Ensure: Policies n", nf and world model components
1: Stage I: Offline World-Model Learning
2: for all (ot: ag. Tt) Tt:
do
3:
Or f Ppre (Ot)
Di Encode observation with pre-trained vision model P> Infer object-centric latent state dynamics- and attribute-level factors
4: si f (s( Or)
5:
Factorize into sf (di, r²)
6:
Learn reward pr- 81, as)
7: Learn transition (s 8t, dz, Gt)
8: Infer interaction graphs Gi ~ (Ge
9: Train interaction policy r'(a, 8c; G?)
10:
end for
11:
Stage Online Policy Learning
repeat
For each environment rollout episode
13:
Observe environment steps; encode current state 8, using world model
14:
G7 ~ nT" (G) st)
b Select goal/target interaction graph (object-centric subgoal)
d, ~ x (aa st, st,G%)
Sample low-level action conditioned on interaction
16:
Execute az; observe rr, Or+1; update 8++1 Update policies x" n with collected data
17:
18:
until episode terminates
Envs
Variational (Cat.)
Variational (Code.)
CTT
Attention-based
objects
0.0944.04
0.08
0.06_o.02
0.12=0.04
Single-Task Task
5 objects
0.12:0.07
0.15
0.13 0..06
0.20
objects
0.16::011
0.19 20.00
0.21
0.29-10.12
9 objects
1..11
0.31
0.35
0.41:0.14
objects
0.08
0.11
0.08
0.15.0.00
Attri, Gen,
5 objects
0.14
0.17
0.13
0.22:1..11
objects
0.17
0.22
0.25
0.
objects
0.30
0.36
0.35
0.50
objects
0.12
0.13
0.09
0.15
Comp, Gen.
objects
0.19 +0.09
0.20
0.19 20.7
0.29
8 objects
0.23
0.28
0.5 0.5
0.35
10 objects
0.322:0.14
0.37 2.
0.39
0.48
Table A1: Full results on the normalized SHD FIOC with different interaction learning models in predicting the ground truth interactions in Sprites World Environment. The bold values indicate the best-performing method, and the underlined ones are the second-hest.
Reconstruction Table A3 presents the complete LPIPS results across domains. LPIPS [83] (lower is better) evaluates the comparison between the generated frames with ground-truth observations at both the pixel and perceptual levels. Our FIOC model achieves the best performance in most cases, particularly in environments with rich interactions such Sprites, Fetch, and Libero. In other cases where DINO-W HMM performs best, our model remains competitive, with I.PIPS scores within 0.05 of DINO-WM.
D.2 Policy Learning
Table A2 gives the full results on single-task learning, attribute generalization, compositional general- ization, and skill generalization in policy learning tasks. The learning curves of single task learning are given in Fig. A3. For some simpler tasks in these single-task learning scenarios, existing baselines, particularly EIT and TD-MPC2, achieve strong performance, for example, on Fetch. However,

---

(a)
(b)
1.
08
Siyees 1ate
0.0
0.
Swoores
0.9
0.0
0.00
6.25
0.30
1.00
1,25 135
2.00
0.0
0.5
022
10.6
Envinummen strgs (154)
Envinmmeent wtrpe (1S4)
(c)
(d)
0.8
0.8
S000000 Italn
Swyoee Baan
0.0
0.4
0.0
125 stepo (150)
0.0
1:25
Eaviooatoost stopo (130)
0.8
0.8
000000 Hlate
0.4
0.4
0.0
0.0
2.1
5,07
735
300
12.
1500
12.6
2000
0.0
0.0
100 12.3 150
20.0
Etrinnment slage (138)
Revtronment Capp (1M)
(g)
(h)
0.5
Succese
0.4
02
0.0
10
0.0
Envinutment staps (134)
steps (1M)
FIOC
Dreamer-V2 V3
EIT
TD-MPC2
Figure A3: The policy learning curves of single task learning scenarios, (a). Gym Fetch Task 1; (b)
Gym Fetch Task 2; (c) Franka Kitchen Task (d) Franka Kitchen Task 2; (e). i-Gibson Task 1; (f).
i-Gibson Task 2; (g). Libero Task I; (h). Libero Task 2.
our method achieves the best performance in 5 out of 8 tasks, second-best in task, and comparable performance (all 90% success rate) on the remaining tasks, which are relatively simple.
D.3
Full Ablations
Online World Model Fine-tuning For computational efficiency and because our experiments indicate that an offline-trained world model already robust given high-quality offline data, we do not update the world model during the online stage by default. This is an empirical choice rather than a limitation; online updates are feasible. To assess the impact, we perform an ablation comparing performance with and without online world-model updates. Results are shown in Table A4.

---

Envs
FIOC
Dreamer- V3
EIT
TD-MPC2
Gym Fetch (Task
0.95
0.98 0.002
0.93
0.97 10.02
Single-Tase
Gym Fetch (Task
0.98
0.97
0.95
0.96 10.02
Franka Kitchen (Task 1)
0.82 00.
0.75
0.69
0. 0.83=0.08
Franka Kitchen (Task 2)
0.79 20.06
0.68
0. 0.75
0.7320.04
i-Gibson (Task 1)
0.76. 12
0.
0.74 20.4400.14
0. 0.72
i-Gibson (Task 2)
0.72
0.68
0.74
0.72++0:12
Libero (Task 1)
0.11:0.11
0.6540.11
0. 0.78 1.0.09
0.76
Libero (Task 2)
0.74+
0.58
0.69 00007
0.65
Push & Switch
0.91
0.90
0.92 00004
0.95
Attri. Gen.
i-Gibson
0.79+t.13
0.70
-0.14
0.65
Libero
0.76.0.14
0.5
0.73
0.73_2..2
10.12
0.69
Push & Switch
0.86.0.10
0..1
0.83
00.02
0.79
Comp. Gen.
Libero
0.70
0.58
0.65
0.63
20.08
Push & Switch
0.81
0.66
0.73
0.65
00
Skill Gen.
Franka Kitchen
0.73
0.59
0.65
=0.
0.62
Table A2: Policy learning (success rate) of world model in Gym Fetch, Franka Kitchen, i-Gibson, and Libero tasks.
Environment
Dreamer-Ve
TD-MPC2
EIT
DINO-WN
FIOC
Sprites
0.026
0.019
0.006
0.012
0.004
Fetch
0.042
0.039
0.026
0.009
0.007
Kitchen
0.102
0.123
0.096
0.035
0,038
i-Gbison
0.135
0.092
0.085
0.063
0.068
Libero
0.089
0.061
0.040
0.035
0.027
Table A3: Comparison of world models on LPIPS metrics.
Using pre-trained embeddings for Dreamer and TD-MPC Specifically, we adapted both base- lines use DINO embeddings input:
For Dreamer-V3, we replaced the pixel encoder with frozen DINO encoder and used DINO-WM's decoder to reconstruct the observations.
For TD-MPC2, used DINO features to predict actions, terminal values, and rewards via its original decoding heads.
The updated results are shown in Table A5. The results indicate that the pre-trained DINO features improves both Dreamer-Ve V3 and TD-MPC2 performance in some cases, but they still under-perfome compared to FIOC. This suggests that while strong visual representations help, our proposed factor- ization (Stage and policy learning (Stage 2) key contributors the performance gain.
Generalize to more objects To assess the model's ability to generalize to a greater number of objects, train FIOC on environments containing three objects and evaluate on tasks with six and eight objects while keeping the world model fixed in Fetch Env. As shown in Table A6, FIOC achieves strong generalization performance, comparable to or better than baselines such as ETT, and it consistently outperforms Dreamer-V V3 and TD-MPC2 under distribution shifts in object count.
Visualization Rollouts Some visualization rollouts are found the project homepage: https: // //sites. google. com/ http//veeeffo//// fio--wiiowwnn

---

Envs
FIOC (w/o Online Tuning)
FIOC (w/ Online Tuning)
Gym Fetch (Task 1)
0.95 = 0.03
0,96 0.02
Gym Fetch (Task 2)
0.98 = 0.01
0.98 = 0.01
Franka Kitchen (Task 1)
0.82 = 0.04
0.79 = 0.06
Franka Kitchen (Task 2)
0.79 = 0.06
0.82 0.05
i-Gibson (Task 1)
0.76 = 0.12
0.78 = 0.14
i-Gibson (Task 2)
0.72 = 0.10
0.75 = 0.06
Libero (Task 1)
0.81 + 0.11
0.83 + 0.08
Libero (Task
0.74 + 0.09
0.71 0.06
Push & Switch (Attri. Gen.)
0.91 + 0.05
0.95 + 0.08
i-Gibson (Attri. Gen.)
0.79 = 0.13
0.81 0.15
Libero (Attri. Gen.)
0.76 = 0.14
0.68 = 0.09
Push & Switch (Comp. Gen.)
0.86 0.10
0.82 0.12
Libero (Comp. Gen.)
0.70 0.09
0.74 = 0.08
Push & Switch (Skill Gen.)
0.81 = 0.06
0.82 0.08 0.72 = 0.07
Franka Kitchen (Skill Gen.)
0.73 = 0.06
Table A4: Performance comparison of FIOC with and without online world model tuning across different environments. Values are mean + standard deviation.
Envs
FIOC
Dreamer-Vi V3 (DINO/original)
TD-MPC2 (DINO/original)
Kitchen
0.82
0.77 / 0.75
0.79 0.83
i-Gibson
0.76
0.71 0.69
0.73 / 0.72
Libero
0.81
0.69 / 0.65
0.74 / 0.76
Table A5: Comparison FIOC with baselines using DINO features as input. Values indicate task
success rates.
E
Network Architectures and Hyper-parameters
E.1
World Models
Learning the Observation Functions For DINO-v2, we use ViT-Base for cases. For R3M, we use ResNet-50 as backbones for all cases. For slot attention parameters, all settings remain consistent. Following VideoSAUR [47], we transform the original features using a two-layer MI.P with an output dimension equal to the slot dimension. The slot attention module is initialized with randomly sampled slots to group the first-frame features. For subsequent frames, we initialize the slot attention module with the slots from the previous frame, which are additionally transformed using predictor module with GRU recurrent unit in the slot attention grouping.
For the VAE used to learn latent states, we employ a two-layer MLP with hidden size of 256. The specific hyperparameters for different environments are detailed in Table A7.
Learning the Regime Variables For variational masks with categorical distribution, directly adopt the hyperparameters and network design from ACD [52]. However, unlike ACD, we use GRU as the encoder, where the MLP has hidden size of 256 with 3 layers. For the codebook-basee approach, we use an MLP with 3 layers. The hidden layer size is set to 128 for Sprites-World while for other environments, it 256. The number of the centered codes are 16 for Sprites World, 8 for Gym Fetch, and 10 for others. All training hyperparameters follow those specified in [52] and [53].
For conditional independence testing, the threshold hyperparameters set follows: 0.02 for Sprites-World, 0.15 for Gym-Fetch, and 0.05 for other environments. All remaining hyperparameters shared across environments.
Learning the State Transitions The state transitions are modeled using MLP layers with different configurations across environments. Specifically, we use 2-layer MI. with hidden dimensionality 32 and SiLU activation for Sprites- Sprites-World, 2-layer MLP with hidden dimensionality 64 for Gym-
10

---

Envs
FIOC (Ours)
Dreamer-Ve V3
ETT
TD-MPC2
3 objects 0.93
0.96
0.94
0.97
6 objects
0.81
0.54
0.77
0.62
8 objects 0.70
0.44
0.62
0.53
Table A6: Generalization increased object count. Models trained with objects and evaluated with 6 and 8 objects using a fixed world model. FIOC shows strong generalization, matching or exceeding EIT and outperforming Dreamer-V3 and TD-MPC2 under distribution shifts in object count.
Parameter
Values (shared not specified)
Used VIT for DINO
Base
Used ResNet for R3M
ResNet-50
Patch Size
16
Feature Dimension
768
Gradient Norm Clip
0.05
Image Crop/Resize
64 World), 224 (others)
Slots
Number of objects + 2
Iterations for Clustering
3
Slot Dimension
32 (SpritesWorld), 64 (Gym-Fetch), 128 (others)
Latent Dimensions for 8
(SoritesWorld); 10, (Gym-Fetch): 16, 12 (others)
Table A7: Hyperparameterss used in learning observation functions.
Fetch, and 3-layer MLP with hidden dimensionality 128 for other environments. The one-step prediction GRU layer consists of 3 MLP layers, with hidden dimensionality 128 for Sprites- World and 256 for i-Gibson, Gym-Fetch, Libero, and Franka Kitchen. The detailed settings are provided in Table A8. All with the learning rate 3e 4.
Environment
MLP Layers (State Transition)
GRU Layers (One-Step Prediction)
Sprites- World
Sprites-World
2 layers, 32 hidden
3 layers, 128 hidden
Gym-Fetch
2 layers, 64 hidden
3 layers, 256 hidden
i-Gibson
3 layers, 128 hidden
3 layers, 256 hidden
Libero
3 layers, 128 hidden
3 layers, 256 hidden
Franka Kitchen
3 layers, 128 hidden
3 layers, 256 hidden
Table A8: Hyperparameters for state transition modeling across different environments.
Offline RL For offline RL experiments, we use the same hyper-paramen parameters as the online ones. Same DINO-WM [27], we do expert demonstrations and inverse dynamics models to learn the mapping pl [s, 1.+1)
Others
For offline training, we collect 3000 episodes with random actions for Sprites- World. For all other environments, we collect 2000 episodes using pre-trained policies from Dreamer-v3, The hyperparameters for the loss terms are set as {a, 3. n {1, 0.05, 0.1, 0.2), and the learning rate set to 3 10
The detailed data collection settings provided in Table A9.
E.2 Policy Learning
Low-Level Policy For MPC, we use gradient descent with a learning rate of 5 x 10-5 For those using PPO, we set the learning rate to 10-4 with a clip ratio of 0.1. The MLP architecture consists of hidden sizes [256, 256] for Gym-Fetch, while for other environments, we use [512, 512]. Generalized Advantage Estimation (GAE) set to 0.95 for all environments, and the entropy coefficient is 0.I.
11

---

Enviromment
Number of Episodes
Action Strategy
Sprites-Wood World
3000
Random Actions
Gym-Fetch, i-Gibson, Libero, Franka Kitchen
2000
Pre-trained Policies (Dreamer-v3)
Table A9: Offline training settings across different environments.
For high-level policy learning, we use PPO with a learning rate of 1 10-4, The MLP architecture follows the same structure as the low-level policy, with hidden sizes [256, 256] for Gym-Fetch and [512, 512] for other environments. The batch size is 1024 for all.
High-Level Policy
E.3
Computes and Training Time
Compute used for training the FIOC-WM:
For Sprites-World, we use 3 hours on 1x NVIDIA A100;
For Fetch, we use hours on NVIDIA 4090;
For i-Gibson, use hours 6x NVIDIA 4090;
For Libero-object, we use 8 hours on 1x NVIDIA A100;
For Kitchen, we use hours on 1x NVIDIA A100.
Task Details
(a)
(b)
dd
Figure A4: visualization of used benchmarks. From left to right: (a). Sprites-Worle World; (b). OpenAl-
Gym Fetch; (c). Franka Kitchen; (d). i-Gibson; and (e). Libero.
OpenAl Gym Fetch [78] is an environment featuring a Fetch robotic arm capable of manipulating cubes and switches. The tasks involve completing sub-tasks that require pushing switching varying number of objects. For single-task learning, we consider 2-push (Task 1) and 2-switch (Task 2), each with 2 million and 1.5 million training steps. For the attribute generalization task, we consider changing the color of the objects. For the compositional generalization task, we add one object to the push task, making it becoming the 3-push task. For skill one, we consider training both 2-push and 2-switch and compose them together for 2-push + 3-switch. All generalization tasks evaluated with zero-shot generalization (for the skill generalization, we let the agent know the compositional task structure by providing separate rewards).
Franka-kitchan [40] is the environment where the 7-DoF Franka Emika Panda arm needs to perform tasks in the kitchen setup. Here we consider several sequential sub-tasks, such as turning on microwave, moving the kettle, turning on the stove, and turning on the light. For single-task learning, we consider these two tasks:
Task- is Turn on the microwave .Move the kettle Turn on the stove Turn on the light; Task-2 Turn on the microwave Turn on the light Slide the cabinet the right Open the cabinet.
All tasks with 2M training steps. The skill generalization one is Turn the microwave Move the kettle Slide the cabinet to the right- Open the cabinet, evaluating with 0.2M training (10% as the base tasks).
i-Gibson [82] is realistic environment with a simulated Fetch robot operating in everyday household tasks with rich objects and interactions. Similar to the setting in [37], we consider the tasks that
12

---

related to the peach object. The peach can be washed or cut, adding complexity to the tasks. The Task-1 is grasping the peach, Task-2 is cutting with knife. Each is with 20M steps to train. For attribute generalization, we change both the color and the size of the peach and follow the Task-1 setting with 1M adaptation steps.
Libero [80] is a benchmark designed for lifelong robot learning and imitation learning in household and tabletop environments. We focus tasks randomly selected from the task library within libero- object. Task I and Task involve picking two different sets of daily objects (boxes, cubes, and glasses) and moving them to designated basket. The compositional generalization setting introduces objects with different colors and requires picking another randomly selected set of objects and placing them in the basket. The number of objects manipulate for Task and Task while the generalization setting includes 7 objects. Number of training steps are 10M for the base tasks and 1M for the generalization task.
13


Reviews:
[{"review_id": "CY5Yo1AK7Y", "reviewer": "Reviewer_E9iB", "summary": "This work proposes a two-stage method called FIOC. First, authos propose to learn object-centric world model to model relationships between objects explicitly. Then, a low-level policy is trained to achieve desired interactions, and a high-level policy is trained to optimize rewards and supply interaction subgoals to the lower level policy. The authors then test FIOC on a variety of tasks, including SpriteWorld and robotic manipulation tasks, and demonstrate impressive performance.", "questions": "- Q1: How do you get the ground truth interaction structures G to get results shown Figure 6? Do I understand correctly that you only do that for SpriteWorld, and that ground truth G is not required unless you want to measure SHD? Also, how exactly is G represented? Is it an NxN matrix of values between 0 and 1? Or is it discrete?\n- Q2: In the online stage, do you also update the world model with the new data you collect?\n- Q3: Do you train the low-level policy fully inside the learned world model or using online observations? (Q4 and Q3 could be answered if you show the algorithms of different stages of your method)\n - Q4: In line 173, you say you only focus on subsets of objects with size of less than 2. Is that correct? Do you only focus on sets of single objects?\n- Q5: FIOC considers two options for low-level policy: MPC and RL, and two options for graph inference: Variational and CIT. What version do you use in Table 2 results?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 1, "significance": 2, "originality": 2, "strengths_and_weaknesses": "###### Strengths\n- This work proposes an effective way to use observation factorization and object relationship graphs to achieve good performance.\n- The performance is impressive. In particular, the boost in performance on generalization showcases the benefit of explicitly modelling the objects and their relationships;\n\n###### Weaknesses\n- The writing is not clear at all: I struggled to understand what G means exactly and how it's learned, and many other things are unclear as well. See questions. \n\nDue to writing, I do not think this paper shoul dbe accepted. However, I find results quite compelling, and I'm willing to increase my score if authors address my concerns.\n\n###### Typos and nitpicks\nFigure 2 caption: gray node -> gray nodes\n112: ave -> have, the sentence doesn't make sense grammatically\n139: a observation -> an observation\n292: fina -> final\nFigure 6: shaded and non-shaded areas look confusing, I'd choose another color/pattern.", "was_revised": true, "final_justification": "The paper was unclear to me when I first read it, but after the discussion with the authors I appreciate the idea more. I think the presented method is interesting, I find the hierarchical decomposition of interactions to be a promising direction, and therefore believe the paper should be accepted. That said, I still have some concerns regarding the presentation clarity, although the authors promised to address these in the final version.", "rebuttal": "Thank you for your thoughtful review and feedback. We sincerely appreciate your suggestions and your recognition of our methodology and performance. \n\nMost of the concerns relate to writing clarity and a few technical details. For clarity, we propose concrete and executable revisions that can be addressed through targeted clarifications in the camera-ready version. For the technical concerns, we provide essential clarifications and additional evaluations. We hope our responses adequately address your questions, and please let us know if you have any further concerns.\n\n---\n\n> what G means exactly and how it's learned\n\n**R1**: We define $\\mathcal{G} =$ { $G_1, \\ldots, G_T$ } as the sequence of **interaction graphs** that represent the relational structure among objects over time (*Lines 72–73* and *137–138*). Each $G_t$ captures the pairwise interactions between objects at time step $t$, where each edge indicates whether an interaction exists between a pair of objects. Concretely, this can be represented as a binary adjacency matrix of size $N \\times N$, where $N$ is the number of objects in the environment.\n\nTo learn $G_t$, we introduce a **surrogate latent variable** $\\mathbf{u}_t$ that parameterizes the distribution over interaction graphs. This modeling choice allows us to capture the fact that interactions and their underlying distributions may vary over time.\n\nSpecifically, for each object pair $(i, j)$ at time $t$, we encode their latent states $s_t^i$ and $s_t^j$ using an RNN-based encoder to obtain a pairwise embedding:\n$$\n\\mathbf{u}_t^{ij} = f\\_{\\text{enc}, \\phi_u}(s_t^i, s_t^j)\n$$\n(as described in Equation (A2)).\n\nWe then use these embeddings to infer the graph structure $G_t$ in one of two ways:\n\n1. **Sampling edges** using a Gumbel-Softmax distribution over $\\mathbf{u}_t$ (Equation A3), or  \n\n2. **Quantizing** $\\mathbf{u}_t$ into a discrete codebook of interaction types (Equation A4), followed by decoding into a graph via:$G_t \\sim g\\_{\\text{dec}}(e_z)$, where $e_z$ is the prototype vector. \n   (as shown in Equation (A5)).\n\nThese mechanisms are designed to flexibly infer time-varying, discrete graph structures from continuous object states. More details can be found in Lines 146–151 and Appendix C.2. [**For clarity**] In the camera-ready version, we will move the above explanation into the main paper and expand it with additional details from Appendix C.2, as we have one extra page available.\n\n---\n\n> Typos and color suggestion for Fig.6.\n\n**R2**:  [**For clarity**] We will correct all identified typos in the camera-ready version. For Fig. 6, we will use more distinguishable colors. For setrence at 112, we will make it as “We encode the observations using object-centric representation learning built on top of pre-trained models such as DINO-v2 [23] and R3M [24], which have been empirically shown to provide high-quality image understanding capabilities [26–28, 18, 29] and facilitate robotic manipulation tasks [19].”\n\n---\n\n> How do you get the ground truth interaction structures G to get results shown Figure 6? Did you only do that for SpriteWorld, and that ground truth is not required unless you want to measure SHD? Also, how exactly is G represented?\n\n**R3**:  You are correct that ground-truth sequence of interaction graphs $G$ are only used in the SpriteWorld environment for evaluation purposes to compute the nSHD and we do *not* use it during training. During world model learning, we infer the estimated sequence of interaction graphs $\\hat{G}$ in a self-supervised manner by maximizing the overall likelihood and optimizing the loss functions described in Section 3.1.\n\nAs for obtaining the ground-truth $G$ in SpriteWorld, we derive it from visual inspection of the collected videos: if two objects are sufficiently close and exhibit abrupt changes in their trajectories, we label them as interacting. [**For clarity**] We will clarify this process and add the above explanation after Line 268 in the revised version.\n\nFor representation, $G$ is a discrete binary adjacency matrix of size $N \\times$, where $N$ is the number of objects. While the model may produce a continuous-valued matrix during optimization, we threshold it to obtain a binary matrix for evaluation. [**For clarity**] We will add this sentence after Line 73.\n\n---\n\n> In the online stage, do you also update the world model with the new data you collect?\n\n**R4**:  For computational efficiency, and because our experiments show that the offline-trained world model is already robust given high-quality offline data, we do not update the world model during the online stage in our default setting. Please note that this is just an empirical choice, and it is feasible to incorporate online updates. To further investigate this, we conducted an ablation study comparing performance with and without online world model updates. The results are shown below:\n\n|Envs|FIOC (w/o Online Tuning)|FIOC (w/ Online Tuning)|\n|--|-|-|\n|Gym Fetch (Task 1)|0.95±0.03|0.96±0.02|\n|Gym Fetch (Task 2)|0.98±0.01|0.98±0.01|\n|Franka Kitchen (Task 1)|0.82±0.04|0.79±0.06|\n|Franka Kitchen (Task 2)|0.79±0.06|0.82±0.05|\n|i-Gibson (Task 1)|0.76±0.12|0.78±0.14|\n|i-Gibson (Task 2)|0.72±0.10|0.75±0.06|\n|Libero (Task 1)|0.81±0.11|0.83±0.08|\n|Libero (Task 2)| 0.74±0.09|0.71±0.06|\n|Push & Switch (Attri. Gen.)|0.91± 0.05|0.95±0.08|\n|i-Gibson (Attri. Gen.)|0.79±0.13|0.81±0.15|\n|Libero (Attri. Gen.)|0.76±0.14|0.68±0.09|\n|Push & Switch (Comp. Gen.)|0.86±0.10|0.82±0.12|\n|Libero (Comp. Gen.)|0.70±0.09|0.74±0.08|\n|Push & Switch (Skill Gen.)|0.81±0.06|0.82±0.08|\n|Franka Kitchen (Skill Gen.)|0.73±0.06|0.72±0.07|\n\nResults averaged over 5 seeds indicate that there is no significant difference, though we observe slight improvements in most environments for both single-task and generalization settings. [**For clarity**] We will include this comparison as an additional ablation study in the appendix, and clarify the experimental setting with the following description: “We also consider the case where online interaction data is used to further update the world model components during policy learning or planning. The results are given in Appendix D.” Additionally, we will emphasize in the main paper that “we fix the learned world model during policy learning and planning”, and will add this clarification after Line 150 to avoid confusion about whether the model is updated online by default.\n\n---\n\n> Do you train the low-level policy fully inside the learned world model or using online observations? \n\n**R5**:  Thank you for the question. We train the low-level policy during the **offline training stage**, and then **fine-tune it during the online learning stage** using the newly collected interaction data. [**For clarity**] We will explicitly state at Line 167: “We learn the low-level policy during world model learning (Stage 1), and then fine-tune it with online data during Stage 2, where the policy is updated each time new interaction data becomes available.”\n\nWe will provide the algorithm in the appendix (here is a sketch):\n\n**Offline World Model and Online Policy Learning Framework**\n\n---\n\n**Stage 1: Offline Model Learning**  \nFor each $(o_t, a_t, r_t) \\in \\mathcal{D}$:\n- Encode observation: $\\hat{o}_t \\leftarrow p\\_\\text{pre-trained}(o_t)$  \n- Infer latent state: $s_t^i \\leftarrow q_\\phi(s_t^i \\mid \\hat{o}_t)$  \n- Factorize latent state: $s_t^i = (d_t^i, c_t^i)$  \n- Learn reward model: $p_r(r_t \\mid s_t, a_t)$  \n- Learn transition model: $p_s(s_{t+1} \\mid s_t, a_t, G_t)$  \n- Infer interaction graphs: $p_u(G_t \\mid s_t)$  \n- Train interaction policy: $\\pi^l(a_t \\mid s_t, G_t^g)$  \n\n---\n\n**Stage 2: Online Policy Learning**  \nFor each environment rollout episode:\n- Receive current state $s_t$ from world model encoder with input environmental steps  \n- Select goal graph: $G_t^g \\sim \\pi^h(G_t^g \\mid s_t)$  \n- Sample action from interaction policy: $a_t \\sim \\pi^l(a_t \\mid s_t, G_t^g)$  \n- Execute $a_t$ in environment, observe $r_t$, $o_{t+1}$  \n- Update policies $\\pi^h$, $\\pi^l$ using collected data  \n\n---\n\n\n> In line 173, you say you only focus on subsets of objects with size of less than 2. Is that correct? Do you only focus on sets of single objects?\n\n**R6**:  Yes, at each step, we focus on a small subset of objects, typically one or two, to induce diverse interactions for them. This does not mean we only consider single-object interactions overall. Instead, we adopt an iterative strategy, where over multiple steps, we cover all objects by sequentially targeting different subsets. This is an empirical design choice that balances interaction diversity and sample efficiency.\n\n---\n\n> FIOC considers two options for low-level policy: MPC and RL, and two options for graph inference: Variational and CIT. What version do you use in Table 2 results?\n\n**R7**:  We use **variational masks** for the results reported in Table 2. Specifically, for all methods (excluding those evaluated under nSHD), we adopt variational masks to infer the interaction structures. For downstream control, we apply **MPC** for the Gym-Fetch and Franka-Kitchen environments, and use **PPO** for LIBERO and iGibson tasks.\nWe also conducted an ablation study to evaluate the impact of using MPC or PPO on all environments. The results of this ablation are shown below. The results indicate that using either PPO or MPC is feasible with similar final success rates. [**For clarity**] We will add this Table in Appendix D. \n|Environment|MPC|PPO|\n|-|-|-|\n|Fetch (Task 1)|0.95±0.03|0.96±0.03|\n|Fetch (Task 2)|0.98±0.01|0.92±0.05|\n|Kitchen (Task 1)|0.82±0.04|0.79±0.06|\n|Kitchen (Task 2)|0.79 ± 0.06|0.76±0.03|\n|i-Gibson (Task 1)|0.72±0.09|0.76±0.12|\n|i-Gibson (Task 2)|0.74±0.13|0.72±0.10|\n|Libero (Task 1)|0.73±0.05|0.81±0.11|\n|Libero (Task 2)|0.76±0.06|0.74±0.09|\n\n---\nWe hope these clarifications address your concerns. If so, we would be truly grateful if you would consider adjusting your rating accordingly. Thank you again for your time and efforts!"}, {"review_id": "phh3hEXmGx", "reviewer": "Reviewer_VDhr", "summary": "This paper proposes the FIOC-WM  framework for learning structured world models that capture both object-level representations and their interactions from high-dimensional visual observations. The core idea is to factor the world model state space across individual objects, decomposing each object into dynamic (time-varying) and static (time-invariant) attributes via DINO-v2, R3M and Slot Attention, and modeling their interactions as sparse, time-varying graphs. FIOC-WM consistently outperforms baselines like DreamerV3, TD-MPC2, and DINO-WM across metrics such as LPIPS, success rate, and generalization gap.", "questions": "1.  Is the same offline training setup used for baseline models such as DreamerV3 and TD-MPC2? Please clarify whether DreamerV3 and TD-MPC2 are also trained in the offline setting using the same dataset as FIOC-WM\n\n2. You mention using both Model Predictive Control (MPC) and Proximal Policy Optimization (PPO) for training the low-level policy. Under what conditions is each method used?\n   * When PPO is applied, is it trained using imagined rollouts from the learned world model (i.e., model-based PPO)?\n   * How is the action sampling process implemented when using MPC with the Cross Entropy Method (CEM)? Specifically:\n\n     * What are the hyperparameters (e.g., number of gradient descent steps, population size)?\n     * Could you provide implementation details or pseudocode of the policy training?\n\n3.  How is the prior distribution $P(G)$ over graph structures defined?  What learning procedure is used to infer the graph structures? This need to be clarified.\n\n4. Equation (6) defines the reward prediction loss, but it is unclear how this learned reward model is utilized during policy training or planning. Is the predicted reward used in MPC rollout evaluation or in PPO optimization?", "limitations": "The model assumes a fixed number of object slots. It is unclear how it generalizes to environments where the number of objects changes, or where previously unseen objects appear.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. The paper proposes a principled factorization of the state representation into dynamic (time-varying) and static (time-invariant) components, and learns their respective dynamics accordingly. This decomposition is intuitive and aligns well with the physical properties of real-world environments.\n\n2. The use of sparse, time-varying graphs to model interactions between objects provides a structured and interpretable inductive bias for object-centric model-based reinforcement learning, enabling more accurate prediction of environment dynamics.\n\n3. The hierarchical policy design—where the high-level policy plans over interaction graphs and the low-level policy executes them—effectively leverages the compositional nature of learned object interactions, resulting in improved performance on long-horizon and generalization tasks.\n\nWeaknesses:\n\n1. While the predictive quality of future observations is quantitatively evaluated using the LPIPS metric, the paper does not provide qualitative visualizations comparing long-term predictions between FIOC-WM and baseline methods. Including side-by-side image rollouts (e.g., from Table A3) would help illustrate in which scenarios FIOC-WM makes accurate predictions and where baselines fail. Such visual comparisons are critical to support the claim of improved long-term predictive fidelity.\n\n2. The paper introduces a surrogate variable ut to model the distribution over interaction graphs given the latent state st, but the role and interpretation of this variable remain unclear. The learning process for the interaction graph, including how ut  is optimized and how it influences the transition dynamics, is under-explained and would benefit from additional clarification.\n\n3. The motivation for adopting two specific approaches—(i) variational mask learning and (ii) conditional independence testing—for learning the state transition distribution is not sufficiently justified. It is unclear why these methods were chosen over other alternatives or what specific advantages they offer in the context of object-centric interaction modeling.\n\n4. Although FIOC-WM shows better performance than baselines in generalization tasks (e.g., Fig. A3), it benefits from pre-trained visual encoders (DINO-v2, R3M), while most baselines do not. This raises concerns about the fairness of the comparison. It remains unclear how much of the performance gain stems from the proposed factorized world model itself versus the use of strong pre-trained features. An ablation comparing against DreamerV3 or TD-MPC2 with access to the same pre-trained encoder would help isolate the contribution of FIOC-WM.", "was_revised": true, "final_justification": "Weaknesses 4, as well as Questions 1-4 and the limitation, have been addressed in the rebuttal. However, Weakness 1, 2, 3 remains unaddressed.", "rebuttal": "Thank you for your careful review. We appreciate the detailed comments and your recognition of our model. Most of the concerns are centered around technical details, and we provide clarifications and additional evaluation results for each point below.\n We hope these responses address your concerns. For points requiring clarification, we outline concrete plans for how they will be incorporated into the camera-ready version. The additional evaluations further support the soundness of our technical design. Please let us know if you have any remaining questions or concerns.\n\n---\n\n> No qualitative visualizations comparing long-term predictions between FIOC-WM and baseline methods. \n\n**R1**: We agree that qualitative visualizations of pixel observations would be useful. Since in the rebuttal we cannot add figures anymore, we will not be able to provide them here, but in the camera-ready version, we will include side-by-side image rollouts in the appendix to compare long-term predictions made by FIOC-WM and baseline methods.\n\n---\n\n> Role of [ $u_t$] and clarification of learning process\n\n**R2**: The $\\mathbf{u}_t$ variable represents the latent parameters of the distribution from which the interaction graph is sampled at each timestep, which allows us to model the fact that the interaction graphs (and their distributions) are time-varying. \n\nSpecifically, for each object pair $(i, j)$ at time $t$, we encode their latent states $s_t^i$ and $s_t^j$ using an RNN-based encoder to obtain a pairwise embedding:\n$$\n\\mathbf{u}_t^{ij} = f\\_{\\text{enc}, \\phi_u}(s_t^i, s_t^j)\n$$\n(as described in Equation (A2)).\n\nWe then use these embeddings to infer the graph structure $G_t$ in one of two ways:\n\n1. **Sampling edges** using a Gumbel-Softmax distribution over $\\mathbf{u}_t$ (Equation A3), or  \n\n2. **Quantizing** $\\mathbf{u}_t$ into a discrete codebook of interaction types (Equation A4), followed by decoding into a graph via:$G_t \\sim g\\_{\\text{dec}}(e_z)$, where $e_z$ is the prototype vector. \n   (as shown in Equation (A5)).\nIn the camera-ready version, we will put the above explanation into the main paper as we have one extra page available.\n\n---\n\n> prior distribution P(G) over graph structures defined? What learning procedure is used to infer the graph structures?\n\n**R3**: The prior distribution $P(G)$ is defined as an independent Bernoulli distribution over each edge:\n$\nG_{ij} \\sim \\text{Bernoulli}(p),$ where $p \\in (0, 1)$ controls the sparsity level. We set $p \\ll 0.5$ (e.g., $p = 0.1$) to encourage sparse and interpretable interaction graphs by biasing toward no-edge configurations.\n\nIn the camera-ready, we will clarify this in Section 3.1 (another paragraph after Line 151) by explicitly describing the sparsity prior and summarizing the graph inference procedure from state encoding to discrete graph generation.\n\n---\n\n> The motivation for adopting two specific approaches—(i) variational mask learning and (ii) conditional independence testing—for learning the state transition distribution is not sufficiently justified.\n\n**R4**: We adopt variational masking and CIT because both are well-established structure learning strategies. Variational methods benefit from scalability and can be efficiently trained via likelihood matching, while CIT offers statistical grounding for inferring dependency structures. We also note that, due to the flexibility of our framework, it can accommodate any off-the-shelf structure learning method.\nTo further support our design choice, we additionally evaluated Dynotears [1], a strong baseline for causal structure learning [2]. We use it on the Sprites-World environment with 3, 5, and 9 objects. The results are shown below:\n| # Objects | Variational (Cat.) | Variational (Code.) | CIT           | Dynotears     | Attention-based |\n|-----------|---------------------|----------------------|---------------|---------------|------------------|\n| 3         | 0.09        | 0.08      | 0.06 | 0.09   | 0.12    |\n| 5         | 0.12        | 0.15      | 0.13     | 0.15   | 0.20      |\n| 9         | 0.27         | 0.31      | 0.35     | 0.36   | 0.41      |\n\nThe results show that DynoTears performs similarly to CIT, but **worse than variational methods** across all object counts. This supports our use of variational masking and CIT as effective and principled choices for object-centric interaction modeling. To clarify our choice, we will add more results for DynoTears for other settings in the Appendix of the revised paper. \n\n[1] Pamfil, Roxana, et al. \"Dynotears: Structure learning from time-series data.\"AISTATS 2020.\n\n[2] Vowels, Matthew J., Necati Cihan Camgoz, and Richard Bowden. \"D’ya like dags? a survey on structure learning and causal discovery.\" ACM Computing Surveys 55.4 (2022): 1-36.\n\n---\n\n> Pre-trained models for Dreamer and TD-MPC\n\n**R5**: Thank you for the suggestion. We originally did not include this comparison because Dreamer-V3 is designed to reconstruct pixel-level observations, while TD-MPC2 predicts actions, terminal flags, and rewards directly from raw pixels. However, to better isolate the contribution of our proposed factorized world model from the benefits of pre-trained visual features, we implemented a fairer comparison.\n\nSpecifically, we adapted both baselines to use DINO embeddings as input:\n- For Dreamer-V3, we replaced the pixel encoder with a frozen DINO encoder and used DINO-WM's decoder to reconstruct the observations.\n- For TD-MPC2, we used DINO features to predict actions, terminal values, and rewards via its original decoding heads.\n\n\nThe updated results are shown below:\n\n|Envs | FIOC | Dreamer-V3 (DINO/original)|TD-MPC2 (DINO/original)|\n|-|-|-|-|\n|Kitchen|0.82|0.77/0.75|0.79/0.83|\n|i-Gibson|0.76|0.71/0.69|0.73/0.72|\n|Libero|0.81| 0.69/0/.65|0.74/0.76|\n\nUsing pre-trained DINO features improves both Dreamer-V3 and TD-MPC2 performance in some cases, but they still underperform compared to FIOC. This suggests that while strong visual representations help, our proposed **factorized world model and hierarchical policy structure** are key contributors to the performance gain.\n\n---\n\n>  Is the same offline training setup used for baseline models such as DreamerV3 and TD-MPC2? \n\n**R6**: Yes, all are trained on the same dataset as we mentioned in Table A6.\n\n---\n\n>  You mention using both MPC and PPO for training the low-level policy. Under what conditions is each method used?\n\n**R7**: We use **variational masks** for the results reported in Table 2. Specifically, for all methods (excluding those evaluated under the nSHD metric), we adopt variational masks to infer the interaction structures. For downstream control, we apply **MPC** for the Gym-Fetch and Franka-Kitchen environments, and use **PPO** for LIBERO and iGibson tasks.\nWe also conducted an ablation study to evaluate the impact of using MPC or PPO on all environments. The results of this ablation are shown below. The results indicate that using either PPO or MPC is feasible with similar final success rates. We will add this Table in Appendix D. \n|Environment|MPC|PPO|\n|-|-|-|\n|Gym Fetch (Task 1)|0.95±0.03|0.96±0.03|\n|Gym Fetch (Task 2)|0.98±0.01|0.92±0.05|\n|Franka Kitchen (Task 1)|0.82±0.04|0.79±0.06|\n|Franka Kitchen (Task 2)|0.79 ± 0.06|0.76±0.03|\n|i-Gibson (Task 1)|0.72±0.09|0.76±0.12|\n|i-Gibson (Task 2)|0.74±0.13|0.72±0.10|\n|Libero (Task 1)|0.73±0.05|0.81±0.11|\n|Libero (Task 2)|0.76±0.06|0.74±0.09|\n\n---\n\n> When PPO is applied, is it trained using imagined rollouts from the learned world model (i.e., model-based PPO)?\n\n**R8**:  Yes, we used both the imagined rollouts and the rollouts from the replay buffer to learn the PPO policy.\n\n---\n\n> How is the action sampling process implemented when using MPC with the Cross Entropy Method (CEM)? What are the hyperparameters (e.g., number of gradient descent steps, population size)?\n\n**R9**: We implement MPC using the Cross Entropy Method (CEM) with the following hyperparameters:\n- CEM iterations: 5 for Fetch, Franka, and i-Gibson; 8 for Libero\n- Population size: 500 for Fetch, 1000 for others\n- Planning horizon: 10 for Fetch, 20 for Franka, i-Gibson, and Libero\nWe will add these in the camera-ready Appendix E.2 as an individual table and include an algorithmic pseudocode as well in the appendix. \n\n---\n\n> Eq6 defines the reward prediction loss, but it is unclear how this learned reward model is utilized during policy training or planning. Is the predicted reward used in MPC rollout evaluation or in PPO optimization?\n\n**R10**: The reward prediction model is used during Stage 1 for learning the world model. Specifically, we supervise it using ground-truth rewards from the offline dataset, following the same setup as in Dreamer-v3 and TD-MPC2. \n\n>  The model assumes a fixed number of object slots. It is unclear how it generalizes to environments where the number of objects changes, or where previously unseen objects appear.\n\n **R11**: To assess the model's ability to generalize to a greater number of objects, we train on environments with 3 objects and evaluate on tasks involving 6 and 8 objects with the fixed world model. As shown in the table below, FIOC achieves strong generalization performance, comparable to or better than baselines like EIT, and outperforms Dreamer-V3 and TD-MPC2 under distribution shifts in object count.\n\n| Envs        | FIOC (Ours) | Dreamer-V3 | EIT   | TD-MPC2 |\n|-------------|-------------|------------|-------|---------|\n| 3 objects   | 0.93        | 0.96     | 0.94  | **0.97** |\n| 6 objects   | **0.81**      | 0.54       | 0.77 | 0.62    |\n| 8 objects   | **0.70**    | 0.44       | 0.62  | 0.53    |\n\n---\n\nWe hope these can address the concerns. Please let us know if you have any further questions. Thank you again for your time and efforts!"}, {"review_id": "AjnjLezLxi", "reviewer": "Reviewer_Kuio", "summary": "This paper introduces FIOC-WM, a framework for object-centric model-based reinforcement learning that learns an object-centric world model and hierarchical policies in two separate stages: offline world model learning and online policy learning. By decomposing environment dynamics into object-level interactions and attribute-level states, world model is able to capture both static and dynamic properties. It begins by extracting object-centric representations directly from pixel observations using pre-trained visual encoders (DINO-v2, R3M) and the slot attention method (based on Videosaur), followed by variational autoencoders with contrastive losses to disentangle static and dynamic attributes. Leveraging this two-level factorization, a hierarchical policy framework is used: the high-level policy selects interaction graphs indicating which objects should interact, and the low-level policy executes these interactions using either MPC or PPO. The approach is evaluated on several robotic manipulation and embodied AI benchmarks (OpenAI Gym Fetch, iGibson, Libero), comparing against model-based RL methods (DreamerV3, TD-MPC2), model-based planning (DINO-WM) and prior object-centric RL baselines (FOCUS, EIT), FIOC-WM consistently achieves superior world model accuracy and better policy performance on most benchmarks.", "questions": "1. The paper mentions using both PPO and MPC frameworks for low-level policies. Are there any recommendations or insights on when to prefer PPO over MPC, or vice versa? If time permits, it would be valuable to see ablation exploring this comparison.\n2. The paper claims that the method can handle long-horizon tasks. How long exactly? Would it be feasible for environments requiring thousands or even hundreds of thousands of steps? Additionally, does the performance degrade as the horizon length increases?\n3. As stated in the original Videosaur paper, the quality of slot extraction tends to degrade with longer video lengths. Has this limitation affected the performance of FIOC-WM in your experiments?", "limitations": "Yes. Limitations are briefly mentioned in the section 6", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "##### Strengths:\n\n1. Policy Performance Across Benchmarks. FIOC-WM outperforms or matches strong baselines (DreamerV3, TD-MPC2, DINO-WM, FOCUS, EIT) on several robotic benchmarks, especially in policy learning, demonstrating its practical advantage.\n2. Generalization and Compositionality. The model shows robustness across attribute generalization, compositional tasks, and skill recomposition, highlighting its strength in tasks involving unseen combinations of known components.\n3. The paper is well written, with some clarity remarks, listed in Weaknesses section.\n\n\n##### Weaknesses:\n\n1. There is no visualization of the slot mappings to the environment observations. Including such visualizations would help clarify that the method's performance is driven by object-centric representations rather than unstructured data. For example, all important objects might otherwise be mapped to a single slot.\n2. Some essential details are missing from the paper. The paper notes that either MPC or PPO could be used for low-level policies in the second step, but does not indicate which was used in the experiments. Clarifying these points would improve transparency and reproducibility.\n3. The architecture is difficult to understand. Including schematic diagrams of the dynamics model and the attributes VAE would greatly improve clarity and help to better understand the structure of the world model.\n\n4. In Appendix E.3, it appears that the Videosaur training time is not included. For the sake of fairness, it would be important to account for the slots attention training time as well, particularly since it needs to be trained separately for each environment.\n\n5. It seems that Table A4 is missing the hidden dimensions for the MLP decoder used in Videosaur.", "was_revised": true, "final_justification": "The paper initially had issues with clarity and lacked details about the proposed method. However, these concerns were addressed in the authors’ rebuttal through additional experiments, and the authors have promised to resolve the remaining clarity issues in the camera-ready version. Therefore, I have increased the clarity score by one point.", "rebuttal": "We thank you for the detailed and constructive review, and we appreciate your recognition of our model contributions and writing. Most of the questions pertain to technical details, which we address through clarifications and additional experiments provided below. In the camera-ready version, we will incorporate these clarifications and include visualizations at the pixel level. We hope these responses address your concerns.\n\n---\n\n> There is no visualization of the slot mappings to the environment observations. \n\n**R1**: We agree that visualizing slot-to-object mappings would help clarify the effectiveness of our object-centric representations. Since in the rebuttal we cannot add figures anymore, we will not be able to provide them here, but in the camera-ready version, we will include such visualizations (e.g., reconstructed images and slot masks). \n\nWhile our current evaluation focuses on probing results (Fig. 5(a)) to show disentanglement of factors like position, velocity, and color, these fine-grained results suggest that object-level separation is also achieved. This is further supported by the use of the Videosaur backbone, which has shown strong object-wise disentanglement in prior work. That said, we acknowledge that a direct qualitative assessment would strengthen this point, and we will ensure that these visualizations are included in the final version.\n\n[1] Zadaianchuk, Andrii, Maximilian Seitzer, and Georg Martius. \"Object-centric learning for real-world videos by predicting temporal feature similarities.\" NeurIPS 2023. \n\n---\n\n> Some essential details are missing from the paper. The paper notes that either MPC or PPO could be used for low-level policies in the second step, but does not indicate which was used in the experiments. \n\n**R2**: Thank you for pointing this out. We apply MPC for the Gym-Fetch and Franka-Kitchen environments, and PPO for LIBERO and iGibson tasks. We will add this clarification after Line 220 in the revision.\n\nAdditionally, we conducted an ablation study to compare the use of PPO and MPC across all environments. The results, which show comparable final success rates, confirm that both approaches are feasible. We will include this ablation table in Appendix D.\n\n|Environment|MPC|PPO|\n|-|-|-|\n|Gym Fetch (Task 1)|0.95±0.03|0.96±0.03|\n|Gym Fetch (Task 2)|0.98±0.01|0.92±0.05|\n|Franka Kitchen (Task 1)|0.82±0.04|0.79±0.06|\n|Franka Kitchen (Task 2)|0.79 ± 0.06|0.76±0.03|\n|i-Gibson (Task 1)|0.72±0.09|0.76±0.12|\n|i-Gibson (Task 2)|0.74±0.13|0.72±0.10|\n|Libero (Task 1)|0.73±0.05|0.81±0.11|\n|Libero (Task 2)|0.76±0.06|0.74±0.09|\n\n---\n\n> The architecture is difficult to understand. Including schematic diagrams of the dynamics model and the attributes VAE would greatly improve clarity and help to better understand the structure of the world model.\n\n**R3**: Thank you for the suggestion. The dynamics model and attribute VAE are currently shown as modules in Fig. 3. To improve clarity, we will include two additional schematic diagrams illustrating their input/output flow and network structures in Appendix D of the camera-ready. \n\nFor reference, the detailed loss functions are provided in Section 3.1, and the corresponding hyperparameters are listed in Appendix E.1.\n\n---\n\n> In Appendix E.3, it appears that the Videosaur training time is not included. For the sake of fairness, it would be important to account for the slots attention training time as well, particularly since it needs to be trained separately for each environment.\n\n**R4**: Thank you for the suggestion. You are right that for a fair comparison, the training time of the Slot Attention (Videosaur) module should be reported. We have now included the training times for each environment (all measured on an A100 GPU with 40GB memory):\n- Sprites-World: 1 hour\n- Fetch: 4 hours\n- iGibson: 5 hours\n- Libero: 5 hours\n- Franka-Kitchen: 4.5 hours\n\nWe will add this information to Appendix E.3 in the camera-ready version.\n\n---\n\n> It seems that Table A4 is missing the hidden dimensions for the MLP decoder used in Videosaur.\n\n**R5**: Thank you for catching this. The hidden dimensions for the MLP decoder in Videosaur were omitted from Table A4. We use:\n- iGibson, Libero, Franka-Kitchen: 4-layer MLP with 1024 hidden units\n\n\n- Fetch: 3-layer MLP with 512 hidden units\n\n\n- Sprites-World: 2-layer MLP with 256 hidden units\n\n\nThese settings are chosen based on the complexity of each environment. We will update Table A4 accordingly in the camera-ready version.\n\n---\n\n> The paper mentions using both PPO and MPC frameworks for low-level policies. Are there any recommendations or insights on when to prefer PPO over MPC, or vice versa? If time permits, it would be valuable to see ablation exploring this comparison.\n\n**R6**: We choose MPC for tasks with relatively simple semantics and shorter horizons (e.g., Gym-Fetch, Franka-Kitchen), and PPO for more complex environments involving long-horizon goals and rich semantics (e.g., LIBERO, iGibson). This decision balances efficiency and learning flexibility. As mentioned previously, we also include an ablation comparing PPO and MPC across environments, which shows that both are feasible with similar final performance. We will add these insights and the corresponding table in Appendix D.\n\n---\n\n> The paper claims that the method can handle long-horizon tasks. How long exactly? Would it be feasible for environments requiring thousands or even hundreds of thousands of steps? Additionally, does the performance degrade as the horizon length increases?\n\n**R7**: By “long-horizon,” we refer to the method’s ability to solve tasks with complex temporal structure by decomposing them into meaningful sub-tasks using learned interaction structures and disentangled features. The actual episode length depends on when the agent completes the task, rather than being fixed.\nWhile we have not explicitly tested environments requiring hundreds of thousands of steps, our hierarchical framework is designed to handle increasing task complexity through temporal abstraction. To support this, we evaluate variants of the Franka-Kitchen environment with different numbers of sub-tasks. For example:\n- Task 1: Open the microwave → Move the kettle → Turn on the stove → Turn on the light (4 sub-tasks)\n- Task 2:  Open the microwave → Turn on the light → Slide the cabinet → Open the cabinet → Turn on the top burner (5-subtasks)\n- Task 2:  Open the microwave →  Move the kettle → Turn on the light → Turn on the stove → Open the cabinet → Turn on the top burner  (6-subtasks)\n\n\n| # Sub-tasks        | Ours (FIOC)      | TD-MPC2        | \n|-------------|-----------------|----------------|\n| 4           | **0.82 ± 0.04**  | 0.83 ± 0.03    | \n| 5           | **0.76 ± 0.02**  | 0.72 ± 0.05    | \n|  6         | **0.70 ± 0.02**  | 0.61 ± 0.03    |\n\nThese results show that while both methods perform comparably on moderate tasks (4 sub-tasks), **FIOC outperforms TD-MPC2 (best-performing baseline for this task) as task complexity increases**, showing better scalability and robustness for long-horizon tasks. This supports our claim that the hierarchical design and object-centric representations in FIOC enable more effective temporal abstraction.\n\n---\n\n> As stated in the original Videosaur paper, the quality of slot extraction tends to degrade with longer video lengths. Has this limitation affected the performance of FIOC-WM in your experiments?\n\n**R8**: In our experiments, we did not observe significant degradation in slot extraction quality with longer video lengths. This is likely because the environments we consider are not as semantically complex or visually diverse as the long-form natural videos (e.g., YouTube-VIS) used in the original Videosaur paper. Most robotic control and embodied AI environments involve more structured and repetitive dynamics. That said, we acknowledge this known limitation of Videosaur and will add a note in Appendix E.1 stating: “*While slot extraction quality may degrade over very long or highly diverse videos, we did not observe such issues in the structured environments used in our experiments.*”\n\n---\n\nThank you again for your positive feedback and insightful questions! Please let us know if you have any further questions!"}, {"review_id": "Up38XGJeMn", "reviewer": "Reviewer_gBUX", "summary": "This paper proposes FIOC-WM (Factored Interactive Object-Centric World Model), a framework for learning structured object-based representations with explicit interaction modeling in reinforcement learning. FIOC-WM decomposes the environment into object-level interactions (e.g., collisions, containment) and attribute-level states (static vs. dynamic properties). It leverages pre-trained vision models and slot attention to extract object-centric representations from pixels, then uses a hierarchical policy for long-horizon reinforcement learning.", "questions": "1. The paper leverages pre-trained vision models like DINO-v2 and R3M to extract high-quality features. A clarification would be helpful as to whether these pre-trained models are fine-tuned during the world model learning stage. \n2. The reconstruction loss is applied in the object-centric embedding space rather than the raw image space. The reasoning behind this choice could be further explained. While this approach is likely computationally efficient, it appears to heavily rely on the quality and generalization of the pre-trained model's features, which might lead to suboptimal learned representations if the pre-trained model's features are not sufficiently expressive for a given complex task.", "limitations": "The authors addressed the limitations and potential negative societal impact of their work.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\nThe work proposes a two-level factorization of representations: an object-level factorization with interactions and an attribute-level factorization for each object (dynamic and static). This approach provides a principled inductive bias that reduces redundancy and highlights the minimal sufficient components needed for planning and control.\n\n\nWeaknesses:\n\n1. The paper is rich in technical detail, but the overall readability could be enhanced. \n2. To further strengthen the paper's claims, it would be valuable to include some qualitative visualizations of the disentanglement effect, which could make the results more intuitive and compelling.\n3. Figure 1 suggests that a low-level interaction policy is learned during the offline model learning stage. However, the description in Section 3.1 focuses on the world model's components and losses, and the discussion of this policy's learning is deferred to Stage 2. A clarification on whether the low-level policy is indeed trained offline.\n4. In Section 3.2, it is stated that the policy is learned \"by leveraging the predicted interactions to learn the inverse mapping from interactions to actions\". A more detailed explanation of this inverse mapping process would be beneficial. \n5. The use of Slot Attention is a critical component for learning object-centric representations. Given that the number of objects can vary across different environments, it would be helpful to know how the number of slots is determined. Conducting ablation studies on the effect of the number of slots on the model's performance in different environments would be better.", "was_revised": true, "final_justification": "Although the original version had poor readability, the authors provided a detailed explanation in their rebuttal and stated they will make changes in the camera-ready version. I am willing to increase my score by one point.", "rebuttal": "Thank you for your constructive feedback and review! We truly appreciate your recognition of our methodology. Most of the concerns relate to technical details and clarity. For clarity-related issues, we provide concrete and executable plans that can be incorporated into the camera-ready version. For the technical concerns, we offer detailed clarifications along with additional evaluations in this rebuttal. Please find our responses below.\n\n---\n\n> The paper is rich in technical detail, but the overall readability could be enhanced.\n\n**R1**: Thank you for the constructive feedback. To improve readability, we have devised a concrete revision plan for the camera-ready version, addressing each question below. Our focus is on clarifying the technical content and improving the overall flow, as suggested. Please refer to the tags “[**For clarity**]” in our responses to see where these revisions are proposed.\n\n---\n\n>It would be valuable to include some qualitative visualizations of the disentanglement effect, which could make the results more intuitive and compelling.\n\n**R2**: Regarding the qualitative visualization of the disentanglement effect, we agree that the illustrations could be useful. We cannot provide it here as we are not allowed to add additional figures during rebuttal. In the camera-ready version, we will make sure to include visualizations of the reconstructed images to provide a more direct qualitative assessment. These will be included in the appendix. \n\nWe would also like to emphasize that in the current submission, we primarily evaluated disentanglement through probing results (see Fig. 5(a)), as our goal is to learn disentangled representations directly from pre-trained models such as DINO and R3M, indicating that our method can extract structured object-centric information without additional supervision. In addition, we also reported LPIPS scores (Table 1), which reflect pixel-level reconstruction quality.\n\n---\n\n> Figure 1 suggests that a low-level interaction policy is learned during the offline model learning stage. However, the description in Section 3.1 focuses on the world model's components and losses, and the discussion of this policy's learning is deferred to Stage 2. A clarification on whether the low-level policy is indeed trained offline.\n\n**R3**: The low-level policy $\\pi^l$ is trained during the offline stage and then fine-tuned in the online stage. We acknowledge that this setup may have caused some confusion, as the details for learning $\\pi^l$ are presented in Section 3.2. \n\nOur original intention was to clearly separate the components: first, explaining how the world model (including dynamics, observation, and reward) is learned, and then detailing how the hierarchical policy is constructed, in order to make the framework easier to follow. \n\n[**For clarity**] In the revision, we will address this potential misunderstanding by adding a clarifying sentence after Line 166: “Note that the learning of $\\pi^l$ is performed in the offline stage and subsequently fine-tuned during the online stage.”\n\nAnd we plan to add the algorithm to the appendix (here is the brief sketch):\n\n**Offline World Model and Online Policy Learning Framework**\n\n---\n\n**Stage 1: Offline Model Learning**  \nFor each $(o_t, a_t, r_t) \\in \\mathcal{D}$:\n- Encode observation: $\\hat{o}_t \\leftarrow q_\\phi(o_t)$  \n- Infer latent state: $s_t^i \\leftarrow q_\\phi(s_t^i \\mid \\hat{o}_t)$  \n- Factorize latent state: $s_t^i = (d_t^i, c_t^i)$  \n- Learn reward model: $p_r(r_t \\mid s_t, a_t)$  \n- Learn transition model: $p_s(s_{t+1} \\mid s_t, a_t, G_t)$  \n- Infer interaction graphs: $p_u(G_t \\mid s_t)$  \n- Train interaction policy: $\\pi^l(a_t \\mid s_t, G_t^g)$  \n\n---\n\n**Stage 2: Online Policy Learning**  \nFor each environment rollout episode:\n- Receive current state $s_t$ from world model encoder with input environmental steps  \n- Select goal graph: $G_t^g \\sim \\pi^h(G_t^g \\mid s_t)$  \n- Sample action from interaction policy: $a_t \\sim \\pi^l(a_t \\mid s_t, G_t^g)$  \n- Execute $a_t$ in environment, observe $r_t$, $s_{t+1}$  \n- Update policies $\\pi^h$, $\\pi^l$ using collected data  \n\n---\n\n> In Section 3.2, it is stated that the policy is learned \"by leveraging the predicted interactions to learn the inverse mapping from interactions to actions\". A more detailed explanation of this inverse mapping process would be beneficial.\n\n**R4**: The inverse mapping is realized through either MPC or PPO, as described later in Section 3.2 and Appendix C.3. Given a predicted target interaction graph at time $t+k$ (e.g., $G_t^k = G^*$), we treat it as the goal with state *s_g* and use the learned transition model to infer the target states. For MPC, we optimize the action sequence to minimize the difference between the predicted future state and the inferred target state:$\\mathcal{L}\\_{\\text{MPC}} = \\|| s_{t+k} - s_g\\||_2^2$.\n\nAt each iteration, we sample a population of action sequences from a Gaussian distribution and update the mean and covariance using gradient descent, enabling the policy to reach the desired interaction outcome.  For PPO, we use the same reward formulation: $-\\|| s_{t+k} - s_g\\||_2^2$,\nand train a goal-conditioned policy to maximize expected return using standard PPO updates. [**For clarity**] We will add these to the second paragraph of Sec 3.2. \n\n---\n\n> The use of Slot Attention is a critical component for learning object-centric representations. Given that the number of objects can vary across different environments, it would be helpful to know how the number of slots is determined. Conducting ablation studies on the effect of the number of slots on the model's performance in different environments would be better.\n\n**R5**: Thank you for the insightful comment. In our experiments, we heuristically set the number of slots to be slightly larger (typically +2 or +3) than the true number of objects present in the environment. \n\nThis allows Slot Attention to stably learn object-specific representations, given its inherent limitations in dynamically adapting to varying object counts. And we think this heuristic is also practical in real-world settings where a rough estimate of object count is often available, either through prior domain knowledge or by incorporating auxiliary tools such as segmentation-based object proposal methods (e.g., SAM).\n\nWhile slot count selection is orthogonal to the core contributions of our method, we agree that it is helpful to include an ablation. Below, we report performance on the Gym-Fetch (Compositional generalization, 5 objects) task by varying the number of slots:\n| # Slots (relative to true object count) | Success Rate |\n|----------------------------------------|---------------|\n| Exact (0)                               | 0.72 ± 0.09   |\n| +1                                      | 0.75 ± 0.13   |\n| +2                                      | 0.86 ± 0.12   |\n| +3                                      | 0.88 ± 0.11   |\n| +4                                      | 0.81 ± 0.13   |\n| +5                                      | 0.75 ± 0.15   |\n| +10                                      | 0.69 ± 0.20   |\n\nFrom the results, we see that using **+2 to +4** slots yields stable and comparable performance. Setting the number of slots to exactly the number of objects can limit flexibility, making it harder to handle occlusions or background clutter. Too many slots may lead to redundancy or object fragmentation. Slight over-provisioning offers a good balance between expressiveness and stability.\n\n---\n\n> The paper leverages pre-trained vision models like DINO-v2 and R3M to extract high-quality features. A clarification would be helpful as to whether these pre-trained models are fine-tuned during the world model learning stage.\n\n**R6**: We do not fine-tune the pre-trained vision models (DINO-v2 and R3M) during world model learning, which makes the framework efficient and flexible. Our goal is to build task-specific world models efficiently by leveraging the rich representations already captured by these models. [**For clarity**] We will clarify this point in the first paragraph of Section 3 in the camera-ready version.\n\n> The reconstruction loss is applied in the object-centric embedding space rather than the raw image space. The reasoning behind this choice could be further explained. While this approach is likely computationally efficient, it appears to heavily rely on the quality and generalization of the pre-trained model's features, which might lead to suboptimal learned representations if the pre-trained model's features are not sufficiently expressive for a given complex task.\n\n**R7**: Yes, we use reconstruction loss in the object-centric embedding space to reduce computational cost while focusing learning on semantically meaningful features. R3M, being pre-trained on robotic data, provides features well-suited for control tasks. DINO-v2 captures strong semantic representations of visual scenes (also validated in DINO-WM (Zhou et al., 2025)). We believe these pre-trained embeddings offer a sufficiently rich foundation, which our disentanglement module further refines to support downstream dynamics learning and decision-making. \n\n\n---\nWe hope these responses address your concerns. If so, we would be truly grateful if you would consider adjusting your rating accordingly. Thank you again for your time and thoughtful review."}]

Author rebuttal:

Meta-review: This paper proposes FIOC-WM, a two-stage framework for object-centric model-based reinforcement learning. It first learns a structured world model by factorizing observations into object-centric representations with static and dynamic attributes and modeling inter-object interactions as sparse, time-varying graphs. A hierarchical policy then plans over these interactions: a high-level policy sets subgoals, while a low-level policy executes interactions to optimize task rewards. Experiments on robotic manipulation and simulated environments show improved policy performance, better generalization to unseen object combinations, and enhanced predictive fidelity compared to strong baselines including DreamerV3, TD-MPC2, DINO-WM, and prior object-centric RL methods.

Reviewers praised the principled object-centric factorization, hierarchical policy design, and strong empirical results, highlighting the method’s compositionality and generalization advantages. At the same time, concerns were raised about presentation clarity, missing qualitative visualizations, experimental details, and motivation for certain design choices. The rebuttal addressed some of these concerns, providing clarifications and committing to substantial revisions in the final version.

After evaluating the reviews and rebuttal, I recommend ACCEPT (Consensus Reached among Reviewers). The authors are expected to implement the promised improvements, including enhanced clarity and complete experimental details, in the final version to fully address reviewer concerns.
Decision: Accept (poster)