SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels
Authors: Malte Mosbach
Jan Niklas Ewertz
Angel Villar-Corrales
Sven Behnke
Venue: ICML 2025 poster (2025)
Keywords: Model-based Reinforcement Learning
Object-centric Learning
World Models
Robotics
Primary area: reinforcement_learning->deep_rl
TL;DR: We introduce an object-centric model-based RL algorithm that learns solely from pixels, enhancing interpretability and outperforming state-of-the-art methods in robotic tasks requiring reasoning and manipulation.

Abstract: Learning a latent dynamics model provides a task-agnostic representation of an agent's understanding of its environment. Leveraging this knowledge for model-based reinforcement learning (RL) holds the potential to improve sample efficiency over model-free methods by learning from imagined rollouts. Furthermore, because the latent space serves as input to behavior models, the informative representations learned by the world model facilitate efficient learning of desired skills. Most existing methods rely on holistic representations of the environment’s state. In contrast, humans reason about objects and their interactions, predicting how actions will affect specific parts of their surroundings. Inspired by this, we propose *Slot-Attention for Object-centric Latent Dynamics (SOLD)*, a novel model-based RL algorithm that learns object-centric dynamics models in an unsupervised manner from pixel inputs. We demonstrate that the structured latent space not only improves model interpretability but also provides a valuable input space for behavior models to reason over. Our results show that SOLD outperforms DreamerV3 and TD-MPC2 - state-of-the-art model-based RL algorithms - across a range of multi-object manipulation environments that require both relational reasoning and dexterous control. Videos and code are available at https:// slot-latent-dynamics.github.io.

SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels
Malte Mosbach I
Jan Niklas Ewertz 23
Angel Villar-Corrales'
Sven Behnke
Abstract
1. Introduction
Advances in reinforcement learning (RL) have showcased the ability to learn sophisticated control strategies through interaction, achieving superhuman performance in domains ranging from board games (Silver et al., 2016) to drone racing (Kaufmann et al., 2023). While these approaches excel when explicit models of the environment are avail- able or abundant data can be collected, learning complex control tasks in a sample-efficient manner remains sig- nificant challenge. Model-basee RL (MBRL) has emerged promising approach to address this limitation by con- structing models of the environment dynamics. For instance, the Dreamer framework (Hafner et al., 2019; 2020; 2025) improves sample efficiency over model-free methods by learning behaviors via imagined rollouts.
Learning a latent dynamics model provides a task- agnostic representation of an agent's 's understand- ing of its environment, Leveraging this knowl- edge for model-based reinforcement learning (RL) holds the potential to improve sample effi ciency over model-free methods by learning from imagined rollouts. Furthermore, because the la- tent space serves as input to behavior models, the informative representations learned by the world model facilitate efficient learning of desired skills. Most existing methods rely on holistic representa- tions of the environment's state. In contrast, hu- mans reason about objects and their interactions. predicting how actions will affect specific parts of their surroundings. Inspired by this, we propose Slot-Artention for Object-centric Latent Dynamics (SOLD). a novel model-basee RL algorithm that learns object-centric dynamics models in an un- supervised manner from pixel inputs. We demon- strate that the structured latent space not only im- proves model interpretability but also provides a valuable input space for behavior models to reason over. Our results show that SOLD outper- forms Dreamer V3 and TD-MPC2 statc-of-the art model-based RL algorithms- across a range of multi-object manipulation environments that re- quire both relational reasoning and dexterous con- trol. Videos and code are available at https:// /// dynamiddddymcmmmm cs. github. io.
While these research efforts have produced world models ca- pable of accurately predicting the dynamics of visual tasks, they rely holistic representation of the environment state. In contrast, humans perceive the world by parsing scenes into individual objects (Spelke, 1990), anticipating how their actions will influence specific components of their surroundings. Relational reasoning, partieularly in environ- ments with multiple interacting objects, is cornerstone of human intelligence and a crucial capability for robotic manipulation tasks (Battaglia et al., 2018). Introducing structured, object-centric representations into MBRL repre- sents a powerful inductive bias, enabling agents to reason about task-relevant components of the environment while ig- noring irrelevant details. Such structured representations not only enhance interpretability but also improve the efficiency of behavior learning by providing models with meaningful latent spaces. Despite these advantages, the integration of object-centric representations and world models remains largely underexplored. To the best of our knowledge, no prior method learns purely inside imagined rollouts of an object-centric world model trained from pixel inputs.
"Equal contribution 'Autonomous Intelligent Systems Group, Computer Science Institute VI Intelligent Systems and Roboties, Center for Robotics, and the Lamarr Institute for Machine Leam- ing and Artificial Intelligence, University of Bonn, Germany "Intelligent Robot Perception Group, Informatics Institute for An- thropomatics and Robotics, Karlsruhe Institute of Technology, Germany "eereedes- Benz AG, Sindelfingen, Germany. Corre- spondence to: Malte Mosbach <mosbach @ais ais iiss.uni-be uni-bonn.de>>.
To address the limitations of holistic representations in MBRL, propose SOLD, novel algorithm that leverages structured, object-centric states within the latent space of its world model. The contributions of our method are twofold. First, we introduce an object-centric dynamics model that predicts future frames in terms of their slot representation.
Proceedings of the ndnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels
Building the OCVP framework (Villar-Corrales al., 2023), introduce action-conditional dynamics model, enabling accurate forecasting of action effects individ- ual objects. Notably, the dynamics model is trained solely from pixels through a loss on the reconstructions and slot representations of the predicted frames, bypassing the need for object annotations and facilitating scalability to com- plex visual tasks. Second, we propose the Slot Aggregation Transformer, novel architectural backbone that aggregates information from the history of object slots to make reward, value, and action predictions. This enables efficient MBRI training grounded in structured representations.
Reach-Specific
Reach-Specitic Relative
Miss-Sppecffic
Specific
Reach-Distinct
Reach-Distinct- Groups
Push-Distinct
Pick- Distinct
For systematic evaluation, we introduce a suite of visual roboties tasks, shown in Figure that require varying levels of relational reasoning and manipulation capabilities. We perform extensive comparison this benchmark, demon- strating that our method achieves superior performance to both sfate-of-thh-ar MBRL algorithms DreamerV3 (Hafner et al., 2025) and TD-MPC2 (Hansen et al., 2024). Further, to highlight its broader potential, we apply SOLD to tasks from two RL. benchmarks that are not object-centric by design, providing evidence of the generalizability of our framework. In summary, make the following contributions:
Figure 1: Suite of visual enviromments requiring relational reasoning and low-level manipulation to be solved.
the previous slot representations conditioned on the current features. Slot Attention performs cross-attention between the slots and image features with the attention coefficients normalized over the slot dimension, thus encouraging the slots to compete to represent feature locations:
A = softmax q (Z) -A(F) ERRXL (1)
N
VD
We introduce SOLD, the first MBRL algorithm to learn inside imagined rollouts of an object-centric dynamics model trained directly from pixel inputs, achieving state-of-the-art performance on visual roboties tasks that require both relational reasoning and manipulation.
where q()) and k(.) are Iearned linear mappings to com- mon dimension D. The slots are then independently updated via a shared Gated Recurrent Unit (Cho et al., 2014) (GRU) followed by a residual MLP:
Z: - MLP(GRR(AA vl (FF).. Zi-1)),
By visualizing learned attention weights, show that method produces human- -ntrrrrratreree attention patterns, providing insights into the decision-making process of behavior models.
And
An. E1 An,i
and v(.) is a learned linear projection. The steps described in Equations 1 and can be repeated multiple times with shared weights to iteratively refine the slots and obtain an accurate object-centric representation of the scene. Finally. SAVi independently decodes each slot of Z, into per-object images and alpha masks, which can be normal- ized and combined via weighted sum render video frames. SAVi is trained end-to-end in self-supervised manner with image reconstruction loss.
We overcome limitations of prior object-centric RL methods by showing that our encoder-decoder module can (i) be adapted to state distributions vastly different from those seen under random pre-training, and (ii) generalize environments that are not object-centric by design.
2. Background
Object- Video Prediction (OCVP) Our dynam- ics model builds on OCVP Villar-Corraled et al., 2023) in order to autoregressively predict future object slots condi- tioned on past object states. OCVP is transformer-encodle model (Vaswani al., 2017) that decouples the processing of object dynamics and interactions, thus leading to inter- pretable and temporally consistent object predictions while retaining the inherent permutation-cquivaraan property of the object slots. This achieved through the use of two specialized self-attention variants: temporal attention up- dates a slot representation by aggregating information from
Slot Attention for Video (SAVi) SOLD employs SAVi (Kipf et al., 2022), an encoder-decoder architecture with structured bottleneck composed of N permutation- equivariant object embeddings, referred to slots. It recur- sively parses sequence of video frames Ou.r Ou. .... O, into their object representations Zo.- = Zo,... Zr, Z, E g ND, At time t, SAVi encodes the input video frame o, into a set of feature maps F, E R Ix LLDD; where L is the size of the flattened grid (i.e. width height), and uses Slot Attention (Locatello al., 2020) to iteratively refine
2

---

SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels
a
a1
0
0
71
1
a1
722
V2
U
Zo
2
Zo
Z1
2
encsvvi
decsAvi
decsAvi
encsAvi
Ô2 (a) World Model Learning: SAVi encodes images Or into slots Z, which are predicted by the dynamics model given the history of slots and actions. We reconstruct the images and compute their actual slot representation shape the dynamics prediction.
O0
Oo
(b) Behavior Learning: Actor and critic are trained via imagined rollouts in the latent space of the world model. Trajectories start after S seed frames (visualized for S = 1) and predict forward with actions a sampled from the actor network.
Figure 2: SOLD is trained by concurrently making the world model consistent with replayed experiences and learning behaviors through latent imagination.
the corresponding slot up to the current time step, with- out modeling interactions between distinct objects, whereas relational attention models object interactions by jointly processing all slots from the same time step.
3.1. World Model Learning
World models compress an agent's experience into a predic- tive model that forecasts the outcomes of potential actions. By simulating rollouts within the internal model, agents can learn desired behaviors in sample-efficient manner. When the inputs are high-dimensional images, it is helpful to learn compact state representations, enabling prediction within this latent space. This type of model, called latent dynamics model, allows for efficient prediction of many latent sequences in parallel.
3. Slot-Attention for Object-centric Latent Dynamics
We propose SOL,), method that combines model-based RL with object-centric representations. The three com- ponents of our method are: the object-centric world model, which predicts the effects of actions on the environment, the critic, which estimates the value of a given state, and the actor that selects actions to maximize this value.
Most prior works rely on generating a single, holistic repre- sentation of the environment state, which contrasts with find- ings from cognitive psychology. Humans perceive scenes as compositions of objects (Spelke, 1990) and reason about how their actions affect distinct parts of their environment. Furthermore, environment dynamics can be compactly ex- plained terms of objects and their interactions (Battaglia et al., 2016). Therefore, we propose to structure the la- tent space by decomposing visual environments into their constituent parts.
Figure 2 gives an overview of the training process. The world model operates on structured latent states by split- ting the environment into its constituent objects and then composing future frames via the predicted states of these individual components. Specifically, we pretrain SAVi encoder- model (Kipf et al., 2022) on random se- quences from the environment to extract object-centric rep- resentations. After pretraining, all components of the world model trained jointly using replayed experiences from the agent's interaction with the environment. The actor and critic are trained on imagined sequences of structured latent states. We execute actions sampled from the actor model in the environment and append the resulting experiences to the replay buffer. Detailed explanations of world model learning and behavior learning are provided in Sections 3.1 and 3.2, respectively.
Components To create a world model that operates on object-centric latent representations, we build on top of OCVP (Villar-Corrals et al., 2023). We begin by pretrain- ing SAVi static dataset of frames collected from random episodes. Having sufficiently large initial dataset is cru- cial for meaningful object-centric representations to emerge. These pretrained representations serve as the foundation for SOLD's s object-centric world model. However, we do not freeze the pretrained encoder-decoder models, allowing
3

---

SOLD: Slot Object-Centrie Latent Dynamics Models for Relational Manipulation Learning from Pixels
Context
Open-loop prediction
True
Model
Slots
True
Model
Slots
t=
2
10
20
30
40
50
Figure 3: Open-loop predictions of our object-centric dynamics model for the Pick-Distinet task (top) and the Hammer environment from Meta World (bottom). Starting from single context frame, our model predicts the next 50 frames by propagating slot representations forward without access any intermediate images.
slots adapt novel configurations that do not occur dur- ing random pre-training. The sequence of object slots Zo: alongside the action commands ag. serve as inputs to our transformer-base dynamics model which predicts the slot representation of the next frame Ze41:
frame 6 and extract the SAVi representations of the tual frame Zt+1 to compute the hybrid dynamics loss:
S4T-1
Ldyn =
Σ
- (,,
llo. 0.11% (4)
embeddiing
Encoder:
Zi = =,((o))
Decoder:
ô, = dn (Zt),
For all loss terms, specify the parameter group that is being optimized and omit stop-grate goo-gradient notations for other models to avoid cluttering the notation.
Dynamics model:
Z,+1 Pvl Zo:t 0oa), and
Reward predictor:
Pr (fr Zo:).
Reward Model Learning The reward predictor solves regression problem that maps slot representations to scalar reward values, where the prediction depends on the set of slots rather than being tied to any specific one. To address this, we introduce the Slot Aggregation Transformer (SAT) as an architectural backbone, which introduces output to- kens and a variable number of register tokens for all time- steps. Register tokens, recently shown to enhance compu- tation in vision transformers (Darcet et al., 2024), can aid computation when processing a set of inputs to produce a singular output. To encode positional information, we adopt ALiBi (Press et al., 2022) place of absolute position en-
Object-centric Dynamics Learning For the dynamics model, we use the sequential attention pattern proposed by Villar-Corrales et (2023), which disentangles rela- tional and temporal attention to decouple object dynamics and interactions. During training, we provide the slot rep- resentation of S seed frames context. We append the predictions to the context and apply this process in an au- toregressive manner to predict the subsequent T' frames. We do not employ teacher forcing so that the dynamics model learns to handle its own imperfect predictions. To shape the predicted representations, we reconstruct the subsequent

---

SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels
coding. ALiBi introduces linear biases directly into the attention scores, effectively encoding token recency. This approach helps generalize to sequences longer than those seen during training. A detailed description of the SAT can be found in Section B.3. To efficiently represent a wide range of reward values, we avoid directly predicting a scalar reward. Instead, the MLP head fc outputs logits of a soft- max distribution over K exponentially spaced bins by. The predicted reward can then be computed as the expectation over these bins:
Table I: Final success rates (% + standard deviation) of SOLD and baseline methods for the specific (top) and dis- tinct (bottom) task variants.
(a) Specific tasks requiring mainly robotic control.
Task
Dreamer
TD-MPC2
w/o OCE
SOLD
Reach
87.4 #1
97.6 +0
83.2
97.9 +0
Reach-Rel.
45.6
79.1
39.2
91.1 42
Push
97.1 =1
72.7 43
75.2 #2
82.8 42
Pick
96.7 #1
87.6 42
22.9 #11
85.8 47
b = symexp([-20,.. ..... +20]).
Average
81.7 #21
84.2 #9
55.1 #25
89.4 #6
(5)
- softmax ( (h.))"b, (h:))
(h) Distinct tasks requiring challenging relational reasoning.
where h2 are the output tokens after being processed by the SAT backbone. To formulate the loss, the true reward rt is first transformed using the symlog function (Webber, 2012) and then encoded via two-hot encoding strategy (Belle- mare et al., 2017; Schrittwieser et al., 2020). The model is trained to maximize the log-likelihood of the two-hot en- coded reward distribution under the predicted distribution:
Task
DreamerV3
TD-MPC2
w/o OCE
SOLD
Reach
14.6 =6
31.4 #3
11.3 #1
91.8 =1
Reach-Gr.
13.9 =2
15.7 #2
5.1
69.6 =2
Push Pick
70.0 15
12.2
10.5 11
80.6 15
33.9 1:36
9.8 11
0.7 1.0
56.4 125
Average
33.1 +23
17.3
6.9 +4
74.6 +13
T-1
Crow =
log (r: Zo:)).
We decouple the gradient scale from value prediction through same approach as in the reward model, predict- ing a categorical distribution over exponentially spaced bins. To stabilize learning, we regularize the critic's predictions towards the outputs of an exponentially moving average (EMA) of its own parameters (Mnih al., 2015; Hafner et al., 2025).
3.2. Behavior Learning
Our strategy of using the world model for behavior learn- ing builds upon the Dreamer framework. At the core of this method lies the process of latent imagination, visu- alized Figure 2b, which trains the actor and critic net- works purely on imagined trajectories predicted by the world model. Since both the actor and critic operate on the latent state, they benefit from the structured representation learned by the world model. The architecture of both models mirrors that of the reward predictor, consisting of SAT backbone that processes the slot histories followed by an MLP head:
Actor Learning The actor is optimized to select actions that maximize its expected return while encouraging explo- ration through an entropy regularizer. Its model architecture is similar to the critic and reward predictor, but instead of regressing a scalar value, it predicts the parameters of the action distribution. Specifically, the MI.P head outputs the mean br and standard deviation or, of a normal distribution N N(p. o, ZZ:z over possible actions. The trade-off in the actor's loss function weights expected returns with main- taining randomness in the outputs and is hence subject to reward scale and frequency of the current environment. To adapt to varying scales of value estimates across different environments, we use a normalization factor sy:
Actor:
at Ta(a Zo:t);
(7)
Critic:
R (R Zo
Critic Learning To account for rewards beyond the imag- ination horizon 7' = 15, the critic is trained to estimate the expected return under the current actor's behavior. Since no ground truth available for these estimates, we com- pute bootstrapped X-returns (Sutton Barto, 2018), R*, via temporal difference learning. These returns integrate predicted rewards f and values R to form the target for the value model:
Cucue (0)
Σ
max(1, +nH((Toaa
Zo:e), (10)
R = fet1 + (11 R) + AR/+1),
(8)
where the value normalization is computed via the EMA of the 5th and 95th percentile of the value estimates (Hafner et al., 2025):
where R Ri, which is trained to minimize the resulting loss:
Leittic (b) =
log u (R)) Zo=))
(9)
Sv = EMA [Per(fe): 95) Per Per(fet,5),000 5..99. (11)
5

---

SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels
Reach
Push Specific
Pick
Specific
Specifio-Relatiie
Specific
20
20
30
Return
20
10
10
0
5
10
15
5
10
20
10
0
Distinct
Distinct-Groups
Distinct
Distinct
20
30
30
Return
20
20
10
10
10
10
0
0
0
5
10
15
0
10
20
30
10
20
0
10
20
Episodes (1e4)
Episodes (1e4)
Episodes (1e4)
Episodes (1e4)
DreamerV3
TD-MPC2
Ours w/o OCE
SOLD (Ours)
Figure 4: Achieved returns over the training duration across the eight benchmark environments. The dashed vertical line represents the offset for our method to account for the samples used during pre-training.
4. Results
its location, and manipulation tasks (Push and Pick), where the agent identifies a target block and moves to a desig- nated goal. To test varying levels of relational reasoning difficulty, we design the following configurations:
In this section, we present the empirical evaluation of SOLD on our suite of visual continuous control tasks. We first de- scribe the comparative baselines and the environments used in our experiments. Using this setup, we aim to answer the following questions: (a) Does SOLD accurately model object-centric dynamics in cetion-conditionall settings, pre- serving the decomposition of visual scenes? (b) Does the structured latent space allow SOLD to outperform SoTA baselines on tasks that require relation reasoning capabili- ties? (c) Can SOLD generalize to visual environments that are not object-centric by design?
Specific The target object is red, with 4 distractor objects of random, distinct colors present in the scene.
Distinct Inspired by the odd-one-out task in cognitive science (Crutch et al., 2009; Beatty & Vartanian, 2015), this task presents 3 to 5 objects, and the target is the one that differs in color from all the others.
For the Reach task, we include two advanced variants:
Specific-Relative The goal is to reach the reddest object, determined by the perceptual CIEDE2000 (Sharma et al., 2005) distance.
Baselines The chosen baselines in evaluation serve two primary purposes: assessing the impact of the object- centric paradigm in our method and benchmarking it against state-of-the-an approaches. To evaluate the effect of object-centric representations, we compare our method to a baseline that replaces the object-centric decoerr modules with standard convolutional architecture (w/o OCE). To benchmark against the best available methods, we include DreamerV3 (Hafner et al., 2025) and TD- MPC2 (Hansen et al., 2024). Both are widely recognized for their strong performance across a wide range of tasks. Addi- tional details for the baselines can be found in Appendix C.
Distinct-Groups The environment contains 5 targets, and the goal to reach the one that appears only once.
On these two additional reach tasks, we reuse the SAVi models that were pre-trained for Reach-Specific and Reach- Distinct, respectively without modification. Further details about these environments are provided in Appendix D.
4.1. Object-centrie Dynamics Learning
The object-centric representations learned by SAVi can be seen in the context frames in Figure 3. The slots effectively decompose the visual scene, with most slots representing distinct objects and three slots capturing different parts of the respective robots. This whote segmentation demon- strates that the slots can meaningfully identify separate parts
Environments We introduce suite of eight object-centric robotic control environments designed to test both relational reasoning and manipulation capabilities. These environ- ments feature two types of problems: Reach tasks, where the agent must identify a target and move the end-effector to
6

---

SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels
Model
Attention
1=2
4
6
17
18
19
20
21
Figure 5: SOLD discovers objects relevant for task completion in unsupervised manner over long horizons. We depict the normalized attention of the [out] token of the actor the object tokens using Attention Rollout (Abnar & Zuidema, 2020). The full slot history is shown in Figure 16.
of a larger object, representing the gripper jaws in the first example and different parts of the kinematic chain of the Sawyer robot in the second. Notably, the sharp mask pre- dictions show that each slot isolates information about the specific object it represents (see also Section E). This prop- erty is crucial for object-centric behavior learning, it en- ables subsequent components to reason about task-relevan, objects while ignoring irrelevant information. Further, the open-loop prediction of 50 future frames starting from sin- gle seed frame, shown Figure 3, demonstrate the model's ability generate physically accurate predictions over long horizons. The movements of the robot and blocks pre- dicted with high accuracy, demonstrating the model's ability to precisely capture physical interactions between objects. Moreover, the model effectively handles occlusions, evi- denced by the continued precise prediction of the spherical target. Importantly, the autoregressive dynamics model maintains a precise and meaningful decomposition of the scene its predictions, even far into the future. We encour- age readers view the videos of object-centric open-loop predictions on our project page for qualitative assessment of these results.
on the more challenging Distinet variants, which require complex relational reasoning between objects. On these tasks, SOLD achieves more than double the performance of the second-best method.
Beyond that, when examining the performance course of training, as shown in Figure 4, we observe addi- tional advantages in terms of sample efficiency. We find that SOL.D consistently outperforms the highly sample-effiieen DreamerV3 and TD-MPC2 baselines on all but the easiest Reach Seach-Specifie task, even after accounting for the samples used during pre-training. While the non-object-centric base- line demonstrates some success the Specific tasks, it struggles with the relational reasoning required to solve the Distinct variants. In contrast, SOLD excels in tasks that demand reasoning about relationships between objects, as evidenced by the substantial performance gap observed on the Distinct tasks.
the
These results support our hypothesis that structured latent representation within the world model significantly benefits tasks requiring object reasoning. This is especially valu- able in robotics, where understanding object interactions is essential for solving complex control problems.
4.2. Behavior Learning
To assess SOLD's performance across our robotic control tasks, we train each method with three different random seeds per environment. The final success rates achieved by each method are presented Table 1. SOLD con- sistently outperforms the non-object-centric baseline, of- by a significant margin, underscoring the effectiveness of object-centric representations for the considered tasks. When compared to state-of-the-art MBRI. methods, SOLD demonstrates competitive or superior performance. Notably, while SOLD narrowly surpasses DreamerV3 and TD-MPC2 across the Specific tasks, it significantly outperforms them
Discovering Tesk-relevant Objects To demonstrate that SOLD's improved relational reasoning capabilities are ac- companied by an interpretable focus on task-relevant ob- jects, we visualize an excerpt of the slot history Figure 5. To illustrate the actor's attention pattern in the current (rightmost) time step, we multiply the attention scores by the masks of the respective objects and show them overlaid with the RGB reconstructions and an individual colormap in the second and third rows, respectively. This visualization shows the Push -eshhSpecifiic task, where we find that the model automatically identifies task-relevant objects, disregarding slots that represent distractor objects across all time steps

---

SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels
Frozen
Finetuned
Full
Target
t= 0
4
9
14
19
t= 0
14
19
Figure 6: Decoded reconstructions of the full scene (top) and the slot corresponding to the target object (bottom) for a frozen (left) and finetuned (right) SAVi model. Finetuning enables the model to represent object configurations not present the random pre-training data.
while focusing primarily the robot and green cube. Al- though the recency bias induced by ALiBi is evident, we find that the model learns to overcome it when necessary, attending to the red sphere (indicating the goal position) after has been occluded for 15 time steps, the last time was visible. We see that the model effectively prioritizes task-relevant information, even when reasoning over long time sequences is required.
significantly different domains typically associated with object-centric learning. These environments are shown in Figure 7. SOLD achieves 100% success rate both the Button-Press and Hammer tasks, highlighting its ability to adapt to visually diverse and challenging object interactions. On the Cartpole-Balande achieves returns of 497 and 645, respectively, demonstrat- ing its capacity generalize tasks where object-centric reasoning is less pronounced. Details about environment decompositions and dynamics predictions for all four tasks can be found in Section E.2 of the Appendix.
and Finger-Spin tasks, SOLD
SAVi Finetuning A common limitation of prior work in object-centric RL is the reliance on encoder-decoder models pretrained on sequences with random behaviors and kept frozen during training, restricting their applicability tasks where random and successful policies encounter similar state distributions. In the Pick tasks, this assumption is explicitly violated, as random behaviors rarely result in blocks being lifted off the table. Consequently, SAVi lacks prior exposure to configurations with blocks in the air, which will inevitably reached by successful policies. Figure illustrates the necessity of continually fine-tuning the object- centric mecderr model. While the fine-tuned model accurately reconstructs lifted blocks, the frozen variant fails to decode this configuration correctly, causing the target block to dissolve during lifting. Additional examples are shown in Figure 15 the Appendix.
5. Related Work
Object-Centric Learning In recent years, the field of unsupervised object-centric representation learning from images and videos has gained significant attention (Yuan et al., 2023). Most existing methods follow an encoder- decoder framework with a structured bottleneck composed of N latent vectors called slots, where each of these slots binds to a different object in the input image. Slot-based methods have been widely applied for images (Burgess et al., 2019; Locatello et al., 2020; Singh et al., 2021; 2023; Biza et al., 2023) and videos (Kipf et al., 2022; Singh et al., 2022; Elsayed et al., 2022; Bao et al., 2022). However, despite their impressive performance on synthetic datasets, they often fail to generalize to visually complex scenes. overcome this limitation, recent methods propose using weak supervision (Elsayed al., 2022; Bao al., 2023). levering large pretrained encoders (Seitzer al., 2023; Aydemir et al., 2023; Kakogeorgiou et al., 2024), using diffusion models as slot decoders (Jiang et al., 2023).
4.3. Generalization to Non-Object-Centrie Tasks
While object-centric methods are commonly applied to en- vironments and datasets that naturally lend themselves to such decompositions, we aim to showcase the potential of our method to generalize beyond this setting. To this end, train SOLD on the Button-Press and Hammer tasks from the Meta- World benchmark (Yu et al., 2019), both of which feature object with complex shapes and textures, challenging the model's ability to handle more diverse and visually intricate inputs. Additionally. we test SOLD on Balance and Finger-Spin environments from the DM-Control suite (Tassa al., 2018), which represent
Object-Centric Video Prediction Object-centric video prediction aims to understand the object dynamics in a video sequence with the goal of anticipating how these objects
will move and interact with each other future time steps. With this end, multiple methods propose to model and fore- cast the object dynamics using different architectures, in-
the Cartpole-Balance
8

---

SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels
cluding RNNs (Zoran al., 2021; Nakano et al., 2023) transformers (Wu al., 2023; Villar-Corrales et al., 2023; Song al., 2023; Daniel & Tamar, 2024; Nguyen et al., 2024) or spate-space models (Jiang et al., 2024), achieving an impressive prediction performance on synthetic video datasets and learning representations that can help solve downstream tasks that require reasoning about objects prop- erties and relationships (Wu al., 2023; Petri al., 2024). Biza et al. (2022) address the integration of action informa- tion into object-centric world models, but focus solely on action-conditional prediction and do not consider behavior learning through RL.
Finger-Spin Non-object centric environments from Meta- World (Yu et al., 2019) and DM-Control (Tassa et al., 2018).
Button-Press
Hammer
Carlpole-Bale Balance
Figure 7:
RL with Object-Centric Representations Recent works have explored integrating object-centric representations into RL frameworks. SMORL (Zadaianchuk et al., 2021) and EIT (Haramati et al., 2024) combined object-centric repre- sentations with goal-conditioned model-free RL for robotic manipulation. Yoon et al. (2023) investigated pre-training object-centric representations for RL, showing benefits for relational reasoning tasks. The field of object-centric model- based RL still largely underexplored. Ferraro al. (2023) introduce FOCUS, which learn object-centric world mod- els for robotic manipulaiton. However, unlike method, FOCUS does not use the object-centric states in forward prediction or action selection, but mainly to form an ex- ploration target. Further, FOCUS requires supervision via ground-truth segmentation masks to learn the object-centric states. Ugadiarov et al. (2025) introduce ROCA, which learns an object-centric transition model from visual obser- vations. However, unlike our method, this model us used to improve return estimates of the critic instead of learning through imagination inside world model rollouts.
Model-based RL Model-based approaches hold the po- tential to improve the sample efficiency of RL methods by learning environment dynamics, and recent years have seen several key contributions advancing this area. Pio- neering work by Ha & Schmidhuber (2018) introduced the concept of recurrent generative model, termed world model, which captures the dynamics of visual RI. environ- ments. By encoding tigh-dimensional observations into a compact latent representation, this model enables RL agents to train policies entirely within imagined rollouts. The Planning Network (PlaNet) (Hafner al., 2019) intro- duced a recurrent state-space model (RSSM) that predicts future states directly in a compact latent space, avoiding the costly step of decoding observations. PlaNet enables efficient planning of action sequences but is limited by the planning horizon. Building on this, Dreamer (Hafner et al., 2020) integrates planning and learning by training agents within a learned world model. Subsequent versions (Hafner al., 2021; ; 2025) improved robustness and generalization through enhanced representation learning and optimization techniques. DreamerV3 has shown superior performance in visual control tasks compared to model-free approaches, but uses holistic rather than object-centric state represen- tations. TD-MPC (Hansen et al., 2022) introduced a task- oriented latent dynamics model to optimize trajectories di- rectly within the latent space of world model. Unlike earlier approaches, TD-MPC avoids reconstructing observa- tions, instead focusing the world model on reward-predictive elements through a loss applied to reward and value predic- tions. TD-MPC2 (Hansen et al., 2024) extends this work by introducing scalability improvements, demonstrating gen- eralization abilities across multiple tasks and action spaces. Micheli al. (2023) proposed IRIS, method combining discrete autoencoder with an autoregressive Transformer to model environment dynamics, demonstrating visually and temporally accurate predictions of game dynamics in Atari environments. While IRIS shares similarities with our ap- proach- encoding an image into a set- based representation and predicting it forward using a Transformer it lacks the object-centric interpretability afforded by our model.
6. Conclusion
We present SOLD, an object-centric model-based RL algo- rithm that learns directly from pixel inputs. By employing structured latent representations through slot-based dynam- ies models, our method offers a compelling alternative to traditional, holistic approaches. While -bject-centric rep- resentations have been valued for their role in forward pre- diction (Villar-Corrales et al., 2023), we demonstrate their synergistic benefits accelerating the learning of behavior models. SOLD achieves strong performance across the in- troduced visual robotics environments, outperforming the state-of-the-art methods DreamerV3 and TD-MPC2, partic- ularly in tasks requiring relational reasoning. Additionally, the learned behavior models exhibit interpretable attention patterns, explicitly focusing on task fask-relevant parts of the vi- sual scene. To enable broader adoption, will be interesting for future work to scale object-centric dynamics models to real- world data and highly stochastic environments. Ulti- mately, SOLD suggests that object-centric approaches hold strong potential for advancing modell-asee reinforcement learning.
9

---

SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels
Impact Statement
Beatry, E. L. and Vartanian, O. The prospects of work- ing memory training for improving deductive reasoning. Frontiers in Human Neuroscience, 2015.
This paper presents work whose goal is to advance the field of machine learning by introducing novel object-centric model-based reinforcement learning algorithm, SOLD, de- signed for improved interpretability and sample efficiency. Our work has potential applications in robotics and automa- tion, particularly in tasks requiring relational reasoning and manipulation. While we do not anticipate specific societal concerns arising directly from this work, the deployment of reinforcement learning systems in real-world applications warrants thoughtful consideration of ethical and safety chal- lenges, including robustness and accountability in decision- making. Improved model interpretability, explored in this work, could provide an avenue toward addressing these challenges.
Bellemare, M. G., Dabney, W., and Munos, R. A distri- butional perspective on reinforcement learning. In In- ternational Conference on Machine Learning (ICML.), 2017.
Biza, O., Platt, R., van de Meent, J.-W., Wong, L. L., and Kipf, T. Binding actions to objects in world models. In ICIR Workshop on the Flements of Reasoning: Objects, Structure and Causality, 2022.
Biza, O., Van Steenkiste, S., Sajjadi, M. S., Elsayed, G. F. Mahendran, A., and Kipf, T. Invariant slot attention: Object discovery with slot-centric reference frames. In International Conference on Machine Learning (ICML), 2023.
Acknowledgment
This work has partially been funded by grant BE 2556/16-2 (Rescarch Unit FOR 2535 Anticipating Human Behavior) of the German Research Foundation (DFG) and by the German Federal Ministry of Education and Research (BMBF) in the project "Robotics Institute Germany (RIG)" under grant No. 16ME0999, Computational resources were provided by the German AI Service Center WestAI.
Burgess, C. P., Matthey, L., Watters, N., Kabra, R., Higgins, I.. Botvinick, M., and Lerchner, A. MONet: Unsuper- vised scene decomposition and representation. arXiv preprint
2019.
Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk. H.,, and Bengio, Y. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Conference on Empiri- cal Methods in Natural Language Processing (EMNLP), 2014.
References
Abnar, and Zuidema, W. Quantifying attention flow in transformers. In Annual Meeting of the Association for Computational Linguistics (ACL), 2020.
Crutch, S. J., Connell, S., and Warrington, E. K. The dif- ferent representational frameworks underpinning abstract and concrete knowledge: Evidence from odd-one-out judgements. Quarterly Journal of Experimental Psychol- ogy, 2009.
Aydemir, G., Xie, W., and Guney, F. Self-supervised object- centric learning for videos. In Advances in Neural Infor- mation Processing Systems (NeurIPS), 2023.
Bao, Z., Tokmakov, P., Jabri, A., Wang, Y.-X., Gaidon, A., and Hebert, M. Discovering objects that can move. In Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
Daniel, T. and Tamar, A. DDLP: Unsupervised object- centric video prediction with deep dynamic latent par- ticles. Transactions on Machine Learning Research (TMLR), 2024.
Bao, Z., Tokmakov, P., Wang, Y.-X., Gaidon, A., and Hebert, M. Object discovery from motion-guided tokens. In Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
Darcet, T., Oquab, M., Mairal, J., and Bojanowski, P. Vision transformers need registers. In International Conference on Learning Representations (ICLR), 2024.
Battaglia, P. W., Pascanu, R., Lai, M., Rezende, D. J., and Kavukcuoglu, K. Interaction networks for learning about objects, relations and physies. In Advances in Neural Information Processing Systems (NeurIPS), 2016.
Elsayed, G., Mahendran, A., Van Steenkiste, S., Greff, K., Mozer, M. C., and Kipf, T. SAVi++: Towards end- to-end object-centric learning from real-world videos. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
Battaglia, P. W., Hamrick, J. B., Bapst, V., Sanche7- Gonzalez, A., Zambaldi, V., Malinowski, M., Tacchetti, A., Raposo, D., Santoro, A., Faulkner, R., et al. Rela- tional inductive biases, deep learning, and graph networks. arXiv preprint arXiv: 0/260 2018.
Ferraro, S., Mazzaglia, P., Verbelen, T., and Dhoedt, B. FOCUS: Object-centric world models for robotic manip- ulation. In NeurlPS Workshop on Intrinsicall-MMotivee and Open- Learning, 2023.
10

---

SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels
Ha, D. and Schmidhuber, J. Recurrent world models facili- tate policy evolution. In Advances in Neural Information Processing Systems (NeurIPS), 2018.
Kipf, T., Elsayed, G. F., Mahendran, A., Stone, A., Sabour, S., Heigold, G., Jonschkwwsii, R., Dosovitskiy, A., and Greff, K. Conditional object-centric learning from video. In International Conference on Learning Representations (ICIR), 2022.
Hafner, D., Lillicrap, T., Fischer, I,, Villegas, R., Ha, D., Lee, H., and Davidson, J. Learning latent dynamics for planning from pixels. In International conference on Machine Learning (ICML), 2019.
Li, R., Jabri, A., Darrell, T., and Agrawal, P. Towards practical multi-object manipulation using relational re- inforcement learning. In International Conference on Robotics and Automation (ICRA), 2020.
Hafner, D., Lillicrap, T., Ba, J., and Norouzi, M. Dream to control: Learning behaviors by latent imagination. In International Conference on Learning Representations (ICLR), 2020.
Locatello, F. Weissenborn, D., Unterthiner, T., Mahendran, A., Heigold, G., Uszkoreit, J., Dosovitskiy, A., and Kipf, T. Object-centric learning with slot attention. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
Hafner, D., Lillicrap, T., Norouzi, M., and Ba, J. Master- ing Atari with discrete world models. In International Conference on Learning Representations (ICLR), 2021.
Loshchilov, I. and Hutter, F. SGDR: Stochastic gradient descent with warm restarts. In International Conference on Learning Representations (ICLR), 2017.
Hafner, D., Pasukonis, J., Ba, J., and Lillicrap. T. Mastering diverse control tasks through world models. Nature, 2025.
Micheli, V., Alonso, E., and Fleuret, F. Transformers are sample-efficient world models. In International Confer- ence on Learning Representations (ICLR), 2023.
Hansen, N., Wang, X., and Su, H. Temporal difference learning for model predictive control. In International Conference on Machine Learning (ICML), 2022.
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidje- land, A. K., Ostrovski, G., et al. Human-level control through deep reinforcement learning, Nature, 2015.
Hansen, N., Su, H., and Wang, X. TD-MPC2: Scalable, robust world models for continuous control. In Interna- tional Conference on Learning Representations (ICLR), 2024.
Nakano, A., Suzuki, M., and Matsuo, Interaction -based disentanglement of entities for object-centric world mod- els. In International Conference on Learning Representa- tions (ICLR), 2023.
Haramati, D., Daniel, T., and Tamar, A. Entity-centric rein- forcement learning for object manipulation from pixels. In International Conference on Learning Representations (ICLR), 2024.
Nguyen, T., Mansouri, A.. Madan, K., Nguyen, K. D., Ahuja, K., Liu, D., and Bengio, Y. Reusable slotwise mechanisms, In Advances in Neural Information Process- ing Systems (NeurIPS), 2024.
Jingg J., Deng, F., Singh, G., and Ahn, S. Object-centric slot diffusion. In Advances in Neural Information Processing Systems (NeurIPS), 2023.
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, I.., al. PyTorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems (NeurIPS), 2019.
Jiang J., Deng, F,, Singh, G., Lee, M., and Ahn, S. SlotSSMs: Slot state space models. In Advances in Neu- ral Information Processing Systems (NeurIPS), 2024.
Kakogcorgiou, I., Gidaris, S., Karantzalos, K., and Ko- modakis, N. SPOT: Self-training with patch-order per- mutation for object-centric learning with autoregressive transformers. In Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
Petri, F., Asprino, L., and Gangemi, A. Transformers and slot encoding for sample efficient physical world mod- elling. arXiv preprint arXiv:2405.220
2024.
Press, O., Smith, N., and Lewis, M. Train short, test long: Attention with linear biases enables input length extrapo- lation. In International Conference on Learning Repre- sentations (ICLR), 2022.
Kaufmann, E., Bauersfeld, L., Loquercio, A., Müller, M., Koltun, V., and Scaramuzza, D. Champion-level drone racing using deep reinforcement learning. Nature, 2023.
Schritwieser., J., Antonoglou, I., Hubert, T., Simonyan, K., Sifre, L., Schmitt, S., Guez, A., Lockhart, E., Hassabis, D., Graepel, T., et al. Mastering Atari, Go, chess and shogi by planning with a learned model. Nature, 2020.
Kingma, P. and Ba, J. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR), 2015.

---

SOLD: Slot Object-Centrie Latent Dynamics Models for Relational Manipulation Learning from Pixels
Seitzer, M., Horn, M., Zadaianchuk, A., Zietlow, D., Xiao, T., Simon-Gabriel, C.-J., He, T., Zhang, Z.,, Schölkopf, B., Brox, T., et al. Bridging the gap to real-world object- centric learning. In International Conference on Learning Representations (ICIRR, 2023.
Villar-Corrales, A., Wahdan, I., and Behnke, S. Object- centric video prediction via decoupling of object dynam- ics and interactions. In International Conference on Im- age Processing (ICIP), 2023.
Webber, J. B. W. bi-symmetric log transformation for wide-range data, Measurement Science and Technology, 2012.
Sharma, G., Wu, W., and Dalal, E. The CIEDE2000 color- difference formula: Implementation notes, supplementary test data, and mathematical observations. Color Research & Application, 2005.
Wu, Z.,, Dvomikk, N., Greff, K., Kipf, T., and Garg. A. Slot- Former: Unsupervised visual dynamics simulation with object-centric models. In International Conference on Learning Representations (ICLR), 2023.
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L.., Van Den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanetot, M., al. Mastering the game of Go with deep neural networks and tree search.
Yoon, J., Wu, Y.-F,, Bae, H., and Ahn, An investiga- tion into pre-training object-centric representations for reinforcement learning. In International Conference on Machine Learning (ICML). 2023.
Nature, 2016.
Singh, G., Deng, F., and Ahn, S. Illiterate dall-e learns to compose. In International Conference on Learning Representations (ICLR), 2021.
Yu, T., Quillen, D., He, Z., Julian, R., Hausman, K., Finn, C., and Levine, Meta-World A benchmark and evalu- ation for multi-task and meta reinforcement learning. In Conference on Robot Learning (CoRL), 2019.
Singh, G., Wu, Y.-F. and Ahn, S. Simple unsuper- vised object-centric learning for complex and naturalistic videos. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
Yuan, J., Chen, T., Li, B., and Xue, X. Compositional scene representation learning via reconstruction: A sur- vey. Transactions on Pattern Analysis and Machine Intel- ligence (TPAMI), 2023.
Singh, G., Kim, Y., and Ahn, S. Neural systematic binder. In International Conference on Learning Representations (ICLR), 2023.
Zadaianchuk, A., Seitzer, M., and Martius, G. Sclf- supervised visual reinforcement learning with object- centric representations, In International Conference on Learning Representations (ICLR), 2021.
Song, Y.JJ., Kim, H., Choi, S., Kim, J.-H., and Zhang, B.-T. Learning object motion and appearance dynamics with object-centric representations. In NeurlPS Workshop on Causal Representation Learning, 2023.
Zhang, B. and Sennrich, R. Root mean square layer normal- ization. Advances Neural Information Processing Systems (NeurIPS), 2019.
Spelke, E. S. Principles of object perception. Cognitive Science, 1990.
Sutton, R. S. and Barto, A. G. Reinforcement learning: An introduction. Bradford Book, 2018.
Zoran, D., Kabra, R., Lerchner, A., and Rezende, D. J. PARTS: Unsupervised segmentation with slots, attention and independence maximization. In International Con- ference on Computer Vision (ICCV), 2021.
Tassa, Y., Doron, Y., Muldal, A., Erez, T., Li, Y., Casas, D. d. L., Budden, D., Abdolmaleki, A., Merel, J., Lefrancq, A., et al. DeepMind control suite. arXiv preprint arXiv: 1801.00690, 2018
Todorov, E., Erez, T., and Tassa, Y. MuJoCo: A physics engine for bosel-based control. In International Confer- ence on Intelligent Robots and Systems (IROS), 2012.
Ugadiarov, L., Vorobyov, V., and Panov, A. Re- lational object-centric actor-critic. arXiv preprint arXiv:2310.17178, 2025.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L.,, and Polosukhin, I. Atten- tion is all you need. In Advances in Neural Information Processing Systems (NeurIPS), 2017.
12

---

SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels
A. Notation
Slot Attention for Video (SAVi)
Dz
The slot dimension
N
The number of slots
Ze
A set of slots Z. E DR at time-step &
Zoo
A history of slot-sets up to time-step t
en
SAVi encoder that maps O, to Z
dn
SAVi decoder that reconstructs Or from Z,
Fr
Features obtained by encoding images
L
Number of spatial locations in F
Reinforcement Learning
The number seed frames
The imagination horizon
Ou
An image observation
a
An action command
A reward
R
return
7
A scalar discount factor
H
The entropy of a probability distribution
FMLP
An MLP head that belongs to parameter group processed output token
h
13

---

SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels
B. Implementation Details
Table 2: Implementation details for each of SOLD modules.
SAVi
(b) Object-centric dynamics
(c) Slot Aggregation Transformer
Hyper-Param.
Value
Hyper-Param.
Value
Hyper-Param.
Value
Slot Dim. Dz
128
# Layers
4
Layers
4
# Slots N 2-10
Heads
8
Heads
8
Slot Init.
Learned
Token Dim.
256
Token Dim.
256
# Iters.
3/1
MLP Dim.
512
MLP Dim.
512
In this section, we describe the network architecture and training details for each of the SOLD components. Our models are implemented in PyTorch (Paszke et al., 2019), have 12 million learnable parameters, and are trained on a single NVIDIA A-100 GPU with 40GB of VRAM. summary of the model implementation details is listed Table 2.
B.1. Slot Attention for Video
We closely follow Kipf et al. (2022) for the implementation of the Slot Attention for Video (SAVi) decomposition model, including their proposed CNN-based encoder Cy and decoder dæ. the transformer-based predictor and the Slot Attention corrector. We employ between 2 and 10 (depending on the environment) 128-dimensional object slots, whose initialization learned via backpropagation. We empirically verified that learning the initial slots performs more stable than the usual random initialization. Furthermore, good initial decomposition, and single iteration for subsequent frames, which is enough to update the slot state given the observed features.
three Slot Attention iterations for the first video frame in order obtain
B.2. Object-centric dynamics model
Our object-centric dynamics model is based the OCVP-Seq (Villar-Corrales et al., 2023) architecture. which is transformer encoder employing sequential and relational attention mechanisms in order to decouple the processing of temporal dynamics and interactions, and has been shown to achieve interpretable and temporally consistent predictions. We use transformer layers employing 256-dimensional tokens, attention heads, and using a hidden dimension 512 the feed-forward layers.
B.3. Slot Aggregation Transformer
The Slot Aggregation Transformer (SAT) forms the architectural backbone for the reward, value and action models. This module aggregates information from object slots across multiple time steps to produce output tokens that are subsequently fed MLP heads in order to predict rewards, values, or actions. An overview of our SAT module is depicted in Figure 8.
SAT' is causal transformer encoder module that receives as input history of object slots, well learnable output token [out] for each time step, which is responsible for producing the final output for the corresponding time step. Additionally, we append to the SAT inputs a number of register tokens [reg] per time-step, which have been shown to aid with processing in autention-based models by offloading intermediate computations from the output tokens and helping the module focus on relevant slots (Darcet et al., 2024).
To encode the positional information into SAT, we employ Attention with Linear Biases (Press et al., 2022) (ALiBi), which introduces linear biases directly into the attention scores, effectively encoding token recency. This approach helps the model deal with sequences of varying length, as well as generalize to longer sequences than those seen during training, thus outperforming absolute positional encodings.
For our experiments, SAT is implemented with 4 transformer encoder layers with causal self-attention, R.MS-Normalization layers (Zhang& & Sennrich, 2019), attention heads, a token dimension of 256, and a hidden dimensionality in the feed- forward layers of 512. We set the number of learnable register token per time step to 4. Furthermore, we enforce in our causal attention masks that tokens belonging to time step cannot directly interact with previous output and register tokens.
14

---

SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels
XJV
MLP
RMS Norm
4:
91
RMS Norm
Zg
Z
Figure 8: The Slot Aggregation Transformer applies causal masking and ensures that output and register token do not attend to themselves on other time-steps. The recency bias induced by ALiBi is visualized through the color gradient the attention mask, with lighter shades blue corresponding to a higher negative bias on the attention scores.
B.4. Training Details
SAVi Pretraining SAVi pretrained for object-centric decomposition on approximately one million frames for 400,000 gradient steps. We use the Adam optimizer (Kingma & Ba, 2015), batch size of 64 and base learning rate of 10-1, which is first linearly warmed-up during the first 2,500 training steps, followed by cosine annealing (Loshchilov & Hutter, 2017) for the remaining of the training procedure. We perform gradient clipping with maximum norm of 0.05.
SOLD Training SOLD is trained using the Adam optimizer (Kingma & Ba, 2015) and different learning rates for each component: 10-4 4 for the dynamics and rewards models, and 3 10 for training the action and value models, as well as for fine-tuning the SAVi encoder. To stabilize training, we perform gradient clipping with maximum norm of 0.05 for the SAVi model, 3.0 for the transition model, and 10.0 for the reward, value, and action models. For all components, we also use learning rate warmup for the first 2,500 gradient steps. Additionally, we implement the exponential moving average (EMA) for the target value network with decay rate of 0.98. We imagination horizon of 15 steps for behavior learning, and the X-parameter is set 0.95.
15

---

SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels
C. Baselines
In our experiments we compare our approach with three different baseline models, namely the state-of-the- model-based RL algorithms DreamerV3 and TD-MPC2 and a Non-Object Centric variant of our proposed model.
DreamerV3 DreamerV3 (Hafner al., 2025) is SoTA model-based reinforcement learning algorithm that learns behaviors from visual inputs without requiring task-specific inductive biases extensive environment interaction. It builds world model that predicts future states and rewards, which is then used to simulate potential outcomes and guide the agent's s decision. DreamerV3 leverages latent dynamics and compact, holistic representation of the environment for efficient exploration, while showing desirable properties such as sample efficiency, scalability, and generalization across a wide range of complex tasks. We select the 12-million- partmmeter variant to match the parameter count of our proposed model. For further details, we refer to (Hafner et al., 2025).
TD-MPC2 TD-MPC2 (Hansen et al., 2024) builds upon its predecessor, TD-MPC (Hansen et al., 2022), with a series of architectural enhancements that improve scalability and sample efficiency. Like its predecessor, it avoids reconstructing high-dimensional inputs and instead focuses on modeling task-relevant dynamies in the latent space. The method employs temporal difference (TD) learning predict future returns in the latent space and uses model predictive control (MPC) to optimize action sequences. Key advancements in TD-MPC2 include enhancements to training stability for larger model architectures and better generalization across tasks. These innovations allow it to achieve state-of-the tee-the-rrr art performance on challenging visual and continuous control problems. For our experiments, we utilize the default 5-million-parameter variant since it recommended by the authors for single-task RL problems.
Non-Object-Centee Centrie Baseline This baseline model follows the same general framework as our proposed model, but replaces the object-centric SAVi encoder and decoder with simple convolutional auto-encoder while keeping the remaining modules unchanged; thus allowing learning. The CNN auto-encoder used in this baseline consists of an encoder comprised of four strided convolutional layers with 64, 128, 256, and 512 channels respectively, each followed by batch normalization and ReLU. The output of the final convolutional layer is flattened and fed through a linear layer to produce decoder mirrors the encoder structure, reconstructing the observations from the latent representation through the use of four transposed convolutional layers. To compensate for the lack of multiple latent vectors and to ensure a fair comparison, we increase the capacity of this baseline model by scaling the actor, critic, and dynamics models. Specifically, we increase the token dimension from 256 512, as well as the MLP hidden dimension from 512 1024. The total parameter count for this baseline is approximately 60 million, thus being five times larger than proposed method and the DreamerV3 baseline.
ablate the effect of object-centric representations for model-based reinforcement
12-dimmeniional latent vector. The
16

---

SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels
D. Environments
In this section, we provide further details about our proposed suite of environments, which includes eight object-centric robotic control tasks designed test relational reasoning and manipulation capabilities. The environments, which are inspired by (Li et al., 2020) and are simulated using MuJoCo (Todorov al., 2012), follow the same basic structure, consisting of robot arm mounted on base, positioned near table where the manipulation tasks take place.
In all environments, the robot is controlled by 4-dimensional action vector a (ax, By: 0z agripl E [-1, where the first three components represent the desired movement direction of the end-effector, whereas the fourth component controls the opening and closing of the gripper. On the Reach and Push tasks, commands to the gripper are ignored, with the gripper fixed in a closed configuration, as gripping is not required to solve these tasks.
For all tasks, we define the following constants:
f11 = 20 and 1/2 = 10: Temperature parameters that determine the steepness of the reward function.
dm 0.05: Distance threshold (in meters) for considering task successful.
Reach In Reach tasks, the agent must identify a spherical target among several distractors and move the end-effector its location. The reward is calculated as:
r= exp(-tt l/p. -P:ll2),
(12)
where Pe is the position of the end-effector and p; the target position. Success is defined through the following condition the last time step of an episode:
[1 if lp. - Pill2 dm
success
0 otherwise
(13)
Push & Pick Both Push and Pick correspond to reasoning and manipulation tasks where the agent must identify single block among several distractors and move it to a target location. In Push tasks, the agent can slide the block to the target position on the table without using its gripper; whereas in Pick the target location can be above the table, thus requiring the agent to grasp the block in order to lift to the target position. In both task variants the reward is calculated as:
r 0.9 exp(- -t1 llp. - ppll2) + 0.1 exp(-ts llpe Pellz),
(14)
where Pe is the position of the end-effector, p. is the target position, and P. is the block position. Success is defined through the following criterion, evaluated at the last time step of the episode:
{
1
if [pp. P:ll2 dm
success
(15)
otherwise
17

---

SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels
E. Additional Results
E.1. Object-centric Decomposition
Individual slots
True
RGB
Model
Mask
Figure 9: Object-centric SAVi decomposition of a video frame. We show the masked RGB image and the segmentation mask corresponding to each object slot. The masked RGB images are combined to reconstruct the observed frame.
Figure 9 depicts the object-centric decomposition of a video frame obtained by SAVi. SAVi parses the input frame into per-object RGB reconstructions and alpha masks, which can be combined via a weighted sum in order to accurately reconstruct the observed video frame. Notably, SAVi assigns an object slot to the scene background, five slots to different blocks, one slot to the red target, and one slot to the robot arm. The sharp object masks demonstrate that SAVi isolates object-specific information each slot, which is beneficial for downstream applications such as behavior learning, allowing the agent reason about object properties and their relationships while abstracting task-irrelevant details. Moreover, we find that SAVi is able adapt to the varying number of objects in our environments, leaving extra slots empty when they not needed represent scene.
E.2. Open-loop Prediction
We visualize action-conditiioal open-loop predictions in the Push-Specific (Figure 10), Button-Press (Figure 11), Hammer (Figure 12), Pittpole-Balanee (Figure 13), and Finger-Spin (Figure 14) environments.
Specifically, we present the ground truth sequence, predicted video frames, instance segmentation where each object mask is assigned a distinct color - and object reconstructions for each slot.
examples, our model parses the scene into sharp, accurate object representations and models action-conditional object dynamics and interactions, enabling precise future frame predictions while maintaining object-centric representations.
We highlight SOLD's ability capture complex physical interactions, such pushing block target location (Figure 10), pressing button (Figure 11), or hammering nail (Figure 12).
Furthermore, we demonstrate that SOLD generalizes to diverse, non-object-centic enviromments (Figure 13 and Figure 14), where sharp object separations and meaningful groupings emerge automatically for instance, an object's shadow, despite being spatially distinct, is assigned to the same slot as the object itself.
18

---

SOLD: Slot Object-Centrie Latent Dynamics Models for Relational Manipulation Learning from Pixels
Context
Open-loop prediction
True
Model
Slots
Individual slats
t=
2
4
10
20
30
40
50
Figure 10: Open-loop prediction on the Push-Specific task. We visualize the ground truth, predicted frames, segmentation obtained by assigning different colors to each object mask, and per-object reconstructions. In this sequence, SOLD assigns one slot to the background, one slot for the robot, one slot for the target, and four different slots for blocks, while one slot remains empty.
19

---

SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels
Context
Open-toop prediction
True
Model
Slots
Individual slots
t=
2
10
20
30
40
50
Figure 11: Open-loop prediction and decomposition results on the Button- Button-Press Press task. We visualize the ground truth and predicted video frames, instance segmentation obtained by assigning a different color to each object mask, and per-object reconstructions. SOLD assigns a slot for the scene background, two slots for different robot parts, and a slot for the button-box.
20

---

SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels
Context
Open-loop prediction
True
Model
Slots
Individua slots
1=0
2
4
10
20
30
40
50
Figure 12: Open-loop prediction and decomposition results on the Hammer task. We visualize the ground truth and predicted video frames, instance segmentation obtained by assigning a different color to each object mask, and per-object reconstructions. SOLD assigns slot for the scene background, three slots for different robot parts, slot for the hammer, and slot for the nail-box.
21

---

SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels
Context
Open-loop prediction
True
Model
T
Slots
Individual slots
t=
4
9
10
20
30
40
50
60
Figure 13: Open-loop prediction and decomposition results on the Cartpole-Balance predicted video frames, instance segmentation obtained by assigning a different color to each object mask, and per-object reconstructions. SOLD assigns a slot for the scene background and a slot for the cart-pole. Notably, the slot represents the object along with its shadow.
Balance task. We visualize the ground truth and
22

---

SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels
Context
Open-loop prediction
True
Model
Slots
Individual slots
t=
10
20
30
50
Figure 14: Open-loop prediction and decomposition results on the Finger-Spin task. We visualize the ground truth and predicted video frames, instance segmentation obtained by assigning a different color to each object mask, and per-object reconstructions. SOLD assigns slot for the scene background, slot for the finger, and slot for the spinning target. Notably, the slots represent the objects along with their corresponding shadows.
23

---

SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels
E.3. SAVi Fine-tuning
The Pick tasks highlight the need to design object-centric encoder-decoder modules that can adapt changing state distribututions. Figure exemplifies this challenge in the Specific variant, where the green target cube dissolves when lifted in the non-fine-tuned model.
Figure 15 underscores the importance of fine-tuning the object-centric encoder-decoder model with another example. Without fine-tuning, the blue color, which appears similarly on both colored blocks and the robot arm, leads to an even more drastic degradation of the reconstructions, where the robot itself no longer accurately captured. In contrast, the fine-tuned model is able reconstruct the sequence accurately.
True Observations
Frozen SAVI
Finetuned SAVi
Figure 15: Comparison of fine-tuned and frozen SAVi models on Pick-Distinct. We visualize the full reconstruction and the slot that reconstructs the cube that is being lifted for both models. When the blue block is lifted off the table, the frozen model merges it with blue elements from the robot arm, deteriorating the prediction and hallucinating the arm going between the gripper fingers. The fine-tuned model, on the other hand, is able reconstruct the sequence accurately.
24

---

SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels
Figure 16: Original, uncut slot- history used Figure 5. The full rollout highlights both the recency bias introduced by ALiBi and the model's ability to overcome this bias when task-relevant information must be retrieved from slots far the past.
Figure 17: Visualizing the actor's attention over the lot-hhistory on the Pick Pick-Specre Specific task reveals that the robot's s gripper and target block in the current time time-step step receive the most attention, highlighting the model's ability to focus on task-relevant components. Further. the red target sphere is mostly occluded in the current time- step making difficult to reconstruct its position accurately. However, the model learns to integrate information from previous steps, where it was still clearly visible,
E.4, Discovering Task-rele relevant Objects
25


Reviews:
[{"review_id": "uBVPRVGhN1", "reviewer": "Reviewer_kT4P", "summary": "The work focuses on the development of a novel model-based reinforcement learning (RL) algorithm that utilizes an object-centered representation of visual scenes. The authors extend the standard model-based approach of Dreamer by incorporating a slot representation generated using the SAVi transformer model. Instead of relying on the traditional Dreamer recurrent state model (RSSM), the authors employ a newly developed object-centric dynamics model OCVP. One of the key innovations introduced by the authors is the utilization of a unique model for the reward function - the slot aggregation transformer (SAT). This model does not directly generate a scalar reward, but instead generates softmax logits of the softmax distribution from which the actual reward is sampled. Additionally, register tokens and ALiBi-based position encoding are employed. The authors conduct experiments in their custom environment involving a robotic arm and larger objects, comparing their approach with Dreamer and TD-MCP. They also indicate extension of their experiments to other platforms such as MetaWorld and simpler versions of DMControl.\n\n## update after rebuttal\nDuring the rebuttal phase, the authors conducted further experiments to evaluate the impact of SAT on the overall architecture and performed preliminary experiments on generalization. I agree with the authors that they have shown the promise of an object-centric approach that works at the level of monolithic approaches. However, this is not the first work in this field to demonstrate this. In general, I believe the proposed method has novelty, so I leave the current assessment Weak Accept.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We want to thank the reviewer for acknowledging that our approach, and the combination of object-centric (OC) representations and RL generally, is interesting and a promising research area. Further, we want to thank them for acknowledging the SAT architecture as one of the key novel contributions of our work. We address the raised questions below and will incorporate all the feedback for the paper and the provided references to related works into the final version.\n\n**Is this ***“the first approach based on an OC world model”***?**\n\nWe want to thank the reviewer for raising this important point and providing a reference to the GOCA paper. While GOCA (called ROCA in the newest version) is specifically focused on value-based MBRL, using the world model to improve return estimates of the critic instead of learning through imagination inside world model rollouts, we want to make sure to characterize the novelty of our method accurately and not overlook any prior contributions. To this end, we add a more detailed discussion of the differentiation of our method to prior work on OC representations (OCRs) in RL (including all references provided by the reviewers) to the Appendix of our paper.\nMoreover, we would appreciate the reviewer’s feedback on the characterization of SOLD as the first method to learn purely inside imagined rollouts of an object-centric world model trained from pixel inputs.\n\n**Why are the evaluations performed on non-standard benchmarks/ custom environments?**\n\nTo make the most of the limited space, we kindly refer to our reply to this question in the response to o1iz.\n\n**Why does SOLD not show a significant advantage over monolithic models in a standard setting?**\n\nWe assume standard setting to refer to tasks that only consider distractor objects, without requiring explicit relational reasoning between those objects (Reach-, Push-, and Pick-Specific). While we want to point out that SOLD still outperforms state-of-the-art baseline methods on the more challenging Push and Pick instantiation of this task, TD-MPC2 performs best on the Reach-Specific environment. \nThe experience used in pre-training to learn OCRs is a cost in the SOLD algorithm that is “paid” upfront and becomes valuable through improved representations over the course of training. On the easiest Reach-Specific task, TD-MPC2 has converged even before the experience required for pre-training SAVi is used up. To us, this is neither unexpected nor troubling, since the problem is too easy to master to warrant the extra effort in acquiring strong representations upfront. Specifically TD-MCP2, which operates in a reconstruction-free manner, can quickly extract the only relevant information from the visual representation, which in this case is the position of the green sphere and the robot’s end-effector.\nWe still include this easier task to paint a fuller picture about when the use of OCRs has yielded a payoff for the total sample complexity and when baseline methods are already performing well. \n\n**How many frames are used to pre-train SAVi?**\n\nWe have included this in the details about SAVi Pretraining in Section D.4 in the Appendix (Line 797). We pre-train SAVi for approximately 1 million frames (including the full episode in which the count of 1 million frames is reached) on all tasks, which we have found empirically to provide a sufficient basis to learn disentangled OC representations.\n\n***\"The code is missing, which makes it impossible to evaluate the correctness of the implementation and experiments.\"***\n\nThe source code is available on the project website linked at the end of the abstract.\n\n**What do $R_t$, $e_\\eta$, etc. refer to?**\n\n$R_t$ represents the return from time-step $t$. $e_\\eta$ and $d_\\eta$ represent a SAVi encoder and decoder model with parameters $\\eta$, respectively. Due to the limited space for the main text, we have added explanations for $R_t$, $e_\\eta$, and $d_\\eta$ to the Notation in Section B of the Appendix. \n\n***\"I would like to see an ablation study on the SAT-related part of the model - how will it work with and without it?\"***\n\nWe have added ablations to key design choices of our method to better justify the decisions. The results of all experiments are available at https://anonymous.4open.science/r/sold-rebuttal/ABLATIONS.md\n\n**How do the authors assess the possibilities of generalizing OC models, for example, to change colors?**\n\nThis is an interesting point for further investigation, and we have conducted a preliminary experiment changing the set of colors from originally 16 to 32 novel values (see https://anonymous.4open.science/r/sold-rebuttal/COLOR_CHANGE.md). The results indicate that both DreamerV3 and SOLD are able to generalize to this setting well and that stronger out-of-distribution experiments are required to inspect this property in a dedicated investigation.\n\n---\n\nWe hope our response has clarified your concerns, and would appreciate it if you considered updating your score.", "claims_and_evidence": "The work is written in a clear and easy-to-understand language, although there are some issues with the notation and the introduction of the RL problem. The authors acknowledge some advantages of object-centric representations over monolithic ones, but they only discuss these advantages in a specific env. While it is possible to agree with some of the claims made by the authors, it is difficult to accept their assertion that this is the first approach based on an object-centric world model (see discussion of the literature in this area below).", "methods_and_evaluation_criteria": "The authors use a combination of existing models, including Dreamer, SAVi, and OCVP. However, the main new contribution of their work is the SAT reward model. For this, they have applied a number of novel techniques, resulting in a high-quality model.\nOne challenge they faced was using their own environment, as it is common in the research community to use more established benchmarks such as RoboSuite, CasualWorld, and Crafter. Additionally, the experiments with MetaWorld and DM-Control were not thoroughly conducted and were not discussed in detail.", "theoretical_claims": "The authors do not offer any theoretical analysis of the proposed model, but generally justify the loss functions used.", "experimental_designs_or_analyses": "The experiments conducted are not very convincing. Even in a setting designed specifically for their method, with large, clearly defined objects and without noise, the SOLD approach does not show a significant advantage over monolithic models in a standard setting, actually demonstrating slower learning. Additionally, the authors did not report the number of steps used to pre-train the OCVP model. There are only good results in a very specific context. Analysis of the results and comparisons with other methods in the MetaWorld and DM-Control environments were not provided. Another major drawback is the lack of comparison to other object-centric, model-free, and MBRL approaches (as described in the literature).", "relation_to_broader_scientific_literature": "The authors do not mention all the available work on object-centric representations in RL. Authors especially miss such works in the field of MBRL, for example, GOCA (see the literature section). At a minimum, they need to be discussed, and not to claim that such works do not exist.", "essential_references_not_discussed": "The authors are encouraged to discuss and conduct an experimental comparison with the following approaches:\n1)\tModel-based - GOCA https://arxiv.org/abs/2310.17178\n2)\tModel-free – OCRL (cited as Yoon et al., 2023), OC-SA (https://arxiv.org/pdf/2208.03374)  и OCARL (https://arxiv.org/abs/2210.07802).", "other_strengths_and_weaknesses": "I am very impressed by the object-centric approach and agree with the authors that it is a promising area of research. However, there are currently some difficulties in applying it in interesting and acceptable environments within the RL community, largely due to limitations of slot attention and visual self-supervised learning methods in general. In this regard, it is important to mention the work done in this area and avoid expanding the range of specific environments. Instead, it would be better to focus on standard environments like CasualWorld or MetaWorld, which would allow for more reliable comparisons and evaluations. I would recommend that the authors conduct their experiments using these environments as a basis, using them as a demonstration and additional reference. Additionally, I would like to see evidence of the benefits of object-based learning, such as improved generalization and transferability to other tasks.", "other_comments_or_suggestions": "The authors did not set the RL task itself, and part of the notation is not entered or deciphered, which makes it difficult to read the article. For example, the concepts of R_t, e_\\eta, etc. are not introduced. Also, when describing the method, it is difficult to separate the authors' introductions and previously used techniques introduced back in DreamerV3.", "questions_for_authors": "1) How do the authors assess the possibilities of generalizing object-centric models, for example, to change colors?\n2) How is the required amount of data estimated for OCVP pre–training?\n3) I would like to see an ablation study on the SAT-related part of the model - how will it work with and without it?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nDba9u4tBf", "reviewer": "Reviewer_o1iz", "summary": "The paper proposes SOLD, a method for model-based reinforcement learning (MBRL) with object-centric world models. The paper is mainly a combination of ideas from OCVP object-centric video prediction and DreamerV3-style MBRL with latent dynamics model being shaped to be object-centric. The main motivation is to use object-centric representations as an inductive bias to improve sample efficiency and interpretability. For evaluation, the authors compare their methods with DreamerV3, TD-MPC2 and a variant of their architecture that excludes the object-centric representation. They build their own in environments on top of what seems to be Gymnasium-robotics environments to create object-centric manipulation tasks. Their approach proves to be equal or better to the baselines and also successful on some select benchmark tasks from DMC and MetaWorld.\n\n## update after rebuttal\n\nThe authors addressed some of my concerns, but not all (missing MFRL baselines and standardized benchmarks). Nonetheless, their arguments and new evaluations are sufficient to raise my score.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We want to thank the reviewer for the detailed feedback. We are encouraged that they found our idea strong and clear, and that they appreciate the improvement in interpretability and sample efficiency. We aim to answer remaining questions and concerns below, but will incorporate all feedback for the paper (such as the discussion of key works from the model-based RL (MBRL) literature) into the final version.\n\n**Why are the evaluations performed on non-standard benchmarks/ custom environments?**\n\nWe agree that comparative evaluation on diverse standard benchmarks, such as RLBench, is a crucial direction to explore. Our motivation for the selected evaluation is twofold: (1) feasibility with current OC representation (OCR) learning methods and (2) explicit investigation into relational reasoning capabilities.\n\nAt the moment, many of these benchmarks are still beyond the capabilities of OC methods. The main purpose of our work is to evaluate the benefit OCRs can bring to learning control policies on environments where current methods can learn sufficiently disentangled representations to accurately investigate this question.\n\nMoreover, while we agree that all robotic manipulation is at its core object-centric, we are interested in specifically studying skills that go beyond the abilities of current methods operating on holistic representations (akin to prior work investigating OCRs in RL (https://arxiv.org/abs/2302.04419, https://arxiv.org/abs/2011.14381, https://arxiv.org/abs/2404.01220)). Improved relational reasoning, characterized as a cornerstone of human intelligence, is such a skill that is insufficiently challenged in standard benchmarks and that becomes increasingly important as RL agents are tasked to perform tasks based on high-level semantic instruction.\n\n**Did you perform ablations of key design choices?**\n\nWe have added ablations to the key design choices here: https://anonymous.4open.science/r/sold-rebuttal/ABLATIONS.md \n\n**I would suggest adding a baseline on model-free RL with the OCRs.**\n\nWe thank the reviewer for characterizing the comparative baselines we employed as adequate, but agree that adding an OC model-free (MF) RL method would add an interesting dimension to compare the performance of MB vs. MF methods operating on OCRs. Given the one-week timeframe for the rebuttal, implementing and evaluating such a baseline is beyond our means, and we have focused our efforts on the extensive ablations and additional experiments requested by the reviewers that were feasible within this time frame. We thank the reviewer for this suggestion and consider the comparison to OC-MFRL to be an interesting direction for future work.\n\n**Is the method ***\"a mere combination of OCVP and DreamerV3\"***?** \n\nWe agree that extending OCVP to an action-conditional setting and designing an architecture to perform MBRL on the basis of this representation is central to our work. However, we believe that *“a mere combination of OCVP and DreamerV3”* undervalues the contribution. Both the action-conditional OC prediction and the MBRL on the basis of sequences of object slots in latent imagination (enabled by the SAT architecture) are novel contributions that address non-trivial challenges in OC-MBRL, which we believe to be of interest to the community.\n\n***\"How would your method perform in environments with more complex action spaces?\"***\n\nWe appreciate this question, since accurately modeling the effects of actions is crucial for learning control. While we have included predictions on DM-Control environments, where the action-spaces are substantially different from the position + gripper control of our task suite, we aim to show the feasibility of our world model to apply to more complex action-spaces. Therefore, we have performed additional experiments on:\n1. The Sketchy dataset (https://arxiv.org/pdf/1909.12200), which features 7-dimensional actions, extending the action-space of our environments by a rotational component, and \n2. A custom Moving Shapes environment featuring multiple shapes of distinct colors, where the action-space encodes information about both *what* happens (e.g., move upwards) and *to which* object (e.g., the red cube) and allows for actions to simultaneously apply to all objects.\n\nOn the Sketchy dataset, results show that our prediction model is capable of accurately predicting the movement of the robot even under gripper rotations. For the Moving Shapes environment, we observe that the dynamics model is able to correctly associate actions given as a single vector to the different objects they apply to.\nWe encourage you to view the result of this investigation here https://anonymous.4open.science/r/sold-rebuttal/COMPLEX_ACTION_SPACES.md \n\n---\n\nWe appreciate the reviewer's characterization of OCRs as long due for MBRL and his recognition of our idea. We hope that we have been able to address outstanding questions and concerns, and would be very grateful if you would consider updating your score.", "claims_and_evidence": "The paper claims to be the first to do object-centric MBRL, which to the best of my knowledge is true.\n\nIn addition, the authors claim that object-centric representations are a good inductive bias to improve 1) interpretability and 2) sample efficiency of MBRL. The qualitative results shown by the authors in terms of slot rollouts do support the claim of improved interpretability. However, sample efficiency is hard to judge given that the authors only demonstrate this feature on environment that they made. It is then hard to say whether the environment was created in a way that favors the method or if the method is generally more sample efficient.", "methods_and_evaluation_criteria": "The proposed method makes absolute sense. Object-centric representations are long-due for MBRL and are a great idea to improve interpretability, performance, and sample efficiency of RL methods.\n\nThe evaluation methods are not sufficient. The authors did a good job on the research direction and method implementation side of things. However, they fail to properly validate their proposed method. It is crucial to evaluate the proposed method on standard RL and manipulation benchmarks (e.g., MetaWorld, Robosuite, RLBench...). Most manipulation benchmarks require object-centric representations since manipulation by definition requires the robot to manipulate some objects in its environment.\n\nThe choice of baselines is adequate, though it is quite minimal (check my comment in the weakness section).", "theoretical_claims": "No theoretical claims.", "experimental_designs_or_analyses": "Experimental design lacks using standard benchmarks from the manipulation literature.", "relation_to_broader_scientific_literature": "The paper is a great combination of OCVP and DreamerV3, bringing recent ideas from video prediction to model-based RL for visual environments.", "essential_references_not_discussed": "The related work section is missing some key works from the MBRL literature to name few:\n\n[1] Deisenroth, Marc, and Carl E. Rasmussen. \"PILCO: A model-based and data-efficient approach to policy search.\" Proceedings of the 28th International Conference on machine learning (ICML-11). 2011.\n[2] Nagabandi, Anusha, et al. \"Neural network dynamics for model-based deep reinforcement learning with model-free fine-tuning.\" 2018 IEEE international conference on robotics and automation (ICRA). IEEE, 2018.", "other_strengths_and_weaknesses": "**Strengths:**\n\n- Using object-centric representation for MBRL is a great idea and a very promising direction.\n- The paper is well-written an easy to follow\n- The method includes multiple key design choices that enabled its performance on the target tasks.\n- The proposed approach improves interpretability of MBRL, which is a major claim of the work that is well-supported by the experiments.\n- The evaluation showcases the improvement in sample efficiency on the chosen tasks.\n\n**Weaknesses:**\n\n- My main concern is with the evaluation of the work. The authors should include some evaluation on more standard benchmarks that are also object-centric such as the ones from RLBench, MetaWorld, and Robosuite. The authors show the success of their methods on some select standardized benchmarks but they do  not compare their method to baselines in these domains.\n- The paper lacks ablations of the key design choices proposed in the paper.\n- The choice of baselines is minimal (sufficient to validate the claims, but not sufficient to highlight the benefits of the paper). I would suggest adding a baseline on model-free RL with the object-centric representations.\n- While the method is interesting, the novelty of the work is quite limited by the fact that it is a mere combination of OCVP and DreamerV3", "other_comments_or_suggestions": "None", "questions_for_authors": "- Given that author benchmark include object-centric tasks, what was the main motivation for building custon environments to evaluate the method? from an outsider's perspective and without you properly motivating this choice, it sounds like the environments were built to favor the proposed method.\n- How would your method perform on environments with more complex action spaces? Since action-conditioning is one novelty from the side of object-centric video prediction, it would be interesting to see this working on more complex action spaces.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "utZungJAE5", "reviewer": "Reviewer_tPrN", "summary": "The manuscript introduces the slot object-centric latent dynamics (SOLD) model, a reinforcement learning (RL) algorithm that leverages an object-centric latent world model, which is learned directly from pixels, for behavior learning. The world model is an extension of object-centric video prediction (OCVP), a model that leverages slot attention for video (SAVi) for object centric representation discovery and a transformer-based prediction model with two attention mechanisms for separately capturing temporal information of the same object and relational information about interactions between objects. The world model in SOLD differs from OCVP in that 1) it conditions the transformer’s predictions on past and current actions in addition to the observation history, 2) SAVi is pre-trained on random observation sequences from random episodes, and 3) a reward prediction model is added to enable behavior learning purely from imagined trajectories.\nFor behavior learning SOLD uses a very similar approach to DreamerV3. The proposed method is evaluated against DreamerV3, TD-MPC2, and SOLD w/ a CNN encoder instead of SAVi on a suite of eight object-centric robotic control environments, exhibiting superior performance. The method is also shown to work on environments that are not designed with object-centric learning in mind.\n\n## update after rebuttal\nThe authors' rebuttal addressed some of my concerns. However, I share the concern with other reviewers that the evaluation is no thorough enough. For instance, the authors mentioned that hyperparameters for baselines were copied from prior work which used different environments, while for SOLD some hyperparameters are tuned. Comparison on more established benchmarks would make the results more convincing. I still believe the work contributes valuable insight for the community and lean towards acceptance.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We want to thank the reviewer for acknowledging the clarity of our paper and methodology and for the valuable references, which we will discuss in the final version. In the following, we aim to address your questions.\n\n***\"The method’s differences to other OC world models should be discussed in (more) detail.\"***\n\nWhile we discuss these references in detail in the final version, we want to briefly point out differences to our method here:\n- While Bize et al. (https://arxiv.org/pdf/2204.13022) consider the problem of adding actions to an OC world model, they do so only for action-cond. prediction, not for behavior learning with MBRL (“A dataset of expert demonstrations is available [...] and the model is evaluated on its ability to predict the block positions”, Section 4.3).\n- Collu et al. (https://arxiv.org/pdf/2402.03326) learns a slot-attention (SA)-based OC dynamics model. While it is referred to as a world model, no reward prediction, action-conditioning or behavior learning is done.\n- Heravi et al. (https://arxiv.org/pdf/2205.06333) use OCR learning on robotic manipulation tasks, showing that SA-based representations yield performance improvements for object location prediction. However, they consider neither world models nor RL (“We only consider the imitation learning setup and learn a policy from a dataset of demonstrations”, Section IV.E).\n- van Bergen et al. (https://arxiv.org/pdf/2209.01258) consider the prediction of video frames on the basis of OCRs but follow the active inference framework.\n- While Zadaianchuk et al. (https://arxiv.org/pdf/2011.14381) use SCALOR to infer OCRs from pixel inputs, they use model-free RL to learn their policies (“our policy can be trained with any goal-conditional model-free RL algorithm”, Section 4.2).\n\nWe want to make sure to depict the novelty of our method accurately and to give credit to all prior works, so we would greatly appreciate the reviewer’s opinion on the rephrased contribution to differentiate SOLD from prior works given in response to Reviewer kT4P (Question 1).\n\n***\"How could one address the problem of SA potentially not separating objects well in more complex environments? What other methods for OCRs have you considered?\"***\n\nWhile the objective of our research is to provide insights into the potential of MBRL from OCRs when such representations can be learned well with current methods, rather than to improve the OCR learning method itself, we concur that learning meaningfully separated objects on complex visual data is a central point for expanding the range of applications of our method.\nIn addition to SAVi, we have experimented with using STEVE (https://arxiv.org/pdf/2205.14065) and VideoSAUR (https://arxiv.org/pdf/2306.04829). However, we observed that the size of the pre-training dataset was the central factor in determining the quality of the decompositions for all three methods. \nBecause STEVE and VideoSAUR add complexity to the OCR learning problem and make it challenging to verify the isolation of object-specific information in each slot (this is because STEVE uses an autoregressive transformer decoder instead of predicting per-slot alpha masks, and VideoSAUR predicts ViT embeddings at the patch level), we found SAVi to be the most suitable model to achieve the goals of our investigation. Namely, to study behavior learning from OCRs that sufficiently disentangle visual scenes, and to explore interpretable patterns in the model's learned attention weights.\nHowever, there are several avenues to extend our work in the future to apply it to visually complex data, including the prediction of ViT embeddings instead of pixels (especially for larger images), and the use of vision-foundation models to guide slot initializations.\n\n**Does SA occasionally collapse to degenerate solutions?**\n\nSAVi training can occasionally collapse (e.g., poor object separation despite good reconstruction) due to misconfiguration (not enough slots), insufficient pre-training data, or seed dependence. See examples: https://anonymous.4open.science/r/sold-rebuttal/SAVI_FAILURE_CASES.md.\nHowever, we found SAVi robust during fine-tuning once converged, which is valuable for MBRL.\n\n**Have you considered partially observable environments?**\n\nOcclusions have induced the demand for challenging long-horizon reasoning for the considered tasks. A key objective of our work was to ascertain whether the proposed models are successfully able to resolve the resulting ambiguities. \nAdditionally, occlusion, as an object-dependent source of partial observability, enables interpretable visualization of the attention patterns to inspect how the model is retrieving missing information. Investigating other sources of partial observability is important future work, especially for real-robot deployment.\n\n---\n\nWe hope that we have been able to address open questions and better situate SOLD in the OC-RL landscape. If so, we would be thankful if you considered updating your score.", "claims_and_evidence": "* The model is shown to work well on the considered relational reasoning tasks.\n* “We introduce SOLD, the first object-centric MBRL algorithm to learn directly from pixel inputs [...]”\nI believe several works mentioned in the “Essential References Not Discussed” section of this review are implementing object centric world models that are trained with pixel supervision. Please clarify or adjust this claim.\n* Regarding SOLD outperforming state-of-the-art methods (SOTA): Those methods are SOTA on other benchmarks than the ones used in this study. The claim should be more narrow (i.e. on a new benchmark suite focusing on relational reasoning) or supported by additional comparisons on widely-used benchmarks.", "methods_and_evaluation_criteria": "At a high level SOLD replaces the DreamerV3 world model with one based on OCVP. The method is compared with DreamerV3, TD-MPC2 and SOLD w/o object-centric representation on a suite of robotic control tasks that require basic relational reasoning skills. The level of relational reasoning is limited to one-vs-all comparisons of color or spatial relation between the arm and one or two objects. How does SOLD compare with the baselines on the two tasks from the Meta-World benchmark or other more commonly used tasks?", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "* I could not find a description of how hyperparameters were selected\n* Appendix E mentions that the capacity of the Non-Object-Centric Baseline was increased for fair comparison. Did you evaluate whether that actually helps its performance? Model size or capacity is not always the best criterion for fairness. Is the CNN a standard architecture used in similar environments?", "relation_to_broader_scientific_literature": "According to my understanding the present manuscript is not the first to propose slot-based world models. The method’s differences to other object-centric world models should be discussed in (more) detail.", "essential_references_not_discussed": "Given the state of object-centric representations being underexplored in RL, as mentioned in the paper, I’d suggest to briefly discuss the relation to the following works:\n1. Biza, O., Platt, R., van de Meent, J. W., Wong, L. L., & Kipf, T. Binding Actions to Objects in World Models. In ICLR2022 Workshop on the Elements of Reasoning: Objects, Structure and Causality.\n2. Collu, J., Majellaro, R., Plaat, A., & Moerland, T. M. (2024). Slot Structured World Models. arXiv preprint arXiv:2402.03326.\n3. Heravi, N., Wahid, A., Lynch, C., Florence, P., Armstrong, T., Tompson, J., ... & Dwibedi, D. (2023, May). Visuomotor control in multi-object scenes using object-aware representations. In 2023 IEEE International Conference on Robotics and Automation (ICRA) (pp. 9515-9522). IEEE.\n4. van Bergen, R. S., & Lanillos, P. (2022, September). Object-based active inference. In International Workshop on Active Inference (pp. 50-64). Cham: Springer Nature Switzerland.\n5. Zadaianchuk, A., Seitzer, M., & Martius, G. (2020). Self-supervised visual reinforcement learning with object-centric representations. arXiv preprint arXiv:2011.14381.", "other_strengths_and_weaknesses": "* The paper is very well organized and explains motivation, methodology and experimental setup clearly.\n* Providing performance comparisons with the baselines on more established benchmarks and a discussion of more works in the intersection of object-centric representations and RL would significantly improve the paper and I'd probably increase the score.", "other_comments_or_suggestions": "The below citation only shows the first of multiple authors without “et al.”:\n\nCho, K. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014.", "questions_for_authors": "On potential limitations:\n* How could one address the problem of slot attention potentially not separating objects very well in more complex environments? What other methods for object-centric representations have you considered?\n* Does slot attention occasionally collapse to degenerate solutions?\n* Have you considered partially observable environments (beyond simple occlusions)?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XIpInLaNqS", "reviewer": "Reviewer_BRLw", "summary": "The paper proposes a model-based RL (Dreamer-like) algorithm that utilizes pre-trained (and then fine-tuned) slot-attention based object-centric representations as the underlying state representation, in contrast to the standard single-vector representation (holistic) typically employed when learning from pixels. The proposed approach outperforms or performs comparably to recent model-based RL approaches.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We want to thank the reviewer for appreciating the potential of our idea and the clarity of the paper. We address your questions below.\n\n***\"I’m worried about the generalizability of this method to more pixel-wise challenging tasks where SAVi fails to decompose the scenes.\"***\n\nPlease see our response to Reviewer tPrN (Question 2) regarding challenges with OCRs in complex scenes and potential future work. SOLD itself is designed to be agnostic to the specific OC model, allowing it to benefit from future OCR improvements.\n\n***\"The claim (line 326) 'This part-whole segmentation highlights the ability of slots to meaningfully identify and represent separate parts of a larger object, such as the gripper jaws of the robot' is not well-positioned.\"***\n\nOur intention with the phrasing “part-whole segmentation” was to highlight SAVi's ability to learn nuanced representations by assigning distinct slots to the gripper jaws (visualized in brown/gray in Figure 3) separate from the main robot arm body (orange). We view this emergent separation as beneficial, particularly as the gripper jaws have independent actuation via a dedicated component of the action vector, making their distinct representation potentially valuable for learning control. We did not intend to claim this specific granularity is universally superior, and indeed, as shown in Figures 10 and 11 in the Appendix, SAVi can represent the robot arm at varying levels of subdivision depending on the visual prominence and context. We will revise the phrasing in the paper for better clarity.\n\n***“I find the environments and tasks used in this work quite simple (also visually simple) and more challenging environments/tasks would help provide more evidence for the claims in this work.”***\n\nWhile we intentionally limit the visual complexity to study control learning when OCRs are obtainable with current methods (see also our response to Question 1 of Reviewer o1iz), we agree that addressing also more visually complex scenarios will be an interesting avenue for future work.\n\n**Why is the performance ***“not much better compared to non-OC baselines in most environments”***?**\n\nWhile SOLD outperforms baselines overall and especially on distinct tasks (Table 1), we deliberately included easier tasks (Reach/Push-Specific) to paint a fuller picture of the level of complexity at which OCRs yield payoffs and to be fair to the baseline methods. We found that the upfront cost of pre-training OCRs pays off as complexity increases. On the easiest Reach-Specific task, TD-MPC2 converges quickly (within the SAVi pre-training window), making the effort to acquire OCRs less warranted *for that specific task.* TD-MPC2 (which operates reconstruction-free) can rapidly extract the minimal necessary information (target/end-effector position) to solve the task. \nWe feel that leaving out such configurations would distort the picture of where significant improvements due to OCRs, accounting for the cost of both pre-training and behavior learning, were made.\n\n***“What does 'whose initialization is learned via backpropagation' mean?”***\n\nThis refers to having a parameter vector to which the slot is initialized, which is trained with back-propagation. We link the implementation at https://anonymous.4open.science/r/sold-rebuttal/slot_initialization/learned.py \n\n***“Ablations: have the authors ablated the choice of positional encoding, or just used AliBi by default?”***\n\nWe added ablations at https://anonymous.4open.science/r/sold-rebuttal/ABLATIONS.md \n\n***“I’m curious regarding the authors’ experience with SAVi’s stability [...]?”***\n\nPlease see our answer to Reviewer tPrN (Question 3).\n\n**How long does it take to train the model?**\n\nOn a single A100 (40GB): SAVi pre-training takes ~2-3 days. SOLD training takes up to ~10 days for the longest tasks (Push/Pick, 50 steps per episode). For comparison, DreamerV3 (optimized JAX code, holistic repr.) took ~4.5 days.\n\n**What is the effect of freezing/fine-tuning the SAVi backbone quantitatively?**\n\nWe added experiments comparing these two settings quantitatively: \nhttps://anonymous.4open.science/r/sold-rebuttal/SAVI_FINETUNING.md \nInterestingly, there is only a small drop in performance from the fine-tuned to the frozen SAVi model. We hypothesize this to be due to the SAT model, which can leverage a long history of slots, compensating for deteriorating representations.\nHowever, we chose to emphasize again the strong qualitative results in Figures 14 and 15 because it clearly indicates a potential failure mode of current OCRL approaches (e.g., SMORL, EIT), which pre-train the OC model and freeze it for the downstream RL task. We believe highlighting this issue and exploring fine-tuning as a strategy to maintain representation quality under distribution shift is an important discussion point for the OCRL community.\n\n---\n\nWe hope our responses have adequately addressed your concerns, and we would be grateful if you consider updating your score.", "claims_and_evidence": "* The paper claims model-based RL can improve sample efficiency over model-free RL, yet does not compare with one. In addition, there are sample-efficiency analyses.\n* There are no comparisons with model-free RL object-centric approaches. While the authors cited several works like SMORL (https://arxiv.org/abs/2011.14381) and Haramati et al. (https://arxiv.org/abs/2404.01220v1), I believe an empirical comparison should be done, especially if the authors wish to claim their method is more sample-efficient (I’ll note that both methods are goal-conditioned, but it seems like it should be possible to apply them on the environments in this paper, and applying SOLD in their environments as well). I also believe that the environments in both mentioned works are more challenging than the ones used in this work.\n* Generalizability: I’m worried about the generalizability of this method to more pixel-wise challenging tasks where SAVi (DDLP - https://arxiv.org/abs/2306.05957), and slot-attention in general (DINOSAUR - https://arxiv.org/abs/2209.14860), do not work well and fail to decompose the scenes.\n* The claim (line 326) “This part-whole segmentation highlights the ability of slots to meaningfully identify and represent separate parts of a larger object, such as the gripper jaws of the robot” is not well-positioned. Do the authors claim that the ability to segment objects like humans (e.g., whole gripper instead of joints) is better for control? Usually state-based representations in robotics include the positions of the joints, so one could claim that aggregating all the joints to one object might actually hurt the performance.", "methods_and_evaluation_criteria": "* The method makes sense for the problem, as well as the datasets.\n* However, I find the environments and tasks used in this work quite simple (also visually simple) and more challenging environments/tasks would help provide more evidence for the claims in this work.", "theoretical_claims": "There are no theoretical claims in this paper.", "experimental_designs_or_analyses": "My only issue, as mentioned above, is that I find the experimental benchmark not challenging and that the chosen benchmark does not shed much light with respect to sample-efficiency.", "relation_to_broader_scientific_literature": "* The main contribution of this work is extending slot-based video prediction models (OCVP) with actions and reward to make them world-models.\n* While I find this a solid contribution, I don’t think the performance of the proposed method, nor the chosen benchmark, are impressive enough to be convinced regarding the claims and the promise of the method.", "essential_references_not_discussed": "I would like to point-out a concurrent work on object-centric world models: OC-STORM (https://arxiv.org/abs/2501.16443). However, the methods are very different.", "other_strengths_and_weaknesses": "**Strengths**:\n* Open-source code (!)\n* Very detailed appendix and a nice project site.\n* Paper reads-well and easy to follow.\n\n**Weaknesses/Limitations**:\n* Reading all the training details, it seems the method is very brittle, and requires a lot of tuning (e.g., for learning rates and clipping values for each component).\n* Performance: I like the idea of the paper and I believe there is a lot of promise in unsupervised object-centric representations, especially for control tasks, and I was surprised that the performance is not much better compared to non-object-centric baselines in most environments. I do wonder if the problem is the choice of easy/simple tasks/environments or the choice of underlying object-centric representations (slot-attention).", "other_comments_or_suggestions": "None", "questions_for_authors": "* (Repeated from above) The claim (line 326) “This part-whole segmentation highlights the ability of slots to meaningfully identify and represent separate parts of a larger object, such as the gripper jaws of the robot” is not well-positioned. Do the authors claim that the ability to segment objects like humans (e.g., whole gripper instead of joints) is better for control? Usually state-based representations in robotics include the positions of the joints, so one could claim that aggregating all the joints to one object might actually hurt the performance.\n* Line 401: the authors report the returns on Cartpole-Balance and Finger-Spin, but without any comparison these numbers are meaningless. Should the reader have knowledge of other algorithms’ performance on these tasks?\n* Appendix, line 737: what does “whose initialization is learned via backpropagation” mean? How does one learn initialization?\n* Ablations: have the authors ablated the choice of positional encoding, or just used AliBi by default? Similarly, the number of register tokens?\n* What is the effect of freezing/fine-tuning the SAVi backbone quantitatively?\n* Stability: from personal experience, Slot-Attention can be very hard to get working (as also been shown in many previous works, like DINOSAUR and DDLP mentioned above), and not very consistent (i.e., different runs with same hyper-parameters but sometimes Slot-Attention does not provide a good decomposition to slots). This is more evident on more visually complex/real-world datasets. I’m curious regarding the authors’ experience with SAVi’s stability in that sense, was it common to have runs where SAVi did not provide good decompositions (e.g., multiple objects in a single slot)?\n* How long does it take to train the model (roughly, wall-clock hours)?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper proposes a model-based RL algorithm that learns object-centric dynamics models in an unsupervised manner from pixel inputs. The proposed method outperforms DreamerV3 and TD-MPC2 - state-of-the-art model-based RL algorithms - in benchmarks introduced in the paper. All reviewers appreciate the object-centric design of the dynamics model, but raise concerns regarding the simplicity of the environments considered. 
They suggest that the paper should include evaluations on more standard benchmarks that are also object-centric such as the ones from RLBench, MetaWorld, and Robosuite. 
However, since there is little research on object-centric dynamics models, all reviewers are positive for the acceptance of the paper.
Decision: Accept (poster)