Language-Guided Object-Centric World Models for Predictive Control

Authors: Youngjoon Jeong, Junha Chun, Soonwoo Cha, Taesup Kim
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords: Object-Centric Representation, World Model, Predictive Control

Abstract:
A world model is essential for an agent to predict the future and plan in domains such as autonomous driving and robotics. To achieve this, recent advancements have focused on video generation, which has gained significant attention due to the impressive success of diffusion models. However, these models require substantial computational resources. To address these challenges, we propose a world model leveraging object-centric representation space using slot attention, guided by language instructions. Our model perceives the current state as an object-centric representation and predicts future states in this representation space conditioned on natural language instructions. This approach results in a more compact and computationally efficient model compared to diffusion-based generative alternatives. Furthermore, it flexibly predicts future states based on language instructions, and offers a significant advantage in manipulation tasks where object recognition is crucial. In this paper, we demonstrate that our latent predictive world model surpasses generative world models in visuo-linguo-motor control tasks, achieving superior sample and computation efficiency. We also investigate the generalization performance of the proposed method and explore various strategies for predicting actions using object-centric representations.

Decision: reject
Reviewer scores: 3, 5, 5, 3

Reviews and discussion:
<thread>
creation date:
1729114707368

thread_type:
official_review

<thread_object>
creator: reviewer_eu5p

<content>
summary:
The authors propose to train a language-conditioned latent dynamics model whose state representations are object-centric “slots” provided by a frozen pre-trained model. They then train an inverse dynamics model that predicts the actions corresponding to the transitions of the autoregressively-generated latent slot representations.

strengths:
- The paper is written quite clearly, I think the authors presented their ideas quite well
- I appreciate the ablation discussions in 4.6.

weaknesses:
- Despite the paper being relatively clearly written, I would highly recommend using ICLR 2025’s extended page limit to increase the size and quality of your visualizations. For instance, for Figure 3, it’s very hard to see where the cube and moon are in the scene. I likewise cannot see any empirical quality differences between your approach’s generations and SuSiE’s.
- The approach is not tested on a wide range of tasks – only the simulated LanguageTable benchmark. It’s not at all clear to me that it would generalize to the real world. Given that SuSiE was evaluated both in sim (CALVIN) and on real-world robots on Bridge-like tasks and showed good performance (compared to strong real-world baselines like RT-2), it is unclear if the present paper’s approach would similarly scale to such more complex tasks.
- Similarly, the authors claim: “However, the major drawback of language-guided video-generation models is the requirement of large-scale labeled language-video datasets and the corresponding high computational cost. Therefore, latent predictive models, which abstract video to predict forward in compact latent state spaces, can serve as an alternative from a computational efficiency perspective.” 
  - If this is true, it seems more sensible to evaluate in the real world, where data is more limited than in sim.
  - Additionally, SuSiE does show that an off-the-shelf image generation model pre-trained on general image-language data can be fine-tuned to work well with robot data just on existing relatively limited robot datasets. If that’s the case, it seems highly unclear that sample efficiency is a problem.
- “The task is to move a block to another block based on the description of the colors or shapes … which are the red moon, blue cube, green star, and yellow pentagon.” This likewise seems very limited – I understand that there are generalization experiments, but the Bridge dataset used for SuSiE’s real-world experiments contain a much wider range of actions and objects, and thus also a much wider range of language (including many noisy labels). It has thus demonstrated to be scalable to a wider range of language and visual entities, which I think would similarly benefit this approach (as it stands, being able to generate latent state trajectories for such a limited number of objects and actions does not say much about its scalability).
- As it stands, given that the approach was only evaluated on a single task setting and said setting is not that representative of real-world language-conditioned visuo-motor robotics tasks, I do not think that this approach has sufficiently demonstrated its general applicability. I think more experiments in a wider variety of domains would be very helpful, especially in real world experiments.
- Finally, I think it would be important to include results that showcase in what settings visual and object-centric world models each excel or break down. I can imagine some cases wherein image or video generation is bad: for example, if I ask a robot to fetch me something from a closed cabinet, the image generator would have to effectively “imagine” what the inside of that cabinet looks like. However, I do not have corresponding intuition for object-centered world models (though it seems like their weaknesses might be quite similar). See last question for more.

questions:
- It’s interesting that Seer gets basically 0% on all tasks. What are the qualitative failure cases there? 
- Since SuSiE was already evaluated on CALVIN, which is simulated, why not evaluate your approach in that setting?
- In what qualitative settings would object-centered world models be more or less effective than image ones? Do the authors have any intuitive examples of this, and is there any experimental evidence to back that up?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730132285255

thread_type:
official_review

<thread_object>
creator: reviewer_vmat

<content>
summary:
- The work proposes to inject language control in object centric world models and show its effectiveness in control. 
- It argues that object centric models, specifically based on slots as studied in the paper, are more efficient and performant than large scale video generation models based on diffusion. 
- They conduct experiments on a simulated table top manipulation benchmark to justify their method and various design choices.
- They present an analysis on how to tune these world models in terms of action decoding, look ahead steps, access to past states to achieve good performance.

strengths:
- The paper is well written and easy to understand. 
- The problem of building a world model for predictive control is a useful and relevant one to solve. 
- The authors have ablated the components of their approach fairly well, including how to do the best action decoding, how many past steps to use in the world model etc.

weaknesses:
- The authors do not have a SlotFormer baseline, which does not use any language conditioning.  Given that one of the key claims of the paper is that language conditioned object centric world models help downstream tasks, checking the importance of being language centric is critical. Adding that baseline would be helpful. 
- For the evaluation of this approach, the authors have used the language table simulation environment, which involves some objects to be manipulated on a table top setting. This makes sense since there is a clear distinction between foreground (the objects) and background, which favors object centric approaches over general video generative models. However, showcasing some other scenarios or evaluation setups where maybe lets say intuitively a video generative model would have an edge, would have been interesting and more convincing to see. 
- Minor: The qualitatives in figure 3 are not the easiest to parse, if the author’s method works well, but having a video to see the predictions would make the difference much clearer, I couldn’t find anything in the suppmat.

questions:
A few questions (some overlapping with Cons. section above):
- Is language table the de-facto setting for studying object-centric control? It seems fairly limited and biased towards object centric approaches since it is clearly possible to discard the background information quite easily. Studying it in cases of ambiguity, where sometimes the background is obvious to ignore and sometimes not would bring more of the community to investigate this topic.  
- In section 4.6, Is a world model really necessary? - Have the authors reported a pixel2action baseline, basically that does the same learning procedure, and except for learning from images directly, extract from some off the shelf network. The current results only ablate the absence of future slots, which makes sense, but that doesn't answer the question generally about needing a world model or not.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730405619662

thread_type:
official_review

<thread_object>
creator: reviewer_ccro

<content>
summary:
The paper proposes to extend Slot Former to conditioned on language instruction object-centric dynamics prediction model. Such model could be used for decoding future actions for given state and instruction. Such predictions are in tern used for decoding the best action for the next time step. The paper showed that in the synthetic environment with large dataset, using such structured repression leads to better performance in comparison to using diffusion models for future state prediction. In addition, authors showed that such model is able to generalize to unseen tasks.

strengths:
- The paper is well-written and mostly easy to follow. 
- The authors provide a comparison with several image generation baselines adapted for the robotics domain, showing large gap from them. 
- The authors study how robust their method to some changes in the environment, such as changing the block type or changing the task to unseen one.

weaknesses:
- Overall, the proposed method is a simple modification of the SlotFormer adding language goal conditioned predictions and trained on a large dataset of the demonstrations.  On it its own it is not a big problem, if the proposed methods would be studied on diverse and challenging environments and compared with other methods that are state-of-the-art world models (e.g. [6, 7])

- While the improved performance on the synthetic dataset is encouraging, it is still not clear how the method would perform in more realistic scenarios where both object-centric models and corresponding agents can struggle. As mentioned by authors, recently it was shown that object-centric methods are able to decompose much challenging images or videos (e.g. see DINOSAUR (Seitzer et al. (2023)) for images or VideoSAUR [5] /SOLV (Aydemir et al., 2024)  for videos). Thus, it would be important to test how object-centric world models perform in more realistic environments with visual more complex scenarios, e.g. by training LSlotFormer on VideoSAUR or SOLV slots on environments like ManiSkill2).  

- It is not clear how the methods compare to the standard baselines on this task: while outperforming diffusion models for video prediction, it is not clear if usage of world model with object-centric representations are comparable or not with state-of-the-art algorithms using the same data for training. 

- Some experimental results would benefit from further analysis: for example, it not clear why using language conditioning for the agent itself is decreasing success rate. 

- Some potentially missing related work in video-based object-centric learning, control with object-centric representation and world models based on the object-centric representations: 

1. Focus: Object-centric world models for robotics manipulation  - also proposed a world model using object-centric representations. (https://arxiv.org/abs/2307.02427)
2. Learning Dynamic Attribute-factored World Models for Efficient Multi-object Reinforcement Learning, NeurIPS 2023 (https://arxiv.org/abs/2307.09205) - learns dynamics graph for more efficient policies. 
3. Self-Supervised Visual Reinforcement Learning with Object-Centric Representations, ICLR 2020 - proposed a goal-conditioned transformer based policy (or action decoder in authors notation), https://arxiv.org/abs/2011.14381
4. Entity-Centric Reinforcement Learning for Object Manipulation from Pixels (https://arxiv.org/pdf/2404.01220)
5. Object-Centric Learning for Real-World Videos by Predicting Temporal Feature Similarities (extention of SAVi to more complex real-world videos using DINOSAUR), https://arxiv.org/abs/2306.04829


6. TD-MPC2: Scalable, Robust World Models for Continuous Control
7. PWM: Policy Learning with Large World Models

questions:
- What is trained during "Predict future dynamics"  Figure 1(b)? If nothing remome "fire" sign near the world model? 

- "nature of slot attention not being robust to variable object number" this could be clarified
</content>
</thread_object>

</thread>
<thread>
creation date:
1730703391219

thread_type:
official_review

<thread_object>
creator: reviewer_nmfa

<content>
summary:
The paper introduces a language-guided, object-centric world model for predictive control, which is both computationally efficient and effective in robotic and autonomous tasks. Using slot attention for object-focused representation and language guidance, it outperforms diffusion-based models in task success, speed, and generalization.

strengths:
- Effectively uses SAVi to extract object-centric frame features, enhancing computational efficiency and model accuracy.
- Compares against two baseline models (Seer and Susie), highlighting the advantages in efficiency and success rate of the proposed approach.
- Demonstrates generalization capabilities to unseen tasks and objects, showing robustness in diverse environments.

weaknesses:
- The contribution in terms of "object-centric" design feels limited, as it primarily substitute the encoder for SAVi without introducing distinct object-centric innovations.
- The lack of an experiment comparing your proposed model with a VAE-based variant (ours + VAE in Tab.1) makes it difficult to conclusively justify the benefits of slot attention.
- Comparison against video diffusion models would be more appropriate than models like instructpix2pix, as diffusion models are more aligned with the proposed model's multi-frame prediction capability.
- The analysis suggesting that future state prediction alone suffices for action decoding is questionable; the low accuracy for "instruction + 0 future steps" (2.5%) compared to near-zero performance for Seer implies that baseline results may lack rigor, potentially outperforming when future states are not predicted.
- The dataset used is overly simplistic, limiting the scope of validation for the world model. Testing across multiple, varied environments would better demonstrate the model’s general applicability and robustness.

questions:
See Weaknesses
</content>
</thread_object>

</thread>
<thread>
creation date:
1732840507708

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response

comment:
Thanks for all the reviewers' valuable feedback. The main concerns for our work that all reviewers had is that our experiments are conducted in a limited and simple environment setting, lacking diversity and scalability to more complex environments. We are aware that the language table environment is not sufficiently complex to represent the model’s general applicability, and would like to address this in the general comment. 

Original SAVi with simple CNN image encoder can be applied to complex simulated image domain (MOVi++) with conditioning slot priors on segmentations or bounding boxes and training with optical flow objective from additional prediction network (PWC-Net). As stated in Appendix A.1, we stick to the non-conditional slot prior and reconstruction loss setting of SAVi for its simplicity and fully self-supervised pipeline without additional networks, which limits slot encoder’s applicability in complex domains. 

We have tried using our slot encoder setting(SAVi with simple CNN) to train in Libero dataset and environment, however, expressiveness capabilities for this slot encoder setting couldn’t able to properly bind objects in complex domains. In theory, since our method is applicable with any slot encoder other than SAVi, we conducted additional experiments on more complex simulated environments (Libero) using our method with SAVi and with state-of-the-art video slot encoder model (SOLV). 

During the rebuttal period, we attempted to extend our methodology by applying SOLV to Libero dataset. However, it has been challenging to complete this within the given period. Unfortunately, our current experiments have not yet demonstrated satisfactory performance when applying SOLV to Libero. Visualizations of the execution results reveal that the robotic arm can identify and move the objects necessary to complete the episode. However, the lack of precision in the actions seems to prevent successful episode completion. 

This issue likely stems from the difference in action complexity between the two environments. In the language table setting, actions are represented as simple 2D displacements in the x and y directions, whereas Libero requires more complex 7D actions, including displacements in x, y, z directions, rotation, and the state of the gripper. 

We explored two approaches for training SOLV: training from scratch using only Libero and fine-tuning from the checkpoint trained on the YouTube-VIS 2019 dataset (Yang et al., 2019) provided by the authors. While fine-tuning appeared to improve object segmentation, it did not have a significant impact on the success rate during evaluation. 

Additionally, we investigated replacing the Invariant Slot Attention used in SOLV with standard Slot Attention, considering that Invariant Slot Attention might exclude object-related information such as position and size from the slot representation, which might be crucial for manipulation tasks. However, this modification also did not improve episode success. 

Observing that the world model trained with SOLV slots fine-tuned using Invariant Slot Attention achieved a significantly lower loss while the action decoder’s loss increased, we hypothesized that the slots lacked critical information for manipulation tasks such as object position, size, and rotation. To address this, we adjusted the training of the world model to also predict relative grid information that reflects object position, size, and rotation. Simultaneously, the action decoder was trained to use both the slots and the relative grid information to predict actions. While this approach slightly increased the world model’s loss, it resulted in improved action decoder performance. Unfortunately, this improvement did not translate into higher episode success rates. 

Based on these experiments, we conclude that the primary reason for failure during evaluation lies not in SOLV or the world model but in the action decoder’s inability to generate sufficiently precise actions. Further refinement of the action decoder is likely needed to overcome this limitation. Nevertheless, as our method is compatible with different action decoders, we believe this issue can be addressed by introducing a more sophisticated action decoder in the future. This would enable our method to handle the increased complexity of Libero’s action space effectively.
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper introduces a language-guided, object-centric world model for predictive control, which is both computationally efficient and effective in robotic and autonomous tasks. Using slot attention for object-focused representation and language guidance, it outperforms diffusion-based models in task success, speed, and generalization.

While the paper tackles an interesting problem with some promising results, reviewers had a number of concerns regarding the novelty and contribution of the approach as well as the baselines considered.