EC-Diffuser: Multi-Object Manipulation via Entity-Centric Behavior Generation

Authors: Carl Qi, Dan Haramati, Tal Daniel, Aviv Tamar, Amy Zhang
Primary area: applications to robotics, autonomy, planning
Keywords: Diffusion, Object-Centric Representation, Robotic Manipulation

Abstract:
Object manipulation is a common component of everyday tasks, but learning to manipulate objects from high-dimensional observations presents significant challenges. These challenges are heightened in multi-object environments due to the combinatorial complexity of the state space as well as of the desired behaviors. While recent approaches have utilized large-scale offline data to train models from pixel observations, achieving performance gains through scaling, these methods struggle with compositional generalization in unseen object configurations with constrained network and dataset sizes. To address these issues, we propose a novel behavioral cloning (BC) approach that leverages object-centric representations and an entity-centric Transformer with diffusion-based optimization, enabling efficient learning from offline image data. Our method first decomposes observations into Deep Latent Particles (DLP), which are then processed by our entity-centric Transformer that computes attention at the particle level, simultaneously predicting object dynamics and the agent's actions. Combined with the ability of diffusion models to capture multi-modal behavior distributions, this results in substantial performance improvements in multi-object tasks and, more importantly, enables compositional generalization. We present BC agents capable of zero-shot generalization to perform tasks with novel compositions of objects and goals, including larger numbers of objects than seen during training. We provide video rollouts on our webpage: https://sites.google.com/view/ec-diffuser.

Decision: accept (poster)
Reviewer scores: 6, 6, 8, 5

Reviews and discussion:
<thread>
creation date:
1730021148458

thread_type:
official_review

<thread_object>
creator: reviewer_wgqn

<content>
summary:
The paper presents a method that tackles the problems of behavior cloning and object manipulation. The method contains three main components: an object-centric encoder (DLP encoder), an entity-centric transformer (for temporal information aggregation), and a goal-conditioned diffusion-based decoder (for simultaneous prediction of states and actions). These components were proposed by previous methods, and EC-Diffuser appears to be a combination of several approaches. The results are superior compared to previous work on three simulation benchmarks. Furthermore, it can generalize to scenarios with a higher number of objects (trained on fewer objects, tested on more objects). The ablation studies (Table 5) show the effect of each component, revealing that the DLP encoder and goal-conditioned diffusion parts are the most important ones. Overall, the proposed method appears to be the new state-of-the-art. However, it seems to be a combination of previous methods: DLP (for object encoding), transformer-based sequence modeling (VQ-Bet), and diffusion-based decoding (diffuser). By combining these components, EC-Diffuser gains advantages over previous work through its use of the DLP encoder (better object-centric representations compared to VQ-Bet), improved entity-centric transformer (temporal information aggregation compared to DLPv2, which uses autoregressive prediction), and diffusion decoder (better handling of multimodality and uncertainty compared to VQ-Bet).

**Important note:** This review's technical content and analysis are my original work. Large Language Models were used solely to improve grammar and writing style, without altering any meanings or substantive feedback.

strengths:
* Experiments:
   - The paper evaluates the method on three different benchmarks and provides ablation studies that demonstrate the effect of each component
   - Section 5.4 demonstrates the necessity of state generation, as it helps match objects across different frames. This evaluation approach was both effective and informative
* Performance:
  - The results demonstrate that EC-Diffuser performs exceptionally well on simulation benchmarks and can generalize to different scenarios (as shown in the object-number generalization experiments in Section 5.2)
* Code is available, and video results are presented on the website for better evaluation
* Overall, the paper clearly explains the problem, tasks, and methodology

weaknesses:
* Novelty:
     - As previously stated, EC-Diffuser appears to be a combination of previous work
     - The model incorporates the best components from several predecessors, such as:
          - Object-centric encoder from DLP (while its usage is fair, the results should perhaps not depend so heavily on the DLP encoder; without it, the performance is comparable to VQ-Bet)
          - Transformer encoder from VQ-Bet, which uses a GPT-like transformer encoder on observations encoded by a VQ-VAE
          - Diffusion decoder that predicts both states and actions, similar to Diffuser which also employs this architecture for denoising both state and action
* Real-world evaluation:
    - Can this model be evaluated on real-world datasets such as NuScenes (as VQ-Bet does)?
    - Can EC-Diffuser be adapted for use as GPT-driver?
     - Is it possible to generalize to real-world setups, or would EC-Diffuser be limited to simulation data?
     - What are the necessary changes to make EC-Diffusers work on real-world data?
* Inference runtime cost:
    - As stated in the limitations and future work section, diffusion inference is time-consuming due to its iterative denoising requirement
     - While real-time inference might not be crucial for simulation data, it is essential for tasks such as autonomous driving where real-time decisions are critical

questions:
* As stated in the limitations, the model cannot capture the object-centric notion in scenes from the FrankaKitchen environment. The paper suggests that the DLP encoder could be modified to incorporate additional information, such as ResNet features. Could Slot Attention be used to obtain better object-centric features in FrankaKitchen?
* Could EC-Diffuser be applied to real-world datasets like NuScenes, setting aside its inference cost considerations? (See Weaknesses section as well.)
* Can EC-Diffuser generalize to different time horizons?
    - For instance, if the training set uses a time horizon H, could the model be tested on longer time horizons such as 2H?
    - Are there any architectural considerations that might enable or limit generalization to longer time horizons?
* Can the diffusion model's conditioning mechanism (goal action) be replaced with cross-attention instead of AdaLN?
    - Could the conditioning be improved this way, since any input could be tokenized and fed to the cross-attention layers? The authors' discussion and ideas would be informative.
    - Most state-of-the-art diffusion transformers (DiT) don't use cross-attention but instead use Multimodal DiTs, where both input and condition are tokenized and concatenated (as seen in SD3, SD3.5, etc.). Therefore, the claim about using AdaLN would also be reasonable.
* Minor formatting error: Table 1 and Figure 4 overlap, appearing to be a vertical spacing error.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730466194562

thread_type:
official_review

<thread_object>
creator: reviewer_3tkq

<content>
summary:
The authors propose EC-Diffuser, a diffusion-based behavioral cloning method designed to enhance multi-object goal-conditioned manipulation tasks from high-dimensional pixel observations. This approach leverages a pre-trained object-centric encoder to map each observation into an unstructured set of latent vectors, where each vector corresponds to a specific object or the background within the observation. The diffusion model is trained to denoise these latent trajectories paired with corresponding actions, using the initial and goal representations as conditional information. To address the unordered nature of these latent representations, the authors employ a Transformer-based diffusion model due to its permutation equivariance.

The authors validate their approach through experiments on several simulated multi-object manipulation environments, demonstrating that it outperforms baseline methods. They also provide ablation studies to highlight the contributions of different components in their method.

strengths:
* __Writing.__ The paper is well-written and easy to follow. The motivation behind addressing the challenges in multi-object manipulation from high-dimensional pixel observations is clearly articulated. 

* __Novelty.__ The authors present a novel integration of existing approaches by combining object-centric representations (Deep Latent Particles) with a diffusion-based behavioral cloning method. The use of an entity-centric Transformer to handle the unordered nature of latent representations is a noteworthy contribution.

* __Positioning.__ The paper is well-situated within the existing body of research, providing a comprehensive review of related work and clearly distinguishing its contributions from prior methods.

* __Experimental design.__ The authors conduct thorough experiments across several simulated multi-object manipulation environments. They adapt and compare against multiple competitive baselines, ensuring a fair and rigorous evaluation of their method. They also thoroughly compare their approach with both structured and unstructured latent representation-based methods, providing a comprehensive evaluation.

* __Performance.__ Experimental results demonstrate that the proposed approach consistently outperforms baseline methods.

weaknesses:
* __Alternative object-centric encoders.__ While the authors utilize DLPv2 as a powerful object-centric encoder, the paper would benefit from experimenting with other similar approaches. Since the DLP module appears to be easily replaceable with other object-centric encoders, exploring alternatives could provide insights into the generality and robustness of the proposed method. This would also demonstrate whether the observed benefits are specific to DLPv2 or applicable across different encoding techniques.

* __More generalization experiments.__ Although the authors present a wide range of experiments showcasing the advantages of their approach, additional generalization studies could strengthen the paper. Specifically, it would be interesting to see experiments involving novel object shapes, textures, or views where the use of object-centric latent representations might not be as beneficial at test time compared to regular unstructured latents. 

Minor Presentation Issues:

* There is a rendering issue with Figure 5, where the figure overlaps with the text.

* In Tables 2 and 3, the authors use __bold text__ to indicate the training setup. It might be clearer to mention this in parentheses in the column header. This is a minor stylistic suggestion but could improve the overall presentation.

questions:
* __Table 1 results interpretation.__ In Table 1, regarding the performance of the __EC Diffusion Policy__ on the __PushCube__ environment, there is an unusual pattern where the success rate decreases when moving from one to two objects but then increases from two to three objects. In contrast, all other methods exhibit a consistent decrease in performance as the number of objects increases. Could the authors provide an explanation for this anomaly in their results?

* __Real-world limitations.__ It appears that the object-centric encoder is the main bottleneck when it comes to extending this model to real-world scenarios. Is this assessment correct? If so, do the authors believe that object-centric approaches are preferable for real-world applications, and how might they address the challenges posed by the encoder in practical implementations?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730620374522

thread_type:
official_review

<thread_object>
creator: reviewer_1wdy

<content>
summary:
This paper propose a diffusion-based behavioral cloning method. It uses the object-centric representations instead of pixel-level representation. Deep Latent Particle is used to get representation, followed by an entity-centric Transformer at the particle level to predict action sequences.

strengths:
- This idea of predicting object manipulation actions through a diffusion-based architecture is interesting. By incorporating object-level information, the proposed method achieves a performance improvement over baseline methods.

- This paper also compare with the state-of-the art non-diffusion baseline, i.e., VQ-BeT, demonstrating the effectiveness of the proposed method.

weaknesses:
- The method depends on the capabilities of the image representation algorithm DLP, and experiments are conducted only in synthetic environments. It is unclear if it will perform well in more complex settings. For example, the method is only compared with one non-diffusion baseline VQ-BeT, which is preformed extremely bad on PushCube and PushT (2 out of 3 test environments used in this paper). I wonder why it is only tested on these three environments. Is it possible to also test the model on other environments that baselines achieve very good results, such as Multimodal Ant, BlockPush, UR3 BlockPush, or even nuScenes self-driving, to better demonstrate the advantage of the proposed method?

- What are the advantages of using a diffusion model over other transformer-based methods? Could you provide an intuitive explanation for why it is suitable for this task?

questions:
- What's the motivation of using diffusion for this task? What are the advantages of using a diffusion model over other transformer-based methods?

- Is it possible to also test the model on other environments, such as Multimodal Ant, BlockPush, UR3 BlockPush, or even nuScenes self-driving, to better demonstrate the advantage of the proposed method?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730690171174

thread_type:
official_review

<thread_object>
creator: reviewer_i8yn

<content>
summary:
The paper proposes entity-centric diffusion transformer architecture to generate the sequence of actions and latent states conditioned on a trajectory of image observations across multiple views. The paper claims that object centric representations facilitate composite generalization of behavior cloning agents. Experiments are conducted across three tasks namely, PushCube, PushT and FrankaKitchen to show the effectiveness of the proposed method.

strengths:
The proposed method leverages object-centric representations to generate action-state sequences for Behavior Cloning and shows that it generalizes to manipulating multiple objects. Experiments are conducted across three different tasks.

weaknesses:
The novelty of the proposed method is limited as it combines existing methods on entity-centric representation with diffusion policy. It builds upon the existing method [1] to use diffusion models instead of transformers for generating the action-state sequences for object manipulation. Diffusion models have already been shown to be useful for object manipulation tasks in prior works [2, 3].

[1] Haramati et. al, Entity-Centric Reinforcement Learning for Object manipulation from pixels.

[2] Mishra et. al. ReorientDiff: Diffusion Model based Reorientation for Object Manipulation

[3] Chi et. al. Diffusion policy: Visuomotor policy learning via action diffusion. 

Missing experiments on evaluating the learned policies on real world robots.

Figure 4 and Figure 5 overlaps with the captions of the Tables above them.

questions:
Is the DLP pretraining done separately for each task in IsaacGym environment? 

Further, if PushCube and PushT image observations were used together for pretraining DLP, can it handle the composition where both objects are present together?

Can the method generalize in manipulating novel objects? For example, what is the success rate when T-cubes are added to the Task 1 environment instead of normal cubes.

Since competing methods VQ-BET and EIT-BC work well for single objects, for multiple objects, can they be used sequentially manipulating one object at a given time? What is the performance of this baseline?

What is the performance of the method when using only the frontal view to train the diffusion model? Are two views essential?

Why is classifier-free guidance not used in the diffusion model? Can classifier-free guidance improve the success rate?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732130695613

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response

comment:
We sincerely appreciate the reviewers' time and effort invested in providing us constructive and actionable feedback. Following is a general response regarding several common points that were raised by the reviewers. In addition, we have uploaded a new revision incorporating content that addresses the reviewers’ questions and suggestions. We summarize the additions at the end of this response. **Modifications are highlighted in blue in the new revision for the reviewer’s convenience**.

**Choice of Object-centric Representation**

Our work advocates the use of unsupervised object-centric image representations in order to facilitate entity-level reasoning. The simplest definition for a *good* object-centric representation for our purposes is one that effectively facilitates downstream decision-making. As our experimental results demonstrate, DLP not only fulfills this criterion but also enables zero-shot generalization in certain scenarios.

Following some of the reviewers' questions and requests, we conducted additional experiments with an alternative object-centric representation: Slot Attention. On FrankaKitchen, EC-Diffuser along with Slot Attention, achieved a new SOTA (see Appendix D.2, Table 13 in the new revision), while in PushCube performance was significantly lower than using DLP but still better than non-object-centric baselines.
In the following points we elaborate on why we believe DLP is better suited than slot-based models for the types of tasks we address:
- The latent particles are composed of explicit attributes, such as position and scale in pixel space, which are learned entirely unsupervised. This provides a valuable inductive bias for object manipulation tasks, as it offers a direct and interpretable signal for the locations of objects, joints, and other areas of interest in the scene.

- The latent particles represent small, localized regions of the image (also referred to as glimpses), unlike slots that encode information spanning the entire image. This makes particles a more lightweight latent representation, reducing the computational cost of diffusion. Additionally, the ability to allocate a larger number of particles per image (~24 compared to the typical ~7 slots) enhances the representation of distinct objects and supports compositional generalization to scenes with more objects than seen in training.

- Extracting particles during inference is simple and fast since the encoder is a feed-forward network, as opposed to extracting slots which is an iterative procedure.

**Real World Applicability**

While we do not experiment with real robots in the physical world, we believe that our general approach, as well as our proposed algorithm, are applicable to real-world object manipulation. We do not see a fundamental limitation in solving real world tasks similar to the ones in our simulated suite using our method. We demonstrate our method’s performance on environments implemented in the high-fidelity IsaacGym simulator, which are more challenging than simulated manipulation environments presented in previous work (see response to reviewer 1WDY regarding comparison to simulated environments used in VQ-BeT). We additionally provide DLP decompositions of real images from the [Language-Table](https://interactive-language.github.io/) dataset (an environment similar to PushCube) in Figure12 (Appendix) of the new revision.
That being said, applying our method in the real world would require enhancement to some aspects of our implementation such as accelerating diffusion sampling for real-time inference, which is not unique to our approach but to all diffusion-based decision-making. We discuss this and more in Section 6 of the paper. We have added a discussion explicitly addressing real-world applicability in the “Limitations and Future Work” portion of this section in the new revision.

**Summarization of Additions**

- Experiments and discussion on using alternative object-centric representations with EC-Diffuser, including Slot Attention (Appendix D.2).
- Experiments and discussion on generalization to novel objects (Appendix D.3).
- Modified Conclusion (Section 6) to incorporate a broader discussion about real-world applicability.
- Details about DLP pretraining in each environment (Appendix B).
- DLP decompositions of real images from the [Language-Table](https://interactive-language.github.io/) dataset (Appendix D.4, Figure 12).
- Rollouts of generalization to novel objects (on the website).
- Rollouts demonstrating multi-modal behaviors (on the website).
- Fixed minor formatting issues pointed out by reviewers.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732750830105

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response - An Update on Language-Table Experiments

comment:
We would like to inform the reviewers of additional preliminary experimental results on applying EC-Diffuser on a real world dataset, Language-Table, for which we have included a new section in Appendix D.3. In summary, EC-Diffuser is able to generate high-quality rollouts in the form of particle states from which we visualize the decoded images in Figure 8. For a quantitative evaluation, we measure the prediction l1 loss for the first action (action to be executed). EC-Diffuser achieves a loss of 7e-3 for actions in the validation set and 6e-3 for actions in the training set (for reference).

We believe that this, in addition to the DLP reconstructions that we have already provided, shows promise for the application of EC-Diffuser in real-world scenarios.

We thank the reviewers for their responsiveness and engagement throughout the rebuttal period thus far.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper introduces a framework for behavioral cloning using an object-centric architecture. The architecture is based on Deep Latent Particles (DLP) and a transformer module, optimised using a diffusion objective. 

Reviewers agree that the paper is well-written and contains a solid experimental evaluation (with reasonable baselines and ablations).

One major concern raised by reviewers is that this paper's strength is mostly in its engineering advancements instead of providing a truly novel approach or method. This is a valid concern, yet the overall quality of the paper, results, and combination of techniques makes this an interesting result worth highlighting in the community.