Venue: CoRL 2022 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper presents SuperQuadric Pushing Dynamics Network (SQPD-Net), a learning-based simulator for modeling the interactions between objects (represented as superquadrics) on a planar surface. This work is motivated by the observation that if the whole pushing configuration is transformed by a planar rigid-body transformation, then the pushing result should also be the result of the same transformation. The authors thus propose to incorporate SE(2)-equivariance in the learning framework by considering an object-centric coordinate frame when modeling the object-object interactions.

The authors have compared the proposed method with a few existing learning-based multi-object interaction simulators and showed improved prediction accuracy thanks to the built-in SE(2)-equivariance. They have used the learned model together with model-based planning algorithms for three types of downstream manipulation tasks (i.e., moving, singulation, and grasping) both in simulation and the real world.
strengths_and_weaknesses: **[Strengths]**

This paper is tackling an important problem of how to add priors and invariances in the learning-based simulators to improve the prediction accuracy and long-term rollout stability.

The idea of incorporating SE(2)-equivariance in the planar pushing tasks nicely captures the structure of the underlying system and has shown improved performance than the alternatives that do not consider such structure.

It is nice that the authors have shown downstream planning results using the learned model both in simulation and in the real world.

**[Weaknesses]**

While I like the general direction of this paper, I have concerns regarding the motivation, the evaluation, and the lack of comparison with more necessary baselines.

I do not feel that the paper has sufficiently motivated why we want to learn the rigid-body planar pushing simulator from data. We have very good physics-based rigid body simulators. Even the proposed model itself is learned from a rigid body simulator (i.e., Pybullet) and evaluated by using Pybullet as the ground truth. If we add Pybullet as a baseline for evaluating the prediction accuracy, the error would be zero. Why don't we directly use Pybullet (or other rigid-body simulators) for real-world experiments? What value do the learning-based dynamics models bring in the specific evaluating scenario?

I could think of the following potential reasons.
- Will it be easier for the learned model to perform online adaptation given real-world observation (e.g., identify the center of mass, friction, distribution of supporting force) --- in other words, would it be more accurate in fitting the real-world observation data? If this is the case, more system identification or online adaptation experiments compared with physics-based simulators are needed for the real-world experiments.
- Will the learned model be able to simulate systems that are otherwise hard for physics-based models to simulate? This could be true if the authors are dealing with objects with more complicated physical properties, but we have a very good understanding of rigid body dynamics. It is hard for me to imagine, in rigid-body planar pushing scenarios, how the learning-based method is better than the physics-based simulators, which already have the priors built in (e.g., SE(2)-equivariance, objects stay rigid).
- The learning-based method can potentially learn models from observation data without the assumption of access to full-state information. However, in this paper, the authors assume a perception model that can produce the full state information of the environment (e.g., object shape and pose), which could also be the inputs to a physics-based simulator to predict the future.
- Can the learned model provide better optimization results for downstream planning tasks? If this is the case, the authors need to show that the learning-based method is more effective at accomplishing the manipulation tasks than using the physics-based simulators (e.g., use the same sampling-based MPCs with Pybullet).

Depending on the authors' motivation, more extensive experiments and comparisons with physics-based simulators are needed to justify the significance of this paper.

The real-world experiments are simple. It seems that only one of the objects is moving during the majority (or all?) of the task completion process, which only constitutes a very small portion of what can happen in the scenario. How well does the method work if there are more complicated multi-object interactions (e.g., the pushed object also pushes other objects, pushing objects in a pile)?

I'm also curious about situations where the SE(2)-equivariance does not apply. For example, what if the friction coefficients are different in different regions of the table? What if the table is not perfectly flat and the distribution of the supporting force is different in different areas?

There are also a lot of works that use graph neural networks that capture the compositionality of the underlying systems and model multi-object dynamics (e.g., [1] and many follow-up works). Therefore, comparisons with these works could also be essential to show the significance of the proposed method, demonstrate the advantages of the incorporated priors on SE(2)-equivariance, and expand its impact.

[1] Peter W. Battaglia, Razvan Pascanu, Matthew Lai, Danilo Rezende, Koray Kavukcuoglu, "Interaction Networks for Learning about Objects, Relations and Physics"
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: While I like the general direction of this paper, I have concerns regarding the motivation, the evaluation, and the lack of comparison with more necessary baselines (especially the physics-based simulators like Pybullet).

=============

**[Post-rebuttal recommendation]**

I have read the rebuttal and the reviews from other reviewers. I appreciate the authors' effort in providing new experimental results in response to my questions, especially the comparison with PyBullet by learning models directly from real-world observations. As a result, I updated my score from Weak Reject to Weak Accept.
issues: Concrete and comprehensive comparisons with physics-based models could be necessary to show the significance of this paper. Depending on the objective of this work, different types of experiments could be needed to justify the necessity of learning in the specific scenarios considered in this paper.

I would like to see how well the model work in more complicated scenarios that involve multi-object interactions (e.g., pushing a chain or a pile of objects). Currently, the real-world experiments mostly involve pushing just one object at each manipulation step.

It would help the readers better understand the difficulty of the tasks and the performance of the method by showing video demonstrations of the task completion processes.

Please see the weaknesses section for more questions and issues.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The main contribution of this paper is the design of an SE(2)-equivariant pushing dynamics model, which is then used in a model-based learning method. Objects are first detected by fitting superquadrics onto depth images and/or point cloud representations, which is an efficient way to improve pose definitions while simultaneously being a simpler description mechanism than the raw visual observation. Then through a series of transformations, they are concatenated and max pooled to enable equivariance, and input into a predictor model that predicts next poses. This model is then used by sampling based MPC methods to learn the sequence of actions needed to achieve the goal.  
strengths_and_weaknesses: Strengths:
- The proposed method seems solid, easy to understand and improves upon the baselines in the paper. The equivariance property is useful for robot manipulation and it was interesting to see how this was encapsulated in the method.
- The use of superquadrics is simplify object representations and improve pose descriptions is elegant.
- The experiments are thorough, with successful sim-to-real transfer. Additionally, the robot learning to push flat objects to the edge of the table in order to grasp them is a good indication of the efficacy of the proposed method.
- The paper is well-written, with proper discussion, good use of figures and clear explanations.

Weaknesses:
- The shapes of objects used seemed to be a bit simple (cylinders and cuboids). Given the use of superquadrics, it would have been great to see some complex shapes, or even a commentary on limitations if unable to.
- Perhaps not directly a weakness of the paper, but it would have been great to see some videos of the experiments.

Improvements:
- A sentence or two explaining how long the training process takes, and a clarification on whether the actions were executed in an open loop or closed-loop manner would be good (are the actions and candidate grasp poses sampled once or resampled every so often?).
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: The proposed method seems good, both in terms of the logic of the design and the experimental results, though I can’t comment on the novelty. The paper is well-written, with extremely detailed description of the method, thorough experiments and clear discussion of the results. The back and forth between the main paper and the supplementary Appendix (especially the related works section being in the Appendix) made it a bit frustrating to read, but that’s just a minor issue. I also have not verified all the derivations in the paper, but there seemed to be occasional missing definitions of variables. The authors took a simple (yet important) observation of equivariance in learning dynamics and used it well to improve robot manipulation. In my opinion, overall, it is a solid paper.
issues: - Some variable definitions seemed to be missing in the equations.
- A brief explanation of why only simple objects were considered in this paper, or some experiments with more complex objects would strengthen the contributions of the paper.
- Clarification on whether open-loop or closed-loop action sampling is used.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper presents an SE(2) equivariant representation of planar object dynamics which can be used to find pushing actions to drive a set of objects to a goal using planar pushing. The approach is trained with depth data to predict dynamic performance in a way that preserves geometric symmetry in planar motion. This approach is used to find pushing actions for a real robotic system and complete a wide variety of tasks. The results demonstrate how this approach outperforms previous paradigms to solve planar manipulation tasks with visual information.
strengths_and_weaknesses: Strengths:

1. Paper is easy to follow and well explained overall, along with helpful descriptive figures.
2. The approach is technically correct and 
3. This method is well validated with a variety of experiments and tasks, demonstrating its benefits.

Weaknesses:

1. The paper doesn't take enough time to describe the fundamental limitations of this approach and the type of tasks where equivariance might be detrimental.
2. The limitations of superquadrics as a shape representation are not discussed.
3. The paper contains several typos and uses passive voice very frequently, which sometimes leads to confusing statements.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: The paper is well written, presents an interesting idea, and is validated with a wide variety of experiments. The reviewer recommends acceptance. However, there are many areas of improvement when describing the limitations of the approach. More specifically, the choice of superquadrics and the impact of equivariance may lead to some fundamental limitations on object shape and mass distributions. 
issues: 1. I would like to point out that equivariance is not always beneficial since different mass distributions will lead to different optimal policies and contact interactions. For instance, a square with a shifted center of mass will tend to rotate even when pushed by the middle. This is a natural limitation of this work and should be discussed with more care.
2. The usage of superquadrics limits the algorithm to convex bodies, which leaves behind a wide range of objects of interest. The authors should also discuss this limitation.
3. The authors briefly mention how they run their real-world experiments in a quasi-static fashion. How does this compare with the motion predictor? Why is this a straightforward way to close the sim-to-real gap? 
4. The proposed approach vastly outperforms previous work on this problem, it would be helpful if the paper briefly explained why the previous methods tend to perform poorly on some of these tasks. Is this related to equivariance or to the object representation?
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The paper studies the SE(2)-equivariance in robotic pushing and proposes a pushing dynamics model that is designed to be SE(2)-equivariant. Experimental results in both simulation and the real world demonstrate that 1. the proposed model achieves high-precision prediction; 2. the model can be used in various tasks.
strengths_and_weaknesses: Strengths:

1. The paper studies an interesting problem that utilizes SE(2)-equivariance to improve a pushing dynamics model.
2. The paper has comprehensive experimental evaluation.
3. The paper has very nice figure illustrations.

Weaknesses:

1. The paper does not have a limitation section (though there is a future direction subsection in Section 5).
2. In my opinion, the idea of performing a transformation on the input before evaluating through the network to implement equivariance is not new [1][A], though the prior works are image-based and perform transformation w.r.t. the gripper frame, but this paper is vector-based and perform transformation w.r.t. the object frame. A proper comparison would be beneficial in a related work section.
3. The paper does not discuss related works in equivariant learning and other works that implement equivariant learning in manipulation [B][C][D].


[A] Ha, Huy, and Shuran Song. "Flingbot: The unreasonable effectiveness of dynamic manipulation for cloth unfolding." *CoRL* 2021.

[B] Wang, Dian, Robin Walters, Xupeng Zhu, and Robert Platt. "Equivariant $ q $ learning in spatial action spaces." *CoRL* 2021.

[C] Wang, Dian, Robin Walters, and Robert Platt. "SO (2) Equivariant Reinforcement Learning." *ICLR* 2022.

[D] Simeonov, Anthony, Yilun Du, Andrea Tagliasacchi, Joshua B. Tenenbaum, Alberto Rodriguez, Pulkit Agrawal, and Vincent Sitzmann. "Neural descriptor fields: Se (3)-equivariant object representations for manipulation." *ICRA* 2022.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are not well addressed
summary_of_recommendation: The idea of the paper, making use of equivariance in robotic pushing, is interesting. I believe the paper makes a good contribution, however, the paper can be improved by adding a related work section and a limitation section. Though the paper talks about some limitations in the future directions subsection in Section 5, the discussion there seems a bit superficial. 
issues: See the major issues in ‘Weaknesses’. 

Minor issues:

1. line 100: $T_j \mapsto C_i^{-1}T_j$ should be $U_j \mapsto C_i^{-1}T_j$
2. Is $I_3$ in Equation 3 the identity matrix? It would be better to mention it in the text. Same applies to the $I_4$ in line 138.
review_title: null

Meta-review: metareview: Strengths:

+ The paper tackles the use of equivariance in learning forward models, which the reviewers agree is a very important topic highly relevant to manipulation.
+ Results show significant improvement compared to baselines.
+ The paper is generally clear and well written (although occasional variable definitions are missing and some notations could be changed for clarity)

Weaknesses:
- While reviewers generally appreciated the use of superquadrics to approximate object shapes, this approach also raises very important questions. For simple objects that can be easily fitted using a single superquadric, the field has good analytical pushing models. What is the advantage of learning, especially since it has to be done through a simulator? Note that this aspect was addressed in the discussion stage and a clear point on on-robot learning was presented.
- How would this method apply to more complex or non-convex shapes? While this is not a topic easily addressed in the current version, the authors present a more clear discussion of it and possibilities for future work.
- Analytical models also have the ability to consider complex phenomena that vision can not perceive, such as different responses due to different pressure distributions of the contact area. In such cases, an incorrect equivariance assumption might in fact be detrimental. This aspect is not discussed.
- The paper does not discuss highly relevant recent literature that makes use of equivariance in the context of representation or reinforcement learning. This point has been addressed in the discussion and revision.

confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept (Oral)