Venue: CoRL 2023 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary_of_paper: The authors propose, essentially, an extension to RoboCraft which models rigid-static object interactions (and gripper state) which they validate on a stowing task in both simulation and on a physical hardware. They estimate the dynamics of the scene (in a simplified uniform-density box-shaped representation)  via a graph neural network and evaluate several methods of planning (finding parameters for three parametric skills) with this dynamics, the most effective one being random shooting.
review: This paper feels a little bit narrow, though it also serves as a limited and partial reproduction of parts of RoboCraft. The primary contribution compared to prior seems to be modeling dynamic/static interactions, which is good, but conversely this work does not evaluate settings with deformable objects or non-box-shaped representations.

That being said, the experimental domain (stowing) used in the paper is interesting with a good mix of difficulty and practical applicability. The author's action formulation was also reasonable and the results shown in real-world experiments are strong (95 percent) though the lack of a RoboCraft comparison in this setting weakens the result.

The paper would also benefit from additional ablation investigations in simulation, such as the effect of pose-estimation error and the dimensionality of the action parameter space on performance. It would be particularly interesting to see if random shooting still outperforms RL approaches such as PPO as the action parameterization becomes higher dimensional and more complex (I expect it would not, but it would be a helpful contribution for this to be evaluated).

robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: The amount of novelty is a bit low, however the expanded experiments make this paper valuable to the field. I advocate for acceptance.
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary_of_paper: The authors propose a graph neural network based approach for placing objects on cluttered bins and shelves. One of the key challenges in their stowing task is that objects must be pushed or manipulated out of the way in order to clear space for them.

The world is represented as a bunch of interacting rigid bodies through a graph neural network (GNN). Each object is represented by particles (nodes in the graph). Edges are formed when distance is under a threshold. The action is also modeled as a trajectory of particles, over three primitives: sweeping, pushing, and transporting.

The authors built a simulated environment on Robosuite with varied shelf sizes and objects. They trained on 300 episodes, and evaluate on simulated and real robotics tasks with a variety of objects, and compare against heuristics and (in sim) against SAC and PPO.

review: I think that the premise of the paper is interesting, and the task - while similar to many other rearrangement problems - is much more realistic. The GNN dynamics model seems novel, especially when applied to robot planning to manipulate in clutter. Generally, I really like the idea of using a dynamics model together with action primitives. The problem is also an interesting manipulation problem that involves a lot of object-object contact.

The GNN represents objects as a collection of particles. This is a different formulation from most GNN policies, and I think it's a very interesting one. However, it seems like there are a lot of hyperparameters involved - how many particules, when to create edges - and I wonder what the effect of these is. Likewise, the choice of key points in the trajectory, which the dynamics model uses to predict where objects will go, seems very important - this detail isn't really in the paper.

They train on 300 trajectories and test on object, including held-out objects with a different shape (it was only trained on boxes, but they test on bottles) - although they still represent it as a box during inference, which implies it might fail in other circumstances where shape matters more. I'm a bit surprised how little data the authors needed to train their dynamics model - 300 trajectories is not a lot! Although the action space is quite limited.

They show the method in simulation and the real world, and in particular they show it succeeding in a range of different challenging stowing tasks in the real world. In the end, I think the idea is straightforward, novel, and interesting. 
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: I think the learned GNN-based dynamics model, the method of planning over primitives, and the interesting task with real-world verification all make this an interesting paper. I would be interested to see how the method could be applied to other tasks.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary_of_paper: This paper implements a pipeline that predicts object pose changes using Graph Neural Networks (GNN) in an environment with a variable number of objects. The system is designed to control the robot gripper, minimizing the expected and actual object pose in order to successfully execute a number of skills, including sweeping, pushing, grasping-transferring-releasing an object. For this, the system needs to learn GNN-based effect prediction on objects given action parameters, i.e. typical motions of objects in response to the actions. These are learned from random executions and human demonstrations. In the use case, the actions are chained in order to stow an object to a rack that includes other objects. The learning occurred in the simulator and was verified in the simulator and the real world. The authors provided results and comparisons with other methods.
review: - The paper is very well-written, easy to follow. 
- The idea of using GNN for predictive modeling of multiple object interactions is not new, as the authors indicated, but the authors missed some very related work [Tekden, 2020; Tekden 2021]. The proposed method uses particle-based object representation, whereas Tekden uses object-level representation in GNN. As all objects are assumed to be box-shaped rigid objects, does more-complex particle-based representation have any advantage?
- As a general remark, although I believe that the framework is quite general, the paper writing and method description is too much geared towards the stowing task, which is, while very common, a specific task among many others. The authors should provide a more general approach.
- The authors mention some challenges related to detecting contacts and friction, but they do not come back to these challenges in the rest of the paper.
- The argument in lines 38-39 (related to failure due to human labeling) needs reference and justification. Most of this work is not self-supervised.
- The argument in lines 83-84 also needs justification/reference.
- Please explain what Random Shooting corresponds to.
- In 103-104, the authors argue that their approach is flexible and adaptable to variations in object shape, size, and geometry. This is not rigorously justified in the experiments section. This can be shown by verifying the learned system with objects not observed during training. 
- The sub-goal explanation for Fig. 2 (l95) is not clear.
- l113: The attributes include gravity; probably, it is supposed to be either density or mass.
- l114: How did you set the threshold to form an edge?
- l116: Please provide the complete set of relations.
- l112: What does 'object it belongs to' mean? Do objects have ids?
- The goal-conditioned trajectory optimization part is not clear. Please precisely explain the algorithm. How exactly the optimization occurs?

- How is the data, collected in data collection, being used precisely? To train GNNs? Are 300 episodes sufficient to cover the wide range of object-action interactions? 
- How is the data obtained from the demonstrations used exactly? The authors mentioned they used teleoperation. How many teleoperations are done? 
- In the explanation of real-world experiments, point cloud representation is indicated to be created. How is this information used?

- In the first experiment, the proposed method is shown to outperform the GNN-based baseline. However, the margin is small, and they are very similar methods with slight differences. Therefore the contribution of this result is not substantial.
- In the second experiment, the proposed method is compared against a completely different approach, RL-based methods, where no details are provided. The authors need to compare theirs with an MPC-based method to show effectiveness.
- How is the randomization realized in the third experiment (in the real world)? The assumption of complete object knowledge is a very strong one, and together with the object recognition step, the details should be moved from the appendix to the main manuscript. The baseline 'heuristic' approach is not clearly explained, and from the videos, it looks not a strong baseline. 


Tekden et al. "Belief regulated dual propagation nets for learning action effects on groups of articulated objects." ICRA 2020.
Tekden et al. "Object and relation centric representations for push effect prediction." arXiv preprint arXiv:2102.02100 (2021).
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: The paper proposed a sound method that combines an existing GNN-based method with a simple action control optimizer. While I appreciate the writing, the working system, and the verification in the simulator and real world, the paper does not make a strong contribution to the field theoretically or as a very strong system in practice.

Update: The paper now shows that their particle-based GNN outperforms the existing object-level GNN architectures. The paper now also includes deformable objects in experiments. Therefore, I increase my recommendation score as the paper now justifies its contribution better.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary_of_paper: This paper proposes a graph neural network framework to address the task of stowing. Also, given the demonstration during training, this paper trains a graph neural networks-based dynamics model to model the interaction of multiple objects. With the learned dynamics, the paper uses a random shooting to choose the best action to achieve several keyframes sequentially to achieve the goal. Based on the training purely in simulation, this framework can generalize to the real-world without any finetuning.
review: Strengths of the paper: 
1: The stowing task is novel and important for the robot manipulation community. 

2: This approach is trained purely in simulation and can generalize to real-world scenarios. 

3: The real-robot demos look cool and interesting. 

Weakness of the paper:
1: the author proposes a graph neural network framework to model the dynamics of multi-object interactions. Both dynamics learning and planning are pretty similar to [1]. The authors should discuss the difference with the previous work. 

2: The author highlights the contribution of long-horizon planning but does not cite enough related works like task and motion planning [2].  

3: The paper is hard to follow. For example, the authors define subgoals at the beginning of the approach section without defining them. Also, the authors do not discuss the details of the loss function. If it’s limited by space, the authors should at least put the details of the loss function in the appendix. 

4: Planning is a very important part of the paper but the authors do not discuss that part clearly. The authors only mention “We employ trajectory optimization to find the skill parameters that yield the lowest cost.” How do you optimize both the discrete part and continuous part of your skill primitives? 

5: The authors claim “In our analysis, we evaluated Random Shooting, the Cross-Entropy Method, and Gradient Descent - all of which exhibited similar performance.” This seems weird to me, it’s clear that the cross-entropy method and gradient descent will perform much better than naive random shooting. My guess is that the assumptions of manually choosing good keyframes make the planning problem easy so even the naive random shooting works well. 

Minor points: 
1: Figure 2 is hard to understand. 

2: training time instead of training times.

[1] Yixuan Huang, Adam Conkey, and Tucker Hermans. Planning for Multi-Object Manipulation with Graph Neu- ral Network Relational Classifiers. In IEEE International Conference on Robotics and Automation (ICRA), 2023.

[2] C. R. Garrett, R. Chitnis, R. Holladay, B. Kim, T. Silver, L. P. Kaelbling, and T. Lozano-Perez, “Integrated task and motion planning,” Annual review of control, robotics, and autonomous systems, vol. 4, pp. 265–293, 2021.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: This paper investigates a novel and interesting stowing task with some cool real-robot demos. However, this paper should discuss more related works like [1][2] to highlight the core contributions of this paper. Furthermore, the presentation of the paper is unclear and the paper should include some important details about the planning algorithm. Thus I recommend “weak reject”.
review_title: null

Meta-review: All reviewers agree this paper proposes a novel model for representing skill-based dynamics using a graph neural network over particle-based representations of the object and robot. This is distinct from recent object-based GNN models and the experiments show the particle-based GNN outperforms existing object-level GNN architectures. This particle-based representation is highlighted in its ability to model deformable objects.

The model is evaluated on a stowing task, which all reviewers agree is an interesting and challenging task. As reviewer NnNZ points out is "is much more realistic" than most existing rearrangement planning papers.

I think this combination of strong architectural improvement with increased difficulty of task makes this an excellent paper for presentation that would help move the manipulation community forward towards more challenging and constrained multi-object manipulation tasks.
Decision: Accept (Oral)