Venue: CoRL 2022 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper submission proposes a method to adapt *a priori* learned skills in open, obstacle-free environments to newly situated environments that contain clutter.  It is motivated by the fact that robots exist in complex and often cluttered environments, and the key hypothesis is that *adapting* a previously acquired skill to newly encountered clutter can prove more sample-efficient than learning from scratch in multiple, diverse cluttered environments.  The proposed method uses a graph-based affordance model with heterogeneous nodes, towards exploiting the underlying structure of the manipulation problem.
strengths_and_weaknesses: This paper addresses an important and relevant open problem in Robotics for the types of complex environments in which robots exist:  adapting object-centric control skills to execution in cluttered environments.  The method proposed is based upon an interesting and intuitive idea, leveraging a graph-based representation to exploit the inherent spatial relationships that exist between the robot, the target object (to be manipulated), and nearby obstacles present.  My primary questions are with regard to the evaluation.

Overall strengths and suggestions for improvement are summarised below.

More detailed strengths of paper:
+  The paper uses a principled and concise formalisation of the manipulation problem.  The representation captures important expressivity by including heterogeneous node and edge types and expected spatial structure that can be exploited by using an underlying graph representation.
+  Results in Table 2 show notable outperformance of the other tested methods in Environments 2 and up (where greater environment IDs map to increasingly complex environments).  The ablation experiments (section 4.4) add value, in characterising to some extent how the input representation impacts method efficacy.
+  There is some evaluation conducted on real robot hardware.
+  The paper is very well written.  The posing of the research questions to be investigated for each experiment was particularly useful in framing why that experiment was selected and how to interpret the results.

I would suggest the following to improve the paper submission:
-  I have questions around whether the set of baselines evaluated is sufficient. While some standard imitation learning methods (e.g Behavioural Cloning) are used for comparison, I would like to understand (and thus recommend explanation/discussion around) if there are more relevant SOA imitation learning or reinforcement learning baselines that actually aim to do constrained optimization or manipulation in cluttered/constrained environments?  If there are, why were those not tested as part of the Evaluation?
-  In Section 4.5 (Table 4), why not use all the same baselines as in Table 2?  For that matter, the same question applies for the results in Table 1.  In Table 4, what actually is the naive baseline and why is it the only one included?  Looking at Table 2, the naive baseline often does not outperform the other baselines.  BC seems to generally perform better; either way, results for BC are reported in the other experiments (Tables 1 and 2), so I would expect it to be included in the real world evaluation.  These questions around the baselines are important because this speaks to how useful the results and insights from the Evaluation are towards assessing the efficacy of the method and the value it adds to the set of existing/SOA approaches.

Additional minor comments:
In section 3.2.2 (page 5, line 139 and Equation 1), what is q?  Perhaps it is the robot starting configuration (as implied in line 138); however, I would recommend explicitly defining q, O, and y. 
In Tables 1 and 2, would suggest **bolding** the best performing method(s) for each column.  This makes it easier to understand how the proposed method ranks comparably across tasks and environments.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: I would recommend accepting this paper, based upon its contribution of a method for adapting acquired skills to new cluttered (and thus more complex environments).  To my knowledge, this is an important and still open problem in Robot Manipulation.
issues: See limitation listed above.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The paper proposed a novel heterogeneous graph-based affordance model to find the optimal robot configuration for executing the a priori acquired skill would complete the task, whilst avoiding collisions with the environment. The main contribution of the paper is an three-step approach that allows the trained policies trained in obstacle-free environments to be excuted in new cluttered environments wihtout any 
collisions. They demonstrate that the proposed method can achieve good performance of the tasks on different tasks in challenging, cluttered and constrained environments.
strengths_and_weaknesses: The idea of 3 steps task deployments that covers finding the starting configuration, move to the start configuration, execute the skill is interesting.

The idea of learning starting configuration with GNN is interesting.

The experiment result shows interesting behaviors learned by the proposed method. The comparison to prior methods in general supports the idea in this work.

It’s not clear if the proposed method can handle tasks with the environment that, the a prior learnt skill would not be successfully executed. 
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: The paper presents a concrete learning algorithm and demonstrates good performance on a suite of difficult problems. However, the exposition of the paper could be improved and more discussions/comparisons would make the paper even stronger.
issues: The result in the video shows that the method can learn to solve a particular task with two steps. Are those environement always  meet the assumption that  there exists a distribution of robot configurations from which an a priori learnt skill would be successfully executed?

It is confusing that, if a priori acquired skills are trained in a enviroment without constraints. Is there any cases that, after finding the starting configuration, the robot still can not successfully complete the task? As the a priori acquired skills can not avoiding the collision. 

Also, how would the method work if a priori learnt skill would not be successfully executed?

Add discussion regarding if the reason that the proposed method outperform the end-to-end algorithm is because the a prior skill reduce the potential search area?
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: The paper presents a approach that can provide suitable starting poses for a priori known skills.
Skills in the paper are for grasping and placing tasks, but there is not reason apparent why it should 
not generalize to other skills. The approach can facilitate robot manipulation in cluttered environments
The learned model can be optimized at runtime to adapt to current environments. The authors propose the 
heterogeneous graph-based representation. 
Evaluation is done in simulation and on a real robot to both show the need of the component and its performance
compared to alternative approaches.
strengths_and_weaknesses: The paper describes the approach quite nicely and is easy to follow. Approach explanation and evaluation is 
nicely done. The performance of the approach is very promising and seems to generalize quite well.

There is no mentioning of any time requirements for training, learning, or optimization at test time. This information 
would be of great interest and a crucial detail to judge the performance and usability of the approach.

The limitations section mentions a few possible reasons for failure cases. A more differentiated discussion would have been 
helpful. How often did a failure get caused by what problem? The failure rate is quite high even in simulation that 
an actual discussion would be useful. Also a discussion on how/what could remedy these issues is missed and what 
side-effects that would cause (e.g., increase input complexity at cost of more learning data needed).
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: The paper is easy to read, lays out the proposed approach clearly, and convincingly shows the performance with several experiments.
A few details were missing, but it is minor. I found the paper very informative and would like to see a continuation of this research line. 
The approach also promises to be very useful for real robot manipulation. 
issues: - There is no mentioning of any time requirements for training, learning, or optimization at test time. This information 
would be of great interest and a crucial detail to judge the performance and usability of the approach.

- A bit extended discussion and details in the limitations section would improve the paper.

- I might have overlooked it, but it would be nice to explicitely state that the learned model is dependent for a specific robot and 
has to be relearned for different robot kinematics.
review_title: null

final_score: rating: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The authors proposed a graph neural network model to determine the initial robot configuration in a cluttered environment. This initial configuration is then deployed into a motion planning model to solve the collision-free manipulation task. The authors show in both simulations and in the real world how this model can help improve the success rate of skill transfer.
strengths_and_weaknesses: The document is generally well written and easy to understand.

The numbers clearly illustrate the proposed methodology.

The number of random seeds used for statistical analysis is insufficient.

I have some doubts about the scalability and generalizability of the proposed approach. The proposed model is trained in a specific scenario with limited deviations. The proposed approach has also been validated in a similar setting. In addition, the affordance model is also conditioned on pre-trained skills. All these settings limit applicability to a broader range of tasks and environments.

This paper lacks rational motivation. The paper's objective is to address the problem of collision avoidance for manipulation tasks. Many different methods already exist to deal with unstructured environments, such as the primitive approximations [1] or the SDF-based approaches [2, 3], so why is the proposed method beneficial compared to these approaches? Also, different approaches exist for skill transfer, such as policy composition [4], and learning residual policies [5]. What is the advantage of only finding the initial configuration?
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: As stated in the strength and weaknesses, I think the paper does not have a clear motivation. 
The scalability and generalizability are not clear to me. I would appreciate it if the author could conduct more experiments on it. 
The advantage and drawbacks are not well discussed. The experiment runs only 3 seeds which are not statistically sufficient. 
I would suggest rejecting the paper in its current status. 

issues: 1. The proposed method tries to learn an initial configuration to simplify the subsequent manipulation task. However, reaching this configuration could already be problematic, especially when the environment has a very narrow feasible space. When a simple motion planner is used to reach the initial configuration as described by the authors, why not use a motion planner to solve the whole problem?
2. How the downsampling of the point cloud could encode the geometry of the environment? What is the local feature vector? Why it’s necessary to learn the affordance model? It seems to me that the model only needs a subset of the point cloud to compute the transformation.
3. What is $y_i$ in section 3.2.2?
4. In the deployment, the author also added a collision avoidance constraint. Why is this constraint needed? Shouldn’t the model learn it?
5. I think the joint limit constraint could be parameterized differently, such as a sigmoid function [1]. In such a way, you could avoid the constraints in the optimization.
6. It would be more beneficial to compare with some proper baselines such as [1], [5]. 

[1] Ratliff, N.D., et al, Riemannian motion policies. arXiv preprint arXiv:1801.02854 (2018)

[2] Finean, et al. "Predicted composite signed-distance fields for real-time motion planning in dynamic environments." *Proceedings of the International Conference on Automated Planning and Scheduling*. Vol. 31. 2021.

[3] Liu, Puze, et al. "Regularized Deep Signed Distance Fields for Reactive Motion Generation." *arXiv preprint arXiv:2203.04739* (2022).

[4] Todorov, Emanuel. "Compositionality of optimal control laws." *Advances in neural information processing systems* 22 (2009).

[5] Johannink, Tobias, et al. "Residual reinforcement learning for robot control." *2019 International Conference on Robotics and Automation (ICRA)*. IEEE, 2019.
review_title: null

Meta-review: metareview: The authors propose a graph neural network model, which allows for training in obstacle-free environments and deploying collision-free policies in cluttered environments for manipulation tasks.  The authors show in both simulations and in the real world how this model can help improve the success rate of skill transfer. Their algorithm demonstrates good performance on a difficult problem.

The paper is original and could potentially represent a major advance in the field.  It is technically competent, with some additional analysis and missing details that need to be addressed.  It is generally clear and concise.   It has the potential to be impactful in the field and demonstrates the algorithm performing well in both simulation and reality.  

However, the introduction needs rewriting to more strongly motivate the work.  Comments on scalability and generalizability need to be addressed – there is no discussion on time and compute requirements.  Limitations need to be more fully fleshed out, with more details required.  The experimentation needs expanding to allow for statistical significance testing to take place. Stronger motivation and explanation for the use of RL in this problem is required, as it seems that some of the ‘learned’ parts could be directly computed.  Some other experimental settings require explanation.  This should be accompanied with an expanded analysis that covers the above issues, as well as provides concrete reasoning for the observed performance differences.

***Update***

The authors were responsive to comments and have provided a strong paper worthy of acceptance.  The methodology itself still has question marks around its value w.r.t other approaches.

confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept (Poster)