Venue: CoRL 2022 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper focuses on the problem of grasping categories of objects with dexterous hands. The paper is built on top of DAPG. The main contribution of the paper is to add a new term to DAPG so that it focuses more on the data that it does not work well. Cause without that term, the policy learns to ignore the difficult objects and focus on the objects that can be easily manipulated. The architecture of the model is pointnet++ for processing the point cloud along with object pose and joint states. All the experiments are done in simulation and the generalization is only considered for different instances within the same category.

strengths_and_weaknesses: Strengths:
- The problem of learning from human demonstration for dexterous manipulation is important

Weaknesses:
- Very limited novelty: Apart from the second term of eq. (2), there is nothing novel about the method.
- There are no real world experiments.
- It seems like the policies are trained for each category separately. So different policies are needed for different categories. 

robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: Limited novelty and no real robot experiments are my biggest concern for this paper.

Post Rebuttal: I appreciate the authors for the new results on all the categories and held out test categories. I improved my rating.
issues: See weaknesses above.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper presents a pipeline for a robot hand to grasp onto different categories of objects and move the object to a target pose. The target grasp of the hand on the object is generated by an affordance model GraspCVAE, which was then reached using CEM for planning. The next step is to perform imitation learning based on DAPG using demonstrations generated from the previous planning algorithm. The policy encourages learning from trajectories that is hard to reproduce by dynamically assigning a normalized likelihood weight during training. A neural network is trained to predict the advantage function that is applied to the partial demonstration. PointNet is used to encode object shape, which is jointly trained with the imitation learning MLP.  The proposed method was compared with multiple baselines, and the ablation study shows the advantage of joint learning. 
strengths_and_weaknesses: **Strength:**
* Extensive experiments to demonstrate the effectiveness of the proposed method in simulation. 

**Weaknesses:**
* No real world demonstration. 
* Clarity or the quality of writing needs improvement. (see “Issues” section for details)
* The scope claimed is not sufficiently backed by what’s demonstrated in the paper. For example the word “manipulation” is not in any of the technical sections. (see “Issues” section for details)

robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Limitations are not well addressed
summary_of_recommendation: The main contribution of the paper is to propose a method to generate actions for grasping object in certain grasp pose and manipulating the grasped object to a target pose. The generated method has a high success rate during the experiment and has demonstrated certain generalizability to objects within the same category. The reviewer believes this work provides a novel approach to object grasping (fetching) than dexterous manipulation as claimed in the title and introduction. Nevertheless, it does showcase algorithmic innovations and demonstrated certain level of success. 
issues: * Figure 4: “JT” in the legends and “JL” in the caption.
* Line 155: $\\rho_{\\pi_{\\theta}}$ instead of $D_{\\pi_{\\theta}}$ in the equation above. 
* The equation between line 151 and 152 is missing the number. 
* Figure 3. Since “Behavior cloning” is a type of “Imitation learning”, the labeling can be confusing and makes the figure difficult to understand. 
* The user study on naturalness demonstrated the benefit of affordance model (a prior work used in this pipeline) rather than that of the proposed method, which does not seem to be very useful in this paper.
* It’s a bit of stretch to use the phrase “dexterous manipulation” in the paper. The action generated by the hands are mostly two stages: (1) grasp onto the object, and (2) move the object to a target pose. The proposed policy is more or less learning a grasp pose of the hand. While the hand does manipulate the object to a final pose. During this stage there’s no relative motion between the object and the fingers, and such manipulation can be achieved through simple inverse kinematics. 
* Please try to have a more elaborated limitation section rather than just focusing on explaining why there is no real-world experiment.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper mainly studies the problem of dexterous grasping. It first generates grasp demonstrations from a human grasp affordance model. Using the generated demonstrations it conduct demonstration-augmented RL to learn dexterous relocation. The proposed method also encode the point cloud observation with a PointNet and pre-train and finetune the PointNet with behavior cloning tasks.
strengths_and_weaknesses: Strengths:
1.  The proposed data generation pipeline (both grasping pose and approaching trajectories) is useful.
2. Pre-training and finetuning visual encoder with behavior cloning task is an interesting idea.
3. The performance of the proposed method is very impressive, surpassing the baselines with a significant margin.

Weakness:
1. The proposed task, relocation, consists of three phases: approaching, grasping, and lifting. The proposed method for demonstration generation seems to already handle the first two phases. For the last phase of lifting and placing it in a target pose, I think it can be implemented with some simple hard code. From the videos, I can see the proposed method mostly translates the object to the desired location and does not handle the orientation properly, so probably it's an even simpler task. The authors should include a baseline with the demonstration generation method combined with a hard-coded lifting policy and evaluate the result.
2. The benefit of pre-training is not demonstrated in the experiment results. The baseline with joint learning but without pre-training, the PointNet should be included. It would be stronger if the author could compare it with other point cloud network pre-training methods like PointContrast.

Minors:
1. Fig 3 is a little confusing: 
  a. the arrangement of the legend of the figure looks like robot state <--> share, object information goes into every T epoch and point cloud embedding goes into every epoch. Leave more margin between the columns.
  b. it looks like the demonstration is the input to the MLP. Adding a line between demonstration and the output actions to show demonstration is also the supervision would make it clearer.
2. It would be great if the authors could try if the proposed method can generalize to objects of different categories other than the training ones.

robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: I'm actually voting for borderline. The results in the simulation are pretty impressive, but I don’t think the argument stands on its ground if we can get the same result with the method that generates the demonstrations. If the authors can address my issues, I'm very likely to change my opinion.

-----

Post-rebuttal:

The author addressed my major issue and demonstrate the necessity of learning in the problem. So I change my recommendation from WR to WA.
issues: 1. Include a baseline with the demonstration generation method and show the learned method can outperform it. Also analyze the reason behind.
2. Include a baseline without PointNet pre-training.
review_title: null

Meta-review: metareview: Strengths:
+ Reviewers agree that the method presented here is effective at better using demonstrations for learning manipulation tasks compared to the main baseline, DAPG. 
+ The method allows the use of significantly more expert demonstrations that previously possible
+ The simulated evaluation is extensive and shows better performance.

Weaknesses:
- The demonstration is somewhat narrow in scope, for a single class of manipulation tasks, and only for objects of the same category as seen in training. Note that this aspect was addressed in discussion where authors provided data on cross-category testing.
- There is no validation on real robots.
- The technical advance is effective, yet also somewhat limited (a ranking term in the use of demonstrations for DAPG). This aspect was also addressed in discussion where authors highlighted the joint training of representation and decision making.

Additional comment: The fundamental improvement compared to DAPG seems to be for generalization, rather than training performance. How do authors explain such a specific improvement? What about their algorithm leads to better generalization, despite similar training speed? The authors provided an answer in discussion attributing the change to avoidance of overfitting via what is essentially a form of regularization.
confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept (Poster)