Venue: CoRL 2022 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The authors introduce a sim2real policy transfer pipeline that learns grasping of rigid objects via an Allegro hand attach to an xArm robot in simulation and transfers the policy learned via reinforcement learning or behavioral cloning to the real robot.
To accomplish the sim2real transfer, the observed point clouds are augmented by an "imagined" point cloud that synthesizes the occluded parts of the robot hand based on proprioceptive state measurements. Furthermore, a contact-based reward function is introduced during training in simulation that encourages contact between the thumb and at least two fingers of the robot hand and the object being grasped. Experiments are conducted on a subset of the YCB objects, as well as as on bottles unseen during training.
strengths_and_weaknesses: # Strengths:
The proposed method is technically sound and well motivated. The introduced contact reward function is shown to have a significant impact on the training performance.

The authors provide a useful discussion of limitations of the current work.

# Weaknesses
The paper is focused on grasping, not "dexterous manipulation". The objects are very similar to each other where the robot motion to grasp it does not require any meaningful variation to grasp even the unseen objects, since they all have roughly cylindrical shape. The robot hand essentially just needs to move to the object and then close itself to grasp it. As such, the work does not sufficiently demonstrate its applicability to truly novel objects.

The experiments are lacking metrics for the stability of the grasp that results from these motions. Judging by the supplied video recordings, the hand often does not firmly hold the object in place, so some quantification of robustness to external disturbances is needed.
robotics_focus: Relevant but unlikely to deploy to hardware in near future
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: The contributions to include imagined point clouds in the observations and to add contact information to the reward function are promising, but more evaluation is necessary to verify the robustness of the sim2real transfer. Given the simplistic experiments where the objects have very similar shapes and the lack of comparison against other dexterous grasping works, the sim2real performance is difficult to assess.

Edit: Updated my review to weak accept in response to the authors (see explanation below).
issues: The success metrics between sim and real differ - in simulation a goal position must be reached, but on the real world experiments it suffices to lift up the object a certain height. I do not understand why it is "inconvenient" (l. 190) to measure distance to a target location. This makes the real-world experiments much easier to accomplish.

Why is the observation augmented by a one-hot encoding to indicate whether the point is real or imagined (line 176)? Has this made any difference in the training process?

The caption of Table 1 needs to be updated, the objects types bottle and can do not appear in the upper and bottom two rows.
The caption of Table 2 also does not fit to the table but talks about two figures that are not there.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This work proposes a policy learning framework for utilizing multi-fingered hands, such as the Allegro platform, to grasp rigid objects. The input corresponds to a point cloud, which is augmented with proprioceptive and object pose information to construct the observation space. The action space consists of the arm’s 6 joint angles and the end-effector's 16 DoFs. Relatively to prior efforts, this work introduces a contact-based reward function (which is where most of the argued benefits arise from) and an “imagination”- based point cloud augmentation technique (for mitigating occlusions and sensor noise). The final outcome is category-based generalization to novel object instances and zero-shot sim2real transfer for grasping. An ablation study has been conducted (in simulation) to compare different training strategies and how training over multiple objects affects the approach. The performance in the real world was tested with four objects. 
strengths_and_weaknesses: Strengths:
- The problem that the paper attempts to solve involves a high-dimensional control challenge, due to the presence of a multi-fingered hand. For such end-effectors, which bring the promise of dexterous manipulation, grasping singulated objects (which is the focus of this work) can still be challenging especially as object variability increases. 
- The ideas on the “oracle contact” and the imagined hand point cloud seem reasonable for the proposed objective. 
- The authors have tested their approach in real-world robot experiments and seemed to have achieved zero-shot Sim2Real transfer, which is highly desirable.
- The paper is mostly clearly written, with proper formalism and an accompanying ablation in simulation. The visualizations are helpful.
- The related work section is rather comprehensive and covers the important subareas related to this work.

Weaknesses:
- The expression "Dexterous Manipulation" used in the title is rather misleading. It should be replaced by the term ‘Multi-fingered Grasping’, which is what this work is focusing on.
- The fact that the method is both trained and tested only on 2 distinct object categories + the nature of the contact-based reward may imply that the approach learns a shape prior and that this is the main reason this method succeeds. What happens in object categories significantly outside this shape prior still remains open for interpretation.
-The reward function seems to have multiple components that are aggregated together with a rather naive weighting scheme. The reward structure does not seem to reflect the structure of the task (e.g., first approach the object and then lift it) as the weighting scheme seems to be constant over the duration of the trajectory instead of time-dependent. It would be helpful to further explore how the different components can be integrated into an effective reward function and the role of the weights in this process.  Some of this discussion is left for the supplementary material but it would be better if the penalty term in the overall reward, as well as the multi-task weights’ scheduling scheme, were introduced in the main paper.
- In contrast to some related efforts (e.g., [3]), access to the object geometries seems to be a requirement for this work during training. 
- The “Limitations” section is more of ‘Future Work’ one. It would be better if the authors acknowledged in a more comprehensive manner the limitations (e.g., there is still a gap until the grasping problem targeted is solved).
- The experiments in the ablation study between single and multiple objects do not communicate a clear message. In particular, the training curves for the two object categories provided do not give a clear indication which method is better and why. Since this is referring to simulation-based experiments, adding more categories may be helpful to evaluate a pattern. 
- There is a mismatch between what is described for the Multi-object case between the main paper and the appendix. In the second one, only the ShapeNet objects are mentioned, while in the first one the authors mention that they randomly sample from both datasets. 
- According to the ablation study, the approximation of the previous, related approach ([3]) achieves 0% success rate. How does this match the results presented in that work, which focused on in-hand object reorientation and must have included contacts. 
- The provided videos do not visualize the goal pose at each test case (like in the paper’s diagram), which makes it harder for the reviewer to evaluate the task’s success/failure. Furthermore, there is no video that clearly showcases the effect of adding solely the contact-based reward term, the strongest point of the paper.

robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are not well addressed
summary_of_recommendation: This a rather well-written paper, with solid fundamentals, aiming to solve a high-dimensional grasping challenge. Both contributions presented are reasonable and presented in sufficient detail. The paper provides real-world experiments and demonstrates Sim2Real transfer. The results are intuitive, even though there are some points that would need further clarification per the comments above. The quantitative results of the method are well-organized and indicate an improvement in terms of grasping success even though there are still failure cases.. Some additional discussion of the qualitative results would be helpful. 
issues: Change the title of the paper to replace “Dexterous Manipulation” with “Multi-fingered Grasping”.

This algorithm is tested on a quite standard setup (single tabletop object, uncluttered, a commonly-used hand) and has virtually no inference comparison point. So, papers like the ‘Multi-fingered Active Grasp Learning’ by Lu et al., ‘Learning Continuous Grasping Function with a Dexterous Hand from Human Demonstrations’ by Ye et al. (admittedly it has been published after the submission deadline)), ‘Multifingered Grasping Based on Multimodal Reinforcement Learning’ by Liang et al.  and other similar works could be included in the related work section. One of them could potentially be used as a comparison point to this algorithm’s performance.

It would be appreciated if the authors provided failure examples and look into which part of their method is causing them and in which scenarios.

How many training iterations would be required if somebody would want to train for a new object category?

Please provide some justification for the following choices: a) The Object/Goal Pose Rotational components are not included in the reward term. b) Only two fingers are needed for the contact reward term to be active.

The authors should add legends to their plot diagrams (including measurement units).
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The paper proposes two techniques to learn dexterous manipulation policies with point clouds as observation. The authors propose using Reinforcement Learning to learn the policy and evaluate the performance of the learned policy when transferred from simulation to reality.

The paper's main contributions are:
1. Show that it is possible to achieve sim-to-real transfer for dexterous grasping policies when using point cloud as observations.
2. Two techniques to guarantee proper learning: 
         (a) a Pointcloud augmentation to deal with possible camera occlusions
         (b) a hand-object contact reward function.

strengths_and_weaknesses: In the reviewer's view, the strengths of the paper are:
1. The demonstration of the utility of point cloud observation for sim-to-real transfer.

In the reviewer's view, the weaknesses of the paper are:
1. The paper is very task specific. The authors present the method they have used to train a reinforcement learning agent to grasp objects given point clouds. To do so, they introduce a reward function and the observation augmentation methods but it is not clear if these methods would also benefit in other possible dexterous manipulation tasks as it has been only evaluated for grasping. Evaluating the method in additional manipulation tasks would be beneficial to show the benefit of the proposed method.
2. The solution is validated only with PPO. It would be interesting to observe the performance of the proposed method with other reinforcement learning algorithms.

robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: The paper presents a problem and provides a solution to it. The authors show that the method works for different objects and show that the method transfer to real robots. The paper itself seems very specific to the problem of learning through reinforcement learning and a grasping policy through point clouds. But, it is properly executed and the authors show the performance of the proposed method and validate the benefit of the proposed techniques. 

Therefore, I suggest a weak acceptance.


issues: 1. Adding a new experiment on a different task. It could be related to grasping, such as opening a drawer.
2. Evaluating the benefits of the proposed techniques with different RL algorithms. The method should work, but the paper would be more complete if an experiment with SAC and TD3 are added.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The paper proposes a method for robotic grasping with a dexterous robot hand in the real world. The method is point-cloud based and fully trained in simulation. The authors propose a shaped reward, including contact information between robotic hand and the object to grasp. Furthermore they propose to construct the point-cloud of the robotic hand from proprioceptive information and use this imagined point cloud as an additional input to their method. 
The authors ablate their modelling choices in simulation and  demonstrate that their trained method can be transferred to the real world without fine-tuning.  
strengths_and_weaknesses: Strengths:
- The authors demonstrate that their method can be transferred to the real world without fine-tuning. 
- The formulation of the reward seems to be useful and lead to good results.
- The idea of adding the imagined gripper point cloud seems improve the training.

Weaknesses:
- The evaluation the authors perform is rather limited: They only train and evaluate on cans and bottles in simulation. In the real world the evaluation is even more limited, as the authors only evaluate on 4 different bottles. 
- There is no comparison to other methods like e.g. Unigrasp [L. Shao et al., "UniGrasp: Learning a Unified Model to Grasp With Multifingered Robotic Hands," in IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 2286-2293, April 2020, doi: 10.1109/LRA.2020.2969946].
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: The ideas seem nice and promising, but the evaluation is not very thorough. There is no comparison to any baseline (e.g. Unigrasp). Also the evaluation is performed only on very view objects of limited variability. In particular the real world experiments are performed only four objects of type bottle. 
issues: Formulation of loss:
- line 140:  how is $x_{obj}$ is defined? Is it the center of the object?
- eq.2: How is inContact defined? How is it determined if contact was established?

Experiments: 
- There is no comparison to a baseline.
- The evaluation is performed only on a very limited set of objects (bottles and cans) and only on four objects of type bottle. 
- Table 2: The caption seems to be wrong. There is no learning curve visible here.
review_title: null

Meta-review: metareview: This paper proposes a sim-to-real approach for dexterous manipulation. It trains a policy for a multi-finger robotic hand in simulation, to grasp rigid objects and transfer the policy to the real-world. The proposed method combines point cloud augmentation based on proprioceptive information and reward shaping for the successful zero-shot sim-to-real transfer. The method is validated using a real Allegro hand to grasp new objects of the same category as in training.

While most of the reviews agree that the method seems technically sound and interesting, they brought up three important areas to improve:
1) There is no baseline which the paper compare the proposed algorithm with, even though there are a lot of related work.
2) The objects to grasp in the evaluations are limited: four objects with similar cylinder-like shapes, which cannot sufficiently evaluate the advantage and limitations of the proposed method.
3) The paper focuses on simpler grasping tasks, instead of the claimed "Dexterous Manipulation" in the title.

During the reviewer discussion, all reviewers agreed that the additional experiments had further strengthened the submission. However, the new door opening and lifting cans or bottles still require similar hand motions to achieve the defined tasks. So it is still questionable whether the method exhibits dexterous manipulation rather than grasping. Reviewers suggest that the paper replaces the term "Dexterous Manipulation" in the title to "Multi-fingered Grasping". Nevertheless, reviewers agree that the technical contributions are interesting and the experiments have been improved significantly. For the above reasons, we recommend accepting this paper.
confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept (Poster)