Venue: CoRL 2022 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper proposes a sim-to-real reinforcement learning approach for grasping objects with external dexterity. The specific focus of this paper is to grasp objects lying at the bottom of shallow flat container, from a grasp pose which would cause the gripper body to collide with the container bottom. This is called an occluded grasp. The solution is to push and tip the object against one of the walls of the container, thereby removing the occlusion from the grasp. Experiments are performed in both simulation and on a real robot.
strengths_and_weaknesses: ## Strengths
- Extrinsic dexterity is under-explored in robot learning. This paper presents a good reference point for environment and reward design and low level robot control.
- The paper is well written and easily understandable.
- Real robot experiments test the validity of simulation results.
- The ablation studies showing the importance of domain randomization and robot control mode (operational space vs. joint space) are useful for readers wanting to draw conclusions about robot RL.

## Weaknesses
- The method depends on live accurate object tracking, because it is a policy observation. However, the paper lacks an analysis of the behaviour of the trained policy with noisy object pose estimates. Noise in different DOFs of the 6-DOF object pose might affect the policy performance differently. Such an analysis would help readers choose object tracking algorithms if they want to replicate this system for their task.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: I am recommending to accept this paper because of the clear research question and well-executed experiments to answer it. The paper can be further strengthened by addressing the weakness pointed out above.
issues: Not necessary for acceptance, but the paper can be strengthened by including experiments with noisy object tracking.
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: The paper proposed an RL-based training framework to train a parallel jaw gripper to grasp objects in configurations that necessitate extrinsic dexterity.  Specifically, objects placed on a flat bin are considered. The reward function is designed such that extrinsic manipulations emerges from training without any dedicated reward terms. Detailed experiments demonstrate that a policy trained in simulation with domain randomization can generalize across different objects parameters, and transfers to a physical robot platform without the need for any additional training.
strengths_and_weaknesses: ### Strengths  (in no particular order)

- I thoroughly enjoyed reading this paper. It is expertly written and clearly organized. I consistently found myself asking a question and finding the answer in the very next paragraph. 
- The framework leverages a simple but powerful idea: A dense yet simple reward measuring progress when combined with good curriculum design and domain randomization will enable the robot to figure out that the external environment can be used to its benefit.
- Design choices across the framework are justified and appropriately motivated with references to backup claims.
- The trained policy enjoys zero-shot generalization to the real-world, thanks to Automatic Domain Randomization. Further, the objects used in the physical robot experiment were out of distribution.
- The evaluation is thorough and the results conclusively demonstrate the capabilities of the proposed methods and provide insights into the need for different design elements. 
- The supplementary website and the videos are very helpful to understand the approach and the learned strategies. I appreciated the discussion of failure modes.

### Questions and suggestions (in no particular order)

- What would happen if the initial location of the object was not in contact with any wall? Will the robot learn to push the object in some direction until contact is made to establish support, or would it simply attempt to lift the object from where it is and fail? I would assume this depends on the training curriculum. But it might be worthwhile to discuss this.
- The paper cites the requirement of a specific gripper design as a limitation of some existing methods. However, the proposed method is trained and tested on the same gripper. How would the proposed approach generalize across grippers or robots? Given that the paper leverage operational space control, it might not be far fetched to investigate if the policy trained on one platform will work on another with some tuning off the controller gains.
- The experiments, while comprehensive and well done as stated above, left me to wonder why the proposed approach was not compared to some of the existing methods (at least the ones that rely on learning such as Refs. [27] and [28] and do not require considerable manual design).
- How sensitive were the choice of weights $\alpha$ and $\beta$ in the reward function, and how much reward engineering was necessary?
- Given that OSC requires a model of the robot, is perfect knowledge of the robot model assumed? This assumption is somewhat buried in the paper and would be better to make it more explicit.
- The grasp selection method involves maximization of the learned Q-function. But it is not clear at what point in time this optimization is carried out. Is it at the beginning before the robot starts its approach? 
- Will the source code be made publicly available?
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: The paper presents a simple and elegant learning framework that expands the capabilities of parallel jaw grippers by training policies that implicitly learn to extrinsic manipulation skills. The paper is well writing and the experiments are detailed and provided convincing evidence to support the claims.
issues: Please respond to my questions under the "Questions and Suggestions" section above and feel free to push back on any of my comments if they exhibit misunderstanding on my part.
review_title: null

Meta-review: metareview: The paper was positively evaluated by 3 reviewers (one as a comment due to emergency reviews). All reviewers aggreed that the proposed task "Occluded Grasping” is new and interesting and the presented method is well motivated. The main issues raised by the reviewers were a better description of the problem formulation as well as the experimental setup and showing the dependency of the algorithm on object tracking errors. Both have been addressed properly.
Due to the positive scores and the innovate problem which is solved in the paper, I recommend an oral presentation. 
confidence: 5: The area chair is absolutely certain
Decision: Accept (Oral)