Venue: CoRL 2021 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper presents a new benchmark of completing pick-and-place tasks using a robot manipulator. In comparison to existing pick-and-place tasks that deal with simple-shape objects, this paper described a parametric way of generating more than a hundred of blocks of diverse shapes that were 3D printed. The agent first learned a policy from simulation using RL methods, where the domain is fully observable (object positions, etc). Then imitation learning methods were used to generate a vision-based policy, where domain randomization was enabled for alleviate the sim-to-real issue. Finally, the policy learned from simulation was improved for real-world deployment. 
strengths_and_weaknesses: The 3D-printed parametric block generation is a strength of this paper, and has the potential of providing a strong benchmark to the community. So the first contribution listed on page 1 is solid: "We present and release a benchmark for stacking that features
a diverse set of geometric objects and tens of thousands of possible stacks." It's also good that the paper used state of the art methods for learning policies in simulation and in the real world. It will be better if the objects shown in Figure 1 (d) can go beyond toy blocks that have been widely used in robot manipulation research. 

The downside is that the paper does not make it clear if any of the used algorithms is new from the literature. As a result, the reviewer is not convinced that the second contribution is valid: "We show that it is possible to learn a single vision-based policy that can handle multiple combinations of objects and can demonstrate a plethora of stacking strategies."  There are many existing methods for vision-based manipulation, and there are competitions on this robot manipulation. The reviewer respectfully believes that there is rich literature on learning vision-based policies for manipulating irregular-shape objects. The indication that no existing work support vision-based handling of multiple objects is questionable. There's a similar concern on contribution (d), which is a claim about sim-to-real policy learning. Improving sim-to-real policies using real-world data is not new. 

Contribution (c) on ablation study makes sense. However, since those are all mature methods from the literature, one probably won't say it's a major contribution to the community. 
summary_of_recommendation: Overall, the reviewer didn't observe anything incorrect from the paper, and the paper was nicely generated. However, the paper mostly demonstrated that existing methods can be applied to vision-based policy learning for manipulation tasks on a new 3D-printed set of objects. The contribution is solid but not significant. 
issues: See above.
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: *This work developed a novel benchmark in the context of stacking manipulation. The benchmark is novel and challenging. The setup cost is reasonable and details have been provided for the ease of reproduction which is valuable to the community.

*It is shown to be possible to learn a single vision-based policy that can handle multiple combinations of objects and can demonstrate a plethora of stacking strategies, emergent from RL training. Zero-shot transfer of these strategies to the real robot has been achieved.

*The ablation studies suggest what matters for sim-to-real transfer and for imitation of a simulation-trained RL policy (that uses privileged state information) with a vision-based general policy. The proposed interactive distillation approach allowed for faster experimentation and iteration.

strengths_and_weaknesses: Strengths:

*The developed benchmark is novel, challenging and interesting. It involves diverse kinds of objects while requiring different manipulation strategies. The setup details have also been provided for reproduction.

*The paper is well written and easy to follow.

*The design choices are reasonable and has been reflected in the ablation study.

Weaknesses:

There is no obvious weakness found, except for some minor issues explained below.

summary_of_recommendation: Accept.
issues: In the video 0:44, in the first row, isn’t the triplet5 a failure case? Triplet3’s execution is also not complete.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper studies robotic stacking of diverse rigid objects. It introduces a benchmark for robotic stacking and proposes a framework that learns a policy to solve the tasks in simulation and transfer the learned model to the real world. The proposed framework achieves a reasonable task success rate in both simulation and the real world. 
strengths_and_weaknesses: Strengths: 

- In this paper, the authors design and implement a simulation environment for stacking rigid objects. A parametric set of objects is proposed to create diverse challenges for the stacking problem. This simulation environment could incentivize interesting future research on table-top robotic manipulation if the authors will release the codebase.

-   The proposed sim-to-real transfer algorithm seems to be effective. The model achieves a reasonable success rate of stacking diverse objects in simulation and the real world. 

Weaknesses: 

- The contributions of this paper are confusing. Although the title claims "Beyond Pick-and-Place", the object stacking task studied in this paper is indeed composed of picking and placing actions. In addition, while stacking diverse objects have been studied in prior work (e.g. Furrer et al ICRA 2017), the title focuses on the application without highlighting any technical novelties proposed in this paper. The authors claim that "We show that it is possible to learn a single vision-based policy that can handle multiple combinations of objects and can demonstrate a plethora of stacking strategies, emergent from RL training" However, this has already been widely studied in the community and demonstrated in prior work such as Duan et al. NeurIPS 2017.

- The introduction section of the paper fails to explain the importance of the problem and does not mention any previous works or their limitations. The first two paragraphs mainly talk about the task design and the experiment setups. 

- The term and the formulation of "sim-to-real policy" in Sec. 4.2 and 4.3 are misleading. In Sec. 4.2, the authors train the policy in simulation and expect it to be directly executed in the real world. So there is anything related to sim-to-real transfer about this policy $\pi_{\theta}^{s2r}$ itself. It would be better to call it just student policy or something else. 

- Although it seems that the proposed sim-to-real transfer approach is one of the main contributions in this paper, the paper lacks a thorough comparison with state-of-the-art sim-to-real transfer approaches in the experiment. Without such comparisons, it is hard to justify the significance of this paper. 

- Many important papers are missing in the related work section, including but not limited to:
    - Furrer et al. ICRA 2017
    - Macias et al. ICRA 2014
    - Noseworthy et al. RSS 2021
    - Duan et al. NeurIPS 2017

- (minor) The listed keywords include CoRL which seems to be a mistake. 
 
summary_of_recommendation: The paper studies an interesting robotic manipulation problem and proposes a valid technical solution. However, the contributions of this paper are not clearly described or justified given its current form and the lacking of comparisons with prior work. Therefore, I believe this work requires another iteration before being ready for publication. 
issues: I would suggest the authors re-think their contributions in this paper and make major revisions to Sec. 1 and Sec. 4. I would also encourage the authors to compare with state-of-the-art sim-to-real transfer learning approaches to justify the significance of their technical contributions.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper proposes a method to deal with the problem of robotic stacking with objects of different geometry by training a vision-based reinforcement learning, and a benchmark for stacking geometric objects is presented. They successfully achieve skill mastery of fixed combinations of objects and also assess skill generalization across them. To achieve sim-to-real transfer, interactive imitation learning is used to distill the state-based policy into a vision-based policy, and the final policy receives relatively high accuracy in the real world. Besides, lots of real-world evaluations including ablation study are provided and discussed, showing what matters in this task.
strengths_and_weaknesses: Strength
1. This paper proposes a benchmark for the interesting but challenging problem of stacking objects with complex geometry.
2. High accuracy is achieved in real-world experiments, and a large ablation study is provided for sim-to-real transfer and imitation of a simulation-trained RL policy.

Weakness
1. The expert policies from state features in simulation are conditioned on parameter ‘y’, which means the object triplet number in skill mastery or object deformation number in skill generalization. However, I think it’s not necessary to add this parameter because the information is contained in the states in simulation or the visual observations in the real world. It might limit the capacity of generalization in this work.

summary_of_recommendation: This paper proposes a benchmark for the problem of stacking objects with complex geometry, and achieves good performance in real-world experiments. However, I think the limited variety of triplets during training limits the contribution of this work because there exist endless object combinations in the real world, which might reduce the policy’s capacity for generalization.
issues: List issues to be addressed during the period of author response and revision. The authors have the opportunity to submit a revised manuscript during the period.
review_title: null

Meta-review: This paper presents a new benchmark for stacking manipulation. Reviewers are overall positive about the paper however there are some concerns:
1. Relationship to prior work
2. Lack of comparison to state of art

Please respond to reviewer SRSs concerns.

==
Post Rebuttal

The authors were able to address the concerns of the reviewers. AC believes this is a strong paper which should be accepted.
Decision: Accept (Poster)