Venue: CoRL 2022 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper suggests that most grasps datasets are biased towards large objects than small objects, and as a result grasp planners compute more grasps on large objects/object-parts and struggle to compute grasps on small objects. To address this, the paper proposes the MsCG method to process local features of a point cloud in varying scales, the SBL loss to fix the imbalance using weights during training, the OBS sampling strategy to fix the imbalance during test time, and a data augmentation procedure that mixes raw data with synthetic data.
strengths_and_weaknesses: Strengths:
1. The problem of data imbalance for 6-dof grasping is an interesting problem, and is well demonstrated in Fig 1 (b).  

2. The paper presents a large amount of experiments in simulation with multiple ablations and the results are quite interesting.

3. The MsCG method is interesting and seems to work well empirically.

Weaknesses:
1. The second contribution the paper makes includes weights that scale the training loss according to the proportion of the sample’s class in the dataset, however this seems like a trivial way to address data imbalance and I’d appreciate additional information that supports the novelty of these weights, for example contrasting it with the commonly used method.

2. The third contribution suggests a data augmentation procedure that mixes raw and synthetic data to increase the number of the number of training examples and address the sim2real gap at once. The paper would benefit from additional experiments that compare the performance of NcM with other common data augmentation techniques, for example training on synthetic data and fine-tuning on raw data. 

3. The results in tables 1 and 2 seem impressive, however it is not clear what their scale represents, and it might be clearer if the results are normalized in a way that makes the scale more intuitive (% for example).

4. The method includes a step in which “M candidates are sampled…”; What are these candidates? Are these points from the point cloud or grasps, and if grasps how are they represented? Where does the farthest point sampling happen in Fig 2? The input to the point encoder seems to be the entire point cloud, but the output is sampled with dimensions MxC.

5. Was this method evaluated on a physical system? It is not clear that it was, and if not it is strongly recommended to test it.
robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: The paper addresses an interesting problem and I’m recommending a weak accept.
issues: The weaknesses section.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper focuses on solving scale imbalance problems in 6-DOF grasp detection tasks. A framework that consists of several modules is proposed. The main contributions include:

1. A Multi-scale Cylinder Grouping (MsCG) module that allows the network to extract features from different scales.
2. A Scale Balanced Learning loss (SBL) to weigh the imbalanced distribution of object sizes in the dataset. 
3. A Object balance Sampling (OBS) strategy to deal with the imbalance during inference.

The proposed method is evaluated on the GraspNet-1Billion dataset, and the ablation study shows the effectiveness of each module in the network. 
strengths_and_weaknesses: **Strengths**

* The proposed method can detect grasps for objects on different scales and is shown to perform better on unseen objects compared to GSNet and other baselines.
* The multi-scale cylinder grouping is an interesting strategy that helps the network to extract features on different scales. It is similar to the image pyramid approach used in the Visual SLAM community. 
* The experimental results are thorough and convincing. The ablation study shows the effectiveness of each module. However, some clarification is needed.

**Weaknesses** (Also see Issues)
* The Noisy-clean Mix (NcM) seems to be very similar to the common noise augmentation approach in machine learning. I would appreciate it if the authors could emphasize the difference between the two.
* The paper is generally well-written and easy to follow. However, some of the writings are ambiguous and need to be clarified. (See issues.)
robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: This paper proposes a grasp detection framework that addresses the scale-imbalance issue. The results show the proposed method achieves better average precision on unseen objects and the ablation studies of each module are consistent with the claims. There exists some issues that need to be addressed. However, the paper is overall convincing, and thus I recommend a weak accept.
issues: * What is the method that denotes "baseline" in the evaluation section? It is unclear what method is being compared. It'll be great if it can be explicitly cited.
* In Figure 5, the proposed method without NcM performs better or equally to the full method. Any idea why this is inconsistent with the results in the ablation study? 
* What is the "CD" in table 2? I don't see it being mentioned in the paper. 

Writings: 
* In Figures 1 & 6, how are the good grasp and bad grasp defined?
* In Figure 1 (b), AP is not introduced before. It was confusing for me until I saw the experimental section. I recommend the authors use Average Precision directly instead of an abbreviation. 
* In Figure 1 (b), I understand the authors are trying to show with the percentage of small grasps increases, the AP drops accordingly. However, it is also not very clear at the first glimpse. I suggest the authors change the "percentage" to maybe "the percentage of small grasps." in the future.

Nit-Picking:
* I recommend the authors also add color legends in Figure 6. It'll help readers to understand the figure more smoothly.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper considers the problem of 6DoF robotic grasp detection from point clouds. Small-scale grasp detection is identified as a particular challenge, not having been a focus in prior work. The authors propose a collection of strategies (MsCG, SBL, OBS) geared toward making improvements for the small-scale grasp detection case that is shown to be beneficial on experiments conducted with the GraspNet 1 Billion benchmark.
strengths_and_weaknesses: Strengths

The authors address an important problem in the field of robotics and provide a good motivation for the problem they solve; Figure 1 is a good visual example that captures the problem well. The ideas are well conveyed and overall the paper is easy to follow.



Weaknesses

The main weakness for me is that no real robot experiments are performed. While results on the GraspNet 1 Billion benchmark showcase the performance gains of the introduced modules, they are not a substitute for real robot experiments. This is missing and would otherwise make the paper a strong submission.

Secondly, I am not entirely convinced by the contribution of the noisy-clean mix augmentations. The results in Table 1 and Figure 5 reveal that its inclusion does not have a significant impact. We also do not see the performance when the network is trained on purely synthetic scenes, which I think is essential to justify the mixed training data. It would be really interesting to see an ablation on the performance from 100% synthetic (clean) to 100% raw (noisy) to find out what the optimal ratio is.

A minor weakness is that there is no ablation on the number of cylinder radii for the MsCG. Would it help to have more? What are the trade-offs?
robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: This paper contains interesting ideas that address a highly relevant in robotics, hence I rate a weak accept given the merit in and contribution of the methodology. Unfortunately, there are key experiments missing (real robot, ablations) that restrict me from giving a strong accept.
issues: Some aspects of the paper are unclear. These are discusses below.

- When is the segmentation network trained? Prior to the main network (point encoder, approach heads and operation head)? If so, the authors could make this more clear to avoid any confusion as I don't see it made explicit in the manuscript. Also, is there a good reason why the same point encoder cannot be used for both the segmentation network as well as the grasp candidate proposal module?

- The loss in equation (2) is not clear without the definition of the terms and sub-losses. While it is a modification of the loss in [8], I suggest to make the definition in equation (2) more clear to make the manuscript self-contained.

- The sentence on line 147-158 that explains how features of new samples are computed is a bit vague. I don't follow how the new feature is the linear interpolation of the three nearest neighbours. Is this the mean of the nearest three? Is there a weight applied to the neighbours based on their distance to the new point?

- The claim on lines 49-51 that point completion and denoising methods are unsuitable is not supported by experiments either conducted in this manuscript or in related work. It would be good to provide quantitative evidence to back up the claim and better justify the design choices of the proposed method.

- In Figure 5, which set of objects are the experiments performed on? Seen, similar, novel or all?

- In Table 2, what does "+ CD" mean?

- Typos
  -- Line 131: "not practical" -> "impractical".
  -- Line 154: "complicate" -> "complicated".
review_title: null

Meta-review: metareview: The paper introduces a new method for 6D grasp detection from point clouds that can deal with small-scale objects. The novelty of the method is on designing feature extraction and training sampling strategies that can handle small-scale objects in 6D grasp detection. The experiments on the GraspNet-1Billion dataset illustrates the effectiveness of the proposed method.

There are some concerns from the reviewers about the Noisy-clean Mix (NcM) strategy in the method. In addition, the paper lacks real-world robot grasping experiments to demonstrate its ability in real robots.

The concerns from the reviewers have been successfully addressed during the rebuttal. The authors are encouraged to revise the final paper accordingly.

confidence: 5: The area chair is absolutely certain
Decision: Accept (Poster)