Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper addresses the problem of 3D object detection in point cloud for indoor scenes (scannet and sun rgbd). The idea is based on voting-based methods: each point predicts its center -> clustering -> classification. The method proposes to classify points first, before center prediction and clustering. They claim contributions in the following:

- Novel "class-aware" 3D proposal strategy as discussed above.

- A refinement stage that re-voxelizes according to each point's classes. 

- Good results on scannet and sun rgb-d. 

The results do seem strong. And both claimed novelties contributed to the improvements. I'll recommend a weak accept for now. Might change opinions based on peers' reviews. 

strengths_and_weaknesses: Strengths:

- Strong results on both datasets as mentioned. Also good ablation on table 2 to demonstrate the improvements from the claimed novelties. 

- Good writing. 

Weaknesses:

- Only somewhat sufficient novelty: I don't feel that reordering the steps of a voting-based method constitutes to sufficient novelty for this venue, but maybe only barely together with the 2nd-stage re-voxelization idea. A 2nd-stage detector is followed by initial points segmentation is not really new either. See [RSN: Range Sparse Net for Efficient, Accurate LiDAR 3D Object Detection]


questions: N/A
limitations: There is one sentence. What do the authors mean by diversity localities?
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: Existing 3D indoor object detection usually abandons semantic consistency within the same group and ignores diverse locality among different categories. To tackle this issue, the authors propose a novel class-aware proposal generation strategy that considers class-specific local groups. Besides, the authors propose an RoI pooling module to revisit the missed surface voxel features due to semantic segmentation errors. Their approach outperforms state-of-art methods on two challenging indoor datasets.
strengths_and_weaknesses: Strengths: 
+ The authors utilize a voxel feature encoding with a class-specific 3D voxel size to obtain the predicted vote voxels and generate class-specific geometric features based on them.
+ The authors propose RoI-Conv pooling, which effectively recovers the missed surface voxel features due to the semantic segmentation errors
+ The proposed method achieves state-of-art results with a remarkable gain on ScanNet V2 and SUN RGB-D.

Weaknesses: 
- The idea of using semantic predictions and geometric shifts to get class-aware clusters is not novel, which has already been used in previous work such as SoftGroup.
questions: - I am not sure about the effect of feature offsets. What is the difference between features extracted from the backbone and features added by feature offsets? Please demonstrate the design goal of feature offsets.
- As the set of neighboring input voxels is dependent on RoI-specific points, sparse abstraction is a time-expensive process. Please compare the inference time between the proposed method and existing methods in the parts of the experiment. 
- In Table 2, it is important for a reader to see all possible combinations of important modules (e.g. class-aware grouping and RoI-Conv) and see their potential influences.  I wonder if a better backbone can alleviate the improvements of local grouping. 
- There is no qualitative analysis in the main paper. It is hard for a reader to judge what are the potential improvements. 
limitations: Yes.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The authors introduced a novel two-stage 3D point cloud framework in indoor environment. In the previous bottom-up framework, the authors firstly modified the class-agnostic local grouping scheme, which may fail in clustering the object close to different categorical object. Instead of grouping class-agnostic manner as VoteNet, the proposed method grouped the voxels with the semantic consistency from the additional semantic prediction branch. Furthermore, the authors proposed an efficient ROI pooling module to compensate the miss-classified semantic prediction in the second stage. In the experimental section, the authors provided experiments on ScanNet V2 and SUN RGB-D benchmarks to demonstrate the proposed methods succeeds in grouping the objects within the same group and shows the better performance compared to existing class-agnostic methods.
strengths_and_weaknesses: [+] First of all, the motivation of the paper seems to be meaningful and pragmatic for indoor 3D object detection, because different types of objects may be located in a short distance in the indoor environment unlike the outdoor environment.

Despite of lacks of technical novelties, the authors well modified the existing class-agnostic model to solve the desired problem. The backbone model is selected to consider the geometric characteristic of the indoor object target, and the pooling module is designed for sparse convolution to handle limitations as computational overhead and geometric information loss.

As the detection paper, the overall framework is well-described, and specifically the details for reproducing and understanding the contribution such as target assignment, handling unique set, comparisons on different kernel size etc. are well presented.


[-] One concern is the proposed RoI pooling module. In sparse convolution, there is a hand-crafted parameter such as voxel size, and according to this factor, geometric loss occurs like existing pooling methods. 

Another concern is the role of the diverse local grouping. In Table 2, this factor has the greatest impact on performance improvement compared to other factors. Among all the contributions, this factor does not look much different from operating a class-agnostic model for each class efficiently. Therefore, I recommend that the authors should provide the opinion to reinforce other contributions.

questions: Feedbacks of weeknesses for validity of RoI pooling and opinion to reinforce other contributions.
limitations: I mentioned all comments including reasons and suggestions in the above sections. I recommend that the author will provide all the concerns, and improve the completeness of the paper (there is few grammatical error).
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper presents an approach for 3D detection from voxels. The authors propose to 2 stage detection pipeline that (1) first generates high quality box proposals by considering the spatial features of the scene. Compared to prior work, this stage also uses semantic information associated with each voxel. (2) Refines the object proposals and aggregates per-object features by a ROI pooling module. While prior work uses Set Abstraction (proposed in PointNet++) or max-pooling, the authors propose a new Sparse Abstraction operator. The final model is evaluated on indoor 3D detection on the ScanNet and SUN RGB-D datasets. 
strengths_and_weaknesses: Strengths
+ The authors report the results averaged over 25 trials in each table. They account for both the training time stochastic behavior as well as inference time stochastic behavior in 3D detection models. This is important for follow-up work as it (a) empirically validates the current paper’s claims; (b) helps understand the robustness of each proposed component.
+ Strong empirical performance on indoor detection benchmarks. In particular, as Table 1 shows, the proposed model achieves a much higher mAP@0.5 than prior work.
+ The Class-aware proposal prediction is well motivated. In particular (L143), predicting a dense semantic map before creating the proposals is a good idea since it additionally supervises the feature backbone. I also liked that the authors directly use the 3D bounding boxes for this part rather than segmentation masks (L158).
+ The 3D backbone inspired by HRNet is a good choice for trading of resolution and compute. It also leads to good gains in performance (Table 2)

Weaknesses
+ The title suggests that the authors use point clouds directly. However, their implementation first converts the point clouds into voxels (L102). The title is misleading. Similarly, the introduction talks about challenges of using unordered point clouds which do not exist once point clouds are converted to voxels, and thus are not challenges either faced by this work or addressed by it. Please update the title/introduction.
+ The re-voxelization step introduced in Eqn (4) seems pretty involved - (a) it requires a threshold per class which is changed with a custom schedule during training. This seems like a non-trivial hyper parameter search. (b) It additionally requires re-voxelization using using a separate parameter \alpha. (c) Finally, it requires aggregation using a kernel k_alpha. I worry that this step will need re-tuning on different datasets.
questions: + How are the weights in Eqn (8) tuned? How sensitive is the model to these weights?
+ Did you try any simpler alternatives to the re-voxelization step? For example, rather than creating separate subsets, did you try feeding in the semantic probabilities of the classes into the aggregation module (which could be class-specific)?
+ Do you mean gIOU loss when you mention IOU loss in L237? Also, is the VoteNet baseline in Table 2 also trained with the IOU loss?
+ Are there particular classes for which this model is better than prior work? Thin/small objects? Since the authors spend effort in accounting for each object class’s size while aggregating features, I expect this design decision may work particularly well for some classes.
limitations: Yes.
review_title: null

Meta-review: metareview: 4 expert reviewers suggest acceptance, based mostly on a strong evaluation section that shows good improvements over previous methods. Novelty of the method is deemed sufficient and well ablated. Overall seems like a good quality paper, although a tiny bit on the incremental side, but enough for recommending acceptance.
confidence: Certain
Decision: Accept