Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper presents a "progressive coordinate transform" strategy for improving monocular 3D object detection. Specifically, the authors identify that the main bottleneck in performance for monocular 3D object detection is the localization performance and tailor a solution towards progressively refining the coordinate proposals with multiple stacked modules before making the final 3D bbox prediction. The authors also propose a "Global context encoding" which processes and concatenates the visual features from the object proposal before the final 3D bbox prediction. This empirically results in an improvement in overall 3D bbox detection performance - achieving state of the art of the KITTI monocular 3D object detection benchmark. The primary contribution of this paper is identifying localization performance as a key bottleneck for 3D monocular object detection and tailoring a simple solution to improve performance over prior work.
main_review: *Strengths*
-  Table 1 in the paper is most insightful in my opinion and identifies localization as a bottleneck (vs size or rotation prediction). This is also somewhat expected because the variation in dimensions for cars is not expected to be high and orientation is also expected to be in a constrained range in driving scenarios. However quantifying this still has value. The authors build from this simple intuition and propose an iterative refinement method after the coordinate proposal module in standard 3D monocular object detection methods. The simplicity is key.

*Weaknesses*
- The two major concerns I have with the paper are technical novelty and significance of improvement over prior work. The major contribution in this work is really the "CLB" module which is primarily an iterative refinement module which predicts residuals between the ground truth and the previous prediction at each stage. The GCE module which appends a visual feature vector computed from the detector backbone isnt necessarily novel and has relatively low effect on performance (<0.4AP in table 4).

- The final improvement on the validation set seems significant for the "Hard" subset (~3.5AP in AP3D) but might not be very significant in my opinion on the other subsets and metrics. This is true when comparing against non coordinate-based methods on the KITTI test set in Table 6. 

- The figures in the paper often do not add much to the discussion. Fig 1 lacks more visuals and primarily a set of blocks. The caption for Fig 2(a) does not do a good job of explaining it and one has to follow the text very closely to understand the schematic here. The top row if FIg 3(a) is also hard to follow. The authors mention "peakier" red distribution curves indicating decreasing localization error over iterations but this is not readily visible to me. The BEV visualization and image bbox visualization in Fig 4 are also very small and the boxes themselves are hard to see. Overall, the figures in the manuscript need quite some improvement.


limitations_and_societal_impact: The authors are encouraged to comment on the technical novelty and significance of improvements over prior work on the KITTI dataset.
review_title: null

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: The paper first analyzes where the main performance loss is in the
inference of 3D bounding boxes from 2D images and concludes it is
in the 3D box center estimation phase. The introduced method
is an add-on iterative refinement which can boost performance of
several algorithms, and improves state-of-art when boosting
PatchNet, as applied to the KITTI dataset.

main_review: The paper introduces an iterative box center coordinate refinement stage
into the 3D BBox estimation (based on 2D box detections and monocular
depth estimates).
Confidence weights are estimated for each progressive refinement.
Semantic information is also added, which also boosts box location
estimation. An ablation study shows that each of the 3 added components
boosts performance a little. The proposed addition is also added to
the Pseudo-LiDAR algorithm, also improving performance.
Comparison of the number of iterations of the localization boost was
done to select the best number of iterations (3).

The paper is good: some nice analysis and ideas, good experiments, clearly written, but is also incremental; hence the proposed score of 8.
limitations_and_societal_impact: I saw no obvious weaknesses. I was curious about the magnitude of the
weights learned.
There was a slight confusion between T=3 and +2 in table 3. Presumably
they mean the same thing, so it might be better to use the T values
rather than the increments.
review_title: null

final_score: rating: 3: Clear rejection
review_content: summary: This paper proposes progressive coordinate transforms to solve the problems in monolar 3D object detection. But there are some problems with workload and experiments.
main_review: 1. The main contribution of this paper focuses on the application of gradient enhancement to localization prediction. Global context encoding seems just a small improvement to former algorithms. Therefore, the overall contribution is not particularly large.

2. The experiment in this paper is carried out on the KITTI data set. The results on the test set in Table 6 should be obtained from the official website submitted to the official website, but the corresponding data (anonymous) is not seen on the KITTI website. So where did the experimental results in Table 6 come from? If it was obtained by KITTI, please submit it anonymously; if not, then there is a problem with the authenticity of the data. In addition, KITTI includes three types of detection object. Why does the experimental result only include the detection result of the car? How about the other two types of test results?

3. The experiment is only carried out on the KITTI dataset, but more experiments on data sets such as waymo and nuscences should be carried out. In addition, the method in this paper is actually a 3D object detection algorithm based on monocular images, so it should be compared with other algorithms of the same type (3D object detection based on monocular image), such as GrooMeD-NMS[1], DDMP-3D[2], CaDDN[3], etc, but this paper does not compare them.

[1] A. Kumar, G. Brazil and X. Liu: GrooMeD-NMS: Grouped Mathematically Differentiable NMS for Monocular 3D Object Detection. CVPR 2021.
[2] L. Wang, L. Du, X. Ye, Y. Fu, G. Guo, X. Xue, J. Feng and L. Zhang: Depth-conditioned Dynamic Message Propagation for Monocular 3D Object Detection. CVPR 2020.
[3] C. Reading, A. Harakeh, J. Chae and S. Waslander: Categorical Depth Distribution Network for Monocular 3D Object Detection. CVPR 2021.

4. This paper claims to design a lightweight coordinate-based network, so what is the amount of calculation and complexity of the algorithm in this paper? How does it compare to other coordinate-based networks? The comparison between the results of this part and other methods is also very important, but it does not appear.

limitations_and_societal_impact: The authors have not addressed the limitations and potential negative societal impact of their work.
review_title: null

Meta-review: At the time of rebuttal, the reviewers had widely varying opinions on the paper. Major concerns included general clarity as well as a set of fairly concrete experimental issues. The authors responded to the reviews with detailed new experiments to address the reviewer concerns. Reviewer vnDo was persuaded by these experiments and raised their score to an acceptance. Reviewer rVMM did not participate in post-rebuttal discussion. The AC examined rVMM's requests and the authors' responses. The AC cannot speak on behalf of rVMM; however, in the AC's view, much of the reviewers' concerns are ostensibly addressed directly with reasonable experiments. On balance, given that the consensus of the active reviewers is acceptance, and the rebuttal seems likely to address the remaining reviewer's concerns, the AC is strongly inclined to recommend acceptance.

The AC would suggest that the authors:
- Incorporate, to the best of their ability, the results presented in the rebuttal. If they do not fit, the authors should report them in the supplement.
- Take Reviewer vnDO's comments on figures and clarity seriously, and update them and their captions. 
- As noted by the reviewers, the authors should extend their discussion of social impacts.

Both promises/new results were instrumental in the paper's acceptance, and the authors will maximize the impact of their paper if they make these changes.
Decision: Accept (Poster)