Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: Targeting monocular 3d object detection, this paper exploits depth information by estimating ground plane and fuses that information with image features via attention layers. Experimental results on KITTI are very strong, outperforming MonoDETR by 2-4points.
strengths_and_weaknesses: Strength:
+ The proposed method is intuitive and clearly presented (i.e. it exploits depth information using ground plane). The paper provides a clear explanation and derivation of how the ground depth is estimated and used in the model. 
+ Experiments are comprehensive and the results are strong. It outperforms previous sota by large margin (Tab.1), provides detailed ablation of different components (Tab.2), and gives good visualization of the output with attention maps.
+ Code is provided with high quality.

Weaknesses: the proposed method could be upbounded by ground-contacting point detection and assumes the ground plane is flat, while it works well on KITTI mono 3d detection, it could be less effective in general.
questions: The paper is well written. For failure cases when ground-contacting points are hard to detect, does the method break fully or can it still output reasonable detections (i.e. it can learn to ignore those features)? including visualization of some failure cases would help better understand the effectiveness.
limitations:  As the authors mentioned, the proposed method is limited by ground-contacting points detection and could fail in cases where those points are occluded or truncated.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The main idea in the paper is to assume a flat ground plane and leverage known camera height above it etc to get depths for the ground points. The method refines the camera pose relative to this plane via horizon/vanishing point estimation in the images by following the DeepVP method[7] thus deriving depths for ground pixels.  For 3D detection, the idea is to fuse these depths with the image features using transformer layers, which pick suitable ground points for the object to derive its depth. For the final detector, the GUPNet head is used. 

The results are claimed to be top on monocular 3D image detection on KITTI, which would be an impressive achievement. Suitable ablations for the method components are done. Additional results, showing the applicability of the introduced pose estimation components to detectors other than GUPNet are also presented. 
strengths_and_weaknesses: Strengths: 
- Insightful use of transformers to fuse estimated depths for object detection. 
- Intuitive overall idea to leverage the flat ground plane assumption and estimate the camera pose relative to it -- it seems to make a difference in KITTI. 
- Strong results on KITTI and meaningful ablations. 

Weaknesses:
- The need for pose estimation on KITTI, what quantities were given and which were not, was not particularly clear. As a result, Sec 3.2 was a bit difficult to read. Consider clarifying this better in a subsequent version. 
- The limitations of the method are not well explained. It assumes a flat ground plane which may not suit hilly scenes. Without reading the papers, it's not particularly clear to me how vanishing point estimation works when the road makes a turn. No experimental results showing model failures in such cases (or any cases really) are presented, in order to better understand the limitations / problems with the model. 
- It is difficult to track based on the links presented, whether this method is #1 on KITTI for monocular depth or not. I am willing to believe it and I see MoGDE represented on the leaderboard here: http://www.cvlibs.net/datasets/kitti/eval_object.php?obj_benchmark=3d. It clearly outperforms GUPNet and MonoDETR, so perhaps the claim is correct. 
- There are other CVPR 2022 methods on the leaderboard stronger than that are not necessarily cited (e.g. Diversity Matters: Fully Exploiting Depth Clues for Reliable Monocular 3D Object Detection, which has better numbers than MonoDETR). 
questions: How exactly did you know where the vanishing point is on KITTI? I don't fully understand how you get the GT. 

Please comment on flat plane / curvy road limitations of the method, if any. 

Any way one can see which papers are monocular detection, on the KITTI leaderboard? 


limitations: No specific limitations apart from what I mention in Questions above.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper utilizes ground depth as geometry cue to improve monocular 3D detection. The ground depth is estimated by camera pose and vanishing point. And the ground depth information is further integrated into the network by the transformer. The experiments demonstrate the effectiveness of the proposed components. And it achieves state-of-art results on the KITTI benchmark. 
strengths_and_weaknesses: + The motivation is good. The ground depth is more reliable in the Mono3D task than other geometry clues. 

+ The fusion of ground depth is well-structured, in which the ground-aware transformer locates the ground-contacting points and integrates their features. 

+ The method is easy to follow. It is flexible to extend to other Mono3D models.

+ Extensive experiments show the effectiveness of the proposed components. And the method achieves SOTA results on the KITTI leaderboard. 

questions: 1. The paper mentions the Mono3D performance of far objects many times. How about the accuracy of objects at long distances (>50m)?

2. The experiments are only performed on the KITTI dataset. Is this method evaluated on other public datasets (e.g. Waymo, Nuscenes)?
limitations: - As mentioned in the conclusion, the ground depth heavily relies on pose estimation. And it is difficult to ensure the accuracy of pose estimation.

- The method assumes that the ground is flat. However, the ground at the long distance and at the side is inconsistent with the nearby place. Therefore the virtual scene constructed from the infinite plane is not convinced. 

- The formulas adopted in the equations are not unified. For example, the EL is suggested to be replaced by a single letter.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposes a framework, namely MoGDE, to consider ground depth cues for more accurate and robust monocular 3D object detection. Specifically, it estimates the camera pose at first and constructs ground depth map to guide the depth estimation of objects. The fusion of ground depth and 2D features is achieved with a depth-aware DETR formulation. Finally, the overall framework achieves state-of-the-art results on the KITTI benchmark, and the qualitative results support the claim that the depth information of the ground assists the localization of 3D objects.
strengths_and_weaknesses: Strengths:
- The paper has a clear motivation, i.e., to use ground depth cues to assist the depth estimation of 3D objects.
- The basic idea is easy to follow, indicating that the presentation of this paper is overall smooth.
- The methodology part is overall clear and rigorous, although there are some points that require clarification.
- The performance on the KITTI test set is impressive, and the qualitative results well support the core idea of methodology.

Weaknesses:
- There are some confusing points in the methodology part:

    a. line 143: The camera pose typically consists of 6 DoF? including 3-DoF translation and 3-DoF rotation. Nevertheless, the author directly takes it as pitch and roll angles, which can be inaccurate and confusing especially when using this concept for the first time.

    b. line 170: How do the "predicted ground plane and vanishing point" formulated? It seems that there is no explanation for their specific definition.

    c. line 171: The matrix A not need any extra annotations? May need to clarify how the matrix is involved in the training procedure, at least more clearly in the main paper.

    d. line 182: The projection matrix in KITTI involves several offset factors, such as bx, by, bz to describe the baseline offset between camera 2 and the referenced camera. How do you deal with such details when deriving the projection equations?

    e. line 193: Why can the virtual stereo setup address the problem that the point near the vanishing point is not continuous? Here needs many words to clarify.

- Experiments:

    a. There seems no ablation or detailed results to reveal that the proposed method has boosted the detection performance for far-away objects, but the author takes this point as an important contribution in the presentation.

    b. Why need experiments on the Odometry Benchmark and why only show the qualitative results on that? Is there any difference between 3D detection Benchmark and Odometry Benchmark? How about the quantitative results on the Odometry Benchmark?

    c. line 267: lack of the explanation for the experiment group (d) (seems there is a typo)

    d. Why does the proposed method perform a little worse on the Easy split of the validation set?

    e. Would be better and more convincing to have experiments on a second dataset, such as nuScenes or Waymo.

- Minor comments:

    a. Can further polish the paper and fix some minor typos, such as need -> needs in line 198.

    b. Related works can be included:
        Ground-aware Monocular 3D Object Detection for Autonomous Driving, RA-L 2021.
        Probabilistic and Geometric Depth: Detecting Objects in Perspective, CoRL 2021.
questions: See the weaknesses.
limitations: The author adequately analyzes the limitation of the proposed methods in the Conclusion Section.
review_title: null

Meta-review: metareview: The paper received positive leaning reviews (2x borderline accept, 1x weak accept, 1x accept). The meta-reviewer agrees with the reviewers' assessment of the paper.
confidence: Certain
Decision: Accept