Kati8/ICLR_2023_2025 huggingface dataset:
authors: Mingqian Ji
authors: Jian Yang
authors: Shanshan Zhang
reviews: 
Comments: The paper proposes DH-Fusion, a depth-aware feature fusion strategy for LiDAR-camera 3D object detection, which dynamically adjusts modality weights using depth information at both global and local levels. By incorporating depth encoding, the method balances the contributions of point cloud and RGB image data, improving detection accuracy and robustness against data corruptions. Extensive experiments on datasets like nuScenes and KITTI demonstrate that DH-Fusion outperforms state-of-the-art methods.
Strengths: 1)The proposed Depth-Aware Hybrid Feature Fusion (DH-Fusion) strategy is well justified, using statistical analysis to show the varying roles of modalities across different depths. This is meaningful to the field of multimodal 3D object detection.
2)This paper includes comprehensive experiments on nuScenes, KITTI, and nuScenes-C datasets, which provide effective evidence of the method’s effectiveness and robustness.
Weaknesses: 1)The proposed fusion method depends on BEV features, which is not general for some point-based, voxel-based in 3D features.
2)The explanation of why depth encoding is more effective for local features could be more comprehensive. The authors should consider adding a theoretical rationale or hypothesis to elucidate why local features exhibit heightened sensitivity to depth variations.
3)It would be important to discuss any potential limitations or key differences compared to existing methods. Additionally, since the method is considerably more complex than BEVFusion, providing metrics on computational efficiency(parameters, GFLOPs, latency) and accuracy would strengthen the evaluation and offer a clearer understanding of the trade-offs involved.
4)The ablation study results are informative; however, more detailed experiments would be beneficial. For instance, assessing the performance of each component under diverse weather conditions in the nuScenes-C dataset could yield deeper insights into the robustness and adaptability of DH-Fusion.
5)The application of the proposed method to the KITTI dataset needs further clarification. Specifically, the baseline used should be explicitly identified, and the improvements brought by each proposed module should be clearly quantified and discussed.
Recommendation: 5
Reviewer confidence: 4
Soundness/correctness: 2
Clarity: 3
reviews: 
Comments: The paper presents a method that uses depth information to enhance the fusion of LiDAR and camera data in 3D object detection. The author proposes to adaptively adjust the fusion weights of the two modalities based on object depth, addressing limitations in prior methods that overlook depth variations. This approach integrates depth-aware fusion at both global and local levels, preserving critical details and improving detection accuracy. Experiments demonstrate that DH-Fusion outperforms existing methods on multiple datasets and shows greater robustness to data corruption.
Strengths: The paper introduces a novel depth-aware fusion approach for 3D object detection that outperforms existing methods on multiple datasets and shows greater robustness to data corruption.

Extensive experimental design with extensive benchmarks on multiple datasets supports its effectiveness. Ablation studies clearly validate the contributions of each module.

The paper is generally well-organized with helpful visual aids, though some mathematical sections could be simplified for accessibility.
Weaknesses: 1. While DH-Fusion introduces a depth-aware module in fusion weights, it applies a similar strategy across the entire depth range, which might not be optimal. Objects at very close or far distances likely require different levels of emphasis on specific feature types (e.g., closer objects may benefit from high geometric fidelity in LiDAR, while far objects may rely more on image data). 

2. While the method offers some contributions, its overall originality appears moderate, as it primarily builds upon established approaches such as LoGoNet and IS-Fusion, with the addition of a depth-guided weighting mechanism for fusion. This depth-aware module is indeed beneficial but does not constitute a fundamentally new approach to multimodal 3D object detection.

3. The paper includes a comparative analysis of speed, showing that DH-Fusion achieves relatively high efficiency. However, the reason behind this speed advantage is unclear, as the authors have not provided an in-depth breakdown of how DH-Fusion’s architecture contributes to these performance gains. Including a detailed computational analysis would improve transparency and help clarify the factors driving the faster inference times.

4. Additionally, in Table 9, it would be valuable to see a more comprehensive comparison of the model’s performance on smaller, more distant objects, where the benefits of depth-aware fusion are likely to be more pronounced.
Recommendation: 5
Reviewer confidence: 4
Soundness/correctness: 3
Clarity: 3
reviews: 
Comments: The paper proposes a depth-aware LIDAR-Camera fusion method, including a depth-aware glocal feature fusion (DGF) module and a depth-aware local feature fusion (LGF) module,  for 3D object detection. Extensive experiments were conducted to validate the effectiveness of the proposed method on both the nuScenes and KiTTI datasets. The core contribution is injecting the depth information into the process of cross-modal feature fusion.
Strengths: 1. The presentation, including the writing and figures, is clear.

2. The motivation and the way to inject depth information are straightforward, easy to understand.

3. Extensive experiments and analyses were conducted to show the effectiveness of each component.
Weaknesses: 1. The ablation study on the effectiveness of depth encoding is conducted separately on DGF and DLF, but a lower baseline might yield larger gains. Could you provide results for the entire model without depth encoding, i.e., DH-Fusion-base without depth encoding?

2. Based on the analysis in Lines 044–053 of the introduction, it seems that the detector should focus more on LiDAR features at close range and on image features at longer range. However, in the proposed method, the output of the depth encoder is multiplied by LiDAR BEV features in DGF and by cross-modal instance features in DLF. Is this design consistent with the findings mentioned above? Additionally, Section 4.6 visualizes the attention weights in DGF, but not the output of the depth encoder or the feature multiplied by it. How can the attention weights applied to the image BEV features in the DGF module support the analysis in Lines 044–053?

3. Why not compare with more recent works from ECCV? For example, SparseLIF achieves better results, with 77.0% NDS on the nuScenes validation set. The submission deadline for ICLR is after the ECCV conference.

[1] SparseLIF: High-Performance Sparse LiDAR-Camera Fusion for 3D Object Detection.
Recommendation: 5
Reviewer confidence: 5
Soundness/correctness: 3
Clarity: 3
reviews: 
Comments: This paper is based on the observation that LiDAR points on a target decay more significantly with distance compared to camera pixels, leading to information loss at greater distances in LiDAR data. Consequently, in multi-modal detection, more emphasis should be placed on camera data to compensate for this loss. Unlike traditional transformer-based methods, this paper introduces a novel approach to "explicitly" leverage distance information, enhancing cross-attention between modalities. The proposed method discretizes target distance and uses positional encoding within a transformer framework to incorporate this information into both global and local feature fusion. Experimental results on the KITTI and nuScenes datasets demonstrate that the proposed approach outperforms baselines with comparable backbones.
Strengths: - The paper offers a clear, statistically grounded analysis that highlights the disparity in information decay between camera pixels and LiDAR points, justifying the architectural adjustments. The experimental validation supports the efficacy of the proposed solution, making the paper well-structured and cohesive.
- The figures in the paper effectively clarify the architecture, aiding in the comprehension of the proposed methodology.
- The proposed approach achieves superior results on both KITTI and nuScenes datasets, surpassing baselines with similar backbones.
Weaknesses: - Although the paper focuses on using positional encoding to make LiDAR query features aware of target distance from the ego vehicle, similar methods (e.g., BEVDet, BEVFormer) have already employed positional encoding. The main novelty here is its application in multi-modal detection for autonomous driving, which limits its originality.
- For a paper emphasizing depth encoding in multi-modal detection, exploring additional variations of positional encoding would provide more comprehensive contributions to the field.
- There is a lack of analysis on whether existing methods implicitly utilize depth information and why explicit use might be preferable.
- Excessive detail on model structure detracts from the primary novelty of the proposed method.
- It is unclear how much additional computation the method requires to achieve improved performance over the baseline.
Recommendation: 5
Reviewer confidence: 4
Soundness/correctness: 3
Clarity: 3