Venue: CoRL 2023 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary_of_paper:   This paper introduces a novel knowledge distillation framework called ADU-Depth for boosting precise depth estimation in monocular images. The framework leverages left-right image pairs to transfer 3D geometry-aware knowledge to a monocular student network. It includes attention-adapted feature distillation, focal-depth-adapted response distillation, and uncertainty modeling to guide effective and smooth knowledge transfer. The paper also presents extensive experiments and an ablation study to verify the effectiveness of each module in the proposed distillation strategy. The results show that ADU-Depth outperforms other state-of-the-art methods in terms of accuracy and robustness.


 

review:   This paper introduces a novel knowledge distillation framework called ADU-Depth for boosting precise depth estimation in monocular images. The framework leverages left-right image pairs to transfer 3D geometry-aware knowledge to a monocular student network. It includes attention-adapted feature distillation, focal-depth-adapted response distillation, and uncertainty modeling to guide effective and smooth knowledge transfer. The paper also presents extensive experiments and an ablation study to verify the effectiveness of each module in the proposed distillation strategy. The results show that ADU-Depth outperforms other state-of-the-art methods in terms of accuracy and robustness.

 
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation:   This paper introduces a novel knowledge distillation framework called ADU-Depth for boosting precise depth estimation in monocular images. The framework leverages left-right image pairs to transfer 3D geometry-aware knowledge to a monocular student network. It includes attention-adapted feature distillation, focal-depth-adapted response distillation, and uncertainty modeling to guide effective and smooth knowledge transfer. The paper also presents extensive experiments and an ablation study to verify the effectiveness of each module in the proposed distillation strategy. The results show that ADU-Depth outperforms other state-of-the-art methods in terms of accuracy and robustness.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary_of_paper: In this work, the authors propose a new single image depth estimation model (student model) by training a deep learning model that learns from a stereo image depth estimation network. It suggests a self-attention mechanism to align the features between the student and teacher models. An uncertainty estimation module is applied to reweight the losses during training. The authors have conducted experiments on two public datasets, KITTI and DrivingStereo, and the presented approach achieves 1st place on the KITTI benchmark.
review: Strengths:
+ The paper is clearly written, and it is easy to follow.
+ The experimental results are compelling.
+ The idea of using a stereo image depth network as the teacher to supervise a single image depth image estimation model is new, although it is not that surprising.

Weaknesses:
+ Although the idea of using a pretrained model to supervise single image depth estimation is new, it is essentially another strategy of knowledge distillation. Knowledge distillation has been used in several prior works, as shown in Table 1. It is hard to say the overall idea of this work is very novel.
+ One weakness is the need for stereo images to train the student network because the teacher network requires stereo images. 
+ It would be good to evaluate the presented approach on other datasets to see the zero-shot performance. 
robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: I think the performance of the presented work is strong, and the proposed idea is new.
review_title: null

final_score: rating: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary_of_paper: The paper introduces ADU-Depth, a knowledge distillation framework for enhancing monocular depth estimation in robotics applications. The main idea is to leverage knowledge distillation to transfer 3D-aware knowledge from a teacher network to a monocular student. The teacher network learns from left-right view image pairs, and this knowledge is incorporated into the student's encoder-decoder architecture. Additionally, ADU-Depth introduces an attention adaptation module, a distillation-score based focal-depth-loss, and an uncertainty estimation module to guide the distillation process effectively and achieve more high-quality responses. The proposed method achieves state-of-the-art performance on the KITTI and DrivingStereo datasets, ranking first on the KITTI online benchmark.
review: Strengths:
1.	The proposed ADU-Depth framework leverages left-right image pairs to transfer 3D geometry-aware knowledge to a monocular student network. The motivation is sound.
2.	The paper is well-written and easy to follow. The proposed framework is described in detail.
3.	The evaluation of three standard benchmarks is convincing.

Weaknesses:
1.	The technical novelty is relatively low. The proposed framework builds upon existing techniques and has no technically novel components. For example, the use of left-right image pairs for 3D-aware knowledge transfer has been explored in previous works. Similarly, distillation techniques have been used in previous works for monocular depth estimation. Essentially, it appears that the proposed method applies existing works to improve performance. 
2.	There are some more recent related works not mentioned:
[1] Wimbauer, Felix, et al. "Behind the Scenes: Density Fields for Single View Reconstruction." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.
[2]  Zhou, Kaichen, et al. "Devnet: Self-supervised monocular depth learning via density volume construction." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022.
[3]  Z. Liu, R. Li, S. Shao, X. Wu and W. Chen, "Self-Supervised Monocular Depth Estimation with Self-Reference Distillation and Disparity Offset Refinement," in IEEE Transactions on Circuits and Systems for Video Technology.
[4]  Ning, Chao, and Hongping Gan. "Trap Attention: Monocular Depth Estimation With Manual Traps." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.
[5] Li, Zhi, et al. "Test-time Domain Adaptation for Monocular Depth Estimation." 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023.
3.	More discussion on the failure cases and limitations would be welcome.

robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: Overall, the paper is well-written and tackles a practical problem. However, It appears that the method is a combination of prior ideas towards this problem. Additionally, the experimental details and discussions are insufficient, and there are no robotic experiments and it is questionable whether the proposed method can perform within robotic scenes.  Thus, my rate is weak reject.
review_title: null

Meta-review: The paper proposes a novel knowledge distillation framework called ADU-Depth for boosting precise depth estimation in monocular images. The framework leverages left-right image pairs to transfer 3D geometry-aware knowledge to a monocular student network. It includes attention-adapted feature distillation, focal-depth-adapted response distillation, and uncertainty modeling to guide effective and smooth knowledge transfer. The paper also presents extensive experiments and an ablation study to verify the effectiveness of each module in the proposed distillation strategy. The results show that ADU-Depth outperforms other state-of-the-art methods in terms of accuracy and robustness, and the results are promising for real-world robotics applications.

Most reviewers agree that the paper is well-written and presents compelling experimental results. However, reviewers also pointed out the technical novelty of the proposed method is limited. And the failure cases and limitations of the proposed method should be discussed more.

During the rebuttal phase, the reviewer acknowledged that some of the concerns have been addressed.

We recommend accepting the paper as poster. And we ask the authors to explain the technical novelty of their work in more detail, and provide more discussion on the failure cases and limitations of the proposed method in the camera-ready version.
Decision: Accept (Poster)