ADU-Depth: Attention-based Distillation with Uncertainty Modeling for Depth Estimation
Authors: ZiZhang Wu
Zhuozheng Li
Zhi-Gang Fan
Yunzhe Wu
Xiaoquan Wang
Rui Tang
Jian Pu
Venue: CoRL 2023 Poster (2023)
Keywords: Monocular depth estimation
Camera perception
Distillation

Abstract: Monocular depth estimation is challenging due to its inherent ambiguity and ill-posed nature, yet it is quite important to many applications. While recent works achieve limited accuracy by designing increasingly complicated networks to extract features with limited spatial geometric cues from a single RGB image, we intend to introduce spatial cues by training a teacher network that leverages left-right image pairs as inputs and transferring the learned 3D geometry-aware knowledge to the monocular student network. Specifically, we present a novel knowledge distillation framework, named ADU-Depth, with the goal of leveraging the well-trained teacher network to guide the learning of the student network, thus boosting the precise depth estimation with the help of extra spatial scene information. To enable domain adaptation and ensure effective and smooth knowledge transfer from teacher to student, we apply both attention-adapted feature distillation and focal-depth-adapted response distillation in the training stage. In addition, we explicitly model the uncertainty of depth estimation to guide distillation in both feature space and result space to better produce 3D-aware knowledge from monocular observations and thus enhance the learning for hard-to-predict image regions. Our extensive experiments on the real depth estimation datasets KITTI and DrivingStereo demonstrate the effectiveness of the proposed method, which ranked 1st on the challenging KITTI online benchmark.

ADU-Depth: Attention-based Distillation with Uncertainty Modeling for Depth Estimation
Zizhang Wu1.2f
Zhuozheng Li
Zhi-Gang Fan
Yunzhe Wu 2
Xiaoquan Wang 3
Rui Tang
Jian P
Fudan University, Zongmu'Tech, "ExploAl
wuzizhang87@ @mmil (zhuozheng.li li, zhigang.fan, nelson.wu, rui.tang) @7zonmuutech.com, rocky. @ wxwleai.pppo....ooooooo jianpu
jianpu @ fudan.edu.cn
Abstract: Monocular depth estimation is challenging due to its inherent ambi- guity and ill-posed nature, yet it is quite important to many applications. While recent works achieve limited accuracy by designing increasingly complicated net- works to extract features with limited spatial geometric cues from a single RGB image, we intend to introduce spatial cues by training teacher network that lever- ages left-right image pairs as inputs and transferring the learned 3D geometry- aware knowledge to the monocular student network. Specifically, we present a novel knowledge distillation framework, named ADU-Depth, with the goal of leveraging the well-trained teacher network to guide the learning of the student network, thus boosting the precise depth estimation with the help of extra spatial scene information. To enable domain adaptation and ensure effective and smooth knowledge transfer from teacher to student, we apply both attention -adated fea- ture distillation and local-depth-ddapted response distillation in the training stage. In addition, we explicitly model the uncertainty of depth estimation guide dis- tillation in both feature space and result space to better produce 3D-awar knowl- edge from monocular observations and thus enhance the learning for hard-to- predict image regions. Our extensive experiments on the real depth estimation datasets KITTI and DrivingStereo demonstrate the effectiveness of the proposed method, which ranked 1st on the challenging KITTI online benchmark?.
Keywords: Monocular depth estimation, Camera perception, Distillation
Introduction
Monocular depth estimation, with the goal of measuring the per-pixel distance from single cam- perception, is absolutely crucial to unlocking exciting robotic applications such autonomous driving [1, 2], 3D scene understanding [3, 4] and augmented reality [5, 6]. However, it is quite dif- ficult to obtain precise depth values from only single 2D input, since monocular depth estimation is ill-posed and inherently ambiguous [7, 8J where many 3D scenes can actually give the same input picture [9, 10]. Early depth estimation methods relied primarily on hand-crafted features [11, 12], geometric assumptions [12], and paramerriiiic depth transfer [13] and achieved limited success.
With the rise of deep learning techniques, the performance of monocular depth estimation has significant boost [14, 15, 16). For supervised learning, list of approaches focus on designing deeper and more complicated networks to improve depth prediction under the supervision of ground- truth depth [17, 18, 19]. The current trend has been to combine convolutional neural networks (CNNs) with vision transformers and attention mechanisms [20, 21, 22, 23]. which can extract more meaningful features in an image and capture long-range dependencies. Inspired by the works of
"The denotion means these authors contributed equally and means the corresponding author. FFrom 24 Jun. 2022 22 Feh. 2023.
7th Conference Robot Learning (CoRL. 2023), Atlanta, USA.

---

Transform within uxcettiaryy
Student Network
Single Imarge
Ponture Enzoder
Feeitre Liecader
Seadect Decd: Map
Far
Depth
Fecture Space
Result Space
Uncersiitty Estometion Modnle
UMR-DOenilation
SadMas
Nest
nligh
Deit product
Remmange dousserde
MRA-Diecllatin
UMP Ditiibaioo
Uacertuasty
Soli
IDvaware Kcowllige
Teacher Network
Depii Hoad
Tefl-Righ Image
Buccular Lawodirr
Fouture Devockrr
Tescher Depti Map
Figure 1: Illustration of ADU-Depth, which consists a student network, teacher network and the distillation scheme. We design attention adaatte feature distillation and focal-depth depth-adapted response distillation. Meanwhile, we introduce an uncertainty estimation module to guide the left- right knowledge transfer, which UMF-Distillation and UMR-Distillation. respectively. During inference, only the monocular student is employed with only a single image as input.
Conditional Random Fields (CRFs) [24, 11, 25], recent method [9] proposed to split the input into windows and incorporate the neural window fully-connected CRFs with multi-head attention mechanism [26] to better capture the geometric relationships between nodes in the graph. However, considering that a single image contains limited 3D scene geometric information, this renders the monocular depth estimation task a difficult fitting problem without extra depth cues [27, 10].
In this work, we present a novel distillation framework named ADU-Depth. We show that is able assist top depth predictors to infer more accurate depth maps, benefiting from 3D-aware knowledge transfer of left-right views as humans do. We first train a teacher network to learn 3D knowledge by concatenating the left-right images together and modify the model structure to accommodate the image pair inputs. This intuitive yet quite effective mechanism leads to a powerful teacher network and can be easily deployed for recent depth networks. Considering the strength of capturing geo- metric relationships between nodes in a graph, we use an encoder-decoder network similar to [9] the student network and add uncertainty branch. To ensure effective and smooth knowledge distillation, the proposed distillation framework should be capable of producing 3D-aware features and responses under the guidance of the teacher. Therefore, introduce self-attention adaptation module for feature imitation and shrink the feature domain gap. Next, we design a focal-depth- to more effectively distill high-quality responses from soft-labels provided by the teacher. Fur- thermore, we combine distillation with an uncertainty estimation module to learn the prediction distribution and better produce 3D-aware features and responses from monocular observations.
To the best of our knowledge, this is the first time in the supervised depth estimation community that left-right view knowledge is transferred from the teacher network to monocular student. In sum- mary, our contributions include: (i) novel ADU-Depth framework that distills 3D-aware knowl- edge from a teacher network into supervised monocular student; (ii) an attention adaptation module for both encoded and decoded feature distillation; (iii) a distillation score based focal weal-depph-loos for response distillation and focuses on hard-to- -ot-ns--ttt knowledge; and (iv) an uncertainty estimation module guide the distillation in both feature and result spaces. The above techniques enable our method to achieve state-of- ofee-----hhh-ar the-art performance of monocular depth estimation across all metrics the KITTI and DrivingStereo datasets and ranked 1st on the KITTI online benchmark.

---

2 Related Work
Monocular Depth Estimation. Neural network based methods have dominated most monocular depth estimation benchmarks. Eigen et al. [14, 15] first proposed a depth regression approach by utilizing two stacked CNNs to integrate global and local information. To model the ambiguous mapping between images and depth maps, Laina et al. [28] presented an end-to-end convolution architecture with residual learning. In contrast, Godard al. [29] formulated depth estimation as pixel-level classification task and also obtained depth prediction confidence. Besides, several multi-view methods [30, 2] employed recurrent neural network based optimization technique that repeatedly alternates depth and pose updates to minimize the metric cost. Furthermore, many self- supervised monocular works took left-right image pairs [31, 32, 33, 34, 35, 36] or multi-frame images [37, 38, 39] as input for the network training. By benefiting from the vision transformers, recent supervised works [18, 21, 23, 22] employed the attention mechanism for more accurate depth estimation. More recently, the neural window FC-CRFs with multi-head attention was proposed to better capture the relationship between nodes in the graph and reduce the computational cost [9]. However, in the absence of 3D geometric information, these complicated transformer and attention based methods achieved limited performance gain for monocular depth estimation.
Knowledge Distillation Methods. Knowledge distillation (KD) was first proposed in [40] to transfer the learned knowledge from a cumbersome teacher model to a light-weight student model through teacher-generatee generated soft targets. Besides its success in image classification, KD strategy has been widely applied in objection detection [41, 42, 43], semantic segmentation [44], and depth esti- mation [45, 46, 47, 36, 48]. Pilzer et al. [46] claimed that distillation is particularly relevant to depth estimation due the limitations of network size in practical real-time applications. They attempted to exploit distillation to transfer knowledge from the refinement teacher network to self-supervised student. For fast depth estimation mobile devices, Wang et al. [47] utilized knowledge distillation to transfer the knowledge of a stronger complex teacher network to a light-weight student network. More recently, DistDepth [48] distilled depth- depth-domain structure knowledge into a selff-supervied stu- dent to obtain accurate indoor depth maps. Unlike existing distillation methods for depth estimation, our ADU-Depth is the first to distill knowledge from left-right views into a supervised monocular depth predictor and incorporates novel distillation techniques into the framework.
Uncertainty Estimation Methods. Uncertainty estimation plays a critical role in improving the robustness and performance of deep learning models [49, 50, 51, 52]. Particularly, it was also intro- duced into monocular depth estimation [53, 54, 55], which was mostly in self- and semi-supervised methods due to their high uncertainty. Poggi et al. [56] examined several manners of uncertainty modeling in monocular depth estimation and suggested the optimum strategy in various scenar- ios. Nie et al. [55] employed a student-teacher network to iteratively improve uncertainty perfor- mance and depth estimation accuracy through uncertainty-awar self-improvement. Inspired by the work [57] that introduces uncertainty estimation to better produce high-quality-like features from low-quality observations for degraded image recognition, design an uncertainty estimation mod- ule and incorporate it into the proposed distillation framework for depth estimation, which can better produce 3D-aware features and responses from monocular observations.
3 Proposed Method
3.1 Overall Framework
As illustrated in Fig. 1, the framework of ADU-Depth can be separated into three parts (i.e., a teacher network, a monocular student, and the distillation scheme). We first pre-train the teacher network on left-right image pairs with dense ground-truth depth. We then freeze it to produce features and responses for training the student network on single image by the proposed distillation strategy. The goal of our framework is train the monocular student to learn to generate 3D-a features and responses under the guidance of the left-right view based teacher network. To meet this goal,
3

---

we first introduce attention attention-adapted feature distillation and focal-depth reral-detth-daapeed response distillation to encourage feature and response imitation, respectively. Then, an uncertainty estimation module (UEM) is designed to model the uncertainty of both the feature and response distribution.
Student model. We employ an encoder-decoder network similar to the work [9] as our baseline depth predictor due to its promising performance and ability to capture relationships between the nodes in the graph. In detail, it adopts multi-level encoder-decoder architecture, where four levels of neural window FC-CRFs modules are stacked. The model employs the wwin-transformer [58] encoder to extract multi-level image features and the CRFs optimization decoder predict the next- level depth according to the image features and coarse depth. We consider that the neural window FC-CRFs module and internal multi-head attention mechanism can better capture the critical scene geometry information of the left-right view data, hence outputting an optimized depth map. In particular, we further introduce an novel uncertainty estimation module alongside the depth head as another branch, which generates the per-pixel uncertainty to better guide the knowledge distillation.
Teacher model. To make easier accessible and more flexible to deploy the teacher network and ensure the model training efficiency in practical applications, while reducing the structural domain gap between the teacher and student networks, our teacher model chooses to leamn 3D knowledge by concatenating the left-view and right-view images together and modifies the initial three channels of the wwin-transformer to six channels to accommodate the image pair inputs. This mechanism is simple yet quite effective and can avoid huge computations and domain gaps brought by complex structural changes. Furthermore, the concatenation operation is also used in the stereo version of the unsupervised method [32], where the additional 3D information of left-right images enables more complete understanding of the input scene. Note that, left-right image pairs commonly used for practical robotic applications, and many existing datasets are available [59, 60, 61].
3.2 Knowledge Distillation Strategy
In this section, we describe our three complementary distillation schemes: attention-adapted feature distillation, hoalldeeth-adappter response distillation, and uncertainty modeling based distillation.
Attention-adapted feature distillation. Due to the domain gap between features extracted from a single image and left-right (stereo) image pair, we consider it sub-optimal to directly execute a monocular student to learn the feature representation of the teacher. While the 3D scene geome- try information learned from the left-right data domain greatly enhances teacher performance, the effective transfer of learned knowledge from teacher to student is an obvious challenge. Since the number of input channels of the student is not compatible with those of the teacher, an adaptation module is important to align the former to the latter for calculating the distance metric and enabling an effective feature approximation between the student and teacher. Unlike the work in [43], which uses full convolution adaptation layer, our experiments show that self-attention adaptation layers are more conducive to bridging the data domain gap and forcing the student to more effectively adapt and imitate the teacher's features. They are able capture long-range feature dependencies and focus the critical spatial cues of the image itself [26]. Specifically, the feature encoder ex- tracts four levels of feature maps following Fig. 1. For each level of feature maps, we first flatten it and then employ self-attention operation to obtain the -rentioo-adaated feature Es:
F = softmax (Q" Ky) X
(I)
where Q Kü and X are the learned query, key, and value of the student encoder, respectively. ()) denotes dot production. Similarly, the attention-adaple decoded feature FA also be obtained via Eq. (I), where Qū KG and X are learned from the multi-level decoded features. Then, given the multi-level extracted feature maps of the teacher model Ft and the attention attention-adapted adapted encoded student feature Fs. the proposed encoded feature distillation loss Le can be formulated as:
C. N
|
F2ll
(2)

---

Similarly, given the multi-level decoded feature maps of teacher model F'J and the iitention-adapteee decoded student feature F our decoded feature distillation loss L can be formulated as:
La
1
Fd F
(3)
focal-depth--dappee response distillation. We use the predicted depth from the teacher as extra soft labels to transfer the learned knowledge to the student in the result space. Furthermore, inspired by the focal loss [62], we design a focal-depth loss to enhance the learning of hard-to-predict sam- ples. Note that the focal loss is originally for classification and the p E [o, 1] in the focal loss the estimated probability for the class with label y 1. To enhance the student's learning of hard-to- transfer knowledge of depth estimation from the teacher network, we introduce a depth distillation score Pa E [0, 1] to indicate the degree of prediction approximation between the student and teacher:
1 N
d df dF
Pa
where N is the number of pixels, i is the pixel index. Then, use the variant an to balance the importance of high distillation-score examples. The proposed focal-depth loss can be described as:
(5) where 7 2 0 is tunable focusing parameter, df and dî denote the depth predictions of the teacher and student, respectively. Furthermore, we also use the soft label L1-loss for distillation in the result space and then combine it with uncertainty modeling, which is represented in Eq. (8). For each pixel i in an image, the initial soft label response distillation loss is computed as:
Liocal (pa) -od (1 pa)" log (pa),
Cyd
|| - dell,
(6)
Uncertainty modeling based distillation. To estimate the pixel-wise uncertainty, we design an uncertainty estimation module (UEM) as branch at the end of the backbone network. The UEM consists of a convolution layer, sigmoid activation function, and a scaling layer. Based on the Bayesian deep learning framework [49], we train the depth network to predict the per-pixel log variance s; = log o². The predicted uncertainty map has the same dimension as the depth map and is then rearranged to the size multi-level feature maps, i.e. 1/4, 1/8, 1/16, and 1/32 of the original size. Given the Gaussian assumption for & loss, the uncertainty modeling based feature distillation loss Lumf is defined as:
Lumf
exp (-8;) Ft | (F;
1
where F denotes left-right feature maps from the teacher network and F g (F) means the restored 3D-aware feature maps from the student network with estimated log variances s; through the uncertainty estimation module. For those Fdt far away from F². the network will predict larger variances to reduce the eтror term exp(-s:)
F 99 [F; (F;;
2 | instead of overfitting to those
erroneous regions. When F is easy to learn, the second term _ performs major role in the loss function, and the network tends to make variances smaller. In this way. it works similarly to an attention mechanism, which enables the network to focus on the hard regions in the image or hard samples in the training set [63, 57]. Lumf is employed on both encoded and decoded features.
For the response distillation in result space, choose Laplace uncertainty loss since it is more appropriate model the variances of residuals with loss. Given the Laplace assumption, d' g d s; to denote the learned stereo-aware student depth prediction. Then, the uncertainty modeling based response distillation (UMR- ) loss Lumr defined as:
N
(8)((())))))))) || (d;:s:) 8:)
Lumr
(8)
5

---

Method
Reference
Sup
Disiil
Sq Rel
Abs Rei
RMSE
RMSE
0
< 1.25
52
< 1.25
01 1.25
CVPR 2018
0.954
Guo et al. [45]
ECCV 2018
/
/
0.515
0.092
3.163
0.159
0.901
0.971
0.988
Refine and Distill [46]
CVPR 2019
4.6%6
0.202
0.882
0.948
0.973
Yin et al. [65]
ICCV 2019
/
X
0.072
3.258
0.117
0.938
0.990
0.998
SAN [66]
CVPR 2021
/
0.062
2.888
0.955
PWA [20)
AAAI 2021
/
0.221
0.060
2.604
0.093
0,958
0,994
0.999
DPT [21]
ICCV 2021
V
0.062
2.573
0.092
0.959
0.995
0,999
SingleNet [36]
0.681
4.392
0.185
0.892
0.9662
0.981
AdaBins [18]
CVPR 2021
/
X
0.190
0.058
2.360
0.088
0.964
01995
0,999
NeWCREs [9]
/
P5Depth [10]
CVPR 2022
V
0.270
0.071
2,842
0.103
0.953
0.993
0.998
Liu al. [67]
TCSVT 2023
0.635
0.096
0.171
0,905
0.969
0.985
ADU--Depth
0.147
0.049
2,080
0.076
0.976
0.997
0.999
Table 1: Quantitative results the KITTI Eigen split with cap of 0-80m. Seven widely-used
metrics are reported and calculated strictly following NeWCRFs [9]. The methods are divided by whether they are supervised (Sup) and the of distillation (Distill).
Method
Dataset
SILog
Sq Rel
Abs Rel
iRMSE
RMSE
01 < 1.25 1
02
1.25 1
0g 1.25
DORN [17]
val
12.22
3.03
11.78
11.68
3.80
0.913
0.985
0.995
BA-Full [19]
val
10.64
1881
8.25
8.47
3.30
0.988
0.997
NeWCRFs [9]
val
8.31
0.89
5.54
6.34
2.55
0.968
0.995
0.998
ADU-Depth
val
6.64
0.61
4.61
5.25
1.98
0.981
0.997
0.999
DORN [17]
online test
I1.77
2.23
8.78
12.98
BA-Full [19]
online
11.61
2.29
9.38
12.23
PackNet-SAN[66]]
online test
11.54
2.35
9.12
12.38
PWA [20]
online test
11.45
2,30
9.05
12.32
NeWCREs [9]
online
10.39
1.83
8.37
ADU-Depth
online test
9.69
1.69
7.26
9.61
Table 2: Quantitative results on the KITTI official split. Eight widely-used metrics are calculated for the validation set and four metrics from the KITTI official online server are used for the test set. Our method ranked 1st on the KITTI depth prediction benchmark (initially named "ZongDepth")..
3.3 End-to-End Network Training
For the training of the teacher network, only the loss Ls from the baseline method adopted [9]. We train our student network in end-to-end manner using the following loss function:
L = Lb + λ1 Lumr + x2 Lumf + λa Liocal,
(9)
where ÅI, A2, Ag are the hyper-parameters to balance the loss of each module.
Experimental Results
4.1 Implementation Details
Our work is implemented in PyTorch and experimented on Nvidia RTX A6000 GPUs. The network is trained for 40 epochs with mini-bare bati-bbtchhae size of 4. The input image is resized as 352 x 1 1120 before being fed into the network and then rearranged and downscaled to four levels in the encoder-decoder
network, i.e. 1/4, 1/8, 1/16, 1 1/22. The learning rate is 2 x 10 10-5, We set 31 = 0.9 and 32 = 0.999 the Adam optimizer for network optimization. For KITTI dataset, A = 0.9, A2 = 0.6, A = 0.8. The output depth map is 1/4x 1/4 size of the original image and finally resized to the full resolution. For the analysis of computation time, the inferred speed of our ADU-Depth is 3.82 FPS, which is faster than the 3.48 FPS of NeWCRFs [9] and without introducing additional inference cost.
4.2 Evaluations
Evaluation on KITTI. We first evaluate our method on the KITTI Eigen split [14]. As shown in Table 1, our ADU-Depth achieves new statc-of-the-ar performance with significant improvements over other top performers and existing distillation-banl depth estimation methods [45, 46, 36) with significant improvements. We then evaluate our method on the KITTI official split [59] as shown in Table 2. Obviously, method achieves state-of-the-an performance across all the evaluation metrics compared to existing depth estimation approaches.

---

99.4
SiI 8867
SiLog 5596
StLog 23.88
S1Log
S:Log - 16.67
SiLog 12:21
Inguut image
CRFs Depth map
Our Depth map
NeWC map
Our Erroz map
Figure 2: Qualitative results generated by the official server on the KITTI online benchmark.
Settings
Sq
Abs Rel
RMSE
Method
Sq Rel
Abs Rel 4
RMSE
Teacher
0.074
1.418
DORN [17]
0.126
0.055
2.173
Baseline
Adabins [18]
0.083
0.036
1.459
Focal
0.152
0.050
2.103
NeWCRFs [9]
0.071
0.032
1.310
UEM
0.154
0.051
2.108
ADU-Depth
0.064
0.028
1.146
Attention
0.161
0.052
2.127
Full-Setting
0.147
0.049
2.080
Table 3: Quantitative results the DrivingSterco.
Table 4: Ablation study on the KITTI Eigen split.
Evaluation on KITTI online test. On the challenging KITTI online benchmark, our method ranked 1st among all submissions of depth prediction for seven months, while NewCRFs [9] ranked 12th. The performance gain is more obvious than its improvement on the Eigen split, since our method better at predicting the depth of hard regions or test images with a domain gap. The scale-invariant logarithmic (SILog) error the main ranking metric used measure the relation- ships between points in the scene [14]. The lowest "SILog" error indicates that method could better capture the relationships between nodes in the scene, reasoning the learning of stereo- aware knowledge. We also provide qualitative comparison results in Fig. 2, including the predicted depth maps and the error maps. The error map depicts correct estimates in blue and wrong esti- mates in red color tones [68]. Dark regions denote the occluded pixels that fall outside the image boundaries. Our method predicts more accurate and reliable depths in various scenes, especially for hard-to-predie image regions, e.g., distant objects, repeated textures and weak light scenes. On the one hand, attention-adapted feature distillation allows model effectively learn more about the spatial information of the scene from the teacher network. On the other hand, the introduction of uncertainty and focal-depth made the model focus on the learning of difficult areas.
Evaluation on DrivingStereo. DrivingStereo [60] a large-scale stereo dataset that consists over 180k images covering diverse set of of driving scenarios. We further compare method with subset of the DrivingSterco dataset with 7000 image pairs for
several well-known methods training and 600 images only for testing. The quantitative results of our method compared with others are shown in Table 3, where four widely used evaluation metrics are calculated for the test set. Our ADU-Depth outperforms these monocular depth predictors in all evaluation metrics.
4.3 Qualitative results of Uncertainty Estimation
The uncertainty estimation module (UEM) should be capable of producing uncertainty values that are well aligned with the depth estimation errors. Fig. shows several examples of our predicted un- certainty maps, where brighter colors indicate high uncertainty. We values are distributed on the distant background region and the object boundaries. These uncertainty values are quite reasonable according to the estimated depth maps and sparse ground truths.
that the high uncertainty

---

Close
Fa
Low
High
Input image
Estimated depth map
Estimated uncertainty map
Ground truth
Figure 3: Qualitative results of ADU-Depth with UEM. High uncertainty values displayed
in far background regions and object boundaries according to the estimated uncertainty maps.
4.4 Ablation Study
To better inspect the effect of each novel component in our distillation strategy, we conduct each module by an ablation study in Table 4. We also report the results produced by our teacher network in the first row as a reference. The simple yet effective design teacher network achieves the promising performance by capturing additional 3D scene information in left-right image pairs.
Since the proposed distillation framework contains three main components, conduct indepen- dent experiments with different combinations to further verify the effectiveness of each module. The attention-adapted feature distillation module shows a significant performance gain. It implies that the key to shrinking the monocular-stereo feature domain gap is to adaptively learn the teacher- student feature approximation. Then, UEM-guided distillation also achieves lower estimation errors due learning for regions of high uncertainty and hard regions. Even though these two compo- nents have shown powerful effects, where the "Sq Rel" error is reduced from 0.188 to 0.154, the introduction of focal-depth adapted response distillation enables our ADU-Depth to achieve further accuracy boost and the optimal performance with the "Sq Rel" error 0.147. It benefits from the enhanced learning of hard-to-transfer knowledge from the teacher model.
5
Limitations
Although the proposed ADU-Depth can boost the performance of monocular depth estimation and leam 3D-aware knowledge from the left-right view based teacher model with novel distillation tech- niques (as shown in Table 4), our method cannot theoretically guarantee the applicability of depth estimation to 3D understanding. In future work, we will predict volumetric scene representation and leverage novel view synthesis to provide spatial cues for monocular depth estimation.
6 Conclusion
We propose a novel distillation framework, ADU-Depth, improve the performance of the monoc- ular depth predictor, especially distant objects well the silhouette of objects. To the best of our knowledge, we are the first supervised monocular depth estimation method that simulates the 3D-aware knowledge learned by a teacher network from left-right image pairs on a monocular student. To guarantee effective and smooth knowledge transfer to the monocular student, we design an attention-ann adapted feature imitation and a focal-depth based response imitation. In addition, we design an uncertainty estimation module model the uncertainty depth estimation, which guides the distillation in both feature space and result space. Extensive experimental results the KITTI and DrivingStereo datasets show that method achieves new state-of of-the-art performance.

---

References
[1] Y. Li, Y. Chen, J. He, and Z. Zhang. Densely constrained depth estimator for monocular 3d object detection. In European Conf. Computer Vision (ECCV), pages 718-734. Springer, 2022.
[2] Z. Wu, Z.-G. Fan, Z. Li, J. Wang, T. Xu, Q. Tang, F. Wang. and Z. Luo. Monocular fisheye depth estimation for automated valet parking: Dataset, baseline and deep optimizers. In 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC), pages 01- 07. IEEE, 2022.
[3] Laidlow, Czarnowski, and Leutenegger. Deepfusion: Real-time dense 3d reconstruction for monocular slam using single-view depth and gradient predictions. 2019 International Conference Robotics and Automation (ICRA), pages 4068-4074. IEEE, 2019.
[4] Wimbauer, N. Yang, L. Von Stumberg, N. Zeller, and D. Cremers. Monorec: Semi- supervised dense reconstruction in dynamic environments from single moving camera. In IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pages 6112-6122, 2021.
[5] F. El Jamiy and R. Marsh. Survey on depth perception in head mounted displays: distance estimation in virtual reality, augmented reality, and mixed reality, IET Image Processing, 13 (5):707-712, 2019.
[6] R. Du, E. Turner, M. Dzitsiuk, L. Prasso, I. Duarte, J. Dourgarian, J. Afonso, J. Pascoal, J. Gladstone, N. Cruces, et al. Depthlab: Real-time 3d interaction with depth maps for mobile augmented reality. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology (UIST), pages 829 843, 2020.
[7] M. Rey-Area, M. Yuan, and C. Richardt. 360monodepth: High-resolution 360deg monocular depth estimation. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 3762-3772, 2022.
[8] V. Guizilini, R. Ambrus, D. Chen, Zakharov, and A. Gaidon. Multi-frame self-supervised depth with transformers. In IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pages 160-170, 2022.
[9] W. Yuan, X. Gu, Z. Dai. S. Zhu, and P. Tan. Newerfs: Neural window fully-connected erfs for monocular depth estimation. In IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2022.
[10] V. Patil, C. Sakaridis, A. Liniger, and L. Van Gool. P3depth: Monocular depth estimation with a piecewise planarity prior. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 1610-1621, 2022.
[11] A. Saxena, M. Sun, and A. Ng. Make3d: Learning 3d scene structure from a single still image. IEEE Trans. Pattern Anal. & Mach. Intell (PAMI), 31 3155):224-840 2008.
[12] D. Hoiem, A. A. Efros, and M. Hebert. Automatic photo pop-up. ACM SIGGRAPH 2005 Papers, pages 577-584. 2005.
[13] K. Karsch, C. Liu, and Kang. Depth transfer: Depth extraction from video using non- parametric sampling. IEEE Trans. Pattern Anal. & Mach. Intell (PAMI), 36(11):2144-2158, 2014.
[14] D. Eigen, C. Puhrsch, and R. Fergus. Depth map prediction from a single image using a multi- scale deep network. Conference and Workshop Neural Information Processing Systems (NeurIPS), 27,207 2014.

---

[15] D. Eigen and R. Fergus. Predicting depth, surface normals and semantic labels with common multi-scale convolutional architecture. In IEEE Int. Conf. on Computer Vision (ICCV), pages 2650-2658, 2015.
[16] D. Xu, E. Ricci, W. Ouyang. X. Wang, and N. Sebe. Multi-scale continuous crfs as sequential deep networks for monocular depth estimation. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 5354-5362, 2017.
[17] H. Fu, M. Gong. C. Wang, K. Batmanghelich, and D. Tao. Deep ordinal regression network for monocular depth estimation. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 2002-2011, 2018.
[18] S. F. Bhat, I. Alhashim, and P. Wonka. AdaBins: Depth estimation using adaptive bins. In IEEE Conf. on Computer Vision and Puttern Recognition (CVPR), pages 4009-4018 8, 2021.
[19] S. Aich, M. U. Vianney, M. A. Islam, and M. K. Liu. Bidirectional attention network for monocular depth estimation. In IEEE Int. Conf. on Robotics and Automation (ICRA), pages 11746-11752 IEEE, 2021.
[20] S. Lee, J. Lee, B. Kim, E. Yi, and J. Kim. Patch-wise attention network for monocular depth estimation. In AAAI Conf. on Artificial Intell. (AAAI), volume 35, pages 1873-1881, 2021.
[21] R. Ranftl, A. Bochkovskiy, and V. Koltun. Vision transformers for dense prediction. In IEEE Inl. Conf. on Computer Vision (ICCV), pages 12179-12188, 2021.
[22] A. Agarwal and C. Arora. Attention attention everywhere: Monocular depth prediction with skip attention. In IEEE Winter Conf. on Applications of Computer Vision ( (WACV), 2022.
[23] S. F. Bhat, I. Alhashim, and P. Wonka. Localbins: Improving depth estimation by learning local distributions. In European Conf. on Computer Vision (ECCV), pages 480-496. Springer, 2022.
[24] A. Saxena, Chung, and A. Ng. Learning depth from single monocular images. Conference and Workshop on Neural Information Processing Systems (NeurIPS). 18, 2005.
[25] X. Wang, C. Hou, L. Pu, and Y. Hou. A depth estimating method from single image using foe crf. Multimedia Tools and Applications (MTA), 74(21):9491-9506 2015.
[26] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, N. Gomez, L. Kaiser, and I. Polo- sukhin. Attention is all you need. In Conference and Workshop on Neural Information Pro- cessing Systems (NeurIPS), pages 6000-6016 2017.
[27] J. Hu, Y. Zhang, and T. Okatani. Visualization of convolutional neural networks for monocular depth estimation. In JEEE Int. Conf. on Computer Vision (ICCV), pages 3869-3878, 2019.
[28) I. Laina, C. Rupprecht, V. Belagiannis, F. Tombari, and N. Navab. Deeper depth prediction with fully convolutional residual networks. In Int. Conf. 3D Vision (3DV), pages 239-248, 2016.
[29] Y. Cao, Z. Wu, and C. Shen. Estimating depth from monocular images as classification using deep fully convolutional residual networks. IEEE Transactions on Circuits and Systems for Video Technology, 28(1 133174-3182 2017.
[30] X. Gu, W. Yuan, Z. Dai, C. Tang, S. Zhu, and P. Tan. DRO: Deep recurrent optimizer for cometure-from-motiie arXiv preprint 1rXiv:2103.13200 2021.
[31] Xie, R. Girshick, and A. Farhadi. Deep3d: Fully automatic 2d-to-3d video conversion with deep convolutional neural networks. In European Conf. Computer Vision (ECCV), pages 842-857. Springer, 2016.
10

---

[32] R. Garg, V. K. Bg, G. Carneiro, and I. Reid. Unsupervised enn for single view depth estimation: Geometry to the rescue. In European Conf. on Computer Vision (ECCV), pages 740-756. Springer, 2016.
[33] C. Godard, O. Mac Aodha, and G. Brostow. Unsupervised monocular depth estimation with left-right consistency. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR),
pages 270-279, 2017.
[34] F. Tosi, F. Aleotti, M. Poggi, and S. Mattoccia. Learning monocular depth estimation infusing traditional stereo knowledge. In IEEE Conf. on Computer Vision and Patlern Recognition (CVPR), pages 9799-9809, 2019.
[35] H. Choi, H. Lee, S. Kim, S. Kim, S. Kim, K. Sohn, and D. Min. Adaptive confidence thresh- olding for monocular depth estimation. In IEEE Int. Conf. on Computer Vision (ICCV), pages 12808-12818, 2021.
[36] Z. Chen, X. Ye, W. Yang, Z. Xu, X. Tan, Z. Zou, E. Ding, X. Zhang, and L. Huang. Revealing the reciprocal relations between self-supervised stereo and monocular depth estimation. In IEEE Int. Conf. on Computer Vision (ICCV), pages 15529-15538, 2021.
[37] T. Zhou, M. Brown, N. Snavely, and D. G. Lowe. Unsupervised learning of depth and ego- motion from video. In IEEE Conf. on Computer Vision and Putlern Recognition (CVPR), pages 1851-1858, 2017.
[38] C. Godard, O. Mac Aodha, M. Firman, and G. Brostow. Digging into self-supervised monoc- ular depth estimation. In IEEE Int. Conf. on Computer Vision (ICCV), pages 3828-3838, 2019.
[39] J. Watson, O. Mac Aodha, V. Priisacariu,, G. Brostow, and M. Firman. The temporal opportunist: Self Self-supervised multi-frame monocular depth. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 1164-1173 174, 2021.
[40] G. Hinton, O. Vinyals, J. Dean, et al. Distilling the knowledge in neural network. Conference and Workshop on Neural Information Processing Systems (NeurIPS), 2015.
[41] P. Chen, S. Liu, H. Zhao, and J. Jia. Distilling knowledge via knowledge review. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 5008-5017, 2021.
[42] Z. Chong, X. Ma, H. Zhang, Y. Yue, H. Li, Z. Wang, and W. Ouyang. Monodistill: Learning spatial features for monocular 3d object detection. In Int. Conf. Learning Representations (ICLR), 2022.
[43] T. Wang, L. Yuan, X. Zhang, and J. Feng. Distilling object detectors with fine- ahe-ggriindd fea- ture imitation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4933 2019.
[44] Y. Liu, K. Chen, C. Liu, Z. Qin, Z. Luo, and Wang. Structured knowledge distillation for semantic segmentation. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 2604-2613, 2019.
[45] X. Guo, H. Li, S. Yi, J. Ren, and X. Wang. Learning monocular depth by distilling cross- domain stereo networks. In European Conf. Computer Vision (ECCV), pages 484-500, 2018.
[46] A. Pilzer, S. Lathuiliere, N. Sebe. and E. Ricci. Refine and distill: Exploiting cycle- inconsistency and knowledge distillation for unsupervised monocular depth estimation. In IEEE Conf. Computer Vision and Puttern Recognition (CVPR), pages 9768-9777, 2019,
[47] Y. Wang, X. Li, M. Shi, K. Xian, and Z. Cao. Knowledge distillation for fast and accurate monocular depth estimation mobile devices. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 2457-2465, 2021.

---

[48] C.-Y. Wu, J. Wang, M. Hall, U. Neumann, and S. Su. Toward practical monocular indoor depth estimation. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 2814-3824, 2022.
[49] A. Kendall and Y. Gal. What uncertainties do we need in bayesian deep learning for computer vision? In Conference and Workshop on Neural Information Processing Systems (NeurIPS), volume 30, 2017.
[50] B. Lakshminarayanan, A. Pritzel, and C. Blundell. Simple and scalable predictive uncer- tainty estimation using deep ensembles. Advances in neural information processing systems (NeurIPS), 30, 2017.
[51] Y. Gal and Z. Ghahramani. Dropout as bayesian approximation: Insights and applications. In Int. Conf. on Machine Learning (ICML), volume page 2015.
[52] Y. Shen, Z. Zhang, M. R. Sabuncu, and L. Sun. Real-time uncertainty estimation in computer vision via uncertainty aware distribution distillation. In IEEE Winter Conf. on Applications of Computer Vision (WACV), pages 707-716, 2021.
[53] J. Hornauer and Belagiannis. Gradient-based uncertainty for monocular depth estimation. In European Conf. on Computer Vision (ECCV), 2022.
[54] N. Yang, L. Stumberg, Wang, and D. Cremers. D3vo: Deep depth, deep pose and deep uncertainty for monocular visual odometry. In IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pages 1281-1292, 2020.
[55] X. Nie, D. Shi, R. Li, Z. Liu, and X. Chen. Uncertainty-aware self-improving framework for depth estimation. IEEE Robotics and Automation Letters (RA-L), 7(1):11-4,, 2021.
[56] M. Poggi, F. Aleotti, F. Tosi, and Mattoccia. On the uncertainty of self-supervised monocular depth estimation. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 3227-3231 2020.
[57] 7. Yang, W. Dong, X. Li, J. Wu, I., Li, and G. Shi. Self-feature distillation with uncertainty modeling for degraded image recognition. In European Conf. on Computer Vision (ECCV), 2022.
[58] Z. Liu, Y. Lin, Y. Cao, H. Hu, Wei, Zhang, Lin, and B. Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In IEEE Int. Conf. on Computer Vision (ICCV), pages 10012-10022, 2021.
[59] A. Geiger, P. Lenz, and R. Urtasun. Are we ready for autonomous driving? The KITTI vision benchmark suite. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 3354-3361, 2012.
(60) G. Yang, X. Song, C. Huang, 7. Deng, Shi, B. Zhou. Drivingstereo: A large-scale dataset for stereo matching in autonomous driving scenarios. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 899-908, 2019.
[61] Y. Cabon, N. Murray, and M. Humenberger. Virtual arXiv:2001.10773, 2020.
KITTI 2.
arXiv preprint
[62] T.-Y. Lin, Goyal, R. Girshick, K. He, and P. Dollár. Focal loss for dense object detection. In IEEE Int. Conf. on Computer Vision (ICCV), pages 2980-2988, 2017.
[63] Y. Huang, P. Shen, Y. Tai, S. Li, X. Liu, J. Li, F. Huang, and R. Ji. Improving face recognition from hard samples via distribution distillation loss. In European Conf. Computer Vision (ECCV), pages 138-154, Springer, 2020.
12

---

[64] D. Xu, W. Wang, H. Tang, H. Liu, N. Sebe, and E. Ricci. Structured attention guided convo- lutional neural fields for monocular depth estimation. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 3917-3925, 2018.
[65] W. Yin, Y. Liu, C. Shen, and Y. Yan. Enforcing geometric constraints of virtual normal for depth prediction. In JEEE Int. Conf. on Computer Vision (ICCV), pages 5684-5693, 2019.
[66] V. Guizilini, R. Ambrus, W. Burgard, and A. Gaidon. Sparse auxiliary networks for unified monocular depth prediction and completion. In IEEE Conf. on Computer Vision and Patlern Recognition (CVPR), pages 11078-10 1088, 2021.
[67] Z. Liu, Li, Shao, X. Wu, and W. Chen. Self-supervised monocular depth estimation with self-reference distillation and disparity offset refinement. IEEE Transactions on Circuits and Systems for Video Technology, 2023.
[68] J. Uhrig, N. Schneider, L. Schneider, U. Franke, Brox, and A. Geiger. Sparsity invariant cnns. 2017 international conference on 3D Vision (3DV), pages 11-20, IEEE, 2017.
13


Reviews:
[{"review_id": "lwDC432t2l", "reviewer": "Reviewer_v8s2", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper introduces ADU-Depth, a knowledge distillation framework for enhancing monocular depth estimation in robotics applications. The main idea is to leverage knowledge distillation to transfer 3D-aware knowledge from a teacher network to a monocular student. The teacher network learns from left-right view image pairs, and this knowledge is incorporated into the student's encoder-decoder architecture. Additionally, ADU-Depth introduces an attention adaptation module, a distillation-score based focal-depth-loss, and an uncertainty estimation module to guide the distillation process effectively and achieve more high-quality responses. The proposed method achieves state-of-the-art performance on the KITTI and DrivingStereo datasets, ranking first on the KITTI online benchmark.", "summary_of_recommendation": "Overall, the paper is well-written and tackles a practical problem. However, It appears that the method is a combination of prior ideas towards this problem. Additionally, the experimental details and discussions are insufficient, and there are no robotic experiments and it is questionable whether the proposed method can perform within robotic scenes.  Thus, my rate is weak reject.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "1.\tHow to choose the hyper-parameters in Eq.(9)? Please include the sensitivity analysis for the hyper-parameters.\n2.\tPlease provide the values of the hyper-parameters in Eq.(9) in 4.1 Implementation Details.\n3.\tIn Table 1-3, more recent state-of-the-art methods should be compared and discussed:\n[2]  Zhou, Kaichen, et al. \"Devnet: Self-supervised monocular depth learning via density volume construction.\" European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022.\n[3]  Z. Liu, R. Li, S. Shao, X. Wu and W. Chen, \"Self-Supervised Monocular Depth Estimation with Self-Reference Distillation and Disparity Offset Refinement,\" in IEEE Transactions on Circuits and Systems for Video Technology.\n[4]  Ning, Chao, and Hongping Gan. \"Trap Attention: Monocular Depth Estimation With Manual Traps.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.\n4.   Please provide a more detailed explanation about how self-attention adaptation layers work for bridging the data domain gap.\n5.    The technical novelty of the work is limited. The authors should argue and explain it.\n6.    Failure cases should be reported and discussed.\n", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Strengths:\n1.\tThe proposed ADU-Depth framework leverages left-right image pairs to transfer 3D geometry-aware knowledge to a monocular student network. The motivation is sound.\n2.\tThe paper is well-written and easy to follow. The proposed framework is described in detail.\n3.\tThe evaluation of three standard benchmarks is convincing.\n\nWeaknesses:\n1.\tThe technical novelty is relatively low. The proposed framework builds upon existing techniques and has no technically novel components. For example, the use of left-right image pairs for 3D-aware knowledge transfer has been explored in previous works. Similarly, distillation techniques have been used in previous works for monocular depth estimation. Essentially, it appears that the proposed method applies existing works to improve performance. \n2.\tThere are some more recent related works not mentioned:\n[1] Wimbauer, Felix, et al. \"Behind the Scenes: Density Fields for Single View Reconstruction.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.\n[2]  Zhou, Kaichen, et al. \"Devnet: Self-supervised monocular depth learning via density volume construction.\" European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022.\n[3]  Z. Liu, R. Li, S. Shao, X. Wu and W. Chen, \"Self-Supervised Monocular Depth Estimation with Self-Reference Distillation and Disparity Offset Refinement,\" in IEEE Transactions on Circuits and Systems for Video Technology.\n[4]  Ning, Chao, and Hongping Gan. \"Trap Attention: Monocular Depth Estimation With Manual Traps.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.\n[5] Li, Zhi, et al. \"Test-time Domain Adaptation for Monocular Depth Estimation.\" 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023.\n3.\tMore discussion on the failure cases and limitations would be welcome.\n", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed."}, "extra_text": {}}, {"review_id": "RpOlmDGFJDk", "reviewer": "Reviewer_nhhw", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "In this work, the authors propose a new single image depth estimation model (student model) by training a deep learning model that learns from a stereo image depth estimation network. It suggests a self-attention mechanism to align the features between the student and teacher models. An uncertainty estimation module is applied to reweight the losses during training. The authors have conducted experiments on two public datasets, KITTI and DrivingStereo, and the presented approach achieves 1st place on the KITTI benchmark.", "summary_of_recommendation": "I think the performance of the presented work is strong, and the proposed idea is new. ", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "What is the performance on other datasets? It would be good to know the current zero-shot performance and limitations. Especially some datasets in the following work:\nRanftl et al. Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer, TPAMI 2022\n\nWhat values are λ1, λ2, λ3 in the implementation?\n", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Strengths:\n+ The paper is clearly written, and it is easy to follow.\n+ The experimental results are compelling.\n+ The idea of using a stereo image depth network as the teacher to supervise a single image depth image estimation model is new, although it is not that surprising.\n\nWeaknesses:\n+ Although the idea of using a pretrained model to supervise single image depth estimation is new, it is essentially another strategy of knowledge distillation. Knowledge distillation has been used in several prior works, as shown in Table 1. It is hard to say the overall idea of this work is very novel.\n+ One weakness is the need for stereo images to train the student network because the teacher network requires stereo images. \n+ It would be good to evaluate the presented approach on other datasets to see the zero-shot performance. ", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed."}, "extra_text": {}}, {"review_id": "ICStkvu69B", "reviewer": "Reviewer_d9wF", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "  This paper introduces a novel knowledge distillation framework called ADU-Depth for boosting precise depth estimation in monocular images. The framework leverages left-right image pairs to transfer 3D geometry-aware knowledge to a monocular student network. It includes attention-adapted feature distillation, focal-depth-adapted response distillation, and uncertainty modeling to guide effective and smooth knowledge transfer. The paper also presents extensive experiments and an ablation study to verify the effectiveness of each module in the proposed distillation strategy. The results show that ADU-Depth outperforms other state-of-the-art methods in terms of accuracy and robustness.\n\n\n \n", "summary_of_recommendation": "  This paper introduces a novel knowledge distillation framework called ADU-Depth for boosting precise depth estimation in monocular images. The framework leverages left-right image pairs to transfer 3D geometry-aware knowledge to a monocular student network. It includes attention-adapted feature distillation, focal-depth-adapted response distillation, and uncertainty modeling to guide effective and smooth knowledge transfer. The paper also presents extensive experiments and an ablation study to verify the effectiveness of each module in the proposed distillation strategy. The results show that ADU-Depth outperforms other state-of-the-art methods in terms of accuracy and robustness.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "1. How are the loss functions $L_e$ and $L_d$ (defined in equations (2) and (3) respectively) adapted during the training of the framework? \n2. The authors should provide a more comprehensive analysis of the knowledge distillation schedule, as it stands as a pivotal contribution of this paper. By offering a detailed examination of this aspect, readers can gain deeper insights into the methodology's significance and potential implications for the field.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "  This paper introduces a novel knowledge distillation framework called ADU-Depth for boosting precise depth estimation in monocular images. The framework leverages left-right image pairs to transfer 3D geometry-aware knowledge to a monocular student network. It includes attention-adapted feature distillation, focal-depth-adapted response distillation, and uncertainty modeling to guide effective and smooth knowledge transfer. The paper also presents extensive experiments and an ablation study to verify the effectiveness of each module in the proposed distillation strategy. The results show that ADU-Depth outperforms other state-of-the-art methods in terms of accuracy and robustness.\n\n ", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a novel knowledge distillation framework called ADU-Depth for boosting precise depth estimation in monocular images. The framework leverages left-right image pairs to transfer 3D geometry-aware knowledge to a monocular student network. It includes attention-adapted feature distillation, focal-depth-adapted response distillation, and uncertainty modeling to guide effective and smooth knowledge transfer. The paper also presents extensive experiments and an ablation study to verify the effectiveness of each module in the proposed distillation strategy. The results show that ADU-Depth outperforms other state-of-the-art methods in terms of accuracy and robustness, and the results are promising for real-world robotics applications.

Most reviewers agree that the paper is well-written and presents compelling experimental results. However, reviewers also pointed out the technical novelty of the proposed method is limited. And the failure cases and limitations of the proposed method should be discussed more.

During the rebuttal phase, the reviewer acknowledged that some of the concerns have been addressed.

We recommend accepting the paper as poster. And we ask the authors to explain the technical novelty of their work in more detail, and provide more discussion on the failure cases and limitations of the proposed method in the camera-ready version.
Decision: Accept (Poster)