# LUMDE: Light-Weight Unsupervised Monocular Depth Estimation via Knowledge Distillation
## 4 Experiments
### Ablation Study
Therefore, we replaced the depth network backbone of SC-SfMLearner with MobileNet V3 (small) and trained the framework with the traditional self-supervised training method. The results are shown in Table 5. Although utilizing the same pose network and training details as with SC-SfMLearner, the performance of the teacher network with a lightweight backbone is significantly worse than LUMDE. This evaluation validates the effectiveness of knowledge distillation in the current study. The qualitative depth maps are shown in Figure 2. It can be seen that the performance of the teacher network is closest to the ground truth, which is consistent with the quantitative results in Table 4. In addition, compared with the baseline depth graphs, our optimized network (LUMDE) presents better prediction capacity on the outlines of objects, and the baseline results are somewhat hazy in some areas. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multicolumn{4}{c}{**Error \(\downarrow\)**} & \multicolumn{4}{c}{**Accuracy \(\uparrow\)**} \\ \cline{2-7}  & **AbsRel** & **Log10** & **RMS** & \(\delta\)**1** & \(\delta\)**2** & \(\delta\)**3** \\ \hline Teacher Network & 0.148 & 0.062 & 0.545 & 0.803 & 0.948 & 0.985 \\ \hline Student Network (Baseline) & 0.163 & 0.068 & 0.581 & 0.769 & 0.940 & 0.983 \\ Performance & \(\uparrow\) 10.1\% & \(\uparrow\) 9.7\% & \(\uparrow\) 6.6\% & \(\downarrow\) 4.2\% & \(\downarrow\) 0.8\% & \(\downarrow\) 0.2\% \\ \hline Baseline + PoseNet & 0.160 & 0.067 & 0.575 & 0.775 & 0.942 & 0.984 \\ \hline Baseline + Transformer & 0.161 & 0.068 & 0.577 & 0.773 & 0.942 & 0.984 \\ \hline Baseline + PoseNet + Transformer (LUMDE) & 0.158 & 0.067 & 0.565 & 0.782 & 0.943 & 0.984 \\ Performance & \(\uparrow\) 6.7\% & \(\uparrow\) 8.1\% & \(\uparrow\) 3.7\% & \(\downarrow\) 2.6\% & \(\downarrow\) 0.5\% & \(\downarrow\) 0.1\% \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Ablation study of different model combinations. \begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multicolumn{3}{c}{**Error \(\downarrow\)**} \\ \cline{2-4}  & **AbsRel** & **Log10** & **RMS** \\ \hline Teacher Network & 0.148 & 0.062 & 0.545 \\ LUMDE & 0.158 & 0.067 & 0.565 \\ Student Network w/o KD & 0.185 & 0.076 & 0.635 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Comparison between LUMDE and student network without KD. In addition, we compared our work with previous unsupervised approaches in Table 6. It can be seen in Table 6 that our model achieves SOTA results in unsupervised depth distillation with a quite low parameter size. The accuracy was also significantly increased compared with [PERSON] et al. [14]. In addition, the performance of LUMDE is quite competitive compared with a dense unsupervised network. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multirow{2}{*}{**Type**} & \multirow{2}{*}{**Method**} & \multicolumn{3}{c}{**Error \(\downarrow\)**} & \multicolumn{3}{c}{**Accuracy \(\uparrow\)**} & \multirow{2}{*}{**Model Size**} \\ \cline{3-4} \cline{6-9}  & & **AbsRel** & **Log10** & **RMS** & \(\delta\)**1** & \(\delta\)**2** & \(\delta\)**3** & **(Depth)** \\ \hline \multirow{4}{*}{w/o KD} & [PERSON] et al. [54] & 0.189 & 0.079 & 0.686 & 0.701 & 0.912 & 0.978 & - \\  & [PERSON] et al. [9] & 0.160 & - & 0.601 & 0.767 & 0.949 & 0.988 & **14.84M** \\ \cline{1-1}  & [PERSON] et al. [46] & 0.147 & **0.062** & 0.536 & 0.804 & 0.950 & 0.986 & **14.84M** \\ \cline{1-1}  & [PERSON] et al. [22] & **0.134** & - & **0.526** & **0.823** & **0.958** & **0.989** & - \\ \hline \multirow{2}{*}{w/KD} & [PERSON] et al. [14] & 0.175 & **0.065** & 0.673 & **0.783** & 0.920 & **0.984** & - \\  & LUMDE & **0.158** & 0.067 & **0.565** & 0.782 & **0.943** & **0.984** & **3.38M** \\ \hline \hline \end{tabular} Note: w/o KD means without using knowledge distillation; w/KD means utilizing knowledge distillation.  
\end{table} 
Table 6: Comparison with previous unsupervised work.