# LUMDE: Light-Weight Unsupervised Monocular Depth Estimation via Knowledge Distillation
## 4 Experiments
### Ablation Study  
With SC-SfMLearner as the teacher network and MobileNet V3 (small) as the optimized backbone of the student network, we first did an ablation study on the losses of baseline, i.e., pixel-wise loss, pair-wise loss, and holistic loss. The results are shown in Table 2. It was found that the combination of the three loss functions can generate the best performance, which is consistent with the original paper [15]. Therefore, we regard the combination of three losses as baseline and explore potential improvements on this. We also did an ablation study on the loss functions of the PoseNet, i.e., \(L_{p}\), \(L_{p}^{M}\), and \(L_{GC}\). This experiment was performed in SC-SfMLearner. We did not perform an ablation assessment of the smoothness loss because many previous works [9; 10; 22] have validated the effectiveness of this approach and used it by default without ablation. The results are shown in Table 3. We find that the combination of \(L_{p}^{M}\) and \(L_{GC}\) is a better choice. Therefore, the following ablation study of the PoseNet module was performed upon the combination of \(L_{p}^{M}\), \(L_{GC}\), and \(L_{s}\). Moreover, we further performed an ablation study to explore potential improvements on the baseline. In the first ablation study, the PoseNet was added to the baseline to assist the convergence of the student depth network. In the second ablation study, Transformer \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Losses**} & \multicolumn{3}{c}{**Error \(\downarrow\)**} & \multicolumn{3}{c}{**Accuracy \(\uparrow\)**} \\ \cline{2-7}  & **AbsRel** & **Log10** & **RMS** & \(\delta 1\) & \(\delta 2\) & \(\delta 3\) \\ \hline L(pa) & 0.333 & 0.128 & 1.007 & 0.506 & 0.777 & 0.903 \\ L(pi) & 0.164 & 0.069 & 0.583 & **0.769** & **0.941** & **0.983** \\ L(ho) & 0.306 & 0.121 & 0.955 & 0.525 & 0.799 & 0.918 \\ L(pa) + L(pi) & **0.163** & 0.069 & 0.583 & **0.769** & **0.941** & **0.983** \\ L(pa) + L(ho) & 0.321 & 0.123 & 0.978 & 0.522 & 0.789 & 0.909 \\ L(pi) + L(lo) & **0.163** & 0.069 & 0.582 & 0.768 & **0.941** & **0.983** \\ L(pa) + L(pi) + L(ho) & **0.163** & **0.068** & **0.581** & **0.769** & 0.940 & **0.983** \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Ablation study of baseline. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Losses**} & \multicolumn{3}{c}{**Error \(\downarrow\)**} & \multicolumn{3}{c}{**Accuracy \(\uparrow\)**} \\ \cline{2-7}  & **AbsRel** & **Log10** & **RMS** & \(\delta 1\) & \(\delta 2\) & \(\delta 3\) \\ \hline L(P) & 0.158 & 0.073 & 0.569 & 0.778 & 0.940 & 0.983 \\ L(PM) & 0.155 & 0.071 & 0.562 & 0.791 & 0.943 & 0.983 \\ L(GC) & 0.319 & 0.118 & 0.938 & 0.539 & 0.808 & 0.919 \\ L(P) + L(GC) & 0.155 & 0.065 & 0.559 & 0.796 & 0.943 & 0.982 \\ L(PM) + L(GC) & **0.148** & **0.062** & **0.545** & **0.803** & **0.948** & **0.985** \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Ablation study of PoseNet losses. was incorporated into the baseline to capture global image information. In the third ablation study, both PoseNet and Transformer are added into the framework to incorporate the advantages of both modules. The results are shown in Table 4. It can be seen that the integration of PoseNet and Transformer can improve the performance of the student depth network. For baseline, the indicators increase by 10.1%, 9.7%, and 6.6% on AbsRel, Log10, and RMS errors, and decrease by 4.2%, 0.8%, and 0.2% on \(\delta\)1, \(\delta\)2, and \(\delta\)3 accuracies compared with the performance of the teacher network. For LUMDE, the indicators increase by 6.7%, 8.1%, and 3.7% on AbsRel, Log10, and RMS errors, and decrease by 2.6%, 0.5%, and 0.1% on \(\delta\)1, \(\delta\)2, and \(\delta\)3 accuracies. It is also useful to evaluate the significance of knowledge distillation in this study. This can be achieved by investigating whether a lightweight depth model can be acquired by simply replacing the backbone of the Teacher Network (SC-SfMLearner) with the lightweight backbone used in this study.