# LUMDE: Light-Weight Unsupervised Monocular Depth Estimation via Knowledge Distillation
## 3 Method
### Pose-Assisted Network  
There is little research on unsupervised depth model distillation. Inspired by the previous unsupervised depth estimation networks, we integrate a pose network [8] into the student network to improve the performance of the baseline network. This module can be trained to predict the relative pose between consecutive frames and the pose information can be utilized to construct the photometric reprojection error. In our framework, it is used to assist the convergence of the student depth network and is only employed in the training stage. This module needs two consecutive frames as input, which is different from the above distillation part, where a single color image is enough. We address this problem by loading 12 images per batch, which means the batch size is 12 for the distillation part, but half for the pose module.  
Just as in [8; 9], \(T_{t\to t^{\prime}}\) is defined as the relative pose for source image \(I_{t^{\prime}}\), with respect to target image \(I_{t}\). The reprojected image from \(I_{t^{\prime}}\) can be written as follows:  
\[I_{t^{\prime}\to t}=I_{t^{\prime}}\langle proj(D_{t},T_{t\to t^{\prime}},K)\rangle \tag{6}\]  
where \(D_{t}\) is the depth map of \(I_{t}\), \(K\) is the intrinsic parameters of the camera lens, \(proj()\) denotes the resulting 2D coordinates of projected \(D_{t}\) in \(I_{t^{\prime}}\), and the angle bracket represents the sampling operator for the sake of aligning image size. With the target image \(I_{t}\) and synthesized image \(I_{t^{\prime}\to t}\), the photometric reprojection loss can be calculated as follows:  
\[L_{p}=\frac{1}{V}\sum_{p\in V}\left(\frac{\alpha}{2}(1-SSIM(I_{t}(p),I_{t^{ \prime}\to t}(p)))+(1-\alpha)\big{\|}I_{t}(p)-I_{t^{\prime}\to t}(p)\big{\|}_{1}\right) \tag{7}\]  
where \(\alpha=0.85\), and \(L_{p}\) is in the form of an L1 norm that is robust to outliers, SSIM [45] estimates the pixel-wise similarity, and \(V\) represents the valid pixels that are reprojected from \(I_{t^{\prime}}\) to \(I_{t}\) plane.  
In addition, we can add another constraint between the two consecutive frames. Because of the widespread geometry inconsistency in depth estimation, we enforce the consistency of the consecutive depth maps [10] by minimizing \(L_{\text{GC}}\). According to the pose information from the pose net, we can warp the source image depth to the target image plane, named \(D_{t^{\prime}\to t}\). If we define depth inconsistency for each pixel \(p\) as:  
\[D_{diff}(p)=\frac{|D_{t}(p)-D_{t^{\prime}\to t}(p)|}{D_{t}(p)+D_{t^{ \prime}\to t}(p)} \tag{8}\]  
Then the geometry consistency loss for each depth map can be defined as follows:  
\[L_{\text{GC}}=\frac{1}{V}\sum_{p\in V}D_{diff}(p) \tag{9}\]  
We use the sum of corresponding depths to normalize the depth inconsistency, thus avoiding the discrepancy in distribution. For pixels of dynamic objects and occlusions,\(D_{diff}(p)\) shows an unreasonable value. Therefore, we can use a weight mask \(M\) to weigh \(L_{p}\), thereby reducing the side effect from these pixels, i.e.  
\[M=1-D_{diff} \tag{10}\]  
\[L_{p}^{M}=\frac{1}{V}\sum_{p\in V}\left(M(p)\cdot L_{p}(p)\right) \tag{11}\]  
Normally, edge-aware smoothness [20] can ensure that the smoothness is guided by the edge of the images. The smoothness loss is defined as follows:  
\[L_{s}=\sum_{p}\left(e^{-\
abla L_{t}(p)}\cdot\
abla D_{t}(p)\right)^{2} \tag{12}\]  
where \(\
abla\) denotes the first derivative along the \(X\) and \(Y\) directions.