# KD-PatchMatch: A Self-Supervised Training Learning-Based PatchMatch
## 3 Method
### Knowledge Distillation
#### 3.2.1 Teacher Training  
The sensitivity of photometric loss to shooting angles and lighting conditions leads to poorer completeness of model predictions. To this end, in addition to the traditional photometric consistency used in our network, following [35], we propose to add the internal feature metric consistency loss to the original loss as additional supervised signals, which are obtained by computing the luminosity loss between the reference and source views. The view reconstruction and loss functions are described below.  
**View Reconstruction.** For a pixel \(p_{0}\) in the reference image \(I_{0}\), given the depth value of the \(j\)-th sample, \(d_{p_{0}}^{j}\), its corresponding pixel \(p_{i}(d_{p_{0}}^{j})\) in the source image \(I_{i}\) is calculated as  
\[p_{i}\Big{(}d_{p_{0}}^{j}\Big{)}=K_{i}\Big{(}R_{0,i}\cdot\Big{(}K_{0}^{-1}\cdot p _{0}\cdot d_{p_{0}}^{j}\Big{)}+t_{0,i}\Big{)} \tag{5}\]  
where \(R_{0,i}\) and \(t_{0,i}\) are the relative rotation and translation of the reference image \(I_{0}\), and the source images \(I_{i}\), \(k_{0}\), and \(k_{i}\) are the intrinsic matrices of the reference image \(I_{0}\) and the source image \(I_{i}\). With the above transformation, the warped feature maps of the source view under the \(j\)-th set of (per pixel different) depth hypotheses, \(F_{i}(p_{i}(d_{p_{0}}^{j}))\), can be obtained by differentiable bilinear interpolation.  
**Loss Function.** The training loss consists of two components: photometric loss \(L_{ph}\) and feature-metric loss \(L_{fm}\). Relying on photometric loss based on the \(L_{1}\) distance between the original RGB reference image and the reconstructed image, previous work can result in large systematic errors in realistic scenes due to reflective surfaces and occlusions. To end this, following [36], we add the feature-metric loss to the photometric loss, i.e., adding the calculation of the photometric loss between feature maps. Feature-metric loss is defined as  
\[L_{fm}=\left|F_{i}(\hat{p})-F_{0}(p)\right|_{1} \tag{6}\]  
where \(F_{i}(\hat{p})\) and \(F_{0}(p)\) represent the reference view's feature map and the reconstructed feature map from the \(i\)-th source view, respectively.  
In the refinement stage, we upsample (from resolution \(\frac{W}{2^{k}}\times\frac{H}{2^{k}}\) to \(W\times H\)) the finest resolution level (Stage 0) through the designed depth residual network and obtain the photometric loss compared to the ground truth, i.e., \(L_{ref}^{0}\).  
Figure 2: Comparison of uniform and proposed probabilistic sampling methods. The single-view depth probability distribution is represented by the blue curve, and the sampled candidates are represented by the red dots.  
To summarize, the final loss function includes all the depth estimation, and the refinement stage is defined as  
\[L_{\text{total}} =\sum_{k=1}^{3}\sum_{m=1}^{n_{k}}\left(\lambda_{fm}L_{fm}^{m,k}+ \lambda_{ph}L_{ph}^{m,k}\right)+L_{ref}^{0} \tag{7}\]  
where \(L_{fm}^{m,k}\) and \(L_{ph}^{m,k}\) are the feature-metric loss and photometric loss of the \(m\)-th iteration of PatchMatch at Stage \(k(k=1,2,3)\), and \(L_{ref}^{0}\) is the loss of final depth map refinement.