# KD-PatchMatch: A Self-Supervised Training Learning-Based PatchMatch
## 3 Method
### Knowledge Distillation
#### 3.2.2 Distillation-Based Student Training  
In this section, we train our network using the knowledge distillation concept, which entails transferring the teacher's probabilistic knowledge to the student model. This procedure is divided into two parts: pseudo-probabilistic knowledge generation and student training.  
**Pseudo Probabilistic Knowledge Generation.** Similar to [37], knowledge is transferred using the probability distribution. However, when applying knowledge distillation in the PatchMatch-based MVS network, we discover that the teacher model's true probabilistic knowledge is not directly available for training the student model. The reason for this is that the depth hypotheses in the coarse-to-fine network must be dynamically sampled based on the previous stage's results, i.e., in subsequent iterations of the PatchMatch stage, we need to use the estimates from the previous iteration, possibly sampled from the previous coarse stage, so that the teacher and student models cannot always share the same depth hypotheses. To this end, following [35], we use probabilistic coding to generate pseudo-probability distributions.  
We denote the final depth prediction as \(D_{0}\), i.e., the depth map generated after the refinement, the depth maps of source views as \(\{D_{i}\}_{i=1}^{N-1}\). We cast the 2D point \(p_{0}\) into 3D space for any given pixel coordinate in the reference image to obtain the 3D point with the depth value \(D_{0}(p_{0})\). Then, to obtain the point \(p_{i}\) in the source view, we back project \(P_{0}\) to the \(i\)-th source view. After that, using the estimated depth \(D_{i}(p_{i})\) in the reference image, we cast \(p_{i}\) into the 3D space to obtain the 3D point \(P_{i}\). Finally, the depth value of \(P_{i}\) observed from the reference camera is denoted as \(D_{0}(P_{i})\).  
**Probabilistic Encoding.** We use the \(\{D_{0}(P_{i})\}_{i=0}^{N-1}\) to generate the pseudo probability distribution \(P_{p_{0}}(d)\) for the depth value \(d\) of each pixel \(p_{0}\) in the reference view. By using maximum likelihood estimation (\(MLE\)), we model it as a Gaussian distribution with mean depth value \(\mu_{p_{0}}\) and variance \(\sigma_{p_{0}}^{2}\):  
\[\mu_{p_{0}}=\frac{1}{N}\sum_{i=0}^{N-1}D_{0}(P_{i}) \tag{8}\]  
\[\sigma_{p_{0}}^{2}=\frac{1}{N}\sum_{i=0}^{N-1}(D_{0}(P_{i})-\mu_{p_{0}}) \tag{9}\]  
\(\mu_{p_{0}}\) fuses the depth information of the reference and source images, while \(\sigma_{p_{0}}^{2}\) reflects the uncertainty of the teacher model's depth value at pixel \(p_{0}\). During the distillation training, it will provide probabilistic knowledge to the student model.  
**Student Training.** With the above pseudo probability distribution \(P\), by making the projected probability distribution of a student model, \(P\), resemble the pseudo-probability distribution, \(\hat{P}\), one can train the model from scratch. For the discrete depth hypothesis \(\{d_{j}\}_{j=1}^{D_{j}}\), we compute their pseudo-probabilities \(\left\{P(d_{j})\right\}_{j=1}^{D_{j}}\) over the continuous probability distribution \(P\) and normalize \(\left\{P(d_{j})\right\}_{j=1}^{D_{j}}\) with SoftMax, resulting in the final discrete pseudo-probability value. The Kullback-Leibler divergence is used to calculate the difference between the student model's predicted and pseudo-probabilities. The distillation loss \(L_{D_{f}}\) is defined as  
\[L_{D_{f}}=L_{KL}(P\|\hat{P})=\sum_{p\in\{p_{0}\}}\left(P_{p}-\hat{P}_{p}\right) \log\left(\frac{P_{p}}{\hat{P}_{p}}\right) \tag{10}\]  
where \(\{p_{0}\}\) represents the pixel in the reference image \(I_{0}\).