## 3 The Proposed Method
### Pixel-Wise Knowledge Distillation for Depth Completion  
Depth completion, unlike conventional image classification, requires dense predictions at the pixel level. Inspired by [PERSON]'s [PERSON] (1982), this approach matches the probability distribution of classes for each pixel between the student and teacher models, as follows:  
\[L_{kd}=\frac{1}{H\times W}{\sum}_{h=1}^{H}\sum_{w=1}^{W}KL(\sigma(\frac{S_{h,w}^ {s}}{T})||\sigma(\frac{S_{h,w}^{t}}{T})), \tag{6}\]  
where \(\sigma(\frac{S_{h,w}^{s}}{T})\) and \(\sigma(\frac{S_{h,w}^{t}}{T})\) indicate the soft class probabilities at pixel positions (\(h\), \(w\)), generated by the student and teacher models, respectively; \(S\) denotes the associated similarity matrix; \(KL\) refers to the Kullback-Leibler divergence Kullback and Leibler (1959); and \(T\) signifies the temperature parameter.  
The similarity matrix represents the pixel-to-region similarity matrix, quantifying the extent to which individual pixels correspond to predefined regions or clusters within the image. This similarity or dissimilarity can be determined by various factors, including color similarity (such as using color histograms or color distances, as applied in Euclidean distance), texture-based similarity (using texture features), or an integrated approach combining both elements. Within the depth completion task, similarity measures determine how closely the inpainted region aligns with its surrounding known regions, deducing the missing or damaged parts based on the available surrounding information. For a given input, \(x\), the resulting pixel embeddings are denoted as \(F\). To simplify the notation, let \(V\) denote the matrix obtained by concatenating elements along the row dimension. Consequently, \(S\) is formulated as follows:  
\[S=FV^{T}. \tag{7}\]  
We utilize pixel-wise class probability distillation for the depth completion task, as illustrated in Figure 4.  
Our approach enhances depth predictions by utilizing pixel-wise class probability distillation, thereby significantly boosting the accuracy, robustness, and reliability of depth estimates, especially in areas with sparse or uncertain data (Figure 5). This method produces more precise and superior outcomes across diverse applications. Following the preliminary depth estimation or the inpainting of missing depth values, a subsequent pixel-wise class probability distillation process refines these predictions further. By leveraging learned probability distributions, this stage adjusts depth values while incorporating contextual information from adjacent pixels and regions. In de  
Figure 4: Application of our proposed dense-pixel KD technique to depth completion.  
technique facilitates more informed decisions when assigning depth values to pixels in inpainted or incomplete areas. By incorporating probability distributions, the approach effectively handles uncertainty and ambiguity in depth estimates. Moreover, it quantifies uncertainty or confidence level for each pixel's depth prediction--critical for applications demanding reliable depth information. By utilizing intricate pixel depth features, this approach enhances adjustments based on local contextual data and neighboring pixel data. Consequently, the outcome demonstrates improved smoothness and structural consistency. By integrating probability distributions with contextual information for each pixel, the approach refines both the accuracy and dependability of depth predictions. This refined methodology proves highly beneficial for depth completion systems, producing high-quality depth maps for advanced applications across various fields.