# Non-Local Means Hole Repair Algorithm Based on Adaptive Block
## 2 Depth Image Inpainting Algorithm
### Algorithm Improvement  
In response to the challenges present in the NLM hole repair algorithm, this section undertakes its enhancement and introduces a novel approach termed the adaptive block-based NLM hole repair algorithm. The detailed methodology is delineated as follows: The principle of the NLM hole repair algorithm based on adaptive blocks is shown in Figure 3.  
Step 1: Aligned depth images and grayscale images captured from the same viewpoint using an RGB-D camera facilitate the direct extraction of texture information from grayscale images, enabling algorithms to perform depth image restoration.  
Step 2: The coordinates of each hole point are recorded, and the computation of connected components within the hole regions is performed. The initial value for the search block \(A(x)\) is predetermined as \(m_{1}\times m_{1}\), where \(m_{1}\) is the larger value between the maximum row width and the maximum column width of the respective hole block, incremented by three. The value of \(m_{1}\) is defined as below:  
\[m_{1}=\max\{(X_{r}-X_{l}),(Y_{t}-Y_{d})\}+3 \tag{4}\]  
Figure 3: Framework of an Adaptive Block-based Non-Local Means Hole Repair Algorithm.  
where \(X_{r}\) and \(X_{l}\) represent the maximum and minimum values of the column indices of the hole points within the hole block, respectively. \(Y_{l}\) and \(Y_{d}\) represent the maximum and minimum values of the row indices of the hole points within the hole block, respectively.  
Step 3: An intelligent block factor \(T\) is introduced. The factor \(T\) dynamically adjusts the initial value of the search block \(A(x)\) based on the proportion of non-hole points within the hole block. Through \(T\), a large-size search and repair block is used to repair a large area of hole block. Conversely, for small-area hole blocks, small-sized search and repair blocks are utilized. This dynamic adjustment enhances the generality of our algorithm, enabling it to adapt to various hole sizes and image features. The intelligent block factor \(T\) is defined as  
\[T=\frac{\sum_{k\in H(p)}I_{k}}{|H(p)|} \tag{5}\]  
where \(|H(p)|\) represents the sum of the total number of pixels in the image and \(I_{k}\) denotes the non-hole points within the target block.  
Step 4: The size of the repair block \(A(x)\) is determined as follows: We calculate the proportion \(T_{1}\) of non-zero pixels within the \(m_{1}\times m_{1}\) neighborhood of the hole point. If \(T_{1}\) is greater than the given threshold \(P\) for the proportion of non-hole points in \(A(x)\), then the final search block size is set as \(p=m_{1}\). Otherwise, increase \(m_{1}\) by 2 (\(m_{1}=m_{1}+2\)), and reevaluate the relationship between \(T_{1}\) and \(P\).  
Step 5: The size of the repair block \(B(x)\) is determined as follows: The initial value of the repair block is set as \(m_{2}\times m_{2}\), where \(m_{2}=m_{1}-3\). The proportion \(T_{2}\) of non-zero pixel points within the \(m_{2}\times m_{2}\) neighborhood of the hole points is calculated. Should \(T_{2}\) surpass \(Q\) (where \(Q\) represents the threshold for the proportion of non-vacant hole points within \(B(x)\)), the ultimate determination for the size of the rectification block is \(q=m_{2}/2\). Otherwise, \(m_{2}\) is incremented by 2 (\(m_{2}=m_{2}+2\)), and the relationship between \(T_{2}\) and \(Q\) is reevaluated.  
Step 6: For each hole point \(x\), calculate the repair weight \(r(x,y)\) for each non-hole point \(y\) in the neighborhood \(A(x)\) of \(x\). Utilize the repair weights to restore the hole point. By iterating through all hole points in the depth image and performing the same operation, the final restoration of the depth image is achieved.