# Non-Local Means Hole Repair Algorithm Based on Adaptive Block
## 1 Introduction  
In recent years, depth cameras have been widely utilized in various fields, such as autonomous driving [1; 2; 3], robot grasping [4; 5; 6], and simultaneous localization and mapping (SLAM) [7; 8; 9]. However, due to object material, specular reflection, or occlusion, the captured depth data often exhibits holes at the edges where objects come into contact with the background. Additionally, limitations in-depth sensor accuracy and range, as well as the influence of optical noise, sometimes result in the reception of ineffective reflection information, thereby degrading the quality of depth images. Considerable research efforts have been devoted by researchers both domestically and internationally to effectively fill the holes in-depth images and obtain high-quality depth images. [PERSON] et al. [10] estimated the boundaries using color images and converged from the edges to the target boundary, effectively addressing holes between edges and multiple edge pixels to achieve seamless inpainting of object boundaries near the camera through an iterative process of alternating filling directions. [PERSON] et al. [11] extended [PERSON]'s inpainting approach [12] by incorporating a block matching algorithm for identifying irregularly shaped voids and a hybrid matching algorithm for selecting optimal repair patches with minimal differences in hole regions, leading to successful restoration,particularly for larger irregular holes. [PERSON] et al. [13] introduced an object-oriented segmentation method to segment different objects in color images, locate source regions for repair patches, and utilize a search algorithm to match missing depth information regions with suitable objects, ultimately filling depth information gaps through optimal repair patch selection within the segmented objects. [PERSON] et al. [14] introduced the statistical nearest neighbors distance measurement (SNNDM) into the NLM filter (NLM-SNN) to measure the distance between two patches, thereby enhancing perceived image quality. [PERSON] et al. [15] introduced a fixing algorithm for the Kinect depth image based on non-local means (NLM), utilizing weights calculated on the corresponding grayscale image through a distance factor and a value-consistency factor to fill holes in the depth image. These methodologies collectively leverage diverse strategies in hole repair techniques to effectively address challenges in in-depth image repair. However, the methods proposed in [10,11] are somewhat complex. The ultimate repair outcome in [13] is influenced by the effectiveness of image segmentation. Additionally, the method in [14] combines the statistical nearest neighbor distance measurement so that it is more complex. In contrast, the approach proposed by [15], inspired by [14], integrates distance factors to enhance restoration efficiency for Kinect cameras but encounters challenges in intricate parameter adjustments. Drawing inspiration from [15], this paper exploits the structural similarity between grayscale and depth images. It utilizes joint grayscale images to estimate depth values within hole regions and applies the non-local means algorithm [14; 15; 16; 17; 18; 19; 20; 21] for effective hole repair. Successful restoration is achieved through the application of structural blocks and NLM-based hole filling. However, in the deep image hole repair algorithm based on the NLM, a requisite task involves the a priori determination of dimensions for both the repair and search blocks. If the dimensions are excessively large, they will significantly increase the computational runtime of the algorithm. Conversely, if the sizes are overly small, they may potentially compromise the quality of the inpainting outcomes. Consequently, a natural question arises: is there a method that can automatically determine the optimal sizes for the search and inpainting blocks of varying dimensions in deep missing regions while ensuring exceptional performance in inpainting quality? It is the motivation for this paper.