# Structure-Guided Image Inpainting Based on Multi-Scale Attention Pyramid Network
## 2 Related Work  
After decades of development, image inpainting can be generally categorized into two main classes: traditional inpainting methods and deep learning-based inpainting methods. The former primarily consists of patch-based and diffusion-based methods. Among the latter, inpainting techniques utilizing adversarial networks have become the most popular. Patch-based methods fill the missing areas of a degraded image by searching within its known regions for patches that could closely resemble the missing area. Initially dominated by techniques such as the region filling approach proposed by [PERSON] [14], these methods demonstrated outstanding results at the time. However, the search for similar patches was time-consuming and computationally intensive. To address this, [PERSON] et al. [15] introduced the PatchMatch method, which employs a randomized nearest-neighbor matching scheme to significantly reduce the computational load and inpainting time. Diffusion-based methods employ mathematical techniques to gradually spread pixel values from the surrounding areas into the missing regions, following specific rules. [PERSON] et al. [16] were pioneers in applying diffusion techniques in image inpainting, adopting a rule that fills inwards from the outer boundary, mimicking the manual restoration of artworks. [PERSON] and [PERSON] [17] later proposed a curvature-driven diffusion rule, which overcame the limitations of previous methods that were only effective in restoring densely textured images, enabling better transmission of edge information and inpainting of non-textured images. While traditional inpainting methods achieved satisfactory results on small missing areas and simple texture-style images, they fundamentally lacked an understanding of semantic information and the ability to generate new content. Thus, they were often inadequate for large missing areas, resulting in the creation of pixel blocks instead of meaningful content. With the advent of deep learning in computer vision, traditional inpainting methods began to fade. [PERSON] et al. [22] were among the first to apply deep learning to image inpainting, introducing an encoder-decoder architecture that transforms damaged images into complete ones. However, their method's adversarial loss was roughly applied to the missing areas, leading to a stark contrast between the generated content and the existing areas, and low consistency. To improve this, [PERSON] et al. [35] proposed the concept of global and local discriminators to enhance the consistency between the overall image and the content generated for the missing areas. This dual-discriminator mechanism, however, was only effective for regularly shaped missing areas and performed poorly on irregular ones. [PERSON] et al. introduced partial convolution [26], which uses an automatic mask update mechanism to reduce the interference of invalid information during inpainting. Yet, this method becomes ineffective in deeper networks as the mask gradually diminishes. These approaches commonly used a single view for image inpainting, which, while easy to implement, often lacked sufficient constraints and reference information, leading to poor performance in complex scenes. To counter this, researchers proposed multi-view inpainting, which leverages features of the missing image from multiple perspectives to achieve comprehensive repair. Methods like EdgeConnect by [PERSON] et al. [36] use edge maps extracted using the Canny edge detector as an additional view, providing more context and structural guidance for the inpainting process. Despite the excellence of the Canny operator in edge detection, its robustness against complex structures or noise can be insufficient, potentially compromising the subsequent inpainting. The E2I (edge-to-image) method introduced by [PERSON] et al. [37], which employs the holistically nested edge detection (HED) technique, brought unique advantages to image inpainting.