# Structure-Guided Image Inpainting Based on Multi-Scale Attention Pyramid Network
## 1 Introduction
Attention mechanisms [32; 33; 34] provide an effective means to ensure the consistency of image inpainting content by focusing on key parts of the image and efficiently utilizing contextual information, significantly improving the quality of image inpainting. However, this approach only guarantees contextual consistency between the generated content and the existing content, failing to ensure internal consistency within the generated content itself, leading to parts of the generated content appearing visually discordant. To achieve visually satisfying repair effects, the internal consistency of the generated content is equally crucial. Current popular image inpainting methods often emphasize achieving large receptive fields, sometimes at the expense of considering the importance of information near the edges of missing areas. However, in the inpainting process for irregular missing areas, not all shapes necessitate assistance from distant information. For instance, when dealing with missing areas resembling spider webs, information surrounding the missing regions is often more critical for accurate inference, while distant information may introduce noise and negatively affect the inpainting results. This phenomenon has been discussed in prior works, which highlight the challenges of irregular missing areas for traditional image-inpainting techniques (e.g., [PERSON] et al. [26]). Overall, the existing image inpainting methods have the following shortcomings in generating satisfactory results: (1) They only rely on a single damaged image for inference, without effectively utilizing the remaining known information within the image. (2) They fail to fully consider the internal consistency of the generated content in the missing regions. (3) During the inpainting process, they overly focus on the distant information from the missing regions and adopt the same receptive field strategy for all types of missing regions. Therefore, this paper proposes an structure-guided inpainting method based on a multi-scale attention pyramid structure. This method consists of two sub-networks: one for generating a complete edge structure and the other utilizing the generated edge structure map as auxiliary information to guide the image inpainting, ultimately producinga structurally constrained complete image. Utilizing additional edge structures as guidance addresses the issue of insufficient utilization of effective information in existing inpainting methods for missing images. Furthermore, within the inpainting network, this paper introduces a spatial attention mechanism using a pyramid structure that can fully utilize multi-scale feature information from the missing image, enhancing the internal consistency of the generated content in the missing regions. Additionally, this cross-scale attention mechanism can expand the receptive field for local information, and the fusion of multi-scale information avoids artifacts in the inpainting results due to limited receptive fields in a single layer. The main contributions of the proposed method can be summarized as follows: (1) Generating accurate edge structure maps as auxiliary information to guide image inpainting, achieving multi-view image inpainting, and obtaining visually satisfying results. (2) Proposing a multi-scale feature fusion mechanism that better preserves shallow texture information and deep structural information of the image, enabling the cross-scale attention mechanism to fuse features that are more beneficial for inpainting results. (3) Introducing a cross-scale attention mechanism based on a pyramid structure, where each scale captures different levels of detail in the image. Utilizing the multi-scale attention mechanism for feature weighting enhances the fineness and internal consistency of the inpainting results.