# Structure-Guided Image Inpainting Based on Multi-Scale Attention Pyramid Network
## 2 Related Work
Yet, the reliance on deep convolutional neural networks (CNNs) for HED necessitates extensive annotated data for training, posing challenges in complex inpainting tasks due to data demands, computational complexity, and accuracy of edge detection. [PERSON] et al. [30] used DeepCut technology to predict precise boundary salient object masks, or contour images, providing clearer edge information for guidance in inpainting. However, the accuracy of DeepCut's predictions can be affected by training data and model complexity, potentially leading to blurred or distorted edges in the inpainting results. [PERSON] et al. 's structure-flow [38] method uses edge-preserving smooth images to provide global structural information during the second phase of inpainting, maintaining overall image structure but sometimes resulting in overly smoothed areas lacking realism. Texture-aware multi-GAN approaches by [PERSON] and [PERSON] [31], involving multiple GAN networks at different resolutions, capture varied texture details but require substantial computational resources and time, and coordination between networks can be challenging, possibly leading to inconsistencies or artifacts in the inpainted images. These methods, based on multi-view approaches, depend on the accuracy of additional views, which, if erroneous, can adversely affect the final inpainting results. Therefore, ensuring the accuracy of each view and applying appropriate constraints is crucial in multi-view inpainting to prevent potential negative impacts on the results. Additionally, selecting the right method based on the specific task and data characteristics, possibly in combination with other techniques, is necessary to optimize the inpainting outcomes. The application of attention mechanisms in the field of image inpainting has significantly improved model performance. [PERSON] et al. [39] introduced a context attention module that can borrow or copy feature information from known background blocks to fill in the missing parts by simulating the long-term correlation between distant context information and the missing area. However, this method has its drawbacks: The generated inpainting content might be inaccurate or distorted if the distant context information is only weakly related to the missing area or if there are complex texture and structural changes. [PERSON] et al. [40] designed a short- and long-term context attention layer that enhances appearance consistency by leveraging the distance relationship between decoder and encoder features. Nevertheless, this approach has high computational resource demands, especially for high-resolution images, potentially resulting in longer inpainting processes. Moreover, inaccuracies in the correspondence between encoder and decoder features can lead to biased inpainting outcomes. [PERSON] et al. [41] proposed the PEN-Net, utilizing a cross-layer attention transfer mechanism to ensure visual and semantic consistency by learning regional affinity from high-level feature maps to guide the repair of adjacent lower layers. However, this method demands a high level of model complexity, necessitating careful design and adjustment of the network structure. Furthermore, if the regional affinity learning in high-level feature maps is inaccurate, it might cause errors or inconsistencies in the inpainting results of the lower layers. While deep learning techniques improve image inpainting by deepening the network architecture and expanding the receptive field, this strategy can sometimes neglect the significance of the surrounding context. This oversight may cause problems such as gradient vanishing or explosion, thus destabilizing the training process. Moreover, while increasing the receptive field size helps to capture more contextual information, it may also bring in unwanted noise and interference. Consequently, this can lead to pronounced color discrepancies and the introduction of undesired details in the inpainting results.