# Image Structure-Induced Semantic Pyramid Network for Inpainting
## 1 Introduction  
Image inpainting technically means to infer and restore the missing content according to the known content of images so that the inpainted image meets the human visual needs as much as possible. As an important component of computer vision and computer graphics, image inpainting is used widely in cultural and lifestyle applications, such as the conservation of damaged paintings, calligraphy and other digital cultural heritage, old photo restoration, and object removals. Prior to the widespread use of deep learning in image inpainting, the most common techniques used were traditional image inpainting techniques. The methods can be classified into two categories. One is the diffusion image inpainting method. Based on the edge information of the missing regions, the known pixels are propagated in a certain direction to the unknown region to obtain a better inpainting effect. The most representative models are the BSCB model raised by [PERSON] et al. [1] and the curvature diffusion (CDD) inpainting model introduced by [PERSON] et al. [2]. However, this method has some limitations, as it is suitable for relatively narrow missing areas, such as cracks and scratches, and it is difficult to infer the specific texture content. The other is the sample-based image inpainting method [3; 4; 5], in which similar sample blocks are searched for in the complete region of the image to fill the missing region of the image. Among them, the most representative is the block-based texture synthesis algorithm of [PERSON] et al. [4]. [PERSON]'s method can restore some large missing areas and generate clear texture details. However, such images are prone to repeated images, and reasonable structures cannot be generated in complex cases. Deep-learning-based image restoration methods address the lack of understanding of traditional methods. [PERSON] et al. [PERSON] et al. (2016) combined a coding-decoding network with a convolutional neural network in 2016 and presented a network model named the context encoder (CE). The model uses adversarial loss [PERSON] et al. (2016) to train a contextual encoder to predict the damaged areas of the image, but this method may suffer from image edge distortion and local ill-definedness. Based on this, [PERSON] et al. (2018) introduced the global discriminator into the context encoder (CE) model to make the restored image more consistent with the global semantics. However, this method has difficulty achieving complex texture inpainting, and the postprocessing is complex. [PERSON] et al. [PERSON] et al. (2018) put forward a two-stage image inpainting adversarial model called EdgeConnect, which uses edge repair results to constrain texture synthesis. Compared with the single-stage forward unified recovery strategy of structure and texture, this method more easily reconstructs clear semantic boundaries and salient objects. The StructureFlow (SF) model proposed by [PERSON] et al. [PERSON] et al. (2018) adopts a similar inpainting strategy as EC--that is, a smooth image with boundary preservation is used to constrain texture synthesis, which alleviates the problem of texture distortion and artefacts. The Learning to Incorporate Structure Knowledge (LISK) model proposed by [PERSON] et al. [PERSON] et al. (2018) uses gradient maps containing structural information and local texture details to constrain the inpainting procedure, reducing network parameters and improving local details. The work in this paper is a deep-learning-based image inpainting approach focusing on the cross-layer transfer of attention to missing images based on the U-Net edge generation structure. U-Net is a deep network structure proposed by [PERSON] et al. [PERSON] et al. (2016) for the field of image segmentation, which can handle the feature extraction task of deep networks well due to its unique upsampling and downsampling plus jump connections. Many advances have been made in research related to U-Net-based image inpainting [PERSON] et al. (2017); [PERSON] et al. (2018); [PERSON] et al. (2018); [PERSON] et al.