# Context-Encoder-Based Image Inpainting for Ancient Chinese Silk
## 3 Approach  
### The Principle of the LISK Model  
The LISK [18] restoration model excels in acquiring and integrating structural knowledge for image restoration. Employing a multitask learning approach, it simultaneously generates both complete images and corresponding structural information, enriching our comprehension of image content. Including structural embedding and attention mechanisms further enhances the use of structural information in the restoration process, as depicted in Figure 2.  
The LISK restoration model generator comprises a spatial context encoder featuring two downsampling stages, eight residual blocks, and a decoder, as delineated in the figure. The spatial Context-Encoder captures the image's spatial information through downsampling, while the decoder, leveraging residual blocks, upscales to produce an  
Figure 2: Schematic of the LISK multitask framework. image of the original size. Importantly, the encoder is shared between image and structure generation, facilitating the exchange of learned features. The decoder is tailored to a multi-scale style for embedding and outputting structure information at different scales. Two modules leverage structure information: the structural embedding layer and the attention layer. The structural embedding layer seamlessly integrates structural features into various decoding stages, which serve as priors for image generation. This process involves segregating and predicting structural features, subsequently merging them through concatenation using standard residual blocks. The attention layer, inspired by non-local mean mechanisms, computes the response at a position in the output feature map as a weighted sum of the entire feature in the input feature map. Through attention, similar features from the surroundings are conveyed to the missing region, enhancing the generated content and structure and effectively addressing issues such as smooth artifacts and improved details. The LISK restoration model is trained using a combined loss approach, encompassing pyramid structural loss (\(\mathbf{L}_{structure}\)) and blended image loss (\(\mathbf{L}_{image}\)). This methodology aims to capture structural knowledge and supervise the image restoration process. The overall loss function is defined as follows:  
\[\mathbf{L}_{total}=\mathbf{L}_{image}+\alpha\mathbf{L}_{structure} \tag{1}\]  
The pyramid structural loss (\(\mathbf{L}_{structure}\)) is primarily employed to guide the generation and embedding of structures, incorporating structural information into the generation process. The formula for \(\mathbf{L}_{structure}\) is defined as follows:  
\[\mathbf{L}_{structure}=\sum_{s}^{n_{s}}\Bigl{[}\Bigl{|}\Bigl{|}\mathcal{C}^{(s)} _{pred}-C^{(s)}\Bigr{|}\Bigl{|}_{1}+\beta\mathbf{L}^{(s)}_{edge}\Bigr{]} \tag{2}\]  
where \(\mathbf{L}^{(s)}_{edge}\) denotes the regularization term, with \(\beta\) as its corresponding coefficient, and \(n_{s}\) represents the total number of scales. \(C^{(s)}_{pred}\) represents the predicted gradient map at scale s, while \(C^{(s)}\) represents the incomplete gradient map at the same scale. The parameters \(\alpha\) and \(\beta\) serve as hyperparameters to balance the contributions of the different loss terms, \(\alpha=0.1\) and \(\beta=100\). The formula is expressed as follows:  
\[\mathbf{L}^{(s)}_{edge}=\Bigl{|}\Bigl{|}\Bigl{|}C^{(s)}_{pred}-C^{(s)}\Bigr{|} \Bigl{|}\Bigl{|}_{1}\odot M^{(s)}_{E} \tag{3}\]  
where \(M^{(s)}_{E}\) denotes the weighted edge mask. To apply regularization to edge structures, the binary true edge map, \(E^{(s)}\), is convolved with a Gaussian filter, \(g\).