# Context-Encoder-Based Image Inpainting for Ancient Chinese Silk
## 3 Approach
### The Principle of the MADF Model  
The MADF [19] restoration model presents a novel paradigm in image processing. Using dynamic convolution filters, it adaptively generates a series of convolution kernels contingent upon configuring effective pixels within each local convolution window, concurrently incorporating mask information. Departing from conventional convolutional methodologies, this model not only recalibrates output activations but also tailors an individualized set of convolution kernels for each spatial position. This customized approach augments sensitivity and adaptability to local features, imparting a distinctly personalized character to the model. This adaptability enables the model to adeptly accommodate diverse image content at varying convolution positions, demonstrating superior efficacy in image restoration endeavors. In Figure 4, the MADF restoration model takes the damaged image (\(I_{m}\)) and its binary mask (\(M\)) as inputs. The model includes three components: (a) the encoder (\(E\)) encodes the damaged image with a mask into high-level feature maps; (b) the restoration decoder (\(R\)) fills in the missing parts of the feature maps; and (c) a series of refinement decoders [\(F_{1},F_{2},\ldots F_{k}\)] refine the feature maps K times and decode the features back to lower-level pixels. Empirically, two refinement decoders balance efficiency and performance. \(MADF^{l}\) and \(R^{l}\) are the \(MADF\) module and restoration decoder block for the \(l\)-th level, respectively. \(F^{l,k}\) represents the \(k\)-th refinement decoder block at the \(l\)-th level. \(m^{l}\) is the mask feature map for the \(l\)-th level of the encoder. Green-marked convolution operations in the encoder take \(e^{l-1}\) as the input, with convolution kernels generated from the corresponding region of \(m^{l-1}\). \"DConv\" denotes transpose convolution, \"LReLU\" is leaky ReLU, and \"up\" signifies increased channels using a \(1\times 1\) convolution layer. \(u^{l-1}\) and \(r^{l}\) are inputs to \(R^{l}\), generating feature map \(r^{l,l}\), and \(F^{l,k}\) takes \(f^{l-1,k-1}\) and \(f^{l,k}\) as inputs to generate feature map \(f^{l-1,k}\). Additionally, \(r^{l}\) and \(f^{l,k}\) are equivalent to \(u^{L}\), and \(f^{l,1}\) equals \(r^{l}\). The model uses a combination of L1 and perceptual losses (\(\lambda_{1}L_{1}+\lambda_{p}L_{perc}\)) for the first refinement decoder's output. Meanwhile, the second refinement decoder's output is supervised by the \(L_{total}\) loss function, incorporating pixel reconstruction, perceptual, style, and total variation losses. The formula for \(L_{total}\) is outlined below:  
\[\mathbf{L}_{total}=\lambda_{1}\mathbf{L}_{1}+\lambda_{p}\mathbf{L}_{perc}+\lambda_{s}\bm {L}_{style}+\lambda_{t}\mathbf{L}_{tv} \tag{15}\]  
where \(\lambda_{1}\), \(\lambda_{p}\), \(\lambda_{s}\), and \(\lambda_{t}\) are weight parameters. According to reference [29], the model empirically sets \(\lambda_{1}=1\), \(\lambda_{p}=0.05\), \(\lambda_{s}=120\), and \(\lambda_{t}=0.1\). Pixel Reconstruction Loss: For the output of the restoration decoder, the model employs the L1 loss function as follows:  
\[\mathbf{L}_{1}=\mathbf{L}_{valid}+\lambda_{h}\mathbf{L}_{hole} \tag{16}\]  
where \(\lambda_{h}\) is weight parameters. According to reference [29], the model empirically sets \(\lambda_{h}=6\). Hole Pixel Loss (\(L_{hole}\)) and Non-Hole Pixel Loss (\(L_{valid}\)) are defined as follows:  
\[\mathbf{L}_{hole}=\frac{1}{N_{I_{gt}}}||(1-M)\odot(I_{out}-I_{gt})||_{1} \tag{17}\]\[\mathbf{L}_{valid}=\frac{1}{N_{tgt}}||M\odot(I_{out}-I_{gt})||_{1} \tag{18}\]  
where \(N_{tgt}\) represents the number of elements in the actual image \(I_{gt}\), \(M\) is the mask, \(I_{out}\) is the generated image, and \(I_{gt}\) is the real image. \(\odot\) represents the convolution operation. Perceptual Loss: The model employs loss calculations based on the _pool_1, _pool_2, and _pool_3 layers of the VGG-16 pretrained on ImageNet. See Formula (7) for details.