# Context-Encoder-Based Image Inpainting for Ancient Chinese Silk
## 3 Approach
### The Principle of the MEDFE Model  
The MEDFE [17] restoration model introduces a network characterized by mutual encoding and decoding for image restoration. Within this framework, CNN features from the shallow encoder are designated to represent texture, while deep features comprehensively capture the image's structural components. To guarantee coherence between structural and textural features, a feature-balancing method is proposed. This method involves concatenating features, reweighting channels, and introducing a bilateral propagation activation function for spatial consistency across the entire feature, effectively mitigating blurring and artifacts resulting from inconsistencies between structural and textural features. A schematic representation of the MEDFE restoration model framework is delineated in Figure 3.  
Figure 3: MEDFE framework schematic. Figure 3 shows that the MEDFE restoration model categorizes texture features as \(F_{te}\) and structure features as \(F_{st}\). Throughout the reorganization process, the model adjusts the sizes of CNN feature maps from different convolutional layers, ensuring uniformity and appropriate connections. Following this reorganization, the MEDFE restoration model encompasses two branches--the structure branch and the texture branch--for hole filling in \(F_{te}\) and \(F_{st}\). Both branches share identical structures, each containing three parallel streams for hole filling at multiple scales. Each stream comprises five partial convolutions with the same kernel size, while the kernel size varies among streams. By employing different kernel sizes, the MEDFE model achieves multi-scale hole filling on the input CNN features for each branch. The features filled from three streams (i.e., three scales) are concatenated and mapped to the same size as the input feature map through a \(1\times 1\) convolution. The model designates the output of the texture branch as \(F_{ft}\) and the output of the structure branch as \(F_{jst}\). To ensure that hole filling emphasizes texture and structure, supervision is applied to Fjst and Ffte. \(F_{ft}\) and \(F_{fte}\) are mapped to color images \(I_{ost}\) and \(I_{ote}\), respectively, through a \(1\times 1\) convolution. The pixel-level L1 loss is formulated as follows:  
\[\mathbf{L}_{rst}= \big{|}\big{|}I_{ost}-I_{st}\big{|}\big{|}_{1} \tag{11}\] \[\mathbf{L}_{rte}= \big{|}\big{|}I_{ote}-I_{gt}\big{|}\big{|}_{1} \tag{12}\]  
where \(I_{gt}\) represents the authentic image and \(I_{gt}\) denotes the structural image derived from \(I_{gt}\) using the edge-preserving smoothing technique RTV [24], following Structureflow [25]. The hole regions within \(F_{te}\) and \(F_{st}\) undergo distinct filling processes via the structure branch and the texture branch. Nevertheless, the incongruent feature representations in \(F_{fte}\) and \(F_{ft}\) are insufficient in accurately capturing the recovered structure and texture, consequently giving rise to blurriness and artifacts in and around the hole regions. In addressing these concerns, the model concatenates \(F_{fte}\) and \(F_{ft}\), followed by a straightforward fusion employing a \(1\times 1\) convolution layer to generate \(F_{gt}\). Feature normalization is systematically applied at discrete CNN feature levels to enhance the refinement of texture and structure representations within \(F_{sf}\). The MEDFE restoration model introduces a suite of loss functions to quantify distinctions between structure and texture. These loss functions encompass pixel reconstruction loss, perceptual loss, style loss, and relative mean LS adversarial loss [26]. The comprehensive loss function is expressed as follows:  
\[\mathbf{L}_{total}=\lambda_{st}\mathbf{L}_{rst}+\lambda_{te}\mathbf{L}_{rte}+\lambda_{r} \mathbf{L}_{rec}+\lambda_{p}\mathbf{L}_{prec}+\lambda_{s}\mathbf{L}_{style}+\lambda_{adv} \mathbf{L}_{adv} \tag{13}\]  
where \(\lambda_{r}\), \(\lambda_{p}\), \(\lambda_{s}\), \(\lambda_{adv}\), \(\lambda_{st}\), and \(\lambda_{te}\) are weighting parameters.