# Cascaded Dual-Inpainting Network for Scene Text
## 3 Methodology
### Structure Reconstruction Module (SRM)  
Text inherently possesses strong structural properties that remain independent of stroke thickness, color, or other visual attributes. The stroke information provides critical guidance for restoring damaged or incomplete characters. To leverage this, the SGM generates an initially repaired foreground character structure. Under the guidance of the SGM-generated text structures, the SRM restores the missing parts of characters to enhance scene text inpainting performance.  
Figure 1: General framework of the cascaded dual-inpainting network for scene text.  
As illustrated in Figure 1, the pipeline of the SRM operates as follows: (1) the SGM first produces an initial text structure \(\hat{S}_{SGM}\); (2) this output \(\hat{S}_{SGM}\) is then processed as the structure-guided condition of the SRM to enhance the text inpainting:  
\[\hat{S}_{SRM}=\mathcal{F}_{SGM}\big{(}I_{cor},\hat{S}_{SGM}\big{)} \tag{5}\]  
where \(I_{cor}\) denotes the corrupted scene text image as input, \(\hat{S}_{SGM}\) serves as the generated structural guidance condition, and \(\mathcal{F}_{SGM}\) represents the SRM's nonlinear transformation function. \(\hat{S}_{SRM}\in\mathbb{R}^{h\times m}\) is the final refined structure, which is improved in spatial coherence and structure preservation compared with the initial SGM output.  
As shown in Figure 1, an Attention U-Net [38] is employed, featuring an encoder-decoder architecture for text structure prediction (input: 4-channel RGB + grayscale, output: 1-channel grayscale). The encoder consists of four down-sampling blocks, each comprising a MaxPool2d layer followed by dual convolutional blocks. The decoder consists of four up-sampling stages, each incorporating bilinear interpolation followed by channel concatenation (combining skip connection features with up-sampled features) and dual convolutional blocks, with attention gates integrated at each skip connection junction. In the Attention U-Net architecture, skip connections fuse low-level spatial features from the encoder with high-level semantic features from the decoder for effective detail restoration through hierarchical feature fusion. The attention gate mechanism demonstrates three functionalities: (1) enhancing focus on text regions to locate missing text areas; (2) concentrating on valid contextual information from surrounding regions; (3) suppressing irrelevant features interference while reducing background noise in complex scenes. During scene text inpainting, the SRM can focus more on the edge information of missing regions to better reconstruct the text structure. In deep decoding stages, the SRM may focus more on semantic information, while shallow layers prioritize detailed textures. The attention gate can dynamically adjust feature importance according to the current stage's requirements.  
SRM loss consists of four components: \(L_{1}\) loss, binary cross-entropy (\(L_{lce}\)), style loss (\(L_{style}\)), and semantic loss [34; 39] (\(L_{sem}\)). These components are formally defined by Equations (6)-(10):  
\[L_{SRM}=\beta_{1}\cdot L_{1}+\beta_{2}\cdot L_{bce}+\beta_{3}\cdot L_{style}+ \beta_{4}\cdot L_{sem} \tag{6}\]  
\[L_{1}=\left\|S-\widehat{S_{SRM}}\right\| \tag{7}\]  
\[L_{bce}=-\theta_{1}\sum S\cdot log\widehat{S_{SRM}}-\theta_{2}\sum\left(1-S \right)\cdot log\left(1-\widehat{S_{SRM}}\right) \tag{8}\]  
\[L_{style}=E\left[\left\|G_{j}^{\phi}(S)-G_{j}^{\phi}\left(\widehat{S_{SRM}} \right)\right\|_{1}\right] \tag{9}\]  
\[L_{sem}=E\left[\sum_{i}\frac{1}{N_{i}}\left\|\phi_{i}(S)-\phi_{i}\left(\widehat {S_{SRM}}\right)\right\|_{1}\right] \tag{10}\]  
where \(\phi_{i}\) denotes the feature map from the \(i\)-th perceptual layer, \(N_{i}\) is the number of elements in feature map obtained by the the \(i\)-th layer, and \(G_{j}^{\phi}\) represents the Gram matrix [40]. \(\beta\)-terms are loss balancing coefficients.