# Task-Driven Real-World Super-Resolution of Document Scans
## 3 The Proposed Framework for Task-Driven Training
### Network Architecture and Loss Function
#### 3.1.5 Multi-Loss Approach  
In the study reported here, we consider eight loss components to capture pixel-level fidelity, structural consistency, and high-level semantic alignment. These are (\(i\)) the pixel-wise MSE, (\(ii\)) the consistency loss component, (\(iii\)) distances in the CTPN feature spaces (three components), (\(iv\)) the distance in the CRNN feature space, (\(v\)) the Key.Net loss, and (\(vi\)) the hue difference. The pixel-wise MSE loss enforces that the super-resolved output matches the HR ground truth:  
\[\mathcal{L}_{\text{MSE}}\ =\ \big{\|}\hat{I}-I_{\text{HR}}\big{\|}_{2}^{2}. \tag{8}\]  
Minimizing \(\mathcal{L}_{\text{MSE}}\) drives the network to reduce pixel-wise differences, which increases the PSNR values. The goal of the consistency loss is to ensure that when a super-resolved image is downsampled back to the original size, it resembles the input image. We employ bicubic interpolation to downsample \(\hat{I}\) to the LR domain, and we compare it with the original \(I_{\text{LR}}\) image to impose the cycle-consistency:  
\[\mathcal{L}_{\text{cons}}\ =\ \big{\|}D(\hat{I})-I_{\text{LR}}\big{\|}_{2}^{2}, \tag{9}\]  
where \(D(\cdot)\) denotes bicubic downsampling by a factor of \(4\times\). This term penalizes the artifacts that do not match with the input LR image. To guide the network toward preserving spatial features relevant to text detection, we incorporate a set of auxiliary loss functions derived from the pretrained CTPN model. Specifically, we extract and compare three types of intermediate representations from both the super-resolved image \(\hat{I}\) and its HR counterpart \(I_{\text{HR}}\): (\(i\)) deep convolutional features,(_ii_) classification logits, and (_iii_) regression outputs. Each feature space yields a separate L1 loss, formulated as:  
\[\mathcal{L}_{\text{CTPN-deep}} =\big{\|}\,\psi_{\text{CTPN-deep}}(\hat{I})-\psi_{\text{CTPN-deep} }(I_{\text{HR}})\big{\|}_{1}, \tag{10}\] \[\mathcal{L}_{\text{CTPN-clss}} =\big{\|}\,\psi_{\text{CTPN-clss}}(\hat{I})-\psi_{\text{CTPN-clss}}( I_{\text{HR}})\big{\|}_{1},\] (11) \[\mathcal{L}_{\text{CTPN-reg}} =\big{\|}\,\psi_{\text{CTPN-reg}}(\hat{I})-\psi_{\text{CTPN-reg} }(I_{\text{HR}})\big{\|}_{1}. \tag{12}\]  
Here, \(\psi_{\text{CTPN-deep}}(\cdot)\) denotes the deepest convolutional activation maps, \(\psi_{\text{CTPN-clss}}(\cdot)\) refers to the classification output logits for text proposal confidence, and \(\psi_{\text{CTPN-reg}}(\cdot)\) captures the predicted vertical bounding-box regressions. By encouraging alignment between these latent representations of \(\hat{I}\) and \(I_{\text{HR}}\), we guide the network to reconstruct text regions in a manner that is focused on text-relevant geometry, even in the absence of explicit bounding-box labels. To further ensure that the super-resolved output improves the capabilities of character recognition, we incorporate the CRNN loss [15]. Let \(\psi_{\text{CRNN}}(\cdot)\) denote the feature extraction function of the CRNN model, which includes both convolutional and recurrent representations. We compute the L1 distance between the feature maps extracted from the super-resolved image \(\hat{I}\) and its HR reference \(I_{\text{HR}}\):  
\[\mathcal{L}_{\text{CRNN}}\ =\ \big{\|}\,\psi_{\text{CRNN}}(\hat{I})-\psi_{ \text{CRNN}}(I_{\text{HR}})\big{\|}_{1}. \tag{13}\]  
This formulation encourages the network to reconstruct the images of enhanced semantic structure required for accurate text recognition. By aligning CRNN-derived features across the input pairs, the model is guided to reconstruct OCR-relevant details even in the absence of explicit textual labels. To teach an SR network reconstructing structural patterns, we incorporate the multi-scale index proposal (MSIP) loss derived from the Key.Net architecture. Unlike previous losses that compare intermediate feature maps, MSIP operates directly on local descriptors extracted from keypoint regions across multiple scales.