Title: Text-Aware Real-World Image Super-Resolution via Diffusion Model with Joint Segmentation Decoders
Venue: NeurIPS 2025
Accepted: yes
Keywords: Image Super-Resolution

Text Structure Preservation

Text Segmentation

Multi-task Learning

Diffusion Model
Abstract: The introduction of generative models has significantly advanced image super-resolution (SR) in handling real-world degradations. However, they often incur fidelity-related issues, particularly distorting textual structures. 
In this paper, we introduce a novel diffusion-based SR framework, namely TADiSR, which integrates text-aware attention and joint segmentation decoders to recover not only natural details but also the structural fidelity of text regions in degraded real-world images. Moreover, we propose a complete pipeline for synthesizing high-quality images with fine-grained full-image text masks, combining realistic foreground text regions with detailed background content. Extensive experiments demonstrate that our approach substantially enhances text legibility in super-resolved images, achieving state-of-the-art performance across multiple evaluation metrics and exhibiting strong generalization to real-world scenarios. Our code is available at [here](https://github.com/mingcv/TADiSR).


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\n- **What**: This paper proposes TADiSR, a text-aware diffusion-based framework that jointly performs image super-resolution and text segmentation, aiming to restore degraded real-world scenes with sharper textual structures.\n- **How**: The method augments a latent-diffusion backbone with (i) text-aware cross-attention, which steers attention toward textual regions, and (ii) joint segmentation decoders that output a super-resolved image and its text mask. Then, the entire model is trained using an SR-oriented loss with edge-aware focal terms and a segmentation loss on the 50k-image FTSR dataset synthesized by a scalable, text-oriented pipeline.\n- **Results**: On FTSR-TE and Real-CE-val, TADiSR shows the best PSNR/SSIM/LPIPS/FID and boosts OCR accuracy by +18.7%p over the previous best method, with ablations showing that removing any main component degrades performance.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n1. **State-of-the-art performance**: TADiSR outperforms all published methods on FTSR-TE and Real-CE. It raises OCR-A by 18.7 percentage points over the previous best and improves PSNR, SSIM, LPIPS, and FID simultaneously. This is clear evidence of perceptual and recognizability gains.\n2. **Novel, well-motivated architecture**: The model explicitly couples SR and pixel-level text segmentation by integrating Text-Aware Cross-Attention (TACA) with a Joint Segmentation Decoder. Ablation studies show that disabling either module sharply degrades OCR and structural fidelity, validating the necessity of the design.\n3. **Scalable synthetic data pipeline**: The authors contribute a 50,000-image dataset generated by a flexible, text-oriented synthesis pipeline. This pipeline provides accurate text masks and rich backgrounds. It offers a reusable resource for future text-oriented SR and segmentation work.\n4. **Thorough empirical validation with clear visualizations**: The paper reports five quantitative metrics, diverse qualitative comparisons, and attention heat maps. These results give readers numerical and visual confidence in the method’s effectiveness.\n\n### Weaknesses\n1. **Narrow degradation and scale coverage**: All experiments use a single ×4 setting with relatively mild Real-ESRGAN degradation, and many of the input examples are already human-readable. The paper does not address how the model behaves under heavier blur, noise, or larger factors (e.g., ×6 or ×8).\n2. **Sparse Analysis of Text-Aware Cross-Attention (TACA)**: The ablation table only shows a binary switch for w/o TACA. It does not test directing attention to alternative tokens or to a global text embedding. Therefore, the paper stops short of proving that steering specifically to the word \"text\" is essential rather than incidental.\n3. **Unexplored Prompt Sensitivity**: All results rely on the fixed prompt \"text.\" The impact of alternative wording (\"sign,\" \"label,\" etc.) or prompt-free inference is unreported, leaving deployment robustness uncertain.\n\n*My confidence in the method's generality and practical deployability would increase substantially if the authors (i) added results for stronger and multi-scale degradations, (ii) provided token-level ablations that isolated the benefit of the \"text\" cue, and (iii) tested performance under varied or absent prompts.*\n\nQUESTIONS:\n1. **Exact definition of the w/o TACA setting (Table 2)**:\nWhen Text-Aware Cross-Attention is disabled, which operations are switched off or replaced? More importantly, what feature tensor is forwarded to the Joint Segmentation Decoder? A concise diagram or textual description would make the ablation easier to interpret.\n2. **Report absolute and ΔOCR-A scores alongside transcripts**:\nList the OCR-A score of the raw LR inputs, followed by the gain (Δ) achieved by each method. Following BTC [1] Fig. 3, could you also overlay the recognized text strings on the qualitative comparison figures?\n\n**Impact on score**\nClarifying the ablation protocol and visualizing the OCR results would remove key ambiguities. Convincing answers could raise my overall rating by 1 point.\n\n[1] Pak et al., \"B-Spline Texture Coefficients Estimator for Screen-Content Image Super-Resolution,\" CVPR 2023.\n\nLIMITATIONS:\nyes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-15"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a diffusion-based SR framework TADiSR to solve the textual structure distorting issues commonly appearing in generative models. TADiSR introduces a unified framework that integrates text-aware attention and dual-stream decoding to recover degraded text in natural images.  **[Network]** The proposed fine-tuning of cross-attention between the word \"text\" and image tokens effectively guides the diffusion models to pay attention to text regions. Meanwhile, the joint image-text segmentation decoder enables more accurate and structure-preserving restoration of textual contents, in a simultaneous generation way. **[Data]** In addition to the network design, the data synthesis pipeline further supports effective training with fine-grained text masks and diverse backgrounds with high quality, making the approach scalable in both Real-SR and text segmentation. **[Loss]** To enhance the preservation of text structures, the paper introduces a modified focal loss that emphasizes hard boundary pixels by leveraging segmentation predictions and ground-truths. TADiSR is compared with GAN-based methods, diffusion-based methods, and SOTA text image SR methods, on both synthetic and real benchmarks, where improved results are shown quantitatively and qualitatively, demonstrating its superiority in both visual perception and text fidelity.\n\nSTRENGTHS AND WEAKNESSES:\n**Paper strengths**\n\n1. The method shows clear advantages in both quantitative and qualitative evaluations. In particular, the qualitative results are impressive, with visually natural and structurally accurate text restoration, significantly outperforming existing methods in preserving the fidelity of textual content.\n1. The integration of text-aware cross-attention supervision with dual-stream decoding in a diffusion framework is a novel contribution. Moreover, the insight that text-aware super-resolution and text segmentation are complementary tasks that can be effectively unified under a multi-task learning paradigm adds depth and originality to the work.\n1. The paper is clearly written and well structured. The methodology is explained in sufficient detail, and the model architecture is effectively illustrated with clear figures. \n\n**Major weaknesses**\n\n1. One major concern lies in the data synthesis pipeline used to construct the FTSR dataset. The copy-and-paste strategy, while effective for generating large-scale training data, may introduce a distribution gap between synthetic images and real-world scenes, potentially limiting the model’s generalization ability. It remains unclear whether the authors have made any attempts to mitigate this domain gap or assess its impact.\n1. Another concern is the potential quality issue introduced during data synthesis, specifically in the step where text samples are first super-resolved using a Real-SR method before being pasted onto high-resolution backgrounds (as mentioned in line 217). This process may introduce distortions into the text regions, lowering the ground truth quality. Moreover, since the text segmentation masks are generated from the original CTR data, they may no longer perfectly align with the super-resolved text patches, leading to mismatches between the image content and segmentation labels during training.\n1. While the Joint Segmentation Decoders offer explicit structural supervision through cross-decoder interactions, it is unclear how this dual-decoder design affects inference efficiency and memory usage. The potential computational overhead introduced by maintaining and interacting between two decoding branches may hinder the model’s practical deployment, yet the paper does not provide a detailed analysis of its impact on inference speed.\n\n**Minor weaknesses**\n\n1. Fig 4. The cropped patch in the boxes of GT is slightly misaligned.\n\nQUESTIONS:\n1. The paper lacks ablation studies to validate the effectiveness of the FTSR dataset. Could the authors evaluate whether training with FTSR improves performance on Real-CE-val? It would also be helpful to show whether other baseline methods benefit from using FTSR to demonstrate its general utility.\n1. Since the CTR dataset mainly contains Chinese text, could the improved performance be partly due to exposure to Chinese characters during training, which competing methods lack? It would be important to show that the performance gains are not solely from language-specific data but also from other proposed designs (e.g., JSD, TACA, MF loss).  Also, it is suggested to provide more examples on other languages in addition to Chinese to show its overall ability.\n1. The evaluation of text recovery primarily relies on a single metric, OCR-A, which may not sufficiently capture the full quality of restored text. Incorporating additional metrics such as OCR-based $F_1$ scores [1] or other metrics would provide a more comprehensive and robust assessment of text fidelity.\n1. It would be helpful to include a table comparing inference speed, memory usage, and model size across all methods. This would provide a clearer picture of the practical trade-offs and deployment feasibility of TADiSR.\n---\n[1] Chee Kheng Chng, Yuliang Liu, Yipeng Sun, Chun Chet Ng, Canjie Luo, Zihan Ni, ChuanMing Fang, Shuaitao Zhang, Junyu Han, Errui Ding, et al. Icdar2019 robust reading challenge on arbitrary-shaped text-rrc-art. In 2019 International Conference on Document Analysis and Recognition (ICDAR), pages 1571–1576. IEEE, 2019.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-29"}

comments: SUMMARY:
This work proposes a novel framework to address the issue that unnatural details of text in image generated by diffusion model, which is practical and  effective.

STRENGTHS AND WEAKNESSES:
**Strength**:
1. The motivation is strong and novelty. It is an important issue that unnatural details of text in image generated by DM, especially for Chinese character.
2. The design of the devised frame work is simple yet effective. The experimental results demonstrate the effectiveness of the proposed framework, which deliver clear and realistic text in the generated image.

**Weakness**:
1. The confused name of the decoder for segmentation. The joint seg decoder is introduced to generate segmentation map for text in the image. However, it is not used to generate SR image at the same time, therefore, the joint is not proper, which would cause confusion.
2. Unclear explanation in Figure 2. For example, there is no explanation about the orange squares, which could be the LoRA?
3. Confusing descriptions. In line 150-151, why does the $c_{tex}$ need to be located? Isn't it the last one in the $c_y$?
4. Unclear motivation of design for $a_{tex}$. Why are $a_{tex}$ concatenated along the channel dimension and how can it be used to generate the segmentation map solely without $z_H$?
5. It would be better to discuss the parameters, FLOP, and inference time compared to other methods by introducing additional VAE decoder.

QUESTIONS:
Please refer to the weakness.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-30

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a Text-Aware Diffusion model for real-world image super-resolution, named TADiSR. Built upon diffusion models, TADiSR integrates Text-Aware Cross-Attention (TACA), a Text Segmentation Decoder (JSD), and Cross-Decoder Interaction Blocks (CDIB) to enhance text extraction and restoration explicitly. The method significantly improves text restoration performance in images.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n1. TADiSR adopts a dual-branch design to extract text information, thereby enhancing text restoration explicitly. The idea is well-motivated and yields strong performance.\n2. The authors also construct a dedicated pipeline for dataset generation, which supports research in this area.\n3. Both quantitative and qualitative results demonstrate excellent restoration performance, which is particularly impressive given the tendency of diffusion models to produce hallucinated textures.\n4. The ablation study validates the contributions of the JSD and TACA modules, and visual comparisons further support these findings.\n5. The paper is well organized. The authors also release code, making the work more solid and reproducible.\n\n\n\n### Weaknesses\n\n1. Although the ablation study validates some components, it does not cover the full model. For example, the impact of CDIB is not studied. The study focuses only on network components, while the proposed pipeline is not experimentally evaluated. The authors should validate the effectiveness of the pipeline itself.\n2. The pipeline section lacks a flowchart, making it harder to understand. Moreover, pasting text onto other images may not accurately reflect real-world scenarios.\n3. In the visual comparisons, the proposed method shows strong performance on Chinese text, but the results of English text are not much different from those of other methods. It is unclear whether this is due to training bias. If other methods like DiffTSR were trained on the same dataset, would the performance gap remain?\n\nQUESTIONS:\nSee weaknesses.\n\nLIMITATIONS:\nThe authors have discussed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-06"}