Title: SRSR: Enhancing Semantic Accuracy in Real-World Image Super-Resolution with Spatially Re-Focused Text-Conditioning
Venue: NeurIPS 2025
Accepted: yes
Keywords: Real-World Image Super-Resolution

Diffusion Models

Attention Mechanism
Abstract: Existing diffusion-based super-resolution approaches often exhibit semantic ambiguities due to inaccuracies and incompleteness in their text conditioning, coupled with the inherent tendency for cross-attention to divert towards irrelevant pixels. These limitations can lead to semantic misalignment and hallucinated details in the generated high-resolution outputs. To address these, we propose a novel, plug-and-play *spatially re-focused super-resolution (SRSR)* framework that consists of two core components: first, we introduce Spatially Re-focused Cross-Attention (SRCA), which refines text conditioning at inference time by applying visually-grounded segmentation masks to guide cross-attention. Second, we introduce a Spatially Targeted Classifier-Free Guidance (STCFG) mechanism that selectively bypasses text influences on ungrounded pixels to prevent hallucinations. Extensive experiments on both synthetic and real-world datasets demonstrate that SRSR consistently outperforms seven state-of-the-art baselines in standard fidelity metrics (PSNR and SSIM) across all datasets, and in perceptual quality measures (LPIPS and DISTS) on two real-world benchmarks, underscoring its effectiveness in achieving both high semantic fidelity and perceptual quality in super-resolution.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a novel Spatially Re-focused Super-Resolution (SRSR) framework for enhancing semantic accuracy in text-conditioned, diffusion-based real-world image super-resolution. The key idea is to refine text conditioning at inference time using visually grounded segmentation masks, which guide cross-attention (SRCA) and selectively disable classifier-free guidance (STCFG) for ungrounded regions. The approach is fully inference-time and plug-and-play, requiring no additional training. It outperforms seven state-of-the-art baselines in fidelity and perceptual metrics on both synthetic and real-world datasets.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1.High quality of methodology and implementation, with strong empirical evidence supporting the claims.\n2.Clear ablation studies, hyperparameter analysis, and comparisons to state-of-the-art methods. Sufficient and reasonable experiments. Fig. 4 not only shows the superiority of the method, but also shows the limitations of existing IQAs.\n3.Novel spatially-aware cross-attention and CFG mechanisms offer a unique contribution.\n4.Strong plug-and-play usability makes it practical for integration.\nWeaknesses:\n1.Potential over-reliance on DAPE and Grounded SAM performance, which could vary in real-world applications. \n2.No user study or real-world deployment evaluation to validate perceptual benefits from an end-user perspective. Fig. 4 displays the limitations of existing IQAs, which further indicates the necessity of user study to evaluate the motivation.\n\nQUESTIONS:\n1. How does the performance change if the prompt extractor (DAPE) is replaced with a newer vision-language model or fine-tuned for specific domains?\n2. Could the authors clarify how sensitive SRCA and STCFG are to errors in the segmentation masks from Grounded SAM?\n3. Have the authors considered joint optimization between segmentation and SR to resolve interdependencies more robustly?\nScore could increase with answers addressing generalization and robustness to segmentation or prompt extraction failures.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces SRSR, a plug-in framework designed to improve semantic accuracy in real-world image super-resolution (Real-ISR). It features two key components: (1) Spatially Re-focused Cross-Attention (SRCA), which restricts cross-attention to semantically grounded regions defined by tag–mask pairs, and (2) STCFG, which mitigates hallucinations in ungrounded areas by adaptively modulating classifier-free guidance. SRSR is compatible with existing models without retraining. Experiments on demonstrate consistent improvements in image fidelity.\n\nSTRENGTHS AND WEAKNESSES:\n**Strength**\n\n1. The paper is well-motivated, clearly identifying critical and unsolved issues in SD-based Real-ISR frameworks. The illustrative examples and experiments tightly support the claims.\n2. This paper is technically sound and clearly written.\n3. The proposed solutions are elegant:\n    - SRCA restricts cross-attention to semantically grounded regions (identified via Grounded-SAM), reducing the influence of irrelevant tags.\n    - STCFG introduces a novel use of classifier-free guidance in Real-ISR. Instead of manually tuning a global parameter, it disables CFG in ungrounded regions, reducing hallucinations.\n4. The plug-in framework enhances Real-ISR models without the need of retraining, making it efficient and practical.\n5. The experiments are comprehensive, and the qualitative results are particularly convincing.\n\n**Weaknesses**\n\n1. The non-reference scores drop significantly when SRSR is applied, raising the concern that the improvements may largely stem from simply reducing classifier-free guidance (CFG) through the proposed STCFG, rather than targeted semantic enhancement.\n    - For example, the ablation study (Table 2) shows that while STCFG substantially improves fidelity (from V2 to V6), the performance gaps between V6 and V4, as well as between V3 and V4, are marginal. This suggests that the gains may primarily result from reducing CFG strength in ungrounded regions, rather than from the semantic refinement provided by SRCA.\n    - To clarify this point, could the authors report the no-reference metric scores for the V2, V3, and V4 models?\n\n2. The grounding threshold described in Section 4.4 appears to act as another hyper-parameter that influences the perception–fidelity trade-off. While removing redundant tags is a reasonable strategy, its impact on performance is not clearly reflected in the results (e.g., the improvement from V1 to V2 is small).\n\nQUESTIONS:\n**Motivation clarity**\n1. In Lines 37–38, you mention that DAPE “is only partially robust to severe degradations.” Could you elaborate on how this conclusion was derived? Is it based on specific failure cases, empirical observations, or quantitative results?\n\n**Perception–fidelity trade-off**\n\n2. In Fig. 7 of SUPIR$^{[1]}$, qualitative examples suggest that non-reference scores better reflect visual quality. In contrast, your work implies that improvements in reference-based metrics (e.g., PSNR, SSIM) better correlate with perceptual quality, and that gains in non-reference metrics may actually result from hallucinations. While I understand that this trade-off remains a challenging and open question in Real-ISR, and I appreciate the qualitative evidence you provide, there appears to be a contradiction with SUPIR’s conclusions. Could you offer further insight into this discrepancy? Specifically, do you see a path toward a more generalizable understanding of the perception–fidelity trade-off, beyond the current reliance on metric scores and qualitative examples?\n\n**Source of improvement: SRCA vs STCFG**\n\n3. In Section B, you show that replacing DAPE with semantic segmentation degrades performance in your framework. Could this be due to segmentation-based method has ungrounded regions where CFG can be selectively reduced? As noted in the weaknesses section, the marginal performance gap between V4 and V6 in Table 2 suggests that most of the gains may stem from CFG reduction (via STCFG), rather than from SRCA. Also, the segmentation-based variant yields better non-reference scores, though you suggest these may prefer artifact-heavy outputs. To clarify this: (1) A direct comparison between **(a) DAPE + Grounded SAM** and **(b) semantic segmentation, both with and without CFG reduction**, would help isolate the individual impact of each component; and\n(2) Further insight on the perception–fidelity trade-off (as discussed in Question 2) could support this claim.\n\n**Incorporating SRSR into training**\n\n4. In Section E, you suggest that integrating SRSR into the training process could lead to further improvements. Could you provide more intuition or preliminary evidence to support this claim? For example, would incorporating grounded attention during training help the model learn better semantic localization?\n\n[1] Yu *et al.* \"Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild\"\n\nLIMITATIONS:\nI would expect the authors to provide a deeper explanation for the significant drop in no-reference metrics, which are currently regarded as important indicators of performance in SD-based Real-ISR frameworks. This issue raises concerns about the practicality of the proposed method in real-world applications, where reference (ground truth) images are typically of low quality and no-reference metrics play a crucial role in evaluating perceptual quality.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-28"}

comments: SUMMARY:
This paper proposes a plug-and-play spatially re-focused cross-attention based on the discovery that traditional cross-attention tends to divert towards irrelevant pixels. Combined visual grounding, the devised super-resolution framework can effectively avoid the interference of unrelated pixels.

STRENGTHS AND WEAKNESSES:
**Strength**
1. Visually, this work reveals the fact that the vanilla cross-attention induces hallucinations, while the devised spatially re-focused cross-attention can address this issue in some extent.
2. The fidelity of super-resolved images are significantly improved.

**Weakness**
1. The design of the proposed modules are not novel, which are guided by the segmentation masks.
2. The writing should be further improved. For instance, the abstract is too short and the first paragraph is too long.
3. In Figure 1 (left), the body of bird is not highlighted in the attention matrix, could you explain the reason?
4. The references are not sufficient, e.g., SFT, CoSeR, etc.
5. As for the issues about uncoverage of full-image, why not consider employing the models for open word panoramic segmentation？
6. Authors are prefer to emphasize the strength of the designed modules before explanation, e.g., Sec. 3.2.
7. Typo. Wrong tense is used in line 179.
8. There are some repeated references, e.g., 9 and 10,45 and 46.

QUESTIONS:
Please refer to the weakness.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-06-30

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents SRSR, a plug-and-play and training-free framework that can be easily applied to Stable-Diffusion-based SR methods with text conditions. The proposed method enhances semantic fidelity by grounding tags to guide cross-attention and suppress irrelevant text influences on ungrounded regions. SRSR improves both visual quality and alignment with semantics and also alleviates hallucination, outperforming baselines across multiple datasets and full-reference evaluation metrics.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths\n+ The proposed method is simple, straightforward, and intuitive.\n+ The experiment results demonstrate that the proposed method can be easily plugged into existing text-conditioned Stable-Diffusion-based SR methods and improve the SR quality with minimal additional burden.\n+ Overall, the writing and figures are clear and enjoyable to read.\n# Weaknesses\n+ This paper argues that previous methods using a segmentation model like Mask2Former and Prompt-Free Anything Detection and Segmentation in DINO-X lack degradation-awareness and thus lead to inferior results. However, the Grounded SAM they used is not degradation-aware, either.\n+ Grounding shows only slight improvements compared to the baseline, as seen in V1 and V2 of Table 2. This could weaken the argument that the accuracy-completeness trade-off is important and that accuracy typically outweighs completeness.\n\nQUESTIONS:\n1. In Table 2 V5, can you explain how you add Ungrounded Tags? Do these ungrounded tags also associate with segmentation maps? Otherwise, how to use these ungrounded tags with SRCA and STCFG?\n2. Do you try to apply the proposed method to other recent works like AddSR and TSD-SR?\n3. It appears that the proposed method offers greater improvement to the multiple-step method, such as SeeSR, compared to the single-step method like OSEDiff. Have you conducted experiments to evaluate how the diffusion steps affect the results with the proposed method?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}