Title: Image Super-Resolution with Guarantees via Conformalized Generative Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: conformal prediction

conformal risk control

image super-resolution

generative models

uncertainty quantification
Abstract: The increasing use of generative ML foundation models for image restoration tasks such as super-resolution calls for robust and interpretable uncertainty quantification methods. We address this need by presenting a novel approach based on conformal prediction techniques to create a `confidence mask' capable of reliably and intuitively communicating where the generated image can be trusted. Our method is adaptable to any black-box generative model, including those locked behind an opaque API, requires only easily attainable data for calibration, and is highly customizable via the choice of a local image similarity metric. We prove strong theoretical guarantees for our method that span fidelity error control (according to our local image similarity metric), reconstruction quality, and robustness in the face of data leakage. Finally, we empirically evaluate these results and establish our method's solid performance.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes an uncertainty quantification framework for generative image restoration models, leveraging conformal prediction to produce confidence masks that reflect the local reliability of generated pixels. The method is black-box compatible (requiring only outputs from the generative model), relies on a calibration dataset of high-resolution images, and supports flexible local similarity metrics for interpretability.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The method works on black-box models, which is practical for commercial or API-locked systems where internals are inaccessible.\n\n2. The framework is grounded in conformal prediction, which gives valid coverage guarantees under minimal assumptions, and extends these ideas to the image domain with localization.\n\n3. The idea of per-pixel confidence masks aligned with perceptual similarity metrics is appealing and potentially actionable for real users or downstream systems.\n\nWeakness:\n\n1. While the application to image restoration is interesting, the core method is a relatively straightforward application of standard conformal prediction, adapted to pixel-wise outputs. The “confidence mask” is essentially a localized conformal interval—novel in context, but not in construction.\n\n2. The method assumes access to “a handful of unseen high-res images,” but doesn't quantify how much is sufficient or how sensitive the method is to domain shift between calibration and test inputs. This could be problematic for models trained on synthetic datasets and deployed on natural images.\n\n3. There’s no empirical evidence (at least from intro/abstract) that the confidence masks are intuitive or actionable for users. User studies or visual comparison with ground-truth errors would help validate this interpretability claim.\n\nQUESTIONS:\n1. How many samples are needed?\n\n2. How does domain shift affect performance?\n\n3. Does the choice of perceptual vs pixel-wise metric (e.g., LPIPS vs L1) affect the mask quality or calibration performance?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-19"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a conformal prediction-based method for uncertainty quantification in image super-resolution. The approach generates a binary \"confidence mask\" that identifies reliable regions in images upscaled by black-box generative models (e.g., diffusion models). The contributions include:\n\n(1) A calibration procedure using easily obtained unlabeled data to quantify uncertainty working atop any black-box diffusion model.\n\n(2) Theoretical guarantees for fidelity error control, PSNR bounds, and robustness to data leakage.\n\n(3) The proposed approach not only works on super-resolution, but other tasks such as colorization.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n+ The method provides provable statistical guarantees (Theorem 2.1, Propositions 3.1–3.2) for fidelity error, PSNR control, and data leakage robustness.\n+ Binary masks offer clearer uncertainty visualization than pixel-wise intervals/continuous scores (Fig. 6).\n+ Works with any black-box super-resolution model (including API-locked ones) and requires only unlabeled calibration data.\n\nWeakness:\n- The evaluation of this paper is somewhat weak in demonstrating the accuracy of the proposed approach. Specifically, if my understanding is correct, the evaluation is mainly based on visualization comparisons in Fig.5 and Fig.6. These visualizations are kind of subjective and lack interpretability. No metric evaluation or human evaluation is involved to strengthen the effectiveness of the proposed method.\n- The validation is simply conducted on the validation set of Liu4K, from the same source as the calibration data. The authors should consider testing on images from diverse sources to validate the robustness.\n\nQUESTIONS:\nMy main concerns are as follows:\n\n1. How robust is the proposed approach given different σ (line108)? What if σ is poorly estimated or how to make sure a stable σ is obtained?\n\n2. The evaluation of this paper is somewhat weak in demonstrating the accuracy of the proposed approach. Specifically, if my understanding is correct, the evaluation is mainly based on visualization comparisons in Fig.5 and Fig.6. These visualizations are kind of subjective and lack interpretability. No metric evaluation or human evaluation is involved to strengthen the effectiveness of the proposed method.\n\n3.  The validation is simply conducted on the validation set of Liu4K, from the same source as the calibration data. The authors should consider testing on images from diverse sources to validate the robustness.\n\n4. In the supp. material, the interpretability of the predicted mask is not that straightforward. Ideally, the red regions should indicate that the color is obviously different from the GT. But from my observation, the unmasked regions are also very different from the GT. I am kind of confused if the predicted results are accurate.\n\nLIMITATIONS:\nNo.\n\nThe paper does not discuss the potential limitations and societal impact in any subsection of the paper. The authors should consider adding such a subsection in the revision.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-20"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the need for interpretable uncertainty quantification in image restoration tasks, specifically focusing on image super-resolution using generative ML models. The authors propose a conformal prediction approach to generate a binary confidence mask that highlights regions where the generated image can be trusted. This method, adaptable to any black-box generative model, offers marginal coverage guarantees for a chosen local fidelity measure between the reconstructed and ground truth images, ensuring the expected maximum fidelity error across pixels is constrained by a user-specified level. They demonstrate that data leakage between calibration and training sets leads to a proportional increase in the fidelity level and establish that the method provides control over the PSNR conditioned on the mask. Experimental results, using the SinSR model on the Liu4K dataset, validate the method's effectiveness in quantifying and visualizing uncertainty in image recovery.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \nThe paper is well-written and presents rigorous derivations for a method that visualizes uncertainty through a binary mask. This mask offers clear visualization and statistical guarantees regarding the recovery error. A significant strength of this approach is its reliance on conformal prediction, which inherently allows it to be applicable to any black-box generative model.\n\nWeaknesses:\n* The related work section could be more comprehensive. Given that the paper's main contributions are the interpretable visualization of uncertainty and clear statistical guarantees for recovery error, I expected a more extensive discussion of existing approaches for uncertainty quantification and visualization, both those that offer statistical guarantees and those that do not.\n\n* The experimental evaluation appears limited. The authors do not compare their method to other competitive approaches for uncertainty visualization, even those that might not offer strict statistical guarantees (e.g. \"Uncertainty quantification via neural posterior principal components\"). Furthermore, the comparison to Angelopoulos's method is misleading; while that work did not explicitly aim to produce a mask, its pixel-wise prediction intervals could be used to generate a mask (as demonstrated in Figure 1 of \"Image-to-Image Regression with Distribution-Free Uncertainty Quantification\"). A more appropriate comparison would be against such a derived mask. Lastly, the comparison with Kutiel's method is unfair, as their continuous masks can be easily converted to binary ones via simple thresholding, an effort not undertaken by the authors.\n\n* The method's requirement to produce M samples for generating each mask introduces additional computational complexity and increased runtime. This aspect should be thoroughly discussed in addition to clarifying which of the M images (if any single one) is ultimately presented to the user.\n\nQUESTIONS:\n* Please extend the related work and the comparison in the experimental section as mentioned in the weakness above.\n* Please elaborate on the point of generating M images.\n* The method seems general. Why restricting it application only to super-resolution?  \n* The use of semantic distance can be seen as providing per-pixel labels and then applying standard conformal risk control. Is this view correct?\n\nLIMITATIONS:\nSee Questions and Weakness.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors work towards answering the interesting question of how to provide guarantees and interpretability for the output of diffusion-based generative models. Specifically, the authors propose a simple but effective method to quantify the uncertainty of images that are inpainted or super-resolved using diffusion models. The authors provide statistical guarantees their marginal conformal prediction will be bounded in expectation. The authors provide empirical evidence showing the effectiveness of their method and how tight the bounds are.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n1. **Simplicity & Interpretability**\n   The proposed algorithm is straightforward to implement and represents an important step toward greater interpretability in diffusion-based generative models. It also raises interesting philosophical questions about the trade off between fidelity and uncertainty (see Figure 2).\n\n2. **Empirical Validation**\n   The authors back up their theoretical claims with solid experimental evidence, demonstrating that the derived bounds are indeed appropriate in practice.\n\n3. **Novel Application of Conformal Prediction**\n   To the best of my knowledge, this work is the first to bring conformal prediction techniques into inverse problems specifically inpainting and super-resolution for diffusion-based generative models.\n\n\n## Weaknesses\n\n1. **Proof Novelty Is Unclear**\n   Based on the Appendix, it’s difficult to distinguish the contributions of this proof from those in prior work by Anastasios Nikolas. The paper would benefit from a clearer discussion of what is fundamentally new.\n\n2. **Loose Bounds**\n   The bounds illustrated in Figure 4 appear rather loose, which makes it challenging to assess how informative they are in practice. Tighter bounds or additional commentary on their practical implications would strengthen the paper.\n\nQUESTIONS:\n## Questions\n\n1. **Clarification of Proof Technique**\n   Could the authors elaborate on the significance of their proof technique and explain how it differs from the original work by Anastasios et al?\n\n2. **Tightness of the Bound**\n   Could the authors comment on the tightness of their derived bound and discuss the main challenges involved in improving it?\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-09"}