Image Super-Resolution with Guarantees via Conformalized Generative Models
Authors: Eduardo Adame
Daniel Csillag
Guilherme Tegoni Goedert
Venue: NeurIPS 2025 poster ()
Keywords: conformal prediction
conformal risk control
image super-resolution
generative models
uncertainty quantification
Primary area: probabilistic_methods
TL;DR: We design a new conformal prediction method with theoretical guarantees that produces 'confidence masks' for uncertainty quantification of image restoration tasks such as image super-resolution.

Abstract: The increasing use of generative ML foundation models for image restoration tasks such as super-resolution calls for robust and interpretable uncertainty quantification methods. We address this need by presenting a novel approach based on conformal prediction techniques to create a `confidence mask' capable of reliably and intuitively communicating where the generated image can be trusted. Our method is adaptable to any black-box generative model, including those locked behind an opaque API, requires only easily attainable data for calibration, and is highly customizable via the choice of a local image similarity metric. We prove strong theoretical guarantees for our method that span fidelity error control (according to our local image similarity metric), reconstruction quality, and robustness in the face of data leakage. Finally, we empirically evaluate these results and establish our method's solid performance.

Image Super-Resolution with Guarantees via Conformalized Generative Models
Eduardo Adame School of Applied Mathematics Getulio Vargas Foundation eduardo salllllllssf..... br
Daniel Csillag
School of Applied Mathematics
Getulio Vargas Foundation
daniel.csillaaeee........... csllalill br
Guilherme Tegoni Goedert
School of Applied Mathematics
Getulio Vargas Foundation guil gfgd br
Abstract
The increasing use of generative ML foundation models for image restoration tasks such as super-resolution calls for robust and interpretable uncertainty quantification methods. We address this need by presenting a novel approach based on confor- mal prediction techniques create 'confidence mask' capable of reliably and intuitively communicating where the generated image can be trusted. Our method is adaptable to any black-box generative model, including those locked behind opaque API, requires only easily attainable data for calibration, and is highly customizable via the choice of a local image similarity metric. We prove strong theoretical guarantees for our method that span fidelity error control (according to our local image similarity metric), reconstruction quality, and robustness in the face of data leakage. Finally, empirically evaluate these results and establish our method's 's solid performance.
1 Introduction
Generative ML foundation models led to massive leaps in the capabilities of modern image synthesis and processing, spanning domains such as image generation, inpainting, and super-resolution. Par- ticularly in the case of image super-resolution, recent methods have become considerably adept at leveraging patterns in images to better recover complex textures, geometries, lighting and more.
In testament to these improvements, leading manufacturers are constantly improving and deploying tools based on these frameworks in every new generation of consumer devices. These widespread applications highlight an important question: How trustworthy are the predictions of these models? When model does some particular inpainting or super-resolutio infill, what guarantees do we have that its predictions are truly accurate to reality, and not mere hallucinations? Therefore, it would be most desirable to have a proper uncertainty quantification over the predicted image.
However, most of the previous contributions to this endeavor have suffered from a lack of interpretabil- ity. In order to be widely adopted, a new framework should clearly communicate its uncertainty estimates to the public in a way that reflects how they will be used. This demand for interpretability fundamentally guides the properties and theoretical guarantees we seek to establish for our predictions. and thus on the underlying procedure, Of course, all of this is compounded by the usual challenges of having to do trustworthy uncertainty quantification that is model agnostic and can be employed effectively atop any "black-box" foundation model.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

In this paper, we address all these issues by proposing method based on techniques from conformal prediction [Vovk al., 2005] and conformal risk control [Angelopoulos et al., 2022a while employing metrics designed for interpretability in concrete applications. All our method requires is handful of unlabelled high-resolution images that were not present in the training set for the diffusion model, and we achieve strong guarantees on our predictions that are also intuitive to the user and robust to violations our key assumptions.
Our main contributions are:
A new method to quantify uncertainty in images inpainted or augmented by diffusion-based models. Our method can work atop any black-box diffusion model, including models that are locked behind an opaque API, and requires only easily-attainable unlabelled data for calibration.
We identify additional theoretical guarantees enjoyed by model. In particular, prove that our method also controls the PSNR of the predicted images, and show that is reasonably robust to data leakage, reinforcing the effectiveness and robustness of our method.
A comprehensive study of modelling choices in our approach, revealing particular modifica- tions from the base procedure that can significantly enhance performance. Particularly, we demonstrate that certain applications of low-pass filters can greatly improve our method's effectiveness.
We note that our procedure is also reasonably generalizable to other image restoration tasks, and provide examples in the supplementary material.
Related work There have been some attempts on formal uncertainty quantification for image super-resolution with varying approaches. But most existing works, e.g., (Angelopoulos et al. 2022h Teneggi et al., 2023], have their goal produce, rather than single image, interval- image (l.e., an image where each pixel is represented by an interval rather than a single value), and ensure that these intervals will, with high probability, contain the 'true' pixel values. However, this is fairly non-intuitive for the user and its underlying guarantees are a bit lax. As far as we aware, the only existing solution that proposes some other way of quantifying uncertainty is [Kutiel et al.) 2023). which produces continuous "confidence" mask over the predicted image, meaning that each pixel in the image is assigned confidence score in [0, 1]. However, their solution has no probabilistic guarantee, which is fundamental for reliable uncertainty quantification (see the supplementary material). Ours, in contrast, is backed by a plethora of such theorems. Finally, our proposed solution is closely related to the existing applications of conformal prediction to semantic segmentation, e.g., [Csillag et al.) 2023] Mossina et al.) 2024]. Also worth noting are methods for uncertainty quantification for image super-resolution that do not have any formal guarantee of correctness; relevant works include both general uncertainty quantification techniques for deep learning such as Bayesian neural networks [Goan and Fookes (020) and Monte Carlo Dropout Gal and Ghahramani] 2015), as well as more task-specific approaches such as NNhhme et al. 2023 Adapa
et al. 2024].
2 Method
2.1 Conformal Mask Calibration
Let us be supplied with any generative image super-resolution model J [0, 1]
[0, [j where w and h stand for the low-resolution image dimensions, and k is the up- scaling factor. Naturally, this is a stochastic function due to the generative nature of the model, so an intuitive (albeit nonrigorous) way to quantify model indecision would be to aggregate many realizations the output image (e.g. by computing the variance of generated pixels). However, our methodology is capable of working naturally when supplied with an arbitrary estimate of model indecision that can be described by a function o : [0, 1 w xhx3 + kkwxkh We further discuss useful constructions of in Section 2.2
Having received lower resolution image X, we must consider how the model prediction P= (X)) differs from the true high resolution image Y. Our goal is to find "confidence mask" M(X that

---

Prediction with conformal mask (non-semantic D2)
Prodiction with conformal mask (semantic D2)
with 64 wixel
Pixel difference (pred gil)
Gaussian blur
Low resolution
Ground truth
Figure 1: Our method highlights meaningful uncertainty regions in generated images. This figure presents comparison of multiple high-resolution images with their corresponding conformal masks. Our conformal masks accurately highlight regions where the predictions significantly deviate from the ground truth, capturing differences in color, texture, and lighting.
indicates the region of the image whose content trust, i.e. the pixels in the predicted image with indecision below a sought threshold. Formally, the mask is a (possibly stochastic) function M [0, 1|=xhxx + {0. 1] J that has image (X) {p Z Z (X)), S t). where [+]p is the image value at pixel p (be it binary, grayscale or colored) and is a desired indecision threshold.
We seek masks that satisfy fidelity guarantees between the true and predicted high-resolution images. This fidelity is measured by fidelity error defined pe M (Y, Y). where Dp is function that measures the difference between two images around some pixel location p. We can employ any local measure of the difference as long as < (Y, Y) < 3 for all p. Y and Y (c.g. Dp Y ||YY, - p11)) There many different useful choices for D,. with few explored in Section 2.3 Equipped with the previous definitions, we are able to produce these fidelity masks Ma (X) for any desired fidelity level a [0, 1] with the guarantee that
sup Dp (x(X)) Sa (X)
E
by thresholding the output of a by some parameter f in the construction of the mask. This parameter can then be calibrated for by using techniques conformal prediction [Vovk et al.] (005) and conformal risk control (Angelopoulos et al. 2022a] with just access unlabelled hold-out data, which has not been used to obtain either j or a (if there data contamination, weaker guarantees hold; see Proposition 3.2 In particular, given such data (X,, Yo) C [0, [0,1)xxbx, x [0, kwxkhx3 we produce
3 ((()) + n+1 (X
ta = sup
lE E (uoo)+oo)
sup
S Sa
(I)
n +
thus obtaining
Ma(X) == (p Zx [o(X) < tta.
(2)
Crucially, all of our guarantees will hold for any μ and o, though the produced trust masks M,, will generally be larger (i.e., more confident) for well-chosen ones.
This methodology comes with strong statistical assurance: a marginal "conformal" guarantee. holds in expectation on the calibration data jointly with an additional new, 'test" sample:
3

---

Theorem 2.1 (Marginal conformal fidelity guarantee). Let a R and n E N. Suppose we have n + 1 Lidd samples , x from an arbitrary probability distribution P and let ta and Ma be as in Equations 7 and (and thus only a function of i, Y with i 1, n...,))) Then holds that
F (X.Y
D2 (m((n+1), pEM.(X.41)
sup
Va+1) (
a.
Thanks to the discrete combinatorial structure of the set the infimum taken over in Equation | the indecision threshold t. (defined in Equation I can be efficiently computed with the use of dynamic programming Ond log d) time, where the number of images in the calibration set and d the number of pixels in each image. In contrast, naive brute force algorithm would take (nn d?) time. The relevant pseudocodes can be found in the supplementary material.
2.2 Producing Score Masks
key component of our algorithm is the model indecision estimate a [0, 1] - R A good should attain higher values for regions of the image where there is more uncertainty, and lower values for regions where we are more certain. Nevertheless, our guarantees hold for any choice of a.
Considering the generative nature our base models, one natural way to produce such a is to take the pixel-wise empirical variance of M generated images:
o var (X)lp VarM [Ix(X))p}
However, this may suffer from being too local: for example, if the model correctly knows that an edge must be present in a particular region an image but slightly misplaces it by one or two pixels, there would be significant mismatch between the "true" model uncertainty and the indecision estimate by g var
To resolve this, we propose to 'smooth out' our predictions by performing a convolution with low pass kernel K. naive way of doing so would be to convolve the images whose pixels we are taking the variances of directly: Var at [[^(X) * K]p]- However, this has an unintended side effect: by applying the convolution directly to the generated images, are effectively undoing the super-resolution done by the model! Hence, we propose to instead apply the convolution to the computation of the variance, via its decomposition terms of the second moment:
kekerrKK (x),, E. [lx(X)² k1,] (Enr [lx(X) Kı,))*
should be noted that when K is 1-box kernel, we recover var, After computing this patch-based variance, we further apply Gaussian blur to the resulting variance map. This additional smoothing step helps mitigate the risk of border artifacts being overly emphasized, which can otherwise lead to an undesired overestimation of uncertainty along edges.
Finally, we remark that ideally this model indecision would be estimated jointly with the upscaled image (XX). This is, however, fairly nontrivial and best left for future work.
Choices Dr
A crucial point of our procedure is the definition of the precise fidelity error are controlling, This given by the choice of D,- which function indexed by a pixel position p that receives the real and predicted images Y and ÿ and returns notion of how similar the two are around p. Our procedure valid for any choice of D, that is bounded within [0, 3), though is best chose one for which (Y, Y) + 0 as + Y. Here we highlight a couple of the most natural and useful:
Pointwise metric P = [[M), (81ll where [У] and [9), correspond to the pixel color of Y and ÿ at pixel position p (and thus the 1-norm is necessary condense their difference into single number).
'Technically. Theorem 2.1 holds under the weaker assumption of exchangeability, with the same proof. We stick to i.i.d. for simplicity.

---

0.15
100%
32-pxchowide Nur For
100%
32-p 30-pexcl-wide blur for
Average Mask Size
Average Mark Size LLmorr (stttrl
11 beteer
[Looor is homer
64 siepleel.nddee buur tor Dp
75%6
sk-pexel--wid hiur for
0.10
Fidelit
W/o method
0.05
Non-temantic D)
25%
25%
Semantic D2
0%
0%
0.2
0.4
0.0
0.1
0.2
0.3
0.2
0.4
Fidelity
Ficciity Level
Fidelity Levd (o))
Figure 2: Our method provably controls the fidelity error with accuracy. Left: The figure shows the non-semantic fidelity error obtained by our method for varying fidelity levels for calibration with both the semantic D, (orange) and non-semantic D, (blue). As is shown in our theoretical guarantee, the error is tightly controlled by our method, being at most Center and Right: The plot displays the size of our confidence masks (i.e., how much of the image we do not trust) for varying fidelity levels a, for calibration done with semantic Dp (Center) and non-semantic D, (Right). As a increases our masks get smaller, eventually reaching zero, ie., trusting the whole image.
hewhhbbroood-avvrraeee metric D, I|Y K], -Y * K]pll, where K is some con- volution kernel corresponding to low pass filter. This makes it that single wrong pixels in the midst many correct pixels do not influence the loss function too much, and generally leads to larger confidence masks.
In both cases, we consider the images in Lab color space. This ensures that all the color comparisons being done are perceptually uniform, which would not be the case in e.g., sRGB space.
Though perfectly valid and useful, it is well known that such pixel-wise comparisons struggle to capture semantic and perceptual properties of the underlying images. Hence, both the options presented above struggle truly capture semantic differences, where user would clearly note difference between the predicted and ground truth images. To this end, we propose a third option based on additional labelled data:
Semantic metric We can suppose a stochastic function S [0, 1/wxkkxx3 [0, 1] x >>
{0, j that indicates mask produced by a human-being denoting the differences between the two given high-resolution images value of I on the image represents differing point). We can then consider (Y, Y) = [S F)]p Note that for the calibration procedure we only need to compute the Dp the calibration images, and thus the only samples of the human annotations S(Y, Y) that we need are on the calibration data.
Additional theorical results
In this section we present additional theoretical properties enjoyed by our method, which highlight its flexibility and robustness.
3.1 Our method provably controls PSNR
So far we have only proven results for the 'fidelity error' defined Subsection 2.1 However, our results can also be directly mapped to more familiar metrics for image quality quantification. In particular, we can prove strong guarantees on the PSNR of our predictions, a common metric of image fidelity and quality in computer graphics:
and n E N. Suppose we have n + lii.d. samples (X, Y:)=+! from an arbitrary probability distribution P and let ta and M. be as in Equations 7 and 2 (and thus only function of X,, Y, with i 1....,,7, nt). Then holds that
Proposition 3.1. Let
E Ex.. [SNRR (u(Xn+1), Yn+1/Ma (Xn+1))l -20 log a.
rather remarkable that, despite our procedure being originally designed in order to establish guarantees for uncertainty quantification, it also maps over to guarantees on standard image quality
5

---

Prediction conformal mask with 64-pixel-wiide Gaussiun blu-
Pixel difference (pred g(()
Low resolution
Ground truth
Figure 3: Unreliable predictions are accurately detected. This figure shows a close-up example where D, is non-semantic (as shown in Figure [}. highlighting failure of the base model to reconstruct a blurred area. Our method correctly identifies this failure and assigns low confidence to the affected region, where the predicted image deviates from the ground truth.
metric. The relative functional simplicity of the PSNR may be contributing factor to this result, but we expect that similar finds will soon follow for other metrics (though the proof would be more involved; the full proof of Proposition 3.1 can be found in the supplementary material).
3.2 Under data leakage
One crucial assumption of Theorem 2.1 is that we assume that the calibration data is independent of the base diffusion model i.c., that the calibration data is independent from (or least exchangeable with) the data used to train the diffusion model.
Though arguably achievable through the collection of new data for calibration purposes, this is considerably harder to ensure when using foundation models which have been trained on massive datasets that attempt span all data the internet. Hence, it becomes essential to explore what happens when there is data leakage from the training data to the calibration data - i.e., some amount of data in the calibration samples is actually already present the training data.
Proposition 3.2 provides worst-case bounds on the miscoverage error when there data leakage. In particular, consider that out of the calibration samples, Nllekeed < actually drawn from the training data (or some other arbitrarily different data distribution), while the remaining Nnew Teeakeed are truly independent of the training samples.
R and E N. with neew + Mleaked Suppose fit procedure as per Equations]] [ and 2 with n data points. Out of these n data points, suppose that the first nee sampled from some arbitrary probability distribution P. and the latter samples (indexed by Nnew + 1,1..., n) be sampled from some arbitrarily different probability distribution Q. Then, as we take sample Xnt1, Yut1 from distribution P, it holds that
Proposition 3.2. Let
nne + Tleakedd + 1
E
EX.
sup
Dp (m(Xn+1)) Xn+1) S a Nhew +
P. (Xn+1)
Note that Q could be the empirical distribution of the data used to train the base generative model. This result shows that our conformal prediction scheme (and split conformal prediction in general) is somewhat robust to data leakage, long as it is not too severe in relation to the amount of calibration samples.
4 Experiments
We dedicate this section to the empirical evaluation of our method, demonstrating its effectiveness.
Data All experiments were conducted using the Liu4K dataset [Liu et al. 2020 which contains 1. 1,600 high-resolution (4K) images in the training set and an additional 400 4K images in the validation set. The dataset features a diverse collection of real-world photographs, including scenie landscapes, architectural structures, food, and natural environments. We use the training set for calibration

---

Semantic Dp
Non-semantic Dp
Fidelity Level
Avg. PSNR
Avg. Fidelity Error
Avg. Conformal Mask Size
Avg. PSNR
Avg. Fidelity Error
Avg. Conformal
Mask Size
0.075
32.75 + 1.55
0.03 = 0.01
0.77 1: 0.07
30.23 + 1.12
0.07 = 0.01
0,43 1 0.09
0.100
32,65 + 1.48
0.03 = 0.01
0.73 + 0.07
28.64 + 0.93
0.09 = 0.01
0.23 : 0.06
31,63
1.32
0.05 0.01
0.60 + 0.08
26.82
1.03
0.14
0.00 0.00
0.300
30.86 + 1.29
0,06 0.01
0.50 + 0.08
26.82 + 1.09
0.14 0.01
0.00 + 0.00
w/o our method
26.83 + 1.06
0.14 + 0.01
N/A
26.82 + 1.08
0.14 + 0.01
N/A
Table Quantitative evaluation of our method under semantic and non-semantic settings. We evaluate the average PSNR, fidelity error, and averrage conformal mask size across different fidelity levels (a) for both semantic and non-semantic D,. For reference, we also include a baseline without our method (i.e., trusting the whole image). Overall our method tightly controls the fidelity error and PSNR while producing precise and informative masks.
38
0.200
0.175
PSNR
[Higiee is bette))
Fiddlyy Timer
LLower bemtrr
0 0.190
0.125
201
Web raehed
Lowerr bound (Phop 3.1)
We paisoring
hound (Prop,
Scmrrnic D.
Nie semoniii D,
Filelit ox)
Loaked Ammount (fawtte))
Figure 4: Our controls the PSNR and is robust and under data leakage. Left: This experiment confirms that the PSNR is theoretically bounded, as established in Proposition [.1, Additionally, PSNR within the conformal masks-in both semantic and non-semantic consistently higher than the baseline, indicating improved prediction quality in trusted regions. Right: This plot illustrates that fidelity error can bounded under data leakage, as per Proposition In both plots, the values plateau once the method reaches the point of trusting the whole images.
remains
procedures, and the test set for evaluation and metrics. We always work in Lab color space for its perceptual uniformity.
Base Model We perform our evaluations atop SinSR (Wang al., 2023 a state-of-the-ar genera- tive super-resolution method based on diffusion models. It performs super-resolution by conditioning the score function on a low-resolution image and applying diffusion in the latent space.
Intel Xeon E5-2696 v2 processor (2.5GHz base, 3.6GHz boost, 18 threads available) with 40GB of RAM and an NVIDIA RTX 6000 Ada Generation 48GB GPU. Notably, the primary computational bottleneck is the inference process of the base diffusion models, while the conformal calibration step is highly efficient and runs fairly quickly on a CPU. For reproducibility, the source code is available in hatpss:/ggitai com/ adwaasalmasleeeea wwwlllllleeeeeeeeeeeeeeae experiieeemm ts-conf
Compute Experiments
concoooooaal-. ormal superres -upprrrr as well as in the supplementary material.
Baselines
We compare our method to the following prior work:
No uncertainty quantification: this correponds trusting the whole generated image. In terms of confidence masks, it is equivalent to producing an empty mistrust mask.
[Angelopoulos et al., 2022b]: the output of this method is an image with interval-valle pixels, with a guarantee that, with high probability, the expected value of the percent of pixel intervals containing their true values will be at least (1 a)).
Kutiel et al., 2023]: the output of this method is a continuous score for each pixel (which they call a 'mask," though their use of this term differs from ours), which is higher for regions of the image we can trust more. The idea is that when reweighting the pixels by
7

---

Prodiction with conformal mask non non-semattic D2)
Prediction conformal mask semantic D2)
Prediction with conformal mask [Kutiel al., 2023]
Interval size heatmap [Angclopoulos al., 2022b]
Pixel difference (ppred ge)
Figure 5: Qualitative comparison to prior work. This figure compares our method both semantic and non-semantic D,, under the same settings as Figure [ against the methods of [Angelopoulos et al. 2022b] and [Kutiel al.) 2023). While our contormal masks highlight precise regions of uncertainty in an interpretable way, the method from [utiel et al. [023] produces continuous masks that closely mirror the original image rather than doing proper uncertainty estimation. Similarly, the heatmaps from [Angelopoulos et al. 2022b do not visually convey uncertainty, making their interpretation more challenging.
this predicted score, the LI loss of the generated image against the predicted image is, with high probability, at most some in expectation. However, they do not prove such validity property, and indeed it generally need not hold (see the supplementary material).
The prior work of [Angelopoulos et al. 2022b and (Kutiel al. 2023 produce pixel-wise intervals and scores, respectively, and are thus not directly comparable to our binary mask-based uncertainty quantification methodology. For this reason, on these two methods we are limited to a qualitative evaluation, which we do in Figure [ That aside, we explore our method and show that it is strict improvement over doing no uncertainty quantification.
In Figure 2 we analyze how well our procedure controls the fidelity error in practice, and how big the confidence masks get. We note that our theorems translate excellent empirical performance, with the actual fidelity error closely following the specified fidelity level a. Indeed, the two are essentially equal up until becomes large that we trust the whole image, which point the fidelity error stays fixed. As for the mask sizes, they steadily decrease as a grows, remaining informative for all but the most extreme levels of a. Table reveals similar patterns.
As observed in Figure 3 our method successfully generates accurate confidence masks even when the base model fails in the super-resolution task. This applies not just to perceptual attributes such as color and brightness but also to cases where the original image blurred; in such scenarios, the base model often misinterprets the blur loss of higher frequency, and hallucinates. Fortunately, our method effectively captures these nuances, preserving essential details and producing faithful, trustworthy results.
In Section 3 we' 've theoretically shown that our method provably controls PSNR and has a certain robustness to data leakage; seek to empirically verify this. Figure 4 left) shows the PSNR
8

---

1-pixel-wide for Dr
16-pixel-wide For Dr
32-pixel-wide For Dr
64-pixel-wide for D;
16 pippili widd blur for 631
3i-ppall-uie wiide blur for 631
piipel wide blur for m1
Figure 6: Blurring improves conformal masks. Red regions correspond the conformal confidence masks, denoting where we mistrust our predictions. Note how, as the radius of the Gaussian blur for D, increases, does the coverage of the trusted regions; and, the radius of the Gaussian blur for increases, not only does the trusted cover also increases, but its regions become more contiguous and visually appealing.
obtained by our method (i.e., the PSNR the reconstruction constrained to the region of the image trust) varying fidelity levels, along with the PSNR obtained the base reconstruction and our bound from Proposition 3.1 We see that our bound is somewhat lax - especially for higher levels seeks to empirically measure our method's robustness
of a but it valid. Similarly, Figure to data leakage. For each level of data leakage (Meakee)) we run our method's calibration with Tijaked of our samples being "perfect predictions" where D, 0, i.c., samples that are perfectly overfit on. Again we see the validity of our bound, and note that is more precise for smaller amounts of leakage. After certain point the fidelity error stabilizes the highest possible fidelity risk (i.e., the fidelity risk when we trust the whole image).
We also investigate the size of the predicted confidence masks, crucial aspect in assessing the reliability of our method. Notably, the choice of D, plays a fundamental role in determining how much confidence
can assign our predictions. If Dp is too local (e.g., defined simply as (Y, = I|[Y)p - I11), the model lacks sufficient spatial context to make robust assessments, resulting in overly conservative confidence estimates and smaller masks.
By incorporating additional contextual information into Dp- such as smoothing the pixel-wise differences with low-pass filter, we observe substantial improvement in mask size and quality. This approach allows for larger, more informative confidence masks, as it accounts for a broader spatial region rather than relying solely on isolated pixel differences. As result, the predictions not only become more faithful to the underlying image structure but also exhibit greater visual coherence. Empirically, this effect is evident in Figure 6 where the use of smoothed D, leads to masks that better capture the regions of uncertainty while preserving the overall perceptual integrity of the image. These findings reinforce the importance of designing D, to effectively balance local accuracy with global consistency, ultimately enhancing both interpretability and reliability in uncertainty estimation.
Finally, we compare against all of our baselines in Figure [5] Our outputs are much more immediately interpretable and useful than those of (Angelopoulos et al. 2022b), whose interpretation is tricky due to the combination of a lower+upper image. Similarly, note that though the pixel scores of [Kutiel al. 2023] are comparatively more interpretable, they are less informative as they follow the original image too closely, whereas selects meaningful regions of the image.

---

Conclusion
In this work, we have presented a new method for performing uncertainty quantification for image super-resolution based generative foundation models endowed with statistical guarantees. Our method requires only easily attainable unlabeled data and is adaptable over any base generative model, including those locked behind an opaque API. We also prove that our proposed solution satisfies properties beyond that of conformal risk control, further strengthening it. We expect our method to be broadly useful in practice, including in scenarios beyond image super-resolution.
Nevertheless, our method has a few key limitations: for one, we assume that our calibration data exchangeable; effectively, this corresponds to giving
together with sample from the test guarantees for in-distribution behaviour. Though conformal prediction methods are known to be relatively robust some changes in distribution, significant distribution shift may void our guarantees in principle. Moreover, it is worth noting that our algorithm requires the generation of multiple high-resolution images, which can be significant increase computation time. Overall, believe these to be exciting directions for future work.
Impact Statement
This paper presents work whose goal is advance the field of trustworthy machine learning. However, it is important to note that advances in image super-resolution technology can have significant societal impacts. For example, as much as our work enables the trustworthy use of super-resolution methods in crucial fields such as medical imaging, it could also be leveraged for harmful tasks such as reversing image censoring (e.g. censoring via blurred pixels). But we highlight that our approach's 's capability is generally constrained by the capabilities of the base super-resolution model, which we believe limits the negative impact of our work.
Acknowledgements
This project is part of global initiative funded by the International Development Research Centre (IDRC) on AI for Global Health, in collaboration with the UK International Development. GTG acknowledges partial funding from FAPERJ.
References
Maniraj Sai Adapa, Marco Zullich, and Matias Valdenegro-Tore Uncertainty estimation for super- resolution using esrgan. ArXiv, abs/2412.15439, 2024. URL wwt.ppssmmatiiccho.l....
Orr//oorussD:227965000
Eirikur Agustsson and Radu Timofte. Ntire 2017 challenge on single image super-resolution: Dataset and study. In The IEEE Conference Computer Vision and Pattern Recognition (CVPR) Workshops, July 2017.
Anastasios Nikolas Angelopoulos, Stephen Bates, Adam Fisch, Lihua Lei, and Tal Schuster. Confor- mal risk control. ArXiv, https/2208.028144, 2022a. URL https:///pp... www.sticsmcoooooo.......o httpp:::/app..smmttttccchheeeeee
org/
CorpusID:251320513
Anastasios Nikolas Angelopoulos, Amit Kohli, Stephen Bates, Michael Jordan, Jitendra Malik, Thayer Alshaabi, Srigokul Upadhyayula, and Yaniv Romano. Image-to-imadt regression with distribution- uncertainty quantification and applications in imaging. ArXiv, abs/2202.052655. www.mtiisccooli......mmm.
2022b. URL htttps://app http://appp.semtnicccccaaaa
warg/CorpusID/... 246706224
Daniel Csillag, Lucas Monteiro Paes, Thiago Rodrigo Ramos, João Vitor Romano, Rodrigo Loro Schuller, Roberto B. Seixas, Roberto Oliveira, Paulo Orenstein. Amnioml: Amniotic fluid segmentation and volume prediction with uncertainty quantification. In AAAI Conference on Artificial Intelligence, 2023. URL. httpps::: api www.nti wwwpsschhhhaaaroo//ooooooooooooe org/CorpusID
259282132
10

---

Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In International Conference Machine Learning, 2015. URL
https://api. semanticscholar. org/CorpusID: 160705
Ethan Goan and Clinton Fookes. Bayesian neural networks: An introduction and survey, ArXiv, 0es/2006.120244, 2020. URL DLLLaattpee//gg..smmmmiiicoooooo .ogaaar..rroooeeeeee wrggCorpusuuo.o 219873953
semanticscholar
Xiaoyang Kang, Tao Yang, Wenqi Ouyang, Peiran Ren, Lingzhi Li, and Xuansong Xie. Ddcolor: Towards photo-realistic image colorization via dual decoders. In Proceedings of the IEEE/CVF
International Conference on Computer Vision, pages 328-338, 2023.
Gilad Kutiel, Regev Cohen, Michael Elad, Daniel Freedman, and Ehud Rivlin. Conformal prediction masks: Visualizing uncertainty in medical imaging. TML4H, 2023. URL URL httpp::/app..sem nticscholar org/ : 259311276
J. Liu, D. Liu, W. Yang. S. Xia, X. Zhang, and Y. Dai. A comprehensive benchmark for single image compression artifact reduction. IEEE Transactions on Image Processing. 29:7845-7860, 2020.
Luca Mossina, Joseba Dalmau, and L'eo And' col. Conformal semantic image segmentation: Post-hoc quantification of predictive uncertainty. ArXiv, abs/2405.05145, 2024. URL https://appis sema
nticscholar wwg/ Chrp/orrpsssDDCC 269626733
Elias Nehme, Omer Yair, and Tomer Michaeli, Uncertainty quantification via neural posterior
principal components. ArXiv, abs/2309.15533, 2023. https:/aaapa stw.aapisssmmmiiccccccccooooooo
hts/CooppuID:223155500
Jacopo Teneggi, Matthew Tivnan, J. Webster Stayman, and Jeremias Sulam. How to trust your diffusion model: convex optimization approach to conformal risk control. International Conference on Machine Learning, 2023. URL http:::t///// //api. semanticscholes www.aroor//rpuuuuueee 1D:266626778
Radu Timofte, Eirikur Agustsson, Luc Van Gool, Ming-Hsuan Yang, Lei Zhang, Bee Lim, et al. Ntire 2017 challenge single image super-resolution: Methods and results. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, July 2017.
Vladimir Vovk, Alexander Gammerman, and Glenn Shafer. Algorithmic learning in a random world. 2005. https///ppp semanticscholart org/ wwt.:ooooooooi::111111110..
Yufei Wang, Wenhan Yang. Xinyuan Chen, Yaohui Wang. Lanqing Guo, Lap-Pui Chau, Ziwei Liu, Yu Qiao, Alex Chichung Kot, and Bihan Wen. Sinsr: Diffusion-berd based image super-resolution in single step. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 25796-25805, 2023. URL https://apt semanticscholar org :
113

---

A Proofs
Proof of Theorem 2.7 This proof is done via the standard conformal risk control argument An gelopoulos al., 2022a].
Consider the "lifted" threshold
(x+ sup (ERUUHO0):
Σ
D2 (u(X,),)))) S sa
sup A (X)))
which, as opposed to their "unlifted" counterpart fo. leverage the (n + 1)-th sample as well and does not include the 3/(nz + 1) term.
Note that it is certainly the case that to (x+1) 2 to. Moreover, note that the fidelity error is monotone with t, and so suffices to show that the fidelity function with g(n+1) is upper bounded by a.
Let Z, be the multiset of the samples (X,, Y) - i... random variable representing the samples, but with their order discarded. Hence, upon conditioning on Z., all randomness that remains that of the order of the samples. It then follows:
E sup
Dp(u(Xn=1), Xn+1) Z.
Σ
sup pi s1) 2=1
(p(u(X))) YJ),
12)
and by the definition of t (n+1) this is upper bounded by a. Thus
E sup
Dp Po(n(Xn+1)) = Ez.
E
sup
Dp(u(Xnt1). Ynt1)
Inlo(Xn+1)ee
(1+11) I Ez, [a] a,
(x
which concludes the proof.
Proof of Proposition 3.1
The PSNR are bounding is given by
(maxpeM, (X
E|PSNR (n(X,,1)) Y, Y = = E 10logio
[M |Ma((x+1)l-- EpeMs ([M(Xn+1)lp -Y,)²
max Mo (X (X
20E
log10
Ma (X Epe M. (X.+1) (Iu(Xn+1)lp Yp)
Now, by Jensen's Inequality and standard properties of logarithms,
max pe M. (X (X++1pp
20E
logi0
(M EpEMa No (Xn+1) (1u(Xn+1)lp - Yp)²
20
max logio
[Y
logio
|a((X
Σ
(lu(X++ Yzy² xx31²/
M (X
(X
20
logio
max
(n+zlpp
E
logio
[M [Mn(Xn-1)1-
Σ
([n(Xn+1)l )2
Pe M. (X.+1)
DEMa(Xa+H
2 20
log10
max
[Yn+1h
- -10810 E
[Na((1
Σ
(l((Xn+1))) Ar - Xx)
pe,,(X,,,, (X,11))
PEM,.X..11)
12

---

And, because the RMSE is upper bounded by the maximum error, we get that
20
E
log10
max Pe Mo (Xn+1)
loggo E
(a
E
(l((Xn+1)ls x2)²
DeMo(Xa+))
2 20
E
max log10
1n-1pp
logau L
sup (xMu(X.11))
(lu(Xn+1)lo - Xp)
And, by Theorem 2.1
20
E
max logio
(Ya+ilo)
log E
sup
(u((X++))o - Xp)
PEMa( Ma (x.) K
(
2
E
logio
max Mo Mo (Xu=1)
[Y [8+tpp
logio a Σ -200log10 a,
where the last step holds as long all pixel values are in [0, 1].
We effectively want to bound the supremum of the expected fidelity error as the leaked data allowed to alter freely. For convenience, let sUP denote the supremum over possible values of the leaked samples (X,, Yi) +1 (and infleaked the corresponding infimum). Note that the error function decreasing on the selected parameter and continuous. Hence:
Proof 3.2
3.2
sup E sup leaked
D) (x)
< E
sup
sup
D2(X,
= IE
D2(r,8)
leaked Pe Ma (X)
( tor
and in turn
n+1 3 Sa
sup to sup sup leaked
t EuUofto))
n 1= mlo Dp(Y..(X.)))
sup
leaked
3
S sup
t t t
inf leaked n pl((X))
sup
Dx(Y..(X,))
Sa
n+1
Nnew
sup
t R [+00]
sup (X)ls D,(Y,,u(x,))
1
3
inf
(2(Y,((Xi))+ + n+1 sa i=Raew (X,)l
sup
leaked
n + 1
3
n + = sup EERRU
==1 pl((X)))sst Dp(Y).u(x))) +
sup
S a
n+
1
nhew
3
n+1
= sup t RU
sup (X
(X(,).(((i))
S a
7new
Neew +
Nhow + 1
Note that this corresponds doing our calibration procedure only the data but with altered fidelity level a (n + (n/ + +1) = a (Nnew + Meaaked + 11)()(nnaaaa + 1), and so, by the same arguments as in Theorem 2.1
Onew + Mllaaeed +
E (x.Y
Dp (1(X Yn+z) pe (X.1)
sup
Sa.
nnew + 1
13

---

B On Kutiel et al., 2023
The method of [Kutiel et al. 2023 produces, given a low-resolution image in 10, 1|wxXx3 and an initial estimate of uncertainty o in I 22
kw a predicted high-resolution image Y in [0, 1 1wwxkkx3 along with a continuous 'confidence mask' M in [0, 1] where higher values denote higher confidence at that region of the image. Ideally, this continuous mask would be such that it would satisty an RCPS-like guarantee
P
E
IiMlss in
[aa
Sa
2 1
low lch
for some chosen a and 5. This goal is enunciated in Definition 2 in Kutiel et al. calibration procedure is then presented in Section 4.2 of their paper, it does the following:
2023
Their
1. For each I..... n, compute
M min
los +
EiMeWees 1 [Yh,) s }
A max
A kw kh
i,j
2. Compute À = - An).
3. The resulting "calibrated" masks are produced by (A),
However, they do not prove that this procedure satisfies Equation 3 other than passing mention at the end of their Section 4.2. Indeed, the guarantee actually does not hold: intuitively this should be fairly immediate:
They write "Finally, taken to the 8 quantile of {A=] i.e. the maximal value for which at least fraction of the calibration set satisfies condition (5). Thus, assuming the calibration and test sets are i.dd samples from the same distribution, the calibrated mask is guaranteed to satisfy Definition 2." However, their condition (5) states that
IF
EIM (M [Mha)] 1.
Sa.
kw khh
with an expectation this expectation is fundamentally absent in the calibration procedure.
2. Additionally, the quantile taken simple empirical quantile. But the guarantee we want that the populational risk is bounded. To satisfy this, one would need to slightly tweak the quantile; this is analogous to how in RCPSs you would apply concentration inequality (e.g. Hoeffding).
Indeed, we provide here a counterexample, in which their procedure does not satisfy their stated guarantee.
Thh difference is the presence of the normalization Jw kR. which is absent in their paper (they refer simple 1-norm). However, we found necessary order for their method to function, and conjecture that was accidentally omitted their work.
Modulo ajustments notation
14

---

Example B.1. Suppose that a is a deterministic O mask for all inputs. Then, to show that Equation does not hold, we just need that
ElMas |Mas - Mesl
S a
< 1 -
kow khh
1
Σ
[alla 118100 - Mrd
> a
> δ
low kh
2.3
P
[M]
- Maol
a
28
kew kh 1g
P
E
ElM MMaal
6,
low kh
[M
for and 5. Further suppose that there is some nonzero probability that lew kh Eigl(Mas
[Yki = O for all samples in the calibration set. Then, with this nonzero probability, A
1 regardless of 8, implying that [M], = 1/(1 + c). By chosing = r/2 and a S
1t E
Joa KR En,10 - Masl] we conclude.
2
C
Pseudocodes
Conerete implementations of these algorithms can be found in code.
Algorithm Conformal mask calibration (computation of t..) with dynamic programming
T + all unique values of oXi)) (Xi))..., (Xx))
T ST))
D + values of Dp each entry of T''
> can be computed jointly with the sorting indices of the original images for each entry of D can be computed jointly with the sorting risk so far on each observation total risk so far
R; +- 0
R n+I
t. &- -00
NA
last threshold seen far
for i L...., nd do
Tf + l then
if R < then
t, +r
end if
end
1 -- T;
r <- max R1,: Di}
R R- R
n+1 Rí, a+1
R, t-r
end for return t,
15

---

Algorithm 2 Conformal mask calibration (computation of (.) with a brute force search
T <- all unique values of (Xx))....,o(())
4. < -0 for / E do
compute risk <
1 x+1 Disl sup st (o((,,((X, + -
< a then
l max
end end for return t,
D Results under Distribution-Shift
Semantic Dp
Non-semantic Dp
Fidelity Level
Avg. PSNR
Avg.
PSNR
0.075
13.34 6.91
13.14 + 6.55
a 100
13.28 7.30
12.36 + 4.47
0.125
13.01
7.11
12.30 3.81
W/o our method
11.73 = 0.39
11.74 + 0.39
Table 2: Quantitative evaluation of our methods under distribution-shift. We evaluate the average PSNR across different fidelity levels (a) for both semantic and non-semantic Dp. as calculated on T We utilized the calibration performed on the LIU4K dataset calculate this metrics on the validation set of DIV2K [Agustsson and Timofte 2017 Timofte et al. (017)] with 100 samples from a different distribution. Overall our method controls the PSNR point-wisely, but the intervals show that performs better an in-distribution scenario.
E
On Metric Comparisons with Baseline Methods
Direct metric comparisons with the baseline methods of [Angelopoulos et al. 2022b and [Kutiel al.) 2023] present fundamental challenges due to output modality differences. These methods produce continuous uncertainty scores rather than binary masks, making objective metric comparisons infeasible without modification. While one could threshold their outputs to obtain binary masks, doing so fundamentally undermines their statistical guarantees, as these methods were not designed for binary mask production. Furthermore, the thresholding process itself introduces methodological complications: for (Angelopoulos et al. 2202b), natural threshold at exists, but for [Kutiel et al. 2023), threshold selection requires an additional calibration split, introducing another degree of freedom that complicates fair comparison.
Given these concerns, we argue that forcing these methods a common evaluation framework would produce misleading comparisons that do not reflect their intended use cases. Instead, we present qualitative visualizations that respect each method's design principles while highlighting the distinct advantages of our approach-namely direct binary mask generation with maintained statistical guarantees and substantially tighter confidence regions (mean mask size 0.73 0.07 for our method at 0.1).
F Results on Image Colorization
Here we showcase direct adaptation our method for image colorization rather than super- resolution. The method works similarly, suggesting broad applicability beyond our domain.
16

---

Prediction with conformal musk 0.0875)
Prediction with conformal musk 0.1)
Prediction with conformal mask (a 0.125)
with wixde--wide
Gray
Figure Results of our method adapted for image colorization. This figure presents comparison of multiple color-restored images generated using the DDColor base model (Kang et al. 2023
alongside their corresponding conformal masks for a E (0.0875, 0.1, 0.155) and non-semantic Dj- Our conformal masks accurately highlight regions where color predictions significantly deviate from the ground truth, effectively capturing uncertainty related hue variations, saturation inconsistencies. and lighting discrepancies. This demonstrates the versatility of our theoretical framework, indicating can be directly extended other image restoration tasks beyond super-resolution.
17

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
All claims are appropriately justified in the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
All assumptions were stated where appropriate.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
18

---

Proofs can be found in the supplementary material, and all assumptions were appropriately stated.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Details for reproduction can be found in the supplementary material.
Guidelines:
The answer NA means that the paper does include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
19

---

Answer: [Yes]
Justification: The code to reproduce all experiments can be found in the supplementary material and will be made available through GitHub. Dependencies are appropriately managed and documented.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please the NeurlPS code and data submission guidelines (https://nips cc/pu blic/guise gggiddes//odeuuinissssiio..
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https for more details.
/ //nips.c///ppp
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Data and procedures are described in the main text, with details provided in the supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We provide bootstrap CIs where appropriate for experiment results. All Cls are over the randomness in the data.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
20

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Described at the start of Section
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make it into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: The topic not particularly sensitive, and all data comes from open datasets. There are no ethics violations.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: The paper is about uncertainty quantification for image super-resolution. There many potential societal consequences of work, none which feel must be specifically highlighted here.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
21

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Given the lack of immediate negative impacts, do not need for safeguards for our work.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All datasets are open and were appropriately cited.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
22

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thoode. canddataeet has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
No new assets were released.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: There was no crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: There was no crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
23

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: No significant use of LLMs.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
24


Reviews:
[{"review_id": "rkvYvuhMdX", "reviewer": "Reviewer_g2HS", "summary": "The authors work towards answering the interesting question of how to provide guarantees and interpretability for the output of diffusion-based generative models. Specifically, the authors propose a simple but effective method to quantify the uncertainty of images that are inpainted or super-resolved using diffusion models. The authors provide statistical guarantees their marginal conformal prediction will be bounded in expectation. The authors provide empirical evidence showing the effectiveness of their method and how tight the bounds are.", "questions": "## Questions\n\n1. **Clarification of Proof Technique**\n   Could the authors elaborate on the significance of their proof technique and explain how it differs from the original work by Anastasios et al?\n\n2. **Tightness of the Bound**\n   Could the authors comment on the tightness of their derived bound and discuss the main challenges involved in improving it?", "limitations": "NA", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "## Strengths\n\n1. **Simplicity & Interpretability**\n   The proposed algorithm is straightforward to implement and represents an important step toward greater interpretability in diffusion-based generative models. It also raises interesting philosophical questions about the trade off between fidelity and uncertainty (see Figure 2).\n\n2. **Empirical Validation**\n   The authors back up their theoretical claims with solid experimental evidence, demonstrating that the derived bounds are indeed appropriate in practice.\n\n3. **Novel Application of Conformal Prediction**\n   To the best of my knowledge, this work is the first to bring conformal prediction techniques into inverse problems specifically inpainting and super-resolution for diffusion-based generative models.\n\n\n## Weaknesses\n\n1. **Proof Novelty Is Unclear**\n   Based on the Appendix, it’s difficult to distinguish the contributions of this proof from those in prior work by Anastasios Nikolas. The paper would benefit from a clearer discussion of what is fundamentally new.\n\n2. **Loose Bounds**\n   The bounds illustrated in Figure 4 appear rather loose, which makes it challenging to assess how informative they are in practice. Tighter bounds or additional commentary on their practical implications would strengthen the paper.", "was_revised": true, "final_justification": "The paper presents strong empirical results; however, its theoretical contribution lacks novelty and is largely a direct application of prior work outlined by the authors. The authors did not address concerns about the originality of their proposed theoretical component, despite it being a major part of the paper. Nevertheless, given the strength of the empirical findings, I lean toward accepting this work.", "rebuttal": "Thank you for your comments. Please refer to our responses below.\n\n**On the novelty of the proof:** Theorem 2.1 indeed follows the arguments from [1], as we state in the proof. The novelty is mainly due to our choice of modelling and definition of the risk, which enables efficient computation and strong guarantees in our setting (there are also some minor differences regarding e.g. the direction of the continuity, but this is trivial). Theorem 2.1 aside, our Propositions 3.1 and 3.2 are entirely novel, and show that our method has desirable properties specific to our context, beyond those shown by [1].\n\n**On the tightness of our bounds:** there are two bounds in Figure 4.\n\n- **Re. bound under data leakage:** note that the bound is reasonably tight up until a certain point, corresponding to when our method becomes confident in the whole predicted images. That said, our bound is fairly pessimistic, since we consider that everything derived from the leaked samples can assume its worst-case value. While this is possible, the typical case is probably not as bad. To improve the bound, we would need to take into account training dynamics of specific model classes, which is best suited for future work.\n- **Re. bound on PSNR:** perhaps the lossiest step in the proof is a use of Jensen's inequality, which could be tightened by e.g. use of the generalized Jensen inequality. However, by doing so the bound will reference some unknown quantities of the underlying data distribution, making it less interpretable. Either way, in our view the main insight given by this result is that as we are more strict with our choice of $\\alpha$, our PSNR increases accordingly, at a rate of $\\Omega(-\\log \\alpha)$ (which seems natural for the PSNR).\n\n[1] \"Conformal Risk Control\", ICLR 2024"}, {"review_id": "sQ5l1bDv64", "reviewer": "Reviewer_z7U3", "summary": "This paper addresses the need for interpretable uncertainty quantification in image restoration tasks, specifically focusing on image super-resolution using generative ML models. The authors propose a conformal prediction approach to generate a binary confidence mask that highlights regions where the generated image can be trusted. This method, adaptable to any black-box generative model, offers marginal coverage guarantees for a chosen local fidelity measure between the reconstructed and ground truth images, ensuring the expected maximum fidelity error across pixels is constrained by a user-specified level. They demonstrate that data leakage between calibration and training sets leads to a proportional increase in the fidelity level and establish that the method provides control over the PSNR conditioned on the mask. Experimental results, using the SinSR model on the Liu4K dataset, validate the method's effectiveness in quantifying and visualizing uncertainty in image recovery.", "questions": "* Please extend the related work and the comparison in the experimental section as mentioned in the weakness above.\n* Please elaborate on the point of generating M images.\n* The method seems general. Why restricting it application only to super-resolution?  \n* The use of semantic distance can be seen as providing per-pixel labels and then applying standard conformal risk control. Is this view correct?", "limitations": "See Questions and Weakness.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths: \nThe paper is well-written and presents rigorous derivations for a method that visualizes uncertainty through a binary mask. This mask offers clear visualization and statistical guarantees regarding the recovery error. A significant strength of this approach is its reliance on conformal prediction, which inherently allows it to be applicable to any black-box generative model.\n\nWeaknesses:\n* The related work section could be more comprehensive. Given that the paper's main contributions are the interpretable visualization of uncertainty and clear statistical guarantees for recovery error, I expected a more extensive discussion of existing approaches for uncertainty quantification and visualization, both those that offer statistical guarantees and those that do not.\n\n* The experimental evaluation appears limited. The authors do not compare their method to other competitive approaches for uncertainty visualization, even those that might not offer strict statistical guarantees (e.g. \"Uncertainty quantification via neural posterior principal components\"). Furthermore, the comparison to Angelopoulos's method is misleading; while that work did not explicitly aim to produce a mask, its pixel-wise prediction intervals could be used to generate a mask (as demonstrated in Figure 1 of \"Image-to-Image Regression with Distribution-Free Uncertainty Quantification\"). A more appropriate comparison would be against such a derived mask. Lastly, the comparison with Kutiel's method is unfair, as their continuous masks can be easily converted to binary ones via simple thresholding, an effort not undertaken by the authors.\n\n* The method's requirement to produce M samples for generating each mask introduces additional computational complexity and increased runtime. This aspect should be thoroughly discussed in addition to clarifying which of the M images (if any single one) is ultimately presented to the user.", "was_revised": true, "final_justification": "Thank you for the rebuttal. Adding comparisons to more suitable baselines has partially addressed my concerns. I still believe the paper would be significantly improved by evaluating the proposed methods against other visualization techniques, even those lacking statistical guarantees. Nevertheless, I acknowledge the authors' efforts and the paper's contribution, and have raised my score accordingly.", "rebuttal": "We appreciate your review. Please check our responses below.\n\n**On the comparison to other methods:**\n\n- Regarding the display of [Angelopoulos et al.] in our Figure 6, the reviewer raises a good point. We will add the \"uncertainty heatmaps\" from their Figure 1.\n- On thresholding continuous masks: we agree that this is an interesting baseline, but note that in doing this we lose the underlying statistical guarantees, since the methods are not geared towards producing binary masks. There is also some nuance on how this threshold is to be chosen: for [Angelopoulos et al.] there is a natural choice of thresholding at the same $\\alpha$ we use for our method, but for [Kutiel et al.] there is no such natural choice; we have to optimize for the threshold on an additional calibration split. Anyways, we'll add these comparisons. In preliminary experiments, we see that even when using their methods post-binarization, our method is substantially tighter (for $\\alpha$=0.1, our avg. mask size is 0.73 $\\pm$ 0.07, binarized [Angelopoulos et al.] is 0.85 $\\pm$ 0.18, and binarized [Kutiel et al.] is 0.88 $\\pm$ 0.12). We attribute this significant difference to the combination of our more direct statistical procedure with our modelling of $\\sigma$.\n- We'll also add some discussion on methods without statistical guarantees and compare to the suggested work, thank you for the pointer.\n\n**On the M images:** this is indeed an additional cost, which we will try to make more explicit; however, that seems necessary to obtain a natural estimate of the uncertainty of the generative model. That said, note that this is trivially parallelized. As for which image to be presented to the user, the method and guarantee hold for any selection rule, as long as it is consistently applied. In all our experiments we simply use the first generated image.\n\n**Why focus on super-resolution?** The method is general, but for the sake of presentation and notation we felt it necessary to focus on one specific setting. Since this work was originally motivated by hallucinations in recent super-resolution methods (which are now even shipped on consumer devices!), we decided to go with that. Note, however, that we also show some results on image colorization in the supplementary material.\n\n**Re. semantic distance and \"standard\" conformal risk control:** the use of semantic distance can indeed be seen as providing per-pixel labels, but we do not believe that our method reduces to any particular proposal from the original Conformal Risk Control paper, besides following the general framework.\n\n[Angelopoulos et al.] \"Image-to-Image Regression with Distribution-Free Uncertainty Quantification and Applications in Imaging\" \\\n[Kutiel et al.] \"Conformal Prediction Masks: Visualizing Uncertainty in Medical Imaging\""}, {"review_id": "e8Y2rxOWJK", "reviewer": "Reviewer_E7RA", "summary": "This paper introduces a conformal prediction-based method for uncertainty quantification in image super-resolution. The approach generates a binary \"confidence mask\" that identifies reliable regions in images upscaled by black-box generative models (e.g., diffusion models). The contributions include:\n\n(1) A calibration procedure using easily obtained unlabeled data to quantify uncertainty working atop any black-box diffusion model.\n\n(2) Theoretical guarantees for fidelity error control, PSNR bounds, and robustness to data leakage.\n\n(3) The proposed approach not only works on super-resolution, but other tasks such as colorization.", "questions": "My main concerns are as follows:\n\n1. How robust is the proposed approach given different σ (line108)? What if σ is poorly estimated or how to make sure a stable σ is obtained?\n\n2. The evaluation of this paper is somewhat weak in demonstrating the accuracy of the proposed approach. Specifically, if my understanding is correct, the evaluation is mainly based on visualization comparisons in Fig.5 and Fig.6. These visualizations are kind of subjective and lack interpretability. No metric evaluation or human evaluation is involved to strengthen the effectiveness of the proposed method.\n\n3.  The validation is simply conducted on the validation set of Liu4K, from the same source as the calibration data. The authors should consider testing on images from diverse sources to validate the robustness.\n\n4. In the supp. material, the interpretability of the predicted mask is not that straightforward. Ideally, the red regions should indicate that the color is obviously different from the GT. But from my observation, the unmasked regions are also very different from the GT. I am kind of confused if the predicted results are accurate.", "limitations": "No.\n\nThe paper does not discuss the potential limitations and societal impact in any subsection of the paper. The authors should consider adding such a subsection in the revision.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n+ The method provides provable statistical guarantees (Theorem 2.1, Propositions 3.1–3.2) for fidelity error, PSNR control, and data leakage robustness.\n+ Binary masks offer clearer uncertainty visualization than pixel-wise intervals/continuous scores (Fig. 6).\n+ Works with any black-box super-resolution model (including API-locked ones) and requires only unlabeled calibration data.\n\nWeakness:\n- The evaluation of this paper is somewhat weak in demonstrating the accuracy of the proposed approach. Specifically, if my understanding is correct, the evaluation is mainly based on visualization comparisons in Fig.5 and Fig.6. These visualizations are kind of subjective and lack interpretability. No metric evaluation or human evaluation is involved to strengthen the effectiveness of the proposed method.\n- The validation is simply conducted on the validation set of Liu4K, from the same source as the calibration data. The authors should consider testing on images from diverse sources to validate the robustness.", "was_revised": true, "final_justification": "I have carefully read the author's rebuttal as well as other reviewers' comments.\nI appreciate the theoretical analysis of this paper, which may introduce interpretability to the uncertainty of image super-resolution, a meaningful direction.\nOn the other hand, I am worried about the generalization ability, which seems hard to provide further verification in the submitted version.\nI hereby tend to maintain my rating.", "rebuttal": "Thank you for your feedback. Below, we provide our responses to your comments.\n\n**On our evaluation:** we have three baselines: (i) not doing any uncertainty quantification, (ii) [Angelopoulos et al.] and (iii) [Kutiel et al.]. However, since the output of the latter two are of different modality from ours (i.e., they don't produce binary masks), we couldn't identify an objective metric with which to compare them. That said, following the suggestions of reviewer z7U3 we will add metric comparisons to thresholded versions of the two methods, even though by doing so they lose their statistical guarantees. In our preliminary experiments, our method seems to obtain substantially tighter confidence masks (for $\\alpha$=0.1, our avg. mask size is 0.73 $\\pm$ 0.07, binarized [Angelopoulos et al.] is 0.85 $\\pm$ 0.18, and binarized [Kutiel et al.] is 0.88 $\\pm$ 0.12).\n\n**Evaluation under distribution shift:** that is an interesting suggestion. We will add an experiment assessing this to the final version. Either way, it is already theoretically known that conformal prediction methods are generally robust to distribution shifts, so we expect this to behave favorably.\n\n**On the image colorization results in the suppl. mat.:** the perceived differences are due to a choice of $\\alpha$ that is too high. Consider, for example, the image in the second row: since the predicted colored image is practically grayscale, the more saturated regions in the original have larger differences, and this is indeed what our method highlights. As the colors become less saturated, the prediction becomes more accurate, eventually being beneath the specified threshold given $\\alpha$; this leads to them no longer being highlighted. We will add some comparisons with other values of $\\alpha$ to the final version, which should clarify the accuracy of the method on all images.\n\n**Additional questions&points:**\n\n- Our guarantee that the fidelity error is controlled holds regardless of how well $\\sigma$ is estimated; however, using a bad estimate may lead to masks that are larger than necessary. Generally speaking, though, the method seems reasonably robust to different $\\sigma$. As for stabilization, note that the blur we apply on $\\sigma$ effectively stabilizes it, making it so that small changes to the estimate are smoothened out and do not significantly affect the rest of the procedure.\n- We'll add a section discussing societal impact & limitations to the final version.\n\n[Angelopoulos et al.] \"Image-to-Image Regression with Distribution-Free Uncertainty Quantification and Applications in Imaging\" \\\n[Kutiel et al.] \"Conformal Prediction Masks: Visualizing Uncertainty in Medical Imaging\""}, {"review_id": "6uPxV7ZvhQ", "reviewer": "Reviewer_cVuS", "summary": "This paper proposes an uncertainty quantification framework for generative image restoration models, leveraging conformal prediction to produce confidence masks that reflect the local reliability of generated pixels. The method is black-box compatible (requiring only outputs from the generative model), relies on a calibration dataset of high-resolution images, and supports flexible local similarity metrics for interpretability.", "questions": "1. How many samples are needed?\n\n2. How does domain shift affect performance?\n\n3. Does the choice of perceptual vs pixel-wise metric (e.g., LPIPS vs L1) affect the mask quality or calibration performance?", "limitations": "Yes", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1. The method works on black-box models, which is practical for commercial or API-locked systems where internals are inaccessible.\n\n2. The framework is grounded in conformal prediction, which gives valid coverage guarantees under minimal assumptions, and extends these ideas to the image domain with localization.\n\n3. The idea of per-pixel confidence masks aligned with perceptual similarity metrics is appealing and potentially actionable for real users or downstream systems.\n\nWeakness:\n\n1. While the application to image restoration is interesting, the core method is a relatively straightforward application of standard conformal prediction, adapted to pixel-wise outputs. The “confidence mask” is essentially a localized conformal interval—novel in context, but not in construction.\n\n2. The method assumes access to “a handful of unseen high-res images,” but doesn't quantify how much is sufficient or how sensitive the method is to domain shift between calibration and test inputs. This could be problematic for models trained on synthetic datasets and deployed on natural images.\n\n3. There’s no empirical evidence (at least from intro/abstract) that the confidence masks are intuitive or actionable for users. User studies or visual comparison with ground-truth errors would help validate this interpretability claim.", "was_revised": true, "final_justification": "I thank the authors for their rebuttal. I have carefully reviewed the comments of the other reviewers. While the work is limited to a specific setting, there are still concerns about its generalization performance. Therefore, I will maintain my original score.", "rebuttal": "Thank you for your review. We have a few comments:\n\n**On the number of samples and domain shift:** much like usual conformal prediction methods, our calibration has fast $O(1/n)$ sample complexity, and so few samples are needed. We find that, in practice, as few as 50 samples are already enough for good guarantees; in the paper, our experiments use just 200. As for domain shift, it's straightforward to theoretically see that our method should be fairly robust: by a change-of-measure via total-variation (TV) distance, we have that for any distributions $P$ and $Q$, $\\mathbb{E}\\_Q[\\text{risk}] \\leq \\mathbb{E}\\_P[\\text{risk}] + O(\\mathbb{E}_X[TV(P|X, Q|X)])$, i.e., it's at most as bad as the expected total variation distance between the distributions conditional on the low-resolution image. We can add an experiment to empirically assess this in the final version.\n\n**Quantitative evidence on intuitiveness/actionability of our masks:** thank you for the suggestion. Note that we already have some comparisons with ground-truth images in Figures 1, 3 and 6; we'll add additional subplots with the pixel differences between the predicted and ground-truth images to facilitate visual inspection.\n\n**Perceptual vs. pixel-wise metric:** Changing the metric can lead to significant differences in mask quality. Moreover, the guarantee offered by our method changes, as it fundamentally involves the metric. Generally speaking, we find that non-semantic metrics lead to smaller confidence masks, likely due to our modelling of $\\sigma$. As for calibration performance, both scale similarly with fast sample complexity of $O(1/n)$, and equal time complexity.\n\nFinally, note that our method is not quite standard conformal prediction, in which one would obtain predictive sets / conformal intervals (i.e., sets of predicted images), but rather an instance of conformal risk control, in which we have many more degrees of freedom on how to design our procedure. But yes, our main contribution is the use of such methods in our particular context."}]

Author rebuttal:

Meta-review: This paper is about applying conformal prediction to super-resolution and other image generation models, which produces mask revealing uncertain pixel regions, and works with black box diffusion models with some theoretical guarantees.

Reviewers appreciated the simple method, empirical validation, excellent paper presentation, statisical guarantees, and connection with perceptual similarity metrics. Some weaknesses are the need for calibration data and not evaluating how much calibration data is needed.

I also note that this paper does not make a full comparison to the state of the art, which might not be strictly needed since the use of a uncertainty mask is novel and not commonly done in the literature. The reference list is on the short side and can be improved with discussion/comparison with state of the art uncertainty methods, for example in particular this one for super-resolution:
"Uncertainty Estimation for Super-Resolution using ESRGAN" at https://arxiv.org/abs/2412.15439

Considering the many strengths and little weaknesses, plus the wide applicability of the proposed method to image generation in diffusion models, in my opinion this paper can be accepted. I recommend the authors to consider the reviewer's comments in their final version.
Decision: Accept (poster)