CONSIGN: Conformal Segmentation Informed by Spatial Groupings via Decomposition
Authors: Bruno Viti
Elias Karabelas
Martin Holler
Venue: ICLR 2026 Poster (2026)
Keywords: Conformal prediction
Uncertainty quantification
Image segmentation
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
TL;DR: We developed a conformal prediction strategy for image segmentation that accounts for spatial correlation to achieve improved results

Abstract: Most machine learning-based image segmentation models produce pixel-wise confidence scores that represent the model’s predicted probability for each class label at every pixel. While this information can be particularly valuable in high-stakes domains such as medical imaging, these scores are heuristic in nature and do not constitute rigorous quantitative uncertainty estimates.
Conformal prediction (CP) provides a principled framework for transforming heuristic confidence scores into statistically valid uncertainty estimates. However, applying CP directly to image segmentation ignores the spatial correlations between pixels, a fundamental characteristic of image data. This can result in overly conservative and less interpretable uncertainty estimates.
To address this, we propose CONSIGN (*Conformal Segmentation Informed by Spatial Groupings via Decomposition*), a CP-based method that incorporates spatial correlations to improve uncertainty quantification in image segmentation.
Our method generates meaningful prediction sets that come with user-specified, high-probability error guarantees.
It is compatible with any pre-trained segmentation model capable of generating multiple sample outputs.
We evaluate CONSIGN against two CP baselines across three medical imaging datasets and two COCO dataset subsets, using three different pre-trained segmentation models. Results demonstrate that accounting for spatial structure significantly improves performance across multiple metrics and enhances the quality of uncertainty estimates.

Published as a conference paper at ICLR 2026
CONSIGN: CONFORMAL SEGMENTATION INFORMED BY SPATIAL GROUPINGS VIA DECOMPOSITION
Bruno Viti
Department of Mathematics and Scientific Computing University of Graz, AT BioTeecMeed-Graz.. bruno. settiiiuiigaaa.aa
Elias Karabelas
Martin Holler IDea Lab University of Graz, AT BioTechMed- martin www.hhoeunniiigraz.....
Department of Mathematics and Scientific Computing University of Graz, AT
BioTechMed-Graz
elias. karabelassuni-ggra..
ABSTRACT
Most machine learning-based image segmentation models produce pixel-wise confidence scores that represent the model's predicted probability for each class label at every pixel. While this information can be particularly valuable in high- stakes domains such as medical imaging, these scores are heuristic in nature and do not constitute rigorous quantitative uncertainty estimates, Conformal predic- tion (CP) provides a principled framework for transforming heuristic confidence scores into statistically valid uncertainty estimates. However, applying CP di- rectly to image segmentation ignores spatial correlations between pixels, a fundamental characteristic of image data. This result in overly conservative and less interpretable uncertainty estimates. To address this, we propose CON- SIGN (Conformal Segmentation Informed by Spatial Groupings via Decomposi- tion), a CP-based method that incorporates spatial correlations to improve uncer- tainty quantification in image segmentation. Our method generates meaningful prediction sets that come with user-specified, high-probability error guarantees. It is compatible with any pre-trained segmentation model capable of generating multiple sample outputs. We evaluate CONSIGN against two CP baselines across three medical imaging datasets and two COCO dataset subsets, using three dif- ferent pre-trained segmentation models. Results demonstrate that accounting for spatial structure significantly improves performance across multiple metrics and enhances the quality of uncertainty estimates.
INTRODUCTION
In many real-world applications, predictive machine learning models are increasingly used to sup- port critical decision-making processes. However, these models often operate under various sources of uncertainty. including noisy data and limited observations. As result, it is essential not only generate accurate predictions, but also to assess the reliability of these predictions. Uncertainty quantification (UQ) provides systematic framework for evaluating and communicating the degree of confidence model outputs, see Abdar al. (202T) for a recent review. Specifically, as deep learning models increasingly dominate segmentation tasks due to their high accuracy, it becomes cqually important to assess the confidence of these predictions through UQ
Several UQ approaches have been proposed in recent years, including Bayesian and ensemble methods, see Abdar et al. (2021): Huang et al. (2024); Lambert et al. (2024) for detailed reviews. One type. Bayesian methods, includes Monte Carlo dropout techniques (Gal & Ghahramani 2016 Kendall
These methods enable uncertainty estimation by applying dropout at test time and sampling multiple forward passes approximate posterior distribution. The second category

---

Published as a conference paper at ICLR 2026
of UQ methods consists of Deep Ensemble Networks (Lakshminarayanan al., 2017 Mehrtash et al. 2020). Ensemble methods estimate uncertainty by combining predictions from multiple in- dependently trained models, capturing diverse hypotheses. [Kohl et al. (2018), instead, proposed a different architecture that combines a U-Net (Ronneberger et al. 20155 with a Variational Au- toencoder (VAH) (Kingma et al. 2019). Along the same line, (Baumgartner et al., (019) proposed PhiSeg, and (Monteiro et al. 2020) introduced the Stochastic Segmentation Network.
A common limitation of the above-diseussed methods is that they do not provide statistical guaran- tees regarding the reliability or coverage of their predicted uncertainty. In other words, while these approaches may produce plausible predictions of uncertainty, there no statistical guarantee that the predicted uncertainty actually matches the true uncertainty associated with the model and the data distribution. Conformal Prediction (CP) (Lei & Wasserman, 2014, Papadopoulos et al.) 2002 Vovk et al., (005) is statistical approach to uncertainty quantification that has recently seen a surge of interest within the machine learning community. Essentially, CP provides principled way to transform informal or heuristic uncertainty measures into rigorous ones (Angelopoulos al. 2023
The general workflow of CP can be outlined as follows:
First, we need a fixed pre-trained model I that has been trained on dataset D Dirain- Usually, the model is required to have some heuristic notion of uncertainty that should be made rigorous. Next, the pre-trained model is evaluated and adapted on the basis of calibration dataset Deat. in order to obtain a calibrated notion of uncertainty with coverage guarantees. Finally, the calibrated model can be evaluated on a new test dataset Drest for which the desired coverage guarantees hold. The main assumption for the latter hold is that the calibration and test datasets are exchangeable, which is true, for instance, if they are independent and identically distributed (i.i.d). We are interested in conformal prediction approach that outputs for each test image Xtest. a set of predictions C( ((ttss)) with some pre-defined guarantees regarding the accuracy of those predictions. In particular, we want to leverage a specific area of CP, Conformal Risk Control (CRC), which provide guarantees of the form
TE [E((((( Yrest) S o;
where is any bounded loss function that shrinks as C grows and a is an user-defined parameter such that 1 is the desired confidence. In particular, during the calibration step we produce prediction sets (-), where the parameter À encodes the level of conservativenes:: the higher the the larger the prediction sets. We are interested in finding the best parameter À that guarantees equation | Given a calibration set {(X,, Y)) the guarantee can be achieved by the choice
Å = inf {x: R(A) S B a YL
where B is the maximum of the loss function and f((A) = E (XC)) Y2) is the empirical risk. The definition of C(-) is crucial, and the quality of the algorithm heavily depends on In standard segmentation approaches, prediction sets are defined as
(x = (1 f(Xx), 2 1 - A}, A [0,
(3)
meaning that each pixel instead of being singleton (the arg max of the softmax probabilities f(x*)) is a set containing all the labels that have a softmax score greater then the threshold 1 X.
Recent works have extended basic conformal prediction methods to quantify uncertainty in image segmentation tasks. WWndrram et al.] (224)) apply pixel-wise CP to different segmentation models and evaluate the performance for binary segmentation tasks, while in Mossina et al. (022)) they ex- tended CRC to address the multi-class segmentation challenge by constructing pixel-wise prediction sets of the form defined in equation Blot et al./ (2025) instead, extended CRC towards group con- ditional risk control, again in mixel-wise setting. Wieslander et al. (2020) were among the first to extended the approach by mak-
introduce pixel-wise CP in medical imaging, (hileeDDavepoor(2210) ing the nonconformity score dependent on the distance to the mask boundaries. [Brunekreef et al.) (2024) tried to overcome pixel-wise CP approaches introducing a method where non-conformity scores are aggregated over similar image regions. The method, however, relies on custom cali- bration strategy that depends heavily on the characteristics of the data and task. [eng et al. (2023) proposed a feature- CP using deep network representations, which, however, leads to need of model internal information that might not be available. A recent contribution from Bereska et al.) (225)) adapts prediction sets based proximity to critical vascular structures in medical imaging.
2

---

Published as a conference paper at ICLR 2026
Mossina & Friedrich (2025) introduced a novel approach based on morphological operations, which, however, is currently limited to binary segmentation. Finally, Liu et al. (2025) introduced SACP, spatial-aware CP method where the scores aggregated across neighborhood pixels.
In summary, most prior works, in particular those who are generically applicable to pre-traines segmentation models, construct the set- set-valued predictions C(.) only for each pixel separately, disre- garding spatial correlations within the image. Since the coverage guarantee still holds by the design of the conformal prediction method, this leads diction; i.e. the set of possible labels for different pixel regions is larger than it would need to be, given the spatial dependence of pixels.
unnecessarily large size of the set-valued pre-
In this work. we address this issue by developing Conformal Segmentation Informed by Spatial Groupings via Decomposition (CONSIGN), method to leverage spatial correlation for improved conformal prediction sets. Building upon techniques that exploit Singular Value Decomposition (SVD) to extract principal directions of uncertainty, as presented in Bellasiin al. (2023); Nehme al. )023) for image restoration, we propose method that transforms any segmentation model capable generating sample predictions - such as those using dropout, Bayesian modeling, or en- sembles into one that produces spatially-aware set predictions with formal coverage guarantees. To achieve this, we defined novel spatially prediction sets and developed a corresponding cal- ibration strategy tailored to their unique characteristics. Most notably, due to the fine-range property of segmentation, our method can provide rigorous uncertainty bounds while only relying on rather low number of principle directions. To showcase the versatility of our method, we apply it to range of pre-trained models. As we show via numerical experiments, the fact that our approach acknowl- edges spatial correlations the segmentation masks, allows to produce much tighter and more meaningful set-valued predictions compared to direct pixel-wise approach that does not account for spatial correlations, see Figure | for an example.
Pixel-Wise
Image
Dog
Honse
Perun
Figure I: Images sampled from the prediction set C* of CONSIGN, and from the pixel-wise one CPW, The pixel-wise method treats each pixel independently, disregarding correlation between pixels and including inconsistent predictions in the prediction set CPW In contrast, our method samples masks by sampling different weights for the principal components extracted via the SVD approach, which captures spatial and contextual dependencies. This results more realistic and coherent segmentations, seen in the smooth transition between the sheep and cow in method's results.
METHODS
2.1 PROBLEM DEFINITION
We want to develop method that provides meaningful statistically valid guarantees for predictions of segmentation models that take into account spatial correlations. That is, instead of having a model f, trained on {(X,, Ner, which outputs single prediction f(Xtest), we want a set of predictions parameter In
Cx (Xeest) such that equation equation holds. The set of predictions depends À that is calibrated using calibration set {(X,, Y.) 1 Neat disjoint from the training X W are images while Y E fl..... L]Wx are the corresponding segmentations. Moreover, we consider models f RW x xxC + x which take in input an image and give output softmax probabilities for L. labels.
3

---

Published as a conference paper at ICLR 2026
Our method consists of two main components: The construction of spatially-aware Set and the Calibration. In the first step, we identify the uncertain regions and create a new basis of vectors {u, that characterizes these areas. Those vectors are the key component for the construction of our prediction set Cx. In the calibration step, we find the best parameter Å and corresponding prediction set C2 C (1.....,)) L x that satisfies equation |
2.2 CONSTRUCTION OF SPATIALLY-AWARE SET
We want to leverage the correlation between pixels to provide more meaningful and precise uncer- tainty regions. In particular, are interested in constructing set Cx(-) that contains predictions whose uncertain pixels change jointly, following a meaningful structure, rather than independently. To this end, principal component analysis (PCA) provides framework for capturing and repre- senting these joint variations in uncertainty. For our purpose, a PCA approach can make use of the different samples obtained from a pre-trained model f to gain insights into both the location of the uncertain regions and the correlation between pixels within those regions. This information is crucial for enhancing the interpretability of the model.
This concept of extracting uncertainty regions using principal components is one of the strengths of our model and has shown its effectiveness in previous works on image regression such as Belhasin et al. (2023); Nehme et al. (2023).
First, we identify the uncertain regions using pre-trained model f. For this, our method does not require any particular model f. as long as the model can be used to generate samples $,,...,SN, SN, W'L that correspond to heuristic uncertainties like softmax scores. Following the main idea of |Belhasin et al. (2023), we construct sample matrix S(X) [S....., SN.). compute its mean and extract the uncertain regions through an SVD (Golub & Reinsch 1971)
((X)
Sn $ (() 12, UEVT.
(4)
Note that here, as detailed below, it is sufficient to use reduced SVD and only compute the first K < min W HL,N, singular values, with E {2, 5} in our experiments. Each column u, E RWHI. of U is basis vector for the space of samples, aligned with the directions of maximum variance in the data. Building on this interpretation, to construct our prediction set, we first compute quantiles of the coefficients of the basis vectors ug, k 1,..., K, over the N,, samples via
a Q2 (f(ug, S,, [(X)) by Qi y (f(u S. (
(5)
Here (.) is the a quannill and (:,)) the scalar product. Then, we define bounds for the basis coefficients
A ak + bb
by ak
a# + by XER by ag
B₁
(6)
2
2
The derivation of equation ij heuristic and is designed to obtain symmetric bounds around the em- pirical quantile midpoint that scale linearly with X. The weighting by Σx. (the k-th singular value) creates wider bounds for principal components with higher variance (larger ΣR,K), allowing more flexibility where the model shows more uncertainty. The parameter will either shrink or enlarge the bounds, and it will be calibrated in the calibration step in order to fulfill equation equation Let = P(r)) represent the predicted labels for an element o IR HL with P beeing the argmax applied along the label dimension of the reshaped element prediction set
R HXL With this, we define the
K CT (x)x= v ECE X
K :Y P(((X)( Σ )
Note that this set contains the predictions such that there exists score vector E RWHL whose de- viation from the mean can be expressed as linear combination of the first K basis vectors uI..... ug with coefficients cs inside of the bounds defined in equation 6 A big advantage of our approach here is that, as we will see in the experiments, meaningful prediction sets this form can already be obtained with K E (2, 5). This significantly reduces the computational load compared full
4

---

Published as a conference paper at ICLR 2026
basis representation with K WHL, and is also major difference to the regression approach of Belhasin et al. (2023): Since our method includes a nonlinear quantization step P(or), map- ping softmax outputs to discrete predicted labels Y. we can enforee the coefficients of most of the basis vectors ux++1,..., UWHL z. to be zero and still be able to reconstruct the ground truth for some combination of coefficients Cg- In other words, even with truncated PCA the prediction set will generally include the ground truth provided that À is large enough (with pathological excep- tions, which we discuss in the next section). In contrast, the regression approach of [Belhasin et al. (023), the authors need to introduce special procedure to achieve coverage guarantees also for a truncated PCA. Independent of this, we still allow for user-defined accuracy rate in our prediction set as follows: We say that two predictions Y1 and Y2 coincide for label-wise accuracy rate of 3 and write Y; _ Y2 if t Eg, 10T =AA?=!)
B, where 3 is second user-defined parameter that control the desired accuracy and ř, pixel coordinates. Notably, adapt the standard accuracy rate from regression to enforce a label-wisc accuracy rate, thereby avoiding a bias towards more frequent labels. Using this, we now define the final prediction set and relative loss function as
E., [()"-t))
K (IX) Y Ec E X (Ax( Y ((((X)+ + Σea Ecand(r))
K
(7)
=
(8)
which is bounded loss function that shrinks as À increases. Here, and trough all the paper, (Y) refers to the indicator function of set C(X), i.e. equal to 1 if Y E C(X). otherwise.
2.3
CALIBRATION
Having defined the X-dependent prediction set (X) as in equation 7 we can in theory use the standard calibration procedure of Angelopoulos et al. (2024) to obtain a coverage of the form
Eie (( Yinss)] P |Yes (Xeee) S a.
This standard calibration procedure iterates trough the calibration set, evaluates the empirical loss ((AA) 1 E,= (X.,, Y2) and checks R(A) < a - (B = 1 in our case). If the empirical loss is above this threshold, the procedure repeated with an increased X. Otherwise, x is calibrated and the desired coverage for a new test point can be guaranteed. For example, 100, the procedure searches for At such that more than 90.9% of the calibration points (X,, Y2) satisfy Y; E (X.). In practice, however, we need to adapt this procedure, as the rather involved form of our prediction set does not allow us to easily check if Y E (X).. In fact, exhaustively checking all possible values of с is computationally infeasible. To address this, we formulate and numerically solve constrained minimization problem such as
if 0.1, and
K
K X [AxX)) Bx(X)] k=
c* = arg min C(Y,P(M(X) cEB
cguR)),
B =
(10)
L Ei T(Y") = =lAP(a) =))
C(Y, P(Y)
(11)
Eij T(Y
the numerical solution c* E B satisfies Y _ ((u(X) + ΣK_ (X)) then we can guarantee that Y E (X). However, due the numerical nature of the optimization process, which is described in Algorithm 2 global optimality cannot be guaranteed. In practice, this means that even suitable e exists within the current bounds, the solver may fail find it, possibly leading to an unnecessary increase in X. Despite this, the statistical guarantee of the overall algorithm is not compromised. Even when output a valid Ā that guarantees equation The only consequence is that the resulting bounds on may be more conservative. Moreover, since previously accepted c remain valid under expanded bounds, and additional segmentations! Y may be in C (X) as the bound increases, the loss equation is guaranteed to be non-increasing, A summary of the resulting procedure is sketched in Algorithm I and the following lemma (which is a direct consequence of Annellopoulss et al. (2024) Theorem T) and proven in Appendix A provides the resulting coverage guarantees for this algorithm.
increased beyond what is strictly necessary, the method will still
5

---

Published as a conference paper at ICLR 2026
Lemma 1. If Algorithm I terminates with X and if the (X1, Vi,...... (Xn, Yn) used in this algorithm are exchangeable with (Xeest Yies), then
P YYese E (((ene)) 2 21
2.3.1 TERMINATION OF THE ALGORITHM
As mentioned in Section 2.2 there may be pathological cases- such as zero or poorly aligned entries in the principal directions ug-ii which our method cannot converge even as À oc. This behavior actually desirable, as it prevents the method from silently producing prediction sets that fail to satisfy the intended coverage guarantees. In practice, we impose maximum value of À and stop the algorithm if it reaches Amaz. so that the user is explicitly informed when no useful or informative prediction set can be obtained for the chosen parameters and B.
Algorithm Calibration algorithm for CONSIGN
Input: a, 3,B.d,, {(X,, ] Neat
Output: Å
1: pre-compute: (()()))) St, U
E Nill
Neat
Neat
as in equation
equation 5
2: A < 0; R 1; I t 0
3: while R
1-a do
Nea!
4: for i
Neal
\ do
5:
(A₁, Bs.)
2
-i-o ΛE
k
2
oi+
AE
oi oi
for each k:
6:
B + X LAk: BR]
e* + approx approx_solveri solver vereerr(rggg m C(Y,, P(u(X,) + Ek caul))) & using e.g. Alg.
7:
8:
a + ((X.) + Ek=1 u
Y _ P(ar) then I t IUii}
9:
R 1 I
10:
Newt
11:
if < Neal
I-a then Å eess X t A
3 EXPERIMENTS
We proved that our method creates prediction sets containing the ground truth with user-defined guarantees. Now validate the method numerically, showing its performances through differ- ent experiments and metrics. In particular, we are interested showing that our method provides prediction sets with lower uncertainty volume compared to the pixel-wise baseline defined in An- gelopoulos et al.] (2020), and the spatial-aware method SACP used in Liu et al. (2025). We define the uncertainty volume the number of predictions in prediction set Cs. With equal theoretical guarantees, we aim for the method that has a lower volume. In the next section, we quantify the volume and show that method reliably produces smaller estimate.
3.1 DATASETS AND BASELINES
We are interested in applying our method to different datasets and pre-trained models f, to show its effectiveness regardless of the setting. We use three medical datasets (M&Ms-CCampelloo et al.
(2021); Martín-Ish et al. (2023). MS- MS-CMRI$Gao et al. (2023); /W & Zhuang (2022): Zhuang
(018). Armaa III et al. 2015), and two subsets the COCO dataset [in et al. (2014). For each dataset we use a different model f, in order to show flexibility of our approach also with respect to the segmentation model. For the two cardiac datasets M&Mss2 and MS-CMR19, we
produce samples through a U-Net [Ronneberger et al. (2015) trained with dropout. For the LIDC dataset, we employ the method proposed by Kohl et al. (0018) which intrinsically contains a way of generating different samples. Finally, for the subsets of the COCO dataset, employ ensemble networks strategy based on DeepLab Chen et al.] (2017) 2018) and generate different samples using different backbones. See Appendix B for further details datasets and pre-trained models.

---

Published as a conference paper at ICLR 2026
As pixel-wise (PW) baseline, we use RAPS (Angelopoulos et. 2020). CP method that forms pre- diction sets by including labels until the cumulative softmax sum plus regularization term exceeds A. Let π be permutation of indices such that f(X*) )x())(())> 2 2 f(x*) )(()))(())) then
TPW (x1) - ... (.)). [k)), k min { (1, LL Li: E(x)) r(1) A..
(12)
The term r(I) defined as (o(2) k where 9 and kyeg are hyperparameter, while o()) is the ranking of I among the label based on the probabilities æ. See Appendix for details. Based on this, pixel-wise prediction set CRW- and a relaxed version with 3 accuracy CF are defined as
{x Vi.j Y (X CR (X) (x
=Ý E (X) -
=
(13)
For comparison with spatial- approaches, employ SACP (Liu et al. 2025), where the pixel- wise cumulative sums of softmax- used to construct the prediction sets are aggregated over local neighborhoods. Similarly RAPS, TSACP is defined as the set of labels whose cumulative scores plus regularization, after aggregation over local neighborhoods, exceed A. Given TSACP, we can define the corresponding c C and C&ACA as in equation 13. See Appendix for further details. The baselines calibration algorithms have similar structure as Algorithm | only that the loss function is given as E(c PC/////SPPPP (X) = Lerw/SACr (x) (Y) and that one can directly check
if a ground-truth is contained in the prediction set. See Algorithm for details.
3.2 SAMPLING AND METRICS
In order to quantitatively compare our approach to the baselines, we want to compute the volume of the prediction sets, which in this case is given as the number of different segmented images contained in this set. Since the definition and value of the accuracy rate 3 is the same for both methods, we focus on the volume of the prediction sets C (X), CSACT (X) and CPW (X). ). For the pixel-wise
method, this volume is given explicitly as ICPW (XW-(X)]= = II W.H PT (x"),, and analogously for SACP. For (X), however, we can only estimate this volume using sampling, and in order to have a fair comparison, we use the same estimates for both methods. Recall that the sampling from our method means sampling coefficients E B and generating the resulting segmentations equation? 7, while for the baselines, it means to sample independently for each pixel (i,j) possible labels contained in TPW (Xx*) and TSACP (X').. Using this sampling, define
in
(-) =
DP = (Ý.) and SSACP (X) = (Ý.)
C*'(""'' (X), CPW (P)-(X),Cm CSACP-(X)) (X), respectively, for given test point X. We evaluate our method and the baselines across three different metrics: Chao estimator (Chao) 1984), Sample-based Estimated Coverage EsEC) and correlation. The first metric taken from the species richness estimation problem, which tries to estimate the true number of species based on sample data. In our setting, we aim to estimate the number of unique segmentations contained in prediction set. The estimator provides a lower bound for the true number of segmentation and is defined as
be samples sets from Ca
P.
NC) :- + 252²
where S₁ is the number of unique samples, is the number of vectors sampled exactly once and fa is the number of vectors sampled exactly twice. The number of unique segmentations can grow rapidly. Therefore,
evaluate the estimator and study its behavior for different sample sizes S. The second metric, tries to estimate the volume of uncertainty through the empirical coverage. The empirical coverage E.C Ni..t Noeat Te, (x.) (Y;). should be, on average, greater equal than 1 - However, while we can evaluate the empirical coverage for the baselines methods, we can only estimate for our method since we do not know the best coefficient c. Therefore, we introduce the Sample-based Estimated Coverage as
Neest
8 &EC()) = Neest
min E I(Y, _ Y.)). Y.EY
7

---

Published as a conference paper at ICLR 2026
This metric should converge to for an increasing number of random samples, and faster convergence suggests that the corresponding set occupies a lower-dimensional subspace. The last metric is the averaged Pearson correlation
2 p()) S(S-
ΣΣ .is
(e,
which can be useful quantify how much different random segmentation are correlated. A strong internal correlation also indicates that the samples are confined to a lower-dimensional manifold. In contrast, the weer-independence of samples suggests higher intrinsic dimensionality. See Appendix flror further details on the metrics.
3.3 RESULTS
We compare the baselines with our CONSIGN approach using two different numbers of principal components, specifically K 2 and K = 5. We evaluate the metrics across five random calibra- tion/test splits, and and the pre-trained model. For safety-critical applications, like medical imaging, smaller values of (e.g a 0.05) preferred to enforce stricter confidence requirements. In contrast, exploratory analyses can tolerate larger a (e.g a 0.3). The parameter 3 controls the acceptable pixel-wise accuracy: medical datasets with fine, small-scale structures benefit from higher 3 (e.g 3 0.8). Whereas segmenting large, distinet objects may allow lower / while still achieving reliable UQ Moreover, optimal choices of a and also depend on the capability of the pre-trained model: stronger models permit more liberal choices, yielding smaller prediction sets without sacrificing reliability, whereas weaker models require more conservative settings. In the following figures, er- ror bars will refer to the standard deviation across the five random splits and will depict + standard deviation.
different combinations of parameters a and 33 depending on the datasets
In Figure 2 we present the Chao estimator for various sample sizes. The estimator for CONSIGN is consistently bounded by the baselines estimators, indicating a smaller volume of uncertainty. In the LIDC experiment, the difference between the methods is maximized, with several orders of mag- nitude separating the estimators. In contrast, during the COCO experiments, the high uncertainty results in our method also producing a high Chao estimator. the case of K 5, the model has access to more principal components, which gives the coefficients с greater flexibility. The higher degree of freedom leads wider range of possible predictions and higher estimated value of NeH. Moreover, CONSIGN's performance depends on the sample quality from the pre-trained model. With the probabilistic U-net, which is designed to produce meaningful samples, results improve by orders of magnitude, but using dropout for sampling leads to less pronounced gains over baselines. In Figure 3
show the behavior of the sEC. CONSIGN, owing to its spatial
CONSIGN;
CONSIGN
PW (RAPS)
SACP
MnM2
mscmr19
LIDC
COCO animals
COCO vehicles a= 0.3 0.3 8= 0.7
0.1 8
0.9
= 0.05 8 0.8
0.8
0.2 $ 0.8
0022 8= 0.8
Chao Estimator
20a
107
10a
1.00
10a
10a
109
304
16a
100
10a
20a0
10
107
10a
30
10000
10000
201
10000
10000
0
10000
#
samples
Figure 2: NCH(Y*), (CH and NCH (YSACP). Larger values indicate larger prediction set
awareness, outperforms RAPS and SACP by achieving empirical coverage with fewer samples. In the COCO-vehicle experiment, even small sample of ten predictions meets the user-defined cov- erage requirement, indicating greater efficiency and precision in capturing relevant segmentations. Finally, in Figure| 4 compare the correlation between predictions sampled from ) SSCCP
8

---

Published as a conference paper at ICLR 2026
1-2
CONSIGN;
CONSIGN;
PW (RAPS)
SACP
MnM2
mscmr19
LIDC
animals
COCO vehicles 0.3 B= 0.7
0=
0.1 (
0.9
a = 0.05 B 0.8
a 0.2 B - 0.8
a 0.25 8 0.8
1.0
0.8
SEC
0.6
0.4
10
5000
10000
10
5000
10000
10
5000
10000
10
5000
10000
10
5000
10000
#
samples
Figure 3: (ECC)"))ECCC(") and sEC(YSACP)) Values close to 1 a indicate better coverage
and yPW, By using a linear combination of principal components to construct predictions, we en- hance the consistency of our predictions in correlated regions, resulting in higher correlation among them. Moreover, CONSIGN exhibits monotonic increase in correlation between samples, indicat- ing consistency in capturing spatial structure. In contrast, the baselines show a decreasing trend in correlation, suggesting that the samples become nearly independent and fail to reflect any coherent shared structure. It can also visually observed that the accounting of spatial correlations, done in CONSIGN, leads to more meaningful set of possible segmentations: In Figure [ we show how our method smoothly transitions between classes, jointly modifying regions that are uncertain and highly correlated. While the visualizations of pixel-wise samples may appear unrealistic, this be- havior expected, as pixel-wise prediction sets contain all combinatorial label configurations rather than structured samples.
In general, we can observe that the results for SACP are comparable the pixel-wise ones. Aggre- gating softmax scores across pixels is mainly post-processing step; however, a similar aggregation happens implicitly during the training of models f. Relying solely on this additional step does not effectively capture the true spatial correlations and results in method that is comparable to pixel-wise approach. In contrast, our method explicitly identifies correlated regions, outperforming the baselines across all metries, proving an advantage in reducing the volume of uncertainty while providing consistent and qualitatively superior predictions.
CONSIGN;
CONSIGN
PW IRAPS!
SACP
mscmm19
LIDC
animals
vehicles
$
05
$-
g 0.2
12 5
g 0.3 5
0.6
Correlation
0.4
0.4
0.2
0
0.2
0.2
0.0
1000
1000
500
1000
# samples
Figure 4: p())), ))WPW) and ((SSACP).
Larger values indicate greater spatial correlation
4 CONCLUSIONS AND LIMITATIONS
We developed a method that transforms heuristic and overconfident softmax scores into predictions backed by user-defined statistical guarantees. We exploit SVD techniques from previous approaches, such as [Belhasin et al. (2023); Nehme et al. (2023). to introduce new spatially-aware conformal prediction approach for image segmentation. Our approach stands out for three main reasons: First, we harness the power of spatial correlation to significantly improve segmentation quality while minimizing uncertainty. This results in a robust tool that allows users sample insightful predictions with solid statistical assurances. Second, our method is casily applicable to any segmentation model that offers samples of predictions. Finally, by exploiting the classification nature of our setting and
9

---

Published as a conference paper at ICLR 2026
CONSIGN
Pixel-Wise (RAPS)
SACP
Imagee
GT
pro.o
Buuigrund
Lun Varride
M
1lggit Verride
MS-CHR199
lanuun
Higi Viniide
pr08
Backgreend
Set
Dog
Mose
Moer
becigroun
CC
Mutotes
Porian
Figure 5: Qualitative comparison between samples from y" (K = 5). Pow and YSA
the non-linear projection P()), able to reformulate the theory of Bellasii et al. (0033 in way that yields more interpretable and practically meaningful bounds.
Currently, main limitations of method prediction based method, the guarantees hold true under exchangeability assumptions on the data. Distribution-shifs out-of distribution data currently addressed by our method. Extensions of conformal prediction in this direct exist, see e.g. Gibbs Candes (2021). and a future goal is extend our method in this direction. A second limitation of our method is the implicit form of the prediction set C (X), which increases the computational cost, see Appendix D for details, and makes it numerically challenging to evaluate if given candidate segmentation is in C (X) not. Nevertheless, believe that this is not major issue, since the online generation and sampling from the prediction set, which is the main application of our method, is still comparably fast.
follows: As with any (standard) conformal-
ACKNOWLEDGMENTS
The authors acknowledge the National Cancer Institute and the Foundation for the National Institutes of Health, and their critical role in the creation of the free publicly available LIDC/IDRI Database used this study.
This research funded in whole in part by the Austrian Science Fund (FWF) 10,55776/F100888 B. V. and E. K. acknowledge funding from BioTechMedd-Graa Young Research Group Grant CICLOPS.
LLMs DECLARATION
The authors utilized large language models (LLMs) to check grammar and rewrite sentences.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
In Appendix A we give the main proof of our method.
The calibration algorithm for CONSIGN is described in Algorithm while the calibration algorithm of the pixel-wise method and SACP are described in 3 The optimization algorithm described in Table while Tables provide the calibrated Å for all the experiments.
In Appendix B we describe the datasets, providing training/calibratiooness splitting and references to download the datasets, which are all publicly available. We provide external links for the pre- trained models, and we discuss the implementation of the U-Net. Finally in Appendix D we discuss the hyper-parameters and hardware specifics.
Moreover, consistent with ICLR guidelines, we plan to share an anonymous repository with review- ers and ACs during the discussion phase, and will make the code public upon acceptance. The code is available at httttts::/// http:::///gtuu..//// com/ Cnurbbruno/CONSIC.
REFERENCES
Moloud Abdar, Farhad Pourpanah, Sadiq Hussain, Dana Rezazadegan, Li Liu, Mohammad Ghavamzadeh, Paul Fieguth, Xiaochun Cao. Abbas Khosravi, U Rajendra Acharya, et al. A Re- view of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges. Information Fusion, 76:243-297, 2021.
Anastasios Angelopoulos, Stephen Bates, Jitendra Malik, and Michael Jordan. Uncertainty sets for image classifiers using conformal prediction. arXiv preprint 1411192009.1199,, 2020.
Anastasios N Angelopoulos, Stephen Bates, al. Conformal Prediction: A Gentle Introduction. Foundations and Trends@ in Machine Learning, 16(4):494-591, 2023.
Anastasios Nikolas Angelopoulos, Stephen Bates, Adam Fisch, Lihua Lei, and Tal Schuster. Confor- mal Risk Control. In The Twelfth International Conference on Learning Representations, 2024.
Samuel G. Armato III, Geoffrey McLennan, Luc Bidaut, Michael F. McNitt-Gray, Charles R. Meyer, Anthony P. Reeves, Binsheng Zhao, Denise R. Aberle, Claudia Henschke, Eric A. Hoffman, Ella A. Kazerooni, Heber MacMahon, Edwin J. R. Van Beek, David Yankelevitz, Anthony M. Biancardi, Patricia H. Bland, Mark S. Brown, Roger M. Engelmann, Gerald E. Laderach, David Max, R. C. Pais, D. P. Y. Qing, Robert Y. Roberts, Ann R. Smith, Andrew Starkey, Priya Batra, Paola Caligiuri, Asim Farooqi, Gregory W. Gladish, Charles M. Jude, Reginald F. Munden, Iva Petkovska, Leslie E. Quint, Lawrence H. Schwartz, Bala Sundaram, Lawrence E. Dodd, Christopher Fenimore, David Gur, Nicholas Petrick, John Freymann, Justin Kirby, Brad Hughes, Adrien Van Casteele, Sonal Gupte, Mohamed Sallam, Mark D. Heath, Michael H. Kuhn, Ekta Dharaiya, Robert Burns, David S. Fryd, Mare Salganicoff, Vineet Anand, Uri Shreter, Sander Vastagh, Byron Y. Croft, and Laurence Clarke. Data From LIDC-IDRI. https: /ttp:///oo........///// 7937/ 73/3//9/// wT/.///T/.../. 2015 LO9QL9SX, 2015. Data set.
Christian F Baumgartner, Kerem C Tezcan, Krishna Chaitanya, Andreas M Hötker, Urs J Muehlematter, Khoschy Schawkat, Anton S Becker, Olivio Donati, and Ender Konukoglu. Phiseg: Capturing uncertainty in medical image segmentation. International Conference on Medical Image Computing and Computer-Assisse Intervention, pp. 119-127. Springer, 2019.
Oner Belhasin, Yaniv Romano, Daniel Freedman, Ehud Rivlin, and Michael Elad. Principal Uncer- tainty Quantification with Spatial Correlation for Image Restoration Problems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(5):3321-3333 2023.
Jacqueline Isabel Bereska, Hamed Karimi, and Reza Samavi. SACP: Spatially-Aware Conformal
Prediction in Uncertainty Quantification of Medical Image Segmentation. In Medical Imaging with Deep Learning, 2025.
Vincent Blot, Anastasios Nikolas Angelopoulos, Michael Jordan, and Nicolas J-B. Brunel. Automat- ically adaptive conformal risk control. The 28th International Conference on Artificial Intelli- gence and Statistics, 2025.

---

Published as a conference paper at ICLR 2026
Joren Brunekreef, Eric Marcus, Ray Sheombarsing, Jan-Jakob Sonke, and Jonas Teuwen. Kandinsky Conformal Prediction: Efficient Calibration of Image Segmentation Algorithms. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp- 4135-41433 2024.
Victor M Campello, Polyxeni Gkontra, Cristian Izquierdo, Carlos Martin-Isla, Alireza Sojoudi, Pe- ter M Full, Klaus Maier-Hein, Yao Zhang, Zhiqiang He, Jun Ma, et al. Multi-centre, Multi-vendor and Multi-Diseand Cardiac Segmentation: The M&Ms challenge. IEEE Transactions on Medical Imaging, 40:3543-3554 2021.
Anne Chao. Nonparametric Estimation of the Number of Classes in Population. Scandinavian Journal of Statistics, pp. 265-270, 1984.
Liang-Chich Chen, George Papandreou. Florian Schroff, and Hartwig Adam. Rethinking Atrous Convolution for Semantic Image Segmentation. CoRR, abs/1706.05587, 2017.
Liang-Chiehh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder- Decoder with Atrous Separable Convolution for Semantic Image Segmentation. In Proceedings of the European conference on Computer Vision (ECCV), pp. 801-818, 2018.
Samuel Duvenport. Conformal confidence sets for biomedical image segmentation. arXiv preprint vrXiv:2410.03400 2024.
Yarin Gal and Zoubin Ghahramani. Dropout as Bayesian Approximation: Representing Model Uncertainty in Deep Learning. In International Conference on Muchine Learning, pp. 1050- 1059. PMLR. 2016.
Shangqi Gao, Hangqi Zhou, Yibo Gao, and Xiahai Zhuang. BayeSeg: Bayesian Modeling for Medical Image Segmentation with Interpretable Generalizability. Medical Image Analysis, 89: 102889, 2023.
Isaac Gibbs and Emmanuel Candes. Adaptive Conformal Inference under Distribution Shift. Ad- vances in Neural Information Processing Systems, 34: 14:1660-11 1660-1672, 2021.
Gene H Golub and Christian Reinsch. Singular value decomposition and least squares solutions. In Handbook for automatic computation: volume II: linear algebra, pp. 134-151. Springer, 1971.
Ling Huang, Su Ruan, Yucheng Xing, and Mengling Feng. A review of uncertainty quantification in medical image analysis: Probabilistic and non-probabilistic methods. Medical Image Analysis, pp. 103223, 2024.
Alex Kendall and Yarin Gal. What uncertainties do we need in bayesian deep learning for computer vision? Advances in Neural Information Processing Systems, 30, 2017.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2017.
Diederik Kingma, Max Welling, al. An Introduction to Variational Autoencoders. Foundations and Trends@ in Machine Learning, (24)):377-32,,201
Simon Kohl, Bernardino Romera-Paredes, Clemens Meyer, Jeffrey De Fauw, Joseph R Ledsam, Klaus Maier-Hein, SM Eslami, Danilo Jimenez Rezende, and Olaf Ronneberger. A Probabilistic U-Net for Segmentation of Ambiguous Images. Advances in Neural Information Processing Systems, 31, 2018.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predic- tive uncertainty estimation using deep ensembles. Advances Neural Information Processing Systems, 30, 2017.
Benjamin Lambert, Florence Forbes, Senan Doyle, Harmonie Dehaene, and Michel Dojat. Trust- worthy clinical AI solutions: unified review of uncertainty quantification Deep Learning models for medical image analysis. Artificial Intelligence in Medicine, 150:102830, 2024. ISSN 0933-3657.
Jing Lei and Larry Wasserman. Distribution-frr prediction bands for non non-parametric regression. Journal of the Royal Statistical Society Series B: Statistical Methodology, 76(1):71-9) 2014.
12

---

Published as a conference paper at ICLR 2026
Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft COCO: Common Objects in Context. In Computer Vision AssionEECV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, pro- ceedings, part 13, pp. 740-755. Springer, 2014.
Kangdao Liu, Tianhao Sun, Hao Zeng, Yongshan Zhang, Chi-Man Pun, and Chi-Man Vong. Spatial- aware conformal prediction for trustworthy hyperspectral image classification. IEEE Transactions on Circuits and Systems for Video Technology, 2025.
Carlos Martín- Vfictor M Campello, Cristian Izquierdo, Kaisar Kushibar, Carla Sendra-Balcelle Polyxeni Gkontra, Alireza Sojoudi, Mitchell J Fulton, Tewodros Weldebirhan Arega, Kumarade- van Punithakumar, et al. Deep Learning Segmentation of the Right Ventricle in Cardiac MRI: The M&Ms challenge. IEEE Journal of Biomedical and Health Informatics, 27:3302- -313,, 2023.
Alireza Mehrtash, William M Wells, Clare M Tempany, Purang Abolmaesumi, and Tina Kapur. Confidence Calibration and Predictive Uncertainty Estimation for Deep Medical Image Segmen- tation. IEEE Transactions Medical Imaging, 39(12)33868-38) -3778, 2020.
Miguel Monteiro, Loic Le Folgoc, Daniel Coelho de Castro, Nick Pawlowski, Bernardo Marques, Konstantinos Kamnitsas, Mark Van der Wilk, and Ben Glocker. Stochastic segmentation net- works: Modelling spatially correlated alcatoric uncertainty. Advances in neural information pro- cessing systems, 33:12756-12767 2020.
Luca Mossina and Corentin Friedrich. Conformal Prediction for Image Segmentation Using Mor- phological Prediction Sets. arXiv preprini arXiv:2503.05611, 2025.
Luca Mossina, Joseba Dalmau, and Léo Andéol. Conformal Semantic Image Segmentation: Post- hoc Quantification of Predictive Uncertainty. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pullern Recognition (CVPR) Workshops, pp. 3574-3584, June 2024.
Elias Nehme, Omer Yair, and Tomer Michaeli. Uncertainty Quantification via Neural Posterior Principal Components. Advances in Neural Information Processing Systems, 36:37128-3711 2023.
Harris Papadopoulos, Kostas Proedrou, Volodya Vork, and Alex Gammerman. Inductive confidence machines for regression. In Machine learning: ECML 2002: 13th European conference on ma- chine learning Helsinki, Finland, August 19-23, 2002 proceedings 13, pp. 345-356. Springer, 2002.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional Networks for Biomed- ical Image Segmentation. In Medical Image Computing and Computer-Assisted Intervention- MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceed- ings, Part III 18, pp. 234-241. Springer, 2015.
Jiaye Teng, Chuan Wen, Dinghuai Zhang, Yoshua Bengio, Yang Gao, and Yang Yuan. Predic- tive Inference with Feature Conformal Prediction. The Eleventh International Conference on Learning Representations, 2023.
Vladimir Vovk, Alexander Gammerman, and Glenn Shafer. Algorithmic Learning in a Random World, volume 29. Springer, 2005.
Häkan Wieslander, Philip Harrison, Gabriel Skogberg, Sonya Jackson, Markus Fridén, Johan Karlsson, Ola Spjuth, and Carolina Wählby. Deep Learning With Conformal Prediction for Hi- erarchical Analysis of Large-Scale Whole-Slide Tissue Images. IEEE Journal of Biomedical and Health Informatics, 25(2):371-380 2020.
Fuping Wu and Xiahai Zhuang. Minimizing Estimated Risks on Unlabeled Data: A New Formula- tion for -uppeviied Medical Image Segmentation. IEFE Transactiony Pattern Analysis and Machine Intelligence, (5(5):6021-6033) 2022.
Anna M Wundram, Paul Fischer, Michael Mühlebach, Lisa M Koch, and Christian F Baumgartner. Conformal Performance Range Prediction for Segmentation Output Quality Control. In Interna tional Workshop on Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, pp. 81-91. Springer, 2024.
13

---

Published as a conference paper at ICLR 2026
Xiahai Zhuang. Multivariate Mixture Model for Myocardial Segmentation Combining Multi-Soures Images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(12):2933-22 2944,, 2018.
A PROOF OF LEMMA 1
Proof. This is direct consequence of (Angelopoulos et al. 2024 Theorem I): It is clear that the loss À + (X)y == 1 - I(Y E Cx(X)) is non- oo--nnceasing for all (X,Y), Further, with finite as provided by our algorithm, it clear that
}
x inf X
Σ Lx (2) +1 saj.
n n
By (Angelopoulos et al.
2024
Theorem 1) and monotonicity of L we hence obtain
P Yeat & O3 (Xiesa) = -E Xun (2) s E X (1)) S
B DATASETS AND PRE-TRAINED MODELS
B.I DATA ACKNOWLEDGMENT
The authors acknowledge the National Cancer Institute and the Foundation for the National Institutes of Health, and their critical role in the creation of the free publicly available LIDC/IDRI Database used in this study.
B.2 DATASETS
The M&Ms-2 datase Campello al. (2021 Martin-Isla (2023) comprises 360 patients with various pathologies affecting the right and left ventricles, as well as healthy subjects. For each patient, the dataset provides cardiac MRI images along with annotations for the left and right ven- tricles and the left ventricular myocardium. It includes both short-axis and long-axis MRI images; however, our experiments utilized only the short-axis images. We adhered to the predefined training and test splits. The training set was used for model training, while the original test set was further divided into two subsets: a calibration set and a reduced test set. The reduced test set included only portion of the original test, i.c. the first 900 MRIs.
The MS datase Gao G et al. (0033) :: Wu Zhuang 2022);; Zhuang (2018) is another cardiac dataset, but it includes different modalities. This variation introduces greater uncertainty in the predictions. The dataset features 45 patients and contains cardiac MR images taken from the short- axis view. In this instance, also utilize pre-defined splits, extracting the calibration set from the original test set.
The third medical datase LIDCAAman III al. (0155) (Licence CC BY 3.0) contains lungs CT images with the corresponding segmentations obtained across over 1000 patients. Two labels are annotated, namely background and cancer.
We created two separate datasets from the COCO dataset Lin et al. (2014). Specifically, we selected images that feature either animals or humans form the COcO-animals dataset, and images that contain vehicles or humans to create the COCO-vehicles dataset. The COCO- animaal dataset in- cludes the following labels: background, cat, dog. sheep, cow, horse, bird, and human. In contrast, the COCO-vehicles dataset contains these labels: background, train, bus, bicycle, airplane, car, boat, and human. All images from both datasets have been used for calibration and testing, as we utilized pre-trained model for this setup.
Table we provide the details regarding the datasets used in the experiment section.
http:://www..bbee//mmmm..m
14

---

Published as a conference paper at ICLR 2026
Table I: Summary of datasets
Dataset
Calibration Images
Test Images
L
Sampling Strategy
M&Ms-2
500
179
4
Monte Carlo dropout
MS-CMR19
500
98
4
Monte Carlo dropout
LIDC
700
809
2
Probabilistic U-Net
COCO an.
275
39
8
Ensemble Networks
COCO veh.
275
46
8
Ensemble Networks
B.3 PRE-TRAINED MODELS
For the two cardiac datasets we used simple U-Net Ronneberger et al. (2015) trained with dropout. The architecture consists of an encoder-decoder structure with skip connections between corre- sponding levels to preserve spatial context. The encoder comprises series of block modules, each with two convolutional layers followed by ReI.U activations, batch normalization, and dropout for regularization. Feature maps are progressively downsampled using max pooling, doubling the num- ber of channels at each depth. The decoder utilizes bilinear upsampling and 1x1 convolutions to reduce channel dimensions. At each stage of the decoder, the feature maps are concatenated with corresponding encoder outputs via skip connections to recover spatial resolution. The final output is produced through a Ixl convolution to map to the desired number of segmentation labels. The U- Net model was trained using a learning rate of 3 10 14, optimized via Adam Kingma Ba (0177). The encoder network utilized initial number of 48 filters, which doubled each layer up fixed depth of 5. Input MRI scans were cropped to a spatial resolution of 128 x 128 pixels, with each pixel representing 1.375 mm in
space. Only MRIs with non-zero ground truth used. A batch size of 2 was used and the model was trained for 1500 epochs. A dropout rate of 0.4 was applied within encoder and decoder blocks (except the final level of the encoder).
For the LIDC experiment we used pytorch re-implementation of the probabilistic U-Net Kohl al. (2018). We trained the model with hyperparameters and splitting provided in the code. Both the original code and the ce-implemennation are published under the Apache License Version 2.0.
Finally, for the COCO experiments we rely on an ensemble networks strategy based on DeepLab Chen et al. (2018 2017). To generate different segmentation samples we used six different models with different backbones"
DpepLabV3-Mobieeee ResNet101, DeepLab V3Puss MobileNet, The code is published under the MIT License.
DeepLabV3-ResNee DeepLabV3.
B.4 BASELINE METHODS
As described in the main text, the SACP method aggregate the score of neighborhood pixels. Let be a permutation of indices such that f(X²) )=(1) ... 2 f(x (x()) then
S(X = Σ f(X*) ((()))(()))))))))))))))))))))))) (mm)) + r()),
(sAcr(x';,1) (1-w) w) - 5(x (1) [N(X
w
Σ S(X"1),
(EN(X
TSACP = (1)),.. (k:),, k == =imin| (1,..,.. SSACp(X > A}.
The hyper-paramen parameter w is the weight that regulate the strength of the aggregation, while N ((x*) is set that includes the neighborhood pixels. The dimension of this set can be also tuned, selecting how many pixels to consider for the aggregation. Then we can define the corresponding prediction sets
CSACP- (x)=(y Y
B. Vi. (SACP(X*)) (X)CP(X)= = -r: 3. E C.P. (()) 2 rr
htunss//ghhbbboomssscaeeeeghhoap Pnet-Prrrc
"https://github... "https//gitub..oom/iinono hwwp://.....m.mooo///oaaaalallla http::///gttttocoomm/a//////e//p
unet
15

---

Published as a conference paper at ICLR 2026
In Liu al. 2025) they introduce iterative aggregation operator V
as
V W = (1 -
w) V (
w
Σ
VA-1
[N(
PE PEN(X
where Vo S. In our experiments, we keep the iterations equal 1, since the over-smoothing of the scores lead to worst results.
C METRIC DETAILS
The Chao estimator is a commonly used non-parametric method in ecology and other fields for estimating the true species richness, the total number of species, in community based on sample data. This method addresses the challenge of unobserved species that may not be detected due to limited sampling efforts Chao (1984). It has been proven that the Chao estimator asymptotically converges to a lower bound of the true species richness as the sample size increases, i.c., as the number of observed individuals 5 - oo, the estimator converges to a consistent lower bound of the total number of species. The Chao estimator is not defined if f2 = 0. In that case the following bias-corrected estimator needs be used
= + fi(fi 1)
2(/2 + 1)
The Pearson correlation Pij between two vectors yı: Yj E R. is computed using the standard formula
Σm (iin F.)( - yj) Σ (Vi. - 53.3% Ex=i(y -,)
(e)(ye,,))
In order to seed up the computations, the Chao estimator and correlation have been computed con- sidering only the non-constant pixels over the samples. It is clear that the results equivalent to computing the metric considering the whole segmentation.
D IMPLEMENTATION DETAILS AND COMPUTATIONAL EXPENSES
We perform each experiment using a GPU NVIDIA A100-SXM4-40GB. For the optimization of the coefficient we utilize Adam optimizer with learning rate equal for the medical datasets and 10 for the COCO datasets. For every experiment in Section 3 we used dA = 0.01 for both CONSIGN and the baselines. For the implementation RAPS chose 0 0.05 and Kreg & where L is the number of labels. For the SACP we chose neighborhood weight w = 0.3 and neighborhood size of 7x7. The hyper-parameters were selected based on optimal performance.
The algorithm to numerically solve the optimization problem is described in Algorithm 2 while the Wise/ACPs calibration algorithm described in Algorithm
Algorithm 2 Optimization algorithm approx_solver
Input: Y, (X),, {u Ir, B,T
Output: e*
1: optimizer t- Adam(c, Ir)
2: for epoch +
do
3: o + (XX) Σk Guuk
4: loss + P(o))
with L as in equation
5: c Adammsteep()
6:
(c)
7: c* f-c
In Table 2 we compare the computational times of CONSIGN and the pixel-wise method. Notice that the computational time of SACP is equivalent the pixel-wise one. The offline time is measured in minutes and considers an average calibration step for one calibration/test split. The online time measured in seconds and refers the sampling of S segmentation from the prediction set. The
16

---

Published as a conference paper at ICLR 2026
Algorithm 3 Calibration algorithm for pixel-wise RAPS and SACP
Input: a, N,B,dA, dA, Y
Output: 2
A 0: R I+
2: while 0 Neal do 3: for + to Neal \ do
1-a
construct label set T TPW)SAC((X) as equation 12
4:
5: if E C PPW/SCCP (X,) then with C cPW/SACH
(X₂) in equation 13 equation B.4
6:
I UU+}
7:
R R-1
I
Neal
8:
if S L-o then Neat
9:
10:
else
11:
- Ad
Table 2: Comparison of offline and online times for CONSIGN and pixel-wise
Method
Offline (min)
Online
(s)
Online $ = 10 (s) Online $ = 10 (s)
CONSIGN
15
0.026 0.040
0.4
40
PW (RAPS)
~ 011-1
~ 0.2 - 0.5
~ 0.4-2 -
~ 15
offline time is higher due to the SVD, but mostly due to the numerical solution of the minimization problem. However, the most important metric the online time. Our method demonstrates faster online processing times for smaller sample sizes S. This is because sample vector in R K instead of selecting a possible label for each pixel in the label set TPW When we sample a large number of segmentations, the reconstruction process becomes computational times. Nevertheless, our method maintains competitive efficiency overall, remaining fast even with larger sample sizes. The online computational time during the online phase includes Singular Value Decomposition (SVD), which adds only constant time of approximately 2 20 ms per image.
expensive, resulting in higher
E ADDITIONAL EXPERIMENTS
In Figures method. In Tables 3 we provide the calibrated À for the experiments of Section 3 and further experiments.
10
2 13 14 15
we show additional quantitative and qualitative result of our
CONSIGN;
(RAPS)
MnM2
ms
LID
COCO animals
COCO vehicles
0.05 $ 0.85
0.1 B 0.85
0.05 B 0.75
0.15 0.7
0.3
0.75
Chao Estimator
3.0
107
1.0
107
103
3.00
100
10a
30a
3.0
10
105
10
30
30"
164
10a
10a
200
102
10a
10000
10000
10000
10000
10000
samples
Figure 6: N, NoH(WWW and NCH (YSACP) for different experiments and principal com-
ponents K
17

---

Published as a conference paper at ICLR 2026
Table 3: Calibrated À across different experiments and splits for CONSIGN
Dataset
(a, B, K)
Fold 1
À Fold 2
Á Fold 3
À Fold
Á Fold
M&Ms-2
(0.1, 0.9, 2)
0.060
0.090
0.070
0.070
0.090
(0.1, 0.9,
0.050
0.070
0.070
0.060
0,070
(0.05, 0.85, 2)
0.250
0.270
0.230
0.260
0.320
(0.05, 0.85, 5)
0.150
0.160
0.120
0.100
0.240
(0.25, 0.95, 2)
0.700
0.500
0.700
0.600
0.800
(0.25, 0.95,
0.250
0.250
0.300
0.250
0.250
(0.05, 0.9,
1.800
4.200
1.800
1.900
3,300
(0.05, 5)
0.700
1.000
0.700
0.700
1.000
MS-CMR19 19
00.1, 2)
0.060
0.060
0.030
0.060
0.080
00.1, 5)
0.050
0.050
0.030
0.050
0.050
(0.05,
0.080
0.080
0.040
0.110
0.100
0.8,
0.060
0.060
0.030
0.060
0.080
(0.15,
0.200
0.250
0.150
0.350
0.200
0.9,
0.120
0.140
0.080
0.140
0.100
(0.2,
0.080
0.100
0.060
0.080
0.070
0.9.
0.060
0.060
0.050
0.060
0.060
LIDC
(0.2, 0.8, 2)
0.010
0.020
0.020
0.010
0.020
(0.2, 0.8.
0.010
0.020
0.020
0.010
0.020
(0.05, 2)
0.020
0.020
0.020
0.020
0.020
0.75,,
0.020
0.020
0.020
0.020
0.020
00.05,
0.050
0.040
0.050
0.050
0.050
(0.05, 0.85,
0.050
0.040
0.050
0.050
0.050
(0.1, 2)
0.050
0.060
0.050
0.050
0.060
0.9.
0.050
0.040
0.050
0.050
0.050
COCO an.
(0.25, 0.8, 2)
0.030
0.030
0.030
0.040
0,040
(0.25, 0.8, 5)
0.020
0.020
0.020
0.020
0.030
(0.15, 0.7,2)
0.060
0.060
0.070
0.060
0.180
(0.15, 0.7.
0.040
0.040
0.050
0.040
0.050
0..2,
0.020
0.040
0.040
0.040
0.080
(0.2, 0.75, 5)
0.010
0.020
0.020
0.020
0.030
0.7,
0.01
0.01
0.01
0.01
0.02
0.7,
0.01
0.01
0.01
0.01
0.01
COCO veh.
(0.3, 0.7,
0.110
0.030
0.060
0.060
0.030
(0.3, 5)
0.060
0.020
0.030
0,030
0.020
00.3,
0.330
0.300
0.300
0.300
0.300
(0.3, 0)
0.120
0.100
0.110
0.110
0.100
(0.25, 0.7.2)
0.500
0.300
0.300
0.300
0.300
(0.25, 0.7.5)
0.150
0.100
0.100
0.150
0.100
(0.35, 0.75, 2)
0.100
0.050
0.050
0.100
0.050
(0.35, 0.75, 5)
0.060
0.030
0.030
0.040
0.020
18

---

Published as a conference paper at ICLR 2026
Table Calibrated À across different experiments and splits for pixel-wise (RAPS) method
Dataset
(ax, 3)
Å Fold
Á Fold
À Fold
Fold 4
Fold 5
M&Ms-2
(0.1, 0.9)
0.610
0.710
0.630
0.640
0,690
(0.05, 0.85)
0.850
0.860
0.850
0.860
0.910
(0.25, 0.95)
0.630
0.640
0.630
0.620
0,630
(0.05, 0.9)
0.930
0.930
0,920
0.930
0.960
MS-CMR19
(0.1, 0.85)
0.670
0.690
0.600
0.670
0.690
(0.05, 0.8)
0.790
0.790
0,680
0.790
0.790
(0.15, 0.9)
0.750
0.750
0.710
0.760
0.740
(0.2, 0.9)
0.650
0.660
0.640
0.670
0.660
LIDC
(0.2, 0.8)
0.690
0.690
0.690
0.690
0.690
(0.05, 0.75)
0.830
0.810
0.810
0.810
0,820
(0.05, 0.85)
0.900
0.900
0.900
0.900
0.900
(0.1, 0.9)
0.900
0.900
0.890
0.900
0.900
COCO an.
(0.25, 0.8)
0.560
0.560
0.570
0.590
0.610
(0.15, 0.7)
0.710
0.710
0.710
0.710
0.720
(0.2, 0.75)
0.590
0.610
0.620
0.620
0.670
(0.2, 0.7)
0.520
0.510
0.540
0.540
0.580
COCO veh.
(0.3, 0.7)
0.710
0.670
0.680
0.680
0.640
(0.3, 0.75)
0,810
0.750
0.760
0.780
0.740
(0.25, 0.7)
0,820
0.800
0.800
0.800
0,800
(0.35, 0.75)
0.700
0.660
0.680
0.680
0.620
Table 5: Calibrated À across different experiments and splits for SACP method
Dataset
(o,, (3)
Å Fold
Î Fold
î Fold
Å Fold 4
Fold 5
M&Ms-2
(0.1, 0.9)
0.580
0.680
0.600
0.620
0.670
(0.05, 0.85)
0.820
0.850
0.820
0.820
0.870
(0.25, 0.95)
0.600
0.610
0.610
0.600
0,600
(0.05, 0.9)
0.870
0.890
0,880
0.880
0.920
MS-CMR19
(0.1, 0.85)
0.660
0.660
0.600
0.660
0.660
(0.05, 0.8)
0.770
0.770
0.680
0.770
0.770
(0.15, 0.9)
0.710
0.720
0.670
0.720
0.710
(0.2.
0.630
0.640
0.610
0.650
0,630
LIDC
(0.2, 0.8)
0.710
0.720
0.720
0.710
0.720
(0.05, 0.75)
0.830
0.820
0.820
0.820
0.840
(0.05, 0.85)
0.900
0.900
0.890
0.900
0.900
(0.1, 0.9)
0.890
0.890
0.890
0.890
0.890
COCO an.
(0.25, 0.8)
0.560
0.560
0.570
0.590
0.610
(0.15, 0.7)
0.710
0.710
0.710
0.710
0.730
(0.2, 0.75)
0.590
0.600
0.630
0.630
0.670
(0.2, 0.7)
0.520
0.510
0.540
0.540
0.590
COCO veh.
(0.3, 0.7)
0.710
0.670
0.680
0.680
0.640
(0.3, 0.75)
0.810
0.750
0.760
0.780
0.750
(0.25, 0.7)
0,820
0.800
0,800
0.800
0,800
(0.35, 0.75)
0.700
0.660
0,680
0.680
0.620
19

---

Published as a conference paper at ICLR 2026
CONSIGN:
CONSIGN,
PW (RAPS)
SACP
MnM2
ms
LDD
COCO animals
COCO vehicles
0=
0.25
Q
0.15 8 0.9
1f
0.05 p= 0.85
0.2 g = 0.75
q 0.25 B= 0.7
Chao Estimator
207
107
30'
10a
10c
10a
10
10a
10a
10a
102
10000
0
10000
10000
0
10000
0
10000
samples
Figure 7: ()*), (Y and NoH (YSACP) for different experiments and principal com-
ponents K
CONSIGNS
CONSIGN,
PW (RAPS)
SACP
MnM2
ms 19
LID
COCO animals
COCO vehicles
10r
0.1
0.9
0.35
0.75
Chao Estimator
3.'
207
3.0
100
10c
10a
20
3.0
10
10
205
100
10
10
30
10"
10
10a
10a
10"
10
10
10'
10000
10000
10000
10000
samples
Figure 8: NoH()*), NcH(()W)) and No (DSAACP) for different experiments and principal com-
ponents K
F MISCELLANEOUS
In Figure 16 we show how CONSIGN can perform well even where the uncertainty is concentrated in small-scale details.
Finally, Figure 17 illustrates the relationship between the model predictions and the principal com- ponents derived from their SVD. For each experimental setting (as defined in Figure 5) we show the segmentation outputs of the pre-trained model f. along with the corresponding first four principal vectors un E R XWW extracted from the label-O channel. Recall that u R xxxxx XWW
20

---

Published as a conference paper at ICLR 2026
1-2
CONSIGN;
CONSIGN:
PW (RAPS)
SACP
MnM2
mscmr19
LIDC
COCO animals 0.15 0.7
COCO vehicles 0.3 8= 0.75
0.05 B 0.85
0.1 0.85
0.05 8 0.75
1.0
0.8
SEC
0.6
0.4
10
5000
10000
10
5000
10000
10
5000
10000
10
5000
10000
10
5000
10000
samples
Figure 9: )EC(""), sEC(YPW) and SEC(SAAC)) for different experiments and principal com-
ponents K
1-2
CONSIGN;
CONSIGN,
PW (RAPS)
SACP
Mn
mscmr19
LIDC
COCO animals
COCO vehicles a= 0.25 $ 0.7
0 =
0.25 8 0.95
a= 0.15 B= 0.9
a 0.05 A 0.85
0= 8 0.75
1.00
0.75
SEC
0.50
0.25
10
5000
10000
5000
10000
10
5000
10000
5000
10000
5000
10000
#
samples
Figure 10: sEC()*) sEC(PPW) and SEC(SAAC)) for different experiments and principal com-
ponents K
1-2
CONSIGN;
CONSIGN
PW (RAPS)
SACP
MnM2
mscmr19
LIDC
COCO animals 0.2 8= 0.7
COCO vehicles
0.05
0.9
0.2 8 0.9
0.1 8 0.9
0.35 8 0.75
1.00
0.75
SEC
0.50
0.25
10000
10000
5000
10000
5000
10000
5000
10000
# samples
Figure 11:
and ACC() for different experiments and principal com-
ponents K
MnM2
mscmm19
LDC
COCO animals
COCO vehicles
== 0.
05.05 P 0.85
a 0.18800.88 0.85
a 0.05 0.75
0.15 (2:555510 ==
0=0 0 0.75
0.4
Correlation
0.4
0.3
0.3
0.2
0.2
0.2
0.2
0.0
0.1
# samples
Figure 12: A PW) and P(YSACP) for different experiments and principal components K
21

---

Published as a conference paper at ICLR 2026
CONSIGN;
CONSIGN
PW (APS)
SACP
UDC
COCO
COCO
0.4
Correlation
0.4
0.2
0.4
0.3
0.1
0.2
0.2
0.1
500
1000
500
1000
500
1000
500
1000
500
1000
# samples
Figure 13: (())), P(YPW) ((YP)) and p()SSCCP) for different experiments and principal components
CONSIONG
CONSSNN
PW IRAPS)
-8 SACP
Mnkk
micm719
UDC
COCO animals
COCO vehicles
0.05
0.9
8 p= 0.9
=
o= 0.2 0.7
o 0.35 0
0.4
Correlation
0.4
0.3
0.2
0.2
0.0
500
1000
1000
500
1000
500
1000
500
1000
# samples
Figure 14: (()*), P(yPW) and p(ySACP) for different experiments and principal components K
CONSIGN
Pixel-Wise (RAPS)
SACP
Image
GT
MANSZ 6:0.55 E-1
Recippurd
Mysuniam
3igglt Hetricee
NS-CHWREE und.
Mynuddan
Shalk
UIC A P-0.7S
aE ur0-15 P-07
Peroon
COCO vel, 2n0.3 Bocii ueel
Pyyoie
CCP
Figure 15: Qualitative comparison between samples from y (K = 5), Pw and SAAP
22

---

Published as a conference paper at ICLR 2026
COWSIOV
cowd
Figure 16: Samples from ) (K 5) showing that CONSIGN is able to capture uncertainty also for small regions.
Raw predictions
Principal components
Image
Figure 17: Example segmentation predictions produced by the pre-trained model f. and the corre- sponding first four principal vectors uf. u². u u& of the label-O channel obtained from the SVD analysis. Each row corresponds to of the experimental settings shown in Figure
23


Reviews:
[{"review_id": "a2vrunG16B", "reviewer": "Reviewer_jv9m", "summary": "The authors propose an approach, CONSIGN, within the conformal prediction framework to calibrate uncertainties output by an arbitrary probabilistic segmentation model. CONSIGN improves upon the existing literature in that it takes into account spatial correlations when outputting prediction sets with user-specified error guarantees. Prediction sets are derived from an SVD decomposition of the samples. The calibration involves an auxiliary constrained optimization step. The approach is demonstrated on 3 medical imaging datasets (M&Ms-2, LIDC and MS-CMR19) as well as two COCO subsets. The approach is tested on top of different segmentation frameworks across these applications: a dropout Unet, probabilistic Unet, and an ensemble networks strategy. The authors show qualitatively superior samples in the prediction sets compared to pixel-wise CP baselines and SACP, as well as report smaller prediction sets (in terms of Chao estimator and higher Pearson correlation between samples) at a fixed error-level, and better compliance to the specified error-level (sampled Empirical Coverage).", "questions": "5. It is not completely clear how Eq. 6 is derived and whether with this form, prediction sets have some guarantee of optimality. In particular as lambda is weighted by Sigma_k,k in Eq. 6, the range of values for A_k, B_k as a function of a_k,b_k is not completely clear. Sigma_k,k could also have appeared as a weight to c_k u_k(X) in the equation afterwards. Could the authors provide more details of derivations?\n\n6. Can the authors justify the following claim : \"In other words, even with a truncated PCA the prediction set will always include the ground truth, therefore we can use the standard CRC approach\" ? I would assume that in some failure cases, none of the segmentation samples generated by the segmentation backbone will cover the ground truth; hence any prediction sets derived from them will in turn not cover the ground truth. Can this have substantial effects on the outcome of the proposed method in some degenerate cases?\n\n7. Given that prediction sets for the proposed approach are derived with spatial awareness vs. pixel-wise for PW (RAPS), it is surprising that the estimated volume of prediction sets does not differ by more orders of magnitude in most experiments in Fig 2. Can the authors comment on this? The number of pixel-wise combinations should grow extremely fast compared to the number of spatially-consistent configurations.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Significance and originality: The approach and the problem addressed are interesting, and this is ultimately what informed my choice of rating. There is potential for clear benefits from this type of approach as ignoring spatial correlations in segmentation uncertainties is problematic.\n\n2. Qualitative and quantitative results over several datasets and segmentation backbones are promising.", "weaknesses": "1. It is not clear how alpha and beta are chosen across all applications (they differ for each application) and whether results generalize well to other values. Can the authors comment on these choices?\n\n2. The method makes an i.i.d. assumption on the calibration and test sets. This is a strong assumption. In practice, guarantees of validity of the prediction sets may silently fail to apply for images with artefacts and other distribution shifts (unseen pathologies, different image quality, etc.). Yet these are exactly the cases that one would like to be flagged by UQ.\n\n3. Overall readability/presentation could have been slightly improved (not a major concern however). \n\n4. See questions and minor weaknesses below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eOJHegbUDZ", "reviewer": "Reviewer_6bMr", "summary": "The submission studies conformal prediction for segmentation models. The submission proposes to construct prediction sets that take into consideration the spatial correlation of the predictions of the base model in order to construct more informative prediction sets. The proposed method uses PCA to find principal directions along which to perturb the raw output of the model.\n\nExperiments compare the proposed method with existing baselines on diverse datasets, containing both medical and natural images.", "questions": "**Exit condition of Algorithm 1**\n\nThe main limitation of the current method is that it is not guaranteed to terminate with a finite parameter $\\hat{\\lambda} < \\infty$. This seems to contradict the claim that the proposed method will eventually find a $\\lambda$ that contains the ground truth.\n\n- Could the authors expand on whether they experienced this behavior in their experiments? \n\n- Could $\\beta$ be chosen adaptively to guarantee termination of the algorithm?\n\n**Experiments**\n\n- Figure 1 should be introduced more clearly. It is only afterwards, in the experimental results, that it is explained how these segmentation masks are generated via sampling. To a first time reader, it is not clear how Figure 1 is generated.\n\n- The fact that sampling from pixel-wise uncertainty intervals does not respect spatial correlations isn't particularly surprising on its own. Pixel-wise uncertainty intervals are not constructed to sample individual segmentation masks. So, it might be misleading to mention these as inherent limitations of existing works, as it is not what they are designed to solve.\n\n- Have the authors considered applying PCA to the pixel-wise uncertainty intervals? This would also allow to sample segmentation masks that take into consideration the spatial correlation of the pixels. I wonder whether comparing with this baseline could provide stronger evidence in support of the proposed algorithm, that integrates spatial correlation at a deeper level.\n\n- The name \"sampled empirical coverage\" might be confusing because it is reminiscent of the marginal statistical guarantee each method provides. That is, readers might expect all methods to provide sampled empirical coverage at level $1 - \\alpha$ by design.\n\n- It might be helpful to compare with Blot et al. \"Automatically Adaptive Conformal Risk Control\", who include examples on semantic segmentation.\n\n---\n\n**Minor comments**\n\n- The operator $P$ is never defined. It is unclear what taking the argmax of a vector in $\\mathbb{R}^{W \\times H \\times L}$ means. I assume $P$ first reshapes the column vector to $W \\times H \\times L$ and then takes the argmax along the $L$ dimension. This should be made clear.\n- Is $\\beta$ an error rate or an accuracy rate? Typo on line 206 $\\geq 1 - \\beta$?\n- It might be helpful to explicitly define the indicator function in Eq. (8), or to spell it out as set inclusion.\n- It might be helpful to include the raw predictions of the model in Fig. 5", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* Calibration of segmentation models is an important and under-developed field.\n* Applying PCA decomposition is novel for conformal prediction of segmentation models.\n* Experiments are comprehensive.", "weaknesses": "* Presentation could be made more precise.\n* The proposed algorithm is not guaranteed to terminate.\n* The construction is interesting, at the cost of less interpretable prediction sets.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KKM94gHJxx", "reviewer": "Reviewer_cy3U", "summary": "This paper introduces CONSIGN to improve uncertainty quantification in image segmentation. By leveraging spatial correlations between pixels, CONSIGN produces more meaningful prediction sets with rigorous coverage guarantees compared to traditional pixel-wise conformal prediction methods. The key innovation of CONSIGN is its ability to create spatially-aware prediction sets by incorporating principal component analysis to capture uncertainties in correlated regions, resulting in tighter, more interpretable prediction intervals. Through experiments on medical imaging datasets and COCO dataset subsets, the authors show that CONSIGN significantly outperforms existing CP-based methods, offering improved performance across multiple metrics, including uncertainty volume, empirical coverage, and correlation of samples.", "questions": "1. Could there be a more robust way to incorporate model uncertainty into the CONSIGN framework to mitigate potential performance degradation?\n2. CONSIGN improves uncertainty quantification by leveraging spatial correlations, how does the method perform in regions with small or subtle uncertainties, such as fine boundaries in segmentation? Are the spatial correlations still effective in these contexts, or does the method tend to overestimate uncertainty in such cases?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. CONSIGN effectively incorporates spatial correlations between pixels, a critical aspect in image segmentation tasks. This spatial awareness improves the quality of uncertainty estimates by reducing the size of prediction sets, providing more meaningful and interpretable results compared to traditional pixel-wise CP methods.\n2. CONSIGN is highly adaptable to various applications and datasets without requiring significant changes to the underlying model architecture.", "weaknesses": "- The method assumes that the calibration and test datasets are exchangeable, which may not hold in many real-world scenarios where distribution shifts or out-of-distribution data are present. The current version of CONSIGN does not address such cases, and extending it to handle distribution shifts could significantly improve its robustness in dynamic environments.\n- it heavily relies on the pre-trained model to generate multiple sample predictions. This means that the method's performance is closely tied to the quality of the pre-trained model, and it may not perform well (like inflated prediction sets) if the base model produces unreliable or poor-quality predictions\n- The method excels at identifying larger uncertain regions, but for small, localized uncertainties (e.g., fine-grained segmentation boundaries), the spatial grouping approach might still lead to overly broad prediction sets. Fine-tuning how small-scale uncertainty is handled, or integrating multi-resolution strategies, could improve the method's accuracy in these cases. \n- The calibration step involves solving a constrained minimization problem to identify the optimal parameter λ, this process can be numerically challenging and does not guarantee global optimality", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)