COMPASS: Robust Feature Conformal Prediction for Medical Segmentation Metrics
Authors: Matt Y. Cheung
Ashok Veeraraghavan
Guha Balakrishnan
Venue: ICLR 2026 Poster (2026)
Keywords: Conformal Prediction
Segmentation
Imaging
Uncertainty Quantification
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
TL;DR: We propose a practical and efficient solution to feature conformal prediction for medical segmentation metrics.

Abstract: In clinical applications, the utility of segmentation models is often based on the accuracy of derived downstream metrics such as organ size, rather than by the pixel-level accuracy of the segmentation masks themselves. Thus, uncertainty quantification for such metrics is crucial for decision-making. Conformal prediction (CP) is a popular framework to derive such principled uncertainty guarantees, but applying CP naively to the final scalar metric is inefficient because it treats the complex, non-linear segmentation-to-metric pipeline as a black box. We introduce COMPASS, a practical framework that generates efficient, metric-based CP intervals for image segmentation models by leveraging the inductive biases of their underlying deep neural networks. COMPASS performs calibration directly in the model's representation space by perturbing intermediate features along low-dimensional subspaces maximally sensitive to the target metric. We prove that COMPASS achieves valid marginal coverage under the assumption of exchangeability. Empirically, we demonstrate that COMPASS produces significantly tighter intervals than traditional CP baselines on four medical image segmentation tasks for area estimation of skin lesions and anatomical structures. Furthermore, we show that leveraging learned internal features to estimate importance weights allows COMPASS to also recover target coverage under covariate shifts. COMPASS paves the way for practical, metric-based uncertainty quantification for medical image segmentation.

Published as a conference paper at ICLR 2026
COMPASS: ROBUST FEATURE CONFORMAL PREDIC- TION FOR MEDICAL SEGMENTATION METRICS
Matt Y. Cheung, Ashok Veeraraghavan & Guha Balakrishnan
Department of Electrical & Computer Engineering
Rice University
Houston, TX 77005, USA
{yc130, av21, gb35 (p35)erice..o
ABSTRACT
In clinical applications, the utility of segmentation models is often based on the accuracy of derived downstream metries such as organ size, rather than by the pixel-level accuracy of the segmentation masks themselves. Thus, uncertainty quantification for such metrics is crucial for decision-making Conformal pre- diction (CP) is popular framework to derive such principled uncertainty guar- antees, but applying CP naively to the final scalar metric is inefficient because it treats the complex, non-linear We introduce COMPASS, a practical framework that generates efficient, metric- based CP intervals for image segmentation models by leveraging the inductive biases of their underlying deep neural networks. COMPASS performs calibra- tion directly in the model's 's representation space by perturbing intermediate fea- tures along oow-dimensional subspaces maximally sensitive to the target metric. We prove that COMPASS achieves valid marginal coverage under the assump- tion of exchangeability. Empirically, we demonstrate that COMPASS produces significantly tighter intervals than traditional CP baselines on four medical image segmentation tasks for area estimation of skin lesions and anatomical structures. Furthermore, we show that leveraging learned internal features to estimate impor- tance weights allows COMPASS to also recover target coverage under covariate shifts. COMPASS paves the way for practical, metric-based uncertainty quantifi- cation for medical image segmentation.
commentttiiontommettien pipeline black box.
INTRODUCTION
Uncertainty quantification is of critical need in medical image analysis, a field used for decision support high-stakes clinical diagnosis and treatment planning applications (Begoli et al. 2019 Abdar al. 2021). A fundamental task in medical image analysis is image segmentation, the task of separating anatomical structures and lesions from each other within an image. Deep learning models, particularly U-Net variants (Ronneberger et al., 2015| [sensee et al.) 2021), have achieved state-of-the-aa performance medical image segmentation. In practice, the outputs of these models ("segmentation maps") are often treated as intermediate result that are then used to automatically derive downstream metrics of interest (known as "radiomics"), such as the areas/volumes or tex- ture patterns of specific anatomic regions (Figure left). These derived metrics then used for decision support to guide clinicians in diagnosis and treatment.
popular, statistically principled uncertainty quantifi- cation framework of choice in machine learning, providing guarantees without restrictive distribu- tional assumptions (Vovk et al., 22055] Shafer Vovk, 20088 Fontana al.) 2023 Papadopoulos et al. 2002) (Angelopoulos & Bates] 2021). While well-studied in the context of typical prediction tasks involving scalar output variables, CP is less explored for tasks such as medical image segmen- tation, in which the output variables are images. Existing CP methods for segmentation typically focus on deriving bounds for pixel-level errors (Mossina et al. 2024 Mossina & Friedrich) 2025] Brunekreef et al. 2024 Angelopoulos et al. 2022), which, while useful for understanding varia- tions of local segmentation contours, may yield meaningless or misaligned intervals for downstream derived metries. On the other hand, recent study shows that treating the segmentation-to-nett
Conformal prediction (CP) has emerged

---

Published as a conference paper at ICLR 2026
pipeline as a black box and performing CP directly the output metric space yields intervals that are well-aligned to the metrics (by construction). but are also often inefficient (i.e., large) because the internal biases of the pipeline are not exploited in the vanilla CP formulation (Cheung et al. 2025).
To achieve more efficient intervals, promising direction is to leverage the powerful inductive biases of neural networks by performing CP on their intermediate representations. Feature Conformal Prediction (FCP) (Teng et al., 2022 Tang al. 2024, Chen et al.. 2024) shows that by working in a semantic feature space, it is possible to generate provably tighter prediction intervals. However, the FCP algorithm requires solving a complex optimization to find the closest adversarial feature vector for each data point, which is computationally prohibitive for high-dimensionas feature spaces of typical CNN and transformer architectures which are the workhorses of modern medical image segmentation.
To bridge this gap, we introduce Conformal Metric Perturbation Along Sensitive Subspaces (COMPASS), framework to perform feature CP in tractable manner to generate valid and effi- cient prediction intervals for any (differentiable) metric derived from the output of a neural network. The core concept behind COMPASS is to linearly perturb the features outputted by the network at a particular layer along data-specific directions that are highly sensitive to the metric of interest (Figure center). To make this process tractable for typical neural network layers with a large num- ber of features, we propose computing a low-dimensional manifold of any given layer by applying principal component analysis (PCA) the gradients of the output metric with respect each of the layer's features. This manifold represents the principal directions of sensitivity of the output with respect that layer. We prove that linear perturbations in the latent space achieve marginal coverage in the output metric space under exchangeability, i.e., for fresh test point, the ground truth will be contained within the prediction intervals generated by linear feature perturbations. Furthermore, we show that simple weighted variant of COMPASS may be used to correct for covariate shifts.
We evaluated COMPASS on public datasets for four medical image segmentation tasks: col- orectal cancer in histopathology images (EBHI) (Hu et al.) [023), skin lesion segmenta- tion (HAM10000) (Tschandl et al.) (018)) thyroid nodule segmentation in ultrasound images (TN3K) (Gong et al.) 22021), and polyp segmentation in endoscopic images (Kvasir) (Jha et al.) 2019). Results show that while standard CP methods achieve valid coverage, they often produce unnecessarily wide prediction intervals. On the other hand, COMPASS finds semantically meaning- ful directions in the latent space that correspond to monotonic metric changes, resulting in efficient intervals. Furthermore, the weighted extension of COMPASS recovers target coverage under co- variate shifts and is the most efficient across all weighted baseline methods. COMPASS paves the way for practical, metric-based uncertainty quantification for medical image segmentation.
2 METHOD
2.1 THEORETICAL COVERAGE UNDER LINEAR LATENT PERTURBATIONS
To provide rigorous uncertainty quantification for the metric Y, we consider linear perturbations in latent space along a direction A E 2 and define intervals in the metric space. We first formalize the nestedness condition, which is a fundamental requirement for the validity of any conformal procedure (Vovk et al.) 2005, Shaferr & Vovk 2008).
Definition (Nestedness). Let Sp(x) be family of prediction sets for an input x E X, parameter- ized by 3 2 O. The family {Sa(z))820 is said to be nested for x if 31 S B2 = S0, (x) C S3, (z).
This condition guarantees that a larger perturbation magnitude yields a larger (or equal-sized) pre- diction set, which is necessary for the quantile-based coverage proof. While standard CP methods satisfy this trivially, it becomes a non-trivial condition for deep feature spaces. We now present our main theorem. Crucially, we construct our prediction sets Sy (z) in a way that guarantees nestedness by definition, thereby ensuring the validity of the conformal procedure.
Theorem
Coverage under Linear Latent Perturbations). Let (X,, Y:) be ex- changeable random pairs with X; X and Y, R, and split the data into training set D and calibration set D Drm = {(X,, Y;) Using Drr. fit segmentation model with decoder g Z + S, and let (xx) E Z denote the latent vector computed deterministically by the trained encoder for in-
2

---

Published as a conference paper at ICLR 2026
Segmentation Pipeline
Calibration
Testing
Output Metric
Input image
FCP hp ball
B
B
p
2 B
Model
Ri
Int1,o
Put1
Pm+1,hi
Conformal Guarantee:
PP (Yn+1 E (g(anti - BBn+z)), h (g(2n+1 + BBn+1)) Der)) 2 1
Figure 1: Overview of COMPASS. (Left) A medical image segmentation network predicts seg- mentation map" from an input image. We conceptually decompose this network into a function which maps the image to latent features E, and function that maps & to the output map. The map may then used to compute a (differentiable) downstream metric ý via the function (Center) We linearly perturb calibration features in sample-specific direction A to find the scores Rg The perturb the features En+1 in the direction An+1 with magnitude B. By Theorem 1, our interval construction is guaranteed to be nested. Therefore, under the assumption of exchangeability. the resulting predic- tion interval achieves marginal coverage (bottom).
scores used to find the conformal quantile $. (Right) At test time for subject +
put v. Let h : S R be measurable metric, and let 2 be any measurable direction that depends only Dic
For x X and B 2 0, define the metric function along this direction as:
m²(b) = (h g)(((=)++++)))))))))))())))))))) bD), for b E R.
(1)
define the prediction set Sz(x) as the range of the metric function over the perturbation interval [-B, +B]:
S2(x) = min (b) max mcr(b)
(2)
[bEl-B,+0]
By construction, this guarantees that the family {Sp(z)}020 is nested (Definition 7 For each calibration pair (X,; Yd), define the non-conformity score:
R; = in 2:: Sz(X,)) E (0,
(3)
and let 3 be the [11 (11-ol (n 1)1-th smallest value among {Ri.... Ru), where (0,1) is the user-specified mis-coverage level.
Then, for a fresh test pair (Xn+1, Yu41), the prediction (2(4n+1) satisfies
PYYn+1 E 5g((n+1) Drt) 2 1-
(4)
Proof sketch. Exchangeability of (X,, Y:) ensures that the rank of the test score Ra41 among [By..... Rn, Rn+1} is uniformly distributed. By the construction of Sz(a) as the range [min(-), max( the nestedness condition is satisfied by definition. Therefore, by construction of 3, the standard conformal guarantee holds. See Appendix A full proof.
Intuition. Our prediction set Sg(z) is defined as the metric range over the perturbation interval, which satisfies the nestedness condition required for CP. Our construction forms a conservative envelope that explicitly accounts for any non-monotonic metric behavior within the perturbation interval. Our non-conformity score R3 is then the minimal perturbation magnitude / required for this envelope to contain the ground truth metric Y,
2.2 COMPASS: CONFORMAL METRIC PERTURBATION ALONG SENSITIVE SUBSPACES
The proposed framework COMPASS is built on the insight that we can directly calibrate down- stream metric by perturbing model's 's representations along specific, data-dependent directions, A.

---

Published as a conference paper at ICLR 2026
As our theoretical guarantee in Theorem I Japplies to any measurable perturbation direction, the key to an efficient method lies in choosing a direction that is highly sensitive to the metric of interest. We explore two natural choices for this representation space for segmentation: the model's final output logits and a deeper internal feature layer.
The first and simplest approach (COMPASS-L) operates the model's final logits. A uniform, scalar shift applied to this tensor directly modulates the model's overall confidence before the final activation function. This is equivalent defining the sensitive direction, A,, tensor of ones. A richer alternative is to perturb an internal feature representation E, as done in FCP (Teng et al., 2022). However, naive search for an optimal perturbation in high-dimensional spaces computationally intractable. Furthermore, arbitrary perturbation directions may cause the metric to change in an erratic or non-smooth ways, resulting unnecessarily wide and inefficient intervals.
To overcome this, we propose a data-driven method (COMPASS-J) to identify a low-dimensional sensitive subspace that is globally effective] Given metric function in Equation [ that is differen- tiable, we first compute Jacobians of the metric ý with respect to to provide a local, linear map of the metric's s sensitivity with respect to the features for each training sample:
:=:= dh(((E))) de;
where 2; - f(; ) C' is the number of channels, and Dj is the i-th spatial dimension. Because the full spatial Jacobian is often too high- -igh-dimensional to be practical, we sum the spatial dimensions and apply Principal Component Analysis (PCA) to the set of these vectors from the training set, (J,), and select the matrix of the top I. eigenvectors Vi. E RCXL
For any given sample, we find its sensitive direction d: by projecting its sensitivity vector onto this learned subspace, and normalization it to produce direction vector A,
d; = V.VV J. d:d////dell
2.3 COMPASS CALIBRATION AND INFERENCE
The goal of COMPASS is to find the smallest symmetric perturbation magnitude B such that the en- tire interval (z) contains the ground truth value y. In general, computing S (z) requires finding the minimum and maximum of the perturbed metric response mz, (b), E 1-3, 3]. When the met- ric response is non-mon mon-monoo,ic, computing the extrema requires a full discretized sweep across the perturbation range. Performing this sweep every step during calibration makes the envelope com- putation expensive because each candidate / requires many forward passes to map out the complete response curve for each sample.
However, when Ma; (b) is monotonic in b, the conservative envelope collapses to evaluations at the endpoints: S3xx;)) = (m,;(-3), my, (+3)]. (+,++)]] Thus, the interval can be computed with only two forward passes, giving a practical and efficient implementation of COMPASS. A perturbation sweep still required. but only once per sample, solely to verify monotonicity rather than to repeatedly compute the envelope during calibration. As we will see in Figures and 12 this monotonicity condition holds across all of our experiments.
Once Sa(;;) can be evaluated, either via the conservative full sweep the endpoint method, the non-conformity scores { {R, follow directly from Equation 3 The conformalized quantile is:
i = @
(1--a)(n + +111
and at test time we return the interval (z(m++1).
For a full outline the algorithms, refer to Appendix C Furthermore, while symmetric perturbations effective when the metric responds similarly to positive and negative perturbations, often observe an asymmetric relationship when the metric responds differently in positive and negative directions. In such cases, asymmetric calibration, where find separate non-conformity scores for the upper and lower bounds, is necessary to construct a more adaptive and efficient interval. We discuss this asymmetric version of COMPASS and provide equivalent algorithm in Appendix B
!Or use of principal directions to restrict the perturbation search space is conceptually related to Belhasin 2023), where authors employed principal directions construct prediction sets for inverse problems.
4

---

Published as a conference paper at ICLR 2026
2.4 WEIGHTED COMPASS FOR DISTRIBUTION SHIFTS
While Theorem provides a robust guarantee of marginal coverage, its validity rests on the critical assumption that the calibration and test data are exchangeable. In many real-world applications, such as medical image segmentation, this assumption is frequently violated due to the variations in how the data collected, processed, and interpreted. Under such distribution shifts, the unweighted quantile 3 is no longer guaranteed to provide the target coverage level on the test set, leading to systematic undercoverage and unreliable prediction intervals.
To address this limitation, we employ Weighted Conformal Prediction (WCP) (iisshirani et al, 2019 Barber et al. (223) to restore the coverage guarantee by re-weighting the calibration samples. The weight for a calibration sample X, is ideally the density ratio w(X,) Ptest (X.)/Pea In practice, this ratio is unknown and typically estimated by training auxiliary classifier A X
[0. 1] to distinguish between samples from the calibration and test sets. The predicted probability p(z) - A(x) that sample belongs to the test set is then used compute the weights, effectively adjusting the calibration procedure to account for the distribution shift. We now extend Theorem to the weighted setting, providing formal coverage guarantee under distribution shift. The setup remains identical.
Proposition 1 (Validity of Weighted COMPASS under Covariate Shift). Lel Deal = {(x;, Y be exchungeable pairs drawn from distribution Peat and let (Xn+1; Yu+1) be fresh test pair from a potentially different distribution Prest. Let the non-conformity scores R₃ = R(X,, Y:) be computed described in Theorem 1 Let the weights w, w(X,) be the true density ratio Peest Xx))/ea((X,) Let Bw be the weighted (1 (1--a)-quantile the calibration scores (Ri...., Rn with corresponding weights {Wi..... ) defined as
2 a )
Bu = inf :20: En w, w
2j=1
Then, for the fresh test pair (Xn+1, Yn+1) ~ Prest, the prediction set SA.,, (Xn+1) satisfies
PV S (Xn+1) Der, Deal) 2
Proof Sketch. Sketch. This is a direct application of WCP (Tibshirani et al. 2019). The validity of this framework applies to any valid non-conformity score. Our scores are valid as they are from a deterministic function of the samples and the pre-trained model. When the weights represent the true density ratio, the weighted empirical distribution of the calibration scores is an unbiased estimator of the test score distribution. Thus, the coverage guarantee from the original theorem directly applies.
In practice the true oracle is not available, and the coverage guarantee holds approximately, with the quality of the approximation depending on the accuracy the density ratio estimates. To correct for the induced distribution shift, we explore three correction strategies: 1) Class: available ground truth class labels as features and weights are computed directly from the known class prevalences, 2) Latent: Model's latent representations summed on the spatial dimension as features (Lambert et al., 2024), and 3) Jacobian: model's internal geometric sensitivity (jacobians) summed as features.
EXPERIMENTS
We evaluated COMPASS across four medical image segmentation tasks: 1) segmentation on H&E
histopathology images from the EBHI dataset (H&E) (Hu et al. 2023). 2) skin lesion segmenta-
tion on dermoscopic images from the HAM10000 dataset (Skin Lesion) (sccannd al. 2018). 3) thyroid nodule segmentation from the TN3K dataset (Nodule) (Gong et al. (2021), and 4) gastroin- testinal polyp segmentation (PolyP) (Jha et al. 2019). We trained all models using the standard U-Net architecture from MONAI (Cardoso al. 2022 Kerfoot et al. 2018). We focused
seg-
mented object size (area) as the downstream climical metric of interest. Object size among the most widely adopted quantitative biomarkers across diverse clinical applications (Smith al.) 2003 O'Connor et al., 2008). For COMPASS-J, which requires a differentiable metric function, we com- pute area by applying soft thresholding sigmoid function) to the output logits and then summing
5

---

Published as a conference paper at ICLR 2026
the resulting probability map. We repeated each experiment over 100 randomized splits to compute average coverage and interval sizes. We provide details on architecture, preprocessing. training, calibration, and testing details in Appendix E We considered three types of calibration strategies:
1. Output-space calibration: These include split conformal prediction (SCP) (Lei ct al. 2018).
Conformalized Quantile Regression (Output-CQHi (Romano al. 2019), and locally adaptive formal prediction (Local CP) (Lei et al. 2018 Papadopoulos et al. 2008 2011).
2. End-to-end calibration: We use Conformalized Quantile Regression (E2E-CQR) (Lambert eet al. 2024) on a model trained to directly produce pixel-wise lower and upper bounds using the Tversky loss (Salehi al. 2017
3. Feature-space calibration:
These include COMPASS-L, which directly calibrates the model's final pre-activation outputs (logits), and COMPASS-J, which constructs prediction intervals by per- turbing latent features along dominant directions. As the strong monotonicity is empirically vali- dated Figures 2 and 12 all our experiments utilize the practical endpoint algorithm. We omitted the original FCP method (Teng et al. (022) from our main empirical comparison because we found its core adversarial search procedure for finding non-conformity scores to be computationally in- tractable for large feature spaces and to fail to reliably converge, preventing the computation of defined scores. We instead offer a comparison against conceptual oracle benchmark for FCP in Table 3 because FCP prediction set is defined by Lp ball in the latent space (Teng et al.] (0222) with bounds computed using Linear Relaxation based Perturbation Analysis (LiRPA) (Xu et al, 2020), empirically find the minimal radius of the Lp ball, which, when propagated through the decoding function using LiRPA, achieves the target coverage for the final metric.
3.1 STANDARD CONFORMAL PREDICTION
First, qualitatively find that COMPASS finds semantically meaningful directions in the latent space that correspond to monotonic changes in metric (Figure 2 for all applications. Next, we compare interval size across methods and datasets for a {0.T5, 0. 0.15) 0.15 (Table 1. We find that output space methods (SCP, CQR, Local) generally achieve valid coverage (Table 2 but yield longer intervals. End-to-end CQR provides tighter intervals. COMPASS (-J and -L) achieves valid coverage with a significantly reduced interval size compared to E2E-CQR. Moreover, we observe an efficiency benefit of using deeper representations (COMPASS-J) compared logits (COMPASS-L) for the majority of datasets. COMPASS-J generally achieves the shortest interval lengths overall. We also find that COMPASS maintains similar calibration stability as baseline methods (Figure 7 This highlights the efficiency of COMPASS methods.
Table 1: COMPASS achieves more efficient interval sizes compared to baseline methods across different target coverages. For datasets and 100 random splits, we compare interval lengths at
{0.15, 0.1, 0.05). We show the output space, end-to-end, and feature calibration methods in
red, blue, and green. The shortest mean interval lengths are bolded. For empirical coverages, see Table 2
Interval Size (Pixels?, Stadaard Deviation)
Dataset
COMPASS-J
COMPASS-L
E2E-CQR
Local
Ouiput-CQR
SCP
H&E
0.05
46372630
4408:432
5121#651
62974722
5646a358
55421676
H&E
0.10
3160+336
3139+375
3433+293
4223+558
3879+369
3509=333
H&E
0.15
2320=252
2354+146
2679+199
31754291
28194207
2550=196
Skin Lesion
0.05
1657=80
1689483
2569+195
3797+237
10857+65
3273#229
Skin Lesion
0.10
1179453
1208+58
1351+75
2433+101
4581436
1813+127
Skin Lesion
0.15
934+30
956+33
943+47
1865+50
2634+44
1124+77
Nodule
0.05
3257=210
3394+280
4150+265
3981+202
7481+46
4589+431
Nodule
0.10
2444=174
2810+180
158
3311+133
5603+57
3076+200
Nodule
0.15
2016+143
2082+142
2150+164
2877+111
4032164
2408=154
PolyP
0.05
54891575
63764769
9162+804
1239442577
8163+722
8570=766
PolyP
0.10
4056+293
43974469
61844616
5965+1011
49814675
62374564
PolyP
0.15
3394=290
3686+361
4528+487
4463+481
3913+326
4504+366

---

Published as a conference paper at ICLR 2026
GAm-20%
GAm-10%
GAm09 0%
GA 10m100 10%
GAm20% 20%
Ground Truth
HGE
3.00
50,00
-1.90
+0.01
-0.00
0.71
0 1.29
Nodule
-0.92
0.44
-0.00
0.45
6 0.71
Polyp
== -4,56
2.13
= -9.02
= 1.49
8 = 2.43
Figure 2: Visual verification of monotonicity to justify Endpoint-COMPASS As latent features are shifted along the COMPASS-J direction A, the induced segmentation volumes (red contours) monotonically expand. This is the key justification for using efficient Endpoint-COMPASS in our experimental setup, as it demonstrates mathematical equivalence with the rigorous Envelope- COMPASS. We show sample from each dataset with perturbation magnitudes 3 targeted at -20% -10,, 0% (original prediction), +10%, and +20% change in area (oA). We provide plot of all metric responses the testing datasets in Figure T2 and more visual examples in Appendix G
3.2 WEIGHTED CONFORMAL PREDICTION
To empirically validate the effectiveness of WCP under distribution shift, we design an experiment with controlled, adversarial label shift for the H&E and Skin Lesion datasets. We adopted dataset- specific partitioning strategy based on sample availability. For H&E, we trained on restricted subset to prevent data exhaustion, ensuring sufficient minority samples remained. Conversely, for Skin Le- sion, we utilized the full dataset, inducing shift by systematically reallocating the class distributions. For H&E, induced shift from "easy" "hard". We allocated 40% of the Adenocarcinoma samples to the calibration set and the remaining 60% to the test set. This results in a test distribution dominated by difficult samples, leading to baseline undercoverage. For Skin Lesion, we induced shift from "hard" to "easy". We allocated 30% of the Melanocytic Nevi (majority/easy) samples to the calibration set, forcing the calibration
be composed primarily of diverse, difficult lesions. The remaining 70% of the Nevi were allocated to the test set. This resulted in a calibration set that was significantly more difficult than the test set, leading to baseline overcoverage. See Appendix for more details.
For output-space calibration methods, we used weights based on the ground truth class labels, which we consider as approximately "oracle" weights. Note that the theoretically perfect oracle weights are defined by the density ratio, which precisely corrects for the change in the full, high-dimensional distribution of features between the calibration and test sets. For end-to-end and feature space cali- bration, we used the latent features (Lambert et al. 2024 Woodland et al. 2023; Anthony & Kam- nitsas 2023) and Jacobians train auxiliary classifiers using gradient boosting machines (Ke al.) 2017) to distinguish between calibration and test sets. For end-to-end, we do not use Jacobian weights consistent with prior work (Lambert et al.
2024

---

Published as a conference paper at ICLR 2026
HSEE
Skin Lesion
6x 10
3x 10
4 10
3 10
Interval
10
2 10
100
Coverage
0.9
0.9
0.8
SCP
Local
Outnut-COR
EZE-COR
COMPASS
COMPASS.
SCP
Local
Outnut-COR
EZE-COR
COMPASS-
COMPASS-
EZE-COR
COMPASS-
COMPASS-
COMPASS
SCP
Local
Output-cOR
E2E-COR
COMPASS-
COMPASS
SCP
Local
Output-CQR
EZE-COR
COMPASS-
COMPASS-
EZE-COR
COMPASS-L
COMPASS-)
COMPASS,
Unweighted
Cass
Feature
jacobian
Figure 3: COMPASS achieves the most efficient interval sizes under covariate shifts. We show results for two datasets and compare weighting methods for 100 adversarial splits that maintain the same covariate shift for a 0.1. For H&E, we increased the proportion of "hard" samples in the test set. For Skin Lesion, we decreased the proportion of "hard" samples in the test set. We find that COMPASS methods achieve valid coverage and the most efficient intervals in each weighting method. We show the 95% confidence intervals.
Our results (Figure 3) demonstrate that both the choice of weighting information and the choice of calibration layer are critical for achieving robust coverage under these covariate shifts. Simple strategies, such class wainssweigg,igg.. do not universally recover target coverage: unweighted methods (grey) fail on the H&E dataset, while class-weighted methods (red) fail on Skin Lesion. Further- more, not all feature-based methods are robust. COMPASS-L and E2E-CQR with feature weighting still fail to maintain coverage under covariate shift on the H&E dataset. In contrast, the COMPASS-I variants with both feature-weighting and Jacobian-weighiing weighting were the only methods to consistently maintain the target coverage across both covariate shifts. Among the methods that proved empiri- cally valid, the COMPASS-J variants were also the most efficient. This suggests that 1) the model's deep features or their Jacobians provide a richer, more adaptive signal for difficulty than simpler class labels or logits, and 2) this deep-layer signal essential, as methods relying on shallower lay- ers (like COMPASS-L. and F2F-CQR) were not robust even when using the same feature- weighttn.. Our results are tabulated in 5
3.3 EMPIRICAL ANALYSIS OF STATISTICAL EFFICIENCY
Unlike the original FCP framework, which provides formal inequality under mild assumptions due to its reliance on the simple and convex C, norm as a score, a similar theorem for COMPASS is intractable, because the COMPASS score RCOM P ASS is defined implicitly through search process that depends on a highly non-linear and non-convex segmentation- iinieenttiiontt-omee to-rit pipeline. Instead, we investigated the reason for efficiency gains of COMPASS compared to output-space methods by finding the relationship between COMPASS scores and SCP scores RSCP.
The statistical efficiency of COMPASS is a direct consequence of a compressive power-law rela- tionship between feature-space scores (RComrass) and output-space errors (Rscr). A log-log plot of these quantities reveals a linear relationship with a scaling exponent slope< (Figure top). This sub-linear scaling is the direct mechanism for the tail-end compression of the score dis- tribution (Figure A bottom). Thus, the largest output-space errors are systematically mapped to disproportionately smaller feature-space scores. A distribution with a compressed tail necessarily has a smaller quantile, which the fundamental mechanism that enables COMPASS to produce tighter, more statistically efficient prediction intervals.
8

---

Published as a conference paper at ICLR 2026
H&E (Slope=0.83)
Skin Lesion (Slope=0.62)
Nodule (Slope=0.65)
PolyP (Slope=0.56)
logi0 (Rcomiass)
-2-
0.0
2.5
5.0
-2.5
0.0
2.5
5.0
-2.5
0.0
2.5
5.0
0.0
1og10 (Rscp)
log20 (Rsce)
log2o (Rsce)
logio (Rscr)
0.4
Probabiltty
0.2
0.0
log10 (Score)
log (Score)
log10 (Score)
log1o (Score)
AsCP
RooMass
Figure 4: The statistical efficiency of COMPASS is driven by a compressive power-law rela- tionship between latent RooMPASS and output RSCP space scores. As RSCP increases, the required latent space perturbation magnitude increases, but at a progressively slower rate since the scaling exponent (slope) < (Figure top). This concave and sub-linear scaling is the direct cause of tail-end compression of the score distribution (bottom). Thus, the long-tail errors are systematically transformed much smaller feature-space scores.
4 DISCUSSION
We introduced COMPASS, a unified framework generate efficient, metric-based prediction inter- vals by leveraging the inductive biases of neural networks. Under the assumption of exchangeability, COMPASS creates practical feature-space calibration by perturbing model's intermediate repre- sentations along (ow-dimensional subspaces maximally sensitive to the target metric. Across four medical segmentation tasks for area estimation, COMPASS achieves valid coverage while produc- ing significantly tighter intervals than traditional output-space and end-to-end baselines for both standard and weighted CP. We discuss several further points below.
COMPASS has several attractive properties. First, it produces instance-adaptive intervals. For COMPASS-J, the perturbation direction for any new sample is found by projecting its Jacobian onto the principal component and reconstructing the direction. Although COMPASS-L applies uniform scalar shift, achieves adaptivity because this shift interacts with the spatially varying logit distribution unique each input, resulting in instance-specific adjustments the segmentation boundary. Second, COMPASS performs better than naive output-space calibration. tasks like segmentation, the final metric is a complex non-linear function of the model's output (the pixel mask). Simply adding a margin to a final object size prediction is a crude approximation that fails capture how the object size is actually derived from the underlying segmentation. In contrast, COMPASS exploits the model's 's internal structure and spatial understanding to directly manipulate and calibrate the metric. COMPASS has more information and degrees of freedom, which allows to achieve more efficient interval sizes. Third, COMPASS performs better than methods trained using pixel-level losses such as E2E-CQR. Such approaches optimize a proxy (pixel quantiles) for the downstream metric, that may may not translate into accurate intervals for the final metric. On the other hand, COMPASS is directly calibrated to the final metric, and is therefore more applicable to practical clinical use cases.
Finally, COMPASS naturally applicable to various segmentation architectures. We also present the interval lengths and coverage for SegResNet (Myronenko) 2018) in Appendix D COMPASS produces the most efficient interval lengths on the majority of datasets across a (0.05, 0.1, 0.15) for Set.

---

Published as a conference paper at ICLR 2026
Limitations. COMPASS's performance is fundamentally dependent the quality of the pre- trained model's representations. Our practical algorithm implementation presupposes that the model has learned inductive bias where the chosen feature space has reasonably monotonic and sen- sitive control over the downstream metric. If the features are poorly aligned with the metric, the resulting perturbation direction may be inefficient or non-monotonic, leading wide or invalid in- tervals. If so, we recommend using the full discretized sweep across the perturbation range to find the minimum and maximum values. Furthermore, as mentioned in prior work (Lambert al. a 2024), WCP with feature or jacobian weights, is effective for moderate distribution shifts where the cali- bration and test distributions have significant overlap in the feature space. However, for large shifts, the estimated weights may be inaccurate, which can compromise the validity of the final interval.
4.1 PRACTICAL CONSIDERATIONS
1.0
Explained Variance (PC1)
While COMPASS-L may be applicable to UNet-style ar- chitectures and object size, other applications may require different setups. Careful design of the different compo- nents of COMPASS, specifically COMPASS-J, will sig- nificantly enhance its upplicability and performance. We provide a list of practical considerations to keep in mind when using COMPASS.
0.8
H&LE
0.6
Skin Lesion
Nodules
PolyP
0.4
Non-monotonic
Monotonic
Optimal Feature Layer. The choice of the feature layer is critical. We find that the optimal layer can be se- lected empirically by measuring the explained variance of the first principal component (I. 1) of the Jacobians. High explained variance in the first principal component indicates that the local gradients of the target metric are globally aligned across the samples. Consequently, there high chance that linear traversal along this princi- pal direction results in consistent, monotonic shift in the output metric for the entire dataset, as the projection scalar maintains a consistent sign. As shown in Figure 5 layers closer to the logits capture the most variance re-
Encoder
Bottleneck
Decoder
Decoder 2
Figure Explained variance is a good indicator of monotonicity. For our 4 datasets, we plot the first principal component's explained variance against the feature layer used for COMPASS-J. Monotonicity and non-monotonicity is indicated by and x.
lated to object size, validating our use of COMPASS-J, We recommend finding layers with high ex- plained variance of the Jacobians and verify monotonicity with a perturbation sweep (Algorithm 2
Number of PCA Components. The number of eigenvectors I. controls the dimensionality of the perturbation subspace. We find that for the segmentation area metric, the first principal component already captures over 90% of the metric's variance (Figure 5). Adding more components does not significantly improve interval tightness (Figure 9). Therefore, we recommend L = 1 for this task, although this may be task-dependent.
Spatial Entanglement vs Global Semantic Alignment. We observe that the explained variance of the Jacobians is significantly higher when computed on spatially aggregated features compared to raw flattened features (Figure TO). This results from the translation-vart variant nature of the flattened space, where sensitivity tied specific pixel coordinates that vary with object position, leading to orthogonal gradients across samples. In contrast, summing the spatial dimensions isolates the channel-wiss semantic sensitivity, which remains consistent across samples. We recommend sum- ming the spatial dimensions of the Jacobians before applying dimensionality reduction techniques.
Computationally Efficiency. COMPASS maintains high computational efficiency, particularly when the metric response is monotonic. The jacobian computation is inexpensive gradients calculated with respect to the scalar metric rather than the high- digh-dimensional output image, leveraging autograd. We further resolve potential intractability in PCA by spatially summing the Jacobians, reducing the feature space to a low-dimensional matrix regardless of spatial resolution. While the jacobian and calibration is fast (Table & and Figure III. we recommend precomputing the jacobians for all data before calibration and testing.
LLM Disclosure. During the preparation of this manuscript, we used LLMs (Google Gemini) to polish the initial draft (grammar and style) and improve readability.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
MC would like to acknowledge support from a fellowship from the Gulf Coast Consortia on the NLM Training Program in Biomedical Informatics and Data Science TI5LM007093.
ETHICS STATEMENT
We develop uncertainty quantification methods for medical image segmentation metrics using pub- licly available and anonymized datasets. The methods are intended research tools to improve reliability and interpretability, but are not validated for clinical deployment. Risks include potential misinterpretatioon of uncertainty estimates. We therefore release code for research purposes only and encourage further evaluation with clinical experts before translation to practice.
REPRODUCIBILITY STATEMENT
We describe the algorithms in our paper extensively in Appendix We also describe the experimental details in Appendix E Our code is available at https::/gitth/ com/ hatthheryccheung//cmm compasss//////sss//o///a//ssssss All packages used in the repository are publicly available. All datasets used are publicly accessible.
REFERENCES
Moloud Abdar, Farhad Pourpanah, Sadiq Hussain, Dana Rezazadegan, Li Liu, Mohammad Ghavamzadeh, Paul Fieguth, Xiaochun Cao, Abbas Khosravi, Rajendra Acharya, et al. A review of uncertainty quantification in deep learning: Techniques, applications and challenges.
Information fusion, 76:243-297, 2021.
Anastasios Angelopoulos and Stephen Bates. A gentle introduction conformal prediction and distribution-fiet uncertainty quantification. arXiv preprint arXiv:2107.07511 2021.
Anastasios Angelopoulos, Stephen Bates, Adam Fisch, Lihua Lei, and Tal Schuster. Conformal risk control. arXiv preprint arXiv:2208. 02811, 2022.
Harr Anthony and Konstantinos Kamnitsas. On the of mahalanobis distance for out-of- distribution detection with neural networks for medical imaging. In International workshop on uncertainty for safe utilization of machine learning in medical imaging, pp. 136-146. Springer, 2023.
Rina Foygel Barber, Emmanuel Candes, Aaditya Ramdas, and Ryan J Tibshirani. Conformal prediction beyond exchangeability. The Annals Statistics, 51 845, 2023.
Edmon Begoli, Tanmoy Bhattacharya, and Dimitri Kusnezov. The need for uncertainty quantifi- cation in machine-assisted medical decision making. Nature Machine Intelligence, 1(1):20-23, 2019.
Omer Belhasin, Yaniv Romano, Daniel Freedman, Fhud Rivlin, and Michael Elad. Principal uncer- tainty quantification with spatial correlation for image restoration problems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(5):3321-333 2023.
Joren Brunekreef, Eric Marcus, Ray Sheombarsing, Jan-Jakob Sonke, and Jonas Teuwen. Kandinsky conformal prediction: efficient calibration of image segmentation algorithms. In Proceedings of the IEEF/CVF Conference Computer Vision and Puttern Recognition, pp. 4135-4143, 2024.
M Jorge Cardoso, Wenqi Li, Richard Brown, Nic Ma, Eric Kerfoot, Yiheng Wang, Benjamin Murrey, Andriy Myronenko, Can Zhao, Dong Yang, et al. Monai: An open-source framework for deep learning in healthcare. arXiv preprint arXiv:2211.0270// 2022.
Baiting Chen, Zhimei Ren, and Lu Cheng. Conformalized time series with semantic features. Ad- vances in Neural Information Processing Systems, 37:121449-12144 2024.
Matt Y Cheung, Tucker J Netherton, Laurence E Court, Ashok Veeraraghavan, and Guha Balakrish- nan. Metric-guided conformal bounds for probabilistic image reconstruction. ArXiv, pp. arXiv- 2404, 2025.
11

---

Published as a conference paper at ICLR 2026
Matteo Fontana, Gianluca Zeni, and Simone Vantini. Conformal prediction: unified review of theory and new challenges. Bernuuli, 29(1):1-23, 2023.
Haifan Gong, Guanqi Chen, Ranran Wang. Xiang Xie, Mingzhi Mao, Yizhou Yu, Fei Chen, and Guanbin Li. Multi-task learning for thyroid nodule segmentation with thyroid region prior. In 2021 IEEE 18th international symposium on biomedical imaging (ISBI), pp. 257-261. IEEE, 2021.
Weiming Hu, Chen Li, Md Mamunur Rahaman, Haoyuan Chen, Wanli Liu, Yudong Yao, Hongzan Sun, Marcin Grzegorzek, and Xiaoyan Li. Ebhi: A new enteroscope biopsy histopathological h&e image dataset for image classification evaluation. Physica Medica, 107:102534, 2023.
Fabian Isensee, Paul F Jaeger, Simon AA Kohl, Jens Petersen, and Klaus H Maier-Hein. nnu- net: self-configuring method for deep learning-baan baaning-basedd biomedical image segmentation. Nature methods, 18(2):203-211 2021.
Debesh Jha, Pia H Smedsrud, Michael Riegler, Pål Halvorsen, Thomas De Lange, Dag Johansen, and Håvard D Johansen. Kvasir-seg: A segmented polyp dataset. In International conference multimedia modeling, pp. 451-462. Springer, 2019.
Guolin Ke, Qi Meng. Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tic- Yan Liu. Lightgbm: A highly efficient gradient boosting decision tree. Advances in neural information processing systems, 30, 2017.
Eric Kerfoot, James Clough, Ilkay Oksuz, Jack Lee, Andrew P King, and Julia A Schnabel. Left- ventricle quantification using residual u-net. In International workshop on statistical allases and computational models of the heart, pp. 371-380. Springer, 2018.
Benjamin Lambert, Florence Forbes, Senan Doyle, and Michel Dojat. Robust conformal volume estimation in 3d medical images. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 123-643. Springer, 2024.
Jing Lei, Max G' G'Sell, Alessandro Rinaldo, Ryan J Tibshirani, and Larry Wasserman. Distribution- free predictive inference for regression. Journal of the American Statistical Association, 113 (523):1094-1111, 2018.
Luca Mossina and Corentin Friedrich. Conformal prediction for image segmentation using morpho- logical prediction sets. In International Conference on Medical Image Computing and Computer- Assisted Intervention, pp. 78 88. Springer, 2025.
Luca Mossina, Joseba Dalmau, and Léo Andéol. Conformal semantic image segmentation: Post- hoc quantification of predictive uncertainty. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5574-3584, 2024.
Andriy Myronenko. 3d mri brain tumor segmentation using autoencoder regularization. In Interna- tional MICCAI brainlesion workshop, pp. 3-320. Springer, 2018.
James PB O' Alan Jackson, Marie-Claude Asselin, David L Buckley, Geoff JM Parker, and Gordon C Jayson. Quantitative imaging biomarkers in the climical development of targeted therapeutics: current and future perspectives. The lancet oncology, 98):766-776, 2008.
Harris Papadopoulos, Kostas Proedrou, Volodya Vovk, and Alex Gammerman. Inductive confidence machines for regression. In European conference on machine learning, pp. 345-356, Springer, 2002.
Harris Papadopoulos, Alex Gammerman, and Volodya Vovk. Normalized nonconformity measures for regression conformal prediction. In Proceedings of the IASTED International Conference on Artificial Intelligence and Applications (AIA 2008), pp. 64-69, 2008
Harris Papadopoulos, Vladimir Vovk, and Alex Gammerman. Regression conformal prediction with nearest neighbours. Journal of Artificial Intelligence Research, 40:815-840, 2011.
Yaniv Romano, Evan Patterson, and Emmanuel Candes. Conformalized quantile regression. Ad-
vances in neural information processing systems, 32, 2019.
12

---

Published as a conference paper at ICLR 2026
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedi- cal image segmentation. In International Conference on Medical image computing and computer- assisted intervention, pp. 234-241 Springer. 2015.
Seyed Sadegh Mohseni Salehi, Deniz Erdogmus, and Ali Gholipour. Tversky loss function for image segmentation using 3d fully convolutional deep networks. In International workshop on machine learning in medical imaging, pp. 379-387. Springer, 2017.
Glenn Shafer and Vladimir Vovk. tutorial on conformal prediction. Journal of Machine Learning Research, 9(3), 2008.
John J Smith, A Gregory Sorensen, and James H Thrall. Biomarkers in imaging: realizing radiol- ogy's s future. Radiology, 227(3):633-66) 638, 2003.
Zihao Tang, Boyuan Wang. Chuan Wen, and Jiaye Teng. Predictive inference with fast feature conformal prediction. arXiv preprint arXiv:2412.0066: 2024.
Jiaye Teng, Chuan Wen, Dinghuai Zhang, Yoshua Bengio, Yang Gao, and Yang Yuan. Predictive inference with feature conformal prediction. arXiv preprint urXiv:2210.00177 2022.
Ryan J Tibshirani, Rina Foygel Barber, Emmanuel Candes, and Aaditya Ramdas. Conformal pre- diction under covariate shift. Advances in neural information processing systems, 32, 2019.
Philipp Tschandl, Cliff Rosendahl, and Harald Kittler. The ham I 1000 dataset, a large collection of multi-source dermatoscopie images of common pigmented skin lesions. Scientific data, 5(1):1-9, 2018.
Vladimir Vovk, Alexander Gammerman, and Glenn Shafer. Algorithmic learning in a random world. Springer, 2005.
McKell Woodland, Nihil Patel, Mais Al Taic, Joshua P. Yung, Tucker Netherton, Ankit B. Patel, and Kristy K. Brock. Dimensionality reduction for improving out-of-distribution detection in medical image segmentation. In International workshop on uncertainty for safe utilization of machine learning in medical imaging, pp. 147-156. Springer, 2023.
Kaidi Xu, Zhouxing Shi, Huan Zhang, Yihan Wang, Kai-Wei Chang, Minlie Huang, Bhavya Kailkhura, Xue Lin, and Cho-Jui Hsich. Automatic perturbation analysis for scalable certified robustness and beyond. Advances in Neural Information Processing Systems, 33:1 (911129-141 1141, 2020.
13

---

Published as a conference paper at ICLR 2026
A PROOF OF THEOREM
Lemma (Guaranteed Nestedness and Valid Scores). For any x E X, let my (b) be defined as in Theorem and let
Sa(z) = min mz(b), max mx bE[-
bei -
This family {Sa(z)}820 is nested. Furthermore, the nonconformity score
R(z,y) = if 20 0: E Sz(x))
is well-defined in [0, ool and satisfies
R(x, v) S. y E S
Proof. First, we prove nestedness.
Let S 31 S 82 By definition, perturbation interval
[-31, +B1] is subset of (-32, +.2].. Let
Sg, (x)
min bEl-Br+tA1
m. (b),
max m. (b) be1-B.
and
(x)
min [bEl-Pa.+0a]
my (b),
max bEl-Ba.-0a)
my (b)
The minimum of function over a set is always greater than or equal to the minimum over a superset. Thus,
min mz( 2 min mz +88-
bel-Ba.+9a1
Conversely, the maximum over a set is less than or equal to maximum over a superset. Thus,
max (1
my (b) S
bEl- P2) (b)
Therefore, by definition of an interval, S3, (1)) C (1), and the family is nested.
Second, we prove the equivalence E (x) = R(x. y) S 8. Since we have just proven that the sets (x) are nested, they expand monotonically in 13. If y Syr (ar) for some 3', then y Ss(x:) for all [ 2 8'. The set {B 2 0 S5(z)) is therefore a closed interval of the form [Ro, oo) (or
[0, ao) y E So(x)). The infimum R(x, y) is thus well-defined (as Ro), and y 83(x) if and only
if 3 Z R(I,y).
Lemma (Exchangeability yields uniform ranks). Let (Xi, YJ) be exchangeable, and let R4 := R(X,, Y.) be derived from Du.- Then the rank of Ra+1 among {RI...., Ba, Ra+13 is uniformly distributed on {1..... 1}, conditional on D
Proof. Since the nonconformity score R(-.-) is measurable, deterministic function of (X,, Yi)
and Du.. the exchangeability of the pairs (X,, Y2) implies the exchangeability of the scalar scores (. Therefore, by the standard properties of exchangeable random variables, the rank of R among all n + 1 scores is uniformly distributed on f1..... 1}.
Theorem (Split-conformal coverage). Under the assumptions of Theorem 1, for any a E (0, 1), the prediction set (X,+1))) satisfies
p (YYt1 E Sz(Xn+1) Diz) 2 1 1
Proof. By Lemma I our construction of Sz(z) guarantees nestedness and also guarantees that Yu41 Sp(Xn+1) if and only if Rn+1 I Applying this to our calibrated quantile 3, we have
Yat1 E Sg(Xn+1)
Rat1 S
1 is uniform on (1...., .n....nt11 n+11 By definition, 3 the (n-a)((+1))) th smallest value among the calibration scores RR,.... R₂ }. By the standard p-value argument for
By Lemma 2 the rank of R2+1
14

---

Published as a conference paper at ICLR 2026
split conformal prediction, the probability that the test score Rn+1 is less than equal B is bounded.
P(R SB Dir) 2 a.
Combining these, we have
P P(Y++1 Sg(Xn+1) Dus) =P(R++1 SB Dw)) 2 -a
which proves the theorem.
Connection to Standard CP. Theorem generalizes the standard split conformal method. In the classical setting for regression, S3(x) [ū(x) - 3. ((z) + 3| corresponds to symmetric absolute- emor balls around predictor ū(x). This family is trivially nested. Here, the sets S3(ø) are gen- erated by perturbations of latent features through A and evaluation under the downstream metric o By rigorously defining (z)r as the [min, max] envelope, we guarantee the nestedness prop- erty required for the proof. The same rank-uniformity argument from exchangeability then ensures coverage. Thus, standard split conformal prediction is recovered as special case of this framework.
B ASYMMETRIC PERTURBATIONS
We extend the symmetric method provide more flexible and efficient prediction intervals to the asymmetric case. This approach is particularly valuable when the relationship between latent pertur- bations and the downstream metric is non-linear or asymmetric. For instance, positive perturbation of a certain magnitude might cause a large increase in metric, while negative perturbation of the same magnitude results only small decrease. It also allows for independent control over the rate of under- and over-estimation errors, specified by Og and Ohi. To account for this, calculate two distinct non-conformity scores (Ri, REa) and two corresponding perturbation magnitudes (3)o, P1)). By calibrating the upper and lower bounds independently, Asymmetric COMPASS constructs an interval that adapts to these skewed error distributions, often resulting in tighter bounds than its symmetric counterpart such cases.
Theorem (Asymmetric
Coverage). Let the setup be the same as Theorem with metric function m. (b) == (h o (2(x)) + bA). We define the one-sided envelope functions:
(3) = min my (b)
(.0)
(3) = max my (b)
bc(0+++]
These functions (g) and (3) are guaranteed to be non-increasing and non-decreasing in B, respectively. For each calibration pair (X,, Yi). define two non-conformity scores:
Ri inf B 2 0: Lx, (3) Y:}, := in 2 0: (3) 2 Y.).
Let Oloo Ohh (0, 1) be user-specified miscoverage rates. Let Bro be the [(1 an) + 1)1-th smallest value among (Ri R,,oo,, and let Bai be the [(1 - an + 1)1 -th smallest value among RRaui..... Rn.ii)
Then, for a fresh test pair (Xn+1, Yn+1), the prediction set
S(X++1) = [Tx... (Bro): Uxner (Bni))
satisfies the marginal coverage guarantee
P(Ynt1 E S(X Dvt) -(oo ons)-
15

---

Published as a conference paper at ICLR 2026
B.1 PROOF OF THEOREM!
Lemma (Monotonic Bounds and Valid One-Sided Scores), For any x E X, let (3) and (x( be defined in Theorem 3 The function (3) is non-increasing in B, and the function (2(3) is non-decreasing in B. Furthermore, the nonconformity scores Rio y) and (E, y) are well- defined in [0, oo] and satisfy
(3) y (3) 2y
(,, S 3 Rai(,, S 3.
Proof. Let S $1 S 52. By definition, the interval [-31. 0] is subset of [- 0]. The minimum of function over a set is always greater than or equal to the minimum over superset. Thus:
L (31) = min my bel-8:.0
2 min mz = (82)) bel-8s.0]
This proves (3) is non-increasing. Similarly, [0, +31] is subset of [0, +32]. The maximum of function over a is always less than or equal to the maximum over a superset. Thus:
Ug (B1) be[o,+8.]
(z S max (b (22)))
bE[0.-8u]
This proves (3) non-decreasing.
This one-sided monotonicity ensures that if a bound satisfied for some 3 (e.g.. Lz(()) S y). remains satisfied for all 2 B'. Hence, the infima in the definitions of the scores exist, and the cquivalences hold.
O
Lemma (Exchangeability yields uniform ranks). Let (X,r Y;) exchangeable, Let Ri, == (X,, Y,) and Rr,i := Rma (X, Y,) be derived from Du. Then the ranks of R,+1.lo among [Rilos.... Ra+11oo J and Ra+imi among {R., Rat1,nd are each uniformly distributed on (1, (1.-.., 1}, conditional Dtr.
Proof. Since the nonconformity scores are measurable, deterministic functions of (X,, Yi) and Drr the exchangeability of the data pairs implies the exchangeability of each sequence of scores, (Rat)) nn+1 and (R, form.
a+1 Therefore, the rank of the test score within each sequence is uni-
Proof of Theorem The total miscoverage event is the union of two one-sided miscoverage events:
P(Y.+1 & S(Xn+1)) = PPYn+1 < Ixaus (310) ) or Yu+1 > Ux (6na))
By the union bound.
P(Y $ 5(Xn=1)) < P(Y++ < Lx,,, P(Y... > I
By Lemma 3 these one-sided events are equivalent to events concerning the scores:
P(Y & S(Xn+1)) <P(Rnti,.o > Bmn) + P > Pna).
By Lemma 4 and the construction of 3im and Bm as quantiles, the probabilities of these rank-based events are bounded by ano and abi, respectively. Thus,
PPYY+1 & 8(Xn+1)) S ano + Ohi-
The claim for the coverage probability, P(Y,-1 E S(Xn+1)) 2 1 - (aso + ami), follows.
This theorem provides a rigorous guarantee for asymmetric intervals. The proof relies on the same core principles of exchangeability, but instead of assuming nestedness, we guarantee the required one-sided monotonicity by defining the bounds (3) and (5) as the [min, max] over the respec- tive perturbation ranges.
16

---

Published as a conference paper at ICLR 2026
C ALGORITHMS
In the COMPASS framework, we decompose the model into an encoder f that maps an input image representation 2, and decoder 9 that maps å the final segmentation. A downstream metric function h then maps the segmentation to a scalar metric ÿ. For simplicity, we show the algorithm using the endpoint (and not the conservative envelope that requires perturbation sweep during the calibration procedure) - i.e. we define the perturbation and metric evaluation function P(2, A, 5) as the interval [h(g(2 h(g) + BA))] This endpoint construction is computationally efficient and theoretically valid given the monotonic metric response observed in our experiments (Figures 2 and T2). The conservative envelope variant follows the same approach, but the symmetric binary search involves perturbation sweep for each sample during the calibration step to find the minimum and maximum.
For COMPASS-L, the representation is the logits; the encoder f is the entire network, and the decoder g is the identity function. The perturbation direction simply A
For COMPASS-J, the representation & is an internal feature map. To find the sensitive direction A, we first compute the Jacobian the metric with respect to the features, = Vgh(g(E)). We then project this Jacobian onto the subspace spanned by the top L principal components (VL) of the training set Jacobians and reconstruct The final direction A is the normalized reconstruction:
d VL J
l 12"
In general, the COMPASS framework follows several key steps:
1. Training: Assume an already trained base segmentation model, decomposed into en- coder f and decoder g. If using COMPASS-J, compute the Jacobians of the metric with respect to the latent features & for the training set and perform PCA to identify the dominant sensitivity subspace Vz.
2. Monotonicity Verification:
Perform perturbation sweep on a subset of the data verify that the metric response m= (b) is monotonic with respect to the perturbation magnitude along the computed direction A. This empirical check justifies the use of the endpoints over the computationally expensive envelope construction.
3. Calibration: For each sample in the calibration set, compute the non-conformity score R using binary search. R the minimal perturbation magnitude 3 such that the computed interval covers the ground truth. Finally, compute the calibrated quantile / from these scores using the standard finite-sample correction.
4. Testing: For a new test sample In+1. compute its specific perturbation direction A (using the learned subspace VL for COMPASS-J 1 for COMPASS-L). Construct the final prediction interval using the calibrated B and the interval endpoints: Sz(En+1)
[ma+1 (-B), mm+ (+3)).
C.1 TRAINING
Algorithm COMPASS Training
Require:
Training data Dtr. I. (number of components for COMPASS-I..
Ensure: Trained model (f. g). and subspace VL (for COMPASS-J).
1: procedure TRAINING(D (tr. L)
2: Train the segmentation model (f.9) arg arg faeg (Dtc). if using COMPASS-J then Compute Jacobians: J {> ESpasial Vg.Moo(((zs))))>ep))
3:
4:
5:
Compute PCA: Vz <- PCA(J.T). 6: return f.g. VL.
7:
else
COMPASS-L requires no extra training
8:
return f.g.
9:
end
10:
end procedure
17

---

if IS-SORTED- BOOTTDD-ACENDINGGGAAA then count + count + end end for Published as a conference paper at ICLR 2026
C.2 MONOTONICITY VERIFICATION
Algorithm Metric Monotonicity Verification
Require: Model (f 9, V2). Dataset D {( bb] Pmar, Nawpp-
Ensure: Percentage of samples with monotonic metric response.
1: procedure VERIFY-MONOOONICTTO. model, h, Smax, Natep))
2: Initialize count 0.
3: for i I...., do
4: 2g + f(x.)
5: if uuing COMPASS-J then
6:
d: < V.V, Vzh(g(2)))
Project Jacobian > Normalize
7:
A d:
8:
else
9:
A +
10:
end
11:
B LinSpace(- (muxx Pmaxs Nsseps).
> Discretized grid
12:
M < 0.
13:
for b B do
14:
M + M U Uhh(0(2 + +..))) end for
Evaluate metric
16:
Check monotonicity
18:
20:
return count/n
21: end procedure
Algorithm is diagmostic step performed after training. It verifies that perturbing the features along the computed direction A results in a monotonic change in the metric (area). High monotonicity validates the of the efficient endpoint-based interval construction in the subsequent calibration and inference steps.
C.3 SYMMETRIC COMPASS
Algorithm 3 Symmetric COMPASS (Calibration & Inference)
Require: Calibration Drat, Test Tat1, a, Model (f,g. Vz).
Ensure: Prediction interval S (In+1)-
procedure CALIBRATION(D model, h, a)
2: Initialize scores R < 0.
3: for 1,...,71 do
4: Define 2; and compute A, (as in Alg. 2 E,, A., v:).
5:
R& f- SYAMETTIC-BINARY-SAAAA R < R (R.).
6:
7:
end for
8: 3 f- Quantile(R, [[--)(a+]]).
9: return
10: end procedure
11: procedure INFERENCH(# model, h, ())
12:
Define E,+1 and compute An+1 (as in Alg. 2
13:
Sp(#n+1) <- P(2n+1, 8).
14: return (g(#n+1)
15: end procedure
18

---

Published as a conference paper at ICLR 2026
deseriis the symmetric formulation. During calibration, we compute non-conformity score R3 for each sample using a binary search (Algorithm 5). R, represents the minimal perturba- tion magnitude 3 such that the interval P(2,, A., 3) covers the true label yt. We then compute the (1 a) quantile of these scores, 3. During inference, we apply this calibrated | to the test sample to produce the final prediction interval.
C.4 ASYMMETRIC COMPASS
Algorithm 4 Asymmetric COMPASS (Calibration & Inference)
Require: Calibration Deal, Test #n+1, Ogo Ohi, Model (f.g. Vz).
Ensure: Prediction interval (zn+1))
procedure CALIBRATION(Daa) model, h, Cno Ohi)
2: Initialize Ri, +- 0, Rui < 0.
3: for 1, ,,..,,,, 7 do
4: Define & and compute A (as in Alg. 2
5: (R RR,ni) <- ASYMMETRIC BINARY- (EANRR--EARCHHE,,.. A1, 31)
6:
R1o <- Rio U RR
7:
Rui +- Rui {-
8:
end for
Bio + Quantile(Rio, [[-aaXK+tIl]
9:
Bui + (uantile(Rhi, [(1- [(1-a)(a+1]]]
n
11: return 8toy Pni
12: end procedure
13: procedure INFERENCE(2.+1 model, h, B,o. 3mm))
14:
Define ên+1 and compute An+1 (as in Alg. 2
S(zn+1) < [h(g(En+1 Blo h(g((((( (2n+1 Am41))
16:
return S(In+1)
end procedure
Algorithm extends the framework to the asymmetric case. We independently calibrate lower bound (3/0) and an upper bound (Bnr) using separate non-conformity scores. This allows the interval to expand differently in the positive and negative directions, which is efficient for metrics with asymmetric sensitivity.
C.5 SCORE-FINDING PROCEDURES
Algorithm 5 Symmetric Binary Search
function
A,y) y)
2:
blow +- 0, bnigh &- Brange-
3:
for I,.... kimax
4:
bund +- (bjo + bnigh) /2
5: Smid 1-6--Pp -(2 bmia)
D Compute endpoint interval
6:
y E Smia then bhigh f- bmid
7:
else blow <- baid
8: end if
end for 10: return bnigh.
9:
11:
end function
19

---

Published as a conference paper at ICLR 2026
Algorithm 6 Asymmetric Binary Search
function
A,
2:
(b) & h(g(2 - ba)), ma(b) t h(g(2 ba))
3: Find Upper Bound Score (Ba)):
4: blow x- 0, bnigh f Brange
5:
for 11..., kimax
6:
bmid + (brow + bhighh/
7:
if y S (biimi then bhigh + bmid
8:
else blow f- bmid
9:
end
10:
end for
11:
Rhi < bnigh-
Find Lower Bound Score (Ro)):
13:
blow f- 0, bnigh f- Brange-
for 1..... kimax do
15:
band t- (blow + /2.
y 2 ma (ma(mmia))) then bnigh +- Umid
17:
else blow <- Drudd
18:
end
19:
end for
20: Rio f bnigh-
21:
return (Ri., Rai).
22:
end function
These are the root-finding functions necessary to determine the non-conformity scores. We use binary search for computational efficiency. This requires O(log N) forward passes, as opposed to (NN) for linear search, making highly scalable.
20

---

Published as a conference paper at ICLR 2026
D ADDITIONAL TABLES
Table 2: Empirical coverage for U-Net.
All methods reach (close to) target coverage.
Coverage (Mean+STD)
Dataset
COMPASS-J
COMPASS L
E2E-CQR
Local
Output-COp CCQutt-QR
SCP
H&E 0.05
0.95240.023
0.953+0.02
0.955=0.023
0.95#0.021
0.951
0,956+0.018
H&E
0.10
0,907=0.029
0,907=0.03
0,903=0.028
0.903=0.03
0.903=0.031
0.91+0.03
H&E
0.15
0.856#0.03
0.856#0.031
0.856#0.035
0.848+0.036
0.85#0.032
0.849+0.032
Skin Lesion
0.05
0.95140.009
0,953#0.008
0.951
0.95+0.01
0.951
0.951#0.01
Skin Lesion
0.10
0.920.014
0.904=0.013
0.910.012
0.899
0.897=0.013
0.899+0.013
Skin Lesion
0.15
0.85340.015
0.86#0.016
0.853#0.016
0.852+0.015
0.852+0.015
0.853+0.015
Nodule
0.94980.018
0.953#0.018
0.958#0.014
0,948+0.017
0.954+0.016
0,953+0.019
Nodule
0.901=0.025
0.904=0.026
0.904=0.022
0,906#0.025
0.908=0.024
0,906#0.024
Nodule
0.15
0.85240.032
0.858=0.031
0.849#0.03
0.85#0.028
0.845+0.026
0.857#0.027
PolyP
0.05
0.955=0.02
0.9620.018
0.957=0.019
0,951=0.022
0.955#0.018
0.9640.022
PolyP
0.10
0.89940.034
0.908=0.032
0.902=0.027
0.897+0.032
0.901=0.029
0.907#0.03
PolyP
0.15
0.84920.035
0.86540.03
0.853#0.034
0.854+0.037
0.856#0.033
0.856#0.032
Table 3: Comparison between COMPASS and FCP interval lengths. Because FCP fails to reliably converge when finding non-conformity score, we instead provide a comparison, where we empirically find the minimal radius of an L, ball that achieves the target coverage across the test set. COMPASS methods consistently achieve the most efficient interval lengths.
Interval Size (Pixels?, Mean
Dataset
COMPASS-J
COMPASS-L
FCP
H&E 0.05
4637+630
4408+432
9702+1535
H&E 0.10
3160#336
31394375
6463+1193
H&E 0.15
2320#252
1354+146
4979+647
Skin Lesion
0.05
1657+80
1689483
14838*10
Skin Lesion
0.10
1179+53
1208+58
12445+51
Skin Lesion
0.15
934+30
956+33
8267+79
Nodule 0.05
3257=210
3394+280
1624742
Nodule
0.10
2444+174
2510+180
16143+3
Nodule
0.15
2016+143
2082+142
1605644
PolyP
0.05
54891575
6376+769
15937+9
PolyP
0.10
4056+293
4397+469
15644+10
PolyP
0.15
3394+290
36864361
15353+14
Table 4: Comparison of layer choice,
For 0.1, we find that while for some datasets the efficiency gains comparable, penultimate features generally achieve shorter interval lengths.
Interval Size (Pixels" (Mean
Dataset
COMPASS-J (Bottleneck)
COMPASS-J (Deep)
COMPASS-J (Shallow)
COMPASS-L
H&E
93942799
3160=336
3140+386
3139#375
Skin Lesion
4084+154
1179:53
1210:51
1208258
Nodule
8070+494
2444+174
2800+187
2510+180
PolyP
Non-monotomic
4056+293
4222+390
4397+469
21

---

Published as a conference paper at ICLR 2026
Table Comparison between baseline and COMPASS methods for Weighted CP. For a 0.1,
weighted COMPASS methods consistently outperform baseline methods in terms of interval length and restoring coverage under covariate shift.
Dataset
Weighting
Method
Interval Size (Mean=STDD, Pixels")
Coverage (Mean=STD)
COMPASS-J
2055+2200
0.911#0.024
COMPASS-L
2065+263
0.903+0.024
E2E-CQR
2444+278
0,900#0.027
Class
Local
31134201
0.907+0.024
Output-CQR
28904265
0.901
SCP
2371+290
0.900
COMPASS-J
1916+177
0.909
H&E
Feature
COMPASS-L
1944+244
0.896=0.024
E2E-CQR
2197+288
0.884+0.029
Jacobian
COMPASS-J
1822+200
0.896+0.08
COMPASS-J
1810#127
0.88620.008
COMPASS-I.
1759+159
0.884+0.000
E2E-CQR
199
0.87140.024
Unweighted
Local
2833+158
0,88#0.021
Output-CQR
2563+134
0.879#0.024
SCP
1003+169
0.878+0.024
COMPASS-J
1318+132
0.89640.019
COMPASS-I.
1373+136
0.895+0.019
E2E-CQR
13354277
0.887+0.021
Class
Local
2426+354
0.893+0.02
Output-CQR
3380+43
0,888+0.01
SCP
1905+475
0.888+0.022
COMPASS-J COMPASS-L L E2E-CQR
1573+158
0.919
Skin Lesion
Feature
1669
0.919+0.011
2030+266
0.919#0.012
Jacohian
COMPASS-J
1485 126
0.914+0.013
COMPASS-J
2183+132
0.94
COMPASS-L
1500
0,93940.00 007
E2E-CQR
3400+2226
0.945+0.000 006
Unweighted
Local
5446+380
0.947#0.000
Output-COO CQR SCP
4201#145
0.945+0.008
47074323
0.948+0.006
Table 6: COMPASS produces the most efficient interval lengths on the majority of datasets across target coverages for SetResNet.
Interval Size (Pixels?, Mean:STD)
Dataset
COMPASS-J
COMPASS-L
E2E-CQR
Local
Output- CQR
SCP
0.05
6129+1147
4680+595
5115+541
7649=821
6860#571
6021#858
H&E
0.10
3283+262
32174247
3575+344
4800=700
4509+389
3911#371
0.15
24834226
2400+244
2563+182
3491+333
33584285
2783+317
0.05
1599:97
1829#191
2281+146
4034=256
10942+53
2987=227
Skin Lesion
0.10
1108+45
1146245
1236466
2311=167
3056+45
1685+135
0.15
873+27
927427
857+52
1710264
2293+32
1083+49
0.05
3675+480
35774413
44614337
3513#212
6197464
4068+388
Nodule
0.10
2671+251
2645+216
3390#197
2762+129
3942+102
2826=200
0.15
2125+203
21274203
2646+128
2355#131
2937495
2188+167
0.05
5890+417
6068+435
8495+1012
1992+1324
82964822
7774+930
PolyP
0.10
4507+529
47374486
49411949
4837+611
50614599
5169+559
0.15
3188+500
3569+572
2668+255
3594=326
3556+401
3467+389
22

---

Published as a conference paper at ICLR 2026
Table Empirical coverage for SegResNet. All methods reach (close to) target coverage.
Coverage (MeanASTD)
Dataset
COMPASS-J
COMPASS-L
E2E-CQR
Local
Output-CQR
SCP
0.05
0,956#0.019
0,949#0.021
0,954+0.021
0,949#0.022
0.949#0.023
0.953+0.02
H&E
0.10
0.905+0.029
0,940.027
0.91=0.027
0,901+0.028
0.902+0.028
0,906+0.027
0.15
0.85840.033
0.852+0.034
0.85640.031
0.854+0.033
0.854+0.033
0.856#0.035
0.05
0,952+0.011
0.95=0.01
0,952+0.01
0.95+0.011
0.951=1001
0.952+0.01
Skin Lesion
0.10
0,903+0.014
0.901=00013
0.9:0.014
0,902+000166
#0.013
0.902=0.015
0.15
0.854x0.015
0.853#0.016
0..51
0.852=0.015
0.854+0.014
0.854+0.015
0.05
0,958+0.017
0,953#0.017
0.954=0.019
0.951=0.019
0.953+0.017
0,952+0.018
Nodule
0.10
199240.026
0,901=0,027
0,909=0.025
0,9+0.024
0.901
0.904+0.026
0.15
0.85240.026
0.85=0.026
0.856#0.029
0.84840.032
0,849#0.032
0.851
0.05
0.948=0.022 0.954=0.022
0.954=0.022
0,949#0.03
0.95+0.021
0.952+0.02
PolyP
0.10
0.905#0.032
O. 0.89=00.033
0.909=0.025
0,904+0.032
0,9:0.032
0.908#0.03
0.15
0,858+0.035
0.85#0.035
0.856#0.031
0.856#0.031
0.855+0.037
0.859+0.033
E EXPERIMENTAL DETAILS
Architectures. We use the standard U-Net. SegResNet, and SwinUNETR architecture available in the MONAI framework (Cardoso et al. 2022). For the U-Net, we used an encoder with channel sizes of (32, 64, 128, 256) and corresponding decoder, using two residual units per block, batch normalization, and a dropout rate of 0.1. For SegResNet, we used a residual encoder-decoder net- work that started with 32 initial filters and included varying numbers of blocks in its down-sampling path (1, 2, 2, 4), batch normalization, and a dropout probability of 0.1. For SwinUNETR, we used a transformer-based model configured for an image size of (128, 128) pixels, with feature size of 48 and multi-head attention mechanisms across four depth levels.
Preprocessing. To standardize all experiments, we resize each image and segmentation mask to a standard resolution of 128x 128 pixels. For data augmentation during training, we apply set of transformations consisting of random crop to the specified size, followed by random horizontal and vertical flips, each with equal probability.
Training details. For each architecture, we trained both a standard model and an End Conformalized Quantile Regression (E2E-CQR) variant (Lambert al. 2024), with all models be- ing trained using an AdamW optimizer with learning rate of Te-4 and a batch size of 32. The stan- dard models were configured for binary segmentation tasks, taking (for grayscale) or 3-channel (for RGB image) as input and producing single-channel output mask. Training was optimized using a Dice score loss function to maximize the overlap between the predicted and ground-truth segmentations. Following the original E2E-CQR model (Lambert et al. (024), we modify the num- ber of output channels to to produce three distinct segmentation masks, corresponding to the lower quantile, the median prediction, and the upper quantile, which together form the prediction interval. These models were trained using the Tversky loss (Salehi et al., 2017). Thus, allows the model to directly learn the uncertainty bounds end-to-end.
Baseline Calibration Details. We follow the standard frameworks for SCP (Papadopoulos et al., 2002 Lei et al. 2018). CQR (Romano al., 2019), and Local CP (Papadopoulos al., 2008 2011 Lei et al.) 2018) For CQR, we use two separate Gradient Boosting Regressors to learn the lower and upper quantiles of the prediction and adjust the resulting We use learning rate of 0. 50 estimators, a maximum depth of 3, minimum leaf samples of 1, and minimum samples to split of 9. For Local CP, we use two separate Random Forest Regressors to learn the mean and mean absolute difference. We use 1000 estimators and minimum leaf sample of 100. For F2E-CQR, we follow the same calibration procedure as CQR. However, in this case, the quantiles are learned by the model.
COMPASS Calibration and Testing Details. For all experiments, we run both the symmetric and asymmetric versions of COMPASS with component on the logits and each layer in the seg- mentation head. We report the layer with the minimum mean interval length and its corresponding
23

---

Published as a conference paper at ICLR 2026
HSE
Skin Lesion
40
(%)
30
Percentage
50
20
30
10
0
Adenocarcinoma
High-grade IN
Low grade IN
Normal
Polyp
Serrated adenoma
bid
df
mel
Class
Class
Calibration
Test
Figure 6: Adversarial shift distribution for H&E and Skin Lesion datasets. For H&E, we place 60% and 40% the Adenocarcinoma (easy) samples into the calibration and test sets. For Skin Lesion, place 70% and 30% of the melanocytic nevi (hard) samples into the calibration and test sets.
coverage. We perform 100 random calibration-tess splits. For weighted CP, we use LightGBM with the default settings (Ke et al. 2017). We also perform 100 calibration-test splits. However, in this case, the samples were shuffled to maintain the target proportion of classes. show the covariate shift in F
F PRACTICAL RECOMMENDATION ADDITIONAL RESULTS
Moan Intervel Laneet (Pazah")
Tayro
5x:00
4x307-
1x007-
2x
1.0
1.
1.0
0.-
08-
64-
0.8
06
05-
02
02-
00
68
2.tenit: lang
Lov Lencoa
Le Ldeios
Jr d.
dd Rdeirt v23
duboa da
del Ldvear
J.devdl 2.34
2. d.exes
C3 dd 103
Adboy Levaa
7deidl 2.eevd
2. d.áexes
Sats c3 403
Loozt 1avvar
Lieede
Gooudd Tuth Ba (Powk))
Cround Tuu Bis (Pioes))
Croand Tulk Dir (Powss)
Cround Tuth Dir (Powks)
CONTAGEL
CONTASS-1
Figure 7: COMPASS results in stable calibration. We the results from a=0.1, binned the ground truth values into non-overlapping bins, and computed the mean interval length and mean coverage (and their 95% confidence intervals). We find that COMPASS methods tend perform better for shorter interval lengths (except for H&E), achieving approximately the same coverage. The count of the bins tends to be a better indicator of calibration stability (See Figure 8). Please note that our guarantees are only marginal; therefore, coverage may be lower for values that are less frequently represented during calibration.
24

---

Published as a conference paper at ICLR 2026
H&LE
Skin Lesion
Thyroid
PolyP
40000
50000
6000
8000
Count
4000
20000
15000
0000
10000
4000
2000
10000
3000
5000
10000
15000
50000
10000
13000
5000
10000
50000
10000
Ground Truth Value (Pieels")
Ground Truth Value (Pikels?)
Ground Truth Value (Pixels")
Ground Truth Value (Pieess
Figure 8: Ground Truth Distribution. We show histogram plots of the distribution of ground truth segmentation area values in Pixels² with a KDE overlaid.
Efficiency Gain (%)
-2
-4
H&E
Skin Lesion
Nodule
-6
PolyP
2
3
Number of PCA Components
Figure 9: One principal component is generally sufficient for COMPASS-J. For 4 datasets, run COMPASS-J with components for 100 iterations. We find that there no additional benefits of using more than 1 principal component, as the first principal component already explained most of the variance (Figure
Explained Variance (PC1)
H&E
0.15
* Skin Lesion
Nodules
PolyP
0.10
Non-monotonic
Monotonic
0.05
0.00
Encoder
Bottleneck
Decoder 1
Decoder 2
Figure 10: Explained variance of the Jacobians is significantly lower when computed on raw flattened features. For our datasets, we plot the first principal component's explained variance against the feature layer used for COMPASS-J. We find all layers result in non-monotonicity indi- cating the iissslatio-vvriann nature flattened space.
25

---

Published as a conference paper at ICLR 2026
Table 8: Jacobian computation for COMPASS-J is fast. We present the average computational time to compute the Jacobians of feature size 64x64x64 across four datasets below using a single NVIDIA A100 GPU. We compute the Jacobians with respect to the metric, which is extremely fast with autograd. This is because the downstream metric is one-dimensions the training and calibration step, we precompute and save the Jacobians summed on the spatial dimension.
Furthermore, to expedite
H&E
Skin Lesion
Nodule
PolyP
Compute time for full dataset
73.6#0.6
305.8+1.1
93.3+0.9
73.6+0.6
(Mean=STT) in seconds)
Hiat
Skón Lasion
Hyyaad
Polye
33-
Average Time
10
5-
COMPRSS
AcoMMiss
COMRASS (Deepp)
coMPASS (Derpi)
COMPASS (liraine))
coMmAss (Snalioo)
COMPASS
A Acompsts4
cOMPASS (Derap))
AcComnASs (Dempp)
COMPASS (Sialiwn)
CONDSSSS (Siaalov))
COMPISSLE
A.COMMEE
cOOPAASS (Deepp)
Accentss (Darpi)
Mothoe ACOMPISS) (ohtalow)
COMPASS (Oinllow))
coMeess
COMMASS (Derp!)
A.OMMAS (Darpi)
coNmASS (livatine)
Acammsss (inalion)
Mothog
Methor
Figure 11: COMPASS methods do not require substantial computing times. The calibration step requires multiple forward passes. However, we find that combining forward passes and binary search enables fast calibration. We show the average computational time required for calibration (for 100 repeats) across 4 datasets using a single NVIDIA A100 GPU. We compute times for the symmetric (without A-) and asymmetric (A-), deep (-Deep) and (-Shallow) layers, and COMPASS-L versus COMPASS-J. The full calibration runtimes are the order of seconds for the full calibration dataset. The number of samples used for calibration was 223 (H&E), 1000 (Skin Lesion), 349 (Thyroid), and 200 (PolyP).
H&E
Skin Lesion
Nodule
PolyP
20
15
10
-15
-20
-1
-1
-1
B
Figure 12: Monotonicity verification. We verify the nestedness of linear latent perturbations
(Definition [). For each dataset, we perform a sweep across different Bs and compute the change in
volume 6 As-Ap 100 where Ag is the area when the original latent is perturbed by 3.
Ao
26

---

Published as a conference paper at ICLR 2026
G SEGMENTATION OVERLAYS FOR COMPASS-J AND COMPASS-L
G.1 SEGMENTATION OVERLAYS FOR DIFFERENT PERCENTAGES CHANGE IN SEGMENTATION AREA (бA) FOR COMPASS-J
-20% -10%%
20%
0.000
1.19
B9321
Figure 13: Segmentation area increases with 3 for H&E. Each row is different sample. 5A = 0% the original prediction.
27

---

Published as a conference paper at ICLR 2026
0A -20%
0A -10%
GA 0%
0A 10%
0A 20%
Ground Truth
m 11.90
B -5.01
B m 0000
[1 D171
- 1:30
8---3,27
B1 1--0
[1 2:71
p 2.05
-1.94
0.00
8 -8.17
B +1.01
B m 22 OX
( D.50
6- 1:21
B
Figure 14: Segmentation area increases with 3 for Skin Lesion. Each row different sample.
6.A = 0% is the original prediction.
28

---

Published as a conference paper at ICLR 2026
AV -20%
DV = -10%
2V 0%
AV 10%
BV 20%
Ground Truth
-2.18
5 -9.9
B m -3
P 3.64
1-122
0.74
5- 0.33
0.00
p 0.20
B- 0.60
0.35
8.18
8 0.00
p 0.37
8- 0.35
-..00
R-
R- 0.65
P-
p 1.1
0.00
p
p
Figure 15: Segmentation area increases with for Thyroid Nodule, Each row a different sample. 8 0% is the original prediction.
29

---

Published as a conference paper at ICLR 2026
0A -20%
04 -10%
dA 0%
04 10%
0A 20%
Ground Truth
m 4.59
6 b2.bb
B m -000
0 1.31
5-24
4.49
5 +
11
F 1.11
2.05
3.77
-773
8 11,01
B m 0.00
6- 2.77
B 2,89
Figure 16: Segmentation area increases with for PolyP. Each is different sample. 8AA 0% is the original prediction.
30

---

Published as a conference paper at ICLR 2026
G.2 SEGMENTATION OVERLAYS FOR DIFFERENT PERCENTAGES CHANGE IN SEGMENTATION AREA (6A) FOR COMPASS-L
-20%
-10%%
20%
0.52
0.80
4,40
B 50000
0.10
0.80
Figure 17: Segmentation area increases with 3 for H&E. Each row is different sample. 5AA = 0% the original prediction.
31

---

Published as a conference paper at ICLR 2026
0A -20%
0A -10%
GA 0%
0A 10%
0A 20%
Ground Truth
m -2.00
B -2.00
B -2
[ 2.00
- 2.00
1m--20 -2.00
B -2.00
11 1--22-m
[1 2:00
p- 2.00
2200
2.00
12.77
B -0.1h
B m 20 OX
D D1.16
12.22
B
Figure 18: Segmentation area increases with 3 for Skin Lesion. Each row different sample.
6.A = 0% is the original prediction.
32

---

Published as a conference paper at ICLR 2026
0A -20%
04 -10%
GA 0%
0A 10%
0A 20%
Ground Truth
-2.00
5 -2.03
0 ---200
P 2.00
2.00
1--22 2--000
P -2.00
1 1--2
F 2.00
2.000
0.13
8 2.08
0.00
b.ce
2.00
A -7.0
B -7 00
p 2.00
6 2.00
B=
Figure 19: Segmentation area increases with for Thyroid Nodule. Each row different sample.
6.A = 0% is the original prediction.
33

---

Published as a conference paper at ICLR 2026
0A -00%
04 -10%
dA 0%
04 10%
0A 20%
Ground Truth
- 2,00
5 -2.03
B 2000000
P 2.00
- 2.00
p- 2.00
5 -2 20
11 -2
F 2.00
p- 2.00
00
8 474
8 -.71
B m 0.00
0 3.71
B- D.41
P
9.19
Figure 20: Segmentation area increases with for PolyP. Each is different sample. 8.A 0% is the original prediction.
34


Reviews:
[{"review_id": "XeWrwLLpXf", "reviewer": "Reviewer_FBpp", "summary": "This paper introduces COMPASS, a novel framework for conformal prediction (CP) tailored to medical image segmentation tasks. Unlike traditional CP methods that operate on scalar outputs or pixel-level segmentation masks, COMPASS generates uncertainty intervals directly for clinically relevant downstream metrics (e.g., organ size) by perturbing intermediate neural network features along directions sensitive to the target metric. The method leverages PCA on Jacobians to identify low-dimensional subspaces that are sensitive to the metrics, enabling efficient and valid interval estimation. COMPASS is evaluated across four medical segmentation datasets and demonstrates superior statistical efficiency and robustness under covariate shift compared to existing CP baselines.", "questions": "1. How sensitive is COMPASS to violations of the exchangeability and nestedness assumptions? Have you observed practical cases where these assumptions fail, and how does that affect performance or calibration stability?\n\n2. Is there any way to relax or empirically verify these assumptions before applying COMPASS?\n\n3. Since COMPASS relies on the alignment between features and evaluation metrics, how can we assess or quantify this alignment in practice?\n\n4. Would using features from different network layers or models influence the calibration quality of COMPASS? If so, how can one optimally select which layer’s features to use for Jacobian computation?\n\n5. The experiments focus on area-based segmentation metrics—how would COMPASS handle non-differentiable metrics such as texture or shape? Is it feasible to apply COMPASS to 3D volumetric segmentation or multi-class settings, and what challenges would arise?\n\n6. Has any attempt been made to test COMPASS on non-segmentation tasks, such as regression or classification, to evaluate generality?\n\n7. How do you handle the computational overhead of large Jacobians for high-dimensional feature maps?\n\n8. Given that the calibration step requires multiple forward passes, are there strategies to make it more efficient for large-scale datasets or models?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper moves beyond pixel-level CP to metric-level CP, addressing a critical gap in medical image analysis---uncertainty quantification for derived metrics rather than raw segmentation outputs. The method is conceptually good and has the potential to improve trustworthiness in clinical settings.\n\n2. The paper introduces a clever and principled way to perturb latent features along sensitive directions derived from PCA on Jacobians.\n\n3. The authors provide rigorous coverage guarantees under exchangeability and nestedness assumptions.\n\n4. The proposed method produces tighter intervals than existing methods across multiple datasets, maintains valid coverage and adapts well to covariate shifts, while outperforming baselines in both interval efficiency and robustness.", "weaknesses": "1. The underlying assumptions---exchangeability and nestedness---may not hold in practice.\n\n2. The method relies heavily on feature quality, i.e., the alignment between extracted features and the evaluation metrics.\n\n3. The experiments focus on segmentation tasks involving area estimation. It remains unclear how well COMPASS generalizes or scales to other metrics (e.g., texture, shape) or modalities (e.g., 3D volumes, multi-class segmentation). In particular, defining a differentiable metric function for such cases may be challenging or even infeasible.\n\n4. The computation of Jacobians can be prohibitively expensive for high-dimensional feature maps. Moreover, the calibration step still requires multiple forward passes, which can be computationally costly for large models or datasets.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "q4qiVpgDzI", "reviewer": "Reviewer_n3H6", "summary": "This paper introduces COMPASS, applying SCP to medical image segmentation tasks, specifically targeting uncertainty quantification for derived metrics like organ size. COMPASS leverages the inductive biases inherent in deep neural networks by performing calibration in the model’s representation space. The method perturbs intermediate features along low-dimensional subspaces that are most sensitive to the target metric. COMPASS yields tighter, more efficient prediction intervals compared to traditional CP methods.", "questions": "- The paper mentions using PCA to select sensitive perturbation directions, which might be computationally intractable in high-dimensional feature spaces. Please provide further discussion or proof that this method remains efficient in very deep or high-dimensional neural networks\n- Theoretically, COMPASS provides valid coverage guarantees under exchangeability and nestedness assumptions. However, in real-world applications, these assumptions might not always hold. how the method ensures valid coverage in scenarios where these assumptions do not fully apply?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. COMPASS calibrates uncertainty in the model’s intermediate feature space, rather than directly on the output metric. This leverages the inductive biases of deep neural networks, resulting in more efficient and tighter prediction intervals\n2. COMPASS incorporates a weighted variant to address distribution shifts.", "weaknesses": "1. The method assumes that perturbations along certain directions in the feature space will produce monotonic changes in the metric, this assumption may not hold in all cases, particularly in highly non-linear models, which could lead to suboptimal intervals or coverage guarantees in some real-world scenarios.\n2. the weighted COMPASS variant helps correct for moderate distribution shifts, but the validity of the coverage guarantee relies on accurately estimating the density ratio for the calibration and test sets, and errors in this estimation could compromise the method’s effectiveness in practice. \n3. COMPASS's performance is heavily reliant on the quality of the pre-trained model’s feature representations. If the model’s features are poorly aligned with the downstream metric, the perturbation directions may become inefficient or non-monotonic, leading to invalid or wide prediction intervals.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "66hlWzRFzS", "reviewer": "Reviewer_TJ6a", "summary": "The submission focuses on metric-guided conformal prediction for image segmentation models with applications in medical imaging. The submission proposes to construct prediction sets by perturbing the feature representations of pretrained segmentation models along the PCA directions of the gradients of the downstream metric of interest. The method is extended to conformal prediction under distribution shifts.\n\nExperiments compare the proposed method with several existing baselines across a variety of medical imaging datasets.", "questions": "**Missing citation to Belhasin et al. (2023)**\n\nThe idea of using principal directions to construct prediction sets was introduced by [1] in the context of imaging inverse problems. Although the current submission studies a different problem, it will be important to include this reference.\n\n**Monotonicity of the downstream metric**\n\nThe submission acknowledges that nestedness and monotonicity are assumptions of the theoretical results. However, the proposed algorithm does not satisfy these assumptions in practice. Even if this does not seem to affect the experimental results, it does render the theorems vacuous for the proposed method in a general sense. Figure 6 provides evidence that the principal directions do approximate monotonic changes in the downstream metric.\n\nCould the authors expand on technical solutions to guarantee monotonicity? If not, the claim that validity of the algorithm follows by Theorem 1 should be reworded.\n\n**Extending approach to downstream functions of both the image and the mask**\n\nThe submission focuses on one downstream task, segmentation area, that only depends on the segmentation mask. Could the authors expand on potential limitations and hurdles to extend the proposed method to:\n\n1. Multi-class segmentation problems.\n\n2. Downstream metrics that also depend on the input image and not the segmentation mask only? This might align better with the motivating example of radiomics in the introduction.\n\n**Dimension of $J_i$**\n\nCould the authors clarify what $C,H$ and $W$ represent? I assume these are not the input image number of channels and spatial dimensions, but the size of the feature embeddings at the chosen layer? Are these vectors flattened in practice to compute the principal directions?\n\n**Experiments**\n\nHow many eigenvectors where chosen? Does performance of the proposed method change with $L$? \n\nWhich layer was used to compute the directions? Does this choice matter? It could be interesting to include visualizations of some directions.\n\nIn Figure 3, why do unweighted CP methods outperform weighted ones?\n\n**Intuition behind Sec. 3.3**\n\nI am not sure I understand the intuitive explanation given in Sec. 3.3. I agree that a skewed distribution has a lower quantile, but how does a lower quantile imply smaller interval lengths here? These quantiles are computed along different directions, which are then pushed forward through another, potentially nonlinear function? \n\n---\n\n**Minor comments**\n\n- It might be worth citing a recent work by Mossina and Friedrich, \"Conformal Prediction for Image Segmentation\nUsing Morphological Prediction Sets\" (2025).\n- Theorem 1: technically, differentiability does not matter for the theoretical result? If I understand correctly, it is needed to compute the principal directions only.\n- Line 175: \"optimal\" in what sense?\n\n---\n\n**References**\n\n[1] Belhasin et al. \"Principal uncertainty quantification with spatial correlation for image restoration problems\" (2023).", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The paper is well-written.\n* Metric-guided conformal prediction is a relevant topic.\n* The proposed method is novel, and experimental evaluation is comprehensive.", "weaknesses": "* Lack of theoretical guarantees on the monotonicity of the sets makes theorems vacuous for the proposed method in a general sense.\n* Method is limited to functions of the segmentation mask only, instead of mask and input.\n\nI have a few clarifying questions and comments and I am looking forward to discussing with the authors!", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iyVVdCEzVG", "reviewer": "Reviewer_ypjS", "summary": "The paper targets uncertainty intervals on downstream clinical metrics derived from segmentations, rather than pixel-wise errors. It proposes COMPASS: apply linear perturbations along the most metric-sensitive low-dimensional directions in a network’s intermediate representation, and use the perturbation magnitude as the nonconformity score for split CP to obtain tighter intervals.\nUnder exchangeability and a nestedness assumption, the method provides marginal coverage guarantees; density-ratio–weighted CP is used to recover target coverage under covariate shift. Empirically, across four medical segmentation tasks, COMPASS (especially the Jacobian variant) achieves shorter intervals while maintaining coverage, and is closer to target coverage under distribution shift.", "questions": "1. How do you detect nestedness of S_β (x_i ) in practice? If responses to ±β are asymmetric or non-monotone, what is the remedy—monotone regression, direction re-estimation, or constructing a conservative upper envelope to retain coverage?\n2. What is the variance of the PCA subspace across random splits/initializations? Would subspace bagging reduce variance and improve coverage stability?\n3. Beyond a LiRPA oracle, can you include approximate adversarial search or low-rank projections + Lp balls as simplified FCP to compare under matched compute budgets?\n4. For tiny targets or filamentary anatomy (very small areas), are the intervals still stable? Please report coverage and interval length stratified by target size.\n5. If commercial black-box models expose only logits, what are the gains of COMPASS-L versus COMPASS-J? Please specify the minimal API needed (which gradients/intermediate layers) and provide a deployment guide.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.\tReplaces per-sample adversarial search with a global low-dimensional “sensitivity subspace,” substantially reducing the computational burden of FCP; practically deployable.\n2.\tProvides standard split-CP coverage under a nestedness condition, and naturally extends WCP with feature/gradient-based weighting to handle coverage under shift.\n3.\tDemonstrates weighted CP effectiveness under adversarial label/class shift.\n4.\tOffers empirical evidence for “why it is more efficient”: a power-law relation between score and error with tail compression, plus visualizations showing monotone metric changes under latent-space directional perturbations.", "weaknesses": "1. Nestedness assumption is strong and insufficiently vetted. For intervals S_β (x_i )=⋯(Section2.3), monotonicity/nestedness in beta is nontrivial with deep networks and nonlinear metrics. The main text lacks a systematic diagnostic/guarantee mechanism (e.g., monotonicity tests, conservative envelopes, or projection fixes), which is critical for theoretical validity.\n2. Feature subspace construction and robustness. Collapsing Jacobians by channel-summing before PCA may sacrifice spatial sensitivity; the choice of layer L, cross-layer comparisons, and PCA stability under small samples/noisy gradients are under-analyzed. A single global subspace may underfit heterogeneous distributions (different anatomies/modalities). Do we need grouped/stratified subspaces or online adaptation?\n3. Baselines and scope. The main text omits feasible approximations/variants of FCP; include a practically runnable approximate FCP or feature-perturbation baseline for a fair comparison. Only area (a differentiable metric) is evaluated; adaptation and experiments for nonsmooth/compositional metrics (perimeter, Hausdorff distance, morphology indices, or multi-metric joints) are missing.\n4. Coverage guarantee depends on fixed Delta learned only from training data. If Delta adapts using calibration or test distributions, do we need data splitting or multiplicity adjustments to preserve validity?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)