Conformal Prediction under Lévy-Prokhorov Distribution Shifts: Robustness to Local and Global Perturbations
Authors: Liviu Aolaritei
Qianyu Julie Zhu
Zheyu Oliver Wang
Michael I. Jordan
Youssef Marzouk
Venue: NeurIPS 2025 poster ()
Keywords: Conformal prediction
distribution shifts
optimal transport
Primary area: probabilistic_methods

Abstract: Conformal prediction provides a powerful framework for constructing prediction intervals with finite-sample guarantees, yet its robustness under distribution shifts remains a significant challenge. This paper addresses this limitation by modeling distribution shifts using Lévy-Prokhorov (LP) ambiguity sets, which capture both local and global perturbations. We provide a self-contained overview of LP ambiguity sets and their connections to popular metrics such as Wasserstein and Total Variation. We show that the link between conformal prediction and LP ambiguity sets is a natural one: by propagating the LP ambiguity set through the scoring function, we reduce complex high-dimensional distribution shifts to manageable one-dimensional distribution shifts, enabling exact quantification of worst-case quantiles and coverage. Building on this analysis, we construct robust conformal prediction intervals that remain valid under distribution shifts, explicitly linking LP parameters to interval width and confidence levels. Experimental results on real-world datasets demonstrate the effectiveness of the proposed approach.

Conformal Prediction under Lévy-Prokhorov Distribution Shifts: Robustness to Local and Global Perturbations
Liviu Aolaritei* UC Berkeley Berkeley, CA liviu.aolarite: www.aaallaaaeaeeeeeeoaaaaa wtw.rritteieeeeeeee.e.eeeee edu
Julie Zhu" MIT Cambridge, MA qianyu_z@m z0mit.
Oliver Wang" MIT Cambridge, MA olivrv@mit.edu
Michael I. Jordan
Youssef Marzouk MIT Cambridge, MA ymarzOmit.edi
UC Berkeley Berkeley, CA
jordan@cs berkeley.edi
Abstract
Conformal prediction provides a powerful framework for constructing pre- diction intervals with finite-sample guarantees, yet its robustness under distribution shifts remains a significant challenge. This paper addresses this limitation by modeling distribution shifts using Lévy-Prokhorov (LP) ambiguity sets, which capture both local and global perturbations. We provide self-contained overview of LP ambiguity sets and their connections to popular metrics such as Wasserstein and Total Variation. We show that the link between conformal prediction and LP ambiguity sets is natural one: by propagating the LP ambiguity set through the scoring function, we reduce complex high-dimensiooal distribution shifts to manageable one- dimensional distribution shifts, enabling exact quantification of worst-case quantiles and coverage. Building on this analysis, we construct robust conformal prediction intervals that remain valid under distribution shifts, explicitly linking LP parameters to interval width and confidence levels. Experimental results on real-world datasets demonstrate the effectiveness of the proposed approach.
Introduction
Conformal prediction has emerged as versatile framework for constructing prediction intervals with finite-sample coverage guarantees 32 2. By leveraging the concept of nonconformity, it provides valid confidence sets for predictions, regardless of the underlying data distribution. This framework has gained significant traction fields such as medicine bioinformatics 14] finance 41, and autonomous systems 27 2x where dccision- making under uncertainty is critical. However, the standard conformal prediction framework relies the assumption of exchangeability between training and test data g. When this assumption is violated due to distribution ion shifts, the coverage guarantees conformal prediction may break down, limiting its applicability in real-world scenarios BY
29 38,
Distribution shifts systematic changes between the training and test distributions are ubiquitous in practice. Examples include covariate shift in medical diagnostics, where
Equal contribution.
39th Conference on Neural Information Processing Systems (NeurIPS 2025).

---

the population characteristics evolve over time [6] or adversarial perturbations in image classification, where small, targeted changes to imputs can drastically alter predictions BI, Addressing such shifts is essential for ensuring the reliability of predictive models, particularly in high-stakes applications.
Existing extensions of conformal prediction under distribution shifts impose restrictive structural assumptions: they assume particular types of covariate or label shift [7 8 purely
local 6z-bounded perturbations or purely global contamination [6. M. shifts measured by
prescribed f-divergence [.. While effective in certain scttings, these approaches can struggle with more complex shifts that involve both local perturbations (e.g., small, pixel-level changes in images) and global perturbations (e.g.. population-wis wide shifts in feature distributions) [ To bridge this gap, we propose a novel framework based on Lévy- Pory-Prokhorov (LP) ambiguity sets, class of optimal transport-based discrepancy measures that simultaneously capture local and global perturbations.
LP ambiguity sets offer a flexible and interpretable way to model distributional uncertainty, Unlike f-divergences, which are limited to absolutely continuous shifts, LP metries naturally handle broader scenarios, including discrete and transport-based perturbations 77. For example, LP metrics can capture local shifts such as minor variations in image textures or sensor readings, as well as global shifts like changes in population demographics. This dual capability makes LP metrics particularly suited for robust prediction in dynamic and heterogeneous environments.
In this paper, we leverage the LP ambiguity set to develop a distributionally robust extension of conformal prediction. By propagating LP ambiguity sets through the scoring function, we simplify high-dimensional shifts into one-dimensional shifts in the score space, enabling exact quantification of worst-case quantiles and coverage. This approach leads interpretable and robust prediction intervals, with explicit control over how the local and global LP parameters influence interval width and confidence levels.
Finally, we validate the proposed approach on three benchmark datasets: MNIST 25 ImageNet [3 and iWildCam [ the latter of which captures real-world distribution shifts, demonstrating its empirical coverage guarantees and efficiency in terms of prediction set size.
1.1 Related Work
Under train-test distribution shifts that violate exchangeability, conformal prediction often fails to maintain valid coverage guarantees [37. Extensions to conformal prediction under such shifts can be summarized into three main categories: sample reweighting, ambiguity sets, and sequential lcarning.
Sample Reweighting. This approach assigns weights to calibration samples based on their relevance to the test data. For instance, [37] proposed weighted conformall prediction for covariate shift, where the marginal distribution Pxx changes while the conditional distribution Pr|x remains fixed. Likelihood ratios are used to adjust for compositional differences, enabling valid predictions. Subsequent extensions address label shift 34, causal inference 266 and survival analysis g 20). However, these methods roly on the accurate estimation of likelihood ratios, which may be challenging in practice. For spatial data, [30 proposed weighting samples based on proximily to test points. Still within the covariate shift setling, B5 and A6 leverage semiparametric theory to design more efficient conformal met hods with asymptotic conditional coverage, bypassing the need for explicit sample reweight Compared to these approaches, our method handles distribution shifts in the joint distribution P of (X, Y), without requiring likelihood ratios, and remains effective under more complex local and global perturbations.
Ambiguity Sets. Ambiguity sets provide a flexible framework for modeling uncertainty in the data distribution. For instance, g used an f-divergence ambiguity set around the training distribution to derive worst-case coverage guarantees and adjusted prediction sets. This work is most closely related to ours, and while their analysis inspired our approach, we rely on fundamentally different lools, particularly drawing on optimal transport techniques. key limitation of f-divergences is that they are restricted to distribution shifts that are absolutely continuous with respect to the training distribution. Building on this line
2

---

of work, [ proposed a robust conformal inference framework that explicitly separates covariate and conditional shifts: the former is handled via sample reweighting without constraints, while the latter is modeled using f-divergence ball. This decomposition enables distinct handling of covariate and conditional shifts, improving efficiency compared to worst-case joint modeling. A related approach is Wasserstein- Reghttiin--gllrrizedd Conformal Prediction (WR-CP) [ which heuristically minimizes an empirical upper bound on the coverage gap under joint distribution shift by combining importance weighting with Wasserstein distance regularization in score space, However, WR-CP requires kernel density estimation and repeated Wasserstein computations during training, and does not offer formal coverage guarantees under worst-case shifts. Differently, [6] proposed robust score functions based on randomized smoothing [2] 24, which ensure valid predictions under adversarial perturbations within fz-norm balls. While ad adversarial mets tend to produce overly conservative uncertainty sets, recent. works [5] 17 [T] have refined prediction sets by considering specific perturbation structures. Other extensions have incorporated poisoning attacks and non-continuous data types such graphs [g However, these methods often assume very specific types of distribution shifts or require solving complex optimization problems. In a related spirit, both 47 and [2 study worst-case coverage under unmeasured confounding, modeled via the I'sselcctonn framework. While their focus is on causal inference and the distributional shifts induced by hidden confounders, their robustness guarantees parallel our LP-based approach in targeting worst-case coverage over structured class of perturbations. In contrast, our method employs unified discrepancy measure that captures both local and global perturbations, imposes no assumptions on the score distribution, and provides a computationally efficient way to construct prediction sets.
Sequential Learning. While most methods assume i.i.d. or exchangeable training data, several works have explored sequential conformal prediction. These methods include updating nonconformity scores [2], leveraging correlation structures [ reweighting samples [3] El, and monitoring rolling coverage [8 10
[ 5 Although our method does not focus on sequential settings, extending it to this context is a promising avenue for future research.
1.2 Mathematical Notation
We denote by P(Z) the space of Borel probability distributions on 2; x==XXx y'C x R. Given IP E P(Z), we denote by Z ~ IP the fact that the random variable 2 is distributed according to P. Projection maps are denoted by ग, and the indicator function of a set A is denoted by {{4]. We implicitly assume that all maps 8: Z + are Borel. We denote by F I the pushforward of P via the map s, defined as (AA == P(s (A)),, for all Borel sets A S Z. Throughout the paper, || denotes an arbitrary norm on Z. Given P, PE E P(3), the co- Wasssesssllnn dislance is defined
Woo (P,, Q) = inf { 2 0: inf
1| =21l e) dx(2) s $0
(1)
TET(F.)) J zxz
where T(P, Q) is the set of all joint probability distributions over Z x Z, with marginals P and Q, often called transportation plans or couplings 39 Moreover, the Total Variation (TV) distance is defined as
/
TV, Q = inf (EI((..) 2 11 - 221l > 0)dr(21, 22).
(2)
At first sight, definition 2 might seem different from the more classical definition (F, Q) =
(up()P(AA) - Q(A))= A C Z is Borel set). We refer to [23] Proposition 2.24 for proof of their equivalence. Here, we prefer definition (2), as it demonstrates that the TV distance is special case of an optimal transport discrepancy, enabling us to leverage the extensive literature on optimal transport 39. Finally, we denote the a-quantile of a distribution IP by
Quant(a: P) == inf E E PPSS < s) 1 a).
2 Lévy-Prokhorov Distribution Shifts
We model distribution shifts as an ambiguity set, i..., a ball of probability distributions
B (P) = (Q P(Z): LP (P, Q) s pt,
(3)
3

---

around the training distribution P, constructed using the Lévy-Prokhorov (LP) meeudo-metric
/
inf LP (P, = -EI((2,)) Jzxz [[] - =2ll > e)dr(21,, 22).
(4)
Note that the LP pseudo-metric belongs to the general class of optimal transport discrepancies, with the particular choice of transportation cost ((21,22) := 1[ -22ll > }} 7 In this section, we provide detailed exposition of the LP pseudo-metric and explore its expressivity in modeling significant distribution shifts. The section culminates with Proposition 2.5 where we study the propagation of IB ((F) thorough the scoring function s, showing that the LP distribution shift can be directly considered in the one-dimensional nonconformity scores.
For more insights into the LP ambiguity set, we begin by presenting alternative represen- lation that decomposes it in terms of the oc- oc-Wasserstein distance and the TV distance.
Proposition 2.1 (Decomposition of the LP ambiguity sct). The LP ambiguity set can be equivalenlly rewrillen as
B (P)
U (..)(s
(Qe TV( Q) sp..
(5)
All proofs of the paper are deferred to Appendix D The decomposition in equation reveals that each distribution Q E (P)) can be constructed through a two-step procedure. First, the center distribution P undergoes a local perturbation, resulting in an intermediate distribution P that lies within a Wo distance of at most from P. This implies that each unit of mass in PP can be arbitrarily relocated within radius of in 2. Secondly, P is subjected to a global perturbation, producing the final distribution Q, which lies within a TV distance of at most from P. Specifically, this step entails displacing up to fraction of P's total mass to any location in the space 2. This decomposition in (5) immediately implics that other well-known distribution shifts can be recovered as extreme cases of the LP ambiguity set (P))
Corollary 2.2 (Relationship to other metrics).
Bo (P) recovers the TV ambiguily set P(Z): TV(P,Q)
(ii) D recovers the oo- Wasserstein ambiguity set (Q E P(Z): (a.
The decomposition in 5 can also be expressed in terms of random variables, which may offer clearer understanding of the LP distribution shifts. We state this in the following proposition, which recovers 7 Theorem 2.1] using a different approach.
Proposition 2.3 (Local and Global Perturbation). Let Zj ~ P. Then Q E (P. if and only if there exists random variable Zz Q of the form
(6) the random variables N, B,C. are as follows: N represents the local perturbation, with support {n E Z ||n|]| S E), B indicates whether the sample is globally perturbed or not, with Prob(B 1) < p, and C represents the globol perturbation, following an arbitrary distribution on Z. In particular, Z, N,NB B, and C' can all be correlated.
Ze (Z1 + N)1
+ C1]
Propositions 2.1 and 2.3 readily imply that the LP ambiguity set allows for distributions Q which are significantly different from P, as the following remark explains.
Remark 2.4 (Absolute continuity). The decomposition in (5) implies that (P)) may contain distributions that are not absolutely continuous with respect to P.. This generality particularly valuable in settings where the test distribution assigns mass to regions unobserved during training. Such shifts are excluded under f-divergence ambiguity sets 9 or models that enforce bounded likelihood ratios between the test and training distributions 37]
So far, we considered the distribution shift modeled via LP ambiguity set in the space 2 = X x Y. This in line with supervised learning, where is more natural to consider distribution shifts in data-space X x V, as opposed to distribution shift in the score-space

---

J). Nonetheless, from a technical viewpoint, it is much casier to deal with an LP ambiguity set in the one-dimensional scores, due to its immediate relationship with the cumulative distribution functions and quantiles. The following proposition shows that the result of the propagation of (PP) through 8 is again captured by an LPP ambiguity set, allowing us to effectively restrict the analysis to distribution shift on the scores.
Proposition 2.5 (Propagation of the LP ambignity set). Let the scoring function s 2 - R be k- Lipichitt over Z, for some & E R4. Then,
s s.BBe,(P)) S (s
(7)
Proposition 2.5 requires to be Lipschitz continuous over Z. This condition is trivially satisfied if, for instance, 8 is continuous and 2 is compact. In light of the inclusion (7) we focus, for the remainder of the paper, distribution shifts over the nonconformity scores. These shifts are modeled via an LP ambiguity set (PP), where, for simplicity, we omit the Lipschitz constant & from the notation and consider IP to be directly the distribution of (Z)) Note that, in this case, all distributions inside (PP) are supported on R.
Remark 2.6 (Lipschitzness of the score function). The Lipschitz assumption in Proposition 2.5 is not required for any other theoretical results in this paper. It merely illustrates how data-space perturbations translate inlo score-space perturbations under a smooth scoring function. All subsequent results, including our coverage guaranlees under distribution shift, are derived by modeling shift directly over the nonconformity scores. This modeling choice aligns with standard practice in conformal prediction under distribution shift (e.g., 9), and enables our framework to accommodate arbitrarily complex, potentially non-Lipschitz score functions such as deep neural networks.
Worst-Case Quantile and Coverage
In this section we introduce and analyze the two key quantities which allow us to construct a robust prediction interval with the right coverage level for any test distribution in the LP ambiguity set. The first quantity is the worst-case quantile, defined below.
Definition 3.1 (Worst-case quantile). For E [0, 1], the worst-case B-quantile in (P,) is defined as
WC Quant ,p (B:P) : sup Quant Q). (E)),
(8)
Equation defines the worst-case quantile through distributionally robust optimization problem, which quantifies the largest 3-quantile for all the test distributions in (P). In other words, Quant WC (P: represents the worst-case impact of the distribution shift on the value of the 3-quantile. This, in turn, affects the size of the confidence interval, as we will show in Section 4 The second quantity is the worst-case coverage, defined next.
Definition 3.2 (Worst-case coverage). Let Fg : R [0, 1] be the cumulative distribution function of Q.or q E IR. Then, the worst-case coverage in (P) at g is defined as
Cov c.n (q: PP) inf (1)
Fg(
Equation 3.2 defines the worst-case coverage as the lowest value among the cunulative distribution functions in the LP ambiguity set evaluated at q E R. For example, if q = Quant(1-a - a; P), Cov WC (q: P) represents the worst-case impact of the distribution shift on the true confidence level when the confidence level for P. is 1 a. In the remainder of this section, we will show that both WC (P) and Cov CC (q; can be quantified in c.p
closed-form, as a function of the training distribution IP and the two robustness parameters E, p. Before doing so, we note that a high value of p. ie., the global perturbation parameter, renders the worst-case quantile trivial. We show this in the following remark.
Remark 3.3 (Case p 2 1- 3). If p 1- 33 then Quant s. PB; = Quant Intuitively, the LP ambiguity set (P) allows to displace p mass from the distribution and move it arbitrarily in R. Since p 2 11--8 3, this implies that we can construct sequence of distributions Qn E Be,((P) for which Quant Qx) + co. To see this, let PP = =(0 1)), and let Q. = U([0, 1 pl) + pón. Then, clearly LP. (P. Qn) p. and Quant (3; Qx) 2
5

---

F5
FF
p
0
p
Cov WC s.p (q; IP)
Quant s, (B:P))
q
Figure 1: (Left) Corst-case quantile; (Right) Worst-case coverage.
Following Remark 3.3 we restrict our attention the case p < 1 3 in the quantity WC Quant 5. (3:P). We are now prepared present the first result of this section.
Proposition 3.4 (Worst-case quantile the LP ambiguity set). The following holds
Quant WC (3; IP) = Quant (3 + (5 +
(10)
In words, the worst-case quantile in the LP ambiguity set IBe.p (P) corresponds to quantile of P that shifted by the local parameter e and adjusted by the global parameter p. We will now present the second result of this section.
Proposition 3.5 (Worst-case coverage in the LP ambiguity set). The following holds
Cover (q; (q: Frlg -)) p.
(11)
The worst-case coverage in the LP ambiguity set ( corresponds to the coverage of P shifted by the local parameter and adjusted by the global parameter p. The proofs of Propositions 3.4 and 3.5 are constructive, in the sense that we propose two sequences of distributions which attain, in the limit, the two quantities Quant WC (B;; respectively. The intuition for both constructions stems from Proposition 2.1 which allows us to construct every distribution in , (I) using two-step procedure that decouples the local and global perturbations. This intuition is illustrated in Figure
(P; and Cov WC (q; IP),
4 Distributionally Robust Conformal Prediction
In this section, we demonstrate how the worst-case quantile and coverage introduced earlier enable the construction of a confidence interval and its worst-case coverage for all distributions in the LP ambiguity set. We start by defining the prediction set
C (x;P) = f E ): s(x, v) S Quant WC (1-
a;; PP:
(12)
WC (1 P; Quant(1 - a + P; + Observe that (r, depends on the training distribution F, which is unknown. Instead, we assume access to exchangcable data points {s(X,, Y)) P. Based on this, we define the empirical distribution P, = E 6s(X.,Yi), and consider the empirical confidence set -1 (zr; P.)). We now state the main result of this paper.
where, as noted in Proposition 3.4 Quant QuanttWor
Theorem 4.1 (Conformal Prediction under L.P distribution shifts). Let a s(xn+1, Yu+1) Prest be independent of (s(X,, vi)) ~ P. Morcover, let TP- (P, IP PPestt) p. Then,
Prob Yott E Ct (Xnti:P.)) [n[1 - a p)1 n 11
(13)
few remarks are order. First, the local parameter e affects only the size of the confidence interval, but not its coverage guarantee. This is expected, given the construction of the two

---

MNIST
ImageNet
2000
Training
2500
Training
Test
3000
Frequency
Freuency
1250
2000
1000
750
500
1000
250
500
-00
155
-10
0
10
-2
y))(s(x.y))
log(s(x, y!)
Figure 2: Score distribution shift. Plots for MNIST and ImageNet under (p= 0.05, u 1.0) perturbation. The score distribution obtained from the unperturbed data (red), and from the perturbed data (blue) are plotted in log scale.
sequences of distributions that achieve the worst-case quantile and coverage in Propositions 3.4 and 3.5 respectively (also illustrated in Figure D. In contrast, the global shift parameter influences both the coverage and the size of the prediction set: it shifts the quantile level from 1 - a to 1 + P, and appears subtractively in the coverage bound. This change in quantile level often has a more pronounced effect on the size of the prediction set than the additive e term, particularly when the score distribution is light-tailed. Meanwhile, the reduction in coverage due to P decreases with the calibration size n, and becomes negligible in the large-sample regime, scaling (1/n). Finally, expected, the distribution shift reduces the coverage below the desired 1 a level. The following corollary provides an adjusted coverage for the worst-case quantile, ensuring 1 confidence level in 13
Corollary 4.2 (1 coverage). Let a (a 2)/n. n.. Under the same conditions as in Theorem 1 we have
Prob { Yat1 C1- (Xut1;P.)) 2 a.
(14)
Recall from Corollary 2.2 that the LP pseudo-metrie recovers the TV and oo- oo-Wasserstein distances if : = and p 0, respectively. As consequence, the guarantee in Corollary 4.2 can be immediately specialized to these additional types of distribution shifts.
Experiments
We conduct experiments three classification problems: MNIST 25 ImageNet 1
and iWildCam G. We also compare our algorithm against five other methods: standard
split conformal prediction (SC), ddviieegeee robust conformal prediction g. conformal prediction under covariate shift (Weight) 37, randomly smoothed conformal prediction (RSCP) [6] and fine-grained conformal prediction (FG-CP) [.. Each method defines its
own prediction set; for our method, this is the robust set Cl-" (x; from Corollary 4.2 While additional methods exist in the literature, they typically constitute minor variations or special cases of the five reprosentative basclincs we benchmark against.
We evaluate methods in terms of validity and efficiency. Validity is computed as the average empirical coverage across M independent calibration-test splits x EM ΣK (() claf9i; ( MH, where fe(i) denotes the empirical distribution of the j-th calibration set, and ((a)), y ) denotes the corresponding test set. Efficiency is evaluated as the average prediction set size across the same M splits and K test samples. For all experiments, we set the miscoverage level to = 0.1 and use the negative log-likelihood (NLL) score, s(x,y) = -loog log a)) as the nonconformity measure. For ImageNet, we use a pre-trained R5s model; for MNIST, we train a small ResNet architecture from scratch; and for iWildCam, adopt the pre-trained ResNet-50 model provided by 6.

---

p=d 01,, u=0.25
0=0.025, U=0.5
p=0 05, u=
Ohe vatoot
033
080
030-
0.03
MNSST
Oeig
LIgTT
FDCP
LPTT
FDOP
LPST
Reggl:
Time
Coverage
Caseeape
Eoverage
0.2
Imagelett
0:06
113
360
10ggh 10:2
144g0 0000 12
Figure 3: MNIST and ImageNet. Coverage (validity) and size (efficiency). In the coverage plots, the long dashed line indicates the target 1 - a level. Scattered points show empirical coverage and prediction set size for each calibration- teation--estt split, while short horizontal lines denote averages across M = 30 splits. The proposed methods highlighted in bold/rod.
5.1 Data-Space Distribution Shift: MNIST and ImageNet
Following the split conformal procedure, we partition the hold-out validation into calibration set of 1000 samples and test set of K = 5000 samples drawn uniformly from the remaining data. We simulate local perturbations by adding i.i.d. noise from 4([[u, ul) to every channel of cach test image. Global perturbations are introduced by randomly corrupting a fraction p of Lest labels, replacing each with a neighboring class label. This setup captures realistic scenarios in which lest-Lime inpuls are noisy and some labels may be incorrect due to annotation errors 15 Figure 2 illustrates the resulting shift the score distribution under the perturbation setting (p 0.05, 1.0).
Calibration NLL scores are computed on unperturbed calibration data points to determine empirical quantiles. Constructing prediction sets is then straightforward for standard conformal prodiction. For the robust algorithms, our method naturally accounts for both global and local perturbations through the parameters and respectively. Following Proposition 2.5 we set p = p to reflect the global label corruption level. While the same proposition suggests setting e = ku, where k is the Lipschitz constant of the score function, estimating from data often leads to overly conservative values, as a global Lipschitz constant may not reflect the local behavior of the score function where the data are concentrated. In practice, we find that a fixed value h = suffices to ensure valid coverage across the full range of data-space shifts a data-driven variant, called LPest, which estimates both and p directly from samples using the algorithm described in Appendix [ This version achieves similar robustness while adapting more fiexibly to the underlying shift. For the x², FG-CP, RSCP, and Weight conformal prediction methods, we follow the original experimental setups described in their respective references; implementation details are provided in Appendix [C
refer to this method as LPs. In parallel, evaluate
Figure reports the empirical coverage and prediction set size (averaged over 30 calibra- tion test splits) for the seven methods under three levels of noise corruption: (p, E {00.01, 0.25), (0.025, 0.5), (0.05, 1.0)). As expected, standard conformal prediction (SC) fails
8

---

Coverage
Size
0.0200
0.944
0.0200
Lov 0.9
Extianted
Grd Swarch
0.934
0.0175
Gnt Search
10.50
9.75
0.0150
0.924
0.0160
9.00
0.0125
8.25
0.0100
0.902
0.904
0,0100
CD
= 0.87, 2
- -7.50
0.0075
0.894
0.0075
6.75
0.884
6.00
1. 10000,0 0.000)
1:00,0 004)
0.0025
Loy - 0,001
0.674
0.0025
Siar - 6.521
5.25
0.0000
E
Figure 4: iWildCams Coverage (left) and prodiction set size (right) across (E. values. The white dashed line marks the set of (E. pairs achieving exactly 90% empirical coverage, and the smallest prediction set along this frontier is shown with black circle. White circles correspond to points estimated by the algorithm in Appendix [ and the best-performing pair among them (yielding the smallest prediction set) is marked by black diamond.
to maintain coverage as the corruption level increases. In contrast, both variants of our method- -L,, and 1.pest_cnsiitenti -cnniiseetly maintain valid coverage across all settings. They also achieve comparable prediction set sizes, demonstrating the effectiveness of data-driven parameter estimation. Among the remaining baselines, only RSCP maintains valid coverage under all shift levels, but it does so at the cost of extremely large prediction sets, particularly for ImageNet. The other three baselines, i.e., x², FG-CP, and Weight, exhibit coverage degradation as the shift intensity increases. This is expected: these methods assume absolute continuity between the training and test distributions, condition violated in our experi- mental setup (see Figure 2). In particular, when test-time perturbations cause the support of the test distribution to lie partially outside that of the training distribution, methods relying on importance weighting or o f-divergence balls struggle to provide valid guarantees. In contrast, our T.P-based approach requires no absolute continuity and remains robust to both global label corruption and local input noise.
This numerical illustration also highlights an important modeling point. The LP-based approach is specifically designed to capture local and global perturbations of the data distribution, as introduced in this experiment. It provides a principled framework for handling such shifts, complete with closed-form expressions for both the worst-case quantile and coverage. As result, the strong empirical performance observed here is not coincidental: our method is theoretically tailored to this class of distribution shifts, and no other method can offer stronger worst-case guarantees within the same ambiguity set.
5.2 Real-world Distribution Shift: iWildCam
We now evaluate our algorithm's ability to handle real-world distribution shifts using the iWildCam dataset [6,, multi-class classification task characterized by naturally occurring train-test discrepancies. These arise from changes in camera trap placement and timing, which induce variability in illumination, color, viewpoint, background, vegetation, and species frequency. As described in [22, the dataset includes a training set, an out-of-distribution test set, and an in-distribution validation/ ttest set consisting of images captured from the same camera locations as the training data but on different dates. We use the in-distribution test set for calibration and the out-of-distriibtion test set for evaluation.
Figure 5.2 illustrates how coverage and prediction set size vary over a grid of (s, p) values in the LP ambiguity set. The left panel shows that all pairs lying to the right of the black dotted contour (the 90% coverage curve) yield valid coverage under the real distribution shift. This demonstrates that. LP ambiguity sets capture the relevant perturbations affecting (WildCam, without assuming prior knowledge of the shift. type or structure. The right panel shows the corresponding prediction set sizes. Notably, moving further right from the 90%

---

contour leads to increasingly conservative sets. White circles in both panels denote (E: pairs estimated by the data-driven procedure described in Appendix B The best among with a black diamond diamond-achiever achieves nearly identical coverage and prediction set size as the optimal point found by an exhaustive grid search (marked by black circle). This proximity confirms that the proposed estimation algorithm reliably recovers high-quality ambiguity set parameters with limited test data.
Taken together, these results support two key takeaways: (1) LP ambiguity sets flexibly model real distribution shifts, delivering valid coverage across a broad region of the parameter space, and (2) the estimated (E, p) pair performs comparably to the best grid-tuned pair, both in coverage and efficiency.
Acknowledgement
Liviu Aolaritei acknowledges support from the Swiss National Science Foundation through the Postdoc.Mobiliee Fellowship (grant agreement P500PT 222215). Michael Jordan was funded by the Chair "Markets and Learning," " supported by Air Liquide, BNP PARIBAS ASSET MANAGEMENT Europe, EDF, Orange and SNCF, sponsors of the Inria Foundation. Youssef Marzouk and Julie Zhu acknowledge support from the US Department of Energy (DOE), Office of Advanced Scientific Computing Research, under grant DE-SC0023188. Youssef Marzouk and Zheyu Oliver Wang acknowledge support from the ExxonMobil Technology and Enginecring Company.
References
[1] Jiahao Ai and Zhimei Ren. Not all distributional shifts are equal: Fine-grained robust conformal inference. arXiv preprind arXiv:2402. 13042, 2024.
[]] Anastasios N Angelopoulos, Rina Foygel Barber, and Stephen Bates. Theoretical foundations of conformal prediction. arXiv preprint arXiv:8411.1188 11824, 2024.
Liviu Aolaritei, Nicolas Lanzetti, Hongruyu Chen, and Florian Dörfler. Distributional uncer- tainty propagation via optimal transport. IEEE Transactions on Automatic Control (Forth- coming), 2025.
[4] Rina Foygel Barber, Emmanuel Candes, Aaditya Ramdas, and Ryan J Tibshirani. Conformal prediction beyond exchangeability. The Annals of Statistics, (2):816 845, 2023.
55] Osbert Bastani, Varun Gupta, Christopher Jung, Georgy Noarov, Ramya Ramalingam, and Aaron Roth. Practical adversarial multivalid conformal prediction. Advances in Neural Information Processing Systems, 35:29362 29373, 2022.
[6] Sara Beery, Elijah Cole, and Arvi Gjoka. The iwildcam 2020 competition dataset. arXiv preprint orXiv:2004. 2020.
[7] Amine Bennouna and Bart Van Parys. Holistic robust data-driven decisions. arXiv preprinl arXio:2207.0955 2022.
[8] Emmanuel Candès, Lihua Lei, and Zhimei Ren. Conformalized survival analysis. Journal of the Royal Statistical Society Series B: Statistical Methodology, 85(1):24-45, 2023.
[9] Maxime Cauchois, Suyash Gupta, Alnur Ali, and John C Duchi. Robust validation: Confident predictions even when distributions shift. Journal of the American Statistical Association, 119 (5488)33033 3044, 2024.
[10] Victor Chernozhukov, Kaspar Wüthrichh and Zhu Yinchu. Exact and robust conformal inference methods for predictive machine learning with dependent data. In Conference On learning theory, pages 732-749. PMLR, 2018
[11] Jase Clarkson, Wenkai Xu, Mihai Cucuringu, and Gesine Reinert. Split conformal prediction under data contamination. arXiv preprint or Xri::24077 2024.
[12 Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In international conference on machine learning, pages 1310-1320. PMLR, 2019.
10

---

[13] Jia Deng, Wei Dong, Richard Sucher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenel: A large- scale hierarchical image database. In 2009 IEEE conference on compuler vision and pallern recognation, pages 248 255. Iowe, 2009.
[14] Clara Fannjiang, Stephen Bates, Auastasios N Angelopoulos, Jennifer Listgarten, and Michael 1 Jordan. Conformal prediction under feedback covariate shift for biomolecular design. Proceedings of the National Academy of Sciences, 119(44))e220456911, 2022.
[15] Shai Feldman, Bat-Sheva Einhinder, Stephen Bates, Anastasios N Angelopoulos, Asaf Gendler, and Yaniv Romano. Conformal prediction is robust to dispersive label noise. In Conformal and Probabilistic Prediction with Applications, pages 624 626. PMLR, 2023.
[16] Asal Gendler, Tsui- Tsui-Wei Weng, Luca Daniel, and Yaniv Romano. Adversarially robust conformal prediction. In International Conference on Learning Representations, 2021.
[17] Subhankar Ghosh, Yuanjie Shi, Taha Belkhouja, Yan Yan, Jana Doppa, and Brian Jones. Probabilistically robust conformal prediction. In Uncertainty in Artificial Intelligence, pages 681-690. PMLR, 2023.
[18] Isaac Gibbs and Emmanuel Candes. Adaptive conformal inference under distribution shift. Advances in Neural Information Processing Systems, 34:1660- 1672, 1116666677,
2021.
[19] Isaac Gibbs and Emmanuel Candès. Conformal inference for online prediction with arbitrary distribution shifts. Journal of Machine Learning Research, 25(162):1 36, 2024.
[20] Yu Gui, Rohan Hore, Zhimei Ren, and Rina Foygel Barber. Conformalized survival analysis with adaptive cut-offs. Biometrika, 111(2):459 477, 2024.
[21] Ying Jin, Zhimei Ren, and Emmanuel Candès. Sensitivity analysis of individual treatment effects: A robust conformal inference approach. Proceedings of the National Academy of Sciences, (e066):e22148811 2023.
[22] Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, et al. Wilds: A benchmark of in-the-wild distribution shifts. In International conference machine learning, pages 5637 5664. PMLR, 2021.
[23] Daniel Kuhm, Soroosh Shafiee, and Wolfram Wiesemann. Distributionally robust oplinization, 2024.
[24] Aounon Kumar, Alexander Levine, Soheil Feizi, and Tom Goldstein. Certifying confidence via randomized smoothing. Advances in Neural Information Processing Systems, 33:5165-5177, 2020.
[25] Yann LeCun. The mnist database of handwritten digits. http://yann, lecun. com/exdb, moobb/mnnit//. 1998.
[26] Lihua Lei and Emmanuel J Candès. Conformal inference of counterfactuals and individual treatment effects. Journal of the Royal Stalistical Sociely Series B: Statistical Methodology, 83 (5):911 938, 2021.
[27] Lars Lindemann, Matthew Cleaveland, Gihyun Shim, and George J Pappas. Safe planning in dynamic environments using conformal prediction. IEEE Robotics and Automation Letters, 2023.
[28] Lars Lindemann, Xin Qin, Jyotirmoy V Deshmukh, and George Pappas. Conformal prediction for stl runtime verification. In Proceedings of the ACM/IEEE 14th International Conference on Cyber- Pheer-Phssica Systems (with CPS-ToT Week 2023), pages 142 153, 2023.
[29] Charles Lu, Andréanne Lemay, Ken Chang, Katharina Höbel, and Jayashree Kalpathy-Crant Cramer. Fair conformal predictors for applications in medical imaging. In Proceedings of the AAAI Conference Artificial Intelligence, volume 36, pages 12008 12016, 2022.
[30] Huiying Mao, Ryan Martin, and Brian J Reich. Valid model-free spatial prediction. Journal of the American Slatistival Associalion, 11(6546):9044999 914, 2024.
[31] Seyed- Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. Universal adversarial verturbations In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1765-1773, 2017.
11

---

[32] Harris Papadopoulos, Kostas Proudrou, Volodya Vovk, and Alex Gammerman. Inductive coufidence machines for regression. In Machine learning: ECML 2002: 13th European conference on machine learning Helsinki, Finland, August 19 23, 2002 proceedings 13, pages 345 356.
Springer, 2002.
[33] Gabriel Peyré, Marco Cuturi, et al. Compatational optimal transport: With applications to data science. Foundations and Trendso in Machine Learning, 11(5-6):355-66) 607, 2019.
[34] Aleksandr Podkopaev and Aaditya Ramdas. Distribution-fies uncertainty quantification for
classification under label shift. In Uncertainty in artificial intelligence, pages 844-853. PMLR,
2021.
[35] Hongxiang Qiu, Edgar Dobriban, and Eric Tchelgen Tchetgen. Prediction sets adaplive to unknown covariate shift. Journal of the Royal Statistical Society Series B: Statistioal Methodology, 85(5):1680-1705, 2023.
[36] Keyvan Rahmani, Rahul Thapa, Peiling Tsou, Satish Casie Chetty, Gina Barnes, Carson Lam, and Chak Foon Tso. Assessing the effects of data drift on the performance of machine learning models used in clinical sepsis prediction. International Journal of Medical Informatics, 173: 104930, 2023.
[37] Ryan Tibshirani, Rina Foygel Barber, Emmanuel Candes, and Aaditya Ramdas. Conformal prediction under covariate shift. Advances in newral information processing systems, 32, 2019.
[88] Janette Vazquez and Julio Facelli. Conformal prediction in clinical medical sciences. Journal of Healthcore Informatics Research, 6(3):241-253) 2022.
[39] Cédric Villani et al. Optimal transport: old and new, volume 338. Springer, 2009.
[40] Vladimir Vovk, Alexander Gammerman, and Glenn Shafer. Algorithmic learning in random
world, volume 29. Springer, 2005.
[41] Wojeiech Wisniewski, David Lindsay, and Sian Lindsay. Application of conformal prediction interval estimations to market makers' net positions. In Conformal and probabilistic prediction and applications, pages 285-301. PMLR, 2020.
[42] Chen Xu and Yao Xie. Conformal prediction interval for dynamic time-series. International Conference on Machine Learning, pages 11569, PMLR, 2021.
[43] Chen Xu and Yao Xie. Sequential predictive conformal inference time series. International Conference on Machine Learning, pages 38707-38727. PMLR, 2023.
[44] Rui Xu, Chao Chen, Yue Sun, Parvathinathan Venkitasubramaniam, and Sihong Xie. Wassersteinn-reguaaiia conformal prediction under general distribution shift. arXiv preprint arXio:2501.133 13430, 2025.
[45] Ge Yan, Yaniv Romano, and Tsui Wei Weng. Provably robust conformal prediction with improved efficiency. Xixiv preprint (tt:2404.1165... 2024.
[46] Yachong Yang, Arun Kumar Kuchibhotla, and Eric Tehelgen Tchetgen. Doubly robust calibration of prediction sets under covariate shift. Journal of the Royal Statistical Society Series B: Statistical Methodology, 86(4):943-965 2024.
[47] Mingzhang Yin, Claudia Shi, Yixin Wang, and David M Blei. Conformal sensitivity analysis for individual treatment effects. Journal of the American Statistical Association, 119(545):122 123-135, 2024.
[48] Margaux Zaffran, Olivier Féron, Yannig Goude, Julie Josse, and Aymeric Dieuleveut. Adaptive conformal predictions for time series. In International Conference on Machine Learning, pages 25834 25866. PMI.R, 2022.
[49] Soroush H Zargarbashi, Mohammad Sadegh Akhondzadeh, and Aleksandar Bojchevski. Robust yet efficient conformal prediction sets. aix preprint arXiv:2407.0911 09165, 2024.
12

---

A Preliminaries in Conformal Prediction
In what follows, we provide a brief introduction to split conformal prediction. Consider a predictive model f X' + y and calibration dataset D = Y.) S X x V/ where the points in D, along with any test sample (Xn+1, Yn41) E X x are assumed to be exchangeable and distributed according to P. Without additional assumptions on the predictive model or the data-generating process, conformal prediction constructs a prediction set C' (Xa+1) that satisfies the finite-sample coverage guarantee:
Prob (Yn+1 E cC'="(X++)) 21
(15)
where the probability is taken over both the calibration dataset D and the lest point (K441,N411)
To achieve this, conformal prediction relies on a scoring function s X' x y + R, which quantifies the nonconformity of label y y for given input E X. The predictive model f is typically used to define the scoring function 8, where f(z) represents the modol's prediction. In regression, f()) might return point estimate of y, with s(x, y) defined as the absolute error [f(z) - yl- In classification, f(z) might output class probabilities, and s(r,, y) could be the negative log- ung-robbabiliyy of the true label y. For each calibration point (x, E D, the nonconformity score s(X,, Y,) is computed. The scores are then used to estimate the empirical (1 qu-antlee with finile-sample correction:
[[1 (I + 1)] 88# sstP.).
Ta = Quant
n
where P is the empirical distribution of the calibration scores {s Y) Finally, the prediction set for a new label Y,+1 is defined as
C1-a(Xa+1 = fy E y:
S 7.
By construction, the prediction set C²1a((X+1)) satisfies the coverage guarantee in (15 provided the data exchangcable. With the conformal prediction framework in place, we now shift our focus to the challenge of distribution shifts. Specifically, consider scenarios where the lest data (Xn+1; Yu+1) is drawn from a distribution that differs from the distribution P. with this shift captured by the Lévy-Prokee Prokhorov ambiguity set around IP. Such shifts introduce additional complexities in ensuring the robustness of the prediction intervals.
B Estimation of the LP ambiguity set parameters E and p
While our theoretical results apply to any pair (s.p) defining an LP ambiguity set, selecting these parameters in practice is critical to balancing robustness and informativeness. This is particularly important when only limited number of calibration and test samples are available. To address this, we propose a systematic estimation procedure for (c, based empirical data. The key idea is to identify the pair that yields the tightest worst-case conformal prediction set while preserving the desired coverage under distribution shift.
The procedure works as follows. Given two independent batches of calibration scores from the training distribution and a batch of test scores from the shifted distribution Q, we evaluate a grid of candidate E values. For each candidate E;. we estimate the corresponding Ps by computing the LP distance between one batch of calibration scores and the test scores using one-dimensional optimal transport with cost. function [|x yl 2 Ei). This transport problem can be efficiently solved either via the Sinkhorn algorithm or using the standard linear programming formulation, both of which are efficient in one dimension due to the sorted structure of empirical distributions 33 The resulting pair (E,, pt) defines a valid ambiguity set, and compute its associated worst-case quantile using the second calibration batch. Specifically, we apply Corollary 4.2 setting 3, = a + (a - P: 2)/n, so that the prediction set C1-8 enjoys a worst-case coverage guarantee al least 1 a. We then select the pair that yields the smallest such quantile. To preserve statistical validily, the calibration scores used to estimate (E,)) must be disjoint from those used to compute the conformal quantile. This ensures that the ambiguity set is selected independently of the scores used for
13

---

Algorithm 1 Estimation of E and p
Input: Independent empirical calibration score distributions 1(0) ? and empirical test score distribution Qmi a grid of {e:]i= values, with & E N; and target coverage 1 a. Output: Pair (Es, Pi) yiclding the tightest prodiction set with valid coverage.
1: for 1...., do
2: Compute one-dimensional LP distance Pi between P and Qm
3: Set [3 = (a Ps 2)/n
4: Compute worst-case quantile:
WC B1; p(2)) Quant (1 ( Ei
5:
qi == Quant
6: end for
7: return (EL. with minimal qi
Smaller qi leads to smaller robust prediction sets
Estimation (pw0 21, 201
t.p Cetimation with be- 0.02, (.1..1
c.0 Estimation with (a--0 05, u-1 0)
Cowrrge 90
2002
Sile 2.47
1.000
2013
g
2012
Soe 1029
4.08
6.000
0.0 0000000000000 000000000000000 00000
0 00000
00
Figure 5: ImageNet (e, estimation. Each point in the 20-point grid corresponds to a candidate (s, pair, where 1.5) and p is estimated using one-dimensional optimal transport between the empirical calibration and test score distributions, each constructed from 1000 samples. The color scale represents the empirical worst-case quantile associated with each pair, computed on a held-out calibration batch. The optimal (e, p) pair, yielding the smallest quantile, is highlighted in red, with the corresponding empirical coverage and prediction set size annotated. The true corruption paramcters (p, u) used to generate the test distribution also indicated for reference.
calibration, avoiding overfitting and maintaining the coverage guarantce. We present this procedure in Algorithm [
Empirical results ImageNet and MNIST validate the effectiveness of this approach. Figures 5 and 6 display the estimated (s, values over a 20-point grid, visualizing the resulting worst-case quantiles through color shading. The selected pair (highlighted in red) yields the smallest worst-case quantile and corresponds to the tightest robust prediction set. Across both datasets, observe that the data-driven procedure reliably identifies ambiguity set parameters that balance coverage and informativeness, leading to prediction sets that respect the desired 1 coverage level.
Remark B.1 (Sensitivity to 8 and p). Tt. is natural ask how sensitive the method is to misspecification of the shift parameters (c. p). While both influence the prediction set, their effects are asymmetric. The parameter e appears additively in the worst-case quantile and controls the width of the prediction set without affecting coverage. In contrast, P shifts the quantile level and also appears subtractively
the coverage bound from Theorem 4.1 As a result, even small underestimation. of p can significantly impact coverage, whereas modest underest
of e tend to reduce the prediction set size only slightly. In both Figures and 6 we observe trade-off: smaller values are typically associated with larger p estimates, and vice versa. Selecting the pair that minimizes the worst-case quantile provides principled way to balance robustness and efficiency without being overly conservative.
Remark B.2 (Use of test samples for shift estimation). The estimation procodure outlined in this section requires access to test samples in order to estimate the distribution shift. While this may initially seem restrictive, we emphasize that only a relatively small number of test samples is needed to ensure stable estimates of (e) in practice. In our experiments, as few as 500-1000 calibration and test scores are sufficient. to obtain consistent estimates across multiple runs. Nonetheless, one might ask: if test samples are available, why not apply
14

---

2.p Estimatioo (p=é 0.0
Extimation (2-0
(p=0
2012
cccen
Size: 10
3012
1200
W.C. Quariile
TAC esdimalte
Ouamite
000000000
X014
3602
00000
0000000
0000
0000
0000000900
0.0
Figure 6: MNIST ps) estimation. Each point in the 20-point grid corresponds to candidate (e, pair, where e E 1.5) and p is estimated using one-dimensional optimal transport between the empirical calibration and test score distributions, each constructed from 1000 samples. The color scale represents the empirical worst-case quantile associated with each pair, computed on held-out calibration batch. The optimal (s, p) pair, yielding the smallest quantile, is highlighted in red, with the corresponding empirical coverage and prediction set size annotated. The true corruption parameters (p,u) u) used to generate the test distribution are also indicated for reference.
conformal prediction directly to them instead of using a distributionally robust approach? In many applications, this is indeed preferable, standard conformal methods yield valid coverage guarantees under the exchangeability assumption. The purpose of this section, however, is not to recommend distributionally robust conformal prediction over standard conformal prediction in the presence of test data. Rather, it is to demonstrate LP-based ambiguity sets as principled model for capturing both local and global distribution shifts. Estimating these parameters from data allows us to instantiate the LP ambiguity set in conercte, data-driven way. Nonetheless, we acknowledge approach that it requires access to test samples for estimating the shift. Developing estimators that rely solely calibration data-sua the variability-based based method proposed by g is an important direction for future work.
a limitation of our current
C Experimental Setup
All experiments were conducted on single Nvidia A100 GPU with 40GB of RAM. We strictly follow the official GitHub implementations provided by the authors of the referenced methods, except for weighted conformal prediction 37. for which we implemented a neural
network-compatibl version based strictly on the algorithm described in 37
For a given level a and n calibration data points, the prediction sets for each algorithm are constructed from the following quantiles:
1. Standard Conformal Prediction:
Quant (In+ a)]/n;
2. Our method
LP Robust Conformal Prediction (following Corollary 4.2
Quant - 3 + P:Pp) tE, == (a pp 22
3. x² Robust Conformal Prediction 1-
Quant (1=);: an = -fP (1+1/m) 97 -a)).
where is the radius of the ambiguity set, f(x) = (x - 1)3 and gr.p and 9 are defined as:
2 3 (1-B)}
(1-2 (1-2 s p 8 10,1]
(3. =- inf E [0, 35
[
(T) sup {8 E [0, 1] (B) < T), TE [0, 1].
The radius p is estimated using the slab estimation procedure described in 9
15

---

4. Conformal Prediction under Covariate Shift 37
Quant (1- Put+ = Σ 1e (x)) + (z)
where the weights defined by
w(X)) (x) Σド= w(X)) + w(x) 1, 1=)
w(x) Σn= w(X,) + w(x)
Here, (X) dl' test (X) /dP /PPcaltbbbbbb (X) denotes the density ratio between the test and calibration distributions, estimated via separately trained classifier.
5. Randomly Smoothed Conformal Prediction 10
Quant (u-a)22+)/(L+ + 8/0, Pn
1
Sx(X.
Here, 3 denotes the smoothed nonconformity score based on an existing score function 8 [6 under which adversarial noise with |/e]|2 S is propagated with distortion bounded by 8/0.
6. Fine enne-graeed Conformal Prediction J
Quant (0 - a), PH++1)
where 9f and P are defined above. For the f-divergence method, estimate the robustness parameter p using the slab estimation procedure described in
D Proofs
D.1 Proofs of Section 2
D.1.1 Proof of Proposition 2.1
Proof. We start by proving the "3" direction. Let Q belong to the right-hand side in 5 and we want to prove that Q (P). From the hand side in (5), we know that there exists P such that W (P, P) S e and TV(P, Q) S p. Using the definition of the Wa distance in T we note that Wo (P, P) S is equivalent to
inf PET(P.F)
1 - 221 > eddr(e1, 22) S 0.
(16)
2xz
Now, since 1[ 2 > $} is lower semicontinuous function, by 39 Theorem 4.1] we know
that there exists coupling vi2 E [(P, which attains the infimum in (6). Analogously, since 1 f 22l > 0} is lower semicontinuous, the same result ensures that there exists a coupling 723 E T(P. @ which attains the infimum in TV(P, Q) S Since
( = (TI) = P.
where mi 2 8g + 2, and 22 2, x Z2 Zg are the canonical projections, the Gluing
lemma 39 [pp. 11-12) guarantees that there exists a distribution 7123 P x 2) such
that (T12) 1123 1123 712 and (T23) 7123 723- We now construct Y13 (T13) #123, which
16

---

can be easily shown to be coupling of P and Q. Then, we have that
1[12
z3ll
e3dria(21, 23) 2xxx2 1[ zall 2)) 22,
2xz
1[1=1 22 22 23 > =)d2123 (21, 22, 23)
2x2x2
|[|=1 =21| + || =a > e) (21, 22, 23)
22xX2
<
(1f122 - 21l > o) + 1fll22 - 23ll > 0}) d2125(21, 23)
2x2x2
J2xz 11|22 - =2 > (1, 22) + f 111l22 zal > 00)ch2a(22- 23)
< 50+p= -= p:
where the first inequality follows by noticing that the event {|| - 22 + || - 23ll > E} is contained in {ll z2ll >
consequence of the triangle inequality, and the second inequality
E) U (122 zall > 03. Therefore,
inf { 1| z1ll > E)dr((21, 23) S: (ET(P. 2x2
showing that (P, Q) S p, and therefore Q E (P)
We now prove the "C" direction. Let Q E (P)). In what follows, we will construct distribution P such that (P, Se and TV(P, Q) S p, showing that Q belongs to the right-hand side in (5) Since 1[|= =2ll > > is lower semicontinuous. again by [0 Theorem 4.1], we know that there exists a coupling 7" T(P,Q) which attains the infimum in LP. (IP, Q) S p. Therefore, 7* (11= - 22 22|2>> =) = and (1 22 S 1 p. for some S p. We define the event A: := {1|=) - 22ll S E), and its complement A = {[|e1 - 22ll > s),
and denote by 7 la and La= the restrictions of the distribution 7* to A and A respectively, We now construct the distribution P' as follows
P = (TI)+7" Las + 77*
note that 3 = 7 + (Id x Id) ((T))7*1A)) is a coupling between and P.. Then, we immediately have that
inf / 1[122 22ll > e)dry(21, 22) 1 / 1[ 2ell > =)dF(a1, 22) 2
jzzz
1fl - 2211 > E7dr (2)
Zxz
/ Zxz
1 - > E)d(Id x Id) ((T1) (21, 22)
which is clearly equal to zero, showing that (x, P) S e. Moreover,
TV(ē. Q) TV ((T) (T2) (T2) Lx + (#2)
inf 7er (a (m)))
11 12ll 0fdr(21, 22)
2xz
inf /
1(12 - 22|1 > O)d(pp + (Id Id) ((T2)
( ((
Exz
inf Ger( (A (a
1(122 2211 > ()1(77)((1.)))
2xx
inf (er()(m)) (00)er)10))
1|ll - 22l1 > 0)d7(21, 22)
2xx
p.
17

---

Here, the first inequality holds since P7 + (Id x Id)# ((T2) #>aa, with 7 E (() 7 is a coupling of (TT) + (T2)# # lA and 7
(me) de. Moreover, the third equality follows from the fact that
[ 111= =2|l > 0ld ((Id Id) ( lA)) (=1, =2) = 2x2
Finally, the last equality follows from the fact that A { 22 > æ}. This shows that (VP Q) S p, and concludes the proof.
D.1.2 Proof of Corollary 2.2
Proof. Assertion (i) follows from by setting to zero, resulting in P= P. Moreover, asscrtion (ii) follows from 5 by sctting O, resulting in P Q
D.1.3 Proof of Proposition 2.3
Proof. We first prove that any distribution Q Be, admits random variable decomposi- tion Zz as described in (6) Since [ 10 22l1 >} is lower semicontimous,, by 39 Theorem 4.1] there exists coupling 7" [[P, Q)) which attains the infimum in LP Q p. Furthermore, given Zi ~ PP, consider the conditional distribution 72| ~ 72 and define the (random) event Az, == 21 - Zill I c). Morcover, we denote by 72, LAz, the restriction of 12, to the event Azi and by 72, LAz, its normalized version. Similarly, 72, is the normalized version of the restriction to the complement A% We then construct the random variables B, N, and C' follows:
Z Bern (72, (1|=2 - Z1|1 >(>))) E))
Z 1(B - 1B+ 1(B = -0) ZI) and
(17)
Z 1[BB=1) 2 +
(1B=
where Z3IZ1 and Z1 follow the probability distributions 72, LAz, and 72, LAg respectively, Here B, N, C' are dependent with marginals satisfying the properties in the statement of the proposition. We now define 7/2 == (7) + N) = 0 + C1 B 1}, and aim to show that Zg d Zg. Following the construction in 17 conditioning Zo on Z1 yields
Zo|2i (Z1 21) 1( 03 + ClZI 1(Bl = 1}
Z [ 0) 7/ 71 (B) = 1).
Now recall from 17 that the conditional random variable B| follows Bernoulli distribu- tion with paramcter 72, [1I Z,1 Z1 >) 72, (42,). Thus, the distribution of Zg|Zi bo- comes 72, (Az,) 72, A2, +72, (A42, . A22 Moreover, since 72, (Az,)-72) 72,1Az, 72, Azz and 72, (AS,) ) 72, A22 73, LAg, we have that 12 12, Therefore, the distribution of Za is equal to
= [Z2/Zi] (T2) Q
which concludes the proof the first direction.
We now prove the converse: any random variable Zg of the form 66 is distributed according to some distribution Q belonging to the LP ambiguity set Br.p(P)) (P). To show this, we employ
Proposition 2.1 which reduces the problem to showing that Q belongs to the union on the right-hand side in We start by defining the random variable
ZB = (Zi + N)1(B 0} + 2,1(B =1
where Z1, N, and are the same random variables as the definition of Zg in (6. Let denote the distribution of Zz. Then, the pair (Z1. Zs) induces coupling 713 E [(P, (IP.)) P). By
construction we have (1 zal > E) 0, implying that
inf / 1f 2all =)d7(21,.5))
11lla 2all e)dris(21, 23) $
(E ) 2x2
2x2
18

---

Using the definition of the Wo distance in (1) this is equivalent to W (P. P) S E. Next,
we verify that TV(P, Q S p. Note that (Zz. Z2) induces a coupling 732 E (TP, 2 satisfying
22 > 03d (2 22) 0- Prob B 0) 1 Prob B 1) p: Jex21 11123
Exx
where the equality follows from [[<3 ZB = 0) = 0 and the fact that the indicator function is bounded by 1. Therefore,
TV(P,Q) ()= = inf / 11 22| > 02) S
TET(P.0) 2x2
Putting everything together, we have that Ue... {(E) TV Q) sp). which completes the proof.
D.1.4 Proof of Proposition 2.5
Proof. Let Q (P)) We will show that s Q belongs the LP ambiguity set Bke, P).
/ 11/2 - 32 > (2),
(P Q)
inf Fe JRR
7E(exs) (B,00) JxxR 1[ أd > ke (21,
inf
inf { TEI(P.Q) Rx3 /
11 -1g1 > ks)d((s 8) (31, E2)
in
1(((22) s(22) ke)
7EI((.Q) 2x2
S inf
1[| =2|1 > e)d7(21,22)
>ET(P.Q) 2x2
(P, Q)
where the second equality follows from the equality I(s P. s* Q) = (s x s) [(P, Q) (sce
B Lemina 2)), and the inequality follows from the fact that s is k-Lipschitz, i.e., [s(21)
8(2) < k||z1 =2ll.
D.2 Proofs of Section 3
D.2.1 Proof of Proposition 3.4
We prove the proposition in two steps. First, we show that the right-hand side in 10) is upper bound on the B-quantile of any distribution in (P,) Second, we prove that there exists a sequence of distributions Qu E (TP), whose 8-quantiles converge to it.
Proof.
Step 1. We prove, by contradiction, that Quant Q) Quant(B + P:P) + E, for all Q E (P)) Suppose there exists ! satisfying
(e) and Quant (3; Quant(3 + pP)P) +e.
(18)
We will show that this leads
(P, inf yEI(B ②) J
11 - 221 > =)dr(211,2)) > p.
To simplify notation, we define a = Quant PP) and b: = Quant (3 (p) +s. Following 18), must. satisfy (b) < B. Hence, there exists A > 0 such that.
Pe(()) P p A.
19

---

Now, for an arbitrary coupling T(P, Q), we have [+(a)=
(2(
a
00 b
22) -
dy(21, z2)
dy(21, 22) + a
pa L" d
dr 22) -
22)
Jat --x dx(21,
6t
<
1111-22/>00 22)
-00 b+
1 ( 22),
Since the above holds for every 7 E (P, Q), we conclude that
inf TEL(P.)) ())
JRRR [| (1an=s 22) 2+
which contradicts the fact that Q E (P). This proves that Quant (3; Q) < Quant + p; P) + E. for all Q E (P).
Step 2. We construct a sequence of distributions Qn E Be (IP) satisfying, as n + oo,
Quant(3; Q.) + Quant(3 + P) + We define the sequence distributions Qn through their cumulative distribution functions
as
FF(q E), q Quant (3- 1 PP) +e Fo. (q) = 3-
Quant (3 PP) + S < Quant (3 + p; + (19)
Fz E), q Quant (B (8-28 + +
To simplify notation, for the rest of the proof, we define g == Quant 1; IP) + and
yn Quant
p; P) + The intuition behind the construction of Qn is follows: first, Q. is obtained by translating the distribution P to the right by E, and then, the mass between [gh (1) - is moved to the point q We refer to the illustration the left in Figure for a visualization this intuition. From this construction, it is clear that the
LP (IP, (2.) is bounded by
Fo. (q! Fe. Fe. (9. Fe. (Quant(3 (33
1 + p;P) + n
Fe. (Quant 1 :)) +e n
-3-
= Fr Quant
+ p; p: (3 - =
n
showing that the sequence Qn belongs to the LP ambiguity set B (P)). Finally, we prove that the sequence of 3-quantiles of Qn converges to Quant + P)) + e from below. From the construction in
19 we know that the following two properties hold:
Fe. (g) < 8
A 4
< Qn
(2)
Fa. 8,
2
n n
Combining these two inequalities, we have that Q3ant(3;00) (3;Qn) = qn which admits a limit as goes to infinity:
(2)
Quant (B- +e " Quant(3 + p) +e
where the convergence follows from the left-continuity of the quantile function, which follows from the right-continuity of the cumulative distribution function. This concludes the proof.
20

---

D.2.2 Proof of Proposition 3.5
Proof. Similarly to Proposition 3.4 wC prove this in two steps. First, we show that the right-hand side in (11) is lower bound on the coverage at g of any distribution (P))
Second, we prove that there exists a sequence of distributions 2n E (P)) whose coverage at y converges to it.
Step We prove, by contradiction, that Fa(9) 2 Fp(q - - p. for all E (T.) Suppose there cxists Q satisfying
Q E and Fg(9) < Fr 2) p.
(20)
We will show that this leads to
LP. inf 1[121 - 221 > E E (21, 22) > p.
TEr(P, RxR
From the incquality in 20 we know that there exists A > 0 such that
p3(1)) S Felg 5) (0 + A).
Mcanwhile, for any coupling ? E [T,, Q we have Fplg - 2) - -))))
p+
[" dy(21, 22)
d7 22)
19- {
19-5 ["r d2(21, - 1
(1-c)) L d=(21,
22) +
df(21, 22)
-oc
14-
100
11 d7(21122)
[| d2(21,22).
RxB
Taking an infimum over y E I(P, Q), we obtain that the LPe(P, () p, which contradicts the fact that @ E (P). This proves that Fo(9)= 2 Fr(g -) -, for all Q E (P,)
Step 2. We construct a sequence of distributions Qa E B (IP) satisfying, as n + oo,
Fq.. (g) + F2(a ( p.
We define the sequence of distributions Qn through their cumulative distribution functions
as
Felr - E), Frlq - E) p n Quant (Fr(g - E) - p 1: 5
-0p-ppp-- < Quant (Fr(q - 8) - p :P +
Fa, (7)
Quant (Fr(g c)+ :P) +
Felr =).
y 2 Quant (Fr(q- - P)) +E.
To simplify notation, for the rest of the proof, we define gn = (F-(q (P)
and ( = Quant(FF(q E) 4: + + The intuition behind the construction of Q. is as follows: first, Qn is obtained by translating the distribution P to the right by E, and then, the mass between [ ()gg())))))))))))))) is moved to the point g! We refer the illustration on the right in Figure [ for a visualization of this intuition. From this construction, it is clear that
the LP- Q.)) is bounded by
Fo. (2. - Fo. (a!
=Fr (Quani (Fo(y =) - [1) Fr
(Quani e) -p+ -p+ 1:)
Fr +)+ Fr(q c) - -p+ 1
p.
n
21

---

showing that the sequence Qn belongs to the LP ambiguity set (P), Moreover, when n 1o we have that o E Ign qn ) holds, and therefore
Fo. (g) Fela - 8) -p+ 3 E3) 5)
This concludes the proof.
D.3 Proofs of Section 4
D.3.1 Proof of Theorem 41
Proof. By conditioning on ((Xi, K))EE obtain
Prob (Yatl E (x Pn)) K))=
WC (1 a;; PP)Pp)) } Froe Quant 5.
Frees Quant
a + p. P,,) +-)
Fp
Quant (1 p; P.Pr) + + + -
Fe Quant (1 p. p. P.
where the first equality follows from Definition 12 the second equality follows from Propo-
sition 3.4] and the first inequality is a consequence of Proposition 3.5 Now, taking the
expectation with respect {(X,, Y))==. we obtain
Prob (Yatz E C (Xnt1:Pen) 2 Fr (Quant(1-og -a---aappe +...a...PP)))) p.P.))) -p
[n(1 a + p)1 P.
n+1
where the second inoquality follows from the guarantce E Lemma D.3)). This concludes the proof.
(F(Quant(::P..)) (B:P.)) [n07//n 1)
(see g
L
D.3.2 Proof of Corollary 4.2
Proof. Note that [n[1 - 3 p)) - 12p21111 - - is guaranteed by n(1 - 3 + p) z
(n + 1)(1 a + p) + 1, which is further guaranteed by a (a 22)/n. This concludes the proof.
22

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: All claims made in the abstract and introduction accurately reflect the paper's contributions and scope, including both theoretical results and empirical findings. Each technical contribution stated in the introduction is supported by formal theorem or proposition with a complete proof.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The rbstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumpt ions and limitations. No or NA answer this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We acknowledge that our method for estimating the parameters of the LP ambiguity set requires access a few lest samples. While this enables data-driven instantiation of the ambiguity set and remains practical in settings with limited test data, we recognize as limitation. This point discussed explicitly in Remark B.2 of the appendix. Developing approaches that avoid relying on test samples-e-ge e-g-; by leveraging calibration variability as g is valuable direction for future rescarch.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate 'Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results to violations of these assumptions (c.g., independence assumptions, noiseless settings, model well-specification asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a spocch-to-text system might not be used reliably to provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algo- rithms and how they scale with dataset size.
applicable, the hors should discuss possible limitations of their approach to address problems of privacy and fairness.
23

---

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play important role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each thcoretical result, does the paper provide the full set of assump- tions and a complete (and correct) proof?
Answer: [Yes]
Justification: All required assumptions are stated the main text as part of the
propositions, thcorems, and corollarics. Complete proofs for all results are provided in Appendix D Each result is properly numbered and referenced.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: All information necessary reproduce the main experimental re- sults including data generation, evaluation procedures, and implementation de- tails provided in Section 5 of the main text and Appendix C ensure the reproducibility of our results.
These details
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution a dataset and/or model, the authors should describe the steps taken to make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same datasct, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access lo a hosted model (e.g., in the case of large language model), releasing model checkpoint, or other means that are appropriate to the research performed.
24

---

While NeurIPS does not require releasing code, the conference does require all submissions provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the datasct).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e-g-, to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: The algorithm and all experimental code are implemented in Python and will be released on GitHub upon acceptance, together with instructions to reproduce the main results.
Guidelines:
The answer NA mcans that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (ittps://nips
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NewrIPS code and data submis- sion guidelines for more details.
The authors should provide instructions data access and preparation, in- cluding how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and basclines. only a subset of experiments are reproducible, they should state which ones are omitted from the script and why. At submission time, to preserve anonymily, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/ setting/details details
Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: All experimental -ocliings-icludinng data splits, calibration procedures, and evaluation metrics are detailed in Section 5 with additional implementation details provided in Appendix
25

---

Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question:
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We report the empirical distribution of accuracy over 30 random calibration-tesl splits by plotting individual points for each split in Figure 3 which transparently reflects the variability induced by data splitting.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to library funetion, bootstrap, etc.)
The assumptions made should be given (o.g., Normally distributed errors).
It should be clear whet the error bar is the slandard deviation or the standard error of the mean.
is OK report 1-sigma crror bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (c.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The computational resources, including hardware type
are described in Appendix C Runtime and memory usage were not tracked, the experiments are lightweight and reproducible on standard hardware.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
26

---

9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethies http::///aarrpp.ccc//aaaa////aaa
Answer: [Yes]
Justification:
This rescarch adheres to the NeurIPS Code of Ethics. It does not
involve human subjects, sensitive data, or real-world deployments with potential societal impact. All results are reproducible, and the code will be released upon acceplance.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negalive societal impacts of the work performed?
Answer: [NA]
Justification: This work is foundational and does not involve specific applications or deployments. While robust prediction under distribution shift can support decision- making in domains such as healthcare autonomous systems, the paper does not target any particular use case and therefore does not entail direct societal impacts.
Guidelines:
The answer NA means that there is societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include polential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance) fairness consid- erations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, optimizing neural networks could enable people to train models that generate Deepfakes faster.
not needed to point out that generic algorithm for
The authors should consider possible harms that could arise when the technology being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional
unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MT.).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
27

---

Justification:
This work does not involve the release of pretrained models, large-scale datasets, or tools that pose significant risk of misuse. The research is theoretical and algorithmic in nature, and all released code is for reproducibility of the experiments. Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.
12. Licenses for existing assets
Question: Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All external assets used in this work-includingg code and datasets are properly cited in the paper, and their licenses have been respected in accordance with the stated terms of use (see Appendix Ch. No proprietary or restricted-access data was used.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from particular source (e.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswithcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 8 creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We introduce new code for our method and experiments, which is documented and will be made publicly available upon acceptanco. The code includes clear instructions for reproducing all results and is structured to facilitate case of use.
Guidelines:
The answer NA mcans that the paper does not release new assets. Researchers should communicate the details of the datasel/codk mode/ as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
28

---

The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question:
For crowdsourcing experiments and research with human subjects, does the paper include the full text instructions given to participants and screenshots, if applicable, well as details about compensation (if any)?
Answer: [NA]
Justification: This work does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental matorial is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for rescarch with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country institution) were obtained?
Answer: [NA]
Justification: This work does not involve human subjects crowdsourcing and therefore does not require IRB approval.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedurcs for this may vary significantly between insti- tutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, non-standard component of the core methods in this rescarch? Note that the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: This research does not involve the use of large language models (LLMs) any part of the core methodology. Any LLM usage was limited to minor writing assistance and had no impact on the scientific content or originality of the work.
Guidelines:
29

---

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should not be described.
30


Reviews:
[{"review_id": "Fxghg4oRth", "reviewer": "Reviewer_odWU", "summary": "This work considers the problem of conformal prediction under distribution shift.  Formally, they consider the problem where the training/calibration data is drawn i.i.d. from some distribution $\\mathbb{P}$, and test data is drawn from another distribution $\\mathbb{Q}$.  Formally they give a method such that if the $\\varepsilon$-Levy-Pokhorov distance between the distribution $\\mathbb{P}$ and the distribution $\\mathbb{Q}$ is less than $\\rho$, then the conformal predictor that is trained on $\\mathbb{P}$ will exhibit coverage on $\\mathbb{Q}$.  The $\\varepsilon$-Levy-Pokhorov distance between $\\mathbb{P}$ and $\\mathbb{Q}$ is the minimum TV-distance between $\\mathbb{Q}$ and any distribution $\\widetilde{\\mathbb{P}}$ that has $\\infty$-Wasserstein distance $\\le \\varepsilon$ to $\\mathbb{P}$.  Thier predictor works by taking a larger prediction set, which expands the quantiles of the score function covered by $\\rho$ and expands in the original space by $\\varepsilon$.", "questions": "1. This notion of robustness is defined for data that are drawn i.i.d., with the training drawn from distrbution $\\mathbb{P}$ and the test data drawn from  a nearby distribution $\\mathbb{Q}$.  Usually, in the standard conformal prediction model, the data is exchangeable, but not necessarily drawn i.i.d. from some unknown distribution.  Are there similar notions that can be defined for exchangeable data?\n\n2. It is not immediately clear whether the exchangeability assumption of standard conformal prediction is stronger or weaker than the assumption that the Levy-Pokhorov distance between $\\mathbb{P}$ and $\\mathbb{Q}$ is bounded.  Is there a natural sense in which it is a stronger assumption?  That is, suppose you had $N$ exchangeable random variables $X_1, \\dots, X_N$.  Is the Levy-Pokhorov distance between the distributions $\\text{Unif}(X_1, X_{N/2})$ and $\\text{Unif}(X_{N/2 + 1}, X_{N})$ small with high probability?  It would be helpful to provide an in depth comparison of these two models. \n\n3. Could you interpret the dependence of the efficiency of your method on $\\varepsilon$ and $\\rho$?  What kinds of values of $\\rho$ and $\\varepsilon$ can you expect nontrivial results for?  As a starting point, to get a non-trivial result, one needs to assume that $\\rho < \\alpha$.  In what settings can we expect this to hold.  Suppose $\\alpha = 0.1$, as it is in many applications of interest.  Then $\\rho < 0.1$ means that you cannot consider any $\\mathbb{Q}$ that has TV-distance more than $0.1$ from $\\mathbb{P}$.  Is this a strong assumption?\n\n4. Your method increases the size of the prediction set in a conservative way (hence the concerns above about efficiency).  Can you show that this is necessary to achieve this definition of robustness?  That is, can you show that, out of all possible conformal methods that achieve robustness in the Levy-Pokhorov sense, yours is the most efficient possible?  I think perhaps you can, if so it would be nice to include this as a formal statement.  \n\n- Line 152: Minor typo: should $W_\\infty (\\mathbb{P}, \\mathbb{Q}) \\le \\rho$ have $\\varepsilon$ instead of $\\rho$?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Weaknesses** \\\nThe technical result is extremely simple.  It is not surprising that increasing the coverage in the training stage makes coverage more robust in the testing stage.  It would nice to have some more theoretical investigation into this method: how much can it affect the efficiency of the method?  Is this the most efficient method possible that can achieve a distributionally robust guarantee?  (Please see questions for authors)\n\nAlso, it is not immediately clear to me how strong of an assumption bounding the Levy-Pokhorov distance is.  Is it possible to get non-trivial guarantees for moderate to large Levy-Pokhorov distances?  (See questions for more details)\n\n**Strengths** \\\nThe problem is very well-motivated.  A major drawback of standard conformal methods is that they assume exchangeability between the training examples and the test examples.  But this is a drawback in deploying this to real world settings, which almost always exhibit drift.  Previous works model specific ways in which the distribution can change, and design methods that fit to that.  However, this strategy requires knowledge of the drift, which is counter to the spirit of distribution-free guarantees.  The modeling of the problem in the framework of distributionally robust optimization is very elegant.\n\nThe numerical experiments are also compelling.  Since the method essentially just increases the coverage on the training set in a conservative way, the natural question is how it affects the efficiency of the method.  The numerical methods suggest that this method does not make the prediction sets too much bigger in natural settings.", "was_revised": true, "final_justification": "I appreciate the authors thoughtful and thorough responses to my concerns.  My main concern remains, that while the observation that it is possible to raise the conformal score threshold in a way that corresponds to Levy-Pokhorov distance is very elegant and clean, it is very simple, and does not require a lot of technical work to prove.  Thus I maintain my score at 4: weak accept.", "rebuttal": "We are very grateful for your time and effort in reviewing our paper. Thank you for the positive assessment of our work. Below, we provide an answer to each of your questions (which also provide an answer to the weaknesses).\n\n------\n\n**Question 1.** Thank you for this insightful question. The standard split conformal prediction framework relies on the assumption that the $n$ calibration scores and the $(n+1)$‑th test score are exchangeable, which in particular requires that they are drawn from the same distribution (though they may be correlated). Under LP distribution shifts, however, the distribution of the test score necessarily differs from that of the calibration scores, violating exchangeability and rendering the standard method inapplicable. In order to address this more general setting, we require the assumption that the calibration scores are i.i.d. from the calibration distribution. This assumption seems technically essential, as it enables us to control the discrepancy between the empirical calibration distribution and its true, unknown counterpart, a property that is used explicitly in the final step of the proof of Theorem 4.1.\n\n-----\n\n**Question 2.** Following the answer provided to the first question, the assumption of exchangeability between the $n$ calibration scores and the $(n+1)$‑th test score is strictly stronger than the LP distribution shift assumption that we impose (indeed, if the scores are exchangeable, then LP distance between the calibration and test distributions must be zero). In fact, since the test distribution and the calibration distribution are different, the standard proof reasoning based on rank statistics fails in the distribution shift case. We hope that this answers the reviewer's question, but are happy to provide more details in case our answer is not fully clear.\n\n----\n\n**Question 3.** Thank you for this sharp question. As noted in Remark A.1 (Appendix A of the submitted version), we discussed the sensitivity of the coverage guarantee in Theorem 4.1 to the parameters $\\varepsilon$ and $\\rho$. However, we acknowledge that this remark does not directly address the statement in question 3 that “to get a non‑trivial result, one needs to assume that $\\rho < \\alpha$.” We emphasize that $\\rho < \\alpha$ is not an assumption required by any of our technical results. Rather, it excludes the degenerate scenario in which the confidence interval covers the entire real line. Indeed, if $\\rho \\geq \\alpha$, one can construct a sequence of distributions whose $(1-\\alpha)$‑quantiles diverge to infinity (see Remark 3.3 in the submitted version). Thus, the condition $\\rho < \\alpha$ is not a limitation of our analysis, but a reflection of the phenomenon itself.\n\nWe also note that the condition $\\rho < \\alpha$ does not imply that the total variation distance is less than $\\alpha$. Such an implication would hold only in the special case $\\varepsilon = 0$, since any local perturbation also contributes to the total variation distance.\n\n-----\n\n**Question 4.** Thank you for the opportunity to clarify this point. Among all conformal methods that achieve robustness in the LP sense, we claim that ours attains the highest possible efficiency. This follows directly from the exact (i.e., approximation-free) results in Propositions 3.4 and 3.5, which characterize the worst‑case quantile and coverage, respectively. These exact characterizations imply that the inequality in the first display equation of the proof of Theorem 4.1 becomes an equality in the worst case. The only source of approximation in Theorem 4.1 arises in its final step, where we relate the $\\beta$‑quantiles of the empirical calibration distribution to those of the true calibration distribution. This approximation appears technically unavoidable in the absence of full knowledge of the true calibration distribution."}, {"review_id": "Ns0bNeL631", "reviewer": "Reviewer_Ro7g", "summary": "This work studies the problem of distribution shift in conformal prediction and presents a novel framework that leverages the Lévy-Prokhorov (LP) metric to provide robustness guarantees. The authors derive theoretical results and validate the approach through empirical studies involving synthetic perturbations and a real dataset.", "questions": "1. Can the authors elaborate on how the robustness parameters are interpreted or estimated from data in a real application setting?\n2. The theoretical results in Theorem 4.1 rely on LP-bounded shifts. In practice, how restrictive is this assumption? Is it easy to verify or approximate?", "limitations": "Yes, the limitations of the paper is provided in Appendix.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The topic is highly relevant to real-world applications, where test-time distribution shifts are often inevitable.\n- The incorporation of the LP metric into conformal prediction is conceptually interesting and mathematically sound. It provides a unified view of local and global shift scenarios.\n- The experiments are generally consistent with the theoretical findings and demonstrate the practical utility of the method.\n\nWeaknesses:\n- While the theoretical framework accommodates a broad class of shifts, the empirical validation remains limited in scope. Specifically, the experiments mainly consider artificial perturbations, and more extensive evaluation on multiple real-world benchmarks would strengthen the practical relevance of the work.\n- The connection between theoretical robustness parameters and their empirical estimation remains unclear, and the method may be difficult to apply in practice without further guidance.", "was_revised": true, "final_justification": "Thank you for your efforts to address my concerns! I maintain my positive score of 4.", "rebuttal": "We are very grateful for your time and effort in reviewing our paper. Thank you for the positive assessment of our work. Below, we first provide answers to your questions and then to the highlighted weaknesses.\n\n----\n**Question 1.** Thank you for highlighting this important aspect. Intuitively, the local perturbation parameter $\\epsilon$ captures small, dense, and continuous shifts (e.g., Gaussian noise or input drift) that alter the distribution’s shape, while the global perturbation parameter $\\rho$ accounts for sparse or structural changes (e.g., label flips, adversarial attacks, or support mismatches). This decomposition is formally described in Propositions 2.1 and 2.3.\n\nFor estimation in practice, Appendix B (Algorithm 1) describes an efficient two‑step procedure using a small number of calibration and test samples:\n\n(i) Grid search over $\\epsilon$: For each candidate $\\epsilon$, estimate $\\rho$ as the LP distance between independent calibration and test score distributions, computed via 1D optimal transport with cost $\\mathbb{1}(|x - y| \\geq \\epsilon)$. Each $(\\epsilon_i, \\rho_i)$ defines a valid LP ambiguity set satisfying empirical coverage.\n\n(ii) Selection for efficiency: Among candidate pairs, choose the one minimizing the worst‑case $(1-\\alpha)$-quantile (Corollary 4.2) to balance robustness and set size. Calibration scores for parameter estimation are kept independent from those used to compute the conformal quantile to ensure validity.\n\nThis yields a natural trade-off: smaller $\\epsilon$ typically corresponds to larger $\\rho$, and vice versa. As shown in Figures 5 and 6, Pareto-optimal pairs (small $\\epsilon$ with moderate $\\rho$) offer the best balance between robustness and efficiency. Importantly, $\\epsilon$ contributes additively to the prediction set size, whereas $\\rho$ shifts the quantile level itself (see the worst-case quantile characterization in Proposition 3.4) and therefore has a more pronounced effect on coverage. Consequently, underestimating $\\rho$ poses a greater risk (potentially violating coverage) while underestimating $\\epsilon$ primarily impacts efficiency.\n\n----\n\n**Question 2.** The LP‑bounded shift model is intentionally designed to be flexible: it only constrains most of the probability mass to remain within a small tolerance while allowing a limited fraction to be displaced arbitrarily. This enables it to capture both smooth, small‑magnitude perturbations and large, sparse deviations, covering a wide range of realistic shifts: from gradual covariate drift to adversarial perturbations and isolated but severe outliers. Many common real‑world shifts (e.g., bounded $\\ell_{\\infty}$ attacks, mild input drift, or mixed noise‑plus-outlier scenarios) fall within this class.\n\nWhile exact verification would require the full test distribution, the parameters $(\\varepsilon,\\rho)$ can be approximated reliably from a modest number of calibration and test samples. In our experiments, we estimate them by comparing the empirical score distributions via a simple 1D optimal transport calculation. Conservative estimates ensure that the method’s coverage guarantee remains valid, and the results show that these estimates are accurate enough to retain efficiency. \n\nWe agree that parameter estimation is critical and have developed an explicit procedure (Algorithm 1 in Appendix B). In the revision, we will move this discussion to the main text (Section 4) for greater visibility, add further intuition and empirical support, and release our implementation to facilitate reproducibility.\n\n----\n\n**Weaknesses.** Answer to Weaknesses: We agree that broader empirical validation would further strengthen the paper. While synthetic shifts are useful for isolating and visualizing the effects of local and global perturbations, we also evaluate on iWildCam, a benchmark dataset with naturally occurring covariate shift. Our estimated parameters align closely with the Pareto frontier in Figure 4 and yield tight prediction sets, supporting the method’s practical utility.\n\nIn response to your suggestion, we are adding experiments on the PovertyMap dataset introduced by Yeh et al. (“Using publicly available satellite imagery and deep learning to understand economic well-being in Africa”). This dataset contains multispectral Landsat imagery paired with a continuous-valued asset wealth index, with natural distribution shifts arising from geographic (cross-country) and urban-rural differences. We have set up access via Google Earth Engine and are awaiting approval; results will be appended once available.\n\nThis setting is analogous to iWildCam but with continuous labels, for which we take the residual $s(X_n, Y_n) = |Y_n - f(X_n)|$ as the score. Based on our theoretical framework, we expect LP-Robust Conformal to perform well in this domain."}, {"review_id": "EEQIQCqmoD", "reviewer": "Reviewer_Y1BD", "summary": "This paper proposes a robust conformal prediction framework under Lévy-Prokhorov (LP) distribution shifts, supported by rigorous theoretical guarantees and various real-data experiments results.", "questions": "1. Beyond the mathematical formulation in Proposition 2.3, how can we intuitively understand the difference between local perturbation and global perturbation?\n2. In Figure 1, what do the blue and red lines represent, respectively? The caption should provide more detail to avoid ambiguity.\n3. About the experiments results, how to estimate the $\\varepsilon$ and $\\rho$ in Figure 6?", "limitations": "Yes, limitations are discussed in the Appendix of the paper.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths: \n- The problem addressed is important and well-motivated.\n- The use of the Lévy-Prokhorov pseudo-metric to model distributional shifts is novel and well-defined, with two robustness parameters, local parameter $\\varepsilon$ and global parameter $\\rho$.\n- The authors conduct well-designed experiments to demonstrate the effectiveness of their method.  \n\nWeaknesses:\n- In practice, the nature of distribution shifts is typically unknown. However, the experimental evaluation is limited to synthetic local and global perturbations and includes only one real-world dataset, without exploring a broader range of naturally occurring distribution shifts. This limitation raises concerns about the practical robustness and generalizability of the proposed method in real-world applications.", "was_revised": true, "final_justification": "Thanks for the responses. I keep my positive score for this paper.", "rebuttal": "We are very grateful for your time and effort in reviewing our paper. Thank you for the positive assessment of our work. Below, we first provide answers to your questions and then to the highlighted weaknesses.\n\n----\n\n**Questions 1+3.** Thank you for these questions. Intuitively, the local perturbation parameter $\\epsilon$ captures small, dense, and continuous shifts (such as Gaussian noise, mild input drift, or other shape-preserving distortions), while the global perturbation parameter $\\rho$ models sparse or structural changes (such as label flips, adversarial attacks, or support mismatches). This decomposition, formalized in Propositions 2.1 and 2.3, allows the LP-bounded shift model to represent both small-magnitude changes and large, isolated deviations. Many realistic shifts (bounded $\\ell_{\\infty}$ attacks, mild covariate drift, or mixed noise-plus-outlier scenarios) fall within this class.\n\nIn practice, $(\\epsilon,\\rho)$ can be approximated from a modest number of calibration and test samples using the procedure in Appendix B (Algorithm 1, which we will move to the main text for clarity). The method proceeds in two steps: \n\n(i) grid search over $\\epsilon$, and for each candidate $\\epsilon$ estimate $\\rho$ as the Lévy-Prokhorov (Wasserstein-$\\infty$) distance between independent calibration and test score distributions, computed via 1D optimal transport with cost $\\mathbb{1}(|x-y| \\geq \\epsilon)$; \n\n(ii) among candidate pairs, select the one minimizing the worst‑case $(1-\\alpha)$-quantile (Corollary 4.2) to balance robustness and prediction-set size. Calibration scores used for parameter estimation are kept independent from those used to compute the conformal quantile to preserve validity.\n\nThis process reveals a natural trade-off: smaller $\\epsilon$ typically corresponds to larger $\\rho$, and vice versa. As shown in Figures 5 and 6, Pareto-optimal pairs (small $\\epsilon$ with moderate $\\rho$) achieve the best robustness-efficiency balance. The parameter $\\epsilon$ affects set size additively, while $\\rho$ shifts the quantile level itself (Proposition 3.4), making underestimation of $\\rho$ riskier for coverage than underestimating $\\epsilon$.\n\n----\n\n**Question 2.** Thank you for pointing this out. In Figure 1, the red curve represents the nominal score distribution $F_{\\mathbb{P}}$, while the blue curve shows an adversarial distribution in the LP ambiguity set $\\mathbb{B}_{\\epsilon,\\rho}(\\mathbb{P})$, chosen to either maximize the $\\beta$-quantile (left) or minimize coverage at a fixed threshold $q$ (right). The figure illustrates exactly the construction of the worst‑case distribution used in the proofs of Propositions 3.4 and 3.5, where it is cited for intuition. We will clarify both the meaning of the curves and this connection in the caption.\n\n----\n\n**Weakness.** We agree that broader empirical validation would further strengthen the paper. While synthetic shifts are useful for isolating and visualizing the effects of local and global perturbations, we also evaluate on iWildCam, a benchmark dataset with naturally occurring covariate shift. Our estimated parameters align closely with the Pareto frontier in Figure 4 and yield tight prediction sets, supporting the method’s practical utility.\n\nIn response to your suggestion, we are adding experiments on the PovertyMap dataset introduced by Yeh et al. (“Using publicly available satellite imagery and deep learning to understand economic well-being in Africa”). This dataset contains multispectral Landsat imagery paired with a continuous-valued asset wealth index, with natural distribution shifts arising from geographic (cross-country) and urban-rural differences. We have set up access via Google Earth Engine and are awaiting approval; results will be appended once available.\n\nThis setting is analogous to iWildCam but with continuous labels, for which we take the residual $s(X_n, Y_n) = |Y_n - f(X_n)|$ as the score. Based on our theoretical framework, we expect LP-Robust Conformal to perform well in this domain."}, {"review_id": "kkUWGfziw0", "reviewer": "Reviewer_YGEH", "summary": "This paper proposes a new framework for robust conformal prediction under distribution shift using Lévy–Prokhorov (LP) ambiguity sets, which capture both local and global perturbations. It shows that the LP ambiguity set can be effectively propagated through scoring functions, simplifying high-dimensional distribution shifts into tractable one-dimensional shifts. Closed-form expressions for worst-case quantiles and coverage are derived. The paper also introduces a data-driven method to estimate LP parameters and demonstrates the efficacy of the proposed method on benchmark datasets, including MNIST, ImageNet, and iWildCam.", "questions": "- See **Weaknesses**.\n- In **Proposition 2.3**, better avoid using $n$ as it stands for sample size later (Section 4) in the paper.\n- Second line of **Definition 3.2**: a typo in \"$\\mathbb{Q}$.or $q\\in\\mathbb{R}$\"?\n- In Figures 5 and 6 in the Appendix, why is the lower bound for the candidate values of $\\varepsilon$ set at 0.5? I wonder what would happen if $\\varepsilon$ is less than 0.5.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n- This paper extends existing works using ambiguity sets, including $f$-divergence ambiguity sets.\n- The theoretical contributions are significant, with rigorous derivations for worst-case quantiles and coverage under LP ambiguity.\n\n\n**Weaknesses**\n- The connection to the work of Cauchois et al. [9] should be discussed in more depth. While the paper does cite [9] and acknowledges some high-level similarities, the key differences in ideas, methodology, theoretical formulation, and proof techniques are not fully discussed.\n- Estimation of the LP ambiguity set parameters $\\varepsilon$ and $\\rho$ is critical for the practical implementation of the proposed approach, and thus I would suggest moving it to the main paper.\n- Although the LP framework is highly general, experiments are limited to classification tasks. Can you add more experiments on regression settings?", "was_revised": true, "final_justification": "Most of my concerns have been addressed and I would like to keep my positive score.", "rebuttal": "We are very grateful for your time and effort in reviewing our paper. Thank you for the positive assessment of our work. Below, we first provide answers to your questions and then to the highlighted weaknesses.\n\n----\n\n**Question 1.** We proceed to answer to the three weaknesses.\n\n*Weakness 1.* We appreciate the suggestion to elaborate on the connection with [9]. Our work was indeed inspired by their formulation (as noted in the introduction) and, in particular, by their treatment of the fact that only samples from the calibration distribution are available. This influence is reflected in Theorem 4.1, whose proof adapts ideas from their Theorem 1 to our setting.\n\nThat said, there are key differences in ideas, methodology, theoretical formulation, and proof techniques. In terms of ideas, we replace the $f$‑divergence ambiguity sets in [9] with Lévy–Prokhorov (LP) sets. $f$‑divergences cannot accommodate distribution shifts with disjoint support, which occur in practice under adversarial perturbations, label shifts, or severe data contamination. The LP framework addresses this limitation and offers a natural decomposition into local and global perturbations, enabling separate reasoning about small‑magnitude shifts and large‑magnitude deviations.\n\nMethodologically, our shift model is defined via an optimal transport (OT) discrepancy, placing it in a different mathematical class than $f$‑divergences. OT‑based sets have a distinct geometry and constraint structure, requiring tools from OT theory rather than the convex‑analytic duality used for $f$‑divergences. This also shapes the proof strategy: in Propositions 3.4 and 3.5, we explicitly construct worst‑case distributions within LP sets and derive closed‑form characterizations of worst‑case quantiles and coverage guarantees. These arguments, tailored to the LP geometry, differ fundamentally from the duality‑based derivations in [9] and, we believe, are more broadly applicable to other OT‑driven robustness models.\n\n*Weakness 2.* Thank you for the suggestion. Space permitting, we will try to move as much of the discussion on the estimation of $(\\varepsilon,\\rho)$ as possible to the main text.\n\n*Weakness 3.* We agree that experiments on regression settings would further strengthen the paper. In response to your suggestion, we are adding experiments on the PovertyMap dataset introduced by Yeh et al. (“Using publicly available satellite imagery and deep learning to understand economic well-being in Africa”). This dataset contains multispectral Landsat imagery paired with a continuous-valued asset wealth index, with natural distribution shifts arising from geographic (cross-country) and urban–rural differences. We have set up access via Google Earth Engine and are awaiting approval; results will be appended once available.\n\nThis setting is analogous to iWildCam but with continuous labels, for which we take the residual $s(X_n, Y_n) = |Y_n - f(X_n)|$ as the score. Based on our theoretical framework, we expect LP-Robust Conformal to perform well in this domain. \n\n\n\n-------------------\n\n\n\n**Questions 2+3.** Thank you for the suggestion to use a different letter for the local perturbation in Proposition 2.3. We have double‑checked and confirm that Definition 3.2 is correct as stated and does not contain a typo.\n\n\n\n-------------------\n\n\n\n**Question 4.** Thank you for the question. For smaller values of $\\epsilon$, we observe a sharp increase in the corresponding $\\rho$. This still yields valid coverage (consistent with the trade‑off described in Remark A.1, where smaller $\\epsilon$ typically corresponds to larger $\\rho$). However, because $\\rho$ shifts the quantile level directly (see Proposition 3.4), even moderate increases in $\\rho$ have a stronger effect on coverage than changes in $\\epsilon$. As a result, parameter pairs with very small $\\epsilon$ and large $\\rho$ tend to be less efficient and are not selected under the coverage‑efficiency trade‑off in Corollary 4.2.\n\nIn the plots, we truncated the $\\epsilon$ search range to $[0.5,\\cdot]$ purely for illustration and readability, since smaller values would make $\\rho$ spike sharply and obscure the relevant Pareto‑optimal region. In the revised version, we can enlarge the window and include an explanation of this phenomenon for completeness."}]

Author rebuttal:

Meta-review: This submission proposes a robust extension of conformal prediction under distribution shifts using Lévy–Prokhorov (LP) ambiguity sets, which simultaneously capture local and global perturbations. The framework is natural, connects well to optimal transport, and yields closed-form characterizations of worst-case quantiles and coverage. While the main technical observation is relatively simple and may not push the theoretical frontier substantially, it is clean, well-motivated, and addresses a practical limitation of standard conformal methods. The experimental evaluation, though somewhat limited in scope, aligns with the theory and provides evidence that the approach maintains good coverage without excessive loss in efficiency. I lean toward acceptance.
Decision: Accept (poster)