Rectifying Conformity Scores for Better Conditional Coverage
Authors: Vincent Plassier
Alexander Fishkov
Victor Dheur
Mohsen Guizani
Souhaib Ben Taieb
Maxim Panov
Eric Moulines
Venue: ICML 2025 poster (2025)
Keywords: conformal prediction
uncertainty quantification
confidence sets
Primary area: theory->probabilistic_methods
TL;DR: We introduce a new conformal prediction method that adjusts conformity scores to improve conditional coverage

Abstract: We present a new method for generating confidence sets within the split conformal prediction framework. Our method performs a trainable transformation of any given conformity score to improve conditional coverage while ensuring exact marginal coverage. The transformation is based on an estimate of the conditional quantile of conformity scores. The resulting method is particularly beneficial for constructing adaptive confidence sets in multi-output problems where standard conformal quantile regression approaches have limited applicability. We develop a theoretical bound that captures the influence of the accuracy of the quantile estimate on the approximate conditional validity, unlike classical bounds for conformal prediction methods that only offer marginal coverage. We experimentally show that our method is highly adaptive to the local data structure and outperforms existing methods in terms of conditional coverage, improving the reliability of statistical inference in various applications.

Rectifying Conformity Scores for Better Conditional Coverage
Vincent Plassier"
Alexander Fishkov 223 Victor Dheur "4 Mohsen 2uizani² Souhaib Ben Taieb 24
Maxim Panoy? Eric Moulines
Abstract
Classical CP approaches guarantee marginal validity but fail ensure the more desirable property of conditional validity, which customizes prediction regions specific co- variates. Prior studies have shown constructing meaningful prediction regions with exact conditional validity is infea- sible without additional distributional assumptions (Vovk) 2012 Lei & Wasserman 2014] Foygel Barber et al.) 2021
We present new method for generating confi- dence sets within the split conformal prediction framework. Our method performs a trainable transformation of any given conformity score to improve conditional coverage while ensuring ex- act marginal coverage. The transformation based on an estimate of the conditional quantile of conformity scores. The resulting method is par- ticularly beneficial for constructing adaptive con- fidence sets in multi-output problems where stan- dard conformal quantile regression approaches have limited applicability. We develop a theo- retical bound that captures the influence of the accuracy of the quantile estimate on the approxi- mate conditional validity, unlike classical bounds for conformal prediction methods that only of- fer marginal coverage. We experimentally show that our method is highly adaptive to the local data structure and outperforms existing methods in terms of conditional coverage, improving the reliability of statistical inference various appli- cations.
Consequently, current research emphasizes developing con- formal methods that maintain marginal validity and achieve
approximate conditional validity (Colombo) 2024 Gibbs
et al. 2025).
A typical relaxation of exact conditional coverage in earlier work involves group-conditional guarantees (Jung et al. 2023 Ding et al. 2024 which provide coverage guarantees For predefined set of groups. Another branch of work partitions the covariate space X' into multiple regions and applies CP within each set in the partition (LeRoy & Zhao 2021 Alaa et al. 2023 Kiyani et al. 20224). However, such partitioning based on the calibration set often leads to overly
large prediction regions (Bian & Barber 2023 Plassier et al. 2024).
An alternative approach weights the empirical cumulative distribution function with "localizer" function that quan- tifies the similarity between calibration points and the test sample (Guan) 2023). Although this method improves the localization of predictions, it has significant limitations, es- pecially in high- -igh-dimensional covariate spaces.
1. Introduction
The widespread deployment of AI models emphasizes the need for reliable uncertainty quantification (Gruber al.]
Finally, several methods focus on the transformation of con- formity scores (Han et al., 2022 Dey et al., 2022 Izbicki ct al. 2022 Deutschmann et al. 2023 Dheur al.. 2024
2023). Although highly flexible in capturing complex statis- tical dependencies, these models can produce unreliable or overly confident predictions (Nalisnick et al. 2018). Con- formal prediction (CP: Vovk et al. (2005); ; Shafer & Vovk (2008) offers a robust, distribution- free framework for pre- dictions with finite-sample validity guarantees (Angelopou- los et
Colombo 2024). These techniques adjust conformity scores to better approximate the conditional coverage. However, they usually require estimating the conditional distribution of conformity scores, which is both computationally inten- sive and difficult to perform accurately.
2023 2024).
'quual contribution 'Lagrange Mathematics and Computing Re- search Center PenterrMMhann 'Mohamed bin Zayed University of Artificial Intelli- gence 'Skolkovo Institute of Science and Technology "University of Mons ÉÉcole Polytechnique. Correspondence to: Maxim Panov <maxim. panov @ ac.a>>.
In this paper, we propose novel CP method, Rectified Conformal Prediction (RCP), extending normalized noncon- formity scores; see, c.g., (Papadopoulos et al., 2008 Pa padopoulos & Haralambous) 201 RCP aims to enhance conditional coverage while preserving exact marginal cov- erage guarantees. By constructing a new conformity score whose quantile at a given coverage level is independent
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Rectifying Conformity Scores for Better Conditional Coverage
of covariates, RCP achieves both marginal and improved conditional validity.
that allows generating reliable prediction regions, even when the predictive model is misspecified or inaccurate.
A significant benefit of RCP is its capacity to generate predic- tion sets without fully modeling the conditional distribution of conformity scores. Instead, RCP concentrates on quan- tile regression to ensure approximate conditional coverage. The main contributions of this work can be summarized as follows.
Split conformal prediction (SCP), Given a possibly mis- specified predictive model g(x), for any input x E X, SCP (Papadopoulos al. 2002) generates prediction set C. (ar) at user-specified confidence level (0, 1) with marginal validity (Papadopoulos 2008):
P(Y E Cx())) 21-a.
(1)
We introduce Rectified Conformal Prediction (RCP), new conformal method designed to enhance condi- tional validity by refining conformity scores (see Sec- The proposed method avoids the need to estimate the full conditional distribution of multi- variate response, relying instead estimating only the conditional quantile of univariate conformity score.
To do so, relies on a conformity score function, V: X'x
y + R. assigning larger value to worse agreement between
g(X) and Y. Let {(x,, Y=)]=== be a calibration set, with
tions and 4
X' C R2 and y C Rd. SCP generates a prediction set Ca (x)
by computing an empirical quantile of the conformity scores V (Xx, Yk), k == 1, I,..., R2:
(a()) = {v: V( v) Q1-a
(Ei=1
6vixp.Ye) mil
We provide a theoretical lower bound on the condi- tional coverage of the prediction sets generated by RCP (see Section (). This conditional coverage is explicitly governed by the approximation error in estimating the conditional quantile of the conformity score distribu- tion.
n
where d, is the Dirac mass at v, and Q1-a (P) denotes the (1 qu-anniill for any distribution P on IR.
Towards conditional validity of CP methods. In many applications, conditional validity is a natural requirement, i.e., for all x E X,
We evaluate our method on several benchmark datasets and compare against state-of-the-ar alternatives[] (see Section 7 Our results demonstrate improved per- formance, particularly in terms of conditional coverage metries such as worst slab coverage (Romano et al. 2020 and conditional coverage error (Dheur et al. 2024
P(Y E Cx(X) X = x) 21
(2)
Conditional coverage 2 is stronger and implies marginal coverage ①I. While classical conformal methods provide
marginal validity ① they do not ensure conditional validity,
Let us denote the conditional distribution Pvix=r with V being a shorthand for V(X,)) The following oracle pre- diction
2. Background
Ca(x)= - (2 E y: V(z, y) s (Pvx(() (3)
Consider a regression problem that aims to estimate a d- dimensional response vector y y = Rd based on a feature vector E X C IH' to predict. We denote by Px, the joint distribution of (X,Y) over X x y.
trivially satisfies conditional coverage by the definition of conditional quantile (Pvxx=2) However, exact conditional validity is not achievable within conformal pre- diction framework (Vovk] 2012 Lei & Wasserman 2014 Foygel Barber et al., 2021). In what follows we will present new conformal prediction method that will achieve approx- imate conditional validity while satisfying exact marginal guarantees.
Construction of prediction regions for regression problems is often based on distributional regression that focuses on fully characterizing the conditional distribution of a re- sponse variable given a covariate (Klein) 2024). This ap- proach improves uncertainty quantification and decision- making (Berger & Smith, 2019). From the conditional pre- dictive distribution, prediction regions can be derived to capture values likely to occur with a given probability. How- ever, these regions rely heavily on the predictive model's quality, and poorly estimated models can result unreliable predictions. In the following, we present split-conformal prediction (SCP: [Papadopoulos et al. 2202). computation- ally efficient variant of the conformal prediction framework
3. Rectified Conformal Prediction
The primary objective of our Rectified Conformal Predic- tion (RCP) method is enhance the conditional coverage of any given conformity score while maintaining their exact marginal validity. Expression (3) suggests that one could approximate the (1 a) qu-uutleeee of the conditional distri- bution of the scores to construct the prediction set:
'The code reproduce main experiments is available at https::/gitthn thub. com wwt/ssattttnntneeeeeeeeece htmm//statmmm/rpppp
C,(x) { ):: V(z.v) S <Qi0 (PvIx==)}
2

---

Rectifying Conformity Scores for Better Conditional Coverage
This prediction set provides approximate conditional guar- antees that depend on the accuracy of the quantile estimator. However, it fails to ensure exact marginal coverage which is an essential property for conformal prediction methods.
Under H1 we denote by 1/ the inverse of the function f.p- i.e., fo (t) = t, for all t E T. Let &p E R be such that
f is invertible (see H 11. Set
(5) and denote V = V(X,Y), and V = Vx(X,, We now define the following prediction set
(2( - (V(r,))
motivation for RCP. Our RCP method is specifically designed to achieve both exact conformal marginal validity and approximate conditional coverage. To achieve this, RCP first constructs specially transformed (rectified) scores to en- hance conditional coverage. To construct the rectified scores, builds on the key observation that marginal and condi- tional coverage coincide precisely when the conditional (1 a)) -quantile of the conformity score is independent of the covariates. RCP then applies the SCP procedure to these rectified scores, ensuring the classical exact conformal marginal validity.
(²(z)= E y: V(z,u) s (p))
(6)
where
T. (2) = Q1-a (Pv | |X = fQ =x= (PvIx-
(7)
i.e., the (1 a1)a) conditional quantile of the transformed score V given X = x. We retrieve Example] [ with Sz(u) = vl, o 1. In this case f (t) = t and (1 v) = (v v). Similarly, for Example 2 f((e) = t = 0. In such a case, fo (t) = / and Vo y) = v)
For any given score Vaæ, y). referred to as the basic score, RCP computes a rectified score (xx, y). which is a transfor- mation of the basic score that satisfies, for Px-a.e. x E X,
Q1-a (Pva(Pvx.)))) = Q1-a
Q -I--a(P(x..)Ixx))
(4)
In the following, we show that the prediction set in 6 satisfies the conditional validity guarantee in 2 and, subse- quently, the marginal coverage guarantee in 1 In fact, can write
Below we present two examples that show how one can construct the rectified scores satisfying (4)
Example 1. Consider the rectified score (z,
F(Y E CX() 1 X = r)= F(V S (x)( 1x xx)
V(x, ((x.))/Q1aa(Pv)x (Pv)x=>). with the assumption that
P(V S (x)) X x)
Q1-a (Pvi==)) > 0 for any x X. We can de- fine the following prediction set, equivalent to (3: (x(r)= = (y E V: V(z,)) S 1]. This prediction set satis- fies conditional coverage. Furthermore, in Appendix B.] we prove that this rectified score satisfies the equality in
(0)
P(V 3 (x) x = x) = 1 1 a.
where we have used in (a) that fo(() = fe(e), in (b) that f is invertible and the definition of V and in (c) the definition of T. (z)) We may rewrite the prediction set (6) in terms of the rectified score V. v) = -T. (V ())
Example 2. Consider the rectified score V(I, y) V(x (Pvix The corresponding prediction set, also equivalent to (3. is: (xx (v E y: Vax,)) S 0}, and it satisfies conditional coverage. Furthermore, in Appendix B.2 prove that this rectified score satisfies the
(x) = (v e D: V. 5 v).
(8)
equality in
In Appendix B.4, we establish that the rectified score sat- isfies (4). more precisely, setting V. V.(X, Y), for all x EEA X,
In the following, we generalize over these two basic exam- ples and present a rich family of general score transforma- tions that allow for score rectification.
Q1o ( (P..).
RCP with general transformations. Recall that starting from basic score function Vzz,g), we develop trans- formed score V(x, y) to achieve conditional validity at given confidence level a. To do so, we introduce a transfor- mation to rectify the basic conformity score V.
With the rectified score, conditional and unconditional cover- age coincide. However, while the oracle prediction set in (6) provides both conditional and marginal validity, it requires the precise knowledge of the pointwise quantile function T, (I). In practice, t, (x) is not known and one must con- struct an estimate f(x) using some hold out dataset. Below we discuss the resulting data-driven procedure.
Consider a parametric family {fiher with (t,v) E T + fe(w)) E and C TR. For convenience, we define ft(t) fv) and proceed under the following assumption. H1. The function v E R U [0o} + f(()) is increasing for any t E T. There exists & E R such that fo is continuous, increasing, and surjective on R.
4. Implementation of RCP
The RCP algorithm. Rectified conformal prediction ap- proach, as discussed above, requires a basic conformity
3

---

Rectifying Conformity Scores for Better Conditional Coverage
Algorithm 1 The RCP algorithm
obtained by minimizing the empirical pinball loss:
Input: Calibration dataset D, miscoverage level a, con-
formity score function V, transformation function fi-
test input 2.
1 Pa (X -x(x)) + Ag(T),
A Calibration Stage
F E arg min TEC
Split D into {(X,, {o) and (x,,))) He=
D, < {(Xg VXX, {0)
(12)
Q1-a - conditional quantile estimate on D,. F ("i-a)
where g is a penalty function and C' is a class of functions. When (ax) 0 (xx) where o is a feature map, and g is convex, the optimization problem in (12) becomes con-
for k = n do
V& < p-1 ( (V (X),))))))
vex. Theoretical guarantees in this setting, are given, e.g.,
in (Chen & Wei 2005 Koenker 2005).
end for
ka
[[1 - an + 1)].
Non-parametric methods have also been extensively ex- plored, see, e.g., (Chernozhukov & Hansen 2005 Cher nozhukov et al., 2022). Takeuchi et al. (2006) introduced the kernel quantile regression (KQR) framework, formu- lating quantile regression as minimizing the pinball loss in an RKHS with Tikhonov (squared-norm) regularization. It established some of the first theoretical guarantees for RKHS- based quantile models, deriving finite-sample gener- alization error bounds using Rademacher complexity; these results were later improved in (Li et al. 2007)
V (ko) f -ia-th smallest value in (Ve)kew) U (+xo).
D Test Stage
(I) (y E y: (V(,,)) s ((nl.
Output: (r).
score function V, a transformation function ft, and a cal- ibration dataset of N n m points. A critical step in the RCP algorithm is estimating the conditional quantile ((x) R Qi-a (Pv, x=o). which we discuss in detail below. F is learned on a separate part of calibration dataset com- posed of m data points {(X,, Yg):
Local quantile regression. The local quantile can be obtained by minimizing the empirical weighted expected value of the pinball loss function P1-a. defined as follows:
h = 1,..., 1...., Subse- quently, RCP uses SCP with the rectified scores V( y) == (F(z) (V (ic, v)) instead of the basic scores Vix, y). SCP is applicd to the rectified scores computed on the second part of the calibration dataset: VA- = V(KR, Y.), k I,..., n.
m Wy (x (Va Vt) - t)
?(x) E arg min EE
Finally, for a given test input x and miscoverage level a, RCP computes the prediction set as
(13)
where {we (x:) m are positive weights; see (Bhattacharya
Cx(2)) = (v y:
y: V( s a
8x
& Gangopadhyay 1990). For instance, we set wx(xx)
7
"Kn (|x XXI... where for h > O, Kn(-) h K2 1.) is kernel function satisfying J K, (z(x)rr= = 1. f (xKi(r)de = 0 and f x=²K( < oo; lxx the ker- nel bandwidth is tuned to balance bias and variance. With appropriate adaptive choice of ((x), this approach can be shown to be asymptotically minimax over Hölder balls; see (Bhattacharya & Gangopadhyay 1990 Spokoiny et al.) 2013 Reiß al. 2009). More recently, [Shen et al. (2024) in- troduced penalized non- parametric approach to estimating the quantile regression process (QRP) using deep neural net- works with rectifier quadratic unit (ReQU) activations. [Shen] et al. (2024) derives upper bounds on the mean-squared error for quantile regression using deep ReQU networks, depending only on the approximation error and network. The bounds are shown to be tight for broad function classes (e.g., Hölder compositions, Besov spaces), implying that ReQU neural networks achieve minimax-optimal conver- gence rates for conditional quantile estimation. Notably, the theory requires minimal assumptions and holds even for heavy-tailed error distributions.
(10)
The resulting RCP procedure is summarized in Algorithm [ We show exact marginal validity of RCP and give bound on its approximate conditional coverage in Section o below.
Estimation of T. (x)). We present below several methods for estimating 7. (z). Interestingly, even coarse approxima- tions of this conditional quantile can significantly improve conditional coverage; see the discussion in Section 7
Quantile regression. For any x E Rdd the conditional quantile, denoted by (x), is minimizer of the expected pinball loss:
(x) = arg min E. [pI-a
(11)
where the minimum is taken over the function T: X' + IR
and P1-a is the pinball loss (Koenker & Bassett Jr 1978
Koenker & Hallock 2001): P1-a(T)= = (1 C1-a0
(1-a)rl, a
or or llgo In practice, the empirical quantile function f is
4

---

Rectifying Conformity Scores for Better Conditional Coverage
5. Related Work
adapt skewed data (Sesia & Romano, 202 to min-
imize the average volume (Sadinle et al. 2019 denoted
is well known that obtaining exact conditional cover- age for all possible inputs within the conformal prediction framework impossible without making distributional as- sumptions (Foygel Barber al., 2021). However, the litera- ture has proposed various relaxations of exact conditional coverage, focusing on different notions of approximate con- ditional coverage.
DCP in our paper) or to define more flexible highest-density
regions (lzbicki et al. 2022)) Plassier et al. 2025). Proba- bilistic conformal prediction (PCP; Wang et al. 2023) by- passes density estimation by constructing prediction sets as unions of balls centered on samples from a generative model. All these methods are either tailored to handle the scalar response (d 1) require an accurate conditional distri- bution estimate which might be hard to obtain in practical scenarios.
first class of methods involves group-conditional guaran- tees (Jung et af. 2023 Ding et al., 2024), which provide coverage guarantees for a predefined of groups. Another class partitions the covariate space into multiple regions and applies classical conformal prediction within each re- gion (LeRoy & Zhao 2021 Alaa et al. 2023 Kiyani et al. 2024). The significant limitation of these methods lies in the need to specify the groups or regions in advance.
Guan 2023) introduces a localized conformal prediction framework that adapts to data heterogeneity by weighting calibration points based on their similarity to the test sample. To do so, kernel-based localizers assign greater importance to nearby points, tailoring prediction intervals to local data patterns. Amoukou & Brunel] (2023) extend Guan's 's ap- proach by replacing kernels with quantile regression forest estimators for improved performance. Although effective, these methods face challenges in high-dimensional or mixed- variable settings.
Other conformal methods aim to approximate conditional coverage by leveraging uncertainty estimates from the base predictor. When d 1, Conformalized Quantile Regres- sion (CQR: Romano et al., 2019)) suggests constructing a conformalized prediction interval (x:) by leveraging two quantile estimates of Y X = Ir, denoted as (1))((r) and hh-a/2 (z). This approach yields prediction intervals that adapt to heteroscedasticity (Kivaranovic et al., 2020). By considering version of CQR by [Sesia & Candès (2020)
Several methods aim to transform conformity scores im- prove approximate conditional coverage. For example, Do hansson et al. (202T). following earlier works by Papadopou
los & Haralambor (201)) Johanssonnnnnno et al. al. (2014); ; Lei et al.
(0088). investigate normalized conformity scores (NCF), which enhance standard conformal prediction by adjusting prediction set according to instance difficulty. NCF can be represented within our framework through a specific choice of the function f- = v/(r+ + (3), where 3-values will put greater emphasis on the difficulty estimation. Notably, the estimation approach employed in these papers uses least- squares regression on residuals, in contrast to the quantile regression approach adopted in RCP, which is essential to satisfy ④ Han et al. (2022) presents an approach that uses kernel density estimation to approximate the conditional distribution. Similarly, [Deutschmann et al. (2023) rescales the conformity scores based on an estimate of the local score distribution using the jackknife+ technique. However, these methods generally rely on estimating the conditional distribution of conformity scores, which is challenging in practice. [ewolff et al. (2025) studies conditional validity of normalized conformal predictors in oracle setting, i.c.. when the optimal normalization is known.
whose conformity score is positive, we can draw a connec- tion with RCP. The conformity score is
V(x, v) = ly - Po (x)) (x))
with Mo (x) = (11= + (2)2 and (x(x) Φ /2 (z) - (z)) Applying RCP with (v) = to yields
the following scaled transformed conformity scores:
Vi, v) = ly pa (x)) /+x),
where F(x) is an estimator of the conditional (1 - a)- quantile of YY- - Na (x)) given X = z. Thus, this particular variant of RCP closely resembles the CQR approach but uses a different quantile estimate.
In the context of multivariate prediction sets, given pre- dictor (u(-), natural choice for the conformity score is (z, v) = -1y--(l))l bo. where |/u]| = maxigega(1u, ) (Diquigiovanni et al. 2024). This conformity score mea-
sures the prediction error associated with the predictor u (Nouretdinov et al., 2001; Vovk et al.) 20055, 2009). Setting fe(v) to and y 1, the rectified confor- mity scores are given by V(x y) V(z, //((()) where F(x) R Qi II==) with V..- = Y.. Thus, RCP is similar to the approach proposed in (Lei et al. 2018). but with different choice of scaling function.
Recent work by Colombo (2024) suggests to transform the conformity score employing a normalizing flow: V y) 6(V (2,, z)) The normalizing flow is trained to map the joint distribution Pv.x of the conformity score and attributes into a product distribution, Pv Ø Px, where Py is an ar- bitrary univariate distribution. Notably, this condition is stricter than the conditional coverage criterion 4 as it enforces Pvix== = Pv for almost every x under Px. Con-
Methods utilizing conditional density estimation have been proposed to produce conformal prediction intervals that
5

---

Rectifying Conformity Scores for Better Conditional Coverage
sequently, learning such a transformation typically necessi- tates a larger sample size; see Section
for any a E (0, 1), follows
1
P E Ca((n)1)) <1 a n + 1
One method (Xie et al. 2024) proposes to use a cross- validated boosting procedure to learn a new score func- tion to be used in split-conformal prediction. The authors consider a specific family of possible score functions and corresponding loss functions tailored either deviation from conditional coverage or interval length. This method has several limitations compared our approach: limited set of score functions, tailored to one-dimensional targets, requires access to the train set, and high computation cost.
The proof is postponed to Appendix B.3 We will now ex- amine the conditional validity of the prediction set. To do so, we will explore the relationship between the conditional coverage of C. (2) and the accuracy of the conditional quan- tile estimator F(x). To simplify the statements, we assume that the distribution of Pv, X=x= where V. (X, Y) continuous. Define
RCP method shares some similarities with that of Gibbs et al. (2025), which also performs a quantile regression of the conformity score with respect to the attribute X. There are Iwo essential differences: firstly, Gibbs et al. (2025) work directly with the conformity score V, whereas we regress on transformed score Vpp. Secondly, the manner in which the quantile regression result is used differs significantly. RCP uses the quantile estimator to define the rectified score V. to which the standard CP procedure, while Gibbs et al. (2025) propose a considerably more complex procedure; see Section 7
c = PV, S (x)) = x) 11 + (14)
The function e, represents the deviation between the current confidence level and the desired level 1 Define the conditional pinball loss
(T) = [ - - (X))xX
(15)
It is shown in Theorem 6 (see Appendix |B.6) that, under weak technical conditions, e, satisfies the following prop- erty: for all x E X.
[e,(z)) Lx
Finally, various recalibration methods have been proposed to improve marginal coverage (Dheur & Taieb (023) or conditional coverage (Dey et al. 2022). While these meth- ods can also be interpreted within the conformal prediction framework (Marx et al., 2022, Dheur & Taieb (2024), they often require modifications to the training procedure, mak- ing them less broadly applicable than purely conformal methods.
where T. (z) is defined in 77. The previous equation bounds e (x)) as a function of the quantile estimate T(ær). If (x:) is close to the minimizer of the loss function E. (as defined in (15) then (>(r) is expected to approach zero
The c.d.f function of the rectified conformity score is defined as Fv = P(V(X,Y) S -). We denote its conditional version by Fvix== = ((( S | x))
6. Theoretical Guarantees
Theorem 2. Assume that 1 and Fy is continuous and that, for any x E X. Fvix o FT is L-Lipschitz. Then, for any a E Kn 13-1, 1, it holds
aand
In this section, we study the marginal and conditional valid- ity of the predictive set (a)) defined (TO). Due to space constraints, we present simplified versions of the results. Full statements and rigorous proofs can be found in the sup- plement materials. Many of the results hold independently of the specific method used to construct the conditional quantile estimator F(z). The only assumption we impose is minimal
PP (Yn+1 E Cx(xxt1) Xa+1 = x) 2 aa +c+(x) al. x [Fo (b) (16)
The proof is postponed to Appendix B.5 According to The- orem 2 the conditional validity of the prediction set (x)
directly depends on the accuracy of the quantile estimate F(x). (fr) closely approximates the conditional quantile then (66) ensures that conditional coverage is approximately achieved.
H2. For any x E X, we have F(z) E T.
Q (iva.Pvv (Pv.
The following theorem establishes the standard conformal guarantee. We stress that for this statement, the definition of 7(r) is not essential. The result is valid for any function T(x),, and the proof follows directly from classical argu- ments demonstrating the validity of split-conformal method.
Local quantile regression. We will now explicitly control eq (z) when the estimate F(x) is obtained using the local quantile regression method outlined in (13 For any E Id.
we define Chy (x) as Ch (x) = E[Kay h (I|= - XI))].
H3. There exists M 2 0, such that for all e E R, $ + ( is M-Lipschitz. Moreover, t E R. H Khy (t) is non-increasing.
Theorem 1. Assume H H2 hold and suppose the rectified
conformity scores (V
1+ k V are almost surely distinct. Then,
6

---

Rectifying Conformity Scores for Better Conditional Coverage
Simulated data
0
1/3
2/3
1
Data Predictor
COVERAGE 90 + 01
84 01
75 + 03
= 07
Orade
Table I: Local coverage on the adversarially selected 10% of the data, w corresponds to the level of contamination of the score quantile estimate.
V(a, V) X == = is known and we denote by Q1-a (z). Given w [0, IJ, we ((z) (1- -)Q we(x). where consider c(x) ~N 2*). We perform 1000 ex- periments and report the 10% lower value of x E [0, 1] + P(Y E C. (c:) X = z); the results can be found in Table 1 If = 0, F(x) corresponds to the true (1 a)- quantile. this case, our method is conditionally valid, as 2 shows. However, while all settings of w yield marginally valid prediction sets, the conditional coverage decreases the quantile estimate 7(x) deteriorates.
0.0
0.2
0.4
0.8
0.8
1.0
X
Figure 1: Oracle data distribution, sample data and predictor for the toy dataset.
Proposition 1. Assume
3 holds. With probability at most Var Kax hxx (l - xI)),, it holds
Theorem
m {1 4Chx (a)
K1 (0) logm
[c+(x)] 2 Cnx (2x(2)) (zx-
7.2. Real-world experiment
hxm
404hx 40xx (x)
sup {tKax (M (t). 05151
We use publicly available regression datasets which are also considered in Tsoumakas et al.) 201 Feldman et al. 2023 Wang al. 2023 and only keep datasets with at least 2 outputs and 2000 total instances. The characteristics of the datasets are summarized in Appendix C
The proof is postponed to Appendix B.7 Proposition | highlights the trade-off associated with the bandwidth pa- rameter hx. Ideally, would like choose hx < to minimize suposisi [tKnx (L (L). However, this
Base predictors. We consider two base predictors, both parameterized by fully connected neural network with three layers of 100 units and ReI.U activations.
sults in an increase of / m mhxx Consequently, there ex-
ists optimal bandwidth parameter hx that depends both the number of available data points and the regu- larity of the conditional cumulative distribution function I +
The mean predictor estimates the mean (4(x) of the distribu- tion for each dimension i E [d] given x E X. Since it only provides point estimate, does not capture uncertainty.
Finally, for the optimal choice of bandwidth hx one can prove the asymptotic validity of RCP:
The mixture predictor models a mixture of K Gaussians, enabling it represent multimodal distributions. Given x X, the model outputs =(x) B.K (logits for mix- ture weights), u(x) E k (mean vectors), and L(x) E k (lower triangular Cholesky factors). The mix- ture weights ((z) E R are obtained by applying the softmax function to 2(x), and the covariance matrices E(x) Rxxdd are computed as Ex(x) La Lx) The conditional density at y E V, given X, is:
P (V E C. (X) + a, n, m oc.
(17)
The exact formulation and its proof are given in Ap- pendix B.8
7. Experiments
7.1. Toy example
K
ply ()) = Σ
Tk(2) - NN mm(x), Ex(x),
Let us consider the following data-generating process:
1=1
X (el.a(1.2,0.))
Y|X=a N(((()),=").
where N(y Ex( is a Gaussian density with mean (x) and covariance matrix Ex(x).
where μ(x) sin) Figure shows a realization with 100 data points. Our goal is to investigate the influ- ence of the quality of the (1 -quantile estimate 7 on performance.
Methods. We compare RCP with four split-conformal pre- diction methods from the literature: ResCP (Diquigiovanni et al. 2024). PCP (Wang et al. 2023). DCP (Sadinle et al.)
We set a = 0.1 and consider the conformity score V V( v) = ly p(x)). this case, the (1 - a--quantile of
2019 and SLCP (Han et al. 2022). ResCP uses residuals outputs,
as conformity scores. To handle

---

Rectifying Conformity Scores for Better Conditional Coverage
Oracle
ResCP
RCP-ResCP
PCP
Cumm
2
1
1 x
-1
-1
-1
-1
2
-1
2
Ys
X
1
Y2
1
Ys
RCP-PCP
DCP
RCP-DCP
2
2
1 x
x
0
-1
1
-1
2
-1
0 x
1
-2
X
X
-2
Figure 2: Examples of prediction sets on synthetic dataset where the output has a bivariate and bimodal distribution.
we follow (Diquigiovanni et al., 2024) and define the con- formity score as the 1R norm of the residuals across dimen- sions, i..., (zz, y) = m (z) yal- PCP constructs the prediction set a union of balls, while DCP defines the prediction set by thresholding the density. ResCP is com- patible with the mean predictor, whereas PCP and DCP are compatible with the mixture predictor. Finally, SLCP, like RCP, is compatible with any conformity score and base predictor. For RCP, we compute an estimate F(x) (see Sec- tion using quantile regression with a fully connected neural network composed of 3 layers with 100 units.
hyperparameter tuning are provided in Appendix
Evaluation metrics. To evaluate conditional coverage, we use worst-slab coverage (WSC, [Cacchoi et al.) 2020 Ro mano et al.) 2020) with 5 = 0.2 and the conditional cover- age error, computed over partition of X, following Dheur et al. (2025). To evaluate sharpness, we also report the me- dian of the logarithm of the prediction set volume, scaled by the dimension d.
Main results. Figure 3 presents the worst-slab coverage and volume for different conformity scores, both with and without RCP. Similarly, Figure compares worst-slab cov- erage between SLCP and RCP. Additional results, including conditional coverage error and marginal coverage, are pro- vided in Appendix A.l
compares
Visualization on synthetic dataset. Figure 2 illustrates example prediction sets for different methods. The orange and black contour lines represent confidence levels of a = 0.1 and a = 0.8, respectively. The first panel shows the highest density regions of the oracle distribution, while the subsequent panels display prediction regions obtained by different methods, both before and after applying RCP. We can see that combining RCP with ResCP, PCP, or DCP results in prediction sets that more closely align with those of the oracle distribution.
In the left panel of Figure we observe that ResCP, PCP, and DCP fail to reach the nominal level of conditional cov- erage for most datasets. In contrast, all variants of RCP significantly improve coverage across all datasets. Similarly, Figure shows that RCP often achieves better conditional coverage than SLCP, particularly on larger datasets. Fig- ure 5 Appendix A.I confirms these findings with the conditional coverage error. Finally, as expected, all methods achieve marginal coverage.
Experimental setup. We reserve 2048 points for calibra- tion. The remaining data is split between 70% for training and 30% for testing. The base predictor is trained on the training set, while the baseline conformal methods use the full calibration set to construct prediction sets for the test points. In RCP, the calibration set further divided into two parts: one for estimating f(x) and the other as the proper calibration for obtaining intervals. This ensures that all methods use the same number of points for uncertainty estimation. When not specified, we used the adjustment fe(v) = L + v. Additional details on implementation and
In the right panel of Figure 3 we observe that RCP improves the median prediction set volume compared to non-RCP variants in addition of improving conditional coverage.
Finally, Appendix A.6 compares average and median vol- umes of prediction sets produced by direct conformal meth- ods and their RCP counterparts. Direct methods obtain smaller average volume while RCP obtains a smaller median
8

---

Rectifying Conformity Scores for Better Conditional Coverage
ResCP
RCP-ResCP
DCP
RCP-DCP
PCP
RCP-PCP
0.90
log(volume) /
0.85
WSC
0.80
0.75
0.70
households scm20Ad
rfl
scnid
meps 21
meps peps__9
maps 20
house
bio blog data calcofi
taxi
households
scm20d
rfl somld meps
meps meps 20 house
19
bio blog
bata-dtta data calcofi
taxi
Figure 3: Worst-slab coverage and volume for three conformal methods and their RCP counterparts, on datasets sorted by
total size.
CQR, showing that CQR already obtains a competitive con- ditional coverage but is outperformed by RCP-DCP in av- erage volume. Appendix |..7 presents preseets an additional study comparing RCP with CP and CQR methods, that adapted to multidimensional target setting. For these experiments, the model predicts parameters of multivariate normal dis- tribution and we use the score based on the corresponding Mahalanobis distance. We demonstrate that RCP improves conditional coverage over classic CP and also benefits from the custom score outperform CQR.
RCP-ResCP
RCP-DCP
RCP-PCP SLCP-PCP
SLCP-ResCP
SLCP-DCP
0.90
WSC
0.85
0.80
households sem20d
rfl
meps -21
meps, 19
meps .20
house
bio blog data calcof
taxi
Appendix A.8 considers approach improve data effi- ciency. Instead of dividing the calibration dataset D into two parts to estimate 7, we compute out-of-sample con- formity scores on the training dataset Drrai using K-fold cross-valie tross-validation. This results in improved conditional cover- age at the cost of training K additional models.
Figure Worst-slab coverage for RCP and SLCP combi- nation with different conformity scores, datasets sorted
by total size.
Appendix A.9 provides an additional comparison with Con- ditional Prediction with Conditional Guarantees (CPCG; Gibbs et al. (2025). CPCG obtains competitive condi- tional coverage but is 100-100000 times slower than RCP overall, limiting its applicability.
volume.
Additional experiments. We complement main results with multiple experiments aiming at studying variations of the proposed method and comparing it with some additional competitors,
8. Conclusion
We present a new approach improve the conditional coverage of the conformal prediction set while preserving marginal convergence. Our method constructs prediction sets by adjusting the conformity scores using appropri- ately defined conditional quantile, allowing RCP to automat- ically adapt the prediction sets against hetcroscedasticity, Our theoretical analysis supports that this approach produces approximately conditionally valid prediction sets; further- more, the theory provides lower bounds the conditional coverage, which explicitly depends on the distribution of the conditional quantile estimator f(z).
Appendix A.2 discusses the estimation of F(x) using either a neural network or local quantile regression for which we have bounds the conditional coverage. On most datasets, the neural network slightly outperforms local quantile re- gression, which expected due to its flexibility. Appen- the choice of adjustment function. For certain adjustment functions, the domain of the scores V(r, y) must be restricted to subset of R to satisfy Notably, fe(v) = tu requires v > 0, fel = exp(fu) requires require v >> 1.
dices an.
Appendix A.5 directly compares the proposed method with

---

Rectifying Conformity Scores for Better Conditional Coverage
Acknowledgments
class and multilabel prediction. J. Mach. Learn, Res., 22: 81:1-81:42, 2020.
V.P. carried out this study during his PhD under the super- vision of E.M., and subsequently as a research engineer the Centre Lagrange in Paris. Part of E.M.'s work was also conducted under the auspices of the Centre Lagrange. E.M. was all partially funded by the European Union (ERC- 2022-SyG, 101071601). Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Re- search Council Executive Agency. Neither the European Union nor the granting authority can be held responsible for them.
Chen, C. and Wei, Y. Computational issues for quantile regression. Sankhyā: The Indian Journal of Statistics, pp. 399 0199417,,00 2005.
Chernozhukov, V. and Hansen, C. An iv model of quantile treatment effects. Econometrica, 73(1):245 261, 2005.
Chemozhukov, V., Fernández- Val, I., and Melly, B. Fast algorithms for the quantile regression process. Empirical economics, pp. 1-27, 2022.
Colombo, N. Normalizing flows for conformai regression. Proceedings of the Fortieth Conference on Uncertainty Artificial Intelligence, pp. 881-893, 2024.
Impact Statement
This work contributes to the broader effort to improve the interpretability and statistical reliability of machine learning algorithms. Prediction intervals with exact marginal and approximate conditional coverage offer a useful tool for conveying uncertainty regarding the accuracy of ML models, which is essential for increasing their fairness, and fostering wider acceptance.
Deutschmann, N., Rigotti, M., and Martínez, M. R. Adap- tive conformal regression with jackknife+ rescaled scores. arXiv preprint arXiv:2305. 19901, 2023.
Dewolf, N., De Baets, B., and Waegeman, W. Conditional validity of heteroskedastic conformal regression. Informa- tion and Inference: A Journal of the IMA, 14(2)iaaff013, 2025.
References
Dey, B., Zhao, D.,, Newman, J. A., Andrews, B. H., Izbicki, R., and Lee, A. B. Conditionally calibrated predictive distributions by probability probbabiity map: Application to galaxy redshift estimation and probabilistic forecasting. arXiv preprint arXiv:2205. 14568, 2022.
Alaa, A. M., Hussain, Z., and Sontag, D. Conformalized unconditional quantile regression. In International Con- ference on Artificial Intelligence and Statistics, pp. 10690- 10702. PMLR, 2023.
Amoukou, S. I. and Brunel, N. J. Adaptive conformal prediction by reweighting nonconformity score. arXiv preprint arXiv:2303. /26955, 2023.
Dheur, and Taieb, S. B. A large-scale study of probabilis- tic calibration in neural network regression. In Interna- tional Conference Machine Learming, pp. 7813-7836. PMLR, 2023.
Angelopoulos, A. N., Bates, S., et al. Conformal predic- tion: A gentle introduction. Foundations and Trends@ in Machine Learning, (644):494-591,2000
Dheur, and Taieb, S. B. Probabilistic calibration by design for neural network regression. In International Conference Artificial Intelligence and Statistics, pp. 3133-3141. PMLR, 2024.
Angelopoulos, A. N., Barber, F., and Bates, S. Theoreti- cal foundations of conformal prediction. arXiv preprint arXiv:2411.11824 2024.
Dheur, V., Bosser, T., Izbicki, R., and Ben Taieb, S. Distribution- conformal joint prediction regions for neural marked temporal point processes. Machine Learn- ing. 13:70555-7102 2024.
Berger, J. O. and Smith, L. A. On the statistical formalism of uncertainty quantification. Annual Review of Statistics and Its Application, 6(1):433-466) March 2019.
Bhattacharya, P. K. and Gangopadhyay, A. K. Kernel and nearest-neighhor estimation of a conditional quantile. The Annals of Statistics, pp. 1400-1415, 1990.
Dheur, V., Fontana, M., Estievenart, Y., Desobry. N., and Ben Taieb, S. A unified comparative study with gen- eralized conformity scores for multi-output conformal regression. The 42th International Conference on Ma- chine Learning, 2025.
Bian, M. and Barber, R. F. leaining-conditional coverage for distribution-free predictive inference. Electronic Journal of Statistics, 17(2):2044 2023.
Ding, T., Angelopoulos, A., Bates, S., Jordan, M., and Tib- shirani, R. J. conditiooa conformal prediction with many classes. Advances in Neural Information Process- ing Systems, 36, 2024.
Cauchois, M., Gupta, S., and Duchi, J. C. Knowing what you know; valid and validated confidence sets multi-
10

---

Rectifying Conformity Scores for Better Conditional Coverage
Diquigiovanni, J., Fontana, M., Vantini, S., al. The impor- tance of being band: Finite-sample exact distribution- free prediction sets for functional data. Statistica Sinica, 1:1-41, 2024.
Kiyani, S., Pappas, G. J., and Hassani, H. Conformal predic- tion with learned features. In International Conference on Machine Learning, 2024.
Klein, N. Distributional regression for data analysis. Annual Review of Statistics and Its Application, 11(1), 2024.
Feldman, S., Bates, S., and Romano, Y. Calibrated multiple- output quantile regression with representation learning. J. Mach. Learn. Res., 24:1-48, 2023.
Koenker, R. Quantile regression. Cambridge University Press, 2005.
Foygel Barber, R., Candes, E. J., Ramdas, A., and Tibshirani, J. The limits of distribution-fie free conditional predictive inference. Information and Inference: A Journal of the IMA, 10(2):455 482, 2021.
Koenker, R. and Bassett Jr, G. Regression quantiles. Econo- metrica: journal of the Econometric Society, pp. 33-50, 1978.
Gibbs, I., Cherian, J. and Candès, E. J. Conformal pre- diction with conditional guarantees. Journal of the Royal Statistical Society Series B: Statistical Methodology. pp- qkaf008. 2025.
Koenker. and Hallock, K. F. Quantile regression. Journal of economic perspectives, 15(4):143-155) 2001.
Lei, J. and Wasserman, I.. Distribution-fres prediction bands for non-parametric regression. Journal of the Royal Sta- tistical Society Series B: Statistical Methodology, 76(1): 71-96, 2014.
Grinsztajn, I.., Oyallon, E., and Varoquaux, G. Why do tree- based models still outperform deep learning on typical tabular data? Advances neural information processing systems, 35:507-520, 2022.
Lei, J., G'Sell, M., Rinaldo, A., Tibshirani, R. J., and Wasserman, L. Distribution-fre predictive inference for regression. Journal of the American Statistical Associa- tion, 1 1136523):1094-1111 2018.
Gruber, C., Schenk, O., Schierholz, M., Kreuter, F., and Kauermann, G. Sources of uncertainty ma- chine learning statisticians" view. arXiv preprint 11X1: 2023.
LeRoy, B. and Zhao, D. Md-split+: Practical local con- formal inference in high dimensions. arXiv preprint 1iXiv:2107.03280 2021.
Guan, L. Localized conformal prediction: A generalized in- ference framework for conformal prediction. Biometrika, 110(1):33-50, 2023.
Li, Y., Liu, Y., and Zhu, J. Quantile regression in repro- ducing kernel hilbert spaces. Journal of the American Statistical Association, 102(477):255-266 2007.
Han, X.. Tang, Z.. Ghosh, J., and Liu, Q. Split localized conformal prediction. arXiv preprint 220i:::2006130992 2022.
Marx, C., Zhao, S., Neiswanger, W., and Ermon, S. Modular conformal calibration. In International Conference on Machine Learning, pp. 15180-15195, PMLR, 2022.
Izbicki, R., Shimizu, G., and Stern, R. B. Cd-split and hpd-split: Efficient conformal regions in high dimensions. The Journal of Machine Learning Research, 23(1):3772- 3803, 2022.
Nalisnick, E., Matsukawa, A., Teh. Y. W., Gorur, D.. and Lakshminarayanan, B. Do deep generative models know what they don't 't know? In International Conference on Learning Representations, 2018
Johansson. U., Boström, H., Löfström, T., and Linusson, H. Regression conformal prediction with random forests. Machine learning, 97:155-176, 2014.
Nesterov, Y. Introductory lectures on convex programming volume i: Basic course. Lecture notes, 3(4):5, 1998.
Johansson, U., Boström, H., and Löfström, T. Investigating normalized conformal regressors. In 2021 IEEE Sympo- sium Series on Computational Intelligence (SSCI), pp. 01-08. IEFE, 2021.
Nouretdinov, I., Melluish, T., and Vovk, V. Ridge regression confidence machine. In ICML, pp. 385-392, 2001.
Jung, C., Noarov, G., Ramalingam, R., and Roth, A. Batch multivalid conformal prediction. In International Confer- ence on Learning Representations, 2023.
Papadopoulos, H. Inductive conformal prediction: Theory and application neural networks. In Fritzsche, P. (ed.), Tools in Artificial Intelligence, chapter 18. IntechOpen, Rijeka, 2008
Kivaranovic, D., Johnson, K. D., and Leeb, H. Adaptive, distribution- prediction intervals for deep networks. In International Conference on Artificial Intelligence and Statistics, pp. 4346-4356. PMLR, 2020.
Papadopoulos, H. and Haralambous, H. Reliable predic- tion intervals with regression neural networks. Neural Networks, 24(8):842-25 2011.

---

Rectifying Conformity Scores for Better Conditional Coverage
Papadopoulos, H., Proedrou, K., Vovk, V., and Gammerman, A. Inductive confidence machines for regression. In European Conference on Machine Learning, pp. 345-356. Springer, 2002.
Takeuchi, I., Le, Q. V., Sears, T. D., Smola, A. J., and Williams, C. Nonparametric quantile estimation. Journal of machine learning research, 7(7), 2006.
Tsoumakas, G., Spyromitros Xioufss E.,, Vilcek, J., and Vlahavas, Mulan: A java library for multi-label learn- ing. The Journal of Machine Learning Research, 12: 241 2411-2414, 2011.
Papadopoulos, H., Gammerman, A., and Vovk, V. Normal- ized nonconformity measures for regression conformal prediction. In Proceedings of the IASTED International Conference Artificial Intelligence and Applications (AIA 2008), pp. 64-69, 2008.
Vovk, V. Conditional validity of inductive conformal pre- dictors. In Asian conference on machine learning, pp. 475-490. PMLR, 2012.
Plassier, V., Kotelevskii, N., Rubashevskii, A., Noskov, F., Velikanov, M., Fishkov, A., Horvath, S., Takac, M., Moulines, E.,, and Panov, M. Efficient conformal predic- tion under data heterogeneity. In International Confer- ence on Artificial Intelligence and Statistics, pp. 4879- 4887. PMLR, 2024.
Vovk, V., Gammerman, A., and Shafer, G. Algorithmic learning in a random world, volume 29. Springer, 2005.
Vovk, V., Nouretdinov, I., and Gammerman, A. On-line predictive linear regression. The Annals of Statistics, pp. 1566 1590, 2009.
Plassier, V., Fishkov, A., Guizani, M., Panov, M.. and Moulines, E. Probabilistic conformal prediction with approximate conditional validity. In The Thirteenth Inter- national Conference on Learning Representations, 2025.
Wang. Z., Gao, R., Yin, M., Zhou, M., and Blei, D. Prob- abilistic conformal prediction using conditional random samples. In International Conference on Artificial Intelli- gence and Statistics, pp.
PMLR, 2023.
Reiß, M., Rozenhole, Y., and Cuenod, C.-A. Pointwise adaptive estimation for robust and quantile regression. arXiv preprint arX4v:0904.0543, 2009.
Xie, R., Barber, R. F., and Candès, E. J. Boosted conformal prediction intervals. In Neural Information Processing Systems, 2024.
Romano, Y., Patterson, E., and Candes, E. Conformalized quantile regression. Advances in neural information pro- cessing systems, 32, 2019.
Romano, Y., Sesia, M., and Candes, E. Classification with valid and adaptive coverage. Advances in Neural Infor- mation Processing Systems, 33:3581 133591,, 2020.
Sadinle, M., Lei, J., and Wasserman, L. Least ambiguous set-valued classifiers with bounded error levels. Journal of the American Statistical Association, 1 114(52)):223- 234, 2019.
Sesia, M. and Candès, A comparison of some confor- mal quantile regression methods. Stat, 9(1):e261, 2020.
Sesia, M. and Romano, Y. Conformal prediction using conditional histograms. Advances in Neural Information Processing Systems, 34:6304-6315, 2021.
Shafer, G. and Vovk, V. A tutorial conformal prediction. Journal of Machine Learning Research, 9()), 2008
Shen, G., Jiao, Y., Lin, Y., Horowitz, L., and Huang, Nonparametric estimation of non-crossing quantile re- gression process with deep requ neural networks. Journal of Machine Learning Research, 25(88):1-75. 2024.
Spokoiny, V., Wang, W., and Härdle, W. K. Local quantile regression. Journal of Statistical Planning and Inference, 149 2013.
12

---

Rectifying Conformity Scores for Better Conditional Coverage
A. Additional Experiments
A.1. Additional results on marginal coverage and conditional coverage error
Figure 5 extends the results of Figure by displaying additionally the marginal coverage and conditional coverage error. As expected, all methods obtain correct marginal coverage. Furthermore, the methods with the best worst slab coverage (closest to 1 ox) also obtain small conditional coverage error, supporting conclusions in Section 7
ResCP
RCP-ResCP
DCP
RCP-DCP
PCP
RCP-PCP
0.90
0.94
Marginal coverage
0.92
0.85
WSC
0.90
0.80
0.88
0.75
0.86
0.70
households scm200
fft
sem d meps 21
meps. 19
maps 20
house
hio blog data calsofi
tax
houscholds som20d
rfl
semid
maps 21
meps. 19 meps: 20
house
bio blog data calcofi
toxi
Figure 5: Marginal coverage and conditional coverage error for three conformal methods and their RCP counterparts, on datasets sorted by total size.
A.2. Estimation of conditional quantile function
Figure 6 compares ways of estimating (see Section 4 RCPMLP corresponds to quantile regression based neural network in Section / while RCP local corresponds to local quantile regression. On many datasets, the more flexible RCPMLP is able to obtain better conditional coverage. However, local quantile regression has theoretical guarantees on its conditional coverage (see 6)
RCPMI.PRRsCP
RCPMIP-DCC
RCPMIPPPPC
RCP RRsCPP
RCP DCCP
RCP Phea--PPC
Marginal coverage
0.90
0.92
WSC
0.88
0.90
0.86
0.88
0.86
households scm20d
fi]
scm d
meps.21 meps. maps. 20
house
bio blog data
calcofi
taxi
houscholds sem20d
tfl
som
meps 21
meps. 19
meps. 20
house
bio blog data calcofi taxi
Figure 6: Marginal coverage and conditional coverage error for two types of quantile estimators in combination with different conformal methods, on datasets sorted by total size.
13

---

Rectifying Conformity Scores for Better Conditional Coverage
A.3. Choice of adjustment function
Figure compares RCP with difference (-) and linear (+) adjustments when combined with the DCP method. Since RCP with any adjustment function adheres to the SCP framework, marginal coverage is guaranteed, as shown in Panel 1.
The conformity score for DCP is defined as (12, v) log p(y a:), which can take negative values, implying that T. = R.
However, the linear adjustment requires T C R4, violating H[ and resulting in a failure to approximate conditional coverage accurately. This issue is evident in Panel 2. In contrast, the difference adjustment does not impose such a restriction.
Panel 3 compares PCP and ResCP when used with difference and linear adjustments. Since the conformity scores for these methods are always positive, i.e., RR, both adjustment methods satisfy H In general, we observe no significant differences between the two adjustment methods.
IXPP
RCP
RCP(P)-DCP
(CPCCPP
RCP RCP, RCP,,,-PCPP
RCP RCss..
Marginal eoverge
0.91
0.92
0.8
WSC
0.88
WSC
0.86
0.86
0.8
houndold sen2ted
somid maps 21
neps, mape. 20
Joster
bin blog dlaa
cakani
households scmo2d
Ifl
somi d
meps
maps, 19
maps. 20
house
blog. dita Galcol
Laxi
scm2td
sould mepel
mape, 19
mapu 20 house
bo blgg Anta caloaa
taxi
Figure 7: Marginal coverage and conditional coverage error obtained for two types of adjustments.
A.4. Additional adjustment functions
We consider two additional adjustments functions, namely ft(v) exp (t + denoted exp -, .nd f((v) = exp (tv),
denoted exp*. To apply these custom adjustment functions we need to ensure that the conditions H[ and H2 are satisfied. For the first function we have: f-1 f-1(v) (v) (In & E and o = Then f (v) 0 = Inv 0 = the second function conformity scores are usually non-negative is the case with PCP and residual scores that we consider here, and we can always add constant 1 to satisfy this requirement.
For
take yP and by similar argument arrive at the same requirement
1. In practice,
Figures and show the marginal coverage and conditional coverage error obtained with these adjustment functions.
PCP
PCPP
RCP (mrp
PCP
RCP (mop-)- PCP
0.94
0.90
Marginal coverage
0.92
0.88
WSC
0.90
0.86
0.88
0.84
0.86
households scm20d
ffl
sem d
meps_2 meps._ 19
21
mepps. 20
house
blog.ut data
bio
calcofi
taxi
houscholds som20d
rf]
scmid
meps_ 21
meps, 19
maps: 20
house
hio blog data calcofi
toxi
Figure& 8: Marginal coverage and conditional coverage error for two additional types of adjustments combined with the method PCP.
14

---

Rectifying Conformity Scores for Better Conditional Coverage
RCP; RCPP-y-ResCP ResCP
RCP (CP
RCP (mop
ResCPP
RCP (eup -RsSCP
0.94
Marginal covernge
0.90
0.92
WSC
0.90
0.88
0.86
0.86
households scm20d
ffl
scm
naps 21
mapss 19
mapss .20
house
bio blog data calcofi
tasi
households scm20d
tfi
scmld
maps 21
meps. 19 meps. 20
house
hio blog data calcofi taxi
Figure 9: Marginal coverage and conditional coverage error for two additional types of adjustments combined with the
method ResCP.
A.5. Direct comparison with CQR
Here present direct comparison of RCP with Conformalized Quantile Regression (CQR; Romano al. (2019). We the same underlying neural network architectures for the models in our main experiment. Similarly to ResCP, to handle multi-dimensional outputs, we follow (Diquigiovanni et al. 2024) and define the conformity score of CQR as the e norm of the CQR conformity scores across dimensions. Specifically, we compare CQR to DCP and its RCP-DCP counterpart, which achieves the best median volume overall.
Figure I0 shows that CQR matches the conditional coverage of RCP-DCP. However, produces larger median prediction sets due to less flexible shapes.
DCP
RCP-DCP
CQR
0.90
T Fd
0.85
WSC
0.80
0.75
0.70
households scm20d
ifl
sem d
maps.21 221
meps.11 19
meps 20
house
bio blog data calcofi
taxi
households s0m
Ifl
scm d
maps. .11
meps. 19
meps 20
house
bio blog data calcofi
toxi
Figure 10: Worst slab coverage and (logarithm) median prediction set volume (scaled by d).
15

---

Rectifying Conformity Scores for Better Conditional Coverage
A.6. Comparison of prediction set volumes
Table 2 shows the average volume obtained by the methods compared in Section Non-RCP methods obtain smaller average volume across all datasets. The larger average volume of RCP is explained by the larger regions produced for instances with larger uncertainty.
Table Mean prediction set volume per dataset.
dataset
PCP
RCP-PCP
RCP-DCP
ResCP
RCP-ResCP
households
88.3
1.33e+02
47.4
1.02c+02
1.81c+02
4.51c402
scm20d
4.26e+005
7.92c406
1.11c+06
3,95c407
5.22c+05
7.15c+12
rf1
0.0274
0.190
0.000562
4.35c+04
0.0276
7.94c+08
semld
1.92e+04
1.30e+08
2.30e+04
1.67e+08
6.27e+04
2.04e+15
meps 21
1.65
10.0
0.746
6.07
5.35
8.32e+12
meps__99
90.0
3.27e+04
3.64
3.27e+04
5.56
3.56e+22
meps_20
1.68
5,50
0.761
6,20
5,38
6,27c+13
house
0.676
0.936
0.519
0.751
2.92
3.88
bio
0.579
1.12
0.414
0.645
1.05
1.16
blog_daie data
0.459
8.45c+02
0.143
6.37e+02
1.26
6.79e421
calcofi
3.47
4.12
2.45
3.06
4.53
4.47
taxi
9.21
9.63
5.69
6.40
12.4
12.8
In contrast, Table 30le3sshor shows that RCP obtains smaller regions across most datasets when comparing the median volume, avoiding outliers.
Table 3: Median prediction set volume per dataset.
dataset
PCP
RCP-PCP
RCP-DCP
ResCP
RCP-ResCP
households
67.4
$6.5
39,2
32.1
1.81e402
1.67++02
scm20d
3.11e+04
1.42c+04
7.03e+05
7.74c+04
5,22e+05
2.00e+05
rfll
0,0110
0.00525
0.000583
33.1
0.0276
0.0231
scmld
1.16c+02
2.05
1.01e+04
25.7
6,27e+04
1.70e+03
meps__] _2
1.04
0.704
0.433
0.227
5.35
2.42
meps_19
3.85
0.754
2.10
0.303
5.56
2.54
meps_20
1.05
0.616
0.416
0,254
5.38
2.51
house
0.596
0.519
0.471
0.386
2.92
2.67
bio
0.507
0.435
0.374
0.344
1.05
0.829
blog data
0.229
0.209
0,0869
0.0597
1.26
1.16
calcofi
3.83
3.98
2.85
2.77
4.53
4.75
taxi
8.67
8.25
5.22
5.27
12.4
10.1
16

---

Rectifying Conformity Scores for Better Conditional Coverage
A.7. Case of ellipsoidal prediction sets
In this section, we will investigate how all parts of our proposed RCP method contribute to the performance. Additionally, we demonstrate the wider applicability of our approach: in this section, To achieve this, modify our base models to predict parameters of the multivariate normal distribution. As bascline method, we have selected CQR because of its popularity and ease of use.
different score and ellipsoid prediction sets,
Figure 11 demonstrates improvements of RCP over simpler methods, with CQR serving as a strong contemporary alternative. Each of these simpler methods employs consecutively more complex model and/or calibration procedure. The first part of the name corresponds the base prediction model, and the second part after the dash denotes the calibration procedure:
Const: the base model for these methods is constant prediction of multivariate normal distribution parameters estimated on the train set.
MLP: the base model is multidhmensional perceptron that predicts the parameters of multivariate normal distribution.
CP: classic conformal prediction using full calibration set. We estimate a fixed prediction ellipsoid size using Mahalanobis distance-based nonconformity score.
RCP: our usual RCP procedure where we split the calibration set and fit quantile regression model to predict the (1 a) quantile of the Mahalanobis score. Similarly to the other experiments, quantile regression is fit using MLP underlying model.
The alternative method CQR based on quantile regression estimates for each dimension of the output variable. First, (univariate) conformalized quantile regression scores (Romano et al. 2019) are computed for each dimension. Then, they are aggregated by taking the maximum score over each dimension, similarly ResCP in the main text. The resulting prediction set in this case is hyperrectangle. Its size is adaptive the input, but the conformal correction is isotropic and constant for all input points.
0.:
0.9
1.0
log(eotume)//
Worst slab coverage.
0.5
0.8
0.0
0.7
0.5
scm20d
semid
meps 21
meps. 19
meps 20
house
bio
blog data
taxi
scm20d
somid
meps.21
meps.
119
meps 20
house
bio
blog data
taxi
CQR
Const-Cr CP
Const- RCP
MIP.-CP
MCPP-RCP
Figure 11: Worst slab coverage and logarithm of prediction set volume (divided by number of dimensions of the response).
The graphs on Figure provide some important insights:
Methods based classic conformal prediction (Const-CP, MLP-CP) often struggle to maintain conditional coverage.
RCP improves conditional coverage: Const-RCP outperforms Const-CP conditional coverage and set size.
RCP in combination with a better predictive model either maintains or improves conditional coverage and volume.
17

---

Rectifying Conformity Scores for Better Conditional Coverage
A.8. Improved data efficiency using cross-validation
As explained in Section 7 RCP requires to divide the calibration dataset D into two parts, one estimate F, and one for SCP.
In this section, we consider more data-efficient approach using the training dataset Danin- Using K-fold cross-validation on Drain for each fold index k, we train a model on the K remaining folds and evaluate the conformity score on the fold k. This yields dataset D, of size [Diiin| with inputs and their associated conformity scores based on which F is estimated. This also removes the need to split the calibration dataset. An additional model fitted on the complete training data set Daiin to produce the non-rectified conformity scores.
Figure 12 shows comparison of learning on half the calibration dataset (cal), using 10-fold cross-validation (CV). The cross velsdavaliationn approach yields improved worst-slab coverage most datasets. This improved conditional coverage comes at the computational cost of training K additional models.
RCsCo
RCPey-ResCP
RCP DCP
DCP
RCPev-DD
RCPou
RCPcy-PCP
Marginal coverage
0.900
0.92
WSC
0.875
0.850
0.825
households sem20d
rfl
som d maps. 21
meps. 19
maps 20
house
hio blog data calcofi
taxi
households scm20d
rfl scm d maps 211 meps. 19 meps. 200
house
bio blog data calc
taxi
Figure 12: Worst-slab coverage of RCP with i trained on half the calibration dataset (cal) or using 10-fold cross- validation (CV).
18

---

Rectifying Conformity Scores for Better Conditional Coverage
A.9. Comparison with CPCG
We conduct an additional experiment comparing RCP with Conditional Prediction with Conditional Guarantees (CPCG; Gibbs al. (0025). We evaluate RCP using both the full calibration dataset (RCPcai) and cross-validation (RCPcv), as deseribed in Appendix A.8 All methods are CPU (AMD Ryzen Threadripper PRO 5965WX) with 6 CPU threads experiment.
Table 4 shows that all methods achieve comparable worst-slab coverage, close to the nominal level. However, Table reveals stark contrast in computational efficiency: CPCG 200-100,000 times slower than RCP and 10-100 times slower than RCPcy overall. This significant overhead because CPCG must solve an optimization problem involving the entire calibration set for each instance. Consequently, CPCG's computational demands become prohibitive for large calibration and test sets, hindering its practical application. Moreover, CPCG failed find solution the "house" and "calcofi" datasets, precluding results for these cases. These factors highlight RCP's 's substantial practical advantage in efficiency, especially for large-scale datasets.
Table 4: Comparison of worst-slab coverage on multi-output datasets.
PCP
-CP
RCPev-PCP
CPCG-PCP
DCP
RCPai-DCP
RCPcv-DCP
CPCG-DCP
households
0.825
0.905
0.899
0.888
0.853
0.891
0,900
0.900
scm20d
0.830
0.892
0.891
0.897
0.877
0.877
0.868
0.899
rfl
0.731
0.830
0.877
0.838
0.715
0.863
0.827
0.872
scm
0.758
0.882
0,895
0.910
0.756
0.902
0.896
0,882
meps_21 21
0.739
0.874
0,904
0.881
0.789
0.881
0.879
0.905
meps_19
0.762
0.875
0.867
0.880
0.788
0.884
0.889
0.878
meps. 20
0.731
0.842
0.871
0.890
0.719
0.880
0.884
0.892
house
0.835
0.895
0,903
0.817
0.878
0,906
bio
0.784
0,860
0,900
0.887
0.774
0.879
0,880
0.880
blog-data
0.770
0.877
0.893
0.886
0.749
0.844
0.888
0.888
calcofi
0.810
0.889
0.888
0.828
0.885
0.892
taxi
0.837
0.885
0.884
0.881
0.846
0.872
0.879
0.879
Table 5: Comparison of computational time (in seconds) multi-output datasets.
PCP
RCPev-PCP PCP
CPCG-PCP
DCP
CCPaI-DCC
RCPev-DCL
CPCG-DCP
households
0,258
0.604
104
8840
0.00759
0.531
104
8164
scm20d
2.63
4.22
772
6409
0.0182
0.852
766
6012
rfl
0.667
1.35
340
10682
0.00836
0.348
339
9674
semld
2.27
3.57
1209
4971
0.0133
0.867
1205
4692
meps_21
0.236
0.607
581
6283
0.0123
0.261
581
6031
meps_
0.272
0.515
493
6411
0.0123
0.184
492
6128
meps_20
0.255
0.520
621
7147
0.0119
0.238
621
7032
house
0.315
0.594
1034
0.0159
0.327
1033
/
bio
0.630
1.22
3161
79422
0.0279
0.782
3163
63178
0.752
0.850
1119
41192
0.0336
0.155
1121
43289
calcofi
0.699
1.02
456
1.
0.0356
0.200
455
/
taxi
0.680
1.17
866
77828
0.0269
0.276
866
70139
19

---

Rectifying Conformity Scores for Better Conditional Coverage
B. Proofs
B.1. Proof for the first example
We provide here a completely elementary proof. The result actually follows from Theorem In this example, we set
T. (x) Q1 (P) where V V)., We assume that for all E X, (x) > 0. We denote V(x,, y)
(Zxx))/T(z)) and V = V)
Qi-aPP)) it e R: : Y) S (T,
a-
We will first prove that, for all x E X, we get that 1 = Q1 (Pv x . for all x E &::
Qi-a =in[(( E R P(V Y) 1T, (7+
2 1 x
=infft E R: Pvix= tQ--
1 a} =1
We then show that Q1 (Pv) 1 1. Indeed, for any 1, by the tower property of conditional expectation, we get:
P(V Y) - ST. (X)) PV v (QIxx
E TP V S (PV1-((Pvx)11))) 2 a
Assume now that Qi-o (Pv) < 1. Then for any E (Q (Q)-a (Pv), (Pv), 1), using again the tower property of conditional expectation, get
1 - s = EP((V (Q1-a(Pvix)(l)
(18)
= FE[Pixx((-a
<1
(19)
by the definition of the conditional quantile. This yields contradiction. Therefore, for Px- X,
(Pv) Qi-o
B.2. Proof for the second example
We set in this case (zz, 3) (zz,g) - (z), where (x) -1-aaPv)x= and V
V(X,Y) We will show that
Q1-a (Fvix=z)= = for all X. We have indeed:
Qi infsle R: P(V(X, 1/- x) x) 22111
(20)
-infft E R: P(V(XX))) S (X) + + = z) 2 21
(21)
We will now show that Q1- < O. Indeed, for all 0, by the tower property of conditional expectation and the definition of the conditional quantile, we get
((V S EIP(V(X, S <x(X)= + +|X]] 2 a.
(22)
On the other hand, assume Q1- (Pv) < 0. Set 8 E (Qi-a 0). We get
- a < (X (0 P) {x++
(23)
E(P(V (X < < <1-a.
(24)
which leads to contradiction.
We first show that Q1-a )P)) < 0. Indeed, by the tower property of conditional expectation, using again the definition of the conditional quantile, get
< = <1 a a,
(25)
which leads to contradiction and concludes the proof.
20

---

Rectifying Conformity Scores for Better Conditional Coverage
B.3. Proof of Theorem
We will now proceed with the proof of Theorem |
which verifies the marginal validity of our proposed approach. First, recall that V: X xx + R conformity score function to which apply measurable transformation (t,z) E TxX - ft (z). Recall that RCP constructs the following prediction sets for X'
{ y E EY: f7 ((z, v) S Q(1-0)(4409))
Cx(x)
For any & E (I.....n 1}, denote V = f(x)) V(Xx, Yk..
Theorem 3. Assume f1 12 hold, and let a E Hn+ + 13 1 1). If V..... Vx+1 are almost surely distinct, then it yields
- a-a s ca((Xn11) < - a 1
(26)
n+1
Proof. By definition, we have
P(Y++1 E (C(()))))== =P (Frx.4)) V(X++1 Yn+1) S
1 ozr))
(27)
= (V S Q(I-a)((an-)(( ( Ek=1 8v2))
(28)
Denote by Fe the cumulative density function of S7 mutually independent uniform random variables. Given a E [[n 1} -1 1), define
VVX Ya+1) and consider fUI....,UA, family of
h. = [m(1 + n )1 an]
Since by assumption 2 11-1, we have k. E (1....,n) n). Additionally, remark that V. has the same distribution that Fe (Uk). Therefore, by independence of the data, we can write
P (Pr-1 S Q(1-a)(ann1)( Σx= )b,))= = =P((F (U)))))
where U (n) denotes the order statistics. Additionally, since the scores Vi,... Va41 almost surely distinct, deduce that
P (F S F7 (U(R))) (Un+1 S U(..)) (U(Nm))
Since UKk.)) follows beta distribution with parameters (Ka: 1 ka), obtain that E [U(R_)] (n 1) ka.
B.4. Proof of equality D
Theorem 4. Assume H hold. For x E X, set Tz(x) (Pv.(Pv,ix=a)))) where Vo(a,g) Foy V(z Set (o( 9) = fr((a)) (z v) and V = Vx(XXY). Then, for all x E X,
s Q1-a (P = Qi-a
Proof. Set (xx) = Qi-a(Pv ). We must prove that x(x) = o for all & E X. First, we will show vb(x) < Note indeed
= = PVV (X)) S x/X .) P(V(X,X)) {=,,)0)))x
(2) P(V(X,)) [((()))
(29)
P V(X,)) [X] x) 2 a,
(30)
where (a) follows from f1(2) [t), (b) from the fact that fo is invertible, and (c) from the definition of (z)
21

---

Rectifying Conformity Scores for Better Conditional Coverage
Now, suppose that (b)) < s. Since for any I, J is increasing, we get that ((())) (p)) Moreover, using that T(x) belongs to the interior T, combined with the continuity of t E T fo (f); it implies the existence of & E T such that < T(x) and also ((()(((())) < fe(()). We can rewrite
I-asP(((x,)) S |X =
= s s(() Ix=x)
= - o v(X, s & x= < a.
which yields to contradiction.
We now show that Qi-a = sp We first show that Qi-a (Pv.) = 4p We first show that (P 5 This
follows from
( 2 a,
P(V,(X, & s) EIP(V - x XJ]
where (a) follows from the tower property of conditional expectation and (b) from Φ Qi-a (I for all X.
Assume now that Qı-a (Pv.)
sp Choose (PI-a(Pv.)..))) Then,
(6)
1 - a PVx E
a,
where (a) follows from the tower property of conditional expectation and (b) Φ Q1-QQ-aa- ( for all X. This yields to a contradiction which conclides the proof.
B.5. Proof of Theorem 2
This section is devoted to the proof of the conditional guarantee given in Section 6 In this section, denote V(x (e(e) (V ()) and for each /. E IR, we denote
= PV
and
(t(() P(((x)) st).
For any x E X, we assess the quality of the quantile estimate T(x) via
(x) =P (fz (V(x,)) S sf(x = xx) 1
For all n E N, note that a(1 - a)=t1 < n If 2 0.1 and n 2 100, then a(1 aa"-1 <
In addition, if 4183n
1
(p( a, then aLF; F (vp S 4183n'
Theorem 5. Assume that H2 hold. Assume in addition that, for any E X. Fy is continuous and Fvix== o FF is
L-Lipschitz. Then for a E [ n + +1 111 it holds
1 a aL ( PY E Ca(Xn+1) =z)
<1-a+e (1l- ( Fb(l))*
Proof. Let E IR be fixed. A first calculation shows that
P (VXX,)))) st) st) - (( = 2) Px Px(dx)
- (f.((a))) S +(x) X== Px(dr).
Now, we introduce the notation (x(x), which quantifies the discrepancy between substituting & with 4:
Ax(x) - (J. 8 s = - ())
22

---

Rectifying Conformity Scores for Better Conditional Coverage
Let's Po denote the distribution of the empirical quantile Q(1-0)(+==1)(+ E )%g). We can rewrite the conditional coverage as follows
P(Y Cx X = xx=)
P (V) Po(d()
p (V) xx= =z) Pa(dtt
P ( / Aa()))(((tt)
=1-a+6(0)) + / (o)
Moreover, consider a set of n i.idd uniform random variables {Ukhsksn and let U11) S ... I U() denote their order statistics. Since Vi,..., Vn are ii.dd, their joint distribution is the same (F (UI)....
(UI", (U,)))))((( Therefore, Pq is also
the distribution of the (1 + n (1 - q--anntiele of 1 Σx= Sp, ((U))) Thus, there exists an integer ka E {l,.... n2 such that
12
(HE ( "pgloss
(U (1-a))+t=1))
7
Moreover, using that (Vr: k [n]] almost surely distinct, deduce the existence of the minimal integer k, [n] such that
1 x E SU
1 # (1 (1-a0 a).
Since Ek=1 (U = ka almost surely, we deduce that ka = [(n + 1)(1 a)1. We also get that Fy (U(ka)) ~ Pq. In the following, we provide lower bound on (z). Since FVix=E is increasing, we can write
/ ((())))(dt) = E FFix= o Fe (U)))) F
(31)
Lower bound. First, using
31 implies that
f (A(())o((d)) 2 -E [L22F2'00 (D)))) {Fixx - Fxxx== F (V
+ ]
Moreover, by definition of the cumulative density function and its inverse, we have FDIx=r o PF' o Fy(()) S Fvix==()) Thus, it follows that
E 1 ((n)) x x - Fvjx -x o (U)((U(n)))) ]
S FFix=z(() - Fixx=r o FF Fy(()
+ E
1 1222, (U x (FrIX=S o FF o Fv(()) Fyxx=z (U
(32)
If (p) = 1, then Fvxx= (v) = 1 Px-almost everywhere, Let's now suppose that Fy(() < 1 and let's define p. = sup(t E R: (t) Fv (4)). For any 0, note that (y, + E) (v()).. This leads to
rē's o Folv. + = inf E R: (1(() 2 Fylve + $)} > p.
Furthermore, using the L- L-Lipschitz assumption on Fvix= Fyl implies that
0 < Fv (v) FiIx= o FT' (p())
< inf FFix= o Fr o Fo +)) - Fv x=x o FF Fo(0)}
c00+
< L inf (Fyle, +e) - Fr(p))
(33)
c00
23

---

Rectifying Conformity Scores for Better Conditional Coverage
From the continuity of F, we deduce that (p( (p.). Therefore, can conclude that lim inf inf (P + (F) O. This computation combined with (33 shows that xz (v) Fvix== o 1 (v((). Lastly, it just
remains upper bound the last term of 32 Once again, using that Fiix= Fe is Lipschitz gives
E [Z225" ()))
FFix= o FF o Fv(()) - Fvi=== FV (U(a))))
- LIE (Fv(P) - U((w))+]
Finally, applying Lemma with B = (P) and & = ka yields the lower bound.
Upper bound.
From 31
we deduce that
/ (()))(( = posp, (())) fF FFx== x=x o (U(R)) - 1 [
(34)
By definition of F we get v F o (v())) Since Fixx=r is increasing and Foix=z o Fr is I-LLipschitz it follows that
/ (x))) E 1 USFF (U(a))) x FFix== Fr (U(b)) - (1)
LE {U(Ko)) -F((1))+]
LE
- ((L-))))
Since the distribution of 1 ((R..) is the same that the distribution of U(n+I-Ru)> applying Lemma with B 1 - Fv((P) and A: = n + 1 hia yields the upper bound.
Let's denote by U(K) the keth order statistic of the i.i.d. uniform random variables UI...., Un. Lemma 1. For any 3 [0, 1] and l [n]. it holds that
(B - = g (1 k
EE
Proof. Let | E [0, 1] be fixed. For any (i,j) N22 define
I(i.j) = (e 3 n0 uddd..
By applying integration by parts for j 2 obtain
I(i,j) E!j (1+1)
I(i [i+ 1(i+15-1)
I(i +,1+5,0)
gi+5+1
(i + f)!
(i+j+1)1
Since UKa follows beta distribution with parameters (ki, n + 1 k), it follows that
E up) /88 (k - 1)1(n - k) 11 k un dl..
nl(3 u)
(35)
Furthermore, we have the following derivations:
3 u)u* 11 (11-))=-ka -u
=3 /'d u k- (1 - du 1 (3- u)ut(1 nku
31(k k) / (k: n k).
(36)
24

---

Rectifying Conformity Scores for Better Conditional Coverage
Lastly, combining 35 with 36
yields the result
nk - 1, - k) nlI -k)
(k: 1)!(n= k:)! (k 1)!(n k:). -3n+1 g k
B
(3 - U(A) B
+
For any B E [0, 1]. observe that
(1- 3)8+t8 v exp ((n + 1) log(1 (n + 2) 2)-
n + 2
Noting that log(1 - (n+ 2)-1) = Exz1 x- (n= 2)- we can show that:
+ (n+ (1-
1i x2+2 (1--t2==
n n+1
(n+ 2)
n n+2 +
S 1. k+
(n+
k20
Consequently, this implies that (1 -1-380 (3) < (n + cc
B.6. Pointwise control of E,
In this section, control the quality of the (1 a:) conditional quantile estimator r(z). To do this, recall that Vz(2, f/ V (x, 3) and consider the following error
x a
Moreover, in this section we denote by (1)a(()) the conditional -1-00 11-aa-qqantilee of (a Y) given X x. Theorem 6. For X, assume that Va Y) has a 1-st moment. for any / I. ((V,(2, || x) then
(x))
- (o)
Proof. Let x E X' be fixed. By definition of E,, we can write
E
[xx(x)]] = E
-1+
Moreover, for any t e R \ {0), it holds that
(t)a 1:50 1+a.
By extension, consider (0) 1. Hence, we get
e, (z) (V,((I, Y) S T
E [1v,(e x= 1+
For è IR, define the loss Co follows
(t) = (V
x]
Since (t) is convex with Lipshitz continuous gradient, applying Theorem 2.1.5 from (Nesterov 1998). it follows that
(C, -
C2 (10)) < 21 De. (41,40);
where L denotes the Lipschitz constant of C'g, and where Dc., is the Bregman divergence associated with C.. For to, t1 E IR the expression of the Bregman divergence is given by
De, (1,,40) = Lx(h)) - f.((to) (10) 1o).
25

---

Rectifying Conformity Scores for Better Conditional Coverage
Let to (1)a((), which represents the quantile. Given that (x, Y) has no probability mass (1-a(x), have f% ( (x)) 0. Moreover, by setting 1 ( observe that
(T(x)) s 21. x (La(T(x)) (x(()
Note that (T((() (x), therefore, the previous line shows
L. (x)] S 22 x L ((1
Finally, since the derivative of the Pinball loss function is 1-Lipschitz, it follows that I, S
In the following, denote for any R
FV,IX= (t) = ((X,))
V(X,Y) = st(x=
Moreover, let's denote by Fv, X=x an estimator of the cumulative density function FVGlx=x. For x E X, define
T(x) inf {1 E R: (0)x
Lemma 2. For x E X, assume that Fv, X= is continuous. Then, for any a E (0,1),
(e((x)] S FV,IX=r Fv.1x=xlloo
Proof. Let x be in X. Since Fv.1x== is supposed continuous, we have (v,(x=((T(z)) = a. Furthermore, using that 1, we obtain that
er (x) = FV,1X=x(
(x))
Fu,IX=x o F-1 Volx=n a)) - a a+1
FvoIX== o f-1 V xmm a) FV.IX=x o volx== (1 - a)
|IVGIX=== Fv.|x==rloo.
B.7. Uniform convergence of cumulative density estimator
For any k E [m], set V..A= = 1-- V( Yg). In the whole section, we assume that the random variables X XI,..., Xm are i.i.d. Therefore, the random variables Ti (x) = Ka, (l|æ xxll) defined for all l E [m] are mutually independent. Moreover, let's consider the empirical cumulative function given for x E X' and v R, by
Pe,x( xx)
Σ us Vo)ss
Theorem 7. H333 holds, then, it holds that
[ xx) - FF 2
<(v
Kille + supIeR. (M)K((t)) hx
2 log m
2Dnx (x)
+
mE (wxx (x)]²
Eet (x)
+ 4E ii (xx)
Var (ar(
m
where Dry (2) defined in (46)
26

---

Rectifying Conformity Scores for Better Conditional Coverage
Proof. Let E A' and E IR be fixed. First, recall that FV,Ix(0 2) (P(V s x). We will control
Foo1x(11 u) -F ur) as below:
Fr,I(0 (o)(01))- xx) -F) Σ (x)
{u I VoNes - PRox((o (x)}
kmn
Σ wx(2)P (Vo(Xss (2) su|xa) (P
(37)
We now apply several results demonstrated later this section:
Applying Lemma
3 shows that
P 2
(x) S mB mB (())
4 Var (x)] mB Riv [x)]
Applying Theorem 8 for any 7 E (0, 1), with probability at least 1 - y, it holds that
Σ vix (x)) { (V - F xaa] < V m FmmllKxlllas log (1/7).
sup eE
Applying Lemma 7 for any 7 (0, 1), with probability at least 7, it follows that
m Σ
m log(1/7)
(x(x)) { (x Xx) (u m)} S MDDnx (z) + sup (MtK((())
sup vER
SER,
2
where Dax (z) is defined (46).
Lastly, set y = m1 and remark that |Kay Iloc l Kr /l.00 Combining all the above bullet points with 37 with probability at most
implies,
(=)( that
+ mB me (x]]
m
(v
2 Iox
2 log m m
2 (x) En (x)
sup
Σ
Fv,x(011))
+ sup {MtK((t) (t) SER4
hx
Corollary 1. If H33 holds, then, it holds that
P [t)(1)] (V (V2 1lo + sup {MEK((()) EER
2 log m mBl uw (1:)²2
2 (x) Ewin (x)
+ 4E (ux (x)l Var (x)) (38)
m
where Dax (x) is defined in 46 and limax 00 Dax (x) =.
Proof. For E X, since F Vi is continuous, applying Lemma 2 with Theorem 7 implies that 38 holds. Moreover, calculation shows that
limsup lim sup Dax (x) S S1 1 (d1 x lim sup (1 1). hx 30
xx +0
Finally, by we know that [x((x))) < oo and Ja, - ( < oc. Therefore, it follows that
lim sup Dax (x)
27

---

Rectifying Conformity Scores for Better Conditional Coverage
The next result shows that k== (x) concentrates around its mean with high probability. Lemma 3. [x)) oc, then
4 [x)( mB ( (2:)
P 2 (x(x)) S mBB(u((>))
Proof.
Since the random variables X,.... X are i.i.d., using the Bienaymé-Tchhtee inequality, obtain
4 Var (E (x(x)) Th k== (x)
Varl (a:)
IP 2
Vi (x) I mBF(ux())))
mB (x)]
A=1
B.7.1. STEP I: INTERMEDIATE RESULTS FOR THEOREM 7
For any & E [m] and E IR, let's recall that (x:) = Kix (1|x0 Xxll) and let's define
G(v) (x) {1 ( -P (VeA
(39)
Theorem 8. Let X and y (0, With probability at least 1 - 7i the following inequality holds
Σ (2) {1 (Iionso FR 1 /mml Km 1l log (1/7).
sup VER
Proof. Let 0 and denote by {en]kemm) sequence of i.i.d. Rademacher random variables. The independence of {wx (r))))))m()))) implies that
II IE [cosh (O))x(x))) k=1
E [exp (aap(-Oric (~0mxx(x))) (2 [exp (0rr)( + 2 2-
E (exp (Deg()((x:))
k=1
For all IR, note that co)h(r) S exp(x2/2). Thus, deduce that
E lexp (0ee exp (2-100
Hence, the previous lines yields that
m
cosh (Orix (x))] I exp (2 (2-'m0?11) 1n02||Kn, (1:0)
(40)
A=
Set A > applying Lemma with defined in 39
gives
IP
sup {G(x))
A
< inf
0A
II
coosh (Bwi (2)a
(41)
(vER
850
km
Now, consider the specific choice of fm given by
A ml Krx
6n
11
Combining (40) with the expression of 8m. it follows that
< exp (-
2
inf
e
AA
E [cosh (Owx
(-m ml|KKx llae)
850
km
28

---

Rectifying Conformity Scores for Better Conditional Coverage
Therefore, combining with the previous inequality implies that
S exp (- (-mll
A ml| Kix 1100
IP
sup VER
[G(b)} 2 A
(42)
For any 7 E (0, 1), setting A Vml Kxx 20 log(1/7) gives
P sup (G(u)) < v /mllKK loo log(1/7)
21-1
11-7 -
(43)
VER
The following statement controls P(sup (G(b)) c). Its proof is similar to the extension of the Dvoretzky-Kieet Kiefer- Wolfowitz inequality provided Appendix (Plassier al. 2024).
Lemma 4, For any A > 0, the following inequality holds
P
sup [G(e)} 2 A)
2 0>0
inf
04 coos (0ri (x)) j
LVER
where G is defined in
39
Proof. First, for any O, applying Markov's inequality gives
P
sup {G(b)} 2 2 s se 1o-0AB
exp
sup (G(w))) VER
(44)
AER
Moreover, Lemma shows that
Y12
E
exp
sup (G(O)) veR
[cosh (0)x(()))
Plugging the previous inequality into 444 and minimizing the resulting expression with respect to yields:
P sup VER
[G(x)} 2 )
inf 030
cosh (0wl (z))
Lemma 5. Let we have
)]
E
exp
sup (G(w)) vER
<
E ccosh (0wx (x))]
Amm
Proof. Let 0 > 0 be fixed, since + Øt is continuous and increasing, the supremum can be inverted with the exponential:
E
exp
sup
HEe (C(O))
suppexs
sup exp (OG(v))
LeeR
For any & [m], consider Vz an independent copy of the random variable Ye, and denote Vo.. = V., Yz). The linearity of the expectation gives
Σ (x) (1 (IV - E Km
B M (xe)) = E
Σ wu (x) (1r I (Xx. [Kx, Yal
29

---

Rectifying Conformity Scores for Better Conditional Coverage
Therefore, the Jensen's inequality implies
(Xx: ya) 1)
E
exp
Oup oRR
(C(o)))
E
exp
OE
Σ (π((x)
{ (I 1 Punsso
PE
(π((x) (I
1l (erass))
E
sup exp
veR
Let {cR [eem) be ii.d. random Rademacher variables independent of {(Xx, YR, Yx) Since 1 Vo,eso 1 Pe is symmetric, we have
wh (x) { (IR.ASs 1 vensse)) se)s [
c (x)x { (Irees - Aennset)
E
sup exp
= E
sup exp vER
A=1
1=1
Using the Cauchy-Schwarr inequality, we deduce that
Osup (G())) VER
c vensor)
E
exp
S E
sup exp
20
VER
Given the random variables (Vo.)mm im holds
denote by a the permutation of [m]] such that (i,0(1)) S ... S V p.((m)) In particular,
if ()))
m
(,)x(())1
(o j=
if V (oo Su < (::(++1))
if 2 V V))mm))
Thus, can rewrite the supremum as
sup cxp
28 Σ
(x)
V
sup cxp Usisn
28 >
Sa #))))))))()))
vE
Applying Lemma 6 we finally obtain that
E
sup exp vER
20
T (x) (z) VO
(Kkr Yal
1=
)
E
sup osism
exp
20
col (()))
(Xx, Vali
< 2 II
cosh (f))(()).
k=1
Lemma 6. Let (Ei)ie()) be i.idd Rademacher random variables taking values in {-1,1), then for any 0 > and (pshiemm) E Rm, we have
E
exp
0 sup Σ Pitj usism
< 2 II
cosh (0px).
j=1
k=1
By convention, we consider Ej PjEj
O.
B.7.2. STEP 2: INTERMEDIATE RESULTS FOR THEOREM 7
For all X and E R, define the conditional cumulative density function Fv, x (v xx) as
(Vo(X,Y) s
( 1 a:)
30

---

Rectifying Conformity Scores for Better Conditional Coverage
Moreover, recall that we denote by fx the density with respect the Lebesgue measure of the random variable X. Using the spherical coordinates, we write by F2.0 (f 01; sin f cos 02................... ...... tsn 01 ... sin 0a-1) the coordinate of # E X, where | t. Additionally, we define
d-2
Fx(t, = 1 (0.0)a-2x1002m)
fx - T1,0)
II sin( d01 d..0--1-
(45)
Note that
1 rd Fx x))tt / fx(x 2)dē = 1.
B4
Under 13 the cumulative density function x. + V (v x)) is M-Lipschitz. In this case, for any hx >> 0, let's consider
Dax (x) = F x / p (i)dk.
(46)
Lemma 7. Assume that H3 holds and les 7y E 10, With probability at least 1 - 7 it holds
(x) FR,I Xx) F.,1x((1 m)) s mDDx (2) + sup (M))((t)) IER4
m ml0g(1/7)
2
vER
Proof.
First of all, using H3 immplies that
m
(x) { (F (xx) - F 12)
Σ th (x) min {1, Xxll).
sup PE
For every E [m]. let's consider Zz = (x(x)) min M||x - Xxll}. Since TH Kix (1|= Xxll), we have
2x S max
sup 00MM51
[MtKax (t))
sup {Kax (K))( Mcs1
(47)
By caleulation, get
1
sup DSME1
{MLK (1) M sup
Ki
M
sup
{IK) (1))
(48)
OSMES1
hx
hx
(k)
We also have
1 Ki
t
)
t Ki
}
sup Mc=1
(Kax (1)
sup
M sup
M
sup
{1K) (I))
(49)
MC>1
hx
hx
Mt21
hx
hx
M)-
Thus, combining
47
48
with 449
yields
<
Z4 < M sup {tK, (t)}.
LER_
Applying Hoeffding's inequality, for any / > 0, it follows
2(1 mBZ))²
P
7+ EZx) 2 t mEZ,
S exp
(50)
mSsuPEE,, (M]K(())
Let E (0, 1) and set:
log
ty mEZ, + sup {Mt (8)
tER
Using
50 it holds that
P
Σ Am/
1
EZt)
2t, mEZ/ 37
31

---

Rectifying Conformity Scores for Better Conditional Coverage
We will now bound b.. To do this, will control 1
E (x) min (1,M]I Xx /
(M 1) Kax (1]x1)/x(x =))d..
(51)
Jzex
Using the spherical coordinates, a change of variables gives
[M | A 1) Kax (Il=l)/xx( x)dē I (2 (M
REX
==
where (x(t, x) is given in
Therefore, immediately follows that
J
{
00 (=xK((0)x((xxx000))
Jeex
[M A 1) Kax (llxl)xx( d)dE S 1xx
Plugging the previous bound in 51 shows that
EZ S Dax (z);
where Dax (2) provided in (46).
B.8. Asymptotic conditional validity
Theorem 9. Assume that H2 H3 hold, and let m be of the same order as If (D & {0, 1} and for every X.
Fiix= Fr is Lipschitz and Ix is continuous, then, for E [[n + 11-1,11 1) and p 0, it follows
lim lim P (IP E Cx( X) 1 al S p) =1 Ax 2x-
Proof. First, let's fix a E [[n + 1)-1,1) and > 0. Our proof is based on the following set:
An { r E X: Pvix=2 o Fæ' is 4 p(Fv(()) A [1 F Lipschitz]
This set contains every point x E A' whose Lipschitz constant of F Vix== F is smaller than certain threshold which tends to oo as n + cc. Let's also define the two following sets
/2 a0 + SUPIER, {MAK(
2 log
2Dxx (x) Chx (z) 2
Bm,hx
XEX:
Chx (x)
m
2D (x)
Boohx DE X:
Chx (x)
2
Lastly, for all consider
Em,hx = {x X: 24 + 4Chx(x)
Var[Knx (1 X(I) < mr),
G fI X: fx(x) 2r}.
Using basic computations, obtain the following line
P (Y e (x) - + > p) P & An n Bm,xx n Em,rxi E G) FF G) + (Y e (X X) 1+ p; e An Bm,hx Em,hx.) (52)
Since m is of the same order n, which means that 0 < lim inf m/ < lim sup m/n n co, it holds
LxgAn Pamm F.m.i 1xec
n 1xgB 1XEG-
Moreover, since Kay is an approximate identity and fx is continuous and bounded, we have limay Chy (x) = fx(x). As stated in Proposition [ it also holds that limpy--o Dix (I) = 0. Therefore, it follows
1x 1 1xeG 4x
0.
32

---

Rectifying Conformity Scores for Better Conditional Coverage
Using the dominated convergence theorem, it yields that
lim sup lim sup (X An Bm.xxx D Em,xxi G)
(53)
-x
n-+00
Given realization X, denoting by Lg the Lipschitz constant of Fvix= F the application of Theorem 2 that
shows
(V (x(( - a > X E A, 0 B,,xx O Fm,hx)
SP c, x)) 21, (F(())) (o( An Bm,xxx Frm.
Since X An. we deduce that Lx - 4 1 [1 Pp(l)] and thus yields that 2Lx [FV())]
(p))))]+) s 2 p. Therefore, it follows
p (X)] 2Lx (F(())) v Pp(p)) X An Bm, n Em,,
S P (2 < (X)( E An Bmixx n Em,xx).
Since x E By,hx O Em, applying Proposition 1
gives that
P(2-'p (e(x) E An n Bm, Em,,i ST.
Equation 52
implies
P ((IP (Cx())))))1 SF & An n Bm, n Embx) $ G) +r.
Lastly, 53
combined with the previous inequality shows
lim sup lim PP (IP (Y (x) - a p) SE [ [ tr.
hx -00 n-
As is arbitrary fixed, from the dominated convergence theorem we can conclude that
lim sup sup P (IP (Y Ca(X) x) 1 1 al p) = 0.
hx -10
n-+00
33

---

Rectifying Conformity Scores for Better Conditional Coverage
C. Details on Experimental Setup
This section aims to provide additional details on our experimental setup and implementation the RCP algorithm.
Models. To facilitate fair comparison of different uncertainty estimation methods, we assume that the base prediction models are already trained. We focus on the regression problem and aim to construct prediction sets for these vre-trained models. All our models are based on fully connected neural network of three hidden layers with 100 neurons each layer and ReLU activations. We consider three types of base models with appropriate output layers and loss functions: the mean squared error for the mean predictor, the pinball loss for the quantile predictor or the negative log-likelihood loss for the mixture predictor. Training performed with Adam optimizer.
Each dataset is split randomly into train, calibration, and test parts. We reserve 2048 points for calibration and the remaining data is split between 70% for training and 30% for testing. Each dataset is shuffled and split 10 times to replicate the experiment. This way we have 10 different models for each dataset and these models' prediction are used by every method that is tailored to the corresponding model type to estimate uncertainty. One fifth of the train dataset is reserved for early stopping.
RCPMLP. This variation reserves part (50%) of the original calibration set to train quantile regression model for the (1 - -leeeel quantile of the scores V. We again use three hidden layers with 100 units per layers for that task. The remaining half the calibration set forms the "proper calibration set" and is used to compute the conformal correction.
the same splitting of the available calibration data. Since only one bandwidth needs to be tuned, use simple grid search on log-scale grid in the interval [10-3,1]
RCP The local quantile regression variant is similar to the previous one,
Datasets. Table 6 presents characteristics of datasets from TToumakas al. 2011 Feldman al. 2023 Wang al.
2023), restricting selection to those with least two outputs and total of 2000 instances. For data preprocessing, follow the procedure of (Grinsztajn et al. 2022).
Paper
Dataset
p
Tsoumakas et al. (001)
scm20d
8966
60
16
rfl
9005
64
rf2
9005
64
semld
9803
279
16
Feldman al. 2023
meps
15656
137
meps_19
15785
137
meps 20
17541
137
house
21613
14
bio
45730
blog data
50000
55
Wang et al. (2023)
taxi
50000
Table 6: List of datasets with their characteristics.
34


Reviews:
[{"review_id": "FwonQRKya6", "reviewer": "Reviewer_AjXf", "summary": "This paper introduces Rectified Conformal Prediction (RCP), a novel framework for improving conditional coverage in conformal prediction while preserving exact marginal validity. The key idea is to learn a transformation of the conformity score such that the ($1-\\alpha$)-quantile of the transformed score becomes covariate-independent. This is done by estimating the conditional quantile of a transformed conformity score and applying a local re-scaling (or shifting) to normalize variability across the feature space. The authors provide theoretical guarantees on marginal and approximate conditional validity, demonstrate the flexibility of the framework across multiple conformity scores and predictors, and show empirical improvements over state-of-the-art methods across synthetic and real-world regression datasets.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the thorough and constructive feedback. Below, we address your questions:\n\nQ1: **RCP's conditional coverage guarantee hinges on accurate estimation ...** Even if the quantile regressor is misspecified, RCP’s conformal calibration guarantees valid marginal coverage by construction. Nevertheless, poor quantile estimates affect conditional coverage: underestimations yield local under-coverage, whereas overestimations produce overly conservative intervals. Such failure modes highlight that RCP inherits biases from quantile regression, reducing conditional efficiency despite correct marginal coverage. This sensitivity was illustrated in our synthetic experiments; additional examples and analysis will be provided.\n\nQ2: **RCP estimates a single...**   RCP computes quantiles of a nonconformity score and thus only concerns with the right tail of the distribution (when the nonconformity score gets large). Therefore, obtaining intervals with lower and upper quantile estimates is not typically needed. Note that for a one-dimensional prediction target, we can, of course, work with CQR-type interquantile intervals as a specific nonconformity score.\n\nQ3:  **How restrictive are the assumptions about monotonicity and invertibility of the transformation functions?** The monotonicity is essential for our approach as we insist on preserving the ordering of the conformity scores. After adjustment by a function with a fixed argument $\\varphi$, a \"large\" nonconformity score must remain larger than a smaller one. Invertibility is a technical requirement that underlines our construction, but it is not as restrictive as the monotonicity assumption. \n\nQ3: **Can your method...** the answer is yes.  As an example, the adjustment function $f_t(v) = t + v$ does not restrict the range of scores. Specifically, it does not impose positivity or negativity constraints, thus naturally accommodating scenarios where the score can assume both negative and positive values. This flexibility is crucial for handling general scoring functions, ensuring applicability across a broad range of prediction tasks without additional transformations or restrictions.\n\nQ4: **The bound in Theorem 4 depends on...**  The Lipschitz property of the quantile function is standard in the statistical literature on conditional quantile estimation — see, for instance, the works [1,2] on this topic. It is difficult to delve into such subtle technical conditions in a short discussion. We will include a more substantial discussion of these contributions in the revised version.\n\n[1] Y.K. Lee, E. Mammen, B. U. Park. \"Backfitting and smooth backfitting for additive quantile models.\" 2010.\n\n[2] M. Reiß, Y. Rozenholc, C. Cuenod. \"Pointwise adaptive estimation for robust and quantile regression.\" arXiv:0904.0543. 2009.\n\nQ5: **Have you evaluated the size...** Thank you for your suggestion. You're absolutely right — because RCP can generate larger sets for test instances with higher uncertainty, the resulting sets tend to have wider volume on average.\n|dataset|PCP|RCP-PCP|DCP|RCP-DCP|ResCP|RCP-ResCP|\n|--|--|--|--|--|--|--|\n|scm20d|4.74e+07|1.01e+08|5.88e+06|1.16e+08|**2.50e+06**|6.20e+12|\n|rf1|77.5|1.27e+03|**1.87**|6.87e+02|9.84|1.84e+08|\n|scm1d|3.06e+06|2.17e+09|3.24e+06|1.00e+09|**1.47e+05**|7.17e+16|\n|meps_21|2.31|4.41|**1.55**|2.40|5.73|6.82|\n|meps_19|5.82|6.07e+03|**1.97**|6.71e+03|5.34|6.09e+06|\n|meps_20|2.35|3.97|**1.48**|2.53|5.49|6.16|\n|house|2.20|2.51|**1.89**|2.05|6.51|8.01|\n|bio|0.823|1.05|**0.584**|0.630|1.14|1.38|\n|blog_data|2.94|1.40e+05|**1.50**|1.80e+05|1.74|3.71|\n|taxi|10.5|10.8|**6.94**|7.38|12.4|12.8|\n\nHowever, when looking at the median volume, avoiding outliers, RCP outperforms baselines:\n\n|dataset|PCP|RCP-PCP|DCP|RCP-DCP|ResCP|RCP-ResCP|\n|--|--|--|--|--|--|--|\n|scm20d|2.91e+06|**1.44e+06**|5.26e+06|3.26e+06|2.50e+06|1.37e+07|\n|rf1|36.0|5.21|1.78|**1.09**|9.84|2.94|\n|scm1d|2.72e+05|3.14e+06|2.57e+06|7.60e+05|1.47e+05|**3.36e+04**|\n|meps_21|1.72|1.27|1.24|**1.03**|5.73|2.40|\n|meps_19|2.60|1.22|1.32|**0.967**|5.34|2.43|\n|meps_20|1.75|1.26|1.12|**0.968**|5.49|2.47|\n|house|1.99|1.82|1.70|**1.60**|6.51|6.38|\n|bio|0.717|0.689|0.530|**0.511**|1.14|0.927|\n|blog_data|1.59|1.81|1.14|**1.08**|1.74|1.30|\n|taxi|9.97|9.18|6.63|**6.24**|12.4|10.6|\n\nWe will report these metrics in Section 7.2.\n\nQ6: **How sensitive is performance...** We discuss the sensitivity with respect to the choice of adjustment function in Appendices A.3-4. The choice has a significant influence on the results, as the ability of fixed single-parameter functions to rectify the results highly depends on the properties of the score distribution. An interesting future work is to design more flexible data-dependent adjustment functions. Sensitivity with respect to quantile regression is discussed in Appendix A.2, showing that a better fit of the quantile model improves the results.", "claims_and_evidence": "Most of the claims in the submission are supported by clear and convincing evidence, particularly the theoretical guarantees for marginal validity and approximate conditional coverage. The empirical results convincingly demonstrate improved conditional coverage across a range of regression tasks, validating the main claim that the proposed rectification improves local adaptivity. However, the paper lacks evaluation on coverage–efficiency tradeoffs.", "methods_and_evaluation_criteria": "1. RCP's conditional coverage guarantee hinges on accurate estimation of $\\tau^*(x)$. How sensitive is RCP to misspecification of the quantile regressor? Could you show examples where quantile regression underperforms and discuss failure modes?\n\n2. RCP estimates a single quantile per point. How would RCP handle two-sided intervals (e.g., $[Q_{\\alpha/2}(x), Q_{1 - \\alpha/2}(x)]$)?", "theoretical_claims": "1. How restrictive are the assumptions about monotonicity and invertibility of the transformation functions? Can your method handle scores with negative values (e.g., log-likelihoods) without heuristic adjustments?\n\n2. The bound in Theorem 4 depends on Lipschitz continuity of the quantile mapping. Can you elaborate on how often this assumption holds in practice? Can you provide empirical values of the bound components?", "experimental_designs_or_analyses": "1.  The paper lacks evaluation on coverage–efficiency tradeoffs. Do RCP sets tend to be wider due to more cautious calibration?\n\n2. The framework requires choosing a transformation $f_t$ and tuning quantile regression models. How sensitive is performance to these choices? Could you provide ablations?", "relation_to_broader_scientific_literature": "The paper builds on and extends works in conformal prediction, particularly methods aimed at improving conditional coverage. This paper proposes a transformation-based approach inspired by recent work on score adjustment and local calibration. It is closely related to conformalized quantile regression in that both seek to adapt prediction sets to local data properties, but RCP generalizes this idea by applying a trainable transformation to arbitrary conformity scores. While prior methods address heterogeneity via weighting or region-specific coverage, RCP’s novelty lies in aligning the conditional and marginal quantiles of transformed scores, thereby offering a new perspective on achieving conditional validity without relying on density estimation or rigid group partitions.", "essential_references_not_discussed": "The paper cites and discusses a wide range of essential related works, including classical methods for marginal coverage, approaches for approximate conditional coverage via stratification or grouping, and more recent developments.", "other_strengths_and_weaknesses": "Strengths:\n1. The rectification strategy is modular and applicable to a variety of conformity scores and models.\n\n2. The authors derive meaningful guarantees on conditional coverage as a function of quantile estimation error.\n\n3. The paper evaluates on diverse multi-output regression datasets, including synthetic setups and real-world benchmarks.\n\n4. The paper is mostly well-written and easy to follow.", "other_comments_or_suggestions": "This paper uses lots of math notations. I suggest the authors summarize the math notations in a table.", "questions_for_authors": "1. RCP's conditional coverage guarantee hinges on accurate estimation of $\\tau^*(x)$. How sensitive is RCP to misspecification of the quantile regressor? Could you show examples where quantile regression underperforms and discuss failure modes?\n\n2. RCP estimates a single quantile per point. How would RCP handle two-sided intervals (e.g., $[Q_{\\alpha/2}(x), Q_{1 - \\alpha/2}(x)]$)?\n\n3. How restrictive are the assumptions about monotonicity and invertibility of the transformation functions? Can your method handle scores with negative values (e.g., log-likelihoods) without heuristic adjustments?\n\n4. The bound in Theorem 4 depends on Lipschitz continuity of the quantile mapping. Can you elaborate on how often this assumption holds in practice? Can you provide empirical values of the bound components?\n\n5.  Have you evaluated the size of the prediction sets produced by RCP compared to standard CP or CQR? Do RCP sets tend to be wider due to more cautious calibration?\n\n6. The framework requires choosing a transformation $f_t$ and tuning quantile regression models. How sensitive is performance to these choices? Could you provide ablations?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7v6DsGo85j", "reviewer": "Reviewer_Vi6b", "summary": "This paper introduces Rectified Conformal Prediction (RCP), a novel method for improving conditional coverage in conformal prediction while maintaining exact marginal coverage. The core idea is to transform conformity scores in a way that aligns their conditional quantiles across different covariates. This transformation is achieved by estimating the conditional quantile of conformity scores and using it to rectify the scores before applying the standard conformal prediction procedure. The authors establish theoretical guarantees for the proposed method, including a lower bound on conditional coverage that depends on the accuracy of the quantile estimate. The paper also presents experimental results demonstrating that RCP outperforms existing methods in achieving improved conditional coverage while retaining valid marginal guarantees.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the thorough and constructive feedback, which helps improve our manuscript. Below, we address your valuable points:\n\n**Effectiveness of quantile estimation in high-dimensional settings:**\nIndeed, quantile estimation accuracy critically affects RCP performance. In our experiments, we selected neural networks and local quantile regression specifically due to their scalability to higher dimensions. However, we acknowledge that our explicit evaluation in very high-dimensional regimes remains limited. Following your suggestion, we will include additional discussions and empirical results to better highlight performance and trade-offs in higher-dimensional settings.\n\n**Choice of the transformation $f_t$** There are numerous possible choices for the function \\\\( f_t \\\\). Thus far, we have always restricted ourselves to relatively simple transformations, often inherited from the literature on \"normalized conformity scores,\" which we recognize should have been more thoroughly credited. We believe it is important that the proposed method remains simple and does not rely on hyperparameters, ensuring that the computational cost of this wrapper stays reasonable.\n\n**Additional experiments with challenging multivariate distributions:**\nWe appreciate your suggestion for extending experiments to more complex multivariate distributions, as this would further reinforce our empirical validation. However, our current synthetic and real-world examples demonstrate clear advantages, while the datasets have dimensions up to 16. In our revised submission, we will incorporate an experiment that highlights RCP’s behavior on more challenging multivariate datasets.\n\n**Impact of incorrect quantile estimation:**\nWe agree that understanding the impact of incorrect quantile estimation beyond synthetic noise is crucial. To address your comment, we plan to include a detailed analysis showing how estimation errors affect coverage in more realistic settings, thereby providing deeper insights into RCP’s robustness and practical utility.\n\n**Extension to sequential settings:**\nYour question regarding sequential adaptation is insightful. While we had not previously explored this application, we find it highly promising. There are no conceptual or methodological difficulties; however, from a theoretical standpoint, everything remains to be developed. RCP’s framework naturally extends to online learning by sequentially updating the quantile estimator based on newly observed data. We envision future work that formally explores sequential conformal adaptations and briefly outline such potential directions in our revision.\n\n**Clarification of minor points:**\n- We will correct inconsistent notation for  \\\\(\\\\hat{\\\\tau}(x)\\\\) throughout Section 4.\n- The Lipschitz constant \"L\" in Theorem 4 will be explicitly defined earlier to improve readability.\n- Figures 3 & 4 will indicate dataset sizes in the revised manuscript for enhanced clarity.\n\nWe appreciate your recognition of RCP’s conceptual novelty and theoretical rigor and your acknowledgment of our thorough experimental evaluation. Your suggestions significantly strengthen the manuscript, and we will diligently incorporate these improvements.", "claims_and_evidence": "The paper provides a well-structured theoretical justification for its claims. The derivation of the conditional coverage bound appears mathematically sound, and the authors clearly articulate how their method improves over traditional approaches. The empirical validation is extensive, comparing RCP against multiple existing conformal prediction techniques across synthetic and real-world datasets. However, while the paper provides strong empirical evidence, the effectiveness of the quantile estimation technique is not thoroughly explored in more complex, high-dimensional settings. Additionally, the impact of the choice of transformation function ft on different types of datasets could have been analyzed in more depth.", "methods_and_evaluation_criteria": "Yes, the benchmark datasets used in the experiments are well-chosen. The paper includes synthetic datasets to illustrate theoretical properties and real-world regression datasets to validate practical performance. The use of worst-slab coverage and conditional coverage error as evaluation metrics is appropriate for measuring improvements in conditional validity. However, additional experiments with more challenging multivariate distributions could have further strengthened the empirical evaluation.", "theoretical_claims": "The theoretical results presented in Section 6 appear to be correctly derived. The proof of Theorem 3 for marginal coverage follows standard conformal arguments. The bound on conditional coverage (Theorem 4) correctly incorporates the accuracy of the conditional quantile estimate, and the derivations align with known results from quantile regression literature. However, I did not rigorously verify all steps in the Appendix proofs.", "experimental_designs_or_analyses": "The experimental setup is methodologically sound:\n\nThe authors compare RCP against multiple state-of-the-art conformal methods (e.g., ResCP, PCP, SLCP, DCP).\nThey use a standard train-validation-test split and ensure calibration data is separated from test data.\nThe quantile estimation methods (neural networks and local quantile regression) are well-justified.\nThe choice of transformation functions for conformity scores is systematically varied.\nOne concern is that the effect of incorrect quantile estimation on performance is not fully explored beyond the toy example. Understanding how estimation errors affect real-world datasets would be crucial for deployment in practical applications.", "relation_to_broader_scientific_literature": "The paper builds upon the conformal prediction framework, particularly methods that aim to approximate conditional validity. Prior work has either:\n\nPartitioned the covariate space (leading to inefficient large prediction sets) or\nReweighted empirical distributions (which struggles in high dimensions).\nThe paper’s key novelty is the idea of rectifying conformity scores through a learned transformation, making conditional quantile estimation more tractable. This idea is conceptually related to:\n\nConformalized Quantile Regression (CQR) (Romano et al., 2019).\nLocalized Conformal Prediction (Guan, 2023).\nCompared to these works, RCP introduces a more flexible and computationally efficient alternative that does not require explicit conditional density estimation.", "essential_references_not_discussed": "The paper does a good job of referencing key works in conformal prediction, including classical results (Vovk, 2005) and recent advances (Angelopoulos et al., 2023). However, some more recent studies on uncertainty quantification could provide additional context:\n\nTraining-Conditional Coverage Methods (Bian & Barber, 2023) discuss techniques that could potentially be adapted into RCP.", "other_strengths_and_weaknesses": "Strengths:\nThe conceptual novelty of rectifying conformity scores is a valuable contribution to conformal inference.\nThe theoretical guarantees are rigorously derived and provide a meaningful lower bound on conditional validity.\nThe experiments are thorough, with comparisons across multiple datasets and methods.\nThe approach is computationally efficient and avoids the pitfalls of full conditional density estimation.\nWeaknesses:\nThe quantile estimation step is critical to the method, but the authors do not explore the trade-offs between different estimation strategies in high-dimensional settings.\nThe impact of outliers in score rectification is not well analyzed.\nThe choice of transformation function ft is somewhat arbitrary, and more discussion is needed on selecting appropriate transformations for different problem domains.", "other_comments_or_suggestions": "Section 4: \"tau(x)\" is sometimes written inconsistently.\nSection 6, Theorem 4: The notation \"L\" for Lipschitz continuity should be explicitly defined earlier.\nFigures 3 & 4: Labels should include dataset sizes for better context.", "questions_for_authors": "How does RCP perform when the conditional quantile estimator is misspecified? The toy example considers synthetic noise, but what about real-world miscalibration?\nWhat is the computational cost of different transformations? Are some ft transformations significantly more expensive than others?\nCould the framework be extended to sequential settings? For example, how would RCP adapt in online learning scenarios?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "c1cblviZm2", "reviewer": "Reviewer_R3jf", "summary": "The paper considers the problem of producing conformal prediction sets with conditional guarantees. The idea is to rectify non-conformity scored by to use additional hold out data to fit a quantile regressor that is then applied to the non-conformity score. Marginal coverage guarantees are obtained using the remaining part of the data to run split cp on the rectified nc scores.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We acknowledge the critical feedback and aim to address the raised points. Below we clarify why **the undiscussed references, while indeed valuable, mostly address aspects different from the specific problem we focus on**. The following discussion highlights the distinctive advantages of our RCP method for constructing confidence sets in multivariate prediction, offering exact marginal coverage and strong theoretical guarantees for conditional coverage.\n\n**Discussion of the references provided in the review.**\n- Among the papers cited, Gibbs et al. (2023) is the one most directly related to our work, as it acts as a wrapper on given conformity scores. It has already been discussed, but additional comments and experiments are worthwhile. From a computational point of view, their method (called CPCG below) is very intensive, mainly because the wrapper uses a form of the \"full conformal\" idea, which requires solving an optimization problem at test time. Its strength lies mainly in finite-sample guarantees tailored explicitly to specific feature subsets and covariate shifts (see Theorem 2 for finite-dimensional class). This targeted approach suits group conditional coverage (Corollary 1) well. In contrast, RCP uses a split-conformal idea that estimates a quantile function on a special subset of the data, which greatly improves computational complexity. Also, for RCP, we provide a more generic theoretical result that is agnostic to a particular estimation method used (Theorem 4), and also specify it for the case of local quantile regression (Proposition 5). The proof of the results is not straightforward and requires the usage of non-trivial technical tools such as very recent extensions of DKW inequality (see Lemma 16).\n\n- Cherian et al. (2024) is a CP framework tailored for LLMs. The method is not easily applicable to conventional regression or standard multivariate classification scenarios, where more broadly effective methods like RCP are more natural.\n\n- Xie et al. (2024) refine conformity scores via gradient boosting to enhance conditional coverage while achieving exact marginal coverage in univariate prediction models. This method is constrained to scalar outcomes and lacks a natural extension to multidimensional settings. Further, the lack of analytically interpretable theoretical guarantees on conditional coverage and reliance on numerous hyperparameters and differentiable approximations complicates both theoretical understanding and practical implementation.\n\n- Kiyani’s (2024) CPL method combines conditional validity and optimized efficiency through constrained optimization, designed exclusively for univariate prediction intervals. While CPL provides exact coverage tailored for a specific class $\\mathcal{F}$, it is fundamentally limited by its complexity. Specifically, the reliance on intricate optimization procedures limits broader practical application, especially for multivariate outputs. In contrast, RCP’s computational simplicity and broader versatility in handling multivariate scenarios position it distinctly ahead.\n\nThus, among these methods, RCP clearly emerges as superior, particularly in multivariate predictive contexts. It balances precision in conditional coverage with practical simplicity, computational efficiency, and broader applicability.\n\n**Empirical comparison with CQR and CPCG.** We conducted an additional experiment to directly compare RCP and CPCG (see https://pdfhost.io/v/26gN4fUeS2_rebuttal-R3jf).\nWe observe that CPCG and RCP give similar conditional coverage, while RCP is at least two orders of magnitude faster. Compared to CQR, it trains a quantile regression model directly on the whole training set, while RCP can be applied to any black-box model and does not require access to the training data or internal model structure. Figure 10 shows that RCP can match or outperform CQR in a multidimensional setting.\n\n**Technical novelty.**\nThe **technical novelty** of RCP lies in its approach to enhancing conditional coverage through the concept of \"rectifying\" conformity scores. Unlike conventional methods requiring estimating the entire conditional distribution for multivariate predictions, RCP simplifies the problem by calculating only the conditional quantile of a univariate conformity score. This quantile estimation is a wrapper around classical methods explicitly tailored for multivariate prediction sets. Furthermore, RCP provides explicit theoretical lower bounds on conditional coverage, directly linking prediction accuracy to the quantile estimation approximation error. These results are based on careful and non-trivial analysis as discussed above. **While competitive methods exist for univariate predictions, such comparisons are irrelevant, as univariate scenarios are explicitly not our targeted application**. Thus, our proposed RCP method represents a **meaningful advancement**, combining clear theoretical foundations with practical efficiency for multivariate prediction tasks.", "claims_and_evidence": "The paper claims to improve conditional coverage compared to existing schemes; however, as shown in the Appendix the proposed method has very similar performance to well established methods as CQR and it is not compared to methods such as those presented in  Conformal prediction with conditional guarantees by Gibbs, I., Cherian, J. J., and Candes, E. J., and Boosted Conformal Prediction Intervals by R Xie, R Barber, E Candes.", "methods_and_evaluation_criteria": "Yes, benchmarks make sense.", "theoretical_claims": "The theoretical claims appear to correct as they follow from the properties of split cp which is applied to the rectified scores. What are the technical challenges and novelty that the current analysis brings?", "experimental_designs_or_analyses": "Yes, experiments are correct. However I believe that other baselines (CQR) should be either moved to the main text and additional ones considered (such as those mentioned above based on boosting, conditional coverage and conformal training should be included.", "relation_to_broader_scientific_literature": "I think the paper does a good job at highlighting similarities with existing literature; however I believe it misses on discussing conformal training methods.", "essential_references_not_discussed": "The idea of optimizing the non-conformity score for improved efficiency and conditional coverage is common in conformal training and CP length optimization. See:\n\nLarge language model validity via enhanced conformal prediction methods by John J. Cherian, Isaac Gibbs, Emmanuel J. Candès\n\nKiyani S, Pappas GJ, Hassani H. Length optimization in conformal prediction. Advances in Neural Information Processing Systems.\n\nR Xie, R Barber, E Candes, Boosted Conformal Prediction Intervals", "other_strengths_and_weaknesses": "I think the idea is simple, but that is not necessarily a weakness of the paper. However, I had trouble understanding the benefits and novelty of the proposed scheme compared to existing methods such as CQR, conformal training, and conditional coverage methods. This concern arises from the fact that these methods are either not benchmarked or, in the case of CQR, have similar or superior performance to the proposed one (e.g., in Figure 9, CQR has the same conditional coverage but a smaller volume?)", "other_comments_or_suggestions": "I have found problems following the methodology section given that at some point is set f_t(v)=\\tilde f_v(t). Is this really necessary?", "questions_for_authors": "What is exactly the technical novelty and benefits of the proposed method as compared to the above mentioned schemes? How do they empirically compare?\n\nUpdate after rebuttal:  I have decided to update my score from reject to weak reject. There exists a fairly large number of relevant papers that are not discussed or benchmarked. This is a common concern among other reviewers; however, they did not consider it as serious as I did—hence my score upgrade.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "16u7Nb58XU", "reviewer": "Reviewer_1t9h", "summary": "The paper presents a novel method to achieve better conditional coverage in conformal prediction for single-output and multi-output regression. The central idea is to start from a classical nonconformity score, and adjust it to improve for conditional coverage. The adjustment is a factor that is obtained by estimating conditional quantiles using classical or local quantile regression. \n\nThe authors present theoretical results, claiming that their proposed method achieves the desired marginal and conditional coverage, provided that conditional quantiles are known. \n\nThe experiments on synthetic and real-world data intend to show that the proposed method works well in practice. On real-world datasets improvements in conditional coverage are observed compared to four baseline methods.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the thorough and constructive feedback, which helps improve our manuscript. Below, we address your valuable points:\n\nThe **main limitations** of RCP can be summarized as follows:\n- The quality of the prediction regions heavily depends on the basic conformity score $V$. For example, if the underlying multi-output conformal method predicts hyperrectangular sets, RCP will also predict hyperrectangular sets. \n- The quality of the conditional coverage of the intervals crucially depends on the quality of the conditional quantile estimator $\\hat{\\tau}(x)$ of the conformity score. The marginal coverage guarantee is **always ensured**; however, the guarantee of conditional coverage clearly depends on how $\\hat{\\tau}(x)$ approaches $\\tau_*(x)$, the \"exact\" quantile. A key contribution of our work is the explicit tracking of how errors in quantile estimation influence conditional coverage error.\n\n**Additional experiment on synthetic data.** Following your advice, we conducted an additional experiment on synthetic data to investigate the case of learned quantile estimate. We used a simple MLP trained on a dataset of varying sizes ranging from 100 to 500 points (note that the calibration dataset size in this experiment is equal to 500). The resulting plot can be found via https://pdfhost.io/v/ND3Dt3PahY_Additional_synthetic_data_experiment\nWe see that the conditional coverage is not perfect, but RCP outperforms the standard CP already for a relatively small data size of 100 points.\n\n**Why assumption H2 is needed?** This assumption ensures that the score function is compatible with the adjustment function. Only valid $t$ values will be supplied to $f\\_t(v)$ to ensure that H1 is satisfied. For example, if $f_t(v)=tv$ then $\\hat{\\tau}(x)$ has to be positive.\n\n**Connection with normalized conformity scores.** \nThere is indeed a connection to prior works on normalized conformity scores. We will incorporate these references into the literature review of the camera-ready version (if accepted). Normalized Conformity Scores (NCS) all share the core idea of normalizing nonconformity scores by the predictive accuracy of the underlying model at new data points. This normalization aims to enhance the efficiency of conformal predictions by assigning wider prediction intervals to challenging instances and narrower intervals to easier ones, with the difficulty determined by the accuracy of the predictive model itself. However, these studies generally lack a detailed analysis of approximate conditional coverage, which distinguishes our work. Furthermore, we can recover the specific formulation of normalized nonconformity scores through an appropriate choice of the function $f_{\\tau}(v)$. However, the criterion employed in our method to estimate $\\hat{\\tau}(x)$ fundamentally differs.\n\nWe provide further details below.\n- U. Johansson et al. [1] and Papadopoulos et al. [2] investigate NCS methods, which enhance standard conformal prediction by dynamically adjusting prediction interval sizes according to instance difficulty. Normalization in their methods involves a parameter $\\beta$, which balances the model's prediction error and the estimation of difficulty. However, these methods lack explicit theoretical guarantees. Their NCS can be represented within our framework through a specific choice of the function $f_{\\tau}(v) = v/(\\tau + \\beta)$. Notably, the estimation approach employed in these papers uses least-squares regression on residuals, in contrast to the quantile regression approach adopted in RCP. The central goal in RCP is to construct a \"rectified\" conformity score that aligns conditional and unconditional quantiles at a target level--an objective distinct from NCS.\n\n- The paper by Dewolf et al. [3] provides an insightful summary of normalized conformal predictors, introducing the concept of a taxonomy function, which they assume to be discrete. In their own terms, the taxonomy function \"divides the instance space based on an estimate of the uncertainty,\" for example, by partitioning the feature space through binning the (conditional) standard deviation. However, their analysis is restricted to an oracle setting, meaning that the theoretical developments rely on an exact, known normalizing function. Consequently, their work does not address the practical scenario in which this normalizing function must be estimated from data.\n\n[1] H. Papadopoulos, A. Gammerman, and V. Vovk. Normalized nonconformity measures for regression conformal prediction. In Proceedings of the IASTED International Conference on Artificial Intelligence and Applications (AIA 2008), pages 64–69, 2008.\n\n[2] U. Johansson, H. Boström, and T. Löfström. Investigating normalized conformal regressors. In 2021 IEEE Symposium Series on Computational Intelligence (SSCI), pages 1–8. IEEE, 2021.\n\n[3] N. Dewolf, B. De Baets ,and W. Waegeman, Conditional validity of heteroskedastic conformal regression. Arxiv 2023.", "claims_and_evidence": "The main goal of the paper is to present a new method that improves on conditional coverage. \n\nI believe that the presented method is novel, but it is a pity that the authors don't discuss the limitations of their approach. \n\nI enjoyed reading the theoretical discussion in Section 3, but I am somewhat less convinced of the practical implementation in Section 4. Conformalized quantile regression has been proposed in the literature as a tool to improve the quantiles obtained by quantile regression, so that better conditional coverage is obtained. Here the authors are reasoning the other way around: they are using quantile regression to improve what conformal prediction is doing wrong... So, others have claimed that quantile regression is not good in obtaining quantiles for regression problems that are strongly heteroskedastic, while here is claimed that quantile regression is the solution. I believe that this deserves more discussion...\n\nIn light of this, the experiments with synthetic data are also not convincing. It is obvious that conditional coverage will be obtained if access to the ground-truth conditional distribution is assumed. I would have liked to see on synthetic data how the method performs when the quantiles need to be estimated using quantile regression. The considered toy problem is strongly heteroskedastic, so I assume that estimating the quantiles is far from trivial, despite the one-dimensionality of the problem in feature space.  \n\nApart from the connection with conformalized quantile regression, I believe that the proposed method is also closely related to the \"normalized\" conformal prediction literature. This literature is not discussed in the related work section, but normalized nonconformity scores have a very similar idea in mind. For regression, the standard nonconformity score based on absolute residuals is divided by (an estimate) of the variance, see e.g.:\n\nH. Papadopoulos, A. Gammerman, and V. Vovk. Normalized nonconformity measures for regression conformal\nprediction. In Proceedings of the IASTED International Conference on Artificial Intelligence and Applications\n(AIA 2008), pages 64–69, 2008.\n\nU. Johansson, H. Boström, and T. Löfström. Investigating normalized conformal regressors. In 2021 IEEE\nSymposium Series on Computational Intelligence (SSCI), pages 1–8. IEEE, 2021.\n\nN. Dewolf, B. De Baets ,and W. Waegeman, Conditional validity of heteroskedastic conformal regression. Arxiv 2023.  \n\nUnder certain assumptions, exact conditional coverage is obtained, similar to the reasoning of the authors. So, I think that this literature should be discussed. Moreover, normalized conformal prediction would also be the most obvious baseline in the experiments. Conformalized quantile regression would also be an obvious baseline. I did not understand the reasoning of the authors for the baselines they chose.", "methods_and_evaluation_criteria": "See previous section.", "theoretical_claims": "The theoretical claims make sense to me. I did not check the proofs in detail, but the claims are pretty straight-forward, so I don't see issues. \n\nI don't understand why assumption H2 is needed. This is a very general assumption that is always fulfilled in practice, isn't it? Perhaps this assumption can be simply omitted.", "experimental_designs_or_analyses": "See above.", "relation_to_broader_scientific_literature": "See above.", "essential_references_not_discussed": "See above.", "other_strengths_and_weaknesses": "Strengths:\n- The paper is very well written (the authors give evidence of a solid math background, which is appreciated) \n- The proposed method is novel\n- The authors present non-trivial theoretical results\n\nWeaknesses:\n- The limitations are not discussed\n- Related work is missing\n- The experiments are a bit underwhelming.", "other_comments_or_suggestions": "None.", "questions_for_authors": "I invite the authors to give feedback on my comments.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "No"}}]

Author rebuttal:

Meta-review: The overall recommendations are 3, 4, 2, 3.

Paper summary: this paper proposes rectified conformal prediction to improve conditional coverage while ensuring exact marginal coverage using a trainable transformation on any given conformity score. In the provided examples and the proposed implementation, the conditional quantile and an estimated conditional quantile are used for the transformation. The paper has theoretical and empirical justification.

Reviewers mostly agreed that the proposed method was novel and theoretical results were non-trivial. Experiments are thorough and improvement of conditional coverage over baselines is found on real-world datasets.

There is a major weakness commonly raised by reviewers:

Many relevant papers were not discussed or benchmarked to clearly highlight the novelty of this paper, while the rebuttal addressed this common concern. Some reviewer(s) thought their concern has been resolved, but some reviewer(s) still believed that this concern should be considered seriously.

Specifically, there are roughly three subareas of conformal prediction literature that reviewers raised during the review/discussion and believed need further detailed discussion:
1. The literature on normalized conformity scores, e.g., [1, 2, 3], since their technical ideas are very similar to the proposed one.
2. Recent papers such as [4] discusses some techniques that could be adapted into the proposed RCP. 
3. Those papers that optimize conformity score to improve the efficiency and conditional coverage such as [5, 6, 7], with the same goal as the paper.

[1] H. Papadopoulos, A. Gammerman, and V. Vovk. Normalized nonconformity measures for regression conformal prediction. In Proceedings of the IASTED International Conference on Artificial Intelligence and Applications (AIA 2008), pages 64–69, 2008.

[2] U. Johansson, H. Boström, and T. Löfström. Investigating normalized conformal regressors. In 2021 IEEE Symposium Series on Computational Intelligence (SSCI), pages 1–8. IEEE, 2021.

[3] N. Dewolf, B. De Baets ,and W. Waegeman, Conditional validity of heteroskedastic conformal regression. Arxiv 2023.

[4] Bian & Barber. Training-Conditional Coverage Methods 

[5] John J. Cherian, Isaac Gibbs, Emmanuel J. Candès. Large language model validity via enhanced conformal prediction methods 

[6] Kiyani S, Pappas GJ, Hassani H. Length optimization in conformal prediction.

[7] R Xie, R Barber, E Candes. Boosted Conformal Prediction Intervals
Decision: Accept (poster)