Accelerating Feature Conformal Prediction via Taylor Approximation
Authors: Zihao Tang
Boyuan Wang
Chuan Wen
Jiaye Teng
Venue: NeurIPS 2025 poster ()
Keywords: conformal prediction
Taylor
fast
feature
uncertainty
deep learning
representation
Primary area: probabilistic_methods
TL;DR: A fast version for feature conformal prediction.

Abstract: Conformal prediction is widely adopted in uncertainty quantification, due to its post-hoc, distribution-free, and model-agnostic properties.
In the realm of modern deep learning, researchers have proposed Feature Conformal Prediction (FCP), which deploys conformal prediction in a feature space, yielding reduced band lengths.
However, the practical utility of FCP is limited due to the time-consuming non-linear operations required to transform confidence bands from feature space to output space.
In this paper, we present Fast Feature Conformal Prediction (FFCP), a method that accelerates FCP by leveraging a first-order Taylor expansion to approximate these non-linear operations.
The proposed FFCP introduces a novel non-conformity score that is both effective and efficient for real-world applications.
Empirical validations showcase that FFCP performs comparably with FCP (both outperforming the vanilla version) while achieving a significant reduction in computational time by approximately 50x in both regression and classification tasks.

Accelerating Feature Conformal Prediction via Taylor Approximation
Zihao Tang*
Boyuan Wang
Chuan Wen
Jiaye Teng*
"Shanghai University of Finance and Economics
Southern University of Science and Technology
Shanghai Jiao Tong University
Abstract
Conformal prediction is widely adopted in uncertainty quantification, due its post- hoc, distribution-free, and model-agnostic properties. In the realm of modern deep learning, researchers have proposed Feature Conformal Prediction (FCP), which de- ploys conformal prediction in a feature space. yielding reduced band lengths. How- ever, the practical utility of FCP is limited due the time-consuming non-linear op- erations required to transform confidence bands from feature space to output space. In this paper, we present Fast Feature Conformal Prediction (FFCP), method that accelerates FCP by leveraging first-order Taylor expansion to approximate these non-linear operations. The proposed FFCP introduces novel non-conformity score that is both effective and efficient for real-world applications. Empirical validations showcase that FFCP performs comparably with FCP (both outperform- ing the Split CP version) while achieving significant reduction in computational time by approximately 50x both regression and classification tasks. The code is available at httpt:://// / thub. com/ Semm/EEviistnn www.wwwwwwiiniii. 11aag////////aaaaarrrr..
Introduction
Machine learning has been successfully applied numerous fields such as computer vision, natural
language processing, and gaming (Jordan and Mitchell 2015 Silver et al. [017]. However, machine learning models usually suffer from overconfidence issues Wei al. 2022 and even hallucinations in large language models (LLMs) [Ji et al.) 2023 which makes them unreliable and unable to be
deployed in fields like finance and medicines Gelijns et al. 2001 Thirumurthy et al. [0119] Morduch
and Schneider 2017]. Therefore, it is essential to develop techniques for uncertainty quantification
and calibrate the original machine learning models Abdar et al. 2021 Guo et al. 2017 Chen et al.
2021
Gawlikowski al. [2021)
Among the uncertainty quantification techniques, Conformal Prediction (Split CP. or split conformal
prediction, Vovk et al. (2005]; (Shafer and Vovk (2008b): Burnaev and Vovk (2044) stands out, because is distribution-free models. Conformal prediction deploys a calibration step to calibrate base model and then construct the confidence band. The goal conformal prediction to return a band (X') such that
does not require retraining, and can be directly applied
various
P(Y' E C1-a(XX)) 2
where (X', Y") denotes test point and 1 represents the confidence level.
In deep learning regimes, researchers try to utilize feature information in Split CP, since the feature space usually contains meaningful semantic information in neural networks (Shen et al. 2014 This leads to Feature Conformal Prediction (FCP, Teng et al. (2022)
Correspondence to teng Langjaye@dufe.ede edw.euueeeeedeecmm.oo.eo
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Is it Cat?
Vanilla CP
Uncertainty
Encoder
Decoder
Output Space
IY f(X)
FCP
Space
Decoder
inf]lv Pll
E (vig(v) Y3
FFCP
Taylor Expansion
Encoder
Space
Decoder
D
Y - f(X)
[[V(
Figure 1: Comparison among Split CP, FCP, and FFCP. FCP and FFCP are more efficient compared to Split CP since they return different band lengths for different individuals. This is achieved by calculating non-conformity score in the feature space. Besides, FFCP approximates FCP using Taylor expansion, which leads to different non-conformity score and accelerates the transformation from feature space output space.
Fortunately, one may get different band lengths on different individuals by utilizing the feature information, leading to shorter confidence band. As a comparison, Split CP only returns the same band length for all individuals in regression tasks, which indicates a longer length.
However, the practical applications of FCP limited because (a) it time-consuming, and (b) only returns estimated bands on the output space, making it less efficient. These two issues come from the step Band Estimation, which transfers the confidence band from feature space to output space. This step involves complex non-lineat operations called LiPRA [Xu et al. [2020] and therefore (a) the non-linear operation requires high computational costs, and (b) the configurations in LiPRA might finally influence the estimated band, further harming the performance of FCP.
In this paper, we present Fast Feature Conformal Prediction (FFCP), which offers fast version for handling the aforementioned nonlinear operations in FCP. Different from Split CP and FCP, FFCP introduces novel non-conformity score sF(-) that is simple compute and does require additional training,
Ser (X, y, h) = |Y - go (X))
where (X, Y denotes a sample, go denotes neural network with feature layer and a prediction head g. and the gradient (g( denotes the gradient g(-) on the trained feature 0 & ixX),, namely,
(ū(i) = 4X),(X), We refer to Algorithm for more details and illustrate the algorithm in Figure dh(X)
The above non-conformity score is closely related to FCP. Specifically, FFCP with this non- conformity score can be regarded as a fast version of FCP, since it equivalently approximates the prediction head using Taylor expansion, which simplifies the aforementioned nonlinear operations. Fortunately, FFCP inherits the merits FCP, for example, it also utilizes the semantic information in the feature.
From theoretical perspective, first demonstrate that FFCP is effective in Theorem in that it returned confidence band with empirical coverage larger than the given confidence Additionally, we demonstrate in Theorem 4.2 that FFCP produces a shorter confidence band than Split CP under a proposed square condition. The square conditions outline the properties of the feature space from two perspectives: expansion and quantile stability, implying that the feature space has smaller distance between individual non-confornity scores and their quantiles. This reduces the cost of the quantile operation and therefore leads to a shorter confidence band. We also validate the square conditions using empirical observations.

---

From an empirical perspective, we conduct several experiments on real-world datasets and show that FFCP performs comparably with FCP, both outperforming Split CP. while achieving nearly 50 times the speed of FCP in terms of runtime for regression tasks. We further validate the approximation ability of FFCP with FCP using the correlation between the non-conformity score of FFCP and FCP. We also apply FFCP to the image segmentation problems to verify its general applications. Besides, we show that the concept in FFCP is pretty general. and can be combined with other variants of CP, e.g., CQR [Romano et al. 2019a] and LCP [Guan 2023 in regression tasks, and RAPS Angelopoulos et al. 2020] in classification tasks.
Overall, our main contributions are summarized as follows:
This work proposes FFCP, which serves as a fast version of FCP. FFCP achieves around 50x faster speed compared to FCP (Table T by utilizing Taylor expansions to approximate the prediction head in FCP. Besides, FFCP inherits the merits of FCP and efficiently exploits semantic information in the feature space.
Theoretical insights demonstrate that FFCP returns shorter band length compared to Split CP (Theorem 4.2) while ensuring coverage exceeds the given confidence level under mild conditions (Theorem
Extensive experiments with both synthetic and real data demonstrate the effectiveness of the proposed FFCP algorithm (Table Additionally, demonstrate the universal applicability our gradient-level techniques by extending them to other tasks such as classification (FFRAPS, Algorithm 5) and segmentation, and to various conformal prediction variants, including CQR (Algorithm 3 and LCP (Algorithm
2
Related Work
Conformal prediction is post-hoc calibration method dealing with uncertainty quantification [Vovk ctt al.
Shafer and Vovk 2008a, Barber et al. 2020J, which is deployed numerous fields [Ye Kumar et al. 2023 Quach et al. [023 The variants of conformal prediction typically revolve ve around the concept of non-conformity scores, with four main branches of development.
2005
al.
2024
Relaxing Exchangeability. The first branch relaxes the exchangeability requirement [Tibshiran]
al. 2019 Hu and Lei, (2020, Podkopaev and Ramdas] 2021 Barber al., 2022]. leveraging weighted or reweighted quantiles to relax exchangeability, By doing so, it gains more flexibility and broader applicability in handling data that may not satisfy the standard exchangeability assumptions.
Diverse Structures. The second branch applies conformal prediction to various data structures, for
example, classification tasks [Romano et al. [2020] Angeloppuulss et al., [2020) time series data (Xu and Xie. 2021 Gibbs and Candès 2021], censored data in survival analysis [Teng et al. [202] I Candès
ct al. 2023 nigh-dimensional data [Candès et al., 2021 Lei et al. 2013), Bellman- based data Yang
et al. 2024
counterfactuals and individual treatment effects Lei and Candès 2021 etc.
Another way involves model structures, such k-NN regression (Papadopoulos et al., 2011a quantiles incorporated [Romano et al.) [019b) (Sesia and Candès 2020). density estimators [lzbicki ctt al. [2000b]. and conditional histograms [Sesia and Romano [021]. These methods further enrich the application scenarios of conformal prediction by adapting it to diverse model frameworks.
Enhancing Methods. The third branch focuses on enhancing the original conformal prediction with band length. Izbicki al. [2020a] introduce CD-split and HPD-split methods, and Yang and Kuchibhotla [202]] develop selection methods to minimize band length. Of particular note is feature contormal prediction [Tegg al. 2002], which leverages neural network training information via feature spaces to improve band length.
Localized Conformal Prediction.
The fourth branch focuses on enhancing the non-conformity | where Y denotes the true score normalization by incorporating difficulty-related terms like o(X)
label, Y denotes the predicted label, and ar (X') denotes the standard deviation related to X'. Here three key approaches:
3

---

(1) Weight Adjustment via Calibration Distances. point to the calibration points and then uses these distances define the weights of non-conformity scores in the calibration process [Hnn et al. 2022 Guan 2023 Our gradient-level techniques can be used to combine with this branch (see FHT.CP Algorithm[4] 4
This approach calculates the distance from the test
(2) Normalization Using Proximity to Training Set. This approach utilizes the observation that testing point exhibits smaller uncertainty when close to the training set, and uses such metrics to approximate oX') Papadopoulos et al. 2008 201 Papadopoulos and Haralambous 2011] In the deep learning regimes, we believe that such procedures can be further improved by calculating the distance in the feature space rather than the input space, since feature layers usually contain more semantic information.
(3) Modeling a (X'). This approach trains a separate model to estimate a (X)). thereby enhancing the accuracy and adaptability of CP (Seedat et al. 2023 2024). However, this line of work heavily relies on the model performance and incurs high computational costs. As a comparison, our approach does not require additional training procedures.
Uncertainty Quantification. Uncertainty quantification is one of the most fundamental questions in
machine learning. In addition to conformal prediction, many other approaches exist for quantifying
uncertainty, including calibration- based techniques [Guo et al., 2017 Kuleshov et al. 2018 Nixon
et al. 2019 Abdar et al.] [2021] [ et al., [2024] and Bayesian- techniques [B Blundell et al.
2015
Hemnandez-Lobrae and Adams 2015 Li and Gal 2017 Izmailov et al. 202 Jospin et al.
2022
3 Preliminaries
We begin by introducing a dataset = {(X,, Hell)) indexed by I. We split the dataset into two folds: a training fold Dira indexed by Iims and a calibration fold Daal indexed by Teat. Denote the testing point by (X', Y"). ) For the model part, define ʃ as neural network. We partition f = g h, where h denotes the feature function (the initial layers of the neural network) and denotes the h(X) as the trained feature. We follow the ideas in Teng al. 2022) and detine the surrogate feature any feature such that =(v)=YY.
prediction head. For a sample (X, Y), we define
Assumption (exchangeability). Assume that the calibration data (X,, Y:) E Deal and the testing point (X', Y') are exchangeable. Formally, define Zi, i = 1, ..... [Teal + 1, as the above data pair. Then Z4 are exchangeable if arbitrary permutation follows the same distribution, i.e.,
(Zi,..., Zzzmi++) (Z)()).... Z (I((ao))))),
with arbitrary permutation π over (1.... [Teat] 1}.
Typically, Split CP composed of three key steps.
I. Training Step. We first train a base model using the training fold Din. II Calibration Step. We calculate a non-conformity score R; = |Y: f(X:) using the calibration fold Deat The form of the score function might vary case by case, quantifying the divergence between ground truth and predicted values.
III. Testing Step. We construct the confidence band for the testing point (X,,") Y") using the quantile of the non con-conformity score Q1-a-
We present Split CF in Algorithm I and provide its theoretical guarantee in Theorem 3.1 Theorem 3.1. Under Assumption
the confidence band C1-x(X)) (X') returned by Algorithm satisfies P(Y' E Ci-a(X')) 2 1 a.
(4)
4 Methodology
In this section, we first illustrate the motivation behind FFCP Section 4.1 address the complexity of non-linean operators in FCP and derive FFCP from FCP. We then formally present the specific form of FFCP, including the non-conformity score, the returned bands, and the corresponding pscudocode. We finally provide thcoretical analyses on the coverage and band length
Specifically, we
in Section 4.2
'8 represents the Dirac function.

---

Algorithm Split Conformal Prediction
Input: Confidence level dataset D {(X,, HEz, testing point X'
1: Randomly split the dataset 2> into training fold Pma {(X, Yi) and calibration fold
Deal & {(X,, )
2: Train a base model f()) with training fold D
3: For each E Teal, calculate the non-conformity score R I2 f(Xi)l:
4: Calculate the (1 a -thh quantile Q1-a of the distribution -ail+1 ZiEZnndn tôoo"
Output: C SSplicp (x')
= (X') [[(X) Q1 J(X) Q1-a].
Algorithm 2 Fast Feature Conformal Prediction
Input: Confidence level a, dataset D {(x,, Ye)ez testing point X'
1: Randomly split the dataset D into a training fold Pma & {(x,, Y))ee and a calibration fold
Deal _ {(x,, YE)
2: Train base neural network with training fold ʃ() g h(-)) with training fold Dimi
3: For each i E Zeal. calculate the non-conformity score R₃ |Y; f(X:)l/ [g(;;) (6:) II. where
Vg(ūs) denotes the gradient of g(-) on the feature ú; h( ()). namely Vg(ū;) = da(x))
d.).(X.).
4: Calculate the (1 -tt quantile Q1-a of the distribution IZ.i|=1 ΣiEz., 6n. + 8009
Output:
ChEp Ci-a (Xx') - [V S(X' + VV (f))() where 0' 1(X)).
4.1 Relationship between FFCP and FCP
In this section, we discuss the motivation behind FFCP. FFCP is inspired by FCP (Teng et al. 2022] which conducts conformal prediction in the feature space. However, since the band is constructed in the feature space, FCP requires Band Estimation process to go from feature space to output space. Specifically, FCP applics LiPRA [Xu et al., [2200 which derives the band the output space {g(u) |I e|| I Q.-a Unfortunately, the exact band is difficuult to represent explicitly since the prediction head g is usually highly non-linear, thereby resulting in significant computational complexity in terms of time. Therefore, propose approximating g using first-order Taylor expansion to simplify the aforementioned non-linear operator. The core steps of FCP include (a) calculating the non-conformity score (from output space feature space), followed by (b) deriving the confidence band (from feature space to output space). We next introduce the concrete formulation of how FFCP approximates FCP.
From output space to feature space. FCP uses the non-conformity score sr(-) in the feature space:
st Y, =Efe-g(o)=))
inf ||0 a|1.
By using the Taylor expansion, one approximates with g(v) z g(t) + Vg(ô)(v -). Plugging into the approximation of g leads to a new non-conformity score st(-)
su Yx |Y - f(X)//VV9(0)I.
(6)
where Vg(E) denotes the gradient g()) on the feature é, namely Vg(ū)= dgch(X) ak(X)
From feature space to output space. After constructing the confidence band in the feature space,
FCP maps this band to the output space. Specifically, FCP derives the following band in the output space which is called Band Estimation:
(o(o) ||v +1l S Qi-a).
FCP proposes to use LiPRA in this process, which is time-consuming. By plugging into the Taylor approximation of g, one can construct the band allep as
clfep (x) = (g()) - V g(0) [1(())
(8)
Remark (Higghdimmnssional Response). When the response Y; = [V,, Y = 2, ... , is high-dimensional, one can deploy conformal prediction at a coordinate level.
5

---

Specifically, for dimension j [ma] we define the non-conformity score as
X Y,, = -
where Vg(O:)²=
(Of(X)) (a((..)
represents the j-th row of the Jacobian matrix of f with respect to h at
X,
We then compute a single quantile Q1-a shared across all dimensions, defined by aggregating non-conformity scores from all coordinates and samples in the calibration set:
Q1 = Quantile (x,,
J E [ml] 1 aa).
(10)
The resulting confidence band for the j-th coordinate at test point X; is given by:
ChiP _ (X.), = [g(0,)' [9(0)) [V 11 g(6x)* + (V((0)))11111
(11)
Based on the above discussion, we present the full algorithm in Algorithm 2 Notably, the Taylor expansion in FFCP is usually different for each sample X, Y. which further leads to confidence bands that are individually different. Besides, FFCP inherits the advantages of FCP. For example, this framework is pretty general and can be combined with other variants of Split CP, e.g., CQR [Romano al. 2019a].
4.2
Theoretical Guarantee for FFCP
This section outlines the theoretical guarantee for FFCP concerning coverage (effectiveness) and band length (efficiency). Below, offer the main theorems and defer the full proof's to Appendix [A.] and A.2 We first demonstrate that the confidence band produced by Algorithm 2 is valid under Assumption
Theorem 4.1 (Coverage). Under Assumption 1, for any >> the confidence band returned by satisfies:
Algorithm 2
PP(Y' E c 2 ax
(12)
where the probability is taken over the calibration fold and the testing point (X',Y Y").
Next, we show that FFCP is provably more efficient than the Split CP. simplify the discussion, we present an informal version of Theorem 4.2 here and postpone the formal version to Theorem A.I
Theorem 4.2 (Band Length). Under mild assumptions, if the following square conditions hold:
1. Expansion. The feature space expands the differences between individual lengths and their quantiles.
2. Quantile Stability. Given a calibration set Deut, the quantile of the band length is stable in both feature space and output space.
Then FFCP provably outperforms Split CP in terms of average band length.
By Theorem FFCP is guaranteed achieve smaller average band length than Split CP. The square conditions imply that the feature space has a smaller distance between individual non- conformity scores and their quantiles. This reduction in the computational overhead of the quantile operation subsequently yields a shorter band length. We provide empirical verifications on this assumption, see Figure 4 for more details. The intuition behind Theorem 4.2 is follows: Initially, FFCP and Split CP perform quantile operations in different spaces, with the Expansion condition ensuring that the quantile step FFCP costs less. The ultimate Quantile Stability condition confirms that the band can be generalized from the calibration folds to the test folds.
5 Experiments
This section presents the experiments to validate the utility FFCP. Firstly, we detail the experimental setup in Scction 5.1 Secondly, present that FFCP achieves both effectiveness and efficiency with

---

Table I: Time comparison among Split CP, FCP, and FFCP. FFCP ensures faster running speed compared to FCP. The last column represents the speed improvement factor FFCP compared to FCP. The time unit is in seconds.
DATASET
SPLIT CP
FCP
FFCP
FASTER
SYNTHETIC
0.008844.0000
3.8939:0033725
0.0902=0.0010
43x
COM
0.0047+0.0010
4.980410.8850
0.0844=0.0187
59X
FB1
0.0245+40.0059
5.9822+:0.95
0.1940=0.066
31x
FB2
0.0414+0.0077
9.353341:0000
0.2510=0.um 0.058
37X
0.0106110.000
3.3237
0.0755 0.755=000037
44x
MEPS20
0.015210.000 0006
5.4003310.3945
0.0948 00.0077
57X
MEPS21
0.0137+0.000
4. 4.1657=00.m 0670
0.0854
49X
STAR
0,0030+0.000 0006
3. 3.5842++0.772
0.0332 0066
108X
BIO
0.0291+0.000 0053
7.5417
0.2042=0.0344
37X
0.0340010.022
8.09131
0.2239 10.0261
36x
BIKE
0.007210.0000
3.5806110.0285
0.053110.0021
67X
Table 2: Comparisons of coverage and band length among Split CP, FCP, and FFCP. FFCP runs faster while performing comparably to FCP in most datasets and outperforming Split CP. For FFCP, we select the shortest band length among all layers.
METHOD
SPLIT CP
FCP
FFCP
DATASET
COVERAGE
LENGTH
COVERAGE
LENGTH
COVERAGE
LENGTH
SYNTHETIC
90,080#0.951
0.176=0.015
89 89.930=+0.956
0.081
90.080#0.w0 951
0,176=00.055
COM
$0.875=20.955
1.974=0.071
89.724+1.087
1.939=1.108 108
20.2266+2.11 179
1.838
FB1
90.254:0.1700
2.004=0.191
90.198:00.07
2.010=0.162
90.168#10.20
1.472
FB2
89.933
2.016=0.215
89 89.966:10.130
1.371 -00370
89.868 00.62
1.425
MEPS19
00.5671103311
3.982
90.605
3.493
90.35210. 1..52.0.4800
MEPS20
89.923
4.184
89 $9.929
2.730
89.615
3,268 &.0...0.... 283
MEPS21
90.01911.3341
3.732=0.655
90.038
3.393=1.313
89.745
1.146-E0.500 506
STAR
90.393:11.44
0.208=0.004
90.300-11 .362
0.174:10033
90.393+11
0.208
BIO
89.875=10.488
1.661
89.930-+0.501
1.412#002253
89.875
1.661
BLOG
90. 10.17720022444
3.524=0.864 850
90.151+0.40s
2.795=1.3m
90.059 101
2.741
BIKE
89.871+00.50
0.703=0.016
89.394400.63
2.147=0.219
89. 89.6244t0.66 688
0.635
faster execution Section 5.2
Thirdly, in Section 5.3.1 we verify that FFCP can be easily deployed
and performs robustly across various tasks, including classification and segmentation. Finally, in Section 5.3.2] we show that the gradient-level techniques used in FFCP can extended to classic models such as CQR [Romano al. 2019a] and LCP Guan 2023 A more detailed account of this extension can be found in Section
5.1 Experiments Setups
Datasets. We consider both synthetic datasets and realistic datasets, including (a) synthetic dataset:
Y WX + where X E [0. 1|200 Y E IR, e ~ (O,, 1), W is fixed random matrix. (b) real- world unidimensional target datasets: ten datasets from UCI machine learning [Asuncion) 2007 and other sources: community and crimes (COM), Facebook comment volume variants one and two (FBI and FB2). medical expenditure panel survey (MEPS19-2) [Cohen et al., 2009 Tennessce's
student teacher achievement ratio (STAR) [Achilles et al. 2008 ) physicochemical properties of protein tertiary structure (BIO), blog feedback (BLOG) Buza 2014]. and bike sharing (BIKE), (c)
real-world semantic segmentation dataset: Cityscapes CCordts et al. 2016). and (d) real-world
semantic classification dataset: Imagenet- Val [Deng et al. [2099].
Algorithms. We compare three methods: Split CP, FCP, and FFCP, with Split CP serving as the baseline. For the one-dimensional scenario, we perform direct caleulations. For higher-dimensional cases, we use a coordinate level non-conformity score.
Evaluation. The algorithmic empirical performance is evaluated with the following metrics:
7

---

Table 3: Coverage and Band Length based on Gradient from Different Layers of Neural Networks. FFCP LAYER()) represents using the gradient between the LAYER(.) and the input. The results in LAYER4 are equivalent to Split CP.
LAYER
LAYER2
LAYER3
LAYER4
DATASET
COVERAGE
LENGTTH
COVERAGE
LENGTH
COVERAGE
LENGTH
COVERAGE
LENGTT
SYNTHETIC
69.810=0.784
0. 0.184:0.018
90.050#0.50 534
0.184=0.017
89.960=0.910
0.18220.028
90.220 .63
0. 8.189=0.083
90.476.11880
1.87820.221
80.22611.20
1.
81.674
1.853110113
89.825110.66
2.037
FB1
90. 10.112:0.111 100
3.540 397
80:212++03355 357
2.880 327
01.083:..221 216
1.507400000 002
90. 80.168::0 220
1.472
FB2
89.9 98.953 =0.250
3.530=0.354
89.85 19.897 20.
3.018=0.5110
89. 159
2.077
59 00.8668 4:0 062
1 1.425=00.10
19PS19
90. 155
0.251
90.352:10.00
3.13410.994
90. 90,440:0.113
3.1841:0-622
90.5861:0
3.795
MUPS20
89 09..334-0.3000 500
4.302
59 88.8880100.00 621
3.573k20 458
80.616220.00
3.268 283
82:0 5899
3.817 305
MEPS21
80 00..496:0.00 962
3.443::0.48 467
80.62311 975
3.218 239
811.745:.0:314
3.146 3.14610.306
90.026 5031
3.4422.0.77
STAR
90.901=1.77
91.039b:1:4422
0.2100+0.00
90 30.3001++1..2
0.209
BIO
89. =0.937=0.391
2.292=0.077
90.0222+0.557 5775
2.042 067
89.991::0.594
2.080+00.06
90.127:4:0.44 478
1.822=00.225
BLOG
80.968 120
4.712:0.014
80.91810
3.404 59s
90.0599:0.101
2.741 50.517
80.01T10.1100 1997
3.058 8085
89.917=0.701
1.701
80.068.0
1.138:0.114
601-466:005579
0.7948.0.000
89-624400.om
0.635
Runtime For runtime evaluation, the timing starts at the score calculation and ends with the final prediction bands returned. FFCP method records the total computation time for each layer, and then selects the layer that achieves the best results.
Coverage (Effectiveness) Coverage refers to the observed frequency with which test point falls within the predicted confidence interval. Ideally, a predictive inference method should yield a coverage rate slightly higher than 1 a for a given significance level a.
Band length (Efficiency) When the coverage exceeds 1 a, our goal is to minimize the length of the confidence band. For FFCP, since we use 5-layers neural network, each layer can be viewed as feature layer. Therefore, in the experiments, we obtain the band length returned by each of the 5 layers of the neural network. In the subsequent results, if only a single band length is presented, it corresponds the shortest band length returned by the different neural network layers. Otherwise, the results for all layers from layer to layer (with the last layer typically representing the Split CP result) will be shown.
Let Y= = Y" E IR denote the d-dimensional response variable, and let C(X) C Rd be the confidence band associated with predictor X. The length of this confidence band in each dimension is represented by the vector (IC(X)"),, I,, l,......,....... (C(,.......C(' I) E Rd. Denote the indices of the test set by Ies and the set of dimensions by [d] = (l,...,.d d]. We then define the coverage and band length as:
Coverage
EI(EEC(X)) E(X.)), Band Length
1
Σ ieI.
(e(xM1) (13)
[Tue
ITes
d
where I[(-) is the indicator function that equals if its argument is true and otherwise.
5.2 Results on Coverage, Band Length and Runtime
Runtime Comparison. The runtime comparison is presented in Table ] The results show that FFCP outperforms FCP with an approximate 50x speedup in runtime. Notably, since Split CP the most basic method and does not utilize additional tools, it exhibits the fastest runtime.
Coverage. Table summarizes the coverage for the dimensiiiio response. Experimental results indicate that the coverage of FFCP all exceeds the confidence level 1 - a, affirming its effectiveness as stated in The
Band Length. The hand length is detailed in Table for a one-dimensional response. It is noteworthy that FFCP surpasses Split CP by achieving shorter band length, thereby validating the efficiency of the algorithm.
5.3 Extensions of FFCP
This section provides the extensions of FFCP, which is divided into two parts. Section 5.3. mainly discusses the applications of FFCP beyond regression tasks, specifically in image classification [An gelopoulos et al.) 2020 and segmentation tasks. Section 5.3.2 focuses on how the gradient-level
8

---

1.
00
Image
Ground Truth Label
Length
(d) Length
Figure 2: Segmentation uncertainty (FFCP vs. FCP). Both FFCP and FCP capture uncertainty concentrated around object boundaries, with FFCP producing more refined and sharper uncertainty bands. Notably, the two methods aim at slightly different goals: FCP operates at the image level, ensuring coverage for the entire predicted mask as whole, while FFCP adopts coordinate-wise approach, offering per-pixel statistical guarantees that better reflect local uncertainty.
techniques in FFCP can be extended to other CP variants, e.g., CQR [Romano et al. 2019a and LCP Guan 2023
5.3.1 Other Tasks
Classification. We extend the FFCP techniques classification tasks using the baseline RAPS [An gelopoulos et al., 2020) model, creating new variant called FFRAPS (Fast Feature RAPS, Algo- rithm55
55in in Appendix B.7). According to the experimental findings presented Table FFRAPS returns shorter band lengths while preserving the coverage compared to RAPS under most model structures.
Segmentation. The gradient-level techniques of FFCP also prove effective segmentation tasks. The segmentation results in Figure 2 reveal that FFCP returns appropriate bands across different regions. Specifically, larger bands are observed in less informative areas, such as at object boundaries, whereas narrower bands are found in more informative regions. This validates the efficiency of FFCP in segmentation tasks.
5.3.2 Extending FFCP into Other Models
Conformalized Quantile Regression (CQR, Romano et al. [2019a] The gradient-level tech- niques of FFCP are adaptable other conformal prediction frameworks like CQR. We develop FFCQR (Fast Feature CQR, Algorithm in Appendix (B.5). which not only significantly reduces runtime compared FCQR but also exhibits better performance than CQR. Additionally, we observe that for the neural network significant level setting [a. 1 - al in the CQR method, as the value increases, approaching 1 a/2, the performance of FFCQR gradually improves. For detailed experimental results in Table 8 10
12 in the Appendix B.5
Locally Adaptive Conformal Prediction (L.PP, Guan 2023)
Integrating gradient-level tech- niques from FFCP into the LCP method leads to FFLCP (Fast Feature LCP, Algorithm Ap- pendix B.6). Experimental results in Table 13 indicate that FFLCP outperforms LCP in terms of group coverage, highlighting an improvement the adaptability of LCP locally adaptive methods.
5.3.3 Comparison of FFCP with Other Baselines
Self-Supervised Conformal Prediction (SSCP, Seedat al. 2023) We additionally evaluate the recent feature-CP method SSCP in Appendix B. 1. SSCP entails training two extra networks and is roughly 50x slower than FFCP. In all cases, FFCP also yields shorter bands, likely because SSCP depends more heavily on the base model's accuracy and on effective auxiliary training.

---

Full CP with CV+ [Barber et al. 2021 framework Although FFCP is a Split CP procedure, we also benchmark against the full CP baseline CV+ and observe consistent advantages notably improved efficiency with comparable coverage. Complete experimental details and results are provided the Appendix B.9
Conclusion
In this paper, propose FFCP, gradient-based non-conformity We establish its thcoretical validity under mild assumptions and demonstrate its broad applicability across regression, classification,, and segmentation tasks. We also introduce FFCQR and FFLCP, based on CQR and LCP, respectively. Finally, we evaluate FFCP in comparison with SSCP and the full CP baseline, CV+.
that is 50x faster than FCP.
Although FFCP gradient-baset can be extended to settings where gradients are not available or are unreliable-e.ge when gradients vanish or the function is non-differentiable-bb incorporating zero-th order methods such as finite difference approximations or perturbation- This further broadens the applicability of our framework beyond differentiable models. For future work, the following points could be considered: (1) We use information from the first derivative and have not delved into higher-order derivatives, which may contain more feature information; (2) The gradient at a single point may be unstable, especially when the gradient zero, so methods such as random smoothing could be considered.
surrogates.
Acknowledgments
This work is supported by Shanghai Science and Technology Development Funds 24YF271 1700 and Fundamental Research Funds for the Central Universities 20241 10586. The authors would like to express their gratitude to Dinghuai Zhang for his constructive insights and valuable discussions.
10

---

References
M. Abdar, Pourpanah, S. Hussain, D. Rezazadegan, L. Liu, M. Ghavamzadeh, Fieguth, X. Cao, A. Khosravi, U. R. Acharya, et al. A review of uncertainty quantification in deep learning: Techniques, applications and challenges. Information fusion, 76:243-297, 2021.
C. Achilles, H. P. Bain, F. Bellott, Boyd-Zaharias, J. Finn, J. Folger, J. Johnston, and E. Word. Tennessee's s student teacher achievement ratio (star) project. Harvard Dataverse, 1:2008, 2008.
A. Angelopoulos, S. Bates, J. Malik, and M. I. Jordan. Uncertainty sets for image classifiers using conformal prediction. arXiv preprint arXiv:2009.14110 2020.
A. U. Asuncion. Uci machine learning repository, university of california, irvine, school of informa- tion and computer sciences. 2007.
R. F. Barber, E. J. Candès, A. Ramdas, and R. J. Tibshirani. The limits of distribution-free conditional predictive inference. Information and Inference: A Journal of the IMA, 2020.
R. F. Barber, E. J. Candes, A. Ramdas, and R. J. Tibshirani. Predictive inference with the jackknife+. The Annals of Statistics, 49(1):486-507, 2021.
R. F. Barber, E. J. Candes, A. Ramdas, and R. J. Tibshirani. Conformal prediction beyond exchange- ability. arXiv preprint arXiv:2202.33415. 2022.
C. Blundell, J. Cornebise, K. Kavukcuoglu, and D. Wierstra. Weight uncertainty in neural network. In F. R. Bach and D. M. Blei, editors, Proceedings of the 32nd International Conference Machine Learning, ICML. 2015, Lille, France, 6-11 July 2015, volume 37 of JMLR Workshop and Conference Proceedings, pages 1613-1622. JMLR.org. 2015.
E. Burnaev and Vovk. Efficiency of conformalized ridge regression. Conference on Learning Theory, pages 605-622. PMLR, 2014.
K. Buza. Feedback prediction blogs. In Data analysis, machine learning and knowledge discovery, pages 145-152. Springer, 2014.
E. Candès, L. Lei, and Ren. Conformalized survival analysis. Journal of the Royal Statistical Society Series B: Statistical Methodology, 85(1):24 45, 2023.
E. J. Candès, L. Lei, and Z. Ren. Conformalized survival analysis. arXiv preprint arXiv:2103.09763 2021.
Y. Chang, X. Wang, J. Wang, Y. Wu, L. Yang, K. Zhu, H. Chen, X. Yi, C. Wang. Y. Wang, al. A survey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology, 15(3):1-45, 2024.
Y. Chen, D. Zhang, M. U. Gutmann, A. Courville, and Z. Zhu. Neural approximate sufficient statistics for implicit models. In International Conference on Learning Representations, 2021.
J. W. Cohen, S. B. Cohen, and J. S. Banthin. The medical expenditure panel survey: A national information resource support healthcare cost research and inform policy and practice. Medical Care, 47:$44-550, 2009.
M. Cordts. M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele. The cityscapes dataset for semantic urban scene understanding. In Proc. of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2016.
Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pages 248-255. Icce, 2009.
J. Gawlikowski, C. R. N. Tassi, M. Ali, Lee, M. Humt, J. Feng, A. M. Kruspe, R. Triebel, P. Jung, R. Roscher, M. Shahzad, W. Yang, R. Bamler, and X. Zhu. A survey of uncertainty in deep neural networks. CoRR, abs/2107.03342. 2021.

---

A. C. Gelijns, J. G. Zivin, and R. R. Nelson. Uncertainty and technological change in medicine. Journal of Health Politics, Policy and Law, 26(5):913-924, 2001.
I. Gibbs and Candès. Adaptive conformal inference under distribution shift. In Neur/PS, 2021.
L. Guan. Localized conformal prediction: A generalized inference framework for conformal predic- tion. Biometrika, 110(1)333-50, 2023.
C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger. On calibration of modern neural networks. ArXiv, abs/1706.04599, 2017.
X. Han, Z. Tang. J. Ghosh, and Q. Liu. Split localized conformal prediction. arXiv preprint MXXi:22006113992, 2022.
J. M. Herbández-Lobato and R. P. Adams. Probabilistic backpropagation for scalable learning of bayesian neural networks. In F. Bach and D. M. Blei, editors. Proceedings of the 32nd International Conference on Machine Learning. ICML 2015, Lille, France, 6-11 I July 2015, volume 37 of JMLR Workshop and Conference Proceedings, pages 1861-1869. JMLR.org, 2015.
X. Hu and Lei. A distribution-free test of covariate shift using conformal prediction. arXiv: Methodology, 2020.
R. Izbicki, G. Shimizu, and R. B. Stern. Cd-split and hpd-split: efficient conformal regions in high dimensions. arXiv preprint arXiv:2007 2020a.
R. Izbicki, G. T. Shimizu, and R. B. Stern. Distribution-fiee conditional predictive bands using density estimators. ArXiv, abs/1910.05575, 2020b.
Izmailov, S. Vikram, M. D. Hoffman, and A. G. G. Wilson. What are bayesian neural network posteriors really like? In International conference machine learning, pages 4629-4640. PMLR, 2021.
Z. Ji, N. Lee, R. Frieske, T. Yu, D. Su, Y. Xu, E. Ishii, J. Bang, A. Madotto, and P. Fung. Survey of hallucination natural language generation. ACM Computing Surveys, 55(12):1-38, 2023.
M. I. Jordan and T. M. Mitchell. Machine learning: Trends, perspectives, and prospects. Science, 349(6245):255-2666 2015.
L. V. Jospin, H. Laga, F. Boussaid, W. Buntine, and M. Bennamoun. Hands-on bayesian neural networks-a tutorial for deep learning users. IEEE Computational Intelligence Magazine, 17(2): 29-48, 2022.
V. Kuleshov, N. Fenner, and S. Ermon. Accurate uncertainties for deep learning using calibrated regression. In J. G. Dy and A. Krause, editors, Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pages 2801-2809, PMLR, 2018.
B. Kumar, C. Lu, G. Gupta, A. Palepu, D. Bellamy, R. Raskar, and A. Beam. Conformal prediction with large language models for multi-choice question answering. arXiv preprint arXiv:2305.184/1 2023.
J. Lei, A. Rinaldo, and L. A. Wasserman. conformal prediction approach to explore functional data. Annals of Mathematics and Artificial Intelligence, 74:29 43, 2013.
L. Lei and E. J. Candès. Conformal inference of counterfactuals and individual treatment effects. Journal of the Royal Statistical Society: Series (Statistical Methodology), 83, 2021.
Y. Li and Y. Gal. Dropout inference in bayesian neural networks with alpha-divergences. In D. Precup and Y. W. Teh, editors, Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pages 2052-2061, PMLR, 2017.
Morduch and R. Schneider. The financial diaries: How American families cope in a world of uncertainty. Princeton University Press, 2017.
12

---

J. Nixon, M. W. Dusenberry, L. Zhang, G. Jerfel, and D. Tran. Measuring calibration in deep learning. In IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2019, Long Beach, CA, USA, June 16-20, 2019, pages 38 41. Computer Vision Foundation / IEEE, 2019.
H. Papadopoulos and H. Haralambous. Reliable prediction intervals with regression neural networks. Neural Networks, 24(8):842-851, 2011.
H. Papadopoulos, A. Gammerman, and V. Vovk. Normalized nonconformity measures for regression conformal prediction. In Proceedings of the IASTED International Conference Artificial Intelligence and Applications (AIA 2008), pages 64-69, 2008.
H. Papadopoulos, V. Vovk, and A. Gammerman. Regression conformal prediction with nearest neighbours. J. Artif. Intell. Res., 40:815-840, 201
H. Papadopoulos, V. Vovk, and A. Gammerman. Regression conformal prediction with nearest neighbours. Journal of Artificial Intelligence Research, 40:815-840 201 lb.
A. Podkopaev and A. Ramdas. Distribution-free uncertainty quantification for classification under label shift. In UAI, 2021.
V. Quach, A. Fisch, T. Schuster, A. Yala, H. Sohn, T. S. Jaakkola, and R. Barzilay. Conformal language modeling, arXiv preprint arXiv:2306.10193 2023.
Y. Romano, E. Patterson, and Candes. Conformalized quantile regression. Advances in neural information processing systems, 32, 2019a.
Y. Romano, E. Patterson, and E. J. Candès. Conformalized quantile regression. In H. M. Wallach, H. Larochelle, A. Beygelzimer, F. d' Alché-Buc, E. B. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems 32: Annual Conference Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 3538-3548, 2019b.
Y. Romano, M. Sesia, and E. J. Candès. Classification with valid and adaptive coverage. arXiv: Methodology, 2020.
N. Seedat, Jeffares, F. Imrie, and M. van der Schaar. Improving adaptive conformal prediction using self-supervised learning. In International Conference on Artificial Intelligence and Statistics, pages 10160-10177. PMLR, 2023.
N. Seedat, J. Crabbé, Z. Qian, and M. van der Schaar. Triage: Characterizing and auditing training data for improved regression. Advances in Neural Information Processing Systems, 36, 2024.
M. Sesia and E. J. Candès. comparison of some conformal quantile regression methods. Stat, 9(1): e261, 2020.
M. Sesia and Y. Romano. Conformal prediction using conditional histograms. In M. Ranzato, A. Beygelzimer, Y. N. Dauphin, P. Liang, and J. W. Vaughan, editors, Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurlPS 2021, December 6-14, 2021, virtual, pages 6304 0444-6315,40 2021.
G. Shafer and Vovk. A tutorial on conformal prediction. J. Mach. Learn. Res., 9:371 2008a.
G. Shafer and V. Vovk. tutorial on conformal prediction. Journal of Machine Learning Research, (3), 2008b.
Y. Shen, X. He, Gao, L. Deng, and G. Mesnil. Learning semantic representations using convolu- tional neural networks for web search. In Proceedings of the 23rd international conference world wide web, pages 373-374, 2014.
D. Silver, J. Schritwieser,, K. Simonyan, I. Antonoglou, A. Huang, A. Guez, T. Hubert, L. Baker, M. Lai, Bolton, et al. Mastering the game of go without human knowledge. nature, 550(7676): 354- 359,359,40 2017.
13

---

J. Teng, Z. Tan, and Y. Yuan. T-SCI: A two-stage conformal inference algorithm with guaranteed coverage for cox-mlp. In M. Meila and T. Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, ICMI. 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pages 10203-10213. PMLR, 2021.
Teng, C. Wen, D. Zhang, Y. Bengio, Gao, and Y. Yuan. Predictive inference with feature conformal prediction. arXiv preprint arXiv:2210.001111111 2022.
H. Thirumurthy, D. A. Asch, and K. G. Volpp. The uncertain effect of financial incentives to improve health behaviors. Jama, 321(15):1451-11)
2019.
R. J. Tibshirani. R. E. Barber, E. J. Candès, and A. Ramdas. Conformal prediction under covariate shift. In H. M. Wallach, H. Larochelle, A. Beygelzimer, F. d' d'Alhéé-Buc E. B. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurlPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages
2019.
V. Vovk, A. Gammerman, and G. Shafer. Algorithmic learning in random world, volume 29. Springer, 2005.
H. Wei, R. Xie, H. Cheng, L. Feng, B. An, and Y. Li. Mitigating neural network overconfidence with logit normalization. In International conference machine learning, pages 23631-23644. PMLR, 2022.
C. Xu and Y. Xie. Conformal prediction interval for dynamic time-series. JCMI, 2021.
K. Xu, Z. Shi, H. Zhang, Y. Wang. K.-W. Chang, M. Huang, B. Kailkhura, X. Lin, and C.-J. Hsieh. Automatic perturbation analysis for scalable certified robustness and beyond. Advances Neural 2020.
Information Processing Systems,
Y. Yang and A. K. Kuchibhotla. Fimite-sample efficient conformal prediction. arXiv preprint arXiv:2104.13877 13877, 2021.
Z. Yang, E. Candès, and L. Lei. Bellman conformal inference: Calibrating prediction intervals for time series. arXiv preprint arXiv:2402.05203. 2024.
F. Ye, M. Yang, J. Pang. I.. Wang, D. F. Wong, E. Yilmaz, S. Shi, and Z. Tu. Benchmarking Ilms via uncertainty quantification. arXiv preprint 47X9::2401.12994, 2024.
14

---

Appendix
The complete proofs are presented in Section A and the experiment details are outlined Section B
A Theoretical Proofs
We prove the thcoretical guarantee for FFCP concerning coverage (effectiveness) in Section A. and band length (efficiency) in Section A.2
A.1 Proofs of Theorem 4.1
The proof based on the exchangeability of data (Assumption] 1 on the calibration fold and test fold, hence the key step we need derive is the exchangeability of the non-conformity scores 9t(X,Y, h) = - f(X)l/I ( Vg((ill- We define the relevant symbols: Lmm represents the train fold, Dies represents the test fold, Deal represents the calibration fold, and D' {(X,, Yi) helm] is the intersection of the two folds. is the number of data points in D'
Similar to Teng et al. 2022]. we first prove that for any function independent of D', h(X, (X Yi) satisfies exchangeability. For the CDF FR of h and its perturbation CDF Fg. π is a random perturbation. We can conclude,
X x y + R. which is
FR(uI,..... Un Dira)
P((( Y1) E uı,..., h(Xn, Yn) S un Dua);
=P((X,, Yi) Cj (m1-)))).... (X V,) C₁ ((n Dun); -P((X=(1)) X=(1)) E C₁ [u) (uI-)..... (Xa(n), Ya(m)) E Cj ((n=) Dma) (14)
=P(h(X+(1), (1(1) S Ui,.... h(N,(,(()))))))) Yz(n)) Un Dua),
-1 Un Dmm)
where (u-) = {(X,Y) 7(x, Y) S w). Next, we need to show the non-conformity score function
s [x Y,g o Y - ((X)l
(15)
which is independent of the dataset D
We can that the non-conformity score s (X, Y, o h) on D' uses information from g and h, both of which depend only on the training set ' Moreover, calculating this non-conformity score in the Algorithm uses only single-point information, not the entire dataset D'.
By integrating the aforementioned, we deduce that the non-conformity scores sp (X,Y, h) on D' exhibit exchangcability. This exchangeability, per Lemma in Tibshirani al. 2019)] lends theoretical support to the efficacy FFCP.
A.2 Proofs of Theorem
Our main conclusions inspired by Theorem 4 in Teng et al.) [002]] The details are follows
Definitions. Let P denote the overall population distribution. The calibration set Deal consists of samples drawn from P. We denote the specific distribution of these samples p" The model under considcration, f = g h, includes h as the feature extractor and g as the prediction head, with g assumed to be continuous function. Vg represent the individual length in output space, given data set D. The term Q1-a a RR) represents the (1 a) -uqantile of the set R. which is adjusted to include the value 0. Furthermore, M|-I signifies the mean value of set, and subtracting real number from set indicates that the subtraction is applied uniformly to all elements within the set.
Split CP. Let V {re hiE,, denote the individual length in the output space for Split CP, given calibration set Deat- Since Split CP returns band length with a quantile of non-conformity score, the resulting average band length is derived by 2Q (V2
15

---

Fast Feature CP. According to the definition of FFCP, Vid = V//
The resulting band length in FFCP is denoted by 2B 2 (1 /
((caa)) (caa))l) Theorem A.1. (FFCP is provably more efficient). Assume that the non non-conformity score is in norm- rype. We assume Holder assumption that there exist 0, L 0 such that [=( X) H(y X)) S L| yla for all X, where H. denotes the length of the prediction interval in the output space for samples. Then if the feature space satisfies the following square conditions:
Qi-a
1. Expansion. The feature space expands the differences between individual length and their quantiles, namely, DE p..p-pn.eee poIMMII (V /V///VV VVg(()) VB/IVv((0lI Do pap.a(QQaaaaV Vgl max [L, (T)lc///
2. Quantile Stability. Given a calibration set Deal the quantile of the band length is stable in both feature space and output space, namely, ED- 1Q [VB/[[] S Fa and ED-P" (Q1 Q a (V
@V
Vn
Then FFCP provably oulperforms Split CP in terms of average band length, namely,
B(x-y'>><1 V((vil Q Qv g < QV-a(VD..))
where the expectation taken over the calibration fold and the testing point (X',Y') Y!).
Proof Theorem A.7 We first proof with Expansion Assumption,
LED-pn ( (
(g()]l - -aapoilllVVVVV///a V//IVv(00)II < Ep.p.MMQ1-a( -V]l
(16)
2 max (x/e/// min miinff/
And obtain
EpMV3 <EpQ1-a(VB)
-2max (/) mijia(aa))) M|D..p. ||
(17)
According to Holder condition for quantile function, we obtain that M(]Vs(i) therefore
Q1-a /V// V MV8 + (VB Q1-a V I)) < EpQ1-a(VB) 2 maxx(1 Lc/e (nimm/) (18)
As the Quantile Stability assumption, we have that EDapn 10:0 (V__ Q Q1 (VB.g/ (V_oVVV__[[VV(((aaa))
Q1 and ED-r= Q Q1-0 - (V (V 1 in Therefore,
Vn
VE( Q1-a ([Vaa()))))
<2Q1-a - -2max [c/ /nn]///]nn mijmmaaaa.)
(19)
<2Q1-a(VB)
B Experimental Details
Section B.T introduces the omitted experimental details. Section B.2 certifies the square conditions. Section B.3 discusses discusses the robustness of FFCP coverage with respect to the splitting point and across each network layer. Section demonstasssss that FFCP performs similarly to Split CP in untrained neural networks, confirming that FFCP's efficiency is due to the semantic information trained in the feature space. Section B.5 proposes FFCQR after applying the gradient-level techniques of FFCP to CQR. Section B.6 proposes FFLCP after applying the gradient-level techniques of FFCP LCP. Section B.7 proposes FFRAPS after applying the gradient-level techniques of FFCP to RAPS. Finally, Section B.8 provides additional experimental results.
16

---

0,35
0.35
0.35
0,30
0.30
0.30
0.25
0.25
Spore
0.25
0.20
0.20
0.20
0.20
0.15
0.15
RCP
0.15
0.15
0.10
0.10
0.10
0.10
0.05
0.05
0.05
0.05
0.00
0.00
0.00
0.00
FFCP Layarl Soore
FFCP Layer2 Score
FFCP Layer3 Score
FECP Layert Sooce
(a) Layerl
(b) Layer2
(c) Layer3
(d) Layer4
Figure 3: Scatter plot of FCP Score and FFCP Score at different layers. The relationship between FCP Score and the FFCP Score is positively correlated, which indicates that the FFCP Score
effectively approximates the FCP Score.
B.1 Experimental Details
All the tests are performed on a desktop with an Intel Core 19-12900H CPU, NVIDIA GeForce RTX 4090 GPU, and 32 GB memory.
We conduct several more experiments in to establish the close relationship between FFCP and FCP, to demonstrate the benefits of FFCP from the good representation of gradient, and to provide empirical validations for the theoretical insights.
Model Architecture. For the one-dimensional employ four-layer neural network, with each layer consisting of 64 dimensions. For the semantic segmentation experiment, we utilize network architecture combining ResNet50 with two additional convolutional layers. We use ResNet50 the base feature extractor h, and the two subsequent convolution layers form the prediction head
Correlation Between FCP and FFCP Scores Across Layers. We compare the relationship between the scores of FCP and FFCP through experiments. the non-conformity scores of the two algorithms, suggesting that FFCP shares similarities with FCP score function. This suggests that FFCP, while computationally efficient, provides a close approximation to FCP. The observed discrepancies may be attributed to the complex, layer-dependent non-linear transformations introduced by the decoder, under which the accuracy of the Taylor-base linear approximation tends to decline the degree of non-linearity increases.
positive correlation between
Decoder Runtime Comparison. We conducted additional experiments evaluate how the decoder's layer depth influences computational efficiency. The rumning times presented below consider the decoder at various depths, explicitly showing the proportional relationship between decoder depth and computational cost.
In our primary comparison (FFCP vs. FCP), we maintained consistency by selecting the layer index as 2 for the FFCP, aligning with the depth used by FCP. The detailed experimental results are summarized in Table
Table 4: Running time comparison (mean + std) for different decoder depths in STAR dataset.
LAYER DEPTH
SPI.IT CP
FCP
FFCP
FASTER
2 LAYER
0.0009
3.1302.10.477
0.0247110.0053
127x
4 LAYER
0.00-7=000000
5.8031 2010
0.023910.0010
243x
6 LAYER
0.0058=0.00 0003
9.4411+0.5277
0.0277+0.000
341x
8 LAYER
0.0064=0.0000
12.2794=0.1668
0.029440.000
00002
418x
10 LAYER
0.0072=0.000
15.8716##.3388 33663
0.0335+0
474x
12 LAYER
0.0077=0.0003 19.435240.8047
0.037344.0021
521x
These results demonstrate that while the computational cost FCP grows significantly with increasing depth of the decoder network, FFCP remains computationally efficient, only exhibiting marginal increases runtime. Thus, FFCP provides substantial efficiency benefits, especially when employing deep network architectures.
17

---

Robustness for FFCP. The empirical performance of FFCP demonstrates its robustness, as seen in the ablation studies on splitting points. We demonstrate that coverage remains robust across different splitting points in neural networks, detailed in Table in Appendix B.3 Furthermore, the results from different layers of the FFCP network are consistent, as presented Table 3
B.2 Verifying Square Conditions
We verify the square conditions in this section. The key component of the square conditions is Expansion condition, which states that performing the quantile step does not result in a significant loss of efficiency.
For computational simplicity, We take exponent a and do not consider the Lipschitz factor We next provide experiment results in Figure On comparing the distribution of the scores between Split CP with FFCP.
From the figure, we observe that the overall distribution of FFCP non-conformity scores is closer the quantile. This numerically validates that M I Vg(o))) VB/[Vg(0)]] is less than M (Q1-a(V)) Vol.
Vantlla
Rak
Daied Ron
Halagram Deeait
Hatogram Dematty
Sas
24
01
Score
Score
(a) Split CP
(b) FFCP
Figure Empirical validation of Theorem A. We plot the score distributions and their corresponding quantiles (a = 0.1) of Split CP (left) and FFCP (right). Compared to Split CP, the non-confornity scores of FFCP are closer their quantiles, leading shorter band. Compared Split, FFCP exhibits more stable distribution with higher quantiles, leading to better performance for FFCP. FFCP selects layer 2 for display.
B.3 Robustness of FFCP
To verify that the coverage by FFCP maintains its robustness despite changes in the splitting point, we performed a network split. The experimental results, detailed Table demonstrate that FFCP indeed robust.
Table 5: Ablation study of the number of layers in h and g in unidimensional tasks. For the sake of avoiding redundancy, set 0.05.
MEPS19
NUMBER(S h)
SPLIT CP
96.24
0.16
4.60 1: 0.50
95.35
7.34 1: 1.01
95.08
0.11
7.88 1: 0.97
h:0
94.88
0.19
5.35
1 0.42
95.18
0.40
5.36 1: 0.52
94.97
1: 0.31
8.28 1: 0.59
h:3
3
94.54
0.14
4.21 0.16
95.13
0.38
5.15 0.19
91.99
0.18
6.37 + 1.22
FFCP
h: 2
2
95.14 0.13
2.48
+ 0.09
96.19
0.36
5.37
+ 0.87
95.08
0.12
5.36
+ 0.93
11
95.16
0.18
2.50
95.34
+ 1.13
95.05
0.11
1.52
h:44
0:
95.24
0.16
4.60 1. 0.50
95.35
0.23
7.34 1: 1.01
95.08 3. 0.11
7.88 J. 0.97
18

---

Table 6: Untrained model comparison between Split CP and FFCP. When the model has not been sufficiently trained, FFCP performs similarly to Split CP. This means that the model's performance determines the quality of the feature information in the gradient. When the model performs poorly, the gradient information obtained by FFCP is inaccurate. On the other hand, this also suggests that FFCP effectively utilizes the feature information in the gradient when the model is well-trained.
METHOD
SPLIT CP
FFCP
DATASET
COVERAGE
LENGTH
COVERAGE
LENGTH
SYNTHETIC
90.23:10.45
2..0
90.2210.96
2.00
COM
90.33++181
4.86+0.13
90.43+1.99
4.73#0.08
FB1
90.18:t0.19
3.57-10.09
90.10+t..13
3.57
FB2
90.16
3.66:10.11
90.1240.114
3.66:0.06
MEPS 19
90.80++0.3
4.33
90.85+0.58
4.38=0.07
MEPS20
90.15++0.55
4.21
90.2740.03
4.46=0.25
MEPS21
89.80+0.46 45
=0.17
89. 09.
4.41
STAR
89.79110.51
1.88 1.001
89.98.10.56
1.94::0.01
BIO
90.16.10.20
1.09.10.02
90.0711.14
4.04
BLOG
90.11.0.30
2.53:10.12
90.1210.28
2.55
BIKE
89.55+00.8
4.56=0.09
89.5740.86
4.60=0.10
Table 7: Untrained model comparison between Split CP and FFCP by Layer on Each Dataset (lower better).
DATASET
LAYER1
LAYER2
LAYER3
LAYER4
LAYER5 (SPLIT)
SYNTHETIC
2.87
2.81+0.03
2.8910.01
2.41+0.01
2.34
COM
5.41+0.14
5.30+0.18
5.03+0.09
4.73+0.08
4.86+0.113
FB1
3.90+1.01
3.77:0.10
3.6240.08
3.56=0.08
3.56.10.09
FB2
4.02+0.09
3.90+0.8
3.74+0.08
3.66=0.06
3.6611011
MEPS19
4.38+0006
4.41#0.07
4.42+0.00
4.38+0.07
4.33
MEPS20
4.43+0.21
4.4410.23
4.48+0.22
4.46-0.25
4.41
MEPS21
1.41
4.46+0.18
4.48+0..18
1..
1.41
STAR
2.4940.06
2.40+0.05
2.1540.03
1.94
1.88
BIO
4.2740.02
4.1440.02
4.07+0.00
4.04+0.02
4.08++0.02
BLOG
2.56+0.15
2.5440.15
2.5740.13
2.5540.14
2.53
BIKE
4.69
4.67+0.07
4.57+0.09
4.60=0.10
4.56
B.4 FFCP works due to semantic information in feature space
One of our primary advantages is that FFCP leverages the semantic information of gradient in feature space. This due to the fact that gradient-level techniques in feature space improve efficiency via the robust feature embedding abilities of well-trained neural networks.
On the other hand, when the base model is untrained and initialized randomly, lacking meaningful semantic representation in gradient, the band length produced by FFCP is comparable to Split CP.
For results, see Table 6
FFCP on untrained network. We propose that FFCP returns shorter band lengths through its deployment of deep representations from the gradients. To test this view, we contrast FFCP's performance using untrained neural network against a baseline model. Using an incompletely trained neural network, FFCP's performance deteriorates and becomes comparable to that of Split CP This is due to the partially incorrect semantic information in the gradient, which mixleads FFCP. We defer the results to Table and have updated the results in Table 7 by selecting the best-performing layer, which confirms that FFCP underperforms when the model is not sufficiently trained.
19

---

B.5 FFCQR
This section highlights the adaptability of FFCP's gradient-level techniques, showing their suitability for a wide range of existing conformal prediction algorithms. We choose Conformalized Quantile Regression (CQR, CoRR.Romano al. 2019b) to propose Fast Feature Conformalized Quantile Regression (FFCQR). The fundamental concept is similar to FFCP Algorithm 2 where calibration steps are performed within the gradient information. FFCQR algorithm is proposed in Algorithm 3
Algorithm 3 Fast Feature Conformalized Quantile Regression (FFCQR)
Input: Confidence level a, dataset 2 {(X,, HE), test point X'; Randomly split the dataset D into a training fold Dua - (X,, Y.)SeI. together with calibration
fold Deal ,X,, Ye)ez..i
2: Train base machine learning model f = o hi-)) and fhi = ghو h(-) using Dira to estimate
the quantile of response Y,, which returns (X;), f (X;)];
3: For each i E Teal, calculate the non-conformity score R. = (X,, /))lIl and
Rhi = (Y; (X.))
where Vg(-) denote the gradient of g(-) on the feature dg"oh((;) and (0,) oh
6, - (X,), namely (6.) dh(X.)
d
4: Calculate the -1-a// a/2)-th quantile Qi-a/2 of the distribution [Zal+ ΣiEI,, oR. + 80 where
R₂ = mmxx
R/o Rbj
Output: C1-a/2
C.llegr (X')
= [ (F) -17- [Vg (f)11-(1-a/1)))) Q1-0/2]
where 0' = h(X").
We summarize run time in Table X and the experiments result in Table (meps19), Table 10 (com), and Table (bike). FFCQR reduces runtime compared FCQR, while achieving better efficiency compared to CQR.
Furthermore, we have observed that as the values of [ax, 1 1 ai used by the neural networks in all CQR methods (CQR, FCQR and FFCQR) become increasingly closer in the training process (The level difference between [0.1, 0.9] is 0.8, while the level difference between [0.49, 0.51) is 0.02, with the difference gradually decreasing), the band length returned by FFCQR gradually narrows. This implies that our method holds an advantage returned band length when the narrower neural network confidence level.
Table 8: Time Comparison among CQR, FCQR and FFCQR. For quantile regression tasks, FFCQR also demonstrates more efficient performance. The last column represents the speed improvement factor of FFCQR compared to FCQR. The time unit is in seconds.
DATASET
CQR
FCQR
FFCQR
FASTER
SYNTHETIC
0.0125:10.0622
0.0742-0.0091 0.3237
4x
COM
0.0045:10.0015
0.2730 1088
0.0210=0.0011
13x
FB
0.0446:20.0157 1.7276=1.1159
0.2532 -0.32=0.0166 0166
7x
FB2
0.0812+t000157
3.9967=20.73 7330
0.0617=0.0122
65x
MEPS 19
0.0187+0.0018
0.767110.0438
0.1189 0048
6X
MEPS20
0.0438+t0.009
1.1876=0.2200
0.1505
8x
MEPS21
0.0187+0.0027
0.800440.0615
0.1120=0.0055
7x
STAR
0.0047+0.0009
0.2352=0.0419
0.0214=0.000 0005
11X
BIO
0.077410.0041
6.9365 145.494
0.64173:0.4877
11X
BLOG
0..12110.0153
1.9591 10.1446
0.39-11 1000618
SX
BIKE
0..008
1.852812.3969
0. 0.238210.3261
6x
20

---

Table 9: Coverage and Band Length Different Net Confidence Levels Used By the Neural Networks in CQR methods with Meps/9 detaset. FFCQR yields shorter band lengths compared to CQR.
CONFIDNNC LEVELS
00.1.0.9
10.2, 0.81
00.3.0.71
[0.4, 0.61
10.49, 0.511
METRICS
COVERAGE
LENGTH
COVERAGE
LENGTH
COVERAGE
LENOTH
COVERAGE
LENOTH
COVERAGE
LENOTH
SPLIT CQR
80.28:0:47
2.432:0.11
900119.0.4
2.5880.40
500
2.33300.11
90.4850042
3.47
90.4839.35
3.48s0.s
FCQR
91.3218-37
1.50..0.0
30-261.0-32
2,01:3.0
90.
2.901.9.36
90.5810.11
90.17.0
4.50.1
50.291..00
2.61..-13
50.221..30
2.0819.54
90. 30.53.10.33
3.08
89.95.0.39
5.90.0.57
19.541013
5.9611-35
2.50
2.0250.45
5.714122
FFCQR
LAYER2
00.14=5.00
2,38:8-12
900.44.0.08
2.22:0232
31..3:00.33
2.00:0.08
89.90
89.9668..4
4.10am.22
80.5749.33
4.768117
LAYER3
00.21=8.49
2.18=0.12
90.14.0.42
2:10
90. 30.33200.0
2.34:0.39
90.25:00.35
2.70a9.25
89.80:9.40
3,162902
1AVI88
50.2818447
2,431.0:11
90.191...0 16
2.58
90.6310033
2,83300.31
90-4810.42
3/47100.36
90/44110:30
3.4.10.5
Table 10: Coverage and Band Length at Different Net Confidence Levels Used By the Neural Networks in CQR methods with com dataset. FFCQR yields shorter band lengths compared to CQR.
CONROENCE LEVELS
[0.1
[0. 2.2, 0.80
[0.3.0.7]
[.64.0.6]
[0.49.0.51]
METRICS
COVERAGE
LENGTH
COVERAGE
LENGTH
COVERAGE
LENOTH
COVERAGE
LENOTH
COVERAGE
LENOTH
1.5750.12
1.7420.16
FOQR
50.83:1.53
0:37:0:06
LAVERO
58.02=2.18
1.62=8.12
89.62e1.78
1.67eaun
1.82:0.12
89.77a1.84
1,80:0.27
82.1221007
1.7649.11
LAYIIS]
58.07..9.0
1.599.8312
1.68
90 18.11:11
10:33
89.8211.75
1.78300.33
80. 00.12211.
1.7410.09
FFCOR
LAYERE
89.77:2.14
1.08::632
89.92233.00
1,683:0322
901 18.5330.43
1.64
89.67a1.28
1,8910.35
48.77.99.75
1.7810.33
1AYERR3
1.58::0:1 12
69.02g)) 20
1.67
80. 5330 46
1.78 18
80.62230.4 as
2.03:10.54
80.27:1900
1.90g0
LAYER4
80.87=1.06
1.57e0.12
90.13ee.50
1.7180.18
89.87=1.ce
1.7480.26
89.27.3.24
2,37
89.57:9940
1.9929.12
B.6 Group coverage
Group coverage is represented by the conditional probability P(Y E (()) The test dataset was categorized into three groups by splitting response Y based on the lower and upper tertiles, and we have reported the minimum coverage for each group.
We present our results in two parts: (a) present the group coverage provided by Split CP, FCP, FFCP, detailed in Table 14 and (b) the group coverage provided by LCP and FFLCP, as shown in Table 1
Analyzing the experimental results, we believe that the group coverage achieved through gradient- level techniques in FFCP reflects improvement over Split CP, albeit with moderate overall performance. We note that the group coverage of gradient-level conformal prediction is contingent upon its Split version. That is, when the Split version demonstrates satisfying group coverage, the gradient-level version tends to mirror this result. Thus, despite FFCP outperforming Split CP, the overall performance is still considered average.
LCP, developed specifically to enhance group coverage, inherently achieves higher coverage. Ex- perimental results further reveal that FFLCP surpasses LCP, demonstrating the superiority of our gradient-level techniques.
Algorithm 4 Fast Feature Localized Conformal Prediction (FFLCP)
Input: Confidence level a, dataset D {(x,, Y)),E, tesing point X', localizer D(X,Y)
1: Randomly split the dataset D into training fold Dum {(X,, Yi) and calibration fold
Deat & {(X,, Y)), :
2: Train a base neural network with training fold f(-) g o h(()) with training fold Dim:
3: For each i E Teal calculate the non-confornity score R; = IY: - ((X))//VVV (l))). where
Vg(0;) denotes the gradient of g(-) on the feature 0; - h(X,), namely Vg(0;) dh(X,) D. and a quantile Qi-a of the
dgoh(X.).
4: Calculate the distance D; A D(X', Xz). di = Σiez, D.
distribution Σiern dPOn, + 6
Output: cT')'(()) = - 1200
where 1' = =((X'.
21

---

Table 11: Coverage and Band Length at Different Net Confidence Levels Used By the Neural Networks in CQR methods with bike dataset. FFCQR yields shorter band lengths compared to CQR.
CONFIDNNE LEVELS
0..1.
10.2, 0.81
00.3.0.71
[0.4, 0.61
10.49,
METRICS
COVERAGE
LENOTH
COVERAGE
LENGTH
COVERAGE
LENOTH
COVERAGE
LENOTH
COVERAGE
LENOTH
SPLIT CQR
89.3888.7
0.8220.07
89.39e0.00
89.
0.77a0.05
90.
00.25e0.42
0.842m042
89.7259.5:
0,50
FCQR
90.2518.47
0.58.8.15
0.651:0:13
89.7710.70
0.8210087
89.58.0
0.91..0.7
89.84..044
0.977..00
89. 99.4210..33
1.201.0.4
89.7510.51
1.54.0133
10.5
1.
60.57
0.90
0.90=a.00
1.049900
1.25:40ss
29.6224.25
1.31490s
FFCQR
LAYER2
89.72=8..2
0.8788.07
89.704e0.27
0.79
89. 03330.c
0.8350.05
89.1420.42
0.5224084
89. 49.4449.43
0.98s9..
LAYER3
89.4928.54
0.8820.00
89.0220046
0.09
0.8999..2
89.5664.33
0.70amu81
89.57:40.8
0.7859.07
1AV188
89.391877
0.821.0.07
89.999.0.00
0.73
89. 19.033.0.44
0.7710..0
90.25,.042
0.8410042
89.72.10:5:
0.96100.5
Table 12: Coverage and Band Length at Different Net Confidence Levels Used By the Neural Networks in CQR methods with bio dataset. FFCQR yields shorter band lengths compared to CQR.
CONRDENCE
(0.1.0.9)
(0.2.0.8)
(0.3.0.7)
[0.4.0.6]
[0.49,0.57] 0.51]
MurricS
COVERAGE
LUNGTII
COVDRAGE
LUNGTI
COVERAGE
LENGTI
COVERAGE
LINGTE
COVERAGE
LENGTII
SHLIT CQR
69, 81
1.4250.22
69.84
1.45e9.0
89 5750.7
1.8100.c
00:0/:0.31
1.804005
00,1649.4
2,00s005
FCUR
90.18:8.35
0.9550.50
90.40ee..4
2.19900.41
00.
1.84s0.43
2.37
90.21:9.45
2.02:0.44
69.7418.32
1.47:0.01
89.981..2
89.
1.7310.04
89.57.0-34
2.22:0:15
80.6410
2.561006
89.7718.30
1.45.0.01
1.4819.00
89. 90.9210.33
1.5010.50
80.6010
2:21:0011
EFCQR
1AYER2
60.77s0.00
1.4380002
50.01:00.8
22
1.89
00.01:0.44
1.7620.11
80.70040:3
1.91190
LAYEIL3
89.7h=8.41
1.41:8.02
89.9890.34
1.38
1.47a0.01
90.07:.032
1.88
N0.57.9.34
1.78s0.02
LAYEB4
89.8988.41
1.42:0002
89.8480.27
1.40e0.02
89.87a0.27
1.8180.02
90.0750.33
1.862004
90.16:9.40
2,0029.03
B.7 FFRAPS
In this section, we show how deploy gradient- techniques in FFCP in classification problems. The basic ideas follow Algorithm 5
Comparing to the experimental part of RAPS, core adjustments are as follows:
(a) During the calibration process, for the model's 's output sorted scores 8, we divide each element by the magnitude of its corresponding gradient: s 8--5.| /g(()]]. Here, 5 is an adjustable hyper-parameter that can be tuned to optimize the performance of the model based on the specific characteristics of the data and the problem at hand.
(b) In the stage of calculating the returned set, we multiply the generalized inverse quantile r by the magnitude of the gradient of the corresponding test data: s' + 5- VV
We summarize the experiment results in Table 15 where we adhere the statistical methodologies of RAPS as described in Angelopoulos et al. 2020
Algorithm 5 Fast Feature Regularized Adaptive Prediction Sets (FFRAPS)
Input: Confidence level a, dataset
{(X,, Yi) tesing point X', and ground-truth label y E {0, 1, K}" for X E 'D and X'; regularization hyperparameters kyeg. 8 and X;
1: Randomly split the dataset D into training fold Pm & {(,,, Yie and calibration fold
Deal & {(X,, He)e...
2: Train a base neural network with training fold f(-) = g o h(.) with training fold Din:
3: For each i Teal, Lú j such that It, yi. where T represents the associated permutation
of index. Calculate generalized inverse quantile conformity score E, s
St. + [[
8 + A(L, k where Vg(ū,) denotes the gradient of g(-) on the feature é; e M(X,).
namely Vg((:)) = dgoA(X.), where & A sort((X) represents the sorted scores. Calculate
dh(X,)
fccal + [[1 - a)(1 + n)] largest value in {E, 1
4: Calculate L f 1 {ji e E/ + g() 8 A(j - -Kire)) S &ocal#) } + 1, where
='= "x'''" and 5 = =xonf((');;
Output: CHFRAPS (X')= = (1.....1.)
22

---

Table 13: Comparison of LCP and FFLCP in group coverage. We divide the datasets into three groups based on the size of Y, and calculate the coverage for each group, returning the maximum coverage. FFLCP shows the results for the 5-layer neural network.
METHOD
LCP
FFLCP
DATASET
COVERAGE
LAYERO
LAYER
LAYER2
LAYER3
LAYER4
SYNTHETIC
87.02=+100
86.93+0.78
86,57
85.43++124
87.11=1.76
87.02
COM
80.33+3.24
81.8442.52
79.56:3.34
77.42+4.12
79.41=2.88
80.33+3.24
FB1
1..71
78.61 #0.91
76.39=1.21
1..16
57.82=1.88
1..71
FB2
1..3
0..8
75.44=0.91
70.45=0.99
60.18=1.73
54.33=1.75
MEPS19
1..3
68.19#2.31
66.44=2.01
64.94+1.63
67.14=1.24
17.35=1.21
MEPS20
65.4911.64
19.3011109
1..5
65.1411.44
6..47
65.4911.64
MEPS21
66.38100.95
17.8211.10
1..9
66.21
6..54=1.02
60.98
STAR
77.2013.97
79.69 1.888
79.28
77.17
7.13
77.2013.97
BIO
81.10=0.61
86. +0.5
86. 36.06=0.50
86.78=0.57
83.26=0.71
81.10=0.61
BLOG
48.99+1.01
61.01
12
46.01=0.65
46.88=0.81
48.99#1.01
BIKE
77.61+1.52
81.02++1.73
82.42:2.08
82.97+1.29
84.41=1.71
77.61#1.52
Table 14: Comparison of Split CP, FCP and FFCP in group coverage.
METHOD
SPLIT CP
FCP
FFCP
DATASET
COVERAGE
COVERAGE
LAYERO
LAYER
LAYER2
LAYER3
LAYER4
SYNTHETIC
87,08:1100
87.92=1.08
86,96.10.8
86 06.63:0.79
85.64+1113
88.46+1.44
87 07.08:1..0
COM
79.41+3.12
79.57
82.00
79.41
78.6444.35
78.65+3.62
79.41
FB1
56.6911 1.35
1..3
79.20:10.95
76.75
68.0911.76
59.33
1..69
FB2
57.98+ 1.28
58.72=0.87
76.27+t0.92
75.64=0.91
70.86+40.69
12.43=1111
57 1.22
MEPS19 19
73.78:111.08
73.82:10911
70.904:2.00 29
70.51+2.25
12.09:1..25
73.53+1.00 1.00
73.78=1.08
MEPS20
12.21+11.47
72.33
70.42+40.88
70.13:1.42
69.5140.79
71.17+2.01
72.21
MEPS21
4..38
72.02
19.40
69.83 1.44
69.8141.68
70.85+10.82
71.39=0.20
83.45
83.17=3.47
82.89+1151
81.22+2..5
81.22:43.60
83.03+22007
83.45-3.09
81.00:10.61
84.45=20.88
87.31110.27
80.27
88.31
84.20.0.70
81.00
BLOG
58.32:10.50
1.44
65.21
59.03#1100
54.55++0777
55.7621.28
58.32=0090
BIKE
77.55:+:1.40
86, 0.82570087
95.36:11.922
94.23#1.40
95.06:1:1.06
84.65=11.85
77.55=1.40
B.8 Additional Experiment Results
This section provides more experiment results. Additional visual results for the segmentation problem also presented in Figure
23

---

Table 15: Comparison of FFRAPS with the state-of of-tte--ttt ahr method RAPS on Imagenet- Val. The FFRAPS method outperforms RAPS in most datasets.
METHOD
ACCURACY
COVERAGE
LENGTH
MODEL
TOP-1
TOP-5
RAPS
FFRAPS
RAPS
FFRAPS
RESNEXTI01
0.793=0.001
0.945
0.908+0.000
0.907
2.012+0.035
2.006=200039
RESNETI52
0.784=0.001
0.941+0.000
0.909:1.000
0.907
2.144
2.128 20.058
RESNETI01
0.774=0.001
0.935++.000
0.906:t0.00
0.906+0.000
2.348 151
2.256=10008
RESNET50
0.761=0.001
0.929=40001
0.907 00.04
0.907=00000
2.560 104
2.59440.000 009
RESNETI8
0.698=0.001
0.891=0.000 001
0.906=0.00m 003
0.903+0.00 000
4.560+0.144 147
4.434=0.160
DENSENET161
0.772=0.001
0.936=+0.001 001
0.907 003
0.907=0.002
2.374+0.008
2.328+0.050
VGG16
0.716
0.90411.000
0.904110.002
0.90211.0022
3.566100.09
3.521
INCEPTION
0.696.2.000
0.887 1.8000.001
0.903.10000%
0.9031100022
5.41011.3550
5.407
SHUFFLENET
0.69411..001
0.883 0..83110.001
0.902.10.001
0.901110.002
5.00110.121
4.971 10.073
(a) Image (b) Ground Truth Label
(c) FCP Length
(d) FFCP Length
Figure 5: Additional visualization results in segmentation task.
B.9 Additional Experiments on Full CP
We further validate the generality of our proposed approach by applying it to the CV+ (Barber et al. (2021) framework. Experiments were conducted synthetic multi-class classification dataset generated using scikit-learn. total 1,200 samples were created, with 1,000 used for training and 200 for testing. The dataset contained 10 features, including 3 informative and 2 redundant ones, distributed across distinct classes, each consisting of a single cluster. Our method achieves higher efficiency while maintaining comparable coverage the standard CV+ approach.
Table 16: Comparison of FFCC+ with CV+ on synthetic dataset.
NUM
CLASS=3
CLASS =
CLASS 8
METHOD
COVERAGE
SIZE
COVERAGE
SIZE
COVERAGE
SIZE
CV+
90.50+0.01
1.23
90.0040.01
1.5744.01
92.004..01 91.50+0.01
2.1310.01
FFCV+
91.00=0.01
1.20=0.01
02.50+0.01
1.5540.01
2.11+0.01
24

---

B.10
Additional Experiments on Self-Supervised CP
SSCP (Seedat 2023] is a method that leverages self-supervised signals to enhance the adaptability and efficiency of conformal prediction intervals. As shown in Table 1, however, SSCP introduces substantially higher computational overhead due to the need train additional networks, making it approximately to 50 times slower than FFCP. Across all settings, FFCP achieves shorter prediction bands, which may be attributed to SSCP's stronger dependence on the base model's predictive quality and the complexity of its auxiliary network training.
Table 17: Results on FFCP and SSCP
METHOD
SSCP
FFCP
DATASET
TIME
LENGTH
TIME
LENGTH
SYNTHETIC
7.1810099
0.25 0.002
0.15:10.02
0.18=0.01
COM
1.3310.16
2.52:10.14
0.03.10.01
1.84
MEPS19
10.86110.76
5.32:10.33
0.1411.00
3..3
STAR
1.96+0.02
0.67+0006
0.67
0.21+0001
BIO
13.49#3.81
1.53=0.03
0.39=0.04
1.66=0.02
BIKE
5.92+0.00
0.73=0.01
0.09=0.01
0.63=0.03
25

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction clearly state the key theoretical and empirical contributions, consistent with the main results.
Justification:
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: We discuss the main limitations, including assumptions data distribution and computational scalability, the paper.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: All assumptions are clearly stated alongside the theorems, with complete proofs included in the appendix.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Experimental settings and evaluation procedures are described in detail in the main text and appendix.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: An anonymized link to the code and data provided the supplemental material along with usage instructions.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: Full details about training, testing, hyperparameters, and optimizers included in the main text and appendix.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We report standard deviations across multiple runs to indicate the statistical significance the results.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
26

---

Answer:
[Yes]
Justification: Details on hardware specifications, runtime, and total compute budget are provided in the appendix.
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: The research adheres to the NeurIPS Code of Ethics and presents no ethical concerns.
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: This work theoretical and does not directly relate to applications with societal impact.
Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
The paper does not involve any models or datasets with potential risk of misuse.
Justification:
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All external assets are properly cited with license terms stated in the appendix main text.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: No new assets are introduced in this work.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The research does involve any human subjects crowdsourcing.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The study does not involve human subjects and thus does not require IRB approval.
27

---

16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The paper does not use LLMs as part of its core methodology.
28


Reviews:
[{"review_id": "Ktp14VIhrm", "reviewer": "Reviewer_4qMa", "summary": "the paper introduces Fast Feature Conformal Prediction (FFCP), an taylor approximation version of prior work Feature Conformal Prediction (FCP).  By using a non-conformity score that uses the gradient of the decoder (part of NN after the feature space embedding used for UQ),  FFCP simplifies the non-linear operations involved in transforming confidence bands from feature space to output space, which is a bottleneck in FCP.  The authors provided theoretical justifications and introduce extensions of FFCP to other CP algorithms.", "questions": "Experiment section \n\n- why is the performance not compared to other feature-based scaled CP methods? You cite Seedat et al 2023 and 2024 - the reason of it needing extra training does not excuse the lack of comparison. One can argue that once amortized, the time consumption of Seedat 2023's method is faster than FCP. If completely without comparison to other methods, where does the reader place this method in the literature? \n\n- \"For FFCP, we select the shortest band length among all layers.\" this is unfair comparison. the authors should choose one layer and stick with it. \n\n- why does Vanilla CP has the exact same numbers as FFCP on the synthetic, star, and bio datasets in table 2? They have different numbers even in table 6. \n\nUnsupported Claims\n\n- even just comparing to FCP, it is unclear from table 2 that FFCP produces \"comparable\" quality uncertainty sets. FFCP's intervals some times 50%-100% wider. \n\n- I'm not sure Figure 2 shows the qualitative result supporting your claim of \"the FFCP producing more refined and sharper uncertainty bands.\", as the presentation makes the difference very subtle. Moreover, what do you mean by : \"FCP operates at the image level, ensuring coverage for the entire predicted mask as a whole, while FFCP adopts a coordinate-wise approach, offering per-pixel statistical guarantees that better reflect local uncertainty.\"? Aren't the proven theoretical guarantees the same?\n\n\n----\nI'm happy to raise my score if my concerns are addressed.", "limitations": "The limitations of the method sparingly discussed, and are not informatively summarized in the discussion/conclusion. Some questions I would appreciate discussions are:\n\n- What are the failure cases of the method? \n- How does it compare to other locally-scaled CP methods in different data setups?\n-  to what extent are the expressivity of FCP limited in FFCP?  (to my understanding FFCP cannot create disjoint prediction intervals for regression tasks but FCP can?)", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n\n- the largest contribution and motivation of the paper is practicality. the authors improve the FCP algorithm such that it bypasses its biggest time bottleneck, LiPRA projection, while retaining most of its strengths. \n\n-  the method is valid and the theory is sound (although heavily building on Teng et al. [2022]).\n\n- the extensions to CQR and RAPS is nice, bringing more usability to the method. (I think there are a lot of potential to section 5.3, but it needs more analysis and organization to integrate to the main paper.)\n\n- overall the paper is clearly written and easy to understand. \n\n----\nWeaknesses\n\n- insufficient contribution. the paper essentially proposes a faster (first degree taylor) approximation to an existing method, which is relatively straightforward. The effectiveness of the algorithm (compared to split CP) highly relies on how accurate the taylor approximation is, where there are no guarantees. Neither the theory or findings brings new insights to CP in my opinion; I'm not sure the contribution is sufficient for a top conference like neurips. \n\n- the experiment results are less than comprehensive and need more analysis. please see questions section for details. \n\n- some unsupported claims in the paper. please also see questions section. \n\n- i think calling the original split conformal prediction algorithm as \"vanilla\" is a bit derogatory and unprofessional/not rigorous. I would just use \"split CP\" in its place; people will know what you are talking about.", "was_revised": true, "final_justification": "the main concern with limited contribution and  limitations discussion remains, but my concerns with the legitimacy of experiments have been addressed.", "rebuttal": "We appreciate the reviewer's insightful comments. We will address these issues one by one.\n\n**Weaknesses:**\n>insufficient contribution. the paper essentially proposes a faster (first degree taylor) approximation to an...\n\nWe understand the reviewer's concern, but we would like to clarify that the core advantages of FFCP lie in two aspects:\n\n1. As a variant of FCP, FFCP significantly reduces computational cost while effectively leveraging feature space information. As a result, it consistently outperforms Vanilla CP (i.e., split CP) and achieves performance comparable to FCP.\n\n2. The feature-level technique in FFCP is highly extensible and largely orthogonal to existing CP methods. It can be readily integrated into various frameworks, including CQR, RAPS, and LCP—and consistently improves performance, as demonstrated in our experiments.\n\nWhile our method leverages a first-order Taylor expansion, it aims to address the core challenge of efficiently utilizing feature space information. FCP improves predictive performance by incorporating feature representations but suffers from high computational costs due to gradient descent and LiPRA. In contrast, FFCP approximates the feature mapping using gradient information, avoiding costly computations while retaining the benefits of feature-guided conformal prediction. It also provides a closed-form prediction band, improving interpretability.\n\nFrom the perspective of extensibility, we have applied the FFCP technique to CQR, RAPS, and LCP (among others), and observed consistent performance improvements over the original models.\n\n>i think calling the original split conformal prediction algorithm as \"vanilla\" is a bit ...\n\nThanks for the reviewer’s reminder, we will make the necessary corrections in the next revision.\n\n**Questions:**\nExperiment section\n>why is the performance not compared to other feature-based scaled CP methods? You cite Seedat et al...\n\nThanks for your constructive suggestion, we have supplemented the results by including those from the referenced literature.\n#### Table A：Results of SSCP and FFCP\n\n\n| Dataset   | Time (s)         |           | SSCP Length       | FFCP Length       |\n|-----------|------------------|-----------|-------------------|-------------------|\n|           | SSCP             | FFCP      |                   |                   |\n| SYNTHETIC | 7.18±0.99        | 0.15±0.02 | 0.25±0.02         | 0.18±0.01         |\n| COM       | 1.33±0.16        | 0.03±0.01 | 2.52±0.14         | 1.84±0.18         |\n| MEPS19    |10.86±0.76        | 0.14±0.01 | 5.32±0.33         | 3.13±0.30         |\n| STAR      | 1.96±0.02        | 0.67±0.06 | 0.67±0.06         | 0.21±0.01         |\n| BIO       |13.49±3.81        | 0.39±0.04 | 1.53±0.03         | 1.66±0.02         |\n| BIKE      | 5.92±0.03        | 0.09±0.01 | 0.73±0.04         | 0.63±0.03         |\n\nIn terms of runtime, SSCP incurs substantially higher computational overhead, requiring the training of two additional networks, and is approximately 2 to 50 times slower than FFCP. In all cases, FFCP also achieves shorter band length. This may be attributed to SSCP’s stronger reliance on the base model’s predictive quality and the effectiveness of its auxiliary network training.\n\n\n\n>For FFCP, we select the shortest band length among all layers.\"...\n\nThanks for the reviewer’s question. This may be due to our lack of clarity in the paper. FFCP method records the total computation time for each layer, and then selects the layer that achieves the best results.\n\n>why does Vanilla CP has the exact same numbers as FFCP on ...\n\nWe sincerely apologize for the confusion caused by our unclear presentation, and we would like to take this opportunity to clarify the issue.First, regarding FFCP, the prediction band takes the form in Eq (8), where $Q_{1-\\alpha}=\\{\\|f(X)-Y\\|/\\|\\nabla g(v)\\|\\}$ When we evaluate the final (top) layer, the prediction function satisfies $g(\\cdot)=f(\\cdot)$, and hence $\\nabla g(v)=\\frac{\\partial f(X)}{\\partial g(v)}=\\frac{\\partial f(X)}{\\partial f(X)}=\\mathbf{1}$. As a result, in this case, FFCP degenerates to the same form as Vanilla CP, meaning their performance is equivalent. This explains why the results of FFCP and Vanilla CP are identical on certain datasets in Table 2.\n\nTable 6 presents a different setting where the base model is under-trained. In this case, feature representations are less reliable, and FFCP tends to perform similarly—or even worse—than Vanilla CP. Due to our oversight, only results from Layer 3 were reported. We have updated the results in Table B by selecting the best-performing layer, which confirms that FFCP underperforms when the model is not sufficiently trained.\n\n#### Table B: Untrained model Length on FFCP \\(full version\\)\n|Dataset|Layer1|Layer2|Layer3|Layer4|Layer5 \\(Vanilla\\)|\n|---|---|---|---|---|---|\n|SYNTHETIC|2.87±0.03|2.81±0.03|2.89±0.01|2.41±0.01|**2.34±0.01**|\n|COM|5.41±0.14|5.30±0.18|5.03±0.09|**4.73±0.08**|4.86±0.13|\n|FB1|3.90±0.01|3.77±0.10|3.62±0.08|3.5699±0.08|**3.5697±0.09**|\n|FB2|4.02±0.09|3.90±0.8|3.74±0.08|3.6625±0.06|**3.6624±0.11**|\n|MEPS19|4.38±0.06|4.41±0.07|4.42±0.07|4.38±0.07|**4.33±0.07**|\n|MEPS20|4.43±0.21|4.44±0.23|4.48±0.25|4.46±0.25|**4.41±0.23**|\n|MEPS21|**4.4081±0.18**|4.46±0.18|4.48±0.18|4.4143±0.15|4.4133±0.17|\n|STAR|2.49±0.06|2.40±0.05|2.15±0.03|1.94±0.01|**1.88±0.01**|\n|BIO|4.27±0.02|4.14±0.02|4.07±0.02|**4.04±0.02**|4.08±0.02|\n|BLOG|2.56±0.15|2.54±0.15|2.57±0.13|2.55±0.14|**2.53±0.12**|\n|BIKE|4.69±0.09|4.67±0.07|4.57±0.09|4.60±0.10|**4.56±0.09**|\n\nUnsupported Claims\n>even just comparing to FCP, it is unclear from table 2 ...\n\nAs the reviewer correctly pointed out, FFCP does exhibit slightly inferior performance compared to FCP in a few cases. However, its primary advantage lies in the substantial reduction of computational cost, making it a practical trade-off between runtime efficiency and predictive performance.\nWithin our framework, we define \"comparability\" as the ability of FFCP to either outperform or closely match the performance of FCP across a wide range of datasets. As shown in Table 2, FFCP achieves superior performance on more than half of the evaluated benchmarks.\n\nFurthermore, unlike FCP, which estimates band length via an implicit and approximate procedure that may lead to instability, FFCP provides a closed-form expression for the prediction band. This enables exact and efficient computation, enhancing both interpretability and robustness.\nFor these reasons, we argue that directly comparing the band lengths of FCP and FFCP on a single dataset may be misleading. A more appropriate evaluation should consider the relative performance across multiple datasets to assess the overall effectiveness and efficiency of FFCP.\n\n\n\n>I'm not sure Figure 2 shows the qualitative result...\n                        \nApologies for the confusion. In Figure 2, our intention was to illustrate that FFCP produces visually sharper edges in segmentation tasks, rather than claiming “sharper uncertainty bands.” We will revise this statement in the next version of the paper to avoid ambiguity.\n\nRegarding the quantile computation, FCP adopts an image-level approach, where each image is treated as a single data point during calibration，and the $\\infty$-norm $||Y-\\hat{Y}||\\_\\infty=\\max\\_{i \\leq d}\\|y\\_i-\\hat{y}\\_i\\|$ is used to define the non-conformity score. In contrast, FFCP computes quantiles in a pixel-level manner, as highlighted in Remark 1, treating each pixel as an individual data instance.\n\nWe believe that uncertainty estimation in image segmentation is more appropriately measured at the pixel level. Therefore, we updated the quantile computation accordingly. This change led to clearer visual boundaries in the prediction masks. (For completeness, we also experimented with applying FFCP under the image-level quantile computation, but the results were suboptimal.)\n\n**Limitations:**\n>What are the failure cases of the method?\n\nWe sincerely thank the reviewer for their comprehensive evaluation of our paper. In the case of FFCP specifically, since the calibration step involves division by the gradient norm, issues may arise when the gradient is very small. We recognize these limitations and view them as important directions for future improvement of our work.\n\n>How does it compare to other locally-scaled CP methods in different data setups?\n\nWe expand on LCP-related models in Appendix B.6, showing that the feature-level technique of FFCP can be extended to other CP frameworks. For instance, we introduce FFLCP as an extension of LCP for group coverage, and Table 12 demonstrates that FFLCP consistently outperforms LCP across datasets.\nOverall, the FFCP feature-based approach is largely orthogonal to existing CP methods, making it a broadly applicable enhancement for improving performance via feature information.\n\n>to what extent are the expressivity of FCP limited in FFCP? ...\n\nWe believe that both FCP and FFCP produce *connected* prediction regions in regression tasks. This stems from two key observations:\n(1) the feature space region is typically assumed to be bounded.\n(2) the prediction head $g$ is a continuous function.\n\nUnder these assumptions, the pre-image set $\\{ v : g(v) \\in \\mathcal{C} \\}$ for a connected set $\\mathcal{C} \\subset \\mathbb{R}^d$ (such as a prediction interval) remains connected.\nWhile FCP may allow more flexible or complex *shapes* within the connected region due to the implicit nature of its band estimation (e.g., via LiPRA), FFCP yields prediction intervals with explicit structure resulting from its first-order Taylor approximation. This structural constraint limits the expressivity of FFCP compared to FCP, but it also brings substantial computational benefits and interpretability. Thus, neither FCP nor FFCP produces disjoint prediction intervals in standard regression settings."}, {"review_id": "FrxXH3tSEj", "reviewer": "Reviewer_JJoX", "summary": "The paper proposes a variation of Feature Conformal Prediction (FCP), which they call Fast Feature Conformal Prediction (FFCP). The core idea is to approximate the prediction head with its linearization, obtained as a first order Taylor expansion. The key benefit of this approximation is efficiency and speed improvement, with the linear model the non-conformity score can be computed analytically for a fraction of the cost of standard FCP. The drawback of this approximation is a potential decrease in performance, but the authors claim that such performance decrease is neglectable and support the claim through extensive empirical evaluation.", "questions": "-Can the author discuss how is it possible that the faster linarized model with FFCP in some settings performs better that the original not-linearized model with FCP? I find this effect very counterintuitive. Do the authors have some explanation of this effect?\n\n-Why didn't the authors include the code for reproducibility in the supplementary material?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 1, "strengths_and_weaknesses": "The paper is well written an easy to read. The concept are well introduced and the proposed method is well explained. However, I think that the first order Taylor approximation of the prediction head should be made more explicit, so far it is only clarified in line 162-163, but this is a very fundamental step of your proposed method. For example you can modify line 8 of the abstract from \"leveraging a Taylor expansion\" to \"leveraging a first-order Taylor expansion\". Despite this and some minor comments I list below, I overall like the paper. \n\nThe idea is not very complex or novel (it somehow resemble the step from Laplace Approximation to Linearized Laplace approximation). Nonetheless the advantages are clear and well supported empirically: a faster method that does not decrease performance. For this reason I'm inclined for acceptance.\n\nMy only main concern is about the performances of FFCP compared to FCP. It is cool that the performance do not degrade much, which is a good motivation for the faster method. However, I find it quite surprising that, in some settings, the performance do actually increase for the faster linearized method. I would expect the performance to at most stay the same when the first order Taylor approximation is applied.\n\nSome minor comments that I'd like to be addressed:\n\n-In line 505 you write \"where H is any function\". This sound too strong and honestly feels wrong. Are you sure that you didn't miss some regularity conditions?\n\n-in Line 130, does it mean that the set of indexes $I=[n]$? Or am I missing something here?\n\n-in Line 204 and 205 there is a clear repetition that could be avoided.\n\n-in Line 495 there is a typo: \"which adjusted\" should be \"which is adjusted\"\n\n-in Line 505 there is a typo: \"then the features\" should be \"then if the features\", right?", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful for the reviewer's valuable feedback and recognition of our work. We will provide additional explanations below.\n\n\n**Weaknesses:**\n>For example you can modify line 8 of the abstract from \"leveraging a Taylor expansion\" to \"leveraging a first-order Taylor expansion\".\n\nWe apologize for the confusion caused and truly appreciate the reviewer’s clarification. As you correctly pointed out, our method is based on a first-order Taylor expansion, and we agree that the description should be more precise and rigorous. We will revise the corresponding statements in the next version to avoid any misunderstanding.\n\n>My only main concern is about the performances of FFCP compared to FCP. It is cool that the performance do not degrade much, which is a good motivation for the faster method. However, I find it quite surprising that, in some settings, the performance do actually increase for the faster linearized method. I would expect the performance to at most stay the same when the first order Taylor approximation is applied.\n\n>Can the author discuss how is it possible that the faster linarized model with FFCP in some settings performs better that the original not-linearized model with FCP? I find this effect very counterintuitive. Do the authors have some explanation of this effect?\n\nWe believe this phenomenon arises from the fact that the band length producted by FCP is not an exact value. Since FCP computes the band in the feature space and then maps it back to the output space, the resulting length is inherently an approximation rather than a directly computed quantity. Specifically, FCP relies on gradient descent during the calibration phase to identify optimal feature representations and employs LiRPA techniques during the test phase. Both procedures involve approximations and can introduce substantial variability. As shown in our experimental results, FCP may even underperform Vanilla CP in certain cases. The performance of FCP is highly sensitive to the number of gradient descent iterations and the choice of hyperparameters in LiRPA, both of which require careful tuning.\n\nIn contrast, FFCP circumvents the costly bidirectional mapping steps and features an explicit computational structure in the output space, resulting in greater stability. This often allows FFCP to outperform FCP in terms of predictive performance under practical settings. However, we acknowledge that if optimal parameters for FCP’s optimization steps can be identified, it typically achieves the best coverage-performance trade-off across most scenarios. In this sense, FCP remains superior on average in terms of predictive guarantees, **while FFCP's primary advantage lies in its substantial reduction in computational cost.**\n\n\n\n>In line 505 you write \"where H is any function\". This sound too strong and honestly feels wrong. Are you sure that you didn't miss some regularity conditions?\n\nWe apologize for the confusion caused by our oversight. To clarify, $H$ refers to $\\mathcal{H}(v, X)$, which denotes the length of the prediction interval in the output space for a sample $X$, given a length $v$ in the feature space. Specifically, for FFCP, this is defined as $\\mathcal{H}(v, X) = Q_{1-\\alpha} \\cdot \\| \\frac{\\partial f(X)}{\\partial h(X)} \\|$. Thus, $\\mathcal{H}$ is not an arbitrary function; rather, it has a well-defined and principled form. We will revise the description in the next version to improve clarity and correctness.\n\n>in Line 130, does it mean that the set of indexes $[n]$? Or am I missing something here?\n\nWe apologize for the confusion，this should indeed be $\\mathcal{I}$, and the use of $n$ is a typo.\n\n>in Line 204 and 205 there is a clear repetition that could be avoided\n>in Line 495 there is a typo: \"which adjusted\" should be \"which is adjusted\"\n>in Line 505 there is a typo: \"then the features\" should be \"then if the features\", right?\n\nWe sincerely thank the reviewer for their careful and thorough review, and for dedicating substantial time and effort to our paper. The issues mentioned above were caused by our oversight, and we will promptly correct these errors.\n\n**Questions:**\n>Why didn't the authors include the code for reproducibility in the supplementary material?\n\nWe regret any confusion this may have caused. We have organized the codebase on GitHub and will make it publicly available in the next version."}, {"review_id": "XREawPs8lz", "reviewer": "Reviewer_UCWd", "summary": "This paper proposes to speed up the computation of Feature CP (FCP) via Taylor approximation. It then extends the proof of FCP to this new method, and carries out experiments comparing CP, FCP, and the proposal (Fast FCP).", "questions": "Presentation:\n- the paper is hardly accessible to those who are not familiar with FCP.\n- the denomination of \"vanilla CP\" is extremely confusing. The original CP is called transductive or (\"full\") CP, and if any CP method should be referred to as vanilla it should be that one. The authors compare against split (or \"inductive\") CP, and that's what the method should be called.\n- Algorithm 1 is a special case of split CP, it is not exactly split CP (despite the description). In general, split CP is defined for an arbitrary nonconformity measure, which takes as input a bag of examples + a new example, and outputs a real number.\n- According to the original definition of split CP, it would strongly seem that FCP is a special case of split CP, in that it uses a specific form of nonconformity measure. Can you clarify?\n- The derivation between eq 5 and 6 should have been spelled out.\n\n\nIn classification problems, isn't eq (5) going to be \"0\" a lot of the times? Indeed, if $g(\\hat{v}) = Y$, as one would hope most of the times for a decent classifier (features+head), then $\\hat{v} \\in \\{v:g(v) = Y\\}$ => eq 5 = 0. This means that \"most of the times\" the whole nonconformity measure computation can be skipped, leading to massive computational gains.\n\nThe empirical results are not as clearcut as the authors describe them. There are cases where FCP is better, cases where FFCP is better, and also cases were split CP is better.\nIn general, split CP is often one order of magnitude faster than FFCP, while its performance (length) isn't too far from it. The authors should consider rephrasing the final message to convey this.\n\nFinally, the comparison should include Full CP, which should in principle achieve the best information coverage (split CP is on the other hand: it has the worst one). There are methods to speed up its computation, which should make it feasible to run this comparison [a.,b,c]\n\n[a] Regression conformal prediction with nearest neighbours (Papadopoulos et al.)\n[b] Exact Optimization of Conformal Predictors via Incremental and Decremental Learning (Cherubin et al.)\n[c] Approximating Full Conformal Prediction at Scale via Influence Functions (Abad Martinez et al.)", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- improves computational speed of FCP\n- in some occasions, it is better than split CP.\n\nCons:\n- presentation\n- the results (method and theory) are relatively incremental", "was_revised": true, "final_justification": "Thank you for taking the time to write your detail rebuttal. I am satisfied with the promised changes, which I encourage the authors to include in their final version.", "rebuttal": "Thanks to the reviewer for the thoughtful comments. We will provide a detailed explanation.\n\n**Questions:**\n>1.the paper is hardly accessible to those who are not familiar with FCP.\n\nWe thank the reviewer for the correction and clarification of the concepts presented in our paper. Since FFCP is a variant of FCP, the current presentation may pose difficulties for readers unfamiliar with FCP. We will revise the relevant descriptions in future versions to improve clarity and minimize potential confusion.\n\n>2.the denomination of \"vanilla CP\" is extremely confusing. The original CP is called transductive or (\"full\") CP, and if any CP method should be referred to as vanilla it should be that one. The authors compare against split (or \"inductive\") CP, and that's what the method should be called.\n\nWe sincerely apologize for the confusion caused in the reading. In future revisions, we will replace \"Vanilla CP\" with the more accurate term \"split CP.\"\n\n>3.Algorithm 1 is a special case of split CP, it is not exactly split CP (despite the description). In general, split CP is defined for an arbitrary nonconformity measure, which takes as input a bag of examples + a new example, and outputs a real number.\n\nWe agree that Algorithm 1 essentially corresponds to the canonical form of split conformal prediction.\n\n>According to the original definition of split CP, it would strongly seem that FCP is a special case of split CP, in that it uses a specific form of nonconformity measure. Can you clarify?\n\nIndeed, from the perspective of Full CP and split CP, FCP can be regarded as a special case of split conformal prediction, as it also partitions the dataset into a training set and a calibration set.\nThe main innovation of FCP lies in how the non-conformity scores are computed during the calibration phase. In classical split CP, the non-conformity score is computed directly as $s = |Y - f(X)|$, and the prediction interval is constructed based on the quantile of these scores.\nIn contrast, FCP similarly trains a model on the training set, but during calibration, it computes the non-conformity scores using Algorithm 2 presented in the paper. Specifically, it searches for an optimal feature representation that leads to a prediction close to the true label, and defines the score as $s_f = |v - h(X)|$. Then, the quantile $Q_{1-\\alpha}$ of these scores is computed, which determines the conformal band radius.\nFCP further employs Band Estimation(incorporate techniques such as LiRPA) to estimate the output band length, and Band Detection (also using Algorithm 2) to evaluate coverage.\nTherefore, in essence, FCP is a variant of split CP, with the key distinction that it leverages information from the feature space in the computation of non-conformity scores.\n\n\n\n>The derivation between eq 5 and 6 should have been spelled out.\n\nThanks for reviewer's question. We provide a detailed approximation procedure below:\n\nIn FCP, the non-conformity score is originally defined as:\n\n$$\n\\inf\\_{v \\in \\{v : g(v) = Y\\}} ||v - h(X)||.\n$$\n\nHowever, this is difficult to compute in practice. Therefore, FCP uses its Algorithm 2 to obtain a good approximation of the feature vector $v$, and then defines the score as:\n\n$$\n\\text{score} = ||v - h(X)||.\n$$\n\nIn Algorithm 2, the optimization objective is:\n\n$$\n\\min_v \\|g(v) - Y\\|^2.\n$$\n\nWe observe that directly optimizing over $v$ can be computationally complex. To simplify this, we consider a first-order Taylor expansion of $g(v)$ around $\\hat{v} = h(X)$:\n\n$$\ng(v) \\approx g(\\hat{v}) + \\nabla g(\\hat{v})(v - \\hat{v}).\n$$\n\nThis implies:\n\n$$\ng(v) - g(\\hat{v}) \\approx \\nabla g(\\hat{v})(v - \\hat{v}),\n$$\n\nand since we are reusing the FCP score form, we obtain:\n\n$$\n||v - \\hat{v}|| \\approx \\frac{|g(v) - g(\\hat{v})|}{||\\nabla g(\\hat{v})||}.\n$$\n\nThis gives an approximate closed-form for the non-conformity score, bypassing the need for gradient-based optimization during inference. Moreover, in a certain sense, FFCP can be viewed as equivalent to FCP when using Algorithm 2 with the number of gradient steps $M = 1$.\n\n\n>In classification problems, isn't eq (5) going to be \"0\" a lot of the times? Indeed, if $g(\\hat{v})=Y$, as one would hope most of the times for a decent classifier (features+head), then $\\{\\hat{v}\\in v:g(v)=Y\\}$ => eq 5 = 0. This means that \"most of the times\" the whole nonconformity measure computation can be skipped, leading to massive computational gains.\n\nWe thank the reviewer for the thoughtful question. Similar to FCP, FFCP is primarily designed for regression tasks, and the formulation presented in the main text reflects its application in the regression setting. **However, we would like to emphasize that the feature-level techniques of FFCP is highly extensible and largely orthogonal to most existing conformal prediction methods**, making it adaptable to a wide range of scenarios beyond regression.\n\n**For example, in classification tasks, we adopt an alternative version of our method, termed FFRAPS, which is better suited for classification.** The detailed implementation is provided in Section 5.3.1 and Appendix B.7, where Algorithm 5 illustrates how the feature-level techniques of FFCP are incorporated into classification tasks under the RAPS framework. In contrast, the version discussed in the main body of the paper is tailored for regression problems, where the goal is to estimate continuous-valued prediction bands.\n\n\n>The empirical results are not as clearcut as the authors describe them. There are cases where FCP is better, cases where FFCP is better, and also cases were split CP is better. In general, split CP is often one order of magnitude faster than FFCP, while its performance (length) isn't too far from it. The authors should consider rephrasing the final message to convey this.\n\nWe understand the reviewer's concern, and we would like to offer the following clarifications: \n\nFirst, **the band length computed by FCP is inherently an approximation**, as it is derived by mapping a region from the feature space back to the output space using methods such as LiRPA. This process introduces potential instability in practice, which may lead to degraded performance compared to Vanilla CP. For instance, Table 2 shows that FCP underperforms on datasets like FB1 and BIKE due to such variability. \n\nSecond, **FFCP can be viewed as an efficient approximation of FCP**. While it significantly reduces the computational burden, its performance remains comparable to that of FCP. In particular, as shown in Table 2, FFCP outperforms FCP on more than half of the evaluated datasets. \n\nLastly, **since FFCP relies on the quality of the trained model**, it may exhibit instability in rare cases. In such scenarios, its performance is generally on par with that of Vanilla CP.\n\n\n\n>Finally, the comparison should include Full CP, which should in principle achieve the best information coverage (split CP is on the other hand: it has the worst one). There are methods to speed up its computation, which should make it feasible to run this comparison [a.,b,c]\n\nWe sincerely thank the reviewer for the thoughtful suggestion. Since FFCP is designed as a fast variant of FCP, and FCP itself can be viewed as a specific instance of split CP, our comparisons have primarily focused within the split CP framework.\n\nThe main reason we did not include comparisons with full CP methods is due to their substantial computational overhead. Full CP typically requires retraining or refitting the model multiple times, which becomes impractical for complex tasks such as image segmentation or image classification. Therefore, implementing full CP in such settings poses significant challenges.\n\nThat said, we fully agree that including full CP in standard regression settings would provide a more comprehensive and objective comparison across CP methods. We will include these results in the next version of our paper."}, {"review_id": "s2u64npYow", "reviewer": "Reviewer_stTu", "summary": "The authors propose an approximation scheme to map Conformal Prediction (PC) sets from the feature to the output space. The idea is to reweight the conformity score with the gradient of the feature map and define CP sets in the output space. The approach does not require extra training, produces intervals that are automatically marginally valid, and reduces efficiency gaps across different datasets.", "questions": "- What do you mean by *utilizing feature information in Vanilla CP*? How does this relate to approximating conditionally valid CP sets? \n- What do you mean by *square condition* and *expanding* the differences between individual length quantiles in Theorem 4.2? How is *stability* in the feature and output spaces defined?\n- Why are FCP and Vanilla's efficiencies so unstable in some cases?", "limitations": "Limitations are not explicitly addressed. The paper does not have any potential negative societal impact.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 1, "originality": 1, "strengths_and_weaknesses": "**Strengths**\n- The proposed conformity score does not require training and generates prediction sets that are automatically marginally valid.\n- Compared to the vanilla and FCP approaches, the proposed conformity measure performs more consistently across datasets (see Table 2). \n\n**Weaknesses**\n- The paper is slightly incremental. While the context is new, approximating a nonlinear map with its linearization is a standard technique. The feature space map of [1] is also based on a linear relaxation. \n- The intervals are computed in the output space. The proposed approach is vanilla CP with a task-specific conformity measure. In the original CP formulation, the conformity score is not required to be $|f(X) - Y|$ as assumed in Algorithm 1.\n- The paper's motivations are the computational costs of computing output space prediction intervals and their non-exactness. It would be useful to expand the explanation on page 2 with some details on how the mapping technique of Xu et al. works and how the proposed approach improves it.\n \n[1]\nTeng, Jiaye, et al. \"Predictive inference with feature conformal prediction.\" arXiv preprint arXiv:2210.00173 (2022).", "was_revised": true, "final_justification": "After reading the authors' rebuttal, I raised my score to 4.", "rebuttal": "We would like to thank the reviewer  for the time and effort spent in reviewing our paper and we will address these concerns and provide our solutions in detail:\n\n**Weaknesses:**\n\n>The paper is slightly incremental. While the context is new, approximating a nonlinear map with its linearization is a standard technique. The feature space map of [1] is also based on a linear relaxation.\n\nWe thank the reviewer for the insightful question. Here, we would like to clarify the motivation behind FFCP. We believe the contributions of FFCP lie primarily in two aspects:\n\n1. FFCP, as a variant of FCP, **significantly reduces computational overhead while maintaining performance comparable to FCP**.\n\n2. **FFCP exhibits strong extensibility. Its feature-level technique is largely orthogonal to most existing CP methods**, allowing it to be seamlessly integrated into classical frameworks such as CQR, RAPS, and LCP—as demonstrated in the experiments presented in this paper.\n\nIt is true that linear relaxation is a widely used technique for approximating nonlinear operators. In the context of FCP [1], the incorporation of feature space information occurs primarily through two key steps:\n(a) mapping from the output space to the feature space; and\n(b) mapping back from the feature space to the output space.\n\nWe believe the reviewer is referring to the linear relaxation used in step (b), which is implemented via LiPRA. Our experiments with FFCP suggest that leveraging feature space information can significantly benefit CP-based models. However, in practical deployment, we observed that\n\nStep (a) requires gradient descent to find the optimal feature representation.\n\nStep (b) relies on LiPRA for the backward mapping.\n\nBoth procedures introduce considerable computational overhead. Our goal was to retain the benefits of using feature space information while substantially reducing computational cost, ideally without sacrificing predictive performance.\n\nTo this end, we applied first-order Taylor approximations (which can indeed be viewed as a form of linear relaxation) to approximate both steps (a) and (b), thereby eliminating the need for gradient descent and LiPRA.\n\nWe found that even such a simple approximation leads to strong empirical performance, comparable to that of the original FCP method. This finding reinforces the value of feature-space guidance in CP while highlighting a more efficient way to leverage it.\n\n>The intervals are computed in the output space. The proposed approach is vanilla CP with a task-specific conformity measure. In the original CP formulation, the conformity score is not required to be $|f(X)-Y|$ as assumed in Algorithm 1.\n\n\nAs the reviewer correctly pointed out, FFCP constructs prediction sets in the output space. However, it is important to emphasize that, similar to FCP, **FFCP also leverages feature information.** From a constructional standpoint, FFCP follows the same conceptual pathway as FCP, mapping from the output space to the feature space and then back to the output space. By applying a first-order Taylor approximation during these two processes, FFCP achieves a significant simplification, resulting in an explicit and closed-form expression for the prediction band in the output space. This is a key advantage of FFCP over FCP: whereas FFCP allows for direct computation of the band length, FCP does not yield an explicit formulation, and thus its band length must be estimated through approximate procedures.\n\nWe acknowledge that there are various possible formulations for computing non-conformity scores in Algorithm 1. We will revise the corresponding description in the next version to reflect this more accurately\n\n\n>The paper's motivations are the computational costs of computing output space prediction intervals and their non-exactness. It would be useful to expand the explanation on page 2 with some details on how the mapping technique of Xu et al. works and how the proposed approach improves it.\n\nWe sincerely apologize for the confusion caused during reading. To clarify, we would first like to briefly explain the role of LiPRA (Xu et al.) in the context of our work.\n\nLiPRA is fundamentally a framework for computing neural network output bounds via linear relaxation-based perturbation analysis techniques, such as CROWN and DeepPoly. Within FCP, LiPRA is used to compute the image of a feature space ball in the output space. Specifically, given Algorithm 2 in FCP, we obtain a feature space set of the form  $\\{g(v) : \\|v - \\hat{v}\\| \\leq Q_{1-\\alpha}\\}$, where $g$ is the prediction head, which is typically highly nonlinear. As a result, computing the corresponding band length in the output space lacks a closed-form expression.\n\nHowever, with LiPRA, given a trained neural network $g$, a reference point $\\hat{v} = g(X)$, and a radius $Q_{1-\\alpha}$ determined via FCP's Algorithm 2 , we can construct the set $\\mathcal{B}_v$ = { $v: ||v-\\hat{v}|| \\leq Q$ }\nand LiPRA allows us to approximate the corresponding output set  $g(\\mathcal{B}_v)$ = { $g(v) : v \\in \\mathcal{B}_v$ }.  This provides an approximation to the prediction band in the output space.\n\nIn practice, however, this step introduces substantial computational overhead due to the complexity of the bound propagation and optimization procedures involved. To address this issue, FFCP adopts a first-order Taylor approximation to map the band length from the feature space to the output space. This significantly reduces computation while still preserving the core idea of leveraging feature-space uncertainty in conformal prediction.\n\n\n\n**Questions:**\n>What do you mean by utilizing feature information in Vanilla CP? How does this relate to approximating conditionally valid CP sets?\n\nIn our work, utilizing feature information in Vanilla CP refers to our attempt to enhance Vanilla CP by incorporating feature-level information. Both FCP and FFCP can be viewed as extensions of Vanilla CP that leverage such information to improve performance.\n\nRegarding approximating conditionally valid CP sets, we would like to clarify that the FFCP technique is not specifically designed to address conditional validity. A typical approach for achieving conditional coverage is LCP (Localized Conformal Prediction). However, the feature-level technique introduced in FFCP can also be integrated into LCP to improve its performance. The resulting method, FFLCP, demonstrates improved conditional coverage properties. Detailed implementation and empirical results are provided in Section 5.3.2 and Appendix B.6 (Algorithm 4).\n\n\n\n>What do you mean by square condition and expanding the differences between individual length quantiles in Theorem 4.2? How is stability in the feature and output spaces defined?\n\nDue to our oversight, the statement of Theorem 4.2 was not clearly presented. The square conditions are intended to capture two key properties:\n\n1. **Expansion**: We informally assume that the difference between the quantile and each individual score in the output space is smaller when measured in the feature space. This implies that performing the quantile operation (Step 4 in Algorithm 1 and Step 4 in Algorithm 2) does not compromise the effectiveness of the prediction bands, thereby supporting the efficiency of FFCP.\n\n2. **Quantile Stability**: This condition ensures that the quantile computed on the calibration set generalizes well to the test set, which is essential for maintaining valid coverage.\n\nUnder these two assumptions, we argue that the average band length of FFCP is theoretically smaller than that of Vanilla CP (split CP).\n\nTo empirically validate the Expansion condition, we provide supporting experiments in Appendix B.2. In the case of FFCP, the difference between each sample and its corresponding quantile in the output space is the same as Vanilla CP. Therefore, Figure 4(a) can be interpreted as the distribution of non-conformity score deviations in the output space, while Figure 4(b) shows the same deviations measured in the feature space. The numerical results indicate that the deviations in the feature space are significantly smaller than those in the output space, thereby confirming that the Expansion condition is satisfied.\n\n\n\n>Why are FCP and Vanilla's efficiencies so unstable in some cases?\n\nWe begin by analyzing the key factors that influence the performance of Vanilla CP, FCP, and FFCP:\n\n1. Vanilla CP is the most basic method, and its performance primarily depends on a) the quality of the trained predictive model.\n\n2. FCP is also affected by a)the quality of model training, but in addition, it is influenced by b)the optimization quality of the gradient descent procedure in FCP's Algorithm 2, and c)the quality of the LiPRA-based bound propagation.\n\n3. FFCP (our method) does not rely on additional models or optimization procedures, so its performance depends solely on a)the quality of the predictive model.\n\nThe band length produced by FCP is an approximation obtained via a two-step mapping from the output space to the feature space and back. This approximation introduces instability, which may result in degraded performance compared to Vanilla CP. For example, in Table 2, FCP performs worse than Vanilla CP on datasets like FB1 and BIKE, likely due to such instability.\n\nFFCP serves as an efficient approximation of FCP, achieving a substantial reduction in computational cost. Despite this simplification, FFCP achieves comparable performance to FCP. Notably, Table 2 shows that FFCP outperforms FCP in over half of the benchmark datasets.\n\nAs FFCP relies on the output of a pre-trained model, it can also exhibit instability under certain conditions. In such rare cases, its performance tends to align with that of Vanilla CP."}]

Author rebuttal:

Meta-review: This paper presents a well-motivated and sound method to significantly accelerate feature conformal prediction using a Taylor approximation, a contribution reviewers found to be practical and valuable. The paper is clearly written, and the proposed approach achieves a notable computational speedup while maintaining comparable performance to the original, more computationally intensive method.
While reviewers initially raised valid concerns about the experimental comparisons and reproducibility, the authors provided a thorough rebuttal with additional results and clarifications that addressed most of these issues. The constructive dialogue significantly strengthened the submission, with the authors clarifying key technical points and committing to release their code, bolstering confidence in the work's contributions to the field.
Decision: Accept (poster)