Title: Accelerating Feature Conformal Prediction via Taylor Approximation
Venue: NeurIPS 2025
Accepted: yes
Keywords: conformal prediction

Taylor

fast

feature

uncertainty

deep learning

representation
Abstract: Conformal prediction is widely adopted in uncertainty quantification, due to its post-hoc, distribution-free, and model-agnostic properties.
In the realm of modern deep learning, researchers have proposed Feature Conformal Prediction (FCP), which deploys conformal prediction in a feature space, yielding reduced band lengths.
However, the practical utility of FCP is limited due to the time-consuming non-linear operations required to transform confidence bands from feature space to output space.
In this paper, we present Fast Feature Conformal Prediction (FFCP), a method that accelerates FCP by leveraging a first-order Taylor expansion to approximate these non-linear operations.
The proposed FFCP introduces a novel non-conformity score that is both effective and efficient for real-world applications.
Empirical validations showcase that FFCP performs comparably with FCP (both outperforming the vanilla version) while achieving a significant reduction in computational time by approximately 50x in both regression and classification tasks.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose an approximation scheme to map Conformal Prediction (PC) sets from the feature to the output space. The idea is to reweight the conformity score with the gradient of the feature map and define CP sets in the output space. The approach does not require extra training, produces intervals that are automatically marginally valid, and reduces efficiency gaps across different datasets.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n- The proposed conformity score does not require training and generates prediction sets that are automatically marginally valid.\n- Compared to the vanilla and FCP approaches, the proposed conformity measure performs more consistently across datasets (see Table 2). \n\n**Weaknesses**\n- The paper is slightly incremental. While the context is new, approximating a nonlinear map with its linearization is a standard technique. The feature space map of [1] is also based on a linear relaxation. \n- The intervals are computed in the output space. The proposed approach is vanilla CP with a task-specific conformity measure. In the original CP formulation, the conformity score is not required to be $|f(X) - Y|$ as assumed in Algorithm 1.\n- The paper's motivations are the computational costs of computing output space prediction intervals and their non-exactness. It would be useful to expand the explanation on page 2 with some details on how the mapping technique of Xu et al. works and how the proposed approach improves it.\n \n[1]\nTeng, Jiaye, et al. \"Predictive inference with feature conformal prediction.\" arXiv preprint arXiv:2210.00173 (2022).\n\nQUESTIONS:\n- What do you mean by *utilizing feature information in Vanilla CP*? How does this relate to approximating conditionally valid CP sets? \n- What do you mean by *square condition* and *expanding* the differences between individual length quantiles in Theorem 4.2? How is *stability* in the feature and output spaces defined?\n- Why are FCP and Vanilla's efficiencies so unstable in some cases?\n\nLIMITATIONS:\nLimitations are not explicitly addressed. The paper does not have any potential negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 1, "ORIGINALITY": 1, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes to speed up the computation of Feature CP (FCP) via Taylor approximation. It then extends the proof of FCP to this new method, and carries out experiments comparing CP, FCP, and the proposal (Fast FCP).\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- improves computational speed of FCP\n- in some occasions, it is better than split CP.\n\nCons:\n- presentation\n- the results (method and theory) are relatively incremental\n\nQUESTIONS:\nPresentation:\n- the paper is hardly accessible to those who are not familiar with FCP.\n- the denomination of \"vanilla CP\" is extremely confusing. The original CP is called transductive or (\"full\") CP, and if any CP method should be referred to as vanilla it should be that one. The authors compare against split (or \"inductive\") CP, and that's what the method should be called.\n- Algorithm 1 is a special case of split CP, it is not exactly split CP (despite the description). In general, split CP is defined for an arbitrary nonconformity measure, which takes as input a bag of examples + a new example, and outputs a real number.\n- According to the original definition of split CP, it would strongly seem that FCP is a special case of split CP, in that it uses a specific form of nonconformity measure. Can you clarify?\n- The derivation between eq 5 and 6 should have been spelled out.\n\n\nIn classification problems, isn't eq (5) going to be \"0\" a lot of the times? Indeed, if $g(\\hat{v}) = Y$, as one would hope most of the times for a decent classifier (features+head), then $\\hat{v} \\in \\{v:g(v) = Y\\}$ => eq 5 = 0. This means that \"most of the times\" the whole nonconformity measure computation can be skipped, leading to massive computational gains.\n\nThe empirical results are not as clearcut as the authors describe them. There are cases where FCP is better, cases where FFCP is better, and also cases were split CP is better.\nIn general, split CP is often one order of magnitude faster than FFCP, while its performance (length) isn't too far from it. The authors should consider rephrasing the final message to convey this.\n\nFinally, the comparison should include Full CP, which should in principle achieve the best information coverage (split CP is on the other hand: it has the worst one). There are methods to speed up its computation, which should make it feasible to run this comparison [a.,b,c]\n\n[a] Regression conformal prediction with nearest neighbours (Papadopoulos et al.)\n[b] Exact Optimization of Conformal Predictors via Incremental and Decremental Learning (Cherubin et al.)\n[c] Approximating Full Conformal Prediction at Scale via Influence Functions (Abad Martinez et al.)\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a variation of Feature Conformal Prediction (FCP), which they call Fast Feature Conformal Prediction (FFCP). The core idea is to approximate the prediction head with its linearization, obtained as a first order Taylor expansion. The key benefit of this approximation is efficiency and speed improvement, with the linear model the non-conformity score can be computed analytically for a fraction of the cost of standard FCP. The drawback of this approximation is a potential decrease in performance, but the authors claim that such performance decrease is neglectable and support the claim through extensive empirical evaluation.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper is well written an easy to read. The concept are well introduced and the proposed method is well explained. However, I think that the first order Taylor approximation of the prediction head should be made more explicit, so far it is only clarified in line 162-163, but this is a very fundamental step of your proposed method. For example you can modify line 8 of the abstract from \"leveraging a Taylor expansion\" to \"leveraging a first-order Taylor expansion\". Despite this and some minor comments I list below, I overall like the paper. \n\nThe idea is not very complex or novel (it somehow resemble the step from Laplace Approximation to Linearized Laplace approximation). Nonetheless the advantages are clear and well supported empirically: a faster method that does not decrease performance. For this reason I'm inclined for acceptance.\n\nMy only main concern is about the performances of FFCP compared to FCP. It is cool that the performance do not degrade much, which is a good motivation for the faster method. However, I find it quite surprising that, in some settings, the performance do actually increase for the faster linearized method. I would expect the performance to at most stay the same when the first order Taylor approximation is applied.\n\nSome minor comments that I'd like to be addressed:\n\n-In line 505 you write \"where H is any function\". This sound too strong and honestly feels wrong. Are you sure that you didn't miss some regularity conditions?\n\n-in Line 130, does it mean that the set of indexes $I=[n]$? Or am I missing something here?\n\n-in Line 204 and 205 there is a clear repetition that could be avoided.\n\n-in Line 495 there is a typo: \"which adjusted\" should be \"which is adjusted\"\n\n-in Line 505 there is a typo: \"then the features\" should be \"then if the features\", right?\n\nQUESTIONS:\n-Can the author discuss how is it possible that the faster linarized model with FFCP in some settings performs better that the original not-linearized model with FCP? I find this effect very counterintuitive. Do the authors have some explanation of this effect?\n\n-Why didn't the authors include the code for reproducibility in the supplementary material?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 1, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nthe paper introduces Fast Feature Conformal Prediction (FFCP), an taylor approximation version of prior work Feature Conformal Prediction (FCP).  By using a non-conformity score that uses the gradient of the decoder (part of NN after the feature space embedding used for UQ),  FFCP simplifies the non-linear operations involved in transforming confidence bands from feature space to output space, which is a bottleneck in FCP.  The authors provided theoretical justifications and introduce extensions of FFCP to other CP algorithms.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n- the largest contribution and motivation of the paper is practicality. the authors improve the FCP algorithm such that it bypasses its biggest time bottleneck, LiPRA projection, while retaining most of its strengths. \n\n-  the method is valid and the theory is sound (although heavily building on Teng et al. [2022]).\n\n- the extensions to CQR and RAPS is nice, bringing more usability to the method. (I think there are a lot of potential to section 5.3, but it needs more analysis and organization to integrate to the main paper.)\n\n- overall the paper is clearly written and easy to understand. \n\n----\nWeaknesses\n\n- insufficient contribution. the paper essentially proposes a faster (first degree taylor) approximation to an existing method, which is relatively straightforward. The effectiveness of the algorithm (compared to split CP) highly relies on how accurate the taylor approximation is, where there are no guarantees. Neither the theory or findings brings new insights to CP in my opinion; I'm not sure the contribution is sufficient for a top conference like neurips. \n\n- the experiment results are less than comprehensive and need more analysis. please see questions section for details. \n\n- some unsupported claims in the paper. please also see questions section. \n\n- i think calling the original split conformal prediction algorithm as \"vanilla\" is a bit derogatory and unprofessional/not rigorous. I would just use \"split CP\" in its place; people will know what you are talking about.\n\nQUESTIONS:\nExperiment section \n\n- why is the performance not compared to other feature-based scaled CP methods? You cite Seedat et al 2023 and 2024 - the reason of it needing extra training does not excuse the lack of comparison. One can argue that once amortized, the time consumption of Seedat 2023's method is faster than FCP. If completely without comparison to other methods, where does the reader place this method in the literature? \n\n- \"For FFCP, we select the shortest band length among all layers.\" this is unfair comparison. the authors should choose one layer and stick with it. \n\n- why does Vanilla CP has the exact same numbers as FFCP on the synthetic, star, and bio datasets in table 2? They have different numbers even in table 6. \n\nUnsupported Claims\n\n- even just comparing to FCP, it is unclear from table 2 that FFCP produces \"comparable\" quality uncertainty sets. FFCP's intervals some times 50%-100% wider. \n\n- I'm not sure Figure 2 shows the qualitative result supporting your claim of \"the FFCP producing more refined and sharper uncertainty bands.\", as the presentation makes the difference very subtle. Moreover, what do you mean by : \"FCP operates at the image level, ensuring coverage for the entire predicted mask as a whole, while FFCP adopts a coordinate-wise approach, offering per-pixel statistical guarantees that better reflect local uncertainty.\"? Aren't the proven theoretical guarantees the same?\n\n\n----\nI'm happy to raise my score if my concerns are addressed.\n\nLIMITATIONS:\nThe limitations of the method sparingly discussed, and are not informatively summarized in the discussion/conclusion. Some questions I would appreciate discussions are:\n\n- What are the failure cases of the method? \n- How does it compare to other locally-scaled CP methods in different data setups?\n-  to what extent are the expressivity of FCP limited in FFCP?  (to my understanding FFCP cannot create disjoint prediction intervals for regression tasks but FCP can?)", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-06"}