Backward Conformal Prediction
Authors: Etienne Gauthier
Francis Bach
Michael I. Jordan
Venue: NeurIPS 2025 poster ()
Keywords: conformal prediction
e-values
uncertainty quantification
machine learning
statistics
Primary area: theory
TL;DR: We introduce Backward Conformal Prediction, a new method that adapts coverage levels to enforce interpretable, data-dependent prediction set sizes, with provable guarantees and practical estimators.

Abstract: We introduce *Backward Conformal Prediction*, a method that guarantees conformal coverage while providing flexible control over the size of prediction sets. Unlike standard conformal prediction, which fixes the coverage level and allows the conformal set size to vary, our approach defines a rule that constrains how prediction set sizes behave based on the observed data, and adapts the coverage level accordingly. Our method builds on two key foundations: (i) recent results by Gauthier et al. [2025] on post-hoc validity using e-values, which ensure marginal coverage of the form $\mathbb{P}(Y_{\rm test} \in \hat C_n^{\tilde{\alpha}}(X_{\rm test})) \ge 1 - \mathbb{E}[\tilde{\alpha}]$ for any data-dependent miscoverage $\tilde{\alpha}$, and (ii) a novel leave-one-out estimator $\hat{\alpha}^{\rm LOO}$ of the marginal miscoverage $\mathbb{E}[\tilde{\alpha}]$ based on the calibration set, ensuring that the theoretical guarantees remain computable in practice. This approach is particularly useful in applications where large prediction sets are impractical such as medical diagnosis. We provide theoretical results and empirical evidence supporting the validity of our method, demonstrating that it maintains computable coverage guarantees while ensuring interpretable, well-controlled prediction set sizes.

Backward Conformal Prediction
Etienne Gauthier" INRIA-ENS-PSL Paris
Francis Bach INRIA-ENS-PSL. Paris
Michael L. Jordan WNRIA-ENS-PAT Paris UC Berkeley
Abstract
We introduce Backward Conformal Prediction, a method that guarantees confor- mal coverage while providing flexible control over the size of prediction sets. Unlike standard conformal prediction, which fixes the coverage level and allows the conformal size to vary, our approach defines a rule that constrains how prediction set sizes behave based on the observed data, and adapts the coverage level accordingly. Our method builds on two key foundations: recent results by Gauthier et al. 2025 on post-hoc validity using e-values, which ensure marginal
coverage of the form P(Y E Ó. (Xtest) )) 2 1 - E[āi for any data-dependent miscoverage a. and (ii) a novel leave-one-out estimator 2-00 of the marginal mis- coverage E[ā] based on the calibration set, ensuring that the theoretical guarantees remain computable practice. This approach particularly useful applications where large prediction sets are impractical such as medical diagnosis. We provide theoretical results and empirical evidence supporting the validity of our method, demonstrating that it maintains computable coverage guarantees while ensuring interpretable, well-controlled prediction set sizes.
Introduction
Conformal prediction (ook et al. 2005) is a widely used framework for uncertainty quantification in machine learning. It produces sel-valued predictions that guaranteed contain the true label with high probability, regardless of the underlying data distribution.
Given calibration set of n labeled examples {(X,, Y))_ and test point (Xuest Yrest), all assumed to drawn from same unknown distribution over X' x Y, conformal prediction constructs prediction set (Xtest) such that
P(Yrett. E C (Xtest)) 2 2-aa
where the target miscoverage E (0,1) is fixed by the practitioner. The probability is taken both the calibration data and the test point, and the guarantee holds under the assumption that all n points are exchangeable generalization of the independent and identically distributed (i.i.d.) setting.
The method uses score function S x R4 R+ypiciallly derived from pre-trained model to evaluate how well the model's prediction of f() at input r aligns with a candidate label y. In this paper, assume that the scores are negatively oriented, meaning that lower score indicates better fit. The basic idea of conformal prediction is that, under exchangeability. the test score should behave like a typical calibration score: should not stand out unusually large. For input X
"etienne, gattuiergautirrreeeee inria.fr
'Exchangeable random variables are a sequence random variables whose joint distribution invariant under any permutation of their indices.
'In the conformal prediction literature, scores also be negative. In this work, we explicitly assume nonnegativity to simplify the construction of e-variables like B
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

the conformal set includes all labels y such that the test score SXXtest y) is not excessively large compared the scores from the calibration set. Standard methods rely on comparing score ranks. which can be interpreted in terms of p-values. We refer the reader to Angelopoulos and Bates [223]] Angelopoulos et al. 2024] for a recent overview of conformal prediction. Our work focuses on classification with finite set of labels ) a setting that has been the focus of extensive research in
conformal prediction [Lei 2014, Sadinle et al. 2019 Hechtlinger et al. 2019
Romano et al. 2020
Angelopoulos et al. 2021 Cauchois et al. 2021 Podkopaev and Ramdas 202
Guan and Tibshirani
2022 Kumar et al. 2023
A key limitation of conformal prediction is that the size of the conformal set is entirely data-driven and canmot be controlled in advance. In many applications, this lack of control can be problematic: overly large prediction sets may be too ambignous to be usefill especially in high-stakes or resource- constrained settings. This has motivated a growing body of work aiming to reduce the size of conformal sets while maintaining valid coverage guarantees (Stutz et al. [022] Ghosh et al. 2023
Dhillon et al. 2024 Kiyani et al. 2024 Noorani et al., 2024 Yan et al. 2024| Braun et al. 2025].
In this work, we take a different perspective. Rather than fixing desired coverage level 1 - a, we fix size constraint rule T that flexibly determines, based on the observed calibration data {(x; Y.) and the test feature Xuest the maximum allowable size of the prediction set. Formally:
Definition 1.1 (Size constraint rule). A size constraint rule is function
T: (((X,, ( Xeest) - T (((,,, Y E (1,...,121),
where | denotes the size of the label space. Given the observed calibration data {(X,, , and test feature Xtest, the rule T determines the maximum allowable size of the conformal set for X Kust- An example of size constraint rule is the constant size constraint rule, which corresponds to fixing a conformal set size independent from the observed data. In this case, T is a constant function, and by abuse of notation, we will also denote by T E (1....,())) ()]) the constant value it returns.
The miscoverage level â is then chosen adaptively to satisfy this constraint. This gives rise to a new method, Backward Conformal Prediction, which ensures that the conformal set Cä (Xtest) has size at most T ({(X;, Y:) 1- Xtest), and provides marginal coverage guarantees of the form:
P(Yestt E Čh (Xtest) 2 E[ā].
(2)
where the probability is taken over both the calibration set and the test point. The guarantee above was established by Gauthier al. [2025] by constructing conformal sets with e-values, a method
known as conformal e-prediction Vovk 2025
This paper then introduces practical component: novel leave-one-oul estimator 6L00 of the marginal miscoverage E[ā]. computed from the calibration set. This allows practitioners to estimate the coverage guarantee in practice and make informed decisions about whether or not to trust the conformal set. The Backward Conformal Prediction procedure is illustrated in Figure] [
Test point (Nuss Yus) Calibration set (x).
Marginal guarantee
F(Yow (Xwe)) 2 Bla
one estirator
Bachward OP
Fix size comstraint rule
Construct (f with adeptive stt. (05(Xx S (f(X,, Y.3 X
miscoverage 2000 ss 3[4]
Accept procedure or adjust 7 based on estimaled coverage
T
Figure 1: Overview of Backward Conformal Prediction. The procedure first fixes a (potentially
data-dependent) size constraint rule T, then constructs a conformal set Ō² (Xtest) using an adaptive miscoverage level chosen respect the size constraint. A leave-one-out estimator L000 is computed on the calibration set to estimate the marginal miscoverage E E[a], enabling practitioners to decide whether to proceed with the current configuration or retune the size constraint T the guarantee is unsatisfactory.

---

In traditional conformal prediction, the miscoverage level a is fixed in advance. Then, conformal set satisfying marginal coverage guarantees (I) constructed based on the calibration set. However, the size of the resulting conformal set is not controlled. In contrast, Backward Conformal Prediction reverses this workflow. It begins by defining rule that dictates how the sizes of the conformal sets should behave, depending on the calibration data and the test feature. In this setting, the miscoverage level ã becomes an adaptive quantity, dynamically adjusted based on the observed data, and is no longer fixed priori. This adaptivity allows for controlling the size of the conformal set, though at the cost indirect control over coverage. Nevertheless, the procedure still yields marginal coverage guarantees (2) This inversion of the standard conformal prediction design, where size control takes precedence over fixed coverage, motivates the name Backward Conformal Prediction.
Our approach is particularly useful in applications where prediction sets must be small and in- terpretable, such as medical diagnosis or inventory demand forecasting. We briefty present two motivating examples from these domains, where the control over prediction set sizes provides natural and principled basis for decision-making.
Healthcare. In medical diagnosis, doctors must often infer a patient's condition from their profile X (symptoms, history, etc.) under time constraints. Standard conformal prediction, applied to calibration data {X,,, )=))==> i= may produce prediction sets that are too large to have practical impact on the diagnosis. Backward Conformal Prediction addresses this by allowing a size constraint which can be fixed or adaptive and data-dependent; for example, expanding the prediction set in rare or unusual cases while keeping it small and actionable for common cases. Our method ensures marginal coverage guarantees 2 and enables doctors to validate external reliability, via the guarantee that the coverage 1 :000 ~ - [[a] remains above a desired level (e.g., 99%), striking balance between diagnostic efficiency and rigor.
commerce, demand forecasting models predict daily sales Y from features X such as day of the week, weather, seasonality, and promotions. To forecast demand for a new day Xust, a seller may wish to define a size constraint rule T that adapts to past demand variability in similar conditions: larger prediction sets for volatile periods (e.g.. holidays) and smaller ones for stable days. Backward Conformal Prediction ensures valid coverage, and the seller can verify if estimated coverage 1 L00 meets reliability goals (e.g., 95%). This balances interpretability and reliability, supporting better stocking decisions under uncertainty.
Inventory demand forecasting.
2 Method
Backward Conformal Prediction builds on two core theoretical components. The first is the post hoc validity of the inverses of e-values, which provides marginal coverage guarantees while ensuring that the conformal sets have controlled size. The second is the estimation of these guarantees. While the first component has been established in conformal prediction by Gauthier et al. [2025), this paper establishes the second guarantee, which is essential for the method to be used in practice and not merely serve as a theoretical benchmark. Before explicating these two key steps, we first review the foundational principles of conformal prediction with e-values, or conformal e-prediction, which serves as the basis for Backward Conformal Prediction.
2.1 Conformal e-prediction
Most conformal prediction methods rely on the comparison of score ranks, which can be interpreted using p-values. However, it is also possible to construct conformal sets using e-values, a method known as conformal e- predicction [Vovk] 2025). E-values are the values taken by e-variables:
Definition 2.1 (E-variable). An e-variable is a nonnegative random variable that satisfies
E S 1.
Backward Conformal Prediction works with any e-variable, For concreteness, we adopt the following
e-variable, first introduced by Wang and Ramdas (2022) and Koning (025b) and later used in the
context of conformal prediction by Balinsky and Balinsky (2024) which optimal some sense
[Koning 2025b Larsson al. 2025
3

---

S(Xtests Yees)
E
2i=1 SX,, Yi)
S 3(Xust; Viest)
It is straightforward verify that EE [test]] 1 under exchangeability. The intuition, much like in standard conformal prediction methods, is that the test score S S(X Yest) should not be excessively large compared the calibration scores. However, rather than comparing ranks, as is typically done in conformal prediction, we directly compare the test score to the average of all the scores.
E-aarialles, when coupled with probabilistic inequalities, allow for the construction of conformal sets with valid a coverage. For instance, by applying Markov's inequality, we get:
P(E <//o) = P(Flest 2 1/a) 2 aFE(Feestt] 2 a.
By carefully designing an e-variable based on the calibration set and the test point, we can construct useful conformal sets.
2.2 Post-hoc validity
E-values offer advantages that p-values alone cannot, including post-hoc validity, which enables more flexible and adaptive inference. The connections between e-variables and post-hoc validity have been explored, either explicitly or implicitly, in the following works: Wang and Ramdas] (2022). Xu et al.)
(2024), Grünwald 2024), |Randas and Wang (2024), , Koning (202aa], and leveraged in conformal prediction by Gauthier et al. [2025]. J. For completeness, we briefly review the main application of post-hoc validity with v-rrialless in the context of conformal prediction.
Proposition 2.2. Consider a calibration set {(X;, Y:) and test data point (Xtest, Yust) such that (X1, Yi,YY,,,... (Xn; Yn); (Xrest, Yust) exchangeable. Let â 0 be any miscoverage level that may depend on this data. Then, for the conformal set
S(x,v) n+1 (E 5(X,, + S(x,1))
<1/ā
y
the coverage inequality
holds, i.e.,
P(Yesst Ca (Xues)
E(a).
Note that when ā is constant, we recover the standard conformal guarantee 11
The flcxibility of e-variables allows for the construction of conformal sets that adapt to the structure of the data, including explicit control over their size. In classification task where wish to control the conformal set sizes, standard conformal prediction techniques do not provide a direct way to enforce this. In contrast, using e-variables allows to define data-dependent miscoverage level a, given a size constraint rule T, as follows:
à = inf {a E 10, (y: Blest <//a] ST([(X, X1ost))
(4)
where, for the e-variable defined in Equation
1(xt 8) (E S(X,, Y2) + S(Xeett v)) yEy
E
Flest
the ratio vector for the test score, where each element corresponds to the ratio of the test score S(Xteat, 3) to the average of the calibration scores and the test score. This formulation allows us to obtain coverage guarantees of the form 2 while explicitly controlling the size of the conformal set.
With additional assumptions on ā, such as sub- Gaussianity, is possible obtain coverage guarantees dependent on the observed miscoverage â and not on the marginal miscoverage E[ā] (see Boucheron Øt al. (2013) for standard reference on concentration inequalities for sub-Gaussian variables). this paper, we propose an approach that does not require any additional assumptions and relies solely on the estimation of the marginal miscoverage term IF [[a]. We suggest estimating using the calibration data, which are fully accessible to the practitioner.

---

2.3 Leave-one-out estimator
In this section, we introduce the leave-one-out estimator Lo00 for the marginal miscoverage term E[ā]. Because it depends only on calibration scores and not on the test label, this estima- tor provides fully computable, real-world proxy for miscoverage, enabling practitioners to obtain empirical guarantees on coverage. The key intuition is that, for large n, the denominator of the value Ftest in 33 closely approximates the expected score [[S(X; Y)]. so Etest effectively measures how far the test score deviates from the true average of the scores. To approximate the expectation of the marginal miscoverage, we emulate pseudo-ratios E based on the calibration set, where we compare ((),,y)
v) to the average of the calibration scores for all E
5(X,,3) Li)1i S Yi) + s(),,
E
for all j ...., n.
yEy
We compute the corresponding pseudo-misscoeerage::
ā = inf
imf {a E (0,1): # {v E, 1/a) (T((X, X,))
and define:
1 n1
n
à L00
aj.
(5)
j=1
This corresponds to averaging the n pseudo-miscoverage terms obtained by artificially designating each calibration score S(X,, Yj) as a pseudo-test score, with the remaining scores (5(X,, Y=) playing the role of the csaudo-calibration set. The construction of & T.00 is schematically detailed in Figure
Marginal Miscoverage E
Leave-one-out Estimator à 100
(Z ,.....
Zn}
X Kvee
[2 Z.3 (2t)
X
{2 2,) (2x)
X,
Avg
E tost
E
E
ā
miscoverage
marginel
E.a
leave one-out estimator
LOO
The left panel illustrates the definition of the true marginal miscoverage E[ā]], which depends on the ratio between the test score S(X, S(Xuest:.. and the average right panel depicts the leave-one-out estimator L000, each calibration point yields pseudo- miscoverage à, by comparing 5(X,,.) to the average calibration score. Averaging these gives à LO0 which approximates E[ā] without using the test score. Feature-label pairs are denoted Z; (X,, Y.).
Figure 2:
n+1 scores. The
We summarize the Backward Conformal Prediction procedure in Algorithm

---

Algorithm 1: Backward Conformal Prediction
Input: Calibration set {(X,, Yi) test feature Xtest size constraint rule 7, score function
Output: Conformal set C. (Xtest) of size S T(((X,, Y;) Xtest) and an approximate
marginal coverage
for i to do
L Compute calibration score SXX,, Y;);
Select å adaptively using Eq. (4) to build conformal set C of size & T([(X, Y.) Xtest),
satisfying guarantee 2
Compute approximate miscoverage â 1.0 using Eq. 5
return C (X approximate marginal coverage L..00
3 Theoretical analysis
In this section, we present theoretical properties satisfied by the estimator L.0 of the marginal miscoverage E[ā].
Let μ E[(((XYY) Y)] denote the expected value of 5(X,Y) For each calibration point E (1, (1..... n we define the normalized score vector:
S((),
E'
DEY
which consists of the normalized score values of S(X,, y) for each y y. Similarly, for the test point, we define the vector:
( (5((X (S( y)
E Frost
1
WEY
which represents the normalized score values S(X y) for each y E Y.
We show that, under certain assumptions the size constraint rule T, the estimator L00 concen- trates around its target E(ā). with an estimation error of order Or vT
as the calibration size
increases, using the Op notation from van der Vaart [1988.
3.1 Constant size constraint rule
To build intuition, we begin with the simple case where the size constraint rule T is a constant, which we also denote by for convenience. This simplified case provides foundation for understanding the estimator's 's properties under more general conditions.
Theorem 3.1. Assume that the score function S is bounded and takes values in the interval [Smin, Smaxl. with 0 Suuin I Suaxx- Suppose that the number of calibration samples satis- fies n Swax/Sunin In addition, assume that the vectors Elest E. Erest and E satisfy the following properties almost surely:
(P1) 1 1 # y E, < <1ST < |/
(P2) For all y V. Ey 1.
Then, if the samples (Xi, Y.) are i.i.d., the leave-one-out estimator satisfies:
|a100 - Eja] = O0p
Note that, under the boundedness assumption on S. the vectors Elest and are well defined. This assumption also implies μ # 0, which guarantees that Etest and E are well defined.
In Property P1) the inequality < # (v E E, < 1) states that at least one label in the conformal prediction framework has relatively low score. This ensures that the model does not assign high scores to all labels, which would make it difficult to differentiate between them. The assumption is

---

not overly restrictive, as it only requires one label to have low score, leaving flexibility for other serve to
labels to have higher scores. The two other inequalities # {y E y E, < < rule out edge cases that would otherwise lead to degenerate behavior. Property (P2) plays a similar role. This assumption mild, as score functions are typically continuous and attain any fixed value with probability zero. Likewise, the boundedness assumption on S is generally unrestrictive; it helps ensure the stability of the analysis by excluding pathological situations.
< [3]
The proof proceeds in several steps. First, we show that the infimum appearing in the definition of the miscoverage is stable under small perturbations of the ratio vectors E. This is done by approximating the set cardinalities using a smooth surrogate based on bump functions, and applying the Implicit Function Theorem. Next, we show that the leave-one-out estimator & is close the miscoverage E[a]. relying on the intuition that when is large, the average of the calibration scores approximately equal to their expected value. We formalize this using Hoeffding's 's inequality to establish concentration (see Hoeffding [1963). The full proof is deferred to Appendix C
Remark 3.2. The explicit bound established in the proof of Theorem 3.1 shows that for any 5 > 0, we have
L2000- E[āa]
Smax
log(4/8) 2n
2 n
log (og(4/))
2
max n +
P (Smin/ Smax
2n
pSSmnn 12
2(n + 1)"
with probability at least 5. While the constants Smin and Smax are typically known to the practitioner, μ is generally not observed. However, we may still upper bound the expression by
log(4/6)
S Smax
log(4/6) 2n
Smax Smin
Pn+1 1
2n
+
+
=: Ro(n),
Smin
Smin/ Smax
n
V 2(n
which guarantees that, long P(Yiest E Ca (Xtest) )) 2 E[ā]] we obtain
P(Yeest E C (Xtest)) 2 T.00 Rs(n),
with probability at least - 8. This provides practical decision-making tool: given a target threshold t, the practitioner can trust the conformal set with probability least - 8 the inequality 1 - L0o (n) 2 r holds, and reject it otherwise.
In addition, the variance of L000 decreases rate O (H), further supporting its concentration around the marginal miscoverage Elā].
Theorem 3.3. Under the assumptions of Theorem 3.1 we have:
Var (a-00) = (a (
The proof of Theorem 3.3 can be found in Appendix C well. The intuition is that the leave-one-out estimator dL00 is sum of terms, each of which is very close independent terms, for which know the variance of the sum. The remaining task is to bound the variance of the deviation, which we do by using a tail bound. This allows us to upper bound the variance of deviation through integration.
Remark 3.4. Note that when T is constant and the samples are i.i.d., coverage guarantees can be estimated in a straightforward way. For each calibration point X,, let the corresponding prediction set consist of the top-T scores among (S(X, (KK,,)))s w/)yey.. One then checks whether the associated label Y lies within this top-T set. The empirical frequency of miscoverage directly estimates the true miscoverage probability, and standard concentration inequalities can be used bound its deviation. In contrast, the general case where T may depend on the calibration data itself requires a more sophisticated treatment. The simplified constant-sizz case provides instructive foundation: it clarifies the main ideas behind our estimator and motivates the proof techniques developed in the more general, data data-adaptive setting presented next.
3.2 General case
In the general case, we can also prove that the estimator L00 is consistent, provided that the size constraint rule preserves form of stability for the miscoverages.
7

---

Theorem 3.5. Assume that the score function S is bounded and takes values in the interval [Smin Smax], with O Smin I Smax- Suppose that the number of calibration samples satis- fies >> Smax/ Smin- In addition, assume that the size constraint rule and the score function satisfy the following stability property almost surely:
(P3) There exists a constant L >> O such that:
Σ ag - E(a])
S
L max DE
Σ j=1
E - E
B
Then, if the samples (X Yi) are i.idd, the leave-one-out estimator satisfies:
|aL00 - Ejal| = =Op
V
implies that the size constraint rule preserves form of stability between the varia- tions in ratio vectors and their impact on miscoverage: small variations in the ratio vectors do not cause disproportionately large fluctuations in the miscoverages. Specifically, 0 depends on E and T(((X, Y)) Xtest), while á depends on E and T(((X,, 1. When T is constant, assumed in Section 3.1 we obtain stability results for the miscoverages. How- ever, when is not constant, we require T to satisfy this stability condition to derive consistency results miscoverages. This assumption miscoverages based on ratio vectors E' worked with were i.i..., allowing to directly apply concentration inequalities. In the general case, we rely on this stability assumption to directly manip- ulate the i..dd ratio vectors É Properties (P1 and (P2) are no longer needed, as they were only used
Property P3
not required when T was constant, since the pseudo-
to establish stability. The proof of Theorem 3.5 is given in Appendix C
4 Experiments
We demonstrate through an image classification experiment how the estimator LLOO effectively approximates the true miscoverage EE[ā), showcasing the effectiveness of Backward Conformal Prediction. In this section, we conduct experiments using a constant size constraint rule TT Additional details and experiments motivate the need for controlling prediction set sizes, followed by an image classification experiment using a more complex data-dependent size constraint rule.
provided in Appendix B, starting with binary classification example
Our approach is evaluated on the CIFAR-10 dataset (Krizhevsky] 2009]. which consists of 50,000 training images and 10,000 test images across 10 classes. For prediction, use EfficientNet-BO model [Tan and Le. 2019) trained on the full training set. This model f is treated as a black box that yields predictions. The model is trained to minimize the cross-entropy loss using stochastic gradient descent (SGD) with a learning rate of 0.1, momentum 0.9, weight decay 5 x 10-4, and cosine annealing over 100 epochs. We use a batch size of 512 and apply standard data augmentation during training. At the end of training, the model f achieves a training accuracy of 98.6% and test accuracy of 91.1%.
We evaluate the estimation 1000 IF [[]ā] across various calibration sizes, n E (100, 1000, 5000), and prediction set sizes T 3,, All the experiments are repeated N 200 times. In each run, we sample a calibration set {(X,, Y of size n uniformly at random from the test set. We then compute scores using the cross-entropy loss, defined by:
S(z, v) - log (3)z),
where py (v)(x) is the predicted softmax probability by f for class y given feature x.
From the remaining test samples (i.e., not included in the calibration set), we draw one point (Xtest, Yiest) uniformly random serve as the test point. Based on the test features and the calibration set, we construct a conformal set Č² (Xtest) of size at most T, along with an estimated coverage level 1 - d1000, using the output of Algorithm [] Both â and the intermediate ä; used to compute L00 are computed via binary search over E (0, 1): we seek the smallest such that the number of labels y with Eiest < 1/a (respectively, E < 1/a) is at most T. The search stops when the candidate value of is within tolerance of 0.005 from the optimal value.
8

---

Prific C
1350
655
033 6.03 9.00
830
010
0.50
0.8
1.08
730 005
170 650
100
Prnquery
030
631
002 838
835
0.50
000 106
655 MIE
(hegency
100
1000
12 5000
Figure 3: Histograms of 1 ā and L00 from N 200 runs for various (n, 7) configurations.
The red dashed line shows the empirical coverage probability. See text for details.
We report the results averaged over the runs Figure We plot the histogram of the N values of ā obtained across runs in green, along with their expected value 1 Efāi shown green dashed line. Overlaid on this, we display the histogram of the corresponding N leave-one-out estimators 1 á 1.00 in blue. A blue dashed vertical line indicates the average value of 1 â LOO for visualization. We also plot the empirical coverage rate PPY C( (X..st) as red dashed line.
Recall that the true coverage probability P(Y Ōn (Xtest) )) satisfies the coverage inequality (2). which confirmed our experiments: the empirical coverage probability, depicted by the red dashed line, consistently exceeds the empirical coverage - E(ū). shown by the green dashed line. Furthemore, while values of coverage always remain above - E[ā]. aligning with theoretical marginal guarantees. We also observe that the leave-one-out estimators a400 closely approximate the marginal miscoverage E [[a with the approximation improving the calibration size increases, suggested by Theorems 3. and 3.3 Note that the histograms of à LO0 and may differ, but this expected, as LOOO targets E rather than higher-order properties. These results show that practitioners effectively make use of available calibration data to approximate marginal coverage guarantees through the leave-one-out estimator.
can exceed the coverage probability, especially for small n, the
5 Conclusion
We introduced Backward Conformal Prediction, a novel framework that prioritizes controlling the prediction set size over guarantees of fixed coverage levels. Built on e-value-based inference, our method enables data-dependent miscoverage and uses leave-one-out estimator to approximate marginal miscoverage. This approach offers a flexible, principled alternative in settings where interpretability and set size control are critical, such medical diagnosis. Our theoretical and empirical results show valid coverage guarantees while enforcing user-specified size constraint. These findings open new avenues for adaptive conformal prediction and data-driven control that go beyond mere coverage.
Backward Conformal Prediction differs from standard conformal prediction in fundamental way: it allows practitioners to control the size of prediction sets while simultaneously estimating coverage. In standard conformal prediction, coverage is guaranteed, but the size of each prediction set is uncontrolled and can vary widely. Our approach provides more actionable information, enabling practitioners to adjust the size constraint if the resulting coverage is too low. While the coverage

---

guarantee in Backward Conformal Prediction might be slightly conservative due to the use of Markov's inequality, this trade-off allows for principled decisions about the balance between set size and reliabiliit,, offering more informative and flexible framework for practical applications.
While focused miscoverage definitions with fixed set sizes, the method is applicable any data-dependent miscoverage. Future work could explore alternative formulations and investigate real-world applications under different constraint rules. Additionally, the stability assumption in Theorem 3.5 could likely be relaxed under regularity conditions.
The size constraint rule also affects the marginal miscoverage E[oi]. and dynamic, anytime adjustments to this rule based on the test sample could be explored, leveraging the leave-one-out estimator for real-time miscoverage estimation.
Finally, although our analysis focuses on classification, the method can also be applied to regression, which could be valuable direction to explore.
Acknowledgments and Disclosure of Funding
The authors would like to thank the reviewers for their thoughtful comments.
Funded by the European Union WEWC--222SYG--CCA---OOOOO 1010716011.. Views and opinions pressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Research Council Executive Agency, Neither the European Union nor the granting authority can be held responsible for them.
This publication is part of the Chair "Markets and Learning," supported by Air Liquide, BNP PARIBAS ASSET MANAGEMENT Europe, EDF, Orange and SNCF, sponsors of the Inria Founda- tion.
This work has also received support from the French government, managed by the National Research Agency, under the France 2030 program with the reference "PRIAT]RIE-PSAI" (ANR-23-IACL. 0008).
References
Anastasios N. Angelopoulos and Stephen Bates. Conformal prediction: gentle introduction. Foundations and Trends in Machine Learning, 16(4):494-591, 2023.
Anastasios N. Angelopoulos, Stephen Bates, Michael I. Jordan, and Jitendra Malik. Uncertainty sets for image classifiers using conformal prediction. In Proceedings of the International Conference on Learning Representations, 2021.
Anastasios N. Angelopoulos, Rina Foygel Barber, and Stephen Bates. Theoretical foundations of conformal prediction. arXiv preprint arXiv:2411.1182. 2024.
Alexander A. Balinsky and Alexander David Balinsky. Enhancing conformal prediction using e-test statistics. In Symposium Conformal and Probabilistic Prediction with Applications, 2024.
Richard Berk, Lawrence Brown, Andreas Buja, Kai Zhang, and Linda Zhao. Valid post-selection inference. The Annals of Statistics, 41(2):802 - 837, 2013.
Stéphane Boucheron, Gábor Lugosi, and Pascal Massart. Concentration Inequalities: Nonasymp- totic Theory of Independence. Oxford University Press, February 2013.
Sacha Braun, Liviu Aolaritei, Michael Jordan, and Francis Bach. Minimum volume conformal sets for multivariate regression. arXiv preprint arXiv:2503. /9068, 2025.
Maxime Cauchois, Suyash Gupta, and John C. Duchi. Knowing what you know: valid and validated confidence sets in multiclass and multilabel prediction. Journal of Machine Learning Research, 22 (81):1-42, 2021.
Tianqi Chen and Carlos Guestrin. XGBoost: scalable tree boosting system. International Conference Knowledge Discovery and Data Mining, 2016.
10

---

John Cherian, Isaac Gibbs, and Emmanuel Candès. Large language model validity via enhanced conformal prediction methods. In Advances in Neural Information Processing Systems, 2024.
Guneet S. Dhillon, George Deligiannidis, and Tom Rainforth. On the expected size of conformal prediction sets. In International Conference on Artificial Intelligence and Statistics, 2024.
Etienne Gauthier, Francis Bach, and Michael I. Jordan. E-values expand the scope of conformal prediction. arXiv preprint arXiv:2503.13050 2025.
Ulysse Gazin, Gilles Blanchard, and Etienne Roquain. Transductive conformal inference with adaptive scores. In International Conference on Artificial Intelligence and Statistics, 2024.
Subhankar Ghosh, Taha Belkhouja, Yan Yan, and Janardhan Rao Doppa. Improving uncertainty quantification of deep classifiers via neighborhood conformal prediction: novel algorithm and theoretical analysis. In AAAI Conference on Artificial Intelligence, 2023.
Peter Grünwald, Rianne de Heide, and Wouter Koolen. Safe testing. Journal of the Royal Statistical 2024.
Society Series B: Statistical Methodology, 86(5):
Peter Grünwald. Beyond alpha. Proceedings of the National Academy of Sciences, 121(39))22302098121, 2024.
Pearson: E-values enable hypothesis testing with datvenriven
Leying Guan and Robert Tibshirani. Prediction and outlier detection in classification problems. Journal of the Royal Statistical Society Series B: Statistical Methodology, 84(2):524 546, 2022.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. Conference on Computer Vision and Puttern Recognition, 2016.
Yotam Hechtlinger, Barnabás Póczos, and Larry Wasserman. Cautious deep learning, arXiv preprint arXiv:1805.00966) 2019.
Wassily Hoeffding. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58(301):13-30, 1963.
Ian T. Jolliffe. Principal Component Analysis for Special Types of Data. Springer, 2002.
Shayan Kiyani, George J. Pappas, and Hamed Hassani. Length optimization in conformal prediction. In Advances in Neural Information Processing Systems, 2024.
Nick W. Koning. Post-hoc hypothesis testing and the post-hoe p-value. arXiv preprint arXiv:2312.08040, 2025a.
Nick W. Koning. Measuring evidence against exchangeability and group invariance with e-values. arXiv preprint arXiv:2310.01153, 2025b.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
Bhawesh Kumar, Charlie Lu, Gauri Gupta, Anil Palepu, David Bellamy, Ramesh Raskar, and Andrew Beam. Conformal prediction with large language models for multi-choice question answering. arXiv preprint 200.v:2005.18401 2023.
Martin Larsson, Aaditya Ramdas, and Johannes Ruf. The numeraire e-variable and reverse informa- tion projection. The Annals of Statistics, 53(3):1015-1044 2025.
Jing Lei. Classification with confidence. Biometrika, 101(4):755-76) 2014.
Drew T. Nguyen, Reese Pathak, Anastasios N. Angelopoulos, Stephen Bates, and Michael I. Jordan. Data-adaptive tradeoffs among multiple risks in distribution-Fi prediction. arXiv preprint 17X6v:2403.19605, 2024.
Sima Noorani, Orlando Romero, Nicolo Dal Fabbro, Hamed Hassani, and George J. Pappas. Confor- mal risk minimization with variance reduction. arXiv preprint 2r11v:2411.01696 2024.
11

---

Henrik Olsson, Kimmo Kartasalo, Nita Mulliqi, Marco Capuccini, Pekka Ruusuvuori, Hemamali Samaratunga. Brett Delahunt, Cecilia Lindskog, Emiel A M Janssen, Anders Blilie, Lars Egevad. Ola Spjuth, Martin Eklund, and ISUP Prostate Imagebase Expert Panel. Estimating diagnos- tic uncertainty in artificial intelligence assisted pathology using conformal prediction. Nature Communications, 13(1):7761, December 2022.
Harris Papadopoulos, Kostas Proedrou, Vladimir Vovk, and Alexander Gammerman. Inductive confidence machines for regression. In European Conference on Machine Learning, 2002.
Aleksandr Podkopaev and Aaditya Ramdas. Distribution-free uncertainty quantification for classifi- cation under label shift. In Conference on Uncertainty in Artificial Intelligence, 2021.
Aaditya Ramdas and Ruodu Wang. arXiv:2410.23614 2024.
Hypothesis testing with e-values. arXiv preprint
Yaniv Romano, Matteo Sesia, and Emmanuel J. Candès. Classification with valid and adaptive coverage. Advances in Neural Information Processing Systems, 2020.
Walter Rudin. Principles of Mathematical Analysis. McGraw- Hill, 1953.
Mauricio Sadinle, Jing Lei, and Larry Wasserman. Least ambiguous set-valued classifiers with bounded error levels. Journal of the American Statistical Association, 114(525):23--34, 2019.
Siddhaarth Sarkar and Arun Kumar Kuchibhotla. Post-selection inference for conformal prediction: Trading off coverage for precision. arXiv preprint arXiv:2304.06158 2023.
David Stutz, Krishnamurthy Dvijotham, Ali Taylan Cemgil, and Arnaud Doucet. Learning optimal conformal classifiers. In International Conference on Learning Representations, 2022.
Mingxing Tan and Quoe Le. EfficientNet: Rethinking model scaling for convolutional neural networks. In International Conference on Machine Learning, 2019.
Aad W. van der Vaart. Asymptotic Statistics. Cambridge University Press, 1998.
Vladimir Vovk. Conformal e-prediction. Pattern Recognition, 166: 11674, 2025.
Vladimir Vovk and Ruodu Wang. E-values: Calibration, combination and applications. The Annals of Statistics, 49(3):1736-177) 2021.
Vladimir Vovk, Alex Gammerman, and Glenn Shafer. Algorithmic Learning in Random World. Springer- Verlag, 2005.
Ruodu Wang and Aaditya Ramdas. False discovery rate control with e-values. Journal of the Royal Statistical Society Series B, 84(3):822 852, 2022.
Xiaoning Wang, Yuyang Huo, Liuhua Peng, and Changliang Zou. Conformalized multiple testing after data-dependent selection. In Advances in Neural Information Processing Systems, 2024.
William Wolberg, Olvi Mangasarian, Nick Street, and W. Street. Breast Cancer Wisconsin (Diagnos- tic). UCI Machine Learning Repository, 1993.
Ziyu Xu, Ruodu Wang, and Aaditya Ramdas. Post-selection inference for e-value based confidence intervals. Electronic Journal of Statistics, 18(1):2292 - 2338, 2024.
Ge Yan, Yaniy Romano, and Tsui- Wei Weng. Provably robust conformal prediction with improved efficiency. In International Conference on Learning Representations, 2024.
12

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract and introduction clearly state our main contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: We consistently discuss the assumptions and their scope for each theorem, and we acknowledge in the conclusion a limitation of our work regarding the stability assumption, suggesting that simpler alternative assumptions may exist.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model cell-speciiiiatinn asymptotic approximations only holding locally). ). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
13

---

Answer: [Yes]
Justification:
We provide proof' of all theorems in Appendix C
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We provide complete implementation details, hyperparameters, and experi- mental setup. Our code will be made publicly available upon acceptance.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
14

---

reproducible, they Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: All datasets used in our experiments are publicly available, and we will also release our code publicly upon acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.c cc/
public/guides/ Staelees//ooeSSSuuesisiiiiiiii
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/
Cooee//ddeeeSbmmmcia ssi onPolicy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide all the training and test details in the paper, and will also release our code publicly upon acceptance.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification:
All experiments are based multiple independent runs.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
15

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma enror bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: See Appendix
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
Our research complies with NeurIPS Code Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We identify the broader societal impacts of our work in the introduction, where we outline relevant applications and the motivation for considering controlled and interpretable prediction set sizes.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
16

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: We believe this paper does not pose such risks as do not release any models or datasets.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All datasets and models used in our work have been properly credited.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
17

---

If assets are released, the license, copyright information, and terms of use in the thcode has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssit
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
Our research does not involve crowdsourcing or human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: No human subjects research was conducted.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
18

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development our paper does not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM for what should or should not be described.
19

---

A Further related work
In this paper, we focus on the split conformal prediction setting [Papadopoulos et al. 2002], where proper training data is used to pre-train a model, and a separate calibration set then formed from other data to construct conformal sets. Conformal prediction has become widely adopted technique due to its reliability, particularly high-stakes or resource-constrained settings, such as medical diagnostics (c.g.. Olsson et al. (2022)) ) This widespread applicability has spurred considerable interest in improving the informativeness of conformal sets, with the goal of providing more precise and actionable predictions.
However, traditional conformal prediction frameworks suffer from key limitation: the miscoverage rate is predetermined, and the resulting conformal sets are not controllable. To address this, recent research has explored methods that allow for adaptive, data data-dependent miscoverage rates. Notable contributions include the work of and Kuchibhotla [2023]. which proposes constructing valid coverage guarantees simultaneously for each miscoverage level with high probability. Their approach relies on constructing confidence bands for cumulative distribution functions, which is analogous to the simultancous inference framework of post-selection inference introduced by [Berk et al. (2013). However, one limitation of this method is that it relies the i..d. assumption. Similar techniques with post-hoc miscoverage have been used in various conformal prediction setups, such as in the transductive setting where multiple test scores are considered [Gazin et al. 2024 in the multiple testing setting [Wang et al. 2024). or in risk control [Nguyen et al. 2024].
Another important related direction is the work of Cherian et al. [2024). which combines two powerful ideas: conformal prediction with conditional coverage guarantees, and level-adaptive conformal prediction. In their framework, the miscoverage level cx(-) is treated as a learned function, optimized to satisfy some quality criterion. In contrast, our goal not to learn ax-)), but rather to estimate the marginal guarantees that result from applying user- specifiee constraint the size of the conformal set. The level a in our case is directly derived to satisfy this constraint. Additionally, we allow the level to depend not only on the test features but also on the calibration set, which introduces further flexibility in adapting the coverage to the data.
We propose more direct approach based on e-values Vovk and Wang 202 Grünwald et al. 2024 robust alternative to p-values that offers several advantages such as stronger data-dependent Type-I error guarantees, which we leverage in this work. Our method yields marginal coverage guarantees, and we provide a practical procedure to estimate the corresponding marginal miscoverage term.
B Further experimental details
All experiments were run on a machine with a 1 13th Gen Intel@ TM i7-13700H CPU and typically takes 0.1-1.5 hours for each trial, depending the calibration size First, provide additional details on the experiment conducted in Section 4 followed by an illustration of Backward Conformal Prediction applied to other experiments.
B.1 Additional details for the experiment in Section
For completeness, we provide samples of histograms of å for = 1,..., in Figures and and which are used to compute the aggregated estimate à 1.00 These samples correspond to the histograms obtained from single run of the experiment conducted Section We use the same setup in that section, with a constant size constraint rule T = 1 and calibration size of n = 5000.
B.2 Introductory setting: binary classification
Now, we highlight the importance of controlling the size prediction sets with a binary classification example.
In binary classification, conformal prediction yields prediction sets of the form Ø, (0). (1), or {0, 1}. However, applying standard conformal methods with a fixed miscoverage level can often result in trivial prediction sets equal to (0, 1}. which offer no information. Ideally, one would prefer prediction sets of size most T 1, providing decisive and actionable predictions.
20

---

1000
lrequeecyy
600
400
Figure 4: Sample histogram of the values örj, for j 1.... n, used compute $L00 with 7
and n = 5000.
1000
800
Trequency
400
0.6
Figure 5: Sample histogram of the values öj. for j 1,.... used to compute L00 with T = 1 and n = 5000.
We perform binary classification experiments on the Breast Cancer Wisconsin (Diagnostic) dataset [Wolberg al. [993]. The dataset consists of 569 instances, each labeled as either benign (y = 0) or malignant (y- = I), with 30 real-valued features computed from digitized images of fine needle aspirates of breast masses.
We randomly split the dataset into 70% training and 30% testing data. We train an XGBoost classifier using cross-entropy [Chen and Guestrin 2016). The trained model achieves an accuracy of 97.7% on the test set.
We repeat the experiment N 200 times. In each iteration, randomly sample calibration set (X,,, Y= consisting of half the original test set, drawn uniformly at random. We compute conformity scores using the cross-entropy loss, defined for binary classification
S(x, y) = ly log(f())) + (1 - y) log(1 f(x))],
where (xx denotes the predicted probability of class produced by the pre-trained model f.
From the remaining test points (i.e., those not included in the calibration set), we select one sam- plc (Xiest Yes) uniformly at random to serve as the test input.
When applying standard conformal prediction methods with fixed miscoverage level, the resulting conformal sets may turn out to be trivial. For instance, over N 200 runs with a fixed miscoverage level a 0.02, obtain empty conformal sets of size 2. This means that out of total of 200 patients, 34 of them receive prediction sets that offer no insight into their diagnosis.
166 informative sets of size and 33 uninformative
To address this issue, we instead use Backward Conformal Prediction with a fixed prediction set size T = 1. Based on X and the calibration set, compute an adaptive miscoverage follow- ing (4) and construct the corresponding conformal prediction set O. (Xws)) of size at most T = 1.
The adaptive miscoverage â described in Equation (4) is computed using binary search with tolerance 0.005. For each run, we record the corresponding value of á and the resulting prediction set C% (Xiea). In Figure 6 we plot the histogram of the N values of 1 á obtained across runs
21

---

PriYoss E ColNnelll
Frequency
90-
10-
Figure 6: Plot of the adaptive miscoverage levels ā and the empirical coverage rate P(Y Ca (Xee)) the binary classification task, using Backward Conformal Prediction with fixed set size T=
in green, with the mean 1 - IE [[]]] shown as green dashed line. We also report the empirical coverage probability P(Yesu E Că (Xies)) red dashed line. We observe that the coverage probability P(Yest Ch (Xtest) consistently exceeds the marginal coverage E[ā]. providing additional empirical support for the coverage guarantee 2
Note that when using standard conformal prediction methods with fixed miscoverage level set to a = E[ we still obtain 196 conformal sets of size and sets of size As a result, some patients receive prediction that provide no informative guidance. This simple binary classification example highlights the need for adaptive control of prediction set sizes, which can be addressed using Backward Conformal Prediction.
B.2.1 Adaptive size constraint rule
We also illustrate Backward Conformal Prediction with a size constraint rule that adapts to the data. Intuitively, features associated with high label variability, as estimated on the calibration set, correspond to greater uncertainty and should be assigned larger prediction sets. We define the size constraint rule in a data-dependent manner, adapting the desired prediction set size to the local uncertainty around each test point.
We begin by extracting features from pretrained ResNer-18 network (He et al. 2016), and project them onto a two-dimensional space using principal component analysis (PCA); for a comprehensive treatment of PCA, seeJolliffe [2022] For each X, in the calibration set, identify its A nearest neigh- bors in the PCA space, denoted (X;),, and collect the corresponding labels {Y, : E NN((.)).
We then compute the empirical label distribution in the neighborhood of X;.
px. 1 K
Σ
1(, c), for E fl,..., D/}
(EN((X.)
and define local label entropy as
-(X.)=->
= - px, (c) log, Px, (c).
This yields a sealar uncertainty score for each calibration point. We compute (X similarly for any test point X by considering its k nearest neighbors among the calibration features.
To map the local entropy tion : + [Tmin...., Tmax), where Tmin and Tmax denote the minimum and maximum allowable sizes for the prediction sets.
discrete prediction set size, define binning func-
Given the entropy value (Xus)) around a test point X the size constraint rule defined as:
T(((,, Y)) X b (Xios))
(6)
22

---

To construct b, we partition the range of entropy values observed on the calibration set Tii
into
mmax
bins. These bins are defined by thresholds:
min H(X)) im!......
(L-1)
(2-1
bins
min
H(X)) +
max H(X;) jml.....
8=1
where the exponent 1/2 is arbitrary and skews the binning towards low entropy regions.
The binning function b then maps entropy value h to size Tmin f, where & is the index of the bin containing h. That is,
L-1 1th
b(h) min
11th > binse
In summary, TX,,,, Y)) Xes) is larger when the local neighborhood of X exhibits high label variability, leading to broader prediction sets in more ambiguous regions of the input space.
12
Sample
Sumple
36
Local Label Eneopy (bb)
RCA2
2:40
Peelction
2.18
1:50
Local entropy of test point: 2.648.
Prediction set size for the test sample: 2.
Figure 7: Illustration of the size constraint rule on the calibration and test samples. Left: local label entropy in PCA space. Right: corresponding values of the size constraint rule T defined in Equation 66) applied to both calibration and test points.
We repeat the experiment in the previous subsection with constant T, except this time
use the adaptive, data-dependent rule defined above with Taun = 1, Tmax 3, and k = 20. An illustration of the adaptive size constraint rule T defined in Equation (6) with 1000 is provided
in Figure] The corresponding coverages are presented in Figure X
Ud
Boquetes
PrtMu Cf.X..ll
PriKu GatNaell
PriViu CfiX..ll
0.70
033
100
1000
5000
Figure 8: Results for the adaptive T defined in Equation
We observe that the coverage probability exceeds the empirical coverage, supporting the validity of (2). Additionally, as in the constant case, the estimator 1.0 increasingly approximates the marginal miscoverage [[ā] as n grows, in line with Theorem 3.5 These observations suggest that Backward Conformal Prediction provides a principled approach to uncertainty quantification in machine learning yielding controlled-size prediction sets, even when the set size is dynamically adapted based on the data.
23

---

Proofs for Section 3
C.1
Constant size constraint rule
Properties
PI
and
P2)
allow us to restrict the analysis to the open subset of R defined by
E
Eyyyey E R
p
1 +
By < 1 ST. E, + 1 vy E f}
E = (E EByyey E ERR E, < < E2 > vj & 1
which is an open set an union of finite intersections of open subsets of R In the following, we denote by II-I] the infinity norm on R!'..11.7 We introduce:
R DLLTT
00 inf {a E (0, # (y:E, <1/a} sT}
F:
(7)
E (Eg)
tool study the behavior of â defined in
C.1.1 Stability F
In the definition of the cardinality # (y Ey < 1//a] is a sum of indicator functions:
E, [[Ev<1/a) which non-differentiable. For À > 0, the bump: a (A(L/a Ey)), where:
cxp (- )
if > 0,
(xx)
(8)
if x < 0,
acts as a soft indicator: is close to when the inequality E, otherwise. Note that
< 1/a is satisfied and close to 0
+ exp (-1)
if x > 0, if x S 0.
of (x)
0
Step function
À 10
À 100
À 1000
0.2
0..75
-0.50
-0.25
0.50
0.75
1.00
Figure Visualization the smooth approximation (Axr) to the step function for various values As À increases, the transition from 0 to 1 becomes sharper, making o (Azz) closer approximation to the discontinuous step function.
In our analysis, we approximate the non-differentiable cardinality condition using a smooth surrogate by replacing the indicator function with this bump approximation. Specifically, we consider:
R P.I.TTT
0. inf {a (0,1) (A(1/a- E,)) sT}
Fx
(9)
E = (E
for À > O. This replacement ensures differentiability with respect to a, and recovers the original cardinality condition in the limit À oc.
We now show that this smooth surrogate is meaningful by proving that the associated infimum Fx converges pointwise to F as À +

---

6.-
Tru cardinalty
Cardiality
65-
A 1000
5---
6.0
35-
Figure 10: Illustration of the difference between # {y: E, < 1/aa (in black) and its smooth approximation a (X(1/a- Ey)). As À increases, (A(1/a Ey)) recovers the original cardinality.
Lemma C.I. Let E E RDI.T Then,
lim Fx FFE).
Proof. Define the following two functions for any (0,
f(E, = # by Ey 1/a Σ <1E))
yEy
AAEE a) = Σ a((11a Ey)),
yEY
where a is the bump function defined B8.
By construction, we have for all > and (0, 1):
fx a) a))
since (A\(1/a- - -Ey)) I 1 and vanishes when E, 2 1/a. It follows that:
EX S F(E).
Now, let > 0. We will show that (X 2 F(E) when is sufficiently large. Assume
F(E) > 0 (otherwise, the result trivial). Define:
0g = F(E) E,
and assume, without loss of generality, that & is small enough so that oz E (0, By definition of FEE), we have:
f(E, 0²) > TT,
because 0g < F(E). Moreover, since fx converges pointwise to f as À oo by definition of the bump function there exists Å > 0 such that for all A 2 Ao
ax, TT
This means that ag is not feasible in the infimum defining Fx (E), and since fx is non-increasing in its second argument:
EX 2 ag = F
for X 2 Ao-
Combining both bounds, for all X > Ao.
F(E) E FX < F(E).
Since this holds for any E > 0, we conclude:
lim
= =
Now, bound the differences between the values of Fx as the inputs vary, and will prove that Fx 1-Lipschitz on any compact set of ('I,, This result naturally follows from the Implicit Function Theorem, standard result in analysis (see, for instance, Rudin [1953)).
25

---

Lemma C.2. Let K. be compuct subset of RR221.,..C Assume that T < (3). Then there exists such that for all X X. there exists constant Rx such that:
|FX Fx & E -E
for all E,, S',E" E K. In other words, Fx is Lipschitz continuous on K. Moreover, ky S 1.
Proof. Let À > To show the existence of kx, suffices to prove that Fx is continuously differen- tiable on an open set of [ (RP.) containing K. since by the Mean Value Theorem, a continuously differentiable function on compact set Lipschitz continuous. We fix such open set U in the following.
To apply the Implicit Function Theorem, may first handle the singularity in = 0. We assume without loss of generality that:
E (Ey)eyy E R.I. B| M
for some constant M which is possible since K. is bounded. Let aM M+1 0, and let
= [g((T" which is > 0 since T < 13%. For all E U and 2 X, we have:
Σ NEY
c((1//a Ey)) Σ a(M(M +1-E,))
NEY
Σ (AA)
NEY
= |
2 T,
where the first inequality comes from the definition of U and the second from the definition of Thus, for all E U and À 2 X, we have (E) 2 aa allowing restrict analysis to (0, 1] instead of [0, 1].
In the following, we fix A 2 X. Define:
& (E, = Σ Oo(A(1/a-B)) Ey)) -T,
BEy
for E (Ey)yey U and (0, 1]. The function Φ is continuously differentiable. Fix E U. For any a E (0. 1], we have:
E ,a) = -
Σ d'(((1/a-E)))) E3)) < <.
(10)
da
a²
BEY
The inequality holds because o' 0, and since E RRP.I.I..T there exists least
such that E6 For such y, have 1/a > E.. hence o' o'(A(1/a Eg)) > which makes the sum strictly positive.
Moreover, for E E U. FXE) is the unique solution to the equation 4 (E, FE)) 0 by the Intermediate Value Theorem. Indeed, 2A(E,,) is continuous on (0, 1] and strictly decreasing with 4x (E, a.) > 0 and (x, Evey (A(1 Ey)) T < # {y E, 1} T < 0, where the strict inequality holds because there exists at least one y E y such that E,, 1.
<
By the Implicit Function Theorem, there exists a neighhorhood Up of E and a unique continuously differentiable function Φx: Un (0. 1] such that xx = Fx (EO) and (E, Φxx 0 for all E E Up. By uniqueness, we have FA = Φx on Up. so Fx continuously differentiable on Un As this holds for any E U, conclude that Fx is continuously differentiable on U.
The Mean Value Theorem guarantees that Fx is Lipschitz continuous on K. with the Lipschitz constant given by xy supgekl! V F (E) It remains show that l < 1.
By the Implicit Function Theorem, we also know that:
1
F (E)
VE,
da
26

---

where 0d da
is given in
100, and
V (E, = (-xoo (M Fx(E) By )))))
Thus, we can express the norm of the gradient as:
Ao F2(F)
Eny)
(E) = max yEy
FX EyEr o' (Λ
FXJEJ Ewy
maX of ()
Egy
Fx(E)
(E)²
Σyey of (* Ex(E) Eyy
< FE)
11,
where the first inquality comes from the fact that o' 2 0, and the second inequality follows from the fact that (X (0, 1). Therefore, [VV S Finally, we conclude that ky 1.
As direct consequence of Lemmas C.1 and C.2 we state the following stability result.
Theorem C.3. Let K. be compact subset of R 1-Lipschitz on K., that is,
110.7 Assume that T < 13%. Then the function F
IF(E') - F(E s |E
E²I, -...
for all E. E.C.
C.1.2 Properties of LO0
We now show how the stability property of F can be leveraged to derive key properties of the estimator LOO, which approximates the miscoverage term E[ā), where à was defined in (4). Theorem C.4 (Theorem I.1p. Assume that the score function S is bounded and takes values in the interval [Smins ) with < Smin Smax- Suppose that the number of calibration samples satisfies a > Smax/ Smin- In addition, assume that the vectors Elest E7, Erest, and E satisfy the following properties almost surely:
(P1) 1 # (u E D' By
(P2) For all E V. Ey 1
Then, if the samples (X,, Y.) are i.i.d., the leave-one-out estimator satisfies:
|aL00 Eia] =0p
Proof. First, since the vectors Etest E,, Etest and E satisfy Properties P1 and P2
they lie in
RIDI.11.7] almost surely. Moreover, since S is bounded, these vectors belong to compact subset of R| RPP.. which allows us to apply Theorem C.3
By the triangular inequality, we have:
La0 - E[a]|
F(E E[F(E
F(B') F
(F2))
+
F
E2
F
ptest ))
+
[
F
ptest
- E[F(Etest)))
1 n
((EE)) (E'))
F
E
Etest
F
E
trsst
E[F(Et EEtest)
n
1=1
Ta
27

---

We will now proceed to bound each of these terms individually, establishing high probability bounds for Ti and 2. Let O.
We start with the expression for 71:
1 Σ ((E)))
((o))(F(())
Ti
By applying the triangle inequality, we have:
Ti F(E') F [
Using the Lipschitz continuity of F (Theorem
C.3 we can bound each term as follows:
Ti
E
E
71
Now, for each j E (1, n) and y E V, we examine the difference between the components E/ and E
S(X,, v)
S(X,, v)
E
E3/
1 S(X,, Yi) + 5(X,,8)
This can be bounded as:
S
Smax
Eia S(X,, Y,) - S(X,:
Simplifying the expression inside the absolute value:
/1 1 R E S(X;; Y.) S(X;a))
S.Sas
1 2i= 1ing 5(X,, Y,) + 5(X,
We continue simplifying:
1 Ei= S(X, Vi) + S(X;,3)) S(Xy,a
Smax
1 ( S(X, Yi) S(X;Y)) St (
We now apply Hoeffding's 's inequality to the numerator and use the bounds on S:
s2 (2g(2/8) 25max
Smax
1 (Smin Smax
Finally, simplifying further:
10g(2/8)
S Smax 1 (Smin/ Smax 1)
2n.
Thus, we obtain the following bound for Ti:
log(2/8) 2n
Ti < Smax
with probability 2 8.
(Smin/ Smax 2)
Next, we bound T2. Since the E. are i.i.d. random variables, we can apply Hoeffding's inequality. Specifically, we have:
log(2/5)
T2
with probability 1 6,
2n
28

---

because F takes values in the interval [0. 11. Finally, bound T3. We begin by rewriting T3 follows:
T3 =
( - E[P(E test }]
F Etest)
F lest
By the triangle inequality, we can express this as:
I3 S E (( F(Etes))
Now, applying the Lipschitz condition on F (from Theorem C.3
we obtain:
T3 E ptest
Etest
Next, examine the difference between the components of Etest and Etest for each y y:
S (exttt 1)
S(Xeest: y)
E
Lest
Elest
Eizl S(,,, Y.) + S(Xtest; y])
Simplifying this, we get:
S
Smax
EE= S(X,, Y,) + S(X test, 3))
n+1
This expression can be further rewritten as:
p - Ei=l S(X,, Y,) 8(X ((N(.))
Smax
n+1
p n++ Ei=1 S(X;, Y.)
S(X n+11
We bound the numerator and denominator separately:
Σ S(X,, Yi)
Smax n+1
n+1
< S max
11 1+I S min
We apply Hoeffding's inequality the sum
Σ S(X,, Y2) and obtain:
2+1
mma 1og(2/8)
Smax n+1
2(n+1)
S S Smax
11 n Smin "
Thus, have:
12g(2/6) 2(++1)
S2 max
n+I
S Smin
H1 Mn+1
Therefore, conclude that:
log(2/8)
Feest
Etest
S2 max
V
n+1 2(n+1)
with probability 2 21-0 5.
Smin
Anfl
Solving for 8, this means that for all 0:
P
Btost
Blost
2 1) 5 2 2exp
12(n + 1)
(I)
11 14--
Swin max
1
2
72 +
and hence:
T3 5
Biest
Frsst
P p
Elest
Feest
2t) dt
0
(40 n
Swin
1
<
2exp
-2(n 11)
dl.
00
1 52 S
nt +
29

---

Substituting:
n Smin
1
Swin dt,
u tq
du 11 + S
+
S
max
n +
max
we obtain:
25
max N +
T3 <
exp (-2(n + 1)u²) du
J4Smin
25 max + (
exp (-2(n 1)u²) du
m.Smin
00
2 n +
π 2(n + 1)
m.min
V
Finally, by applying union bound, we obtain:
(ng(4/8)
log (4/6)
25 max
LL00 -Eaa]] S Smax
2n
π V 2(n + 1)
(Smin/ Smax
2n
S.S.inn
n
that holds with probability 2 - 5. This shows that:
|a_00 E|al = Op
1
Vn
Note that when applied Hoeffding's concentration inequality to the sums of 5(,,, Yi), could have used (Smax - Smin)² instead of Stax. in the error term, given that S takes values in the interval [Smin Smax]. However, chose keep S max in order keep the error terms as simple possible. have:
Theorem C.5 (Theorem 3.3 Under the assumptions of Theorem 3.7
Var (a = =0(=)
Proof. We the same notations in the proof of Theorem 3.1 We begin by decomposing the leave-one-out estimator 1.00
1 プー=
F(E2) into a sum of f.i.d. components and deviation
term:
1
à 1.0
Σ n
F
E
F(E) F E
G.idd.
=:
where the F
E
are i.i.d. copies. Since F takes values in [0, 1]. the variance of each i.i.d. term is
bounded by Var
F E
1. yielding:
1 = 0 (2)
Var (a (ii.d.)
4n
11
The variance decomposes as:
Var (a--0) Var (alid)) + Var (A) + 2Cov (atid. A).
We have already bounded the first term, Var (a)i.))). It remains to control the variance and covariance terms involving A.
For Var (A), note that A Ti, and we established in the proof of Theorem 3.1 that:
log(2/8) / (Smin/ Smax I
A Ti < Smax
with probability 8.
30

---

Thus, we obtain tail bound:
tgc
Smin
1 ) 2
P(| t) S 2 exp
-2n
Smax
Smax
72
71
Then:
Var (A) = E [ E[A]
S E [
P ([A] 2 Vu) du
2
IP (P 2 1) dt (substituting a = l,, du = 2dt)
111
Smin Smax
2
Σ
texp
-2n
dt.
Smax
Substituting:
tj
Smin Smax
1 Smax
Smin
1 11
du
dl,
Smax
Smax
obtain:
2
100
v + 2 71
exp( exp(-2no'* dv.
Smin Smax
Smas
Now, we split the integral:
[
2
100
0c 2
exp(
(2nx-22 du
Texp( -nw ) do +
exp( -nnx?) du
2
texp( 2ne do
exp( (221eo2) du
00
1
2
exp(
4n
n
2n
8/n
π
e
4n
2n
Therefore:
2
2
2
Var (A) S
8/n
0
Smin
+)
4n
n
2n
mace
Smav
Finally, by the Stuchy-Schward inequality:
[Cov (alid, A)] S / Var (áhi.d.) Var (A)
11
Smin
1
V
4n
4n
8-8/n
8/n
+
2
2m
π
= 0
Smax
Smax
Putting everything together, we have shown that:
Var (a = =0 (-)
LI
C.2 General case
Theorem C.6 (Theorem 3.5
Assume that the score function S is bounded and takes values the interval [Smin Smax) with < Smin Smax. Suppose that the number of calibration sumples satisfies n > Smax/ Smin- addition, assume that the size constraint rule and the score function satisfy the following stability property almost surely:
Smin-Smin-
31

---

(P3) There exists constant L > 0 such that:
n1 Σ
n
Gā F(a))
L max yEy
Σ
E
F
B
j=1
j=1
Then, if the samples (X,, Y.) are i.i.d., the leave-one-oul estimator salisfies:
aa E'āl = =0p
1 vn
Proof. We have:
1o.00
Efa]|
1
Σ
(āy E(ā))
E
E
E
Σ
E
E
E
E
E
teett
E
E Etest
E E
E
17
B
E
Brost
+
LE
Ftest
Etest
where the first inequality is precisely the stability property P3 and the second follows from the triangle inequality.
In the proof of Theorem
3.1
we have shown that:
log(2/6) 2n
+
E
E
S Smax
with probability 2 1 - 6,
14 (Smin/ Smax
and
E
test
Etest
252
max n +
π
p.S.
2(n 1)
remains to bound combined with Hoeffding inequality yields:
Σj=
Ereat
)
with high probability. For any 2 0, union bound
P
1
E
F
E
test
26 =P
E
F
Frestt
2
71
yEy!
j=1
< Σp P
E y
E
Etest
yEy
<2exp(
2ne 2,2
(Bmax - Smin 1
2np
$ | exp
(~(Smaxx - Smin
Therefore:
Smax
Smin
log (2
1
b test
<
with probability 2 1
72
/
2n
1=1
32

---

Finally, applying union bound and combining the above inequalities, we obtain:
2 2n +
Smax
Smin
[og(44)))/0)
2LS ++
JáLO0_ -Eaā] S LS max
n
11 (Smin/ S Smax )
1
2n
u.min
n
V 2(n + 1)
with probability 2 1 - 6. We conclude that:
|a_000_ - Eial = =0p
Vn
D Proof of Proposition 2.2
For completeness, give here proof of Proposition 2.2 which can be found in [Knnin 2025a
Proposition D.1. Consider calibration set {(X;, Y.) and a test data point (Xtest Yiea) such that (X1, Yı),.... (Xn, Yn); (Xtests Yeea) are exchangeable. Let & be any miscoverage level that may depend on this data. Then, for the conformal set
S(z, v) m41 i== 5(X,, Yi) S(x,u))
=
<1/ā
the coverage inequality 2 holds, i.e.,
P(Yest E Ch (Xeest)
21-E
Proof. Since Etest is an e-value (that does not depend on a)), we have E[sup 1 E > 1/a]/a] S 1, because 1 Etest 2 1/a) 1 (aEtt 2 1} I a.ttet Rewriting this inequal-
ity gives 1/E(sup, {sE 2 //a]/a| 2 1, and applying Jensen's inequality then yields E[L/(sup.>0 [ 1/a)/a]] 2 1, which is equivalent to [i aj 1, or, equíva-
lently, E[1 in 21/0 S.
21/0 Noting that on 1/a a = 1/a} - a, we obtain E[sup, .> 1f 2 1/a] aa S 0...e, supa>o E[1fEtest 2 1/a] al < 0.
1{Etest 2 1/a) - a = supa>o 1
Therefore, for any possibly data-dependent à 0, E[1 2 1/ā)| S F[ā),, i.e., P(
1 1/ā) < [[ The result then follows by substituting E and C. with their definitions.
33


Reviews:
[{"review_id": "VwmmLP5hkk", "reviewer": "Reviewer_i8Nu", "summary": "This paper proposes \"backward conformal prediction\" a conformal method that instead of fixing the coverage before-hand, it fixes the set size, and then make prediction sets that admits to that set size constraint aiming for largest possible coverage guarantees. Their method rely on recent advancements in connecting e-values to conformal prediction. Furthermore, the introduce a novel leave-one-out style estimator of the achievable coverage level of their conformal method. They also provide empirical evidence for the performance of their method.", "questions": "Already asked above.", "limitations": "yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 2, "originality": 4, "strengths_and_weaknesses": "- The idea of providing set-size constrained CP sets is interesting and useful for a set of real-world applications.\n- The presentation of the paper is very good. I like the easy-to-understand introduction to e-values and their application to cp without making the notations messy. \n- The LOO method to estimate the coverage level sounds novel to me, and might also be of independent interest beyond CP. \n- Despite the pleasant characteristics of e-values, I am still not fully convinced why e-values are actually needed in this scenario, at least from the algorithmic perspective. One can propose the following, perhaps simpler, algorithm: Take any conformity score you like. Look at the prediction sets of the form ${y | S(x, y) \\leq q}$, as is the convention in CP. Then sweep the value of q. if you pick q small enough, you will eventually satisfy the set size constraint. Then one can pick the largest value of q such that the set size constraint is satisfied over the calibration points. Then, perhaps by some very mild assumptions (iid and bounded scores) one can derive guarantees for set size constraint at test time. Similarly, by looking at what quantile of the calibration scores the selected q belongs to, one can also derive an estimate (along with a guarantee) for the realized coverage in test-time. How is this algorithm different than yours? and why should some one take the route of (potentially more complicated) e-values?", "was_revised": true, "final_justification": "I have read other reviewers comments, and I believe some of the points raised by the reviewer A3K2 are valid and has to be addressed properly. For instance, the point about approximation error is valid to me. Also, about the importance of set size control CP, although I find it interesting, the use of it in practice is not entirely straightforward. The thing about coverage control is that it has some clear implications for the down-stream decision makers, for instance look at the recent work of [1]. It is not obvious how a set size controlled prediction set should be used or affect the decisions. This needs more discussions.\n\nThat being said, I still believe this is a novel point of view to CP and can benefit the community potentially with opening new perspectives and research directions.", "rebuttal": "*First, we would like to thank the reviewer for their time and detailed review. We also appreciate all the positive feedback received regarding the presentation, clarity, and originality of the paper. We will respond to all comments in order and also address the main question raised by the reviewer regarding the benefits of using e-values for this problem.*\n\n*We note that the points made by the review are similar to that of reviewer kQ7Q, and that our response to their review may contain additional complementary explanations that could help guide the reviewer in their comments.*\n\n* The idea of providing set-size constrained CP sets is interesting and useful for a set of real-world applications\n\nWe appreciate that the reviewer shares our perspective on the value of size-constrained uncertainty quantification. We believe it is indeed a key method with important real-world applications, notably in medicine, as highlighted in our paper. This view is also supported by reviewer kQ7Q. However, reviewer A3K2 does not find this aspect to be of any relevance, which we found somewhat surprising.\n\n* The LOO method to estimate the coverage level sounds novel to me, and might also be of independent interest beyond CP\n\nWe are pleased that the reviewer perceives the scope of our new method even though we preferred to highlight its simplicity in our presentation. Indeed, we specifically designed a proof tailored to our problem, based on an analytical study of the quantile function combined with probabilistic inequalities. This proof could indeed be reused as is, adapted, or serve as inspiration for tackling other problems. The LOO method is indeed novel, and we hope it may inspire similar types of proofs in the literature.\n\n* Despite the pleasant characteristics of e-values, I am still not fully convinced why e-values are actually needed in this scenario, at least from the algorithmic perspective\n\nFirst, we understand the simple algorithm and method proposed by the reviewer. It is indeed possible to obtain theoretical guarantees under the assumption that the data are i.i.d. However, this only works in the simple case **where the size-constraint rule does not depend on the data**. While we did treat this case in the paper, it was primarily to build intuition for the proof in the more general setting **where the size-constraint rule does depend on the data (including all the calibration data), and the confidence level is random**. To achieve sound theoretical guarantees in this more general setting it’s essential to make use of the post-hoc validity that is provided by e-values.\n\nTo help future readers to understand this (critical, but somewhat subtle) issue, we will make use of the reviewer’s suggested algorithm as a didactic device to compare and contrast to our general approach.  In particular, we will add a remark comparing the different approaches in the case where the size-constraint rule is constant.\n\n*We would like to thank the reviewer for their useful and insightful comments.*"}, {"review_id": "eMAwIptQZM", "reviewer": "Reviewer_kQ7Q", "summary": "This paper introduces Backward Conformal Prediction, a method that ensures conformal coverage while offering flexible control over prediction set sizes. Unlike standard conformal prediction, which fixes the coverage level and allows set sizes to vary, this approach defines a size constraint rule to dictate how prediction set sizes should behave based on observed data and adapts the coverage level accordingly. It builds on two key foundations: post-hoc validity using e-values to ensure marginal coverage and a novel leave-one-out estimator (α^LOO) to compute the marginal miscoverage from the calibration set, making theoretical guarantees practical. The method is particularly useful in applications like medical diagnosis where large prediction sets are impractical. Theoretical results show the estimator's consistency, and experiments on CIFAR-10 and binary classification tasks demonstrate that it maintains computable coverage guarantees while controlling prediction set sizes effectively. The approach prioritizes set size control over fixed coverage, offering a flexible framework for uncertainty quantification in machine learning.", "questions": "See above.", "limitations": "See above.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "(+)\n1. This paper focuses on an important topic of uncertainty quantification, with a popular technique conformal prediction. \n2. The idea in this paper is pretty novel. To the best of my knowledge, I did not find papers similar to this one, which starts from the constraints of the confidence interval. \n3. Based on the second point, I am convinced that the constraints of the confidence interval are indeed important in many fields like medicine. \n4. The authors further provide theoretical and empirical evidence to validate their findings. \n\n(-)\n1. Could the authors provide evidence on why this paper uses conformal e-prediction as the basis? Is it necessary for the backward conformal prediction? What if we directly use other conformal prediction techniques?\n2. The experiments part might be a little bit weak. More realistic experiments might enhance the scope of this paper.", "was_revised": true, "final_justification": "Post-rebuttal: The authors' replies (as well as their discussion with other reviewers) address my concerns properly. Therefore I keep my score of 5 unchanged.", "rebuttal": "*We would like to thank the reviewer for their detailed review highlighting the strengths of our paper, and we are grateful to know that they appreciated the work. We are also pleased to see that the reviewer found the paper to be well written, clear, original, and addressing an important problem. We will provide a few additional comments on the strengths and questions raised by the reviewer, which are closely aligned with those of reviewer i8Nu.*\n\n* The idea in this paper is pretty novel. To the best of my knowledge, I did not find papers similar to this one, which starts from the constraints of the confidence interval\n\nThis idea is indeed novel, and we are pleased to see that the reviewer shares our enthusiasm for our new method, which quantifies uncertainty based primarily on a size constraint. It is a promising approach that we believe has potential applications beyond the scope of this paper.\n\n* I am convinced that the constraints of the confidence interval are indeed important in many fields like medicine\n\nWe appreciate that the reviewer shares our view on the importance of our method, particularly in fields such as medicine, as illustrated in the paper. We note that reviewer i8Nu agrees on the usefulness of size-constrained uncertainty quantification. However, we are surprised to see that reviewer A3K2 does not share this common enthusiasm.\n\n* The authors further provide theoretical and empirical evidence to validate their findings\n\nWe thank the reviewer for highlighting the theoretical contribution of this paper, as well as the experiments that validate our theoretical findings.\n\n* Could the authors provide evidence on why this paper uses conformal e-prediction as the basis? Is it necessary for the backward conformal prediction? What if we directly use other conformal prediction techniques?\n\nWe thank the reviewer for their question, which provides an opportunity to highlight the importance of using conformal e-prediction in this context.\n\nFirst, traditional conformal prediction methods are not designed to control the size of prediction sets, which can be problematic in certain applications (medicine, for instance). We propose a method that is dual to conformal prediction, shifting the focus from controlling coverage at a fixed level to prioritizing control over the size of prediction sets. Our approach is not intended to replace conformal prediction; rather, we believe it complements it and may be more suitable in scenarios where controlling the size of prediction sets is critical.\n\nRegarding the use of e-values, their key advantage lies in the post-hoc guarantees they enable. This allows us to retain theoretical guarantees even when the size-constraint rule depends on the data. Simpler alternatives may exist in more restricted settings, as noted by reviewer i8Nu. However, our method is more general and remains valid even when the size-constraint rule is data-dependent.\n\nWe would be happy to add a remark in the final version of the paper to clarify why we rely on e-values in this setting rather than on alternative methods that only apply to the simpler case.\n\n* The experiments part might be a little bit weak. More realistic experiments might enhance the scope of this paper.\n\nWe demonstrated the validity of our method on a simple binary classification example and experiments on the popular CIFAR-10 dataset. In future work, we would be happy to apply our method to real-world datasets and even deploy it in practical settings, such as medical applications.\n\n*We once again thank the reviewer for their detailed review and their enthusiasm for our work.*"}, {"review_id": "aENlUlCloR", "reviewer": "Reviewer_A3K2", "summary": "This paper studies backward conformal prediction, where we constrain the size of the prediction set while adapting the coverage level accordingly. Based on the results of Gauthier et al., this paper proposes an estimation of the miscoverage expectation so that we can estimate the expected coverage of backward conformal prediction methods. Experiments show that the estimation is pretty good in practice.", "questions": "Please refer to the weaknesses.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 1, "originality": 2, "strengths_and_weaknesses": "### Strengths\n\n- The paper proposes an estimation of $\\mathbb{E}[\\tilde{\\alpha}]$ and then provides theoretical guarantees about the estimation error.\n\n\n### Weaknesses\n\n- The contribution of this paper is to provide an estimator for $\\mathbb{E}[\\tilde{\\alpha}]$. However, the main results for backward conformal prediction are provided by Gauthier et al. All the paper does is provide an estimator, which is sadly not so meaningful due to the unknown approximation error caused by the first-order Taylor approximation in lines 134-135.\n- There may be a mismatch between the claim \"an estimator of $\\mathbb{E}[\\tilde{\\alpha}]$ can help us reject predictions with low coverage\" and the experimental results. The experiments show that the estimated value $\\hat{\\alpha}^\\text{LOO}$ is close to the true value $\\mathbb{E}[\\tilde{\\alpha}]$. However, this does not mean that the theoretical bounds here are tight. Note that given the theoretical results, we are going to trust prediction sets with $1-\\hat{\\alpha}^\\text{LOO}- R_ \\delta(n) \\ge \\tau$, which is a **distribution-free result**. So the experimental results are not sufficient to show the effectiveness of Backward Conformal Prediction as claimed in lines 244-245.\n- In conclusion, compared with conformal prediction (CP), I do not see the necessity of studying backward conformal prediction (BCP).  Since the approximation error of the first-order Taylor approximation in lines 134-135 is not controlled, BCP can only control the prediction set size, but not the coverage level. So, it seems that CP and BCP focus on controlling different aspects of the prediction set, i.e., the coverage level and the size, respectively. From this point, it seems that BCP is still worth studying. However, the claim in this paper that we can control the size while rejecting predictions with bad coverage is not as good as controlling the coverage while rejecting predictions with large size. It is because although the former can control the size, its rules on the coverage are ambiguous due to the unknown approximation error of the Taylor approximation. On the contrary, the latter can control the coverage, and the rules for prediction set size are really easy to set. So, it seems that BCP can be regarded as a dual method for CP only if the approximation error can be controlled. By the way, in situations where errors can cause serious consequences, controlling the accuracy (coverage) is obviously more important than controlling the set size. Furthermore, the spirit of confidence sets prediction (which includes conformal prediction) is to control the coverage level but not the set size, just like the spirit of hypothesis testing is to control the type-1 error but not the type-2 error.", "was_revised": true, "final_justification": "After the discussion, I was convinced that BCP is worth studying, so one of my main issues was addressed. Another main issue is that the contribution of this paper is not so much because it just provides an estimator of $E[\\tilde{\\alpha}]$. However, I believe such a contribution deserves a positive score since it makes the BCP algorithms potentially applicable in practice. So I changed my mind and decided to give Borderline Accept.", "rebuttal": "*First, we would like to thank the reviewer for their time. We are confident in our approach and hope that our response will clarify our aims, the interpretation of our results, and comparison to classical statistical methodology.*\n\n* The contribution of this paper is to provide an estimator for E[α]. However, the main results for backward conformal prediction are provided by Gauthier et al. All the paper does is provide an estimator, which is sadly not so meaningful due to the unknown approximation error caused by the first-order Taylor approximation in lines 134-135.\n\nOur method indeed builds on the use of e-values in conformal prediction, in particular drawing from the work of Gauthier et al. However, that paper did not focus (at all) on obtaining marginal guarantees. In contrast, this is the central contribution of our work. We emphasize that our approach to providing the marginal guarantees (specifically the expected coverage E[α]) is entirely novel. This contribution is crucial in practice, as it enables practitioners to understand the actual coverage they can achieve with this method and make informed decisions accordingly.\n\nRegarding the final part of the comment, the Taylor expansion is never used to approximate any quantity; it is solely a technical tool to establish the coverage property: Pr(Y_test \\in C(X_test)) \\ge 1 - E[α]. Both the experiments from Gauthier et al. and our own results support and justify the validity of this approximation. We find empirically in a range of experiments that this coverage inequality always holds in practice.\n\n* The experiments show that the estimated value α^LOO is close to the true value E[α]. However, this does not mean that the theoretical bounds here are tight. Note that given the theoretical results, we are going to trust prediction sets with 1-α^LOO-R(n) \\ge \\tau, which is a distribution-free result. So the experimental results are not sufficient to show the effectiveness of Backward Conformal Prediction as claimed in lines 244-245.\n\nWe are somewhat surprised by the comment about tightness, given that the 1/\\sqrt{n} and 1/n convergence rates that we obtain for the mean and the variance, respectively, are ubiquitous in statistics. These rates arise from fundamental properties of random variables, probability theory, and the general structure of estimation problems.  While one could pursue analyses that establish lower bounds, that effort would duplicate statistical literature and be far beyond the scope of the paper.\n\nWe certainly agree that our results are distribution-free, and we view this as a key strength of our method. We are puzzled as to how the reviewer could view this property negatively, especially since the experimental results align closely with the theoretical guarantees we establish.\n\nWe would appreciate it if the reviewer could clarify what they intended by this comment. We would be happy to provide further explanation if it would help clarify our approach and make our contributions in the paper more understandable.\n\n* In conclusion, compared with conformal prediction (CP), I do not see the necessity of studying backward conformal prediction (BCP)\n\nThe CP and BCP methods are fundamentally different and are designed to address two distinct problems. BCP is not intended to replace CP, but rather to be used in situations where the practitioner prioritizes control over the size of the prediction sets.\n\n* Since the approximation error of the first-order Taylor approximation in lines 134-135 is not controlled, BCP can only control the prediction set size, but not the coverage level.\n\nWe do not claim that BCP controls the coverage level. Once again, the Taylor approximation is only used to establish a coverage inequality, and it is consistently satisfied in practice. \n\nOur method, however, does allow for the estimation of coverage. In a sense, BCP even provides more flexibility than CP: while CP controls the coverage but offers no way to estimate the size of the prediction sets, BCP explicitly controls the size of the prediction sets and allows for coverage estimation.\n\n* It is because although BCP can control the size, its rules on the coverage are ambiguous due to the unknown approximation error of the Taylor approximation\n\n**Again, the Taylor approximation is only used to derive the coverage inequality**; moreover, we observe that this inequality always holds in practice, which is consistent with previous results from Gauthier et al. Furthermore, our estimator allows for the estimation of the coverage guarantees.\n\n* By the way, in situations where errors can cause serious consequences, controlling the accuracy (coverage) is obviously more important than controlling the set size. Furthermore, the spirit of confidence sets prediction (which includes conformal prediction) is to control the coverage level but not the set size, just like the spirit of hypothesis testing is to control the type-1 error but not the type-2 error.\n\nThe reviewer is referring to classical statistical practice, dating back to Neyman and Pearson and to Fisher.  But just because it is classical does not mean that it is always the preferred approach.  Indeed, in the paper, we provided motivating examples, for example in medicine, where controlling the size of prediction sets is crucial. In such examples, the key quantity to control is the time devoted to having experts chase down possible leads that arise from a statistical analysis.  Obtaining a large uncertainty set can be unrealistic and indeed useless if requires too much time for experts to perform a follow-up verification to whittle down the possibilities. It may be preferable to allocate expert time/effort in advance and to proceed if the estimated coverage is sufficiently large.  That’s what our method makes possible.\n\nOur method is not meant to replace existing classical approaches where the coverage level is fixed in advance. Rather, it is complementary and orthogonal, offering a new perspective on uncertainty quantification. \n\nOne may view it as surprising that the statistical literature has focused entirely, for several decades, on controlling coverage, with set size a secondary consideration.  It is surprising but true.  But perhaps it is because the shift to the inverted logic of our paper is made possible by recent developments in the use of e-values and their associated post-hoc guarantees.  This makes it possible to obtain theoretical guarantees even when the coverage level is data-dependent, something that is not possible with p-values.\n\nTo say this another way, classical statistical methods have been built around a focus on the control of type I error, typically with a pre-specified, data-independent significance level α and/or p-values.  Generations of statisticians and applied researchers (particularly in medicine) have criticized this focus.  Accordingly, the controversy around p-values persists.  Our work provides one way to escape the tyranny of p-values and to solve inference problems in a new way that is complementary to classical methodology.\n\n*We would like to thank the reviewer for giving us the opportunity to clarify the nature of our method relative to classical statistical practice.  Our revision will make this comparison more clear and emphasize that our approach is novel, for interesting reasons.*"}]

Author rebuttal:

Meta-review: This paper introduces Backward Conformal Prediction (BCP), a variant of conformal prediction that inverts the usual framework: instead of fixing coverage and allowing prediction set sizes to vary, it fixes the set size in advance and then derives prediction sets that maximize coverage under this constraint. The method uses connections between e-values and conformal prediction, contributing a novel LOO estimator of achievable coverage, which the authors show to be consistent and practically useful. Empirical studies suggest that BCP can maintain reliable coverage guarantees while controlling set size. This feature is particularly appealing in settings like medical diagnosis where unwieldy prediction sets are impractical.

Reviewers have generally found the idea of set-size constrained prediction sets both interesting and relevant for real-world applications, with a clear and well-presented introduction to e-values. The LOO coverage estimator is regarded as novel and potentially useful beyond this paper. The work is also appreciated for its theoretical grounding and empirical validation, with novelty in shifting the focus of conformal prediction toward size constraints rather than coverage constraints. However, several weaknesses were noted. Some argue that the reliance on e-values is not fully justified and that simpler, more direct algorithms might achieve similar outcomes without the added complexity. Others note that the main contributions of BCP were already introduced by Gauthier et al., with this paper primarily providing an estimator whose value is diminished by uncontrolled approximation errors (e.g., the first-order Taylor expansion). The experimental section is seen as somewhat limited in scope and lacking more realistic applications. Conceptually, two reviewers also questioned whether prioritizing set size over coverage aligns with the spirit of conformal prediction, since coverage guarantees are traditionally considered more fundamental, especially in high-stakes domains.

After extensive discussion between the authors and the reviewers, I believe the authors have successfully addressed their concerns. Overall, the paper offers a promising and original direction, but it could benefit from expanding discussion around its necessity, methodological choices, and experimental validation. In particular, the authors are encouraged to include many of the points raised during the discussion period strategically in the paper.
Decision: Accept (poster)