Conformalized Survival Analysis for General Right-Censored Data
Authors: Hen Davidov
Shai Feldman
Gil Shamai
Ron Kimmel
Yaniv Romano
Venue: ICLR 2025 Poster (2025)
Keywords: conformal prediction
survival analysis
PAC
covariate shift
uncertainty quantification
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Abstract: We develop a framework to quantify predictive uncertainty in survival analysis, providing a reliable lower predictive bound (LPB) for the true, unknown patient survival time. Recently, conformal prediction has been used to construct such valid LPBs for *type-I right-censored data*, with the guarantee that the bound holds with high probability. Crucially, under the type-I setting, the censoring time is observed for all data points. As such, informative LPBs can be constructed by framing the calibration as an estimation task with covariate shift, relying on the conditionally independent censoring assumption. This paper expands the conformal toolbox for survival analysis, with the goal of handling the ubiquitous *general right-censored setting*, in which either the censoring or survival time is observed, but not both. The key challenge here is that the calibration cannot be directly formulated as a covariate shift problem anymore. Yet, we show how to construct LPBs with distribution-free finite-sample guarantees, under the same assumptions as conformal approaches for type-I censored data. Experiments demonstrate the informativeness and validity of our methods in simulated settings and showcase their practical utility using several real-world datasets.

Published as a conference paper at ICLR 2025
CONFORMALIZED SURVIVAL ANALYSIS FOR GENERAL RIGHT-CENSORED DATA
Hen Davidov Shai Feldman', Gil Shamai!, Ron Kimmel', Yaniy Romano" (dahen, heldman @ cem.pusstechiionac...t (sgils, ron) wec.ssecciion.c.... yromano @teecnion.acc.i
ABSTRACT
We develop framework to quantify predictive uncertainty in survival analysis, providing a reliable lower predictive bound (LPB) for the true, unknown patient survival time. Recently, conformal prediction has been used construct such valid LPBs for rype-1 rightt-coecee data, with the guarantee that the bound holds with high probability. Crucially, under the type-I setting, the censoring time is observed for all data points. As such, informative LPBs can be constructed by framing the calibration as an estimation task with covariate shift, relying on the conditionally independent censoring assumption. This paper expands the confor- mal toolbox for survival analysis, with the goal of handling the ubiquitous general right-censored setting, in which either the censoring or survival time is observed, but not both. The key challenge here that the calibration cannot be directly formulated covariate shift problem anymore. Yet, we show how to construct LPBs with distribution-free free fimite-sample guarantees, under the same assumptions conformal approaches for type-I censored data. Experiments demonstrate the informativeness and validity of our methods in simulated settings and showcase their practical utility using several real-world datasets.
INTRODUCTION
Survival analysis is essential numerous fields, including medicine (Cole Hudgens 2010 Selvin 2008), engineering (Ma & Krings 2008), and social sciences (Cloyes et al., 2010). The primary objective is to predict the survival time The time-to-event such as death, failure, or relapse- based covariates X that may include clinical markers, machine specifications,
sociological
factors. The underlying challenge in survival analysis is that T is not observed for all subjects due to censoring. In particular, in this paper, focus right-censored data, the most common setting in survival analysis, where the survival event may not occur by the end of the follow-up period. This means that for some subjects X, the true survival time T is obscured by the censoring time C. As such, the observed censored survival time is given by T (T, CT,C).
To formalize the above data generation process, we assume the triplets sampled i.i.d. from Px,T,O. The observed data, however, include censored observations, where we consider two possible forms of right-censored survival data.
N.
type-I right censoring: Here, we assume that censoring time C' is observed for all subjects, which typically occurs in studies with a fixed duration, where all subjects are followed until the study ends. Under this setting, the observed dataset is of the form Drype. = {(X,, T., C.
General right censoring: Unlike the type-I setting, in this more general case, we observe either C' or T for each subject, but not both. For example, in situations where the censorship time is obscured by early mortality (Candès et al. 2023 Ahmad al., 2017 Chandrashekar et al. 2014). Concretely, denote by
ITT, < C.F binary indicator variable that gets the value if T; is observed; that is, / = when = 1 and / = C' otheewisee The observed dataset is then given by
Department of Computer Science, Technion - Israel Institute of Technology 6 Department of Electrical and Computer Figgnneriing Technion Israel Institute of Technology

---

Published as a conference paper at ICLR 2025
Deennrat (x, Due to its greater generality, this setup is applicable for many survival analysis problems in biomedical research (Klein 2003 Cole & Hudgens 2010) and other fields. Thus, it is the focus of this work.
Given the observed censored data, the learning task is to estimate the distribution Tf X. How- ever, since we do not fully observe the survival time T for all subjects, we need to impose further assumptions to make the learning task feasible. Indeed, a common assumption in survival analysis is that, conditional on the covariates X, the survival time T and the censoring time C are independent. Formally, this is expressed as follows.
Assumption 1.1 (Conditionally Independent Censoring (Kalbfleisch & Prentice 2011
C X.
Building on the above assumption and related ones, various machine learning approaches have been successful in estimating the distribution of T|X (Cox) 1992. Wei 1992. Lee et al. [0118 Nagpal jet al. 202 Katzman et al. 2018). Despite their notable successes, these methods often lack reliability guarantees for the resulting predictions. This is attributed either to the opaque nature of modern deep learning algorithms or to the simplified modeling assumptions of more traditional statistical methods. Yet, such guarantees are of great importance given the harsh consequences of making erroneous predictions in high-stakes domains, such as healthcare (Navarro et al. 2021 Obermeyer et al. 2019).
1.1 THE NEED FOR RELIABLE LOWER PREDICTIVE BOUNDS FOR SURVIVAL ANALYSIS
In response to this challenge, we aim to construct a lower predictive bound (LPB) on the survival time, supporting any given predictive model with distribution-fri finite-sample guarantces (Vovk iet al.) 20055: Vovk 2012: Tibshirani et al. 2019; Romano et al. 2019 Candès et al., 2023). Our goal is to construct an LPB function L(-) such that, for a new test instance X with an unknown true survival time Treas the event i(Xeee) < Ties occurs with at least
probability. We refer to the probability of this event as the coverage rate of the LPB. For example, if the LPB satisfies the coverage property at level (1 - ax) = 0.9, then L(X(es) is guaranteed to be lower than the unknown Tust at least 90% of the time. To further enhance reliability, we seek to obtain a coverage guarantee that holds conditional on the observed dataset, in a manner similar to Gui et al. (2024); Park et al. level that accounts for the randomness in the realization of the dataset, and define the probably approximately correct (PAC)-type LPB as follows.
(209 ; Bates al.) (2021): Angelopoulos al. (2021). To this end, let o (0, 1) be tolerance
,,..., N be i.i.d. samples from Px,,,C, with L being function of the observed dataset 'D = Dyenenal = {(X,, Tú, e.) where T, min(T;, C.) and = [T; < Cil. L is a PAC-rype LPB at level a E (0, 1) with tolerance 8 E (0,1) if. with probability least 8 over the realization of
Definition 1.1. Let (X,, C.),
P(Trea 2 i.(Xrex) () 2
where the probability is taken with respect to a new data point (Xresty Tist) PX.T-
Recently, conformal prediction methods have been applied to construct valid LPBs for type-I right- censored data. The key idea is to use holdout samples to calibrate the predictions of a survival analy- sis model, ensuring the desired coverage level is obtained for new test points. Under the assumption of conditionally independent censoring, Candès et al. (022) introduced a conformal method that constructs valid and informative LPBs in finite samples. This approach was further refined by Gui jet al. (2024), offering more informative LPBs with PAC-type guarantees. Both methods rely on the availability of the censoring time C for all subjects, which is used to discard early-censored sam- ples. Intuitively, discarding such samples brings the observed time T closer to the true event time T. yielding more accurate LPBs. Specifically, leveraging Assumption 1.I Candès et al. (2023); Gui et al.] (2024) reformulate the LPB construction as estimation problem under covariate shift. Notably, the above calibration methods produce LPBs that are doubly robust, meaning they remain valid if either the predictive model or the estimation of the covariate shift likelihood ratio is accurate.
In the general right-censored setting. however, the literature is less developed, and existing methods lack such strong validity guarantees. This is primarily because the censoring time C' is not available
2

---

Published as a conference paper at ICLR 2025
for all subjects, making the calibration process more challenging. Qi et al. (2024) suggest imputing the value of T with a 'best guess' to calibrate the estimated survival distribution. Naturally, the va- lidity of the method is affected by the quality of the imputation, and our experiments show that their method often fails to achieve the desired coverage in finite-sample. Concurrently with our work, Qin al. (2024) and Meixide et al. (2024) proposed distribution- methods for constructing L.BSs in the general right-censored setting, but their bootstrap-basee approaches rely solely on asymptotic validity under certain regularity conditions. As a result, finite-sample calibration remains elusive. This is especially concerning because small sample sizes are common in biomedical and psycho- logical research, often undermining the validity and reproducibility of statistical findings (Errington)
et al.
2021 Nosek et al. 2022 Button et al. 2013: Nüesch et al. 2010).
1.2 OUR CONTRIBUTIONS
We build on the foundations of Gui et al. (2024) and propose the first method for constructing fiite-sample valid LPBs for general right- segtocensored survival analysis data. To produce informative LPBs, we leverage the event indicator and introduce novel approach to selectively discard certain censored examples in data-driven manner, bringing the censored event time T closer to the true event time T. Crucially, although the calibration cannot be directly formulated as a covariate shift problem, our approach forms doubly robust, valid LPBs under the same conditional independence assumption 1.1 of conformal methods for the type-I setting. Additionally, unlike Meixide et al.) 2224 Qi et al. (2024): Qin et al. (2024), our methods' validity relies accurate estimation of the tully observed e, rather than the partially observed or C. We validate our theory and assess the applicability of the proposed methods using synthetic data, and further demonstrate their effective- ness on real breast cancer data. A Python implementation of our methods provided in our github
repository
BACKGROUND AND RELATED WORK
Denote by gr (xx the true T-th quantile of | X x, and define the oracle LPB for Tiest as the a-th conditional quantile function, i.e., L(Xusti (x) = qa (Xus)) While this oracle LPB is guaranteed to attain - a coverage by definition, in practice we do not have access to the true qa (z), rendering this approach infeasible. As way out, one could use survival analysis tools to estimate the con- ditional quantile function, which we denote by (z)), and form a heuristic LPB for Tiest by setting [(Xisti a) = q.. (Xus). This heuristic approach, however, may not attain the desired coverage, unless the quantile estimates are accurate.
In this paper, we alleviate this issue by building on conformal prediction-4 general framework to calibrate the estimated LPB. The appeal of conformal prediction is that allows us to work with any predictive model and guarantee the desired 1 - coverage at test time. The key idea is to modify the LPB function L(x;)) by rigorously tuning hyperparameter E R using holdout calibration data Teal to attain the desired coverage rate. We denote the tuned hyper-parameter as f, and the resulting LPB as f(:;)).
2.1 FIRST STEPS: NAIVE CONFORMALIZED SURVIVAL ANALYSIS
Before presenting our method, we first outline a naive conformal prediction approach for tuning т. In addition to using this approach as baseline method in our experiments, introduces the core principles of LPB calibration and motivates the need for more powerful solutions.
The key observation here is that Ý < T by definition, and therefore a valid LPB for T is also a valid (but conservative) LPB for T. As such, naive approach to tune 7 is discard the information provided by the event indicator e; and construct a valid LPB for T using the calibration points h.) In more detail, define the LPB function L(X; T) a- (X), which is tightly
connected to the methods by Chernozhukov et al. (2021); Gui et al. (2024) 1 Then, we formulate
(conservative) empirical estimator of the true miscoverage rate a(7) P(7 qr (X)) for each
'There are other possible design choices for i. 7(X;7) such the conformalized quantile regression (CQR)
approach (Romano al. 2019) used by Candès al. 2023
3

---

Published as a conference paper at ICLR 2025
choice of (+),,, E
[0, 1]. defined as follows:
nuive (T) [Teal
Σ < TE
(I)
Above, we use the indicator function T(.) to count the events where T. < (x,) and so Onaive (T)
is an unbiased estimator of O'naive (T) = P(T < Ôr (X)). Since 7 - 'I' we get that Oinaive (T) 2 a(T), implying that the empirical quantity (miie(() is anticipated to overestimate the miscoverage rate, on average. Finally, we define nave as the smallest value of r for which ( meets the user-defined miscoverage level a, i.e..
naive
sup {T E [0,1]: sup @naive (T)) S aa.
(2)
r'ST
In turn, the resulting calibrated LPB is given by L(X; navve) ) = Ftum (X). Notably, this L(x; F PAC-type LPB on T, and thus also PAC-type LPB on T, since 7 S T. While this result is not novel, for completeness, we present Theorem A.I in the Appendix, which proves its validity.
2.2 UTILIZING TYPE-I CENSORED DATA FOR POWERFUL CALIBRATION
Recall that &naive from Equation is an overly conservative miscoverage estimator, particularly when early censoring occurs. This is because ónaive counts the events in which either C. < År (X,) or T; < (x,) hold. To form a less conservative estimator, Gui et al. 20024) suggest discarding the samples for which C; < (x(),,, strategy that is inline with Candès et al. (2023). Under
Assumption T.T] it can be shown that such a selection rule induces a covariate shift. In turn, Gui et al. (2024) introduced the following miscoverage estimator:
1type1(7) EiEra
W, wppes (X,) (X,, s C.) 1[T, < (x.))
Eiera w. 1ype (X,) [X, < C.}
where the weights (.pe-l(2)) (z). approximating I/P (X) <C X = z). rigorously account for the covariate shift induced by the selection rule I(4-(X, I C.). Notably, (type-((7) differs from (T)ive(T) from Equation in that the latter uses all the samples for calibration, i.e,, the selection rule indicator is always set 1. Armed with (ry the hyper-parameter fiype- can be tuned analogously to Equation 2 Finally, the LPB of a new test point is given by L(x; ftype-1 = @iigpes (X).
The advantage of the estimator (T) that it mostly less conservative compared to Qnaive ( (T), resulting in more informative LPBs. However, for general right- -igh--ce data, using Crype-1(7) (T) is not feasible as the censorship time C, is not available for some subjects. Our work addresses this knowledge gap, showing how to rigorously utilize the event indicator e, = T(T; < C.} to construct not only valid but also useful LPBs, as detailed in the next section.
PROPOSED METHODS
An intuitive adaptation of the miscoverage estimator proposed by Gui et al. (2024) general right- censored data would be to examine only calibration points for which e; = T. The challenge with this selection rule that the distribution shift it induces cannot be formulated because, even under Assumption [.I we expect statistical dependence between I and conditional on X. Furthermore, without careful adjustments, estimating the miscoverage using only uncensored subjects jeopardizes the LPB's 's validity as the survival times of uncensored subjects are generally lower than the ones of the general population.
covariate shift. This is
Nevertheless, in this section, we show that with proper weighting scheme, the above set of ideas can be formalized into a valid calibration method. Concretely, we propose two calibration techniques. Our first method, called focused calibration, only uses uncensored subjects for LPB construction. We prove the validity of this method and show that, while it can lead to more informative LPBs than the naive ve approach, it remains conservative to some extent. This observation drives the proposal of our second approach, which builds upon and further improves the first. Our key idea is to reduce conservativeness by including certain censored examples-ae in addition to the uncensored subjects- when calibrating the LPB. We term this method fused calibration, as it utilizes both censored and uncensored samples while maintaining the theoretical guarantees.
4

---

Published as a conference paper at ICLR 2025
3.1 FOCUSED CALIBRATION
In what follows, we formally introduce our focused calibration method, then provide an intuitive explanation for its correctness, and finally characterize the condition under which it leads to LPBs that are more informative than those produced by the naive method. Following Gui et al. (2024). we use the LPB function L(;;T) = (X), but we tune the hyper-parametee т by utilizing the special structure of the general right-censored setting.
In more detail, we calibrate the LPB using samples with e, i.e., those for which T, T, With this selection rule in place, we define the resulting miscoverage estimator as
Ófecus (T) Eier, ((,,) l1fe = 1 11T; < (X.))
(3)
ΣiEr w( 1e: 1
where the weights (x) approximate (e 1 z). Later, explain the rationale behind
=
the use (x) in detail. For now, we remark that: (i) the estimation of Peee = X
can be done by fitting any binary classifier the training pairs (X; and (ii) in Section 3.3
characterize the effect of the estimation error on the resulting coverage.
Next, the tuned Foccs is defined the smallest such that disccus (T) passes the user-specified mis- coverage level o, i.e.
Tacus sup {T E [0. sup deoccus (T') S sa..
TST
Finally, the LPB for a new test point is given by ((X frccs)) = @ (Xust). For ease of reference, the algorithm summarizing our focused calibration procedure is in Appendix [B.]
Building upon the foundations of Gui et al. (2024), we establish a double robustness result for the validity of our method. This result implies that the LPBs are approximately valid if either (i) the weights ((r are well approximated; or (ii) the estimated quantiles (x), E (0, 1) of the condi- tional distribution 7' X are accurate. Additionally, if (ii) is satisfied, the LPBs are approximately valid conditional on Xiest The theorems formalizing the validity of the focused method is post- poned to Section 3.3] as this procedure actually a specific instance of the fused calibration method, which is presented in the next section.
Having presented the algorithm, we turn to discuss why the use of the oracle weight w(x) 1/P(e =1 X = z), estimated by w(x), conservatively accounts for the distribution shift in- duced by the selection rule from Equation 3 Our analysis follows Gui al.) (2024), providing an upper bound for the true miscoverage rate a(T) obtained by the quantile estimator 9, for some
r E (0, 1):
=P() < (X)) = E [P(T < (X) X)]
(4)
[P((T (X) | x)- =0 ( x(X)
-x(X)
(5)
< 1l)))(()))))))
(6)
[ạ (x)) -)ll-()))))))))) w(X)] = Offocus (T).
Above, steps and 6 hold by the tower property, and the inequality in step 5 is due to Lemma B.T. provided in the Appendix. Importantly, the above derivation shows that, while we do not have direct access the true ((T), we can upper bound it using weighted average of observed quantities, as revealed by 7 In turn, the miscoverage upper bound (T) provides the basis for constructing its empirical estimator d'ocuu (T) from Equation 3 with the denominator serving to normalize the weighted average.
While being conservative, our experiments show that the LPBs derived from the focused method tend to be more informative than those produced by the naive approach from Section 2.1 To better understand when this is the case, we now present a condition under which aiooeus (T) is less conservative than (T). Proposition 3.1. Under Assumption
Il the relation Orfocus (T) < OOnaive (T) holds when
P(F (X) X =x < PTT (x( X =x Ve X.
(8)
5

---

Published as a conference paper at ICLR 2025
The proof is given in Appendix[B.2 To highlight the practical implications of the above proposition, recall the definition of T min (T, C) and and re-write Condition 8 as follows:
IP (X) z, 1) < P(C < (x) X 2, 2,
Va E X.
As such, our focused method is anticipated to outperform the na approach when the proba- bility of early survival time for subjects with = 1 smaller than the probability early censorship time for subjects with O. Such phenomenon can occur in medical trials, where early censor- ing is prevalent due to factors such as non-compliance (Zhou et al. 2020). withdrawal of consent (Wilson et al. 2021, and loss to follow-up (Fontana al. 2018: Gill et al. 2018 Monfared et al. 2021).
With that said, Proposition 3.1 also reveals that the nai ve approach from Section 2.1 may produce tighter LPBs than the focused method. Specifically, this occurs for samples that do not satisfy the inequality in Equation 8 This insight naturally brings the idea of detecting such samples and leveraging them to mitigate the conservativeness of Öoouus (T). Indeed, this is the key principle behind our fused calibration method, described hereafter.
3.2 FUSED CALIBRATION
Following the above discussion, we show how to fuse the naive calibration approach with the facused calibration method to enhance statistical efficiency, making better use of available data. In addition to using all the uncensored points for calibration, we aim to selectively include in the calibration set the censored subjects that violate the inequality Equation 8 criterion is thus formally stated as
The oracle selection
f1, P(T < (X) X = < P(T < X- (x) {6,
otherwise.
do not have access to the conditional distribution T' e. However, this indicator sz() can be estimated using a single classifier, as
Observe that cannot compute (z) in practice
follows. First, train binary classifier with ex) imput, predicting the label I[T; qr (X.)). Then, use this classifier to compare the estimated probabilities of the label given (X = 2, e = 0)
and (X x, e 1). We refer to the fitted estimator of sx(x) as 8 (z). Crucially, inaccurate estimation of & does not impact the validity of the calibration procedure, however it can affect the gain in statistical efficiency.
With 8 (x) in place, we formulate the fused selection rule, which uses the calibration point (X,, TT) for miscoverage estimation if either e; = 1 or 8- (xx))
CX,,, 1, if ix,, = eis otherwise.
(9)
Next, the fused miscoverage estimator is formulated as follows:
C (T) EiET, vi (X.) I(G = - < (X,))
EiEE., de (X,) H[G, =1]
where the weights wi (x) approximate w, (z) = (x, = x = x). We note that since 8, (X.) is deterministic given X,, we have that
[1, { 1// = (2).
& (X,) =
w- (X,)
otherwise.
Continuing with the same rationale as in focused calibration, we define Trssed
Fused = sup {T E [0, sup d (T') S sa),
T'ST
and set the LPB for a new test point L(Xiest Plssed @rowes (Xieu).
For ease reference, the fused calibration method is presented Algorithm 3 Notably, we use training data (independent from the calibration points) fit the estimated quantile functions /r (x:),

---

Published as a conference paper at ICLR 2025
the classifiers 8y (z)), and the weights (z);; these models serve inputs to the fused calibration algorithm.
We remark that Algorithm 3 reduces to the focused approach with the choice of 8, (z) == 0 for all и, and to the naive approach when (z) := 1 for all æ. For this reason, we argue that the interpolation between two valid calibration strategies, with the optimal interpolation hyperparameter being estimated by (z)). Indeed, our theory shows that the fused method is valid regardless of how accurately the model (xr) estimates sy (xx).. This is because any selection rule, determined by the model 8 (2), yields a conservative estimate of the miscoverage; see Appendix [.I as well as the formal results given in the next section. Although validity is maintained, we may lose power when using an inaccurate (z). To see this, imagine an extreme case where for any т the focused calibration approach is less conservative than the na i ve approach, i.e., 8- (x) = 0 for all x, and yet our estimate (1)) always predicts the (incorrect) value In such case, the fused method reduces to the na ve approach, resulting in power loss, but maintaining validity.
fused method can be viewed
3.3 THFORETICAL GUARANTEES
Our theoretical framework builds on the work by Gui et al. (2024) and adapts their double robustness result to our proposed methods for general right-censored data. Concretely, states that if the weights are well-estimated, i.e., (2) s w-(x), then the resulting coverage is approximately higher than - a conditional on the training data Drr = {(X,, Pi, Hez, and the holdout calibration data Deat {X; Ti,e)))sz.., where D = Drr U Deat- In particular, this guarantee holds even when the estimated conditional quantiles 4- are inaccurate. Further, following Theorem below, we can see that the guaranteed coverage rate gets closer to the desired level as (i) w gets closer to wy; and (ii) the size of the calibration set [eail increases.
first validity result
Theorem 3.1 (Approximate calibration with accurate weights). Fix any a, E (0. 1). Given that 1r (x) is non-decreasing and continuous in T, and that there exists some constant frr > 0 such that
w- (x) S frr for Px-almost all x. Under Assumption 1.1 with probability at least 1 5 over the
draw of D, the LPB L. produced by either Algorithm 2 or by Algorithm 3 satisfies
[
P T
2 D 1 a Am.
where
1 1 22 max
(1,
2 -1) m
12
Wi (Xi)
-1 Dr +
1
Aw = sup
E
In
rE[0.1]
Wy (Xi) no
0.4 [cal
8
and By = E
d (X) Wir (X)
Du
The proof is given in Appendix D.2
The next result states a stronger, approximated conditional coverage guarantee that holds even when the weights are inaccurate. In essence, this result holds under a stronger assumption that the condi- tional quantiles are well-estimated, i.e., (x) x (-(x)) This is stated formally in assumption (b) Before formalizing this intuition, we define the smallest estimated quantile level to bound T with probability least B, as
(() = sup (r' [] < (x.(X) Dx) S $)
Theorem 3.2 (A (Aprroximate calibration with accurate conditional quantiles). Assume the same con- ditions as Theorem 3.1 and assume further that the distribution of T X is continuous and its density upper bounded by a constant B > O, and that there exists a constant r > 0 such that
(a) For Px-alman all x: (uPpe)r(a) we(x) 7 and supperrla) th s
Fy for some constants 7,, 0
(b) |a suDzex x maxx B, 1)l2r(9) (x) (x)-q((z)) + / is the B-quantile of the distribution T X
[og(1/8) 0.4 Tea ]
S r, where q²(x)
7

---

Published as a conference paper at ICLR 2025
D, the LPB L produced Then, under Assumption 1.1 with probability least 1 1-6 over the draw Df by either Algorithm or by Algorithm 3 satisfies that for Px-almon all x,
T L(X)DD,
Ag.
where Ag suppeea.aar suPeex {2B (0(0))()) as (x)1} + t 0.4 log 3
The proof is deferred to Appendix D.3
4 EXPERIMENTS
4.1 SIMULATION STUDIES
We follow the experimental protocol outlined by Candès et al. (2023) and Gui et al. (2024) to eval- uate the performance of methods across various simulated scenarios. Notably, since real-world data lack the true survival time 7 for all test points, synthetic data experiments are important to verify the correctness of our theory.
Base predictive models In all experiments, we utilize the DeepSurv method (Katzman et al., 2018), as the base predictor to estimate the conditional quantiles d-- We also fit Random Forest classifiers to estimate the weights w, and indicator 3, for calibration methods. For more details, see Appendix
Data We test calibration methods six simulated settings designed mimic real-world clin- ical trials, as in Candès et al. (2023): Gui et al. (2024). Refer to Appendix E. for details.
Methods and performance metries To evaluate our approach, we compare our method to base model, applied without calibration, and to the CSD calibration method proposed by Qi et al. (2024). all simulation studies, we set the desired coverage level a to 90% and evaluate both the coverage rate and the average LPB over the test set.
Results The performance metrics for the synthetic experiments are presented in Figure 1 Fol- lowing the top panel in that figure, we can that the naive, focused, and fused calibration methods achieve valid coverage in all six settings. This stands in contrast to both the uncalibrated base model and the CSD methods which produce LPBs that do not obtain the desired coverage level in settings 1, 3, and 4. Notably, our fused method tends to be less conservative than the focused and na ve approaches, highlighting the advantage of the fused approach. Settings and 6 further stress the advantage of the fused method, where one can that (i) the focused approach is less conservative than the naive approach, suggesting a violation Equation and (ii) the fused method is more powerful than the focused and naive methods.
Turning to LPB comparisons, the bottom panel of Figure Iillustrates that the fused method gener- ates the most informative lower bounds out of all the methods with finite sample guarantees (higher better). This statistical gain is in line with the tighter coverage rate of the fused method. The uncalibrated baseline and CSD calibration methods produce the highest LPBs, but these I.Bs often invalid.
Additional synthetic experiments, demonstrating the robustness of method to varying censor- ship rates, are given in Appendix G Experiments demonstrating the effect of the sample size on the performance of methods presented in Appendix H An ablation study, showing the bustness of our methods to changes in key hyperparameters of the underlying predictive models, is detailed in Appendix[I]
4.2 APPLICATION TO REAL DATA
We demonstrate the practical utility of our methods by applying them six real-world datasets: The Northern Alberta Cancer Dataset (NACD) (Haider et al. 2020). Rotterdam German Breast Cancer Study Group (GBSG), Molecular Taxonomy of Breast Cancer International Consortium
8

---

Published as a conference paper at ICLR 2025
Setting
Setting
Setting
Setting
Setting
Setting
1.00
Coverage
0.90
0.80
Focused
0.75
Fused
Naive
0.70
CSD
Uncalibrated
Relative LPB
Figure 1: Performance of the different calibration methods for each of the six simulated settings. Top:
empirical coverage rate, with red dashed line indicating the nominal 90% level. Bottom: relative LPB, defined as the LPB obtained by each method divided by the naive method's median LPB. A higher relative LPB is better. The reported performance metrics are evaluated on 50 inde- pendent trials, each consisting of newly sampled train, validation, calibration, and test sets of sizes 600, 200, 1000, and 200, respectively. The notably large proportion of calibration samples is inten- tionally chosen to ensure that the CSD method has sufficient data, as advised by Qi et al. (2024).
(METABRIC). Study to Understand Prognoses Preferences Outcomes and Risks of Treatment (SUP-
PORT) (vamme et al. 2019 Katzman et al. 2018), user churn dataset (Fotso et al. 2019
present,, as well The Cancer Genome Atlas Breast Invasive Carcinoma (TCGA-BRCA) multi-
modal dataset collection (Tomezak et al, 2015). The datasets' description are detailed Appendix
and the TCGA-BRCA dataset's s acquisition and preprocessing detailed in Appendix Cen- sorship rates for all datasets are detailed in T
In contrast with the synthetic experiments, here it is not possible rigorously verify that the desired coverage level is achieved in practice-we do not have access to ground truth survival times. Yet, our theory indicates that the na ve method should attain conservative coverage level under minimal assumptions. Further, under well-approximated estimation of the weights or quantiles, the proposed focused and fused methods should produce less conservative and approximately valid LPBs.
To evaluate how the various methods perform in practice, we present in Figure 2 their correspond- ing average LPB values. Among the methods that provide umite-sample guarantees, the fused approach achieves LPB values comparable to the stronger -enngrrrerroomingg method between the nair haive and focused. Furthermore, the fused method outperforms both the na ve and focused meth- ods, albeit to limited extent, the METABRIC dataset; this is reminiscent of settings 2 and 6 in the synthetic experiments. Finally, we note that while there are datasets for which the uncalibrated base model and the CSD approach yield higher LPB values, synthetic experiments highlighted that there are cases where these methods fail to achieve the desired coverage in practice.
DISCUSSION
We introduced a flexible uncertainty quantification framework with finite-sample LPB coverage guarantees for general right-censored survival analysis data. The validity of the proposed calibration methods relies on the conditionally independent censoring and i.idd assumptions, which may not hold in all real-world cases. It is therefore of great importance to study the effect of violations of these assumptions both to understand the practical limitations of our methods and to further enhance their robustness. In particular, the work by Oliveira al. (2024) may serve valuable starting

---

Published as a conference paper at ICLR 2025
tcga
support
metabric
churn
nacd
gbsg
40
13.0
12.5
35
Focused
LPB (months)
40
12.0
Fused
Naive
30
30
11.5
CSD
000
11.0
Unca llbrated
7
20
25
10.5
10
3 10.0
20
Figure 2: Real-world data experiments: comparison of the average LPB obtained by different cal- ibration methods. For each dataset, base models are trained once using randomly selected training and validation sets, comprising 60% and 15% of the dataset, respectively. The remaining dataset (25%) forms the calibration (10%) and test (15%) sets. LPBs are calculated over 100 random cali- bration/test splits.
point to move beyond the i.i.d. assumption; and the work by Feldman & Romano (2024) can be further utilized to build robustness to imperfect training data.
We should also emphasize that our methods may fail to generate valid LPBs in cases where the weights ti, (x) are poorly estimated. Stressing this point. the estimation of w, (x) can be particularly challenging when the censoring rate high, resulting in imbalanced training data. A related issue handling extreme ui (I),, where one possible remedy is bound the estimate of TP(c
X) in manner similar to Gui et al. (2024). It would be of great interest to offer more principled solutions the above challenges, possibly by following the ideas Li al. (2018) Cheng al. (0022 Elze
et al. (2017).
ACKNOWLEDGMENTS
This research partially supported by the Israel Science Foundation (ISF grant 3864/21), the Israel Cancer Research Fund (ICRF grant 1281495), and the Zimin Insititute for Al solutions in Healthcare. Y. R., S. F,, and H. D. were supported by the Israel Science Foundation (ISF grant 729/21). Y. R., S. F., and H. D. also thank the European Union (ERC, SafetyBounds, 101 163414) for providing additional support. Views and opinions expressed are however those of the authors only and do not necessarily reflect those of the European Union or the European Research Council Executive Agency. Neither the European Union nor the granting authority can be held responsible for them. Y. R. thanks the Career Advancement Fellowship, Technion, for providing additional research support.
REFERENCES
Tanvir Ahmad, Assia Munir, Sajjad Haider Bhatti, Muhammad Aftab, and Muhammad Ali Raza. Survival analysis of heart failure patients: A case study. PLOS ONE, 12(7)xe0111001 2017. doi: 10.1371/jjurnal.pom pon..0181000.
Anastasios N Angelopoulos, Stephen Bates, Emmanuel Candès, Michael Jordan, and Lihua Lei. Learn then test: Calibrating predictive algorithms to achieve risk control. arXiv preprint arXiv:2110.01052, 2021.
Stephen Bates, Anastasios Angelopoulos, Lihua Lei, Jitendra Malik, and Michael Jordan. Distribution-Free Disk-Controlling Prediction Sets. Journal of the ACM (JACM), 68(6):1-34, 2021. doi: 10. 10.1 145/3478535
10

---

Published as a conference paper at ICLR 2025
Katherine S Button, John PA loannidis, Claire Mokrysz, Brian A Nosek, Jonathan Flint, Emma SJ Robinson, and Marcus R Munafö. Power failure: why small sample size undermines the reliability of neuroscience. Nature reviews neuroscience, (M5):365-376, 2013. doi: 10.1038/mm3375.
Emmanuel Candès, Lihua Lei, and Zhimei Ren. Conformalized survival analysis. Journal of the Royal Statistical Society Series B: Statistical Methodology, 85(1):24 2023. doi: 10.1093/
jrsssb/qkac004.
A Chandrashekar, S Ramakrishnan, and D Rangarajan. Survival analysis of patients maintenance hemodialysis. Indian journal of nephrology, 24(4):206-213, 2014. doi: 00110000971-40666 132985.
Chao Cheng, Fan Li, Laine E. Thomas, and Fan (Frank) Li. Addressing extreme propensity scores in estimating counterfactual survival functions via the overlap weights. American Journal of Epidemiology, 191(6): 140-1151, 03 2022. doi: 10.1093/aje/kwacc44..
Victor Chernozhukov, Kaspar Wüthrich, and Yinchu Zhu. Distributional conformal prediction. Pro- ceedings of the National Academy of Sciences, 18848)):2107774111, 18, 2021. doi: 10. p073/as..... 2107794118.
Kristin G Cloyes, Bob Wong, Seth Latimer, and Jose Abarca. Time prison return for offend- ers with serious mental illness released from prison: A survival analysis. Criminal Justice and Behavior. 37(2):175-187, 2010. doi: 10.1177003334489554577
Stephen Cole and Michael G Hudgens. Survival analysis in infectious disease research: describing events in time. AIDS, 24(16)):423-2431 2010. doi: (0.1097/QAD0000) 3e32833dd0cc.
David R. Cox. Regression Models and Life-Tables, pp. 527-541. Springer New York, New York, NY, 1992. ISBN 11781-46112-488--- doi: 10.1007/978- 1-4612-4380-9_37.
Christina Curtis, Sohrab P Shah, Suet-Feung Chin, Gulisa Turashvili, Osear M Rueda, Mark Dun- ning, Doug Speed, Andy G Lynch, Shamith Samarajiwa, Yinyin Yuan, et al. The genomic and transcriptomic architecture of 2,000 breast tumours reveals novel subgroups. Nature, 486(7403): 346-352, 2012. doi: 10. 1038/nature 10983.
Christine Desmedt, Benjamin Haibe-Kains,
Kains, Pratyaksha Wirapati, Mare Buyse, Denis Larsimont, Gi- anluca Bontempi, Mauro Delorenzi, Martine Piccart, and Christos Sotiriou. Biological Processes Associated with Breast Cancer Clinical Outcome Depend on the Molecular Subtypes. Clinical Cancer Research, 14(16):5158-511 2008 doi: 10. 1.55//177.04332CCC---7777
Markus C. Elze, John Gregson, Usman Baber, Elizabeth Williamson, Samantha Sartori, Roxana Mehran, Melissa Nichols, Gregg W. Stone, and Stuart J. Pocock. Comparison of propensity score methods and covariate adjustment. Journal of the American College of Cardiology, 69(3):345- 357, 2017. doi: j0.10066iaacc.221...0.0 10.060.
Timothy M Errington, Maya Mathur, Courtney K Soderberg, Alexandria Denis, Nicole Perfito, Eliz- abeth Iorns, and Brian A Nosek. Investigating the replicability of preclinical cancer biology. Elife, 10:e71601, 2021. doi: 10.7554/eLife.llee 16001.
Shai Feldman and Yaniv Romano. Robust conformal prediction using privileged information. arXiv preprint arXiv:24006.0 2000000 2024.
John Foekens, Harry A Peters, Maxime Look, Henk Portengen, Manfred Schmitt, Michael Kramer, Nils Brünner, Fritz Jänicke, Marion E Meijer-van Gelder, Sonja C Henzen-Logmans, et al. The urokinase system of plasminogen activation and prognosis 2780 breast cancer pa- tients. Cancer research, 60(3):636 (63)633,,33,,3 2000.
Matteo Fontana, Luca Boeri, Andrea Gallioli, Elisa De Lorenzis, Franco Palmisano, Stefano Paolo Zanetti, Gianluca Sampogna, Giancarlo Albo, Fabrizio Longo, Franco Gadda, et al. Prevalence and predictors of being lost to follow-up after transurethral resection of the prostate. Scientific reports, 8(1)x6406, 2018. doi: 10. (0.108//41158-00 218-2286--2
Stephane Fotso et al. PySurvival: Open source package for survival analysis modeling, 2019- present.
11

---

Published as a conference paper at ICLR 2025
Amitoj Gill, Rohit Gosain, Shruti Bhandari, Rahul Gosain, Gurkirat Gill, Joseph Abraham, and Kenneth Miller. "Lost to Follow-up" Among Adult Cancer Survivors. American Journal of Clinical Oncology, 41(10):1024-1022 2018. doi: 10.
Yu Gui, Rohan Hore, Zhimei Ren, and Rina Foygel Barber. Conformalized survival analysis with adaptive cut-offs. Biometrika, 111(2)459-477, 2024. doi: 10.1099/biomeet/ssad077
Humza Haider, Bret Hoehn, Sarah Davis, and Russell Greiner. Effective ways to build and evaluate individual survival distributions. Journal of Machine Learning Research (JMLR), 21(1). 2020. doi: 10.555/34557163445580)
John D Kalbfleisch and Ross L Prentice, The statistical analysis of failure time data. John Wiley & Sons, 2011. doi: 10.1002/9781118032985
Jared L. Katzman, Uri Shaham, Alexander Cloninger, Jonathan Bates, Tingting Jiang, and Yu- val Kluger. DeepSurv: personalized treatment recommender system using a cox proportional hazards deep neural network. BMC Medical Research Methodology, 18(1):24, 2018. doi:
10.1186//12974--188-44933
Mert Ketenci, Shreyas Bhave, Noemie Elhadad, and Adler Perotte. Maximum Likelihood Estimation of Flexible Survival Densities with Importance Sampling. In Machine Learning for Healthcare Conference, pp. 360-380. PMLR, 2023.
John P Klein. Survival Analysis, Techniques for Censored and Truncated Data, pp. 70. Springer Science Business Media, Inc, second edition, 2003.
William A Knaus, Frank E Harrell, Joanne Lynn, Lee Goldman, Russell Phillips, Al- fred Connors, Neal V Dawson, William J Fulkerson, Robert Califf, Norman Desbi- ens, al. The SUPPORT prognostic model: Objective estimates of survival for seriously ill hospitalized adults. Annals of internal medicine, 122(3):191-203, 1995. doi: 10.7326/ 0003-4819. 122-3- 199502010-00001..
Håvard Kvamme, Ørnulf Borgan, and Ida Scheel. Time-to-event prediction with neural networks and cox regression. Journal of Machine Learning Research, 20(129):1-30 2019.
Changhee Lee, William Zame, Jinsung Yoon, and Mihaela van der Schaar. DeepHit: A Deep Learn- ing Approach to Survival Analysis With Competing Risks. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1), 2018. doi: 10.1609/a 132...
Fan Li, Laine E Thomas, and Fan Li. Addressing extreme propensity scores via the overlap weights. American Journal of Epidemiology. 188(1):250-257, 2018. doi: 10. w0.1
Zhanshan Ma and Axel W. Krings. Survival analysis approach to reliability, survivability and prog- nostics and health management (phm). In 2008 IEEE Aerospace Conference, pp. 1-20, 2008. doi: 00.1009/ARR..2008.5550 4526634.
Carlos García Meixide, Michael R. Kosorok, and Marcos Matabuena. Uncertainty quantification for intervals. arXiv preprint arXiv:2408. 16381, 2024.
Sara Monfared, Anna Martin, Don Selzer, and Annabelle Butler. Travel distance reduces follow- up compliance but has no effect on long-term weight loss success in bariatric patients. Surgical endoscopy, 10.1579--583, 2021. doi: 10.10077.004444000-075555-55
Chirag Nagpal, Xinyu Li, and Artur Dubrawski. Deep Survival Machines: Fully Parametric Survival Regression and Representation Learning for Censored Data with Competing Risks. IEEE Journal of Biomedical and Health Informatics, 25(8):3163-3177 2021a. doi: 0.1180000401106..199 10485206.
Chirag Nagpal, Steve Yadlowsky, Negar Rostamzadeh, and Katherine Heller. Deep Cox Mixtures for Survival Regression. In Machine Learning for Healthcare Conference, pp. 674-708. PMLR,
2021b.
12

---

Published as a conference paper at ICLR 2025
Constanza L. Andaur Navarro, Johanna Aag Damen, Toshihiko Takada, Steven W. J. Nijman, Paula Dhiman, Jie Ma, Gary Stephen Collins, Ram Chandra Bajpai, Richard D. Riley, Karel G. M. Moons, and Lotty Hooft. Risk of bias in studies on prediction models developed using supervised machine learning techniques:
systematic review. The BMJ, 375, 2021. doi: 10. 1 336/mjj.n2288
Brian A Nosek, Tom Hardwicke, Hannah Moshontz, Aurélien Allard, Katherine S Corker, Anna Dreber, Fiona Fidler, Joe Hilgard, Melissa Kline Struhl, Michèle B Nuijten, al. Replicability, Robustness, and Reproducibility in Psychological Science. Annual review of psychology, 73(1):
719-748, 2022. doi: 10. 10.1144/annurev-ppst 440ammree-psycca-0222111111111 11417.
Eveline Nüesch, Sven Trelle, Stephan Reichenbach, Anne WS Rutjes, Beatrice Tschannen, Dou- glas G Altman, Matthias Egger, and Peter Jüni. Small study effects in meta-analyses of os- teoarthritis trials: meta-epidemiological study. Bmj, 341, 2010. doi: 10.1 1o.1m366/...33555..
Ziad Obermeyer, Brian W. Powers, Christine Vogeli, and Sendhil Mullainathan. Dissecting racial bias in an algorithm used to manage the health of populations. Science, 366:447 -453, 2019. doi: 10.1 20.1126/science.aant a2232222.
Roberto I. Oliveira, Paulo Orenstein, Thiago Ramos, and João Vitor Romano. Split Conformal
Prediction and Non-Exchangeable Data. Journal of Machine Learning Research, 25(225);1-38, 2024.
Sangdon Park, Osbert Bastani, Nikolai Matni, and Insup Lee. PAC Confidence Sets for Deep Neural Networks via Calibrated Prediction. arXiv preprint wrXiv:2001000100 2019.
Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. Scikit-leam: Machine learning in python. Journal of machine learning research, 12(Oct):2825-22)) 2830, 2011.
Shi-Ang Qi, Yakun Yu, and Russell Greiner. Conformalized Survival Distributions: A Generic Post- Process Increase Calibration. In Proceedings of the 41st International Conference on Machine Learning. volume 235 of Proceedings of Machine Learning Research, pp. 41303 41339, PMLR, 21-27 Jul 2024.
Jing Qin, Jin Piao, Jing Ning, and Yu Shen, Conformal predictive intervals in survival analysis: re-sampling approach. arXiv preprint Stti::200006553333 2024.
Yaniv Romano, Evan Patterson, and Emmanuel Candes. Conformalized Quantile Regression. Ad- vances neural information processing systems, 32, 2019.
M Schumacher, Bastert, H Bojar, K Hübner, M Olschewski, W Sauerbrei, C Schmoor C Beyerle, RL Neumann, and HF Rauschecker. Randomized 2 x 2 trial evaluating hormonal treatment and the duration of chemotherapy node-positive breast cancer patients. german breast cancer study group. Journal of Clinical Oncology, 12(10))2086-200 2093, 1994. doi: 10.1200/JCO.1994.10 2086.
Steve Selvin. Survival analysis for epidemiologic and medical research. Cambridge University Press. 2008. doi: 10.111 154111/1151--02220000000111 012447..x...
Ryan Tibshirani, Rina Foygel Barber, Emmanuel Candes, and Aaditya Ramdas. Conformal Predic- tion Under Covariate Shift. In Advances in Neural Information Processing Systems, volume 32.
Curran Associates, Inc., 2019.
Katarzyna Tomezak, Patrycja Czerwińssa, and Maciej Wiznerowiez. The Cancer Genome Atlas (TCGA): an immeasurable source of knowledge. Contemporary Oncology (Poznan, Poland), 19 (1A):A68 77,6 2015. ISSN 1428-2526. doi: 10.51 1/.111//00.211.471336
Vladimir Vovk. Conditional validity of inductive conformal predictors. In Steven C. H. Hoi and Wray Buntine (eds.), Proceedings of the Asian Conference on Machine Learning, volume 25 of Proceedings of Machine Learning Research, pp. 475-490, Singapore Management University, Singapore, 04-06 Nov 2012. PMLR. doi: 10. 10.1007//100994-13-5553
Vladimir Vovk, Alexander Gammerman, and Glenn Shafer. Algorithmic Learning in a Random World, volume 29. Springer, 2005. doi: 10.1007/1106115
13

---

Published as a conference paper at ICLR 2025
Lee-Jen Wei. The accelerated failure time model: a useful alternative to the Cox regression model in survival analysis. Statistics in medicine, 11(14-15)187111187, 1992. doi: 10.1002/sim.
4780111409.
Brooke Wilson, Michelle Nadler, Alexandra Desnoyers, and Eitan Amir. Quantifying Withdrawal of Consent, Loss to Follow-Up. Early Drug Discontinuation, and Censoring in Oncology Trials. Journal of the National Comprehensive Cancer Network JNCCN, 19:1-8, 09 2021. doi: 10.
4004//jccnn2222..000
Hanwen Xu, Naoto Usuyama. Jaspreet Bagga, Sheng Zhang, Rajesh Rao, Tristan Naumann, Cliff Wong. Zelalem Gero, Javier González, Yu Gu, Yanbo Xu, Mu Wei, Wenhui Wang. Shuming Ma, Furu Wei, Jianwei Yang, Chunyuan Li, Jianfeng Gao, Jaylen Rosemon, Tucker Bower, Soohee Lee, Roshanthi Weerasinghe, Bill J. Wright, Ari Robicsek, Brian Piening, Carlo Bifulco, Sheng Wang, and Hoifung Poon. whole-slide foundation model for digital pathology from real-world data. Nature, 2024. doi: 10. 10.1038/s41586 00533/41586--2--0000 024-07441-10
Brenda Zhou, Travis C Mitchell, Alexander M Rusakevich, David M Brown, and Charles C Wykoff. Noncompliance in Prospective Retina Clinical Trials: Analysis of Factors Predicting Loss to Follow-up. American journal of ophthalmology, 210:86-96, 2020. doi: 10.1016/j.ajo.201 10. 012.
A NAIVE METHOD'S S ALGORITHM AND PROOF OF VALIDITY
A formal description of the algorithm for the naive calibration method given by Algorithm D
Algorithm Conformaliied survival analysis for general right-censored data: naive calibration
desired miscoverage level o;; calibration data Deal {(X,, Tú, Vez estimated quantiles T X, ()) rE((.1]
Input:
Procedure:
1: for in grid over [0, 1] do
2:
Compute the fused miscoverage estimator:
@ =
Σ I[T: < (x,))
ITal
TETal
3: end for
4: Calibrate the hyperparameter T:
Anaiee = =sup fT [0, 1] sup ánaive (r') S a}
-'ST
5: Return: The calibrated LPB: Ê(.) = L(: fnaive) = &eren (.)
Theorem A.1 (PAC-type validity of the naive LPB). Fix any a, 8 E (0, 1). Assume that (x(x) is non non-deccgg т. With probability at least 1 over the draw D. the LPB produced by the maive ve calibration method from satisfies
TP
T 2 iXx; naive
) )DD] 2 An.
where A, = V
In
Zaal
Proof. For the proof we denote the smallest estimated quantile bound T' with probability least 8, conditional on the training data, as
T(3) sup (r' [0, :P PT (x(()) D.)) B).
14

---

Published as a conference paper at ICLR 2025
As in the proof for theorem 3 in the work by Gui al. (2024). proving that - v Faavve S <a + An) Dul is sufficient for proving the theorem. Analyzing the naive miscover- age estimator, we get that
P Mmive (T (a + An) + a Drr
T: < dr(atta)+e (Xi) Teail a D.
P
(I{
fr(a+ -a 50 s0
LieT
< E texp
Σ
(a- < araatantt (xi)))) Dir
(10)
where the first transition is by the definition of dnaive, and the last by the Markov inequality. Further conditioning on (Xi) since Pr(a+An)++ (X,) -I I < drea+An)+e (X.) is 2-sub-gaussian, Hoeffding's 's lemma we get that
E eex
Σ
(pr(ots.+t(X,)) (X,) I driatsntee (x.]) (Xi)iEz Drr
VEI
12
I exp
Pr(ots.)t((X)) - I T; < Ar(atstte (xi)
(Xi) Du
+ Σ 12
TETa
t² Ical
Pria+Aa]+e (Xi) -P { < (ratt.))e(()))
(Xi) Dr
exp
iE
Since T; < T, by definition, we have that
Pr(a+An)+e (Xi) - PP {PTT:: < (ro-Aw)HeXx)) so.
and so by the law of total expectation, we get that
Σ (Priatsa)teXx)) (Xi) -If < Cr(a+ante (xi))) Dr
exp
[Taal t2
S exp
Combining with 10 we get that
P Unaavv (T (a + An) + +) S <al Dul
Teal t²
(a Pr--otataatee (x,)) [pu]
< exp
E
texp
Σ IET
By the definition of T (a + An), we have that
P [ < Deee (X) Duj S An,
15

---

Published as a conference paper at ICLR 2025
and so,
PP mnive (T (ax + Da) + e) S a Drr
Tcal (2
S cxp
E exp
Σ
(P (r(o+aa)+ (Xi) Dul
VET
-n Pr(att, to (xi))) Du
Teal t
tAn Icaal) )
< exp
where the last inequality follows from the 1 --uu-gauusiaaii of Pr(a+da)+e (X:) - PP TT < dr(atAnte (X.) Dr!
Taking
4An:
we get that
TP [Onaive (T (a + An) + S a Dur S exp (-2 [Tall (-al
(An)²).
Isul-
E
[x()) [x(x
vt
Dr
-n
Since
(-2|Ical) (An)²)
get that
x2474+77
P [anaiie (T (a + A,) + S a Del S 6,
where the last inequality follows from the definition of An. Taking the limit O, we get that 1-6 (ónnaive (r (ax + An)) > [Dr] S P [fnive < (ar + A.. [rr],
as needed.
B FURTHER DETAILS REGARDING THE FOCUSED CALIBRATION METHOD
B.1
POCUSED CALIBRATION LEMMA
Lemma B.1. Suppose that æ E N. : X +t. Rt ((X) X x) 0 and under
Assumtion T.I
P(c= 2)) < xx
where the probability is taken over Pix,C.T)
Proof. Suppose E X. All derivations in this proof are conditional on X x, and denote P.(.) = P(- X = -) and = g(X).
We begin by developing P, < < | <):
(T <
= =
<c)
= - +) P.0<CT<<<<<000000 p.(<g)
P,(T<4..'<.)
P2 4)
<9)
P.(<<))P,(C<9)
<TT<CTT<C<<))
P,(C2
(T(T<q))
P2(C2
P.( <.
16

---

Published as a conference paper at ICLR 2025
where the first transition is by the law of total probability and the third is by Assumption L.
We now develop Pz (T < C):
= (CTCC)))..<<CCC0HHH<<<CCCCC
By subtraction the two quantities, get
(T <[ -
<))
= =2(CC' g) + < - -3000000000000000000000000
<x))<
= =Pa(C - PP < < C,C2 P, -
(x(T<4)
= =P,(T CCC21) + P <)) - P,T<<CCC))
and so, multiplying by (T < 7) we get that
(T C,T <g) (T <
That is,
PA < C,T < < P < ()P <g)
B.2 FOCUSED CALIBRATION INFORMATIVENESS CONDITION
In this section, we prove Proposition 3.1
Proof. By looking
have that
afocus (T) E [T (x) -(x] = E [F(T < 4X)) |x,e xx,== - 1)
Analogously, we can re- re-write Ornaive (T)
Onaive (T) P(T <x) PPF (P()) xx]
Therefore, Arocuus (T) Onaive (T) when
PPT < d (X) X = =x
=xz,c=1)<<(F P(T (x)) Vo E X.
Which, by the law of total probability, can be stated as
P( < (x) X x, =x 1)< < 4x) x= =x Var X
B.3 FORMAL DESCRIPTION OF THE FOCUSED CALIBRATION ALGORITHM
A formal description of the focused calibration algorithm is given in Algorithm
Note that the thcorems for the validity of this algorithm in Section 3.3 are valid under the definition VT E [0, 1], W, wh U.
C PSEUDO-CODE DESCRIPTION OF THE FUSED CALIBRATION METHOD
A formal description of the fused calibration method is presented in Algorithm 3
17

---

Published as a conference paper at ICLR 2025
Algorithm 2 Conformalized survival analysis for general right-censsoed data: focused calibration
Input:
estiattteee desired miscoverage level calibration data Deal (X,, T3 e4) Nez estimated quantiles Tf x, ( weights ((r: approximating 1/P(e
X = x).
Procedure:
for grid over [0, do 2: Compute the fused miscoverage estimator:
asfcus (T) = Eiera w(x;)) TTe; = 113 -[TT (x:))
Eiera w(Xi) Ifee = 1}
3: end for
4: Calibrate the hyperparameter T:
(omeus(T))) foous = sup { E [0, 1]: sup a (T)) S a}
+'ST
5: Return: The calibrated LPB: Ż(-) == (.: frocus)
@f (-)
Algorithm 3 Conformalized survival analysis for general right-censored data: fused calibration
desired miscoverage level calibration data Deat {Xx,, T, ei)iez estimated quantiles of T X, {+) rE weights w; (z) approximating 1/P(s, = 1 X z), with the
Input:
corresponding selection rule Sr X' x {0, 1} + {0, 1} defined in Equation Procedure:
for grid over [0, d
2: Compute the fused miscoverage estimator:
Örfuxed (T) EiEr,, ( IIS, = 1] I[T; (x()))
Eiera (X,) [[G
3: end for
4: Calibrate the hyperparameter T:
Trused sup fT [0, 1] sup afused (T') S a}
T'ST
5: Return: The calibrated LPB: L(-) = L(: Frused) @oreal (.)
D PROOFS FOR FUSED CALIBRATION DOUBLE ROBUSTNESS
D. CALIBRATION THEORETICAL DERIVATION AND LEMMA
The derivation for the fused calibration method very similar that of focused calibration, with (xXX e) replacing e:
a(r)= = P(T < (X))) = E X X)]
(11)
E [P(T < (X) (XX w(x)
< E (I (
(12)
E [[ =I w(X)
(13)
E [[ (((==<(X)))-I(((X..... = =1] w(x) = ()occs((T)
OOsserve that the values of that lead change in â(T) correspond to those for which 7, (x()). for
i Teal- Consequently, the grid defined over the values that satisfy this equality, with 0 included.
18

---

Published as a conference paper at ICLR 2025
Where this time w(x)
1/P e)
= =) min(1/P(e = = x)) s, (X));
steps Equation 11 and Equation 13 hold by the tower property, and step Equation 12
holds by the
following lemma-
Lemma D.1. Let X {0.1), and & X (0.1) {0.1} defined in Equation 9 Then Vx E x.:: x R
Under Assumtion
1.1
(X, 1, < x()
P(G,(X -(G()) = 1 1|X = = < (X) =x] 0.
Proof. We'll prove this lemma by separating the cases into the possible values of (X). which is constant given X = I. Suppose that I E X' satisfies 8, (x) = 0. We have that
(X c) = xxx)x =]
= Pl(e = 1,7 < x() | = x]
PPle = 1,7 1 XX x],,
and
= e()
< =
= P/c = x - < x (
And so, by Lemma B.1 we have that
(X, 1,T
= - (X)
=
= Ple P| -1
For x X with 8, (X) = 1, we have that
ex) 1,T q())
= PPT < ( x),
And
xPP q()) |X
=P(T < q(X) | = x].
And so, since by definition / - T, we have that Yo E X::
((,, = 1,T < x X x PX,, (X, 1|X x())x==
= =P <x(x) - (X)) 220 0.
D.2 FUSED CALIBRATION WITH APPROXIMATELY ACCURATE WEIGHTS
In this section we prove Theorem 3.1 relying on the proof of (Gui et al. 2024 Theorem 3).
Proof. Here, we provide a single proof for the validity of both the focused and fused, using the appropriate notations for each method. For the focused method, consider dc(-) = OHocuse (.). f : Frocused. and (X,, ex) = For the fused method, we denote â(-) == å'usod f == Frused and (A (X,, e4) (X,,
or e, 1}. Importantly, in both cases, the weights are defined as
w- (x) = (XP((((Xe) = X = x). We remark that all our claims hold for each of the two choices of these terms.
We define the error term by:
+ + max (1, (1, 22 1)
52
(X)
A : sup
E
1
Dr + \
log
(X)TX
0.4 0.4
19

---

Published as a conference paper at ICLR 2025
Recall that the oracle quantity is formulated as:
T(a A) sup {> :P < (x( D2) I a 4)
The outline of this proof builds the proof of (Gui al. 2024 Theorem 3). First, if - < P s
T(a A) Dr), then the event {f - (a + A] holds with probability at least 1 - 6. Therefore:
P(T 2 f+ [D)
PT 2 (r(a-a) D)
a A.
Above, the first inequality is due to the monotonicity of dr and the second one follows from the left-continuity of P(T 2 q. (X) D) in In what follows, we focus showing 1 - 6 s IP s
T(a A) Dr). Suppose that s > For simplicity, we denote À == T(a + A) + E, Following the definition of (T),, we get:
((á(T((a + +))
D())
-P(((A) 3 a I
Eier, tix (XX,, < (x)))
P
S a Drr
EiEra tix (xX,,,)
Σ
ii (X)) < <x,)) a Σ ii (Xx) (xx,, (s) De
(14)
ETTal
EETal
P
Σ x (X.)
(x et) [ xx(x))) a] < <0/bu)
VET
=P
Σ Te
Sn (X,) (A (ax [a- < (x(x)))] 2 p
Next, we apply Markov's inequality for any t > 0, and have:
(x et) (a-1ạt; < (A(x))1) (p.)
E exp
t
iEZal
(X)) - (x.)) - -1(7 < <a(x.)))) (p..)
= E
exp
/
Σ Φi (X,,,
iEI.l
(15)
Above pA px(x)) = PPT < (x(X) x=x,
x. Dr)). We now conditioning on (Xi, ei)ie
E
exp
Σ
ii (X (x(Xx, es) [Pa(X.) -IZ 1 <xxxx))))))) (m. (Xi, ei)ieza
(a)
t2 Σ EiEz, (200 (X)))²
exp
AE
Σ Φ> (Xi) (x (t) [px(X)) - I[π. <xxxx,)]
Du. (X, (X,, ei)izz.
8
ViET..
(b)
exp
HE
Σ き (X
(a(XX) [px(x.) 1[ū: < (x(x),)] Dr (X eilezz
+
Ieal Ieaul 11222
2
Icau kEul
((x,,)))(T Σ ii (Xi) (x)) - G(x, TTT < [(xx)]] Der (Xe, (eliei
< Eexp
II
+
VEIT
- (XX,, < (.NNcapp((k--ccc...c.T<<<<<))<< Drs Pr. 1)
[Taa 1292
12
Σ のx (X:)
S exp
2
(16)
20

---

Published as a conference paper at ICLR 2025
where step (a) uses Hoeffding's inequality; step (b) follows from the boundness of ú;; step (c) is due to the independence assumption between the samples. We now turn to develop the expectation inside the sum.
E
Dr, X1,6;
=R (X (3) Dr, X1 B Pr.. X,, - (x (x) < (x())) Drr Xi,es
=P (X (1) = Dr X, 6, E px(X,) Pu. X,,e]] Sa(X, [= 1, < <x((x)) Du. Xci
=P[()(X, 6i) = Pr, B (X,, X,, e,] -P (x(Xx,) -P(ss-)) = 1, < xx(i)) Dr. =)
=P, = Dr, X,, 63 P(T: < (x.)) Drr Xi,es)
e Dr, X ei)
[; < (x(x)) Dro Xi;
-P(A e,) = 1 X,, e,l
P(T, < (x,,)) Du. X2, P [i < (x(x,) Du. Xx,
00
The last inequality holds since 7, S T,. By plugging this in we get: 16 I cxp (Zall(²3)) We now condition 15 on (X,, eiee
194, < (a(x))1))
E
exp
úx (X,) (xX, px(())
Dur (Xi, ci)ieza
iETa
E
exp
t
Σ Ιn (X+) (X.)) (Dx(X.))
(px(x)))) - 14T, < (x())))]
Σ
Ιi (xX,, p2(X.31) Dan (X ei)EEt))
+
iELal
(a (xxxxx))))))x(((() - IiT < ((xx))
E
exp
t Σ
أu (Xi)
iEInl
Σ Ti (a)
Dus (X,, eslEZaa
exp
TT
ex) (p) - T97. - (acx.))) Drr (Xi, eivets
E
exp
sEInal
E exp
Σ
ix (a
Per Pur (Xi
VEIN
TTaal
1t222
t Σ txx (X) -aaiaa))ppx))))))))))))))))))- pa(Kas)
<exp
E
cxp
Dw. (Xi, ei)iez.
2
VeT
By the law total expectation, we get:
12
Σ Φx (X)) (
Du (17)
exp
E
exp
TE
21

---

Published as a conference paper at ICLR 2025
We now condition (Xi and by the sub- gaussianity of <A (X,, cs) -x(x)) -1
we get:
E
cxp
Σ Φx
(Xx (Sx(X,, (xx - Dur (xitieie
TET.
12
Σ のx (x,) [a (a-a((x))
Px())))2
S exp
8
iE
A Teal
exp
We plug the above in 17 and bound it as follows:
5 T 12
xb (X;) (x,)),)
(..)
Ii S exp
E
exp
1
aa
px(X
(18)
8
iEZna
By combining all of the above, and by following the derivations in the proof of (Gui et al. 2024
Theorem 3) we get
5 T| It
(x)
mAa))
18
exp
(T) + 12 +5())))
+ Iswlt t
E
Du
(19)
wx(X)
By taking
ATTA
E
(x) (x))
TTA (Dr]]
ワス + 93 + -(A22
we have
T.4
TT E
(xx(X) (x(X)
- TT Drr 2
19
exp
5.
5 + 1 7())2
where the last inequality follows from the definition A. Therefore,
1-5 S P(á(T(a + A) + > D.)) P(t < 7(a + A) + | Dr))
By taking + and by the continuity of the probability measure, we obtain 1 8 P(f T(a A) s Dr)), which concludes the proof.
D.3 FUSED CALIBRATION WITH APPROXIMATELY ACCURATE QUANTILES
Following Gui al. 2024), instead of proving Theorem 3.2 directly, we provide more general theorem that implies Theorem 3.2 Our proof relies on the proof of (Gui et al.) 2024| Theorem 5).
Theorem D.2. Fix any 8, E (0, 1). Under the same conditions of Theorem 3.7 assume further that there exists constant > 0 such that:
1. For Px-almost all suDpetrde) we(x) and suppeeelaa)))))taa)))))) ()g())) S
5 for some constants 7, 7, 2
2. for any 1) r., for any 8 (a, a + r), and for Px-almost all x::
P(T (X) + X x) S 3 Bn,
P(T < go (X) 17 x = x) 3 Bn-
for some family of oracle functions (g.(-))=(oo1)) and some constant B > 0.
3. suPpe(a.ar]] SUP maax B3 1)l - (2(2)) + vy
log(1/8)
S r.
0.4 Ieal
Then with probability at least 1 over the draw of D, the LPB produced by either Algorithu 3or by Algorithm 2 satisfies that for Px-mlmost all x:
log log(1/5)
P(T 2 i(x) D.X =1x==11a-a
sup BEla,atri ZE
sup (2B g=(x)1} m//
0.4[Tcal
22

---

Published as a conference paper at ICLR 2025
the following proof guarantees the validity of both the focused and fused. For each method, we embrace its appropriate notations as follows. For the focused method, we use a(-) := Ocfocused f Tocused and <A (X, e,) eç. For the fused method, consider (()) = (ffwed (.), == Tuseed and (x(X,, e,) = T(8A (X,) = 1 or e = 1}. Notice that in both settings, the weights are formulated as Wy == 1/P((x
Proof. Similarly to Appendix D.2]
| X = æ). As in the proof in Appendix D.2] our claims hold for each choice of method and its corresponding terms.
We begin by defining the error terms:
log (1/0) T.a
3 = sup sup (16-(0)(1) - (x))) A = BE + Vy BE(a,atr| sE
(20)
For simplicity, we also denote À == 7(a + for some
> O. We further adopt the definition of
(a(x) = P(T I (X) X = z) from Gui et al. 2024). The rest of the proof follows from Guui
et al. 2024, Theorem 5), except for substituting f by q. I[C, 2 fx(x)) by ((X,, L[C; 2 (x(,) Ti} by [((((,, ex) (xx,) > T.J. and I[T; < fx())) by UiT: < (X.)). All derivations from the proof of (Gui et al. 2024| Theorem 5) apply to our setting except for the upper
bound of the following term, which we develop next.
(Xx)
(pa(X,) I[T. < (x(x))))
Pus |Pan Pu.(Xibieza) }
E exp
(X,,,)
12
(X,)²
wA(X,)
E [Px(X))) -I(X))-IIT 11T, < <x(x)))) Pu. 1xiteeeell
<exp
+t
2
wx(x,)2
wx(X,)
ToT
iE Tex
12
ex(x))²
(X1x))
[P(T: (x.))),,, P(T. < <xx()))
exp
+1
WA(X,)2
(X;))
t2 Ical??
< <exp
Above, the first inequality follows from Hoeffding's 's inequality and the last one holds since T; < Ti and due the upper bounds of the weights (-). We plug in this bound in the derivations of Gui et al. 2024) and get that
P(á(TTa +) > a Dw) Sexp { I
ITeal ((A - BE) T 1/
Therefore, take t By5z A BЕ) get
P(á(T(a + A) +) Drt) < exp 00.4
A
BE)²
2
2222
where the last inequality follows from the definition of A from Equation 20 Therefore, we get that with probability 1- over the draw of D: A)) > a). This implies T(a+ +e f, and thus:
P(r(a
As in Gui et al. 2024). we take = + 0, and by the continuity of the probability measure, we get that T( + A) + 7 holds with probability at least 1 - 8, which concludes the proof.
E DATASET DETAILS
E. SYNTHETIC DATASETS
In all six simulated datasets from Section 4.1 the covariates X are drawn from uniform distribu- tion, X O, 4), The base conditional survival time T Tasee follows a lognormal distribution,
as in Candès et al. (2023): Gui et al. (2024):
T husse
X exp(N(x(X),
23

---

Published as a conference paper at ICLR 2025
where (x) oX is the normal distribution with mean (xX) and standard deviation a (X) defined in Table [ for each setting, along with the dimension p of X. To simulate patients with random early mortality,
choose 5% of subjects, and assign them the parameters μ In 0.1 and 0.2 instead. For the remaining 95% event time is defined as 7 Thaee
The censorship time C X is designed mimic real-worke censorship scenarios commonly countered in clinical trials (Zhou et al. 2020 Wilson et al. 2021
End-of-trial censorship: All settings impose censorship at time 10.
Early censorship: For 12% of subjects with X x[0] [0, 0.5), very early censorship is applied at C 0.1, representing scenarios such as early loss to follow-up, treatment non- compliance, withdrawal of consent.
Parametric follow-up censorship:
The remaining subjects are censored based on the para- which simulates loss to follow-up
metric distribution Colllow-u described Table I over time.
Table summarizes the overall censorship rates across different settings, ranging 30%-71.6%.
Table I: Parameters specifying the distribution of survival time X and censoring time C|X in the six simulated data sets.
Setting
m(x)
(xx
Crollow
1
4
Vai
Exp(5)
2
4
(2 VT4) 11 2] + / 1222 2
Exp(10)
3
10
(2 (TT) 11111 2] + zΑ 11/1 $ 2}
1.5
Exp 0.01+
60
4
10
3- I[x1 > 2) 1.5xg 1//11 2]
1.5
lognormal
(2 2-21 1.5)
5
20
0.126(x: min 2244)
Exp
6
20
0.126(x1 + VI +
#2 2
Exp
Table 2: Censorship rates for each simulation setting.
Setting
2
3
5
6
Censorship Rate (%)
64.6%
40.5%
30.0%
71.6%
45.3%
52.6%
E.2 REAL-WORLD DATASETS
What follows are descriptions for the datasets used in the experiments of Section 4.2
The Northern Alberta Cancer Dataset (NACD), as presented by Haider et al. 2200)). includes data on 2,402 patients diagnosed with a range of cancers, such as lung, colorectal, head and neck, esophageal, stomach, and other types. After excluding patients with negative or zero survival times, dataset is reduced to 2,396 patients.
The Rotterdam & German Breast Cancer Study Group (GBSG) dataset combines information from two sources: the Rotterdam tumor bank Foekens et al. (2000) and the German Breast Cancer Study Group Schumacher et al. (1994). The Rotterdam dataset includes records for 1,546 patients with node-positive breast cancer, while the GBSG dataset provides complete data for 686 patients.
The Molecular Taxonomy of Breast Cancer International Consortium (METABRIC) dataset, de- tailed by Curtis et al. (2012), provides survival data for breast cancer patients. incorporates a wide array of features, including clinical traits, gene expression profiles, copy number variation (CNV) profiles, and single nucleotide polymorphism (SNP) genotypes. These features are derived from breast tumor samples collected during the METABRIC study.
24

---

Published as a conference paper at ICLR 2025
The Study to Understand Prognoses. Preferences, Outcomes, and Risks of Treatment (SUPPORT) dataset, as described by Knaus et al. (1995), includes data from 9.105 participants. It was designed to investigate survival outcomes and clinical decision-making processes for critically ill hospitalized patients.
The customer churn prediction dataset (Churn) is designed predict customer attrition (Fotso al. 2019 present). For analysis, individuals censored at time 0 are excluded.
The Cancer Genome Atlas Breast Invasive Careinoma (TCGA-BRCA) multi-modal dataset collec- tion (Tomezak et al., 2015) is also used, and includes various clinical features such as age, cancer grade, and tumor size, along with pathological biomarkers, genetic data, and visual biopsy samples. To effectively estimate the conditional survival times from such diverse features, we designed ded- icated preprocessing and training procedures for this data. These include the use of the GigaPath foundation model (Xu et al., 2024) to extract features from the visual biopsy samples, extraction of cancer grade from patients' medical reports, and more.
A detailed description of the entire processing pipeline can be found in Appendix F
Table summarizes the overall censorship rates across the different datasets.
Table 3: Censorship rates for each real-world dataset.
Setting
TCGA
SUPPORT
METABRIC
churn
NACD GBSG
Censorship Rate (%)
86.6%
32.0%
42.1%
53.4%
36.6%
43.2%
F EXPERIMENTAL SETUP
In all experiments, the dataset was split into four parts: 60% for training. 20% for calibration, 10% for validation (used for early stopping), and 10% for testing to evaluate performance. Synthetic data was generated through distribution simulations, as outlined in Table T while the process for acquiring the TCGA data is described in Section F3 The synthetic data generation function and the processed TCGA-BRCE dataset are available in the github repository. The TCGA-BRCA dataset was further normalized and imputed using the dataset pre-processing code by Ketenci et al. 2023) utilizing the pre-processing of Nagpal et al. (2021a) for data imputation and normalization.
In all experiments, we approximated the distribution of T X using the DeepSurv method Katzz man et al., 2018). implemented in the pycox package (Kvamme et al.] 2019), implemented using PyTorch MLP regressor with ReLU activation, early stopping (triggered after 5 epochs without improvement), and a training cycle of 1000 epochs. The Adam optimizer optimized the model with parameters Ir = 1e 3. 31 0.9 and f3 0.999, batch size of 256, dropout layers with rate of = 0. batch normalization layers, and varying configurations of hidden layers, detailed in Table These configurations were selected be similar to those found in the PyCox notebooks, with the real-world datasets getting a deeper model to account for their more complex and interconnected nature.
Additionally, we employed scikit-learn (Pedregosa et al. 2011) to train a Random Forrest Classifiers with max depths of 4 and 2, to estimate the weights w; and the indicator š, respectively. The higher max depth for estimating úi, is meant to make conditional calibration more accurate, promoting validity. In comparison, the lower max depth of 8, is there to reduce potential overfitting, that might lead to overly conservative estimators.
F.1 MACHINE SPECIFICATIONS
The hardware and OS used for the experiments are as follows.
CPU: AMD EPYC 7443 24-Core Processor
GPU: NVIDIA RTX A6000
OS: Ubuntu 20.04
25

---

Published as a conference paper at ICLR 2025
Table 4: Parameters specifying the number of hidden layers of the MLP regressor employed for the DeepSurv survival analysis model in the synthetic and TCGA settings.
Setting
Hidden layers
Synthetic settings
TCGA-BRCA
[32, 32
F.2 COMPUTATIONAL EFFICIENCY
Note that the values of r that cause shift in a(T) are those for which T; = gr (X,), for i E Teal- As result, to compute the miscoverage estimator one need only check these values and = 0. As such, the computational complexity of the calibration ((Taa)) and so the brunt computing is allocated towards the model training, both for the quantile and weight estimation.
F.3 TCGA ACQUISITION PREPROCESSING DETAILS
The TCGA-BRCA cohort is accessible through the GDC portal To create dataset suitable for DeepSurv, we exclude patients without clinically relevant Estrogen Receptor (ER) status or scanned biopsies. We then merge several tables and select the following clinically significant features for model training, imputing the Progesteron Receptor (PR) value with the ER value, and the rest of the features with the value value -1:
ER status (0, 1}
PR status E {0,
Her2 status (0, 1)
Presence of infiltrating ductal carcinoma E (0, 1}
Presence of infiltrating lobular carcinoma E {0, 1]
PGR gene expression [0, 1]
ESR1 gene expression E [0, 1]
ERBB2 gene expression defined by Desmedt al. (2008) [0
Gender {0, 1}
Age {26, .90
Tamoxifen drug sensitivity E [0,
Lapatinib drug sensitivity E [0
Additionally, we incorporate the cancer grade of each patient by combing through their written medical reports. Finally, we process a single whole-slide H&E biopsy image for each patient, which can vary significantly in size, typically around 300,000 pixels. To identify the tissue regions, we apply Otsu's segmentation method. After segmentation, we use the GigaPath foundation model from Xu et al.] (2024) to generate compact embedding vectors that capture the essential features of each biopsy. These embeddings are then further reduced to 3 dimensions using PCA (Principal Component Analysis). All of the above features are concatenated to create covariate vector X of size 19. The survival time T is defined as the time to mortality, with the censored time f, and event indicator given by the dataset.
G STUDYING THE EFFECTS OF CENSORSHIP RATE CHANGE ON THE CALIBRATION METHODS' PERFORMANCE
In this section, examine the performance of the proposed calibration methods under varying levels of censorship. We test all three of our methods on Setting from Section 4.1 for vari- ety of end-of-trial times. Figure presents the empirical coverage rate and I.B obtained by each
26

---

Published as a conference paper at ICLR 2025
technique. This ablation study demonstrates the superiority of the fused calibration method com- pared to the focused and naive methods. We can see that this setting the fused method archives a relatively tight, yet valid, LPB regardless of the censorship rate, while the focused method's perfor- mance depends much more on the censorship rate, and is similar to that of the fused method only with minimal censorship. The naive method underperforms throughout.
This hierarchy performance teaches us important lessons about each method's reactivity to changes in censorship:
The naive method has stable performance across the varied end of trial times since it cali- brated the estimator the 0.1-th quantile of T', Since changed only the upper quantiles of I' by bounding C"s value, the lower quantiles remain unchanged, and the calibration output remains unchanged.
The focused method on the other hand, calibrated the estimator to the 0.1-th quantile of i = 1, i.e. of T with making the focused method less conservative.
= As end trial time increases, more and more high values 1 get included in our calibration set, and so all quantile levels get higher,
The fused method manages to avoid the dichotomy of 7 and choosing examples that make it less conservative. The degree which it succeeds this task is surprising, and motivates the authors to delve deeper into the reasons for this success future work.
by selectively
Censorship
0.75
Rate
0.50
Coverage
0.95
Focused
0.90
Fused Naive
LPB
0.2
0.1
2
4
6
8
10
End of trial time
Figure 3: Performance of the naive, focused, and fused calibration methods as function of end of trial time. Top: Censorship rate change. Middle: Empirical coverage rate, with red dashed line indicating the nominal 90% level. Bottom: LPB. Higher is better. The reported performance metrics are evaluated on 10 independent trials, each consisting of newly sampled train, validation, calibration, and test sets of sizes 600, 200, 1000, and 200, respectively.
H RELATIVE PERFORMANCE OF THE METHODS AS FUNCTION OF THE NUMBER OF SAMPLES
We demonstrate the effect of varying sample sizes in Setting 3 from Section 4.1 on the naive, fo- cused, and fused calibration schemes. The coverage rate and LPB constructed by each method are presented in Figure 4 This figure indicates that methods attain a valid coverage rate even when used with small sample size.
We can see that when we use up to 800 samples using 50% calibration set, both the focused and fused methods show relatively high uncertainty. although their performance is still superior to that of the naive method, and they maintain validity. Above the 800 sample range, all methods perform about the same with high sample number as they do with lower sample number. We
27

---

Published as a conference paper at ICLR 2025
infer from these results that the validity of the calibration methods is unaffected by varying the number of samples, but the variance of the LPBs generated by the focused and fused methods, which don't always utilize all the calibration samples, gets predictably lower when calibrated using more samples.
1.00
Coverage
0.95
0.90
Focused
0.30
Fused
Naive
0.25
LPB
0.20
0.15
0.10
200
400
600
800
1000
1200
1400
1600
1800
2000
Number of Samples
Figure 4: Performance of the naive, focused, and fused calibration methods as function of sample size. Top:
empirical coverage rate, with red dashed line indicating the nominal 90% level. Bottom: LPB. A higher LPB is better. The reported performance metrics are evaluated on 10 independent trials, each consisting of newly sampled train, validation, calibration, and test sets which consist of 30%, 10%, 50%, and 10% proportions of the dataset, respectively.
T ABLATION STUDY: ROBUSTNESS TO WEIGHT AND QUANTILE ESTIMATION HYPERPARAMETERS
For weight estimation, we test the methods detailed in the paper using either shallow random forest classifier (max depth 2) or a deeper one (max depth 6), and for conditional quantile estimation, we test the methods using either shallow network (I hidden layer) deeper network (3 hidden layers).
We analyze the performance of the proposed calibrations under this ablation, and display the cover- age rate and LPB in Figure 5 This figure shows that our methods are robust to arbitrary selection of these hyperparameters in terms of both coverage validity and informativeness. and reinforce the superiority the fused calibration method, which manages to perform on par or better than both the focused and the naive methods.
28

---

Published as a conference paper at ICLR 2025
Shallow and Shallow Deep and Shallow
Shallow and Deep w
Deep and Deep w
1.00
Coverage
0.95
0.95
0.95
0.95
Focused
0.90
0.90
0.90
0.90
Fused
Naive
LP8
0.2
0.2
0.2
0.2
0.1
0.1
0.1
0.1
Figure 5: Performance of the naive, focused, and fused calibration methods for each combination of hyperparameters. Top:
empirical coverage rate, with red dashed line indicating the nominal 90% level. Bottom: L.B.. A higher LPB is better. The reported performance metrics evaluated 10 independent trials, each consisting of newly sampled train, validation, calibration, and test sets of sizes 600, 200, 1000, and 200, respectively.
29


Reviews:
[{"review_id": "1RBYstkrQF", "reviewer": "Reviewer_R8TU", "summary": "This paper introduces an extended conformal survival analysis framework specifically designed to handle general right-censored data, aiming to provide a reliable lower predictive bound (LPB) for patient survival times. The main contribution of the paper is the development of a novel, doubly robust calibration method that constructs valid LPBs with finite-sample guarantees, even under uncertain censoring conditions.", "questions": "1. In the \"focused calibration\" and \"fused calibration\" methods, the estimation of weights w(x) is crucial. How much impact does error in weight estimation have on the prediction results? Are there established standards for evaluating the accuracy of weight estimation, or strategies to reduce its sensitivity?\n\n2. In the experiments, the proposed method is compared with the \"naive method.\" Are there other commonly used methods for handling censored data that could serve as benchmarks? For example, traditional weighted Cox regression models or other survival analysis methods.\n\n3. What is the computational complexity of the proposed \"fused calibration\" method when handling large-scale datasets in practice? How does its computational efficiency compare to other methods?\n\n4. The proposed method relies on the estimation of weights and conditional quantiles; however, the article does not include a sensitivity analysis of these critical hyperparameters. How do the hyperparameters impact model performance? Is it possible to provide a robust hyperparameter selection strategy?\n\n5. The effectiveness of doubly robust calibration depends on the accuracy of conditional quantile or weight estimation. Could explicit theoretical bounds be provided to describe the calibration method’s performance under inaccurate estimations? It is recommended to clearly specify how such biases impact the final coverage.", "limitations": "", "rating": 3, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 1, "strengths": "The paper proposes a doubly robust calibration method capable of constructing valid lower predictive bounds (LPBs) for general right-censored data, addressing a gap in survival analysis. The proposed method provides finite-sample guarantees, which are particularly important for ensuring model reliability in high-stakes survival analysis applications.", "weaknesses": "Although the paper introduces a doubly robust calibration method, it fundamentally relies on existing conformal prediction methods (e.g., the work by Gui et al., 2024). The real innovation lies in extending these techniques to the general right-censored data setting, representing a relatively incremental contribution. The comparisons in the paper are primarily against a relatively simple \"naive\" calibration method rather than other state-of-the-art survival analysis methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4u6TeURDxl", "reviewer": "Reviewer_rRgu", "summary": "This paper extends on Gui et al.'s paper (Conformalized survival analysis with adaptive cut-offs), to construct valid and reliable lower predictive bound for survival analysis in the more general right-censoring settings, where either censoring time or event time can be observed but not both. The paper provides distribution-free finite-sample guarantees under some necessary conditions, and provides some experiments to demonstrate the effectiveness.", "questions": "Lines 294-295 state \"inaccurate estimation of $\\hat{s}_\\tau$ does not impact the validity of the calibration procedure\", what do you mean by that exactly? Theorem 3.1 states that the calibration is achieved with *accurate weights*, which seems to contradict the first statement.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "This paper is well-written and a pleasure to read. It tackles the important topic of achieving calibrated (probably approximately correct) predictions in survival analysis with the right censoring settings, an area where previous research has not provided methods with finite-sample distribution-free guarantees. The authors try to fill this gap by utilizing the concept of covariate shift. The paper is well-motivated and introduces three methods to address the problem. Starting with the simplest approach, it gradually highlights the limitations of each method and proposes solutions to overcome them. The authors provide finite-sample guarantees for the fused method, and the results on synthetic datasets support the theoretical findings.", "weaknesses": "Despite all the merits I mentioned, I have concerns regarding the experimental results. The experiments are limited to oversimplified synthetic datasets and a single small real-world dataset, without comparisons to existing methods. This raises questions about the practical effectiveness and generalizability of the proposed approach.\n\nBelow are my major concerns with the paper:\n1. **Lack of Comparison with Existing Methods:** \nThe paper does not compare the proposed method with any existing approaches. Although it is the first to provide finite-sample guarantees, this does not inherently mean it outperforms methods without such guarantees. Previous studies, such as Cresswell et al. [1], have shown that methods lacking finite-sample guarantees can perform equally well or better in practice. It is crucial to include comparative analyses with methods like those proposed by Qin et al., Meixide et al., and Qi et al. [2] to compare the empirical performance of the proposed method.\n2. **Simplistic Synthetic Datasets:** The main claim relies solely on 6 synthetic datasets with simplistic settings where survival and censoring times follow lognormal and exponential distributions. These settings are not reflective of real-world scenarios. For example, event and censor distributions can be non-parametric and censoring can stem from various sources (e.g., a combination of loss to follow-up, competing risks, and administrative censoring). I recommend designing more complex synthetic datasets that better mimic the complexities encountered in practice.\n3. **Inadequate Evaluation on Real-World Data:**\nThe evaluation of the real-world dataset is insufficient. The TCGA dataset used is small (n=1044), and the chosen evaluation metrics do not adequately support the paper's main findings. There are established metrics for assessing coverage and calibration in censored datasets, such as the distribution calibration metric discussed in Haider et al. [3]. The distribution calibration is proved to be asymptotically unbiased under the conditional independent assumption (Assumption 1.1 in this paper). Although it is not perfect (as it doesn't have a finite sample guarantee), incorporating such metrics would strengthen the validity and reliability of the experimental results.\n4. **Sensitivity to High Censoring Rates:** \nThe proposed method employs inverse probability weighting, which is known to have high variance and be sensitive to the number of uncensored subjects, especially in datasets with high censoring rates. The paper should report the censoring rates in both synthetic and real datasets and assess the method's performance across varying levels of censoring. Additionally, testing on larger datasets would provide more robust evidence of the method's practical applicability.\n5. **Challenges with Double Robustness in Practice:** While the double robustness property is theoretically appealing, achieving either condition in practical settings is challenging. The paper should discuss the practical implications of this limitation and explore potential strategies to mitigate its impact.\n\n---\n\nBelow are my additional thoughts on the lower predictive bound (LPB) and marginal coverage/calibration:\n\nAchieving a Probably Approximately Correct (PAC) lower predictive bound is important, but focusing solely on calibration or coverage is insufficient. Simple methods like the Kaplan-Meier estimator can provide calibrated predictions without the need for complex modeling. For example, a PAC-type LPB at 90% can be contained by finding the time when the KM curve drops below 90%.  A more important question is: how does the proposed method affect (or possibly decrease) other performance metrics by achieving a better calibration/coverage? Some important metrics include ranking accuracy (Concordance index), probabilistic accuracy (Brier score), and time-to-event prediction accuracy (mean absolute/squared error).\n\nThe requirement of a separate calibration set reduces the number of samples available for training, which could negatively impact predictive performance. Qi et al. [2] proposed a conformal-based method that addresses this issue by enhancing calibration without sacrificing other metrics. Discussing such alternatives would provide valuable context and highlight potential limitations for the current manuscript for improvement.\n\nMoreover, providing only the LPB may not offer sufficient information for real-world applications compared to generating survival or hazard functions (which are the standard prediction outcomes for survival analysis models). These functions provide a comprehensive view of the time-to-event distribution, which is crucial for decision-making in clinical and other applied settings. LPB is a degenerated output that can be obtained from the survival function while losing a lot of other information.\n\n\n[1] Cresswell et al., Conformal Prediction Sets Improve Human Decision Making. ICML 2024\n\n[2] Qi et al., Conformalized Survival Distributions: A Generic Post-Process to Increase Calibration. ICML 2024\n\n[3] Haider et al., Effective Ways to Build and Evaluate Individual Survival Distributions. JMLR 2020\n\n\n---\n**Update after rebuttal**\nThank you for your comprehensive rebuttal and the additional experimental results on complex synthetic datasets, real-world datasets, and supplementary baselines. Your responses have effectively addressed my concerns. I find the current draft to be a strong paper and am now more inclined toward acceptance. Accordingly, I am revising my rating to 8.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vYxMm7CI6R", "reviewer": "Reviewer_ovAp", "summary": "The manuscript is well-structured, offering solid theoretical support and a thorough experimental evaluation. The authors present a framework to quantify predictive uncertainty in survival analysis with general right-censoring, specifically addressing cases where the censoring time is not directly observed. The main contribution lies in adapting conformal prediction methods to this broader right-censoring context, providing a lower predictive bound (LPB) for patient survival times. Unlike previous approaches that assume known censoring times, this work extends the methodology to general right-censored data, where either the censoring or survival time is observed, but not both.", "questions": "- This is a general question. Why do researchers tend to prioritize the lower predictive bound rather than the upper predictive bound for survival time? Is there any existing work on the upper predictive bound for survival time?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "- The paper extends conformal prediction methods to general right-censored data, addressing a significant gap where censoring times are unknown.\n- The framework is backed by strong theoretical support, providing distribution-free, finite-sample guarantees for lower predictive bounds.\n- The authors provide both simulated and real-world data experiments, showcasing the method’s validity and practical effectiveness.", "weaknesses": "- In Section 4.1, the proposed methods appear to underperform compared to the naive method in Setting 6 regarding the lower predictive bound (LPB). Could the authors provide additional insights into why the naive method performs exceptionally well in this specific setting?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AQJa7fuxao", "reviewer": "Reviewer_bciN", "summary": "The paper proposes a reliable lower predictive bound (LPB) for the survival time under the general right-censored data based on the work in Gui et al. (2024). The proposed LPB is a PCA-type with distribution-free finite-sample guarantee.", "questions": "1.\tHow do you handle the extreme $\\hat{\\omega}$?\n2.\tHow does the estimator $\\hat{s}_{\\tau}(x)$ affect your method?\n3.\tThe fused and focused LPB may not as tight as the LPB in Gui et al. (2024), then how loss are they? Can you quantify?\n4.\tIn what settings, the fused and focused LPB are not better than the naïve LPB?\n5.\tWhy the authors not include the average LPB in the experiments?\n6.\tWhat is the coverage result of the real data experiment? Can you run experiments on other real data set? For example, the data used in Candes et al. (2023) and Gui et al. (2024).\n\n\nEmmanuel Cande`s, Lihua Lei, and Zhimei Ren. Conformalized survival analysis. Journal of the Royal Statistical Society Series B: Statistical Methodology, 85(1):24–45, 2023.\n\nYu Gui, Rohan Hore, Zhimei Ren, and Rina Foygel Barber. Conformalized survival analysis with adaptive cut-offs. Biometrika, 111(2):459–477, 2024.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 1, "presentation": 2, "contribution": 1, "strengths": "1.\tThe LPB is doubly robust by theorem.\n2.\tThe paper extends the result in Gui et al. (2024) to the general right-censoring setting.", "weaknesses": "1.\tI don't see a strong motivation for using the general right-censoring setting, which appears to be quite similar to the Type-I right-censoring setting. Other than two citations (which only mention the definition), the authors do not provide a clear explanation of why the general right-censoring setting is timely or important in real-world data.\n2.\tThe entire framework builds on Gui et al. (2024), except for the miscoverage estimators. The proposed focus calibration and fused calibration use only part of the information in the calibration set, which may lead to a waste of information. Additionally, I have concerns about the sufficiency of the proposed calibrations in cases of a large imbalance between censored and uncensored data in the calibration set (e.g., when the censoring rate is high). Therefore, there are doubts regarding the novelty of the proposed method.\n3.\tMany parts of the paper are derived from Gui et al. (2024), including the main idea, the theoretical results, the simulations, and the proofs. Several sections are not clearly explained in this paper, requiring a read of Gui et al. (2024) to understand the context. For example, why is a PAC-type LPB considered instead of a marginally calibrated LPB? How does the PAC-type LPB feature in your method? As a result, phrase like \"Notably, ...\" in line 162-163 is vague and confusing.\n4.\tSection 3.2 presents a list of results without a clear interpretation of the underlying ideas. The nested notation further complicates the readability of the section.\n5.\tIn your experiments, there is no comparison between your method and other methods, except the naïve LPB. The experimental section is mainly a collection of results without sufficient interpretations. For instance, the focused and fused LPBs are not consistently better than the naïve LPB in settings 4 and 6. In the real data experiment, the focused and fused LPB are not significantly better than the naïve LPB.\n6.\tThe data used in your experiments does not reflect a general right-censoring setting.\n7.\tSome notations in the proof are clunky and confusing. For example, in the proof of Lemma B.1, $q=q(X)$ in your $\\mathbb{P}_x(\\cdot)$ \n\nis deterministic? In Section B.2, the form \n$\\mathbb{E}[\\mathbb{P}(\\tilde{T}<\\hat{q}_{\\tau}(X)|X=x,e=1)]$ \n\ndoes not make sense, \nas $\\hat{q}_{\\tau}(X)$ is not random given $X=x$.\n\n8. Some references are not in correct format. For example, the arxiv references should be in arxiv format.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)