Title: KSP: Kolmogorov-Smirnov metric-based Post-Hoc Calibration for Survival Analysis
Venue: NeurIPS 2025
Accepted: yes
Keywords: survival analysis

calibration

Kolmogorov-Smirnov metric

post-processing
Abstract: We propose a new calibration method for survival models based on the Kolmogorov–Smirnov (KS) metric. Existing approaches—including conformal prediction, D-calibration, and Kaplan–Meier (KM)-based methods—often rely on heuristic binning or additional nonparametric estimators, which undermine their adaptability to continuous-time settings and complex model outputs. To address these limitations, we introduce a streamlined $\textit{KS metric-based post-processing}$ framework (KSP) that calibrates survival predictions without relying on discretization or KM estimation. This design enhances flexibility and broad applicability. We conduct extensive experiments on diverse real-world datasets using a variety of survival models. Empirical results demonstrate that our method consistently improves calibration performance over existing methods while maintaining high predictive accuracy. We also provide a theoretical analysis of the KS metric and discuss extensions to in-processing settings.

KSP: Kolmogorov-Smirnov metric-based Post-Hoc Calibration for Survival Analysis

Abstract

We propose a new calibration method for survival models based on the Kolmogorov-Smirnov (KS) metric. Existing approaches-including conformal prediction, D-calibration, and Kaplan-Meier (KM)-based methods-often rely on heuristic binning or additional nonparametric estimators, which undermine their adaptability to continuous-time settings and complex model outputs. To address these limitations, we introduce a streamlined KS metric-based post-processing framework (KSP) that calibrates survival predictions without relying on discretization or KM estimation. This design enhances flexibility and broad applicability. We conduct extensive experiments on diverse real-world datasets using a variety of survival models. Empirical results demonstrate that our method consistently improves calibration performance over existing methods while maintaining high predictive accuracy. We also provide a theoretical analysis of the KS metric and discuss extensions to in-processing settings.

1 Introduction

Calibration plays a vital role in ensuring reliable risk estimation for decision-making in survival analysis. Since poor calibration can misrepresent failure risk, leading to suboptimal or misguided decisions, it is especially critical in high-stakes areas such as healthcare and infrastructure systems. Recently, deep neural networks (DNNs) have significantly improved predictive performance in survival analysis (Wiegrebe et al., 2024), as evidenced by models such as DeepSurv (Kim et al., 2019), Transformer-based architectures (Hu et al., 2021), and SurvTrace (Wang and Sun, 2022). These models are typically evaluated using the concordance index (C-index; Harrell Jr et al., 1996), which measures a model's ability to correctly rank individuals. Although DNNs improve discrimination, they often suffer from over-confidence, a widely recognized issue in classification (Guo et al., 2017; Kumar et al., 2018; Mukhoti et al., 2020), which results in poor calibration. Nevertheless, survival calibration remains underdeveloped and is further complicated by censoring and time-dependent risks.

One notable contribution to survival calibration is D-calibration (Distributional calibration; Haider et al., 2020), which compares the predicted probability with the observed failure proportion within

predefined intervals. Building on this idea, X-cal (eXplicit calibration; Goldstein et al., 2020) incorporates D-calibration directly into model training. However, it relies on binning, which may underestimate calibration error (Kumar et al., 2019) and often substantially reduces discrimination performance (Qi et al., 2024a; Park et al., 2025). To reduce such trade-offs, recent studies (Avati et al., 2020; Fuhlert et al., 2022; Qi et al., 2024a,b; Lee et al., 2024) have aimed to improve calibration while maintaining predictive performance. Some leverage conformal inference (Qi et al., 2024a,b) or outcome-aware sampling (Lee et al., 2024), but their reliance on fixed sampling schemes or predefined percentiles may limit adaptability.

A promising direction for addressing these limits comes from classical statistical tools. The Kolmogorov-Smirnov (KS) metric has long been used to assess discrepancies between empirical and estimated distributions (Fernández and Gretton, 2019), and has also been employed as a calibration metric in classification (Gupta et al., 2021; Arrieta-Ibarra et al., 2022), offering a nonparametric, distribution-level evaluation of calibration error. Although the KS metric has also been applied in survival analysis (Rao, 1998; Fleming et al., 1980; Schumacher, 1984; Wu, 2018; Ansin, 2015; Cox and Snell, 1968), such studies have predominantly focused on testing, with limited attention given to its role in assessing or improving calibration error. A recent study (Park et al., 2025) introduced a KS metric for survival models, using the maximum deviation between predicted and empirical survival functions as a calibration measure. Building on this insight, we propose an efficient post-processing method that leverages the KS metric to improve calibration without compromising predictive accuracy. Our approach avoids binning, surrogate losses, and quantile estimation, providing a bin-free, global measure of calibration error.

Our main contributions are as follows:

We introduce a simple and scalable post-processing method for calibrating survival models, eliminating the need for surrogate losses, sampling, and quantile estimation.

We provide theoretical and algorithmic insights into the KS metric, including its connection to calibration in survival analysis.

We conduct extensive empirical evaluations across diverse real-world datasets and models, showing that the method improves calibration without sacrificing predictive performance.

The remainder of this paper reviews the background and motivation for calibration, examines the theoretical properties of the KS metric, and presents the post-processing method along with experimental results. All proofs of the theorem and proposition are deferred to Appendices A and C.

2 Related work

Evaluation In medical research, the calibration of survival models is often assessed graphically using calibration curves, where perfect calibration aligns with the 45-degree line representing equality between observed and predicted probabilities (Alonzo, 2009; Crowson et al., 2016). Goodness-of-fit can be evaluated using a Hosmer-Lemeshow-type test (Hosmer and Lemeshow, 1980). For DNNbased survival models, 1-calibration, which evaluates predicted survival probabilities at a specific time point, is commonly used (Yan et al., 2022; Li et al., 2023; Xia et al., 2023), though it does not capture calibration across the full survival distribution. To address this, D-calibration was introduced to assess calibration over the entire distribution (Haider et al., 2020). However, its reliance on fixed bins may underestimate calibration error (Kumar et al., 2019). Yanagisawa (2023) uses KM-calibration as an evaluation metric.

In-processing Calibration has also been incorporated into the model training process (Goldstein et al., 2020; Chapfuwa et al., 2020; Avati et al., 2020; Lee et al., 2024; Park et al., 2025). X-cal (Goldstein et al., 2020) introduces a bin-based D-calibration penalty term, while S-cal (Park et al., 2025) replaces it with a random-interval-based term. Chapfuwa et al. (2020) integrates KM estimates directly into training. These methods enhance marginal calibration, but often at the cost of reduced discrimination performance. A notable exception is the contrastive learning-based method by Lee et al. (2024), which aims to enhance discrimination without sacrificing calibration.

Post-processing To balance calibration and discrimination, post-processing approaches have been proposed primarily using conformal prediction techniques (Candès et al., 2023; Qi et al., 2024a,b; Gui et al., 2024; Qin et al., 2025; Davidov et al., 2025). The CSD method (Qi et al., 2024a) refines

survival probabilities via KM-based percentile adjustments, ensuring marginal calibration across models. It offers a model-agnostic, post-hoc calibration framework. Qi et al. (2024b) further extends this to calibrate conditional distributions for censored data.

3 Discrimination and calibration in survival analysis

Weconsider right-censored survival data, where the non-negative failure time T is subject to censoring. Let C be the censoring time, and define the observed time as Y = min( T, C ) and the censoring indicator as δ = I ( T ≤ C ) , where I ( A ) denotes the indicator function, taking the value 1 when A is true and 0 otherwise. Each observation consists of ( Y, δ, z ) , where z is a vector of covariates. Assuming independent censoring given covariates, T ⊥ ⊥ C | z , the full dataset is denoted by { ( Y i , δ i , z i ) } N i =1 for a sample size of N . Let F ( t | z ) = P ( T ≤ t | z ) denote the conditional cumulative distribution function (CDF). The corresponding survival function S , hazard function λ , and cumulative hazard function Λ are as follows:

with the relationship S ( t | z ) = exp {-Λ( t | z ) } , where Λ( t | z ) = ∫ t 0 λ ( s | z ) ds .

3.1 Discrimination

The C-index is a standard discrimination metric in survival analysis, assessing how well the model ranks individuals by risk. Harrell's C-index (Harrell Jr et al., 1996) is defined as

where M i denotes the predicted mean or a related quantity. The C-index ranges from 0 to 1, with higher values indicating better discrimination. In our case, we use the predicted mean as M i .

3.2 Calibration and metrics

Calibration in survival analysis refers to the agreement between the estimated survival distributions and actual event occurrences. While discrimination is important, calibration must also be considered to assess the uncertainty in predicted survival probabilities. A well-calibrated model should accurately estimate the probability of an event occurring by a given time. One fundamental approach to assessing calibration relies on the estimated CDF of event times. Specifically, if T ∼ F , then F ( T ) ∼ Unif [0 , 1] ; that is, the CDF evaluated at the true event time follows a uniform distribution over the interval [0 , 1] . This probabilistic property forms the basis of several calibration metrics, including D-calibration and KM-calibration.

3.2.1 D-calibration

Haider et al. (2020) introduced D-calibration, a measure designed to evaluate calibration under censoring by leveraging the distributional properties of the estimated CDF. For uncensored data, the CDF values satisfy F ( T | z ) ∼ Unif [0 , 1] . In the presence of right censoring (i.e., T > C ), the distribution becomes F ( T | z ) ∼ Unif [ F ( C | z ) , 1] . Combining both cases, the following equality holds:

Here, I = [ a, b ] ⊆ [0 , 1] is a subinterval. The term P ( F ( T | z ) ∈ I ) is computed as P ( F ( T | z ) ∈ I ) = b -F ( Y | z ) 1 -F ( Y | z ) I ( F ( Y | z ) ∈ I )+ b -a 1 -F ( Y | z ) I ( F ( Y | z ) < a ) . Based on this, D-cal is defined as the sum of bin-wise squared differences:

where I = { I 1 , . . . , I B } is a set of B ( > 0) disjoint intervals partitioning [0 , 1] , and ˆ F θ denotes the estimated CDF with model parameters θ . D-cal measures the discrepancy between the observed proportion of estimated CDF values falling into each bin and the corresponding predicted probability.

3.2.2 KM-calibration

While D-calibration evaluates calibration in the probability space by assessing the uniformity of the estimated CDF values, an alternative approach focuses on the time domain. Specifically, Yanagisawa (2023) and Qi et al. (2024a) proposed calibration metrics based on the KM estimator. One such

KM estimator and ˆ S θ ( t | z ) = 1 -ˆ F θ ( t | z ) is the model-based survival function. This metric assumes that the KM estimator is well-calibrated and measures the squared discrepancy between the empirical and predicted survival probabilities over time.

4 Motivation for new algorithm

Several recent algorithms leverage D-calibration and KM-calibration metrics, including X-cal (Goldstein et al., 2020), SFM (Survival Function Matching; Chapfuwa et al., 2020), CSD (Conformalized Survival Distributions; Qi et al., 2024a), and CSD-iPOT (individual survival Probability at Observed Time; Qi et al., 2024b). X-cal and SFM incorporate empirical variants of D-cal and KM-cal as penalty terms during model training. For example, X-cal replaces the discontinuous indicator I ( x ∈ [ a, b ]) with a smooth surrogate ζ γ ( x ; [ a, b ]) = 1 1+exp( -γ ( x -a )( b -x )) (for γ > 0 ), which enables gradientbased optimization. In contrast, CSD and CSD-iPOT adjust estimated survival distributions using conformal inference over quantile levels. These approaches are effective in reducing calibration error while maintaining predictive accuracy, but they have some limitations.

X-cal Binning-based methods, such as those underlying D-cal and X-cal, may underestimate calibration error (Kumar et al., 2019), a limitation that extends to the survival setting. Additionally, penalty-based calibration can lead to degradation in discriminative performance, as observed in Qi et al. (2024a); Park et al. (2025).

SFM This method involves computing E z [ ˆ S θ ( t | z )] , which can be computationally intensive. Moreover, when the model's estimated survival function is already better calibrated than the KM estimator, the KM-based correction may not further reduce calibration error.

CSD and CSD-iPOT CSD relies on the KM estimator for handling censored observations, whereas CSD-iPOT avoids KM-based estimation. However, both methods involve sampling procedures, which can introduce additional variance, especially in the tail regions, and may result in non-monotonic quantiles (i.e., quantile crossing).

These limitations motivate a calibration approach that (i) avoids discretization and binning, (ii) preserves discriminative performance (e.g., C-index), (iii) does not require computationally expensive sampling, and (iv) remains stable in tail regions.

The KS metric is a classical tool for quantifying discrepancies between empirical and target distributions. In the calibration context, it has been applied to classification tasks (Gupta et al., 2021; Arrieta-Ibarra et al., 2022) as a nonparametric, bin-free measure of calibration error. Among existing tools, Cox-Snell residuals (Cox and Snell, 1968) can be used to evaluate calibration error. However, they rely on the cumulative hazard function and are less practical than CDF-based approaches. Therefore, we focus on an alternative form of calibration error. Previous work of Park et al. (2025) proposed using the KS metric as a calibration measure in survival analysis. We refer to this metric as KS-cal and build upon it by introducing a post-processing algorithm that improves calibration while preserving predictive accuracy.

For finite samples, we approximate KS-cal using the evaluation points q j = ˆ F θ ( y j | z j ) as KS-cal = max 1 ≤ j ≤ N D j , where D j = max { D j,u , D j,l } with D j,u = | ˜ F ( q j ) -q j | and D j,l = | ˜ F ( q j ) -δ j /N -q j | . The term D j,l accounts for the left-limit behavior of ˜ F at uncensored points.

We observe that ˜ F ( x ) exhibits jumps at uncensored points while increasing linearly over censored points. Figure 1 illustrates this behavior, showing how D j is computed at uncensored observations

Pictorialexampleofhowtofind Dj

Uncalibratedmodel

Calibrated model

0.55

F(x)

1.00

F(x)

1.00

F(x)

0.53

Perfect Calibration

Perfect Calibration

Perfect Calibration

0.50

0.80

0.80

0.0866

D

0.48

0.60

0.60

0.45

0.43

Dj.1

0.40

0.2936

LL

0.40

0.40

0.20

0.20

0.38

D=max{Dj.uDj}=Dj

0.00

0.00

0.35

0.40

0.42

0.44

0.46

0.48

0.50

0.00

0.20

0.40

0.60

0.80

1.00

0.00

0.20

0.40

0.60

0.80

1.00

X

and contrasting uncalibrated and calibrated models in terms of KS-cal. This metric offers a binfree assessment of calibration error and is conceptually related to D-calibration (Haider et al., 2020), although key differences in how calibration is measured remain. D-cal evaluates deviations within fixed bins, whereas KS-cal captures the maximum discrepancy at data-adaptive points where miscalibration is most evident, as determined by ˆ F θ . In the following section, we formalize the theoretical properties of the KS-cal, including consistency and convergence behavior.

4.1 Property of KS-cal

We now analyze the theoretical behavior of KS-cal. We begin with the simplified setting without covariates, i.e., U i = ˆ F θ ( Y i ) , where calibration corresponds to ˆ F θ = F , with F denoting the true CDF of T . The following result has been established in the previous work of Park et al. (2025).

Proposition 4.1. Assume the Regularity Conditions in Appendix A. Then calibration holds if and only if

We now extend this result to the setting with covariates, i.e., U i = ˆ F θ ( Y i | z i ) . Following Goldstein et al. (2020), calibration with covariates is defined as:

Then, we have the following theorem.

Theorem 4.1. Assume the Regularity Conditions in Appendix A. Then calibration (with/without covariates) holds if and only if

This result implies that ˜ F converges to the CDF of Unif [0 , 1] when the model is calibrated, where the theorem has two folds (with/without covariates). Therefore, we can establish KS-cal as a theoretically grounded, bin-free metric for evaluating calibration in survival models. Moreover, under the regularity conditions in Appendix A, the convergence of ˜ F (and thus KS-cal) occurs at the rate of O p ( N -1 / 2 ) up to log N.

Remark. Based on Eqn. (3), an alternative calibration metric termed S-cal (Summation-based Calibration) has been proposed (Qi et al., 2024a,b; Park et al., 2025). This metric captures cumulative deviations from ideal calibration and is defined as:

S-cal, D-cal, and KS-cal are all variants of D-calibration, derived from the same foundational definition, but they emphasize different aspects of calibration behavior. We include all three in our evaluation to provide a comprehensive view of model calibration.

5 Calibration method based on KS-cal

In this work, we propose a post-hoc calibration method based on the KS-cal introduced in Section 4. We also briefly describe how the KS-cal can be incorporated as a penalty term during model training.

5.1 Post-processing

We propose KS-cal based post-processing (KSP), similar to Platt scaling (Platt, 1999) in classification tasks. We simply transform the original ˆ F θ into a modified ˆ F ∗ θ ⊆ [0 , 1] that minimizes the KS-cal. The procedure is summarized in the following algorithm.

Algorithm. KSP

1: Input: Estimated CDFs ˆ F θ , strictly monotone increasing link function G : [0 , 1] → ( -∞ , ∞ )

2: Initialize parameters a ( > 0) , b , α ( > 0)

3: Sort ˆ F θ for computational efficiency

4: while KS-cal not improved do

5: Compute transformed CDF: ˆ F ∗ θ = { G -1 ( a · G ( ˆ F θ ) + b ) } α

6: Compute KS-cal on validation set: max 1 ≤ j ≤ N D ∗ j , where D ∗ j denotes D j evaluated using ˆ F ∗ θ

7: Update ( a, b, α ) via gradient descent (ADAM) to minimize the KS-cal

8: end while

9: Apply final calibrated transformation to the test set using optimized ( a, b, α )

10: Output: Calibrated CDF ˆ F ∗ θ

The function G can be any link function satisfying the two predefined conditions described in the Input, such as the logit function, inverse hyperbolic tangent function, or an inverse CDF. Although a DNN-based transformation is feasible, it is challenging to preserve the ordering of the transformed CDFs consistent with the original ones. In this work, we adopt the logit function as G .

The three hyperparameters a, b, and α control different aspects of the CDF transformation: a influences the tails, b shifts the distribution relative to the target x , and α controls nonlinearity. Their effects are visualized and discussed in the Appendix I. As long as a > 0 and α > 0 , the transformation preserves the ordering of CDF values, thereby maintaining the time-dependent C-index (Antolini et al., 2005), consistent with Qi et al. (2024b). Specifically, if ˆ F θ ( t | z 1 ) > ˆ F θ ( t | z 2 ) , then the transformed CDF also satisfies ˆ F ∗ θ ( t | z 1 ) > ˆ F ∗ θ ( t | z 2 ) . While the CDF ordering is preserved, the ordering of expected survival times may not be, as the transformation can nonlinearly distort the survival curve. This is particularly relevant when survival curves intersect. In such cases, monotonicity is maintained at each time point, but the area under the curve may change, which in turn affects the expected value. In practice, such changes are typically small, and overall concordance may even improve. We also formalize the conditions under which the mean ordering is preserved. In particular, if the original survival curves do not cross (preserving the order across all times), the transformation preserves the ordering of expected survival times. Theoretically, these are summarized as:

Proposition 5.1. Let E [ T ∗ | z ] = ∫ ∞ 0 S ∗ ( t | z ) d t denote the expected survival time under the KSP , where S ∗ ( t | z ) = 1 -F ∗ ( t | z ) . If the original survival curves S ( t | z 1 ) and S ( t | z 2 ) do not cross, then the ordering of expected survival times is preserved under the KSP:

The proof is provided in Appendix C. The non-crossing condition holds for some standard models, including the Cox proportional hazards (PH) model (Cox, 1972) and the Weibull Accelerated Failure Time (AFT) model (Stute, 1993). By Proposition 5.1, the C-index is preserved since we use the predicted mean of survival time for the C-index.

Compared to prior methods such as CSD (Qi et al., 2024a) and CSD-iPOT (Qi et al., 2024b), KSP provides better computational efficiency. CSD requires O ( N · |P| · R ) operations, and CSD-iPOT requires O ( N · R ) , where N and |P| are the sample size and the number of quantiles, respectively, and R is the number of samples used for censored data. In contrast, KSP requires only O ( B · N + N log N ) , which simplifies to O ( B · N ) in practice, where B is the number of optimization iterations. The N log N term accounts for the initial sorting of CDF values, and each iteration involves linear-time operations. Although the number of iterations may increase when the initial calibration error is large, the overall overhead remains modest and can be controlled through the learning rate. This scalability makes KSP particularly suitable for large datasets.

5.2 In-processing

KSP is introduced as a post-processing method to reduce calibration errors while preserving predictive accuracy. Alternatively, the KS-cal defined in (2) can be incorporated directly into training as a penalty term: P k ( θ ) = ∑ k j =1 ( D ( N -j +1) ) 2 , where the subscript denotes the order statistic. Unlike the original KS-cal, which considers only the maximum deviation, this variant aggregates the topk largest deviations. Using the squared form improves gradient stability, making it more suitable for optimization. Since the deviations are evaluated only at observed points, the indicator function in ˜ F ( x ) naturally disappears when the CDF values are pre-sorted. In contrast, methods like X-cal require a surrogate function to approximate the indicator for gradient-based learning. We refer to this variant as KS-cal( k ). While our primary focus is on post-processing, we also explore this in-processing approach in Appendix J. Although the penalty function is not convex, we observe that training remains stable and converges well in practice. However, using excessively small batch sizes can lead to large approximation errors; thus, we recommend employing relatively large batch sizes for reliable performance.

6 Experiment

We first compare post-processing methods, including CSD (Qi et al., 2024a), CSD-iPOT (Qi et al., 2024b), and KSP, across various models and datasets. We also evaluate in-processing approaches such as X-cal (Goldstein et al., 2020) and SFM (Chapfuwa et al., 2020). While SFM originally combines a calibration penalty with a discriminative loss, we use only the calibration component for consistency. For all in-processing methods, we tune the regularization parameter over { 1 , 10 , 100 , 1000 } . Experiments are conducted with an Intel Xeon Silver 6226R CPU and an NVIDIA GeForce RTX 3090 GPU.

6.1 Experimental setup

Baselines We consider six baseline models: DeepSurv-based Cox PH model (Katzman et al., 2018), Multi-task Logistic Regression (MTLR; Yu et al., 2011), a parametric model (Goldstein et al., 2020), Survival CRPS (Avati et al., 2020), DeepHit (Lee et al., 2018), and the Weibull AFT model (Stute, 1993). As a reference, we include the KM estimator fitted on the training set and evaluated on the test set, following Qi et al. (2024a,b). The KM estimator serves as an empirical lower bound for calibration error (see Appendix B in Qi et al. (2024a)). Further details on the baseline models are provided in Appendix B.

Datasets We evaluate all methods on ten benchmark datasets: WHAS, METABRIC, GBSG, NACD, NB-SEQ, SUPPORT, MIMIC-III, SEER-liver, SEER-stomach, and SEER-lung. These are grouped by sample size into three categories: Small (WHAS, METABRIC, GBSG, NACD), Medium (NBSEQ, SUPPORT, MIMIC-III), and Large (SEER-liver, SEER-stomach, SEER-lung). Details on the datasets and preprocessing steps are provided in Appendix D. Each dataset is randomly split into training, validation, and test sets in a 3:1:1 ratio, with balanced censoring rates. All experiments are repeated 30 times with different random seeds. For CSD and CSD-iPOT, we use the validation set as the conformal set to enable a fair comparison with KSP.

Evaluation metrics For predictive accuracy, we report the (time-independent) C-index (Harrell Jr et al., 1996) rather than the time-dependent variant, as the latter may favor our method and CSDiPOT (Qi et al., 2024b), potentially inflating their discrimination performance. For calibration, we evaluate five metrics: S-cal(20), D-cal(20), KS-cal, KM-cal, and the Integrated Brier Score (IBS;

100

KSP

CSD

CSD-iPOT

(in sec.)

80

Inference Runtime(

60

40

20

0

Small

Medium

Large

Sample Size

C-index

S-cal(20)

D-cal(20)

1e-01

1.0

0.100

1e-02

0.7

0.010

1e-03

0.5

1e-04

0.001

Non-calibrated

KSP

CSD

CSD-IPOT

Non-calibrated

KSP

CSD

CSD-iPOT

Non-calibrated

KSP

CSD

CSD-POT

KS-cal

KM-cal

IBS

0.100

0.3

0.30

0.010

0.2

0.10

0.03

0.001

0.1

Non-calibrated

KSP

CSD

CSD-IPOT

Non-calibrated

KSP

CSD

CSD-iPOT

Non-calibrated

KSP

CSD

CSD-iPOT

Graf et al., 1999). S-cal(20) is computed over 20 equally spaced points s ∈ { 0 . 05 , 0 . 10 , ..., 1 . 00 } , while D-cal(20) uses 20 equal-width bins. The results based on a 10-bin evaluation are provided in Appendix H. KS-cal is a bin-free metric. KM-cal and IBS assess 1-calibration and offer complementary perspectives. Each model is thus evaluated using one discrimination metric and five calibration metrics. For CSD and CSD-iPOT, we follow the original setup by using P = { 0 . 1 , 0 . 2 , ..., 0 . 9 } for quantile adjustment and set R = 1000 to sample censored observations.

6.2 Experimental results

Comparison with post-processing methods We conduct 6 × 10 = 60 experiments across models and datasets. For each setting, we perform pairwise comparisons among the non-calibrated baseline, CSD, CSD-iPOT, and KSP, counting how often each method outperforms the others. As summarized in Table 1, KSP outperforms the alternatives in approximately 70% of cases. Compared to the non-calibrated baseline, KSP consistently achieves better calibration and significantly improves the C-index in 10 cases, especially for the parametric model and CRPS. Against CSD, KSP shows better calibration, while CSD slightly leads in the C-index due to its guaranty of preserving the time-independent C-index. Compared to CSD-iPOT, KSP offers a similar C-index with clearly better calibration.

KSP's improvements are more evident in D-cal(20) and KS-cal, although it still performs competitively in S-cal(20), which tends to favor CSD and CSD-iPOT due to their quantile-based structure. A similar pattern appears in discretized models like MTLR and DeepHit, where KSP shows slightly weaker calibration, possibly due to structural mismatch. Nonetheless, KSP outperforms in KM-cal and IBS, confirming its overall calibration strength. Figure 3 further illustrates that CSD and CSDiPOT often exhibit calibration mismatches near the left tail due to boundary interpolation, which D-cal(20) and KS-cal capture effectively. Full results are presented in the Appendix F.

KSP also offers efficiency advantages. As shown in Figure 2, runtime increases with sample size for all methods, but the growth is slower for KSP. This is because CSD and CSD-iPOT require additional bootstrap sampling for the monotonicity of quantiles, while KSP depends only on the number of optimization steps. The complete runtime summary is provided in Appendix K. Figure 2 also presents the distribution of each metric across all experiments, with the median values in the boxplots serving as a key point of comparison. All methods show comparable C-index values, confirming that postprocessing can improve calibration without compromising discrimination. KSP consistently achieves the best performance in S-cal(20), D-cal(20), and KS-cal. Although CSD records the lowest median KM-cal, KSP surpasses it more frequently in pairwise comparisons, indicating greater robustness. KSP shares a similar property with CSD-iPOT in preserving time-dependent discrimination, as reflected in their comparable performance on KM-cal and IBS.

Calibration plot Figure 3 shows calibration plots for MTLR and CRPS on the MIMIC-III dataset. Although all three post-processing methods achieve similar quantitative scores, CSD and CSD-iPOT exhibit slight mismatches near the left tail, likely due to boundary interpolation in their quantile adjustments. In contrast, KSP reduces the maximum deviation, resulting in more uniform calibration across the entire range. This pattern aligns with the D-cal(20) and KS-cal results in Table 1, where KSP outperforms other methods by better capturing localized discrepancies. Full results are provided in the Appendix G.

MTLR

MTLR

CRPS

CRPS

KSP

KSP

KSP

KSP

CSD

CSD-iPOT

CSD

CSD-iPOT

Observedprobability

probabilit

probabilit

Observedprobability

Observedp

Observed

9.2

9.2

Predictedprobability

Predictedprobability

Predictedprobability

Predictedprobability

Comparison with in-processing methods We report comparisons with in-processing methods (SFM and X-cal) for CRPS on the MIMIC-III dataset, which illustrate typical trade-off patterns. As λ increases, both in-processing methods reduce calibration error but degrade the C-index. SFM, which penalizes KM-calibration, lowers KM-cal and IBS but performs worse on D-calibration. X-cal improves D-calibration but incurs a greater C-index loss and increases KM-cal and IBS, particularly at λ = 1000 . In contrast, KSP achieves a better trade-off with stable performance across metrics and no need for tuning. While in-processing may be helpful when calibration is the sole priority, KSP offers a more practical and balanced solution when both discrimination and calibration are important. Figure 4 summarizes these results, showing the mean and standard error across multiple λ values.

C-index'

S-cal(20)

D-cal(20)

KS-cal

KM-cal*

IBS↓

王

I1

T

0.8

0.08

王

王

工

0.20

0.15

0.9

0.15

0.2

0.6

王

0.06

0.10

0.8

0.10

0.1

0.4

工

0.04

亚

0.7

0.05

王

T⊥

0.2

+

工

0.02

工

工

工

0.05

王

0.00

王

0.0

王

0.0

0.00

工

Non-calibrated→KSP→SFM→X-cal

Ablation study We conduct an ablation study to assess the effects of various design choices in the KSP optimization procedure. Specifically, we investigate (1) the choice of link function G , (2) the role of hyperparameters a, b, and α , and (3) alternative loss formulations beyond the default max j D ∗ j ,

as detailed in the Appendix I. In the main text, we present partial results for (2) on the MIMIC-III dataset using DeepSurv, summarized in Table 2, with full results deferred to the Appendix I. We find that using b alone substantially improves calibration, reducing D-cal(20) and KS-cal by 35% and 27%, respectively, without degrading the C-index. Introducing a and α yields further gains, especially in D-cal(20), by enabling scale adjustment and smoothing. While the magnitude of improvement may vary across datasets and models, all three hyperparameters contribute positively to the KSP procedure. Note that 'Non-calibrated' in Table 2 refers to the default setting with a = 1 , b = 0 , α = 1 , that is, when KSP is not applied.

7 Conclusion

Our work highlights the effectiveness of KS-cal as a calibration metric and introduces KSP as a practical post-processing method for improving the reliability of survival models. Beyond its empirical and computational advantages, KSP offers a theoretically grounded framework and avoids the limitations of bin-based or sampling-dependent approaches. While KSP shows clear advantages, it tends to be less effective in datasets with heavy ties or severe skewness; however, it is more robust when calibration errors are large and in capturing local discrepancies, such as tails. As an important direction for future research, we aim to extend KSP toward conditional calibration, enabling personalized and subgroup-level reliability assessments. Such developments are particularly valuable for high-stakes domains where accurate uncertainty quantification is critical.

Acknowledgment

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) [RS-2023-00218377], Global - Learning & Academic research institution for Master's · PhD students, and Postdocs (LAMP) Program of the NRF grant funded by the Ministry of Education [RS-2024-00443714].

References

Alonzo, T. A. (2009). Clinical prediction models: A practical approach to development, validation, and updating: By ewout w. steyerberg. American Journal of Epidemiology , 170(4):528.

Ansin, E. (2015). An evaluation of the cox-snell residuals. Master's thesis, Uppsala University, Department of Statistics, Uppsala, Sweden.

Antolini, L., Boracchi, P., and Biganzoli, E. (2005). A time-dependent discrimination index for survival data. Statistics in Medicine , 24(24):3927-3944.

Arrieta-Ibarra, I., Gujral, P., Tannen, J., Tygert, M., and Xu, C. (2022). Metrics of calibration for probabilistic predictions. Journal of Machine Learning Research , 23:1-54.

Avati, A., Duan, T., Zhou, S., Jung, K., Shah, N. H., and Ng, A. Y. (2020). Countdown regression: sharp and calibrated survival predictions. In Proceedings of the Conference on Uncertainty in Artificial Intelligence , pages 145-155. PMLR.

Candès, E., Lei, L., and Ren, Z. (2023). Conformalized survival analysis. Journal of the Royal Statistical Society Series B: Statistical Methodology , 85(1):24-45.

Chapfuwa, P., Tao, C., Li, C., Khan, I., Chandross, K. J., Pencina, M. J., Carin, L., and Henao, R. (2020). Calibration and uncertainty in neural time-to-event modeling. IEEE Transactions on Neural Networks and Learning Systems , 34(4):1666-1680.

Cox, D. R. (1972). Regression models and life-tables. Journal of the Royal Statistical Society: Series B (Methodological) , 34(2):187-202.

Cox, D. R. and Snell, E. J. (1968). A general definition of residuals. Journal of the Royal Statistical Society: Series B (Methodological) , 30(2):248-265.

Crowson, C. S., Atkinson, E. J., and Therneau, T. M. (2016). Assessing calibration of prognostic risk scores. Statistical Methods in Medical Research , 25(4):1692-1706.

Davidov, H., Feldman, S., Shamai, G., Kimmel, R., and Romano, Y. (2025). Conformalized survival analysis for general right-censored data. In the International Conference on Learning Representations . https://openreview.net/forum?id=JQtuCumAFD .

Fernández, T. and Gretton, A. (2019). A maximum-mean-discrepancy goodness-of-fit test for censored data. In Proceedings of the International Conference on Artificial Intelligence and Statistics , volume 89, pages 2966-2975. PMLR.

Fleming, T. R., O'Fallon, J. R., O'Brien, P. C., and Harrington, D. P. (1980). Modified kolmogorovsmirnov test procedures with application to arbitrarily right-censored data. Biometrics , pages 607-625.

Fuhlert, P., Ernst, A., Dietrich, E., Westhaeusser, F., Kloiber, K., and Bonn, S. (2022). Deep learningbased discrete calibrated survival prediction. In Proceedings of the IEEE International Conference on Digital Health , pages 169-174.

Goldstein, M., Han, X., Puli, A., Perotte, A., and Ranganath, R. (2020). X-cal: explicit calibration for survival analysis. In Advances in Neural Information Processing Systems , volume 33, pages 18296-18307.

Graf, E., Schmoor, C., Sauerbrei, W., and Schumacher, M. (1999). Assessment and comparison of prognostic classification schemes for survival data. Statistics in Medicine , 18(17-18):2529-2545.

Gui, Y., Hore, R., Ren, Z., and Barber, R. F. (2024). Conformalized survival analysis with adaptive cut-offs. Biometrika , 111(2):459-477.

Guo, C., Pleiss, G., Sun, Y., and Weinberger, K. Q. (2017). On calibration of modern neural networks. In Proceedings of the International Conference on Machine Learning , volume 70, pages 1321-1330. PMLR.

Gupta, K., Rahimi, A., Ajanthan, T., Mensink, T., Sminchisescu, C., and Hartley, R. (2021). Calibration of neural networks using splines. In the International Conference on Learning Representations . https://openreview.net/forum?id=eQe8DEWNN2W .

Haider, H., Hoehn, B., Davis, S., and Greiner, R. (2020). Effective ways to build and evaluate individual survival distributions. Journal of Machine Learning Research , 21(1):3289-3351.

Harrell Jr, F. E., Lee, K. L., and Mark, D. B. (1996). Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Statistics in Medicine , 15(4):361-387.

Harutyunyan, H., Khachatrian, H., Kale, D. C., Ver Steeg, G., and Galstyan, A. (2019). Multitask learning and benchmarking with clinical time series data. Scientific Data , 6(1):96.

Hosmer, D. W. and Lemeshow, S. (1980). Goodness of fit tests for the multiple logistic regression model. Communications in Statistics-Theory and Methods , 9(10):1043-1069.

Hu, S., Fridgeirsson, E., van Wingen, G., and Welling, M. (2021). Transformer-based deep survival analysis. In Proceedings of Survival Prediction: Algorithms, Challenges and Applications , pages 132-148.

Karandikar, A., Cain, N., Tran, D., Lakshminarayanan, B., Shlens, J., Mozer, M. C., and Roelofs, B. (2021). Soft calibration objectives for neural networks. In Advances in Neural Information Processing Systems , volume 34, pages 29768-29779.

Katzman, J. L., Shaham, U., Cloninger, A., Bates, J., Jiang, T., and Kluger, Y. (2018). Deepsurv: Personalized treatment recommender system using a cox proportional hazards deep neural network. BMC Medical Research Methodology , 18(1):24.

Kim, D. W., Lee, S., Kwon, S., Nam, W., Cha, I.-H., and Kim, H. J. (2019). Deep learning-based survival prediction of oral cancer patients. Scientific Reports , 9(1):6994.

Kumar, A., Liang, P. S., and Ma, T. (2019). Verified uncertainty calibration. In Advances in Neural Information Processing Systems , volume 32, pages 3792-3803.

Kumar, A., Sarawagi, S., and Jain, U. (2018). Trainable calibration measures for neural networks from kernel mean embeddings. In Proceedings of the International Conference on Machine Learning , volume 80, pages 2805-2814. PMLR.

Lee, C., Zame, W., Yoon, J., and Van Der Schaar, M. (2018). Deephit: A deep learning approach to survival analysis with competing risks. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 32, pages 2314-2321.

Lee, D., Park, H., and Lee, C. (2024). Toward a well-calibrated discrimination via survival outcomeaware contrastive learning. In Advances in Neural Information Processing Systems , volume 37, pages 30985-31014.

Li, R., Qu, W., Liu, Q., Tan, Y., Zhang, W., Hao, Y., Jiang, N., Mao, Z., Ye, J., Jiao, J., et al. (2023). Development and validation of a deep learning survival model for cervical adenocarcinoma patients. BMC Bioinformatics , 24(1):1-15.

Lin, D. Y. (2007). On the breslow estimator. Lifetime Data Analysis , 13(4):471-480.

Mukhoti, J., Kulharia, V., Sanyal, A., Golodetz, S., Torr, P., and Dokania, P. (2020). Calibrating deep neural networks using focal loss. In Advances in Neural Information Processing Systems , volume 33, pages 15288-15299.

Park, J., Kang, S., and Kim, G. (2025). Stochastic explicit calibration algorithm for survival models. IEEE Access , 13:78679-78706.

Platt, J. C. (1999). Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In Advances in Large Margin Classifiers , pages 61-74. MIT Press.

Qi, S.-A., Yu, Y., and Greiner, R. (2024a). Conformalized survival distributions: A generic postprocess to increase calibration. In Proceedings of the International Conference on Machine Learning , volume 235, pages 41303-41339. PMLR.

Qi, S.-A., Yu, Y., and Greiner, R. (2024b). Toward conditional distribution calibration in survival prediction. In Advances in Neural Information Processing Systems , volume 37, pages 8618086225.

Qin, J., Piao, J., Ning, J., and Shen, Y . (2025). Conformal predictive intervals in survival analysis: a resampling approach. Biometrics , 81(2):ujaf063.

Rao, M. B. (1998). Survival analysis, techniques for censored and truncated data. Technometrics , 40(2):159-160.

Schumacher, M. (1984). Two-sample tests of cramér-von mises- and kolmogorov-smirnov-type for randomly censored data. International Statistical Review , 52(3):263-281.

Stute, W. (1993). Consistent estimation under random censorship when covariables are present. Journal of Multivariate Analysis , 45(1):89-103.

Van der Vaart, A. W. (2000). Asymptotic Statistics , volume 3. Cambridge University Press.

Wang, Z. and Sun, J. (2022). Survtrace: Transformers for survival analysis with competing events. In Proceedings of the ACM International Conference on Bioinformatics, Computational Biology & Health Informatics , pages 1-9.

Wiegrebe, S., Kopper, P., Sonabend, R., Bischl, B., and Bender, A. (2024). Deep learning for survival analysis: a review. Artificial Intelligence Review , 57(3):65.

Wu, T. (2018). Randomized survival probability residual for assessing parametric survival models. Master's thesis, University of Saskatchewan, School of Public Health, Saskatoon, Canada.

Xia, Y., Zhang, B., and Zhang, Y. (2023). Deep survival analysis using pseudo values and its application to predict the recurrence of stage iv colorectal cancer after tumor resection. Computer Methods in Biomechanics and Biomedical Engineering , 27(15):1-10.

Yan, L., Gao, N., Ai, F., Zhao, Y ., Kang, Y ., Chen, J., and Weng, Y . (2022). Deep learning models for predicting the survival of patients with chondrosarcoma based on a surveillance, epidemiology, and end results analysis. Frontiers in Oncology , 12:967758.

Yanagisawa, H. (2023). Proper scoring rules for survival analysis. In Proceedings of the International Conference on Machine Learning , volume 202, pages 39165-39182. PMLR.

Yu, C.-N., Greiner, R., Lin, H.-C., and Baracos, V. (2011). Learning patient-specific cancer survival distributions as a sequence of dependent regressors. In Advances in Neural Information Processing Systems , volume 24, pages 1845-1853.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract and introduction clearly state the claims.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We included our limitations on Section 6.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We provide the theoretical result in Sections 4.1 and 5.1 and the proofs in Appendices A and C.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: All experimental settings, hyperparameters, and evaluation protocols are fully described in the main paper and Appendix. The corresponding code is also publicly available.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may

be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: All datasets used are public benchmarks, and we provide open-source code with instructions to reproduce all experiments at https://github.com/wjdgh4325/KS-cal , as also noted in the Appendix F.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and Non-calibrateds. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We specify details on data split in Appendix D, other details in Appendix K.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: Table 1 shows statistical significance, and we provide error bars for experimental results in Appendix E.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We specify compute resources in Section 6.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We follow NeurIPS code of ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We discuss broader impact in Appendix.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: No high-risk models or datasets are used or released in this work.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We used publicly available code assets under the MIT License:

https://github.com/rajesh-lab/X-CAL

https://github.com/shi-ang/CSD

https://github.com/shi-ang/MakeSurvivalCalibratedAgain

All licenses and attributions have been properly acknowledged, and relevant papers are cited in the Appendix F.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [No]

Justification: We will open new assets after the paper is accepted.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [No]

Justification: Not applicable.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [No]

Justification: Not applicable.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [No]

Justification: We did not use the LLM for the core parts of the paper.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

Appendix

Broader impacts

Improving survival calibration has broader impacts on clinical decision-making, fairness, resource allocation, and policy development. By producing accurate risk estimates, it supports more reliable and balanced decisions across medical and public health domains.

Code availability

The source code for reproducing our experiments is available at: https://github.com/ wjdgh4325/KS-cal

A Proofs of Proposition 4.1 and Theorem 4.1

We denote ˆ F θ = F ˆ θ for notational clarity.

Regularity Conditions

With or without covariates, F -1 ˆ θ ( x ) exists due to the required properties of F ˆ θ , such as left-continuity, the existence of a right-hand limit, and monotonicity. For simplicity, we omit the dependence on z in the notation when covariates are present.

For the case without covariates, when non-calibration holds,

there exists an x ∗ ∈ (0 , 1) such that

and for the above x ∗ , ∫ x ∗ 0 dG ( F -1 ˆ θ ( s ) ) < 1

where F denotes the true CDF of the failure time, and G denotes the CDF of the censoring time.

Proof of Sufficient Condition in Proposition 4.1

For the proof, we let ˜ F ( x ) = 1 N ∑ N i =1 ˜ F i ( x ) where ˜ F i ( x ) = I ( U i ≤ x ) { δ i +(1 -δ i ) x -U i 1 -U i } for 0 ≤ x ≤ 1 , U i = F ˆ θ ( Y i ) . By Eqn. (8) in Lemma A.2, we have E [ ˜ F i ( x ) ] = x, E [ ˜ F i ( x ) 2 ] = x -(1 -x ) ∫ x 0 x -s 1 -s dG ( F -1 ( s ) ) ≤ 2 and 0 ≤ ˜ F i ≤ 1 . Then, by the Bernstein inequality (Van der Vaart, 2000) stating that for an independent random variable X i such that E [ X i ] = 0 , | X i | ≤ M ,

For any 0 < ϵ < 1 / 2 , we have a set S of points between [0 , 1] such that { 0 , ϵ, 2 ϵ, [1 /ϵ ] ϵ, 1 } . It implies ∀ t ∈ [0 , 1] , ∃ k ∈ B s.t. | t -k | ≤ ϵ. This implies that

Then,

By the Bernstein inequality, P ( N | ˜ F ( x ) -x | > ϵ ) ≤ 2 exp( -ϵ 2 / (2 N + ϵ/ 3)) , due to the symmetry of the absolute value and the expectation of the square is less than 2. This implies

Finally, we have as N →∞ .

Proof of Necessary Condition in Proposition 4.1

Assuming the regularity conditions, the fact that non-calibration implies that

has the deviation cη > 0 from some x ∗ is sufficient for the proof. By Lemmas A.2-A.3 and the Regularity Conditions imply that when the non-calibration holds, sup x ∈ [0 , 1] | ˜ F ( x ) -x | ≥ cη/ 2 with probability approaching 1 as N →∞ . This completes the proof.

Proof of Theorem 4.1

Lemma A.1 and the definition of calibration such as E [ z x i ( U i , δ i )] = x imply

in probability as N → ∞ . Furthermore, when non-calibration holds such as sup x ∈ [0 , 1] | E [ ˜ F ( x )] -x | > δ , we have P ( sup x ∈ [0 , 1] | ˜ F ( x ) -x | > δ/ 2 ) → 1 as N → ∞ . This implies that calibration holds if P ( sup x ∈ [0 , 1] | ˜ F ( x ) -x | > δ/ 2 ) → 0 as N → ∞ . It is by contraposition.

Lemma A.1. Assuming that the Regularity Conditions hold, we have that for any 0 < ϵ < 1 / 2

goes to 0 as N →∞ where z x i ( U i , δ i ) = I ( U i ≤ x ) { δ i +(1 -δ i ) x -U i 1 -U i } and U i = F ˆ θ ( Y i | z i ) .

Note that we have | z x i ( U i , δ i ) | ≤ 1 and ( z x i ( U i , δ i ) -E [ z x i ( U i , δ i ]) 2 ≤ 2 . Then, by the Bernstein inequality (Van der Vaart, 2000) stating that for an independent random variable X i such that E [ X i ] = 0 , | X i | ≤ M,

For simplicity, we let

For any 0 < ϵ < 1 / 2 , we have a set S of points between [0 , 1] such that { 0 , ϵ, 2 ϵ, [1 /ϵ ] ϵ, 1 } . It implies ∀ t ∈ [0 , 1] , ∃ k ∈ S s.t. | t -k | ≤ ϵ. This implies that

Then,

By the symmetric applications of Bernstein inequality,

which implies

This is due to the fact that the square-expectation is less than 2. Finally, we have

as N →∞ .

Remark. If we let ϵ ≡ ϵ N = M N log N √ N for any diverging sequence M N , then the probability in the right term of Eqn. (5) approaches 0 as N →∞ . This ensures the convergence rate of O p (1 / √ N ) up to log N .

Lemmas for the proof of Proposition 4.1

Lemma A.2. Under the Regularity Conditions and F ˆ θ = F , we have

Proof. Regularity Conditions and calibration of F ˆ θ imply that

Proof of E [ ˜ F i ( x ) ] = x

Then ˜ F i ( x ) def = Z x ( U i , δ i ) . We have

Similarly,

By combining (6) and (7), and using integration by parts with respect to dG ( · ) ,

It implies that

Lemma A.3. Under the non-calibration assumption such as the Regularity Conditions, we have that for some c > 0 and x ∈ [0 , 1] ,

Proof. We use the notations from the proof of Lemma A.1 such as Z x ( U, δ ) , and denote that E [ Z x ( U, δ )] ≡ E U,δ | z [ Z x ( U, δ )] . Using the proof in Lemma A.2, we have

Furthermore, the right term of Eqn. (9) is

By the Regularity Condition 2, we can pick x ∗ for x such that ∣ ∣ ∣ F ( F -1 ˆ θ ( x ∗ ) ) -x ∗ ∣ ∣ ∣ = η > 0 .

by the triangle inequality. Note that

Therefore, we can let ∣ ∣ ∣ ∣ ∫ x ∗ 0 { F ( F -1 ˆ θ ( s ) ) -s } 1 -x ∗ 1 -s dG ( F -1 ˆ θ ( s ) ) ∣ ∣ ∣ ∣ = c ′ η for some c ′ ∈ (0 , 1) . Finally, we have

where c = ∣ ∣ ∣ ∣ 1 -∫ x ∗ 0 dG ( F -1 ˆ θ ( s ) ) -c ′ ∣ ∣ ∣ ∣ . Furthermore, c ∈ (0 , 1) by Regularity Condition 2. This completes the proof.

Lemma A.4. Under the Regularity Conditions, we have

goes to 1 as N →∞ .

Proof. By Eqn. (4), replacing x with the E [ ˜ F ( x )] , we have that for any δ > 0 ,

as N → ∞ . Lemma A.3 implies ∣ ∣ ∣ E [ ˜ F ( x ∗ )] -x ∗ ∣ ∣ ∣ ≥ cη for x ∗ in the Regularity Conditions. Consequently, the probability of ∣ ∣ ∣ ˜ F ( x ∗ ) -x ∗ ∣ ∣ ∣ ≥ cη/ 2 goes to 1 as N → ∞ . It suffices for the proof.

B Likelihood function

B.1 DeepSurv

The classical Cox PH model (Cox, 1972), λ ( t | z ) = λ 0 ( t ) exp ( β ⊺ z ) , assumes linearity for the effect of the variables z . Here, we can adopt a DNN to estimate the effects of variables beyond linearity. The hazard function is then modeled as λ ( t | z ) = λ 0 ( t ) exp { g ( z ; θ ) } where g is a nonlinear function of z obtained through the DNN. The original Cox PH model optimizes the logarithm of the partial likelihood which is given by

where R i = { j : Y j ≥ Y i } is a risk set and β is the vector of regression coefficients. Maximizing this partial likelihood or minimizing the negative log-partial likelihood yields the estimated coefficients ˆ β . If we adopt DeepSurv (Katzman et al., 2018), the negative log-partial likelihood is changed to -∑ N i =1 δ i [ g ( z i ; θ ) -log ∑ j ∈ R i exp { g ( z j ; θ ) } ] for the loss function. In DeepSurv, we only use the estimated g ( z i ; θ ) to calculate the C-index since the baseline hazard function λ 0 ( t ) is equivalent for all subjects if the time t is the same. But to get ˆ F θ , we need to estimate λ 0 ( t ) . Rather than estimating itself, we estimate the cumulative baseline hazard function Λ 0 ( t ) using the Breslow estimator, denoted by ˆ Λ 0 ( t ) (Lin, 2007). Then, by using the relationship between the distribution function and the cumulative hazard function, we have

It is known that ˆ Λ 0 ( t ) is consistent for Λ 0 ( t ) . However, ˆ F θ includes the effect of z , which tells us ˆ F θ is not consistent for F , even when ˆ Λ 0 ( t ) is consistent. Therefore, we need to consider the calibration. Instead, the calibration error we observed is lower than that of other models.

B.2 MTLR

MTLR (Yu et al., 2011) models the probability of failure using a logistic function. Let ( t 1 , t 2 , . . . , t B ) be the previously determined time points, ( y 1 , y 2 , . . . , y B ) be the survival status according to ( t 1 , t 2 , . . . , t B ) , and ( s 1 , s 2 , . . . , s N ) be the actual survival times of the subjects. The probability of failure after the l -th time point is modeled as (1 + exp ( -θ ⊺ l z )) -1 . When the survival time is observed over the l -th point, the survival time is vectorized into (0 , 0 , . . . , 0 ︸ ︷︷ ︸ # of l points , 1 , 1 , . . . , 1) when δ = 1 , and (0 , 0 , . . . , 0 ︸ ︷︷ ︸ # of l points ) when δ = 0 . For the uncensored case, we obtain the probability of observing the

For the censored case, we only know that the failure would be observed after the censoring time s i that we have observed. Thus, if we denote t l as the closest time point following s i , the likelihood of a

censored subject is given by

By combining these, the negative log-likelihood for the loss function is defined as

Here, f θ , S θ are the conventional notations, not the true probability density function and survival function of failure time. For more details, refer to Yu et al. (2011). For calibration error, ˆ F θ would be obtained, which is accomplished by

where t ∈ [ t l , t l +1 ) . We use B = 20 for all experiments, except for SEER-stomach and SEER-lung, where B = 10 is used.

B.3 Parametric model

We use the parametric model proposed by Goldstein et al. (2020), referred to as LognormalNN. We parametrize the location and scale parameters ( µ, σ ) of the lognormal distribution. Let f and S be the probability density function and survival function of the lognormal distribution, respectively, with parameters estimated by DNN. The negative log-likelihood is as ℓ ( θ ) = -∑ N i =1 [ δ i log f θ ( y i | z i ) + (1 -δ i ) log S θ ( y i | z i )] where θ = ( µ, σ ) ⊺ and y i is an observed time of subject i . Then, we calculate the distribution function with two parameters at the given observed time.

B.4 CRPS

where y is an observed time and δ is a censoring indicator. See Appendix B in Avati et al. (2020) for a lognormal distribution. Since there is a closed form for the loss, we use the lognormal distribution for ˆ F θ .

B.5 DeepHit

DeepHit (Lee et al., 2018) is a discrete time survival model that estimates the probability mass function P ( T = t | z ) over discrete time intervals. Let T max denote the maximum time point. Given z , the model outputs a probability vector ˆ p = (ˆ p 1 , . . . , ˆ p T max ) with ∑ T max t =1 ˆ p t = 1 using the softmax function. The model is trained to minimize the negative log-likelihood. For an uncensored sample with event time t , the loss is -log ˆ p t , and for a censored sample with censoring time t , the loss is -log ∑ T max s = t +1 ˆ p s , corresponding to the estimated survival probability ˆ S ( t | z ) . We adopt DeepHit in the single-event setting and train it using only the log-likelihood loss, without the ranking loss originally proposed in Lee et al. (2018). Similar to MTLR, DeepHit depends on the number of bins, and we use a different number of bins depending on the dataset.

B.6 Weibull AFT model

The Weibull AFT model (Stute, 1993) assumes the following regression form:

where ϵ follows a standard Gumbel distribution. Under this formulation, T follows a Weibull distribution with a scale parameter exp( β ⊺ z ) and a shape parameter 1 /σ . The survival function is given by:

The likelihood function is defined similarly to the parametric model.

C Proof for Proposition 5.1

In this section, we demonstrate that if survival curves do not cross, the KSP preserves the ordering of mean survival times. Furthermore, we show that this property holds for both DeepSurv and the Weibull AFT model after applying KSP.

First, recall that for two individuals with variables z i and z j , if

then the CDFs after the KSP also satisfy F ∗ ( s | z i ) < F ∗ ( s | z j ) due to the monotonicity of the transformation.

For a non-negative random variable T , it is known that its expectation can be written as

Applying this identity to both the original and the transformed CDFs, we obtain

where T ∗ ∼ F ∗ . Thus, the ordering of the mean survival times is preserved; consequently, the C-index based on the mean survival time remains unchanged after KSP.

DeepSurv Under the PH assumption, the ordering of mean survival times is preserved. The model assumes that

where g ( z ; θ ) is the log-risk function. Hence, the C-index derived from the risk scores is preserved under KSP.

Weibull AFT The Weibull AFT model defines the survival function as

Therefore, the expected survival times maintain their original ordering.

And,

D Data generation and description

We first summarize the ten datasets used in this study. Then, we describe each dataset in detail, along with the corresponding pre-processing steps.

D.1 Details for WHAS, METABRIC, GBSG, and SUPPORT

The DeepSurv package (Katzman et al., 2018) in Python includes four pre-processed real-world datasets: the Worcester Heart Attack Study (WHAS), Molecular Taxonomy of Breast Cancer International Consortium (METABRIC), German Breast Cancer Study Group (GBSG), and the Study to Understand Prognoses Preferences Outcomes and Risks of Treatment (SUPPORT). To ensure consistency across experiments, we combined the training and test sets provided by the package and re-split the data into training, validation, and test sets in a 3:1:1 ratio. No additional data pre-processing was performed.

D.2 Details for NACD

The Northern Alberta Cancer Dataset (NACD) (Qi et al., 2024a) includes patients diagnosed with various cancers, such as lung, colorectal, head and neck, esophageal, stomach, and other types of cancer. The event of interest in this dataset is failure time. We obtain the dataset from http://pssp.srv.ualberta.ca under the 'Public Predictors" section. It consists of 2,396 patients with 51 features. For our experiments, we split the dataset into training, validation, and test sets (1,436 / 480 / 480), ensuring that the censoring rate remains balanced at 36%.

D.3 Details for NB-SEQ

NB-SEQ consists of neuroblastoma data and other supervised penalty learning benchmarks specifically designed for censored regression in the context of supervised penalty function learning for change-point detection. The dataset includes various summary statistics, as well as the minimum and maximum values of λ , achieving high performance, where λ is a hyper-parameter for the penalty function. The primary response variable (considered a failure time) is the maximum value of the estimated λ . It is important to note that the maximum value of λ is not directly observed in some cases; we only have information that the value exceeds a certain threshold, indicating that the true value is censored. We analyze 4,789 sequence data points with 24 variables. The list of the 24 variables used in the NB-SEQ dataset is provided in Table 4. All the variables are continuous variables. The censoring rate is 31%. The dataset is partitioned into sizes 2,873, 958, and 958 subsets.

D.4 Details for MIMIC-III

MIMIC-III consists of de-identified health-related information from patients admitted to the Beth Israel Deaconess Medical Center (BIDMC) in Boston, Massachusetts, USA, with a primary focus on intensive care unit (ICU) admissions. Following the pre-processing steps introduced in Harutyunyan et al. (2019), provided by https://github.com/YerevaNN/mimic3-benchmarks. This dataset involves

quartile.25. / quartile.50 / quartile.75 / quartile.100 / mean / sd / bases / data / log.1.quartile.25 / log.1.quartile.50 / log.1.quartile.75 / log.1.quartile.100 / log.1.mean / log.1.sd / log.1.bases / log.1.data / log.quartile.100 / log. mean / log.sd / log.bases / log.data / log.log.quartile.100 / log.log.bases / log.log.data multiple instances of length-of-stay data, with each subject having repeated measures at different time points. After excluding ICU transfers and patients under 18, the training and test sets consist of 2,925,434 and 525,912 instances, respectively. Recognizing that using the provided data directly is not suitable due to the correlation among data points for a single subject, we undertake additional steps. First, we exclude variables with excessively high missing rates. Next, we create one instance per subject by defining survival time as the duration between enrollment and discharge, which has no censoring. The dataset includes a list of 15 variables, as shown in Table 5. For variables that are repeatedly measured, we transform them into a single representative value, such as the mean for continuous variables and the mode for categorical variables. Finally, we exclude subjects whose variables are still missing, even after applying imputation, resulting in a dataset with 8,873 instances that combines the training set with the test set. We partition the entire dataset into training, validation, and test sets, with sizes of 5,353, 1,785, and 1,785, respectively.

D.5 Details for SEER-liver, SEER-stomach, and SEER-lung

The Surveillance, Epidemiology, and End Results (SEER) Program dataset (National Cancer Institute, DCCPS, Surveillance Research Program, 2015) is a large-scale, population-based cancer registry that covers approximately 49% of the U.S. population (Qi et al., 2024a). It contains comprehensive information on cancer incidence, patient demographics, treatments, and survival outcomes. In this study, we utilize three distinct subsets from the SEER database-SEER-liver, SEER-stomach, and SEER-lung-which respectively include patients diagnosed with liver, stomach, and lung cancers. The objective is to model the time from cancer diagnosis to a failure event, specifically death, with time measured in months. For our analysis, we selected a subset of clinically relevant variables: Sex, Race, Summary stage, Malignant behavior, Record number, Total tumor size, and Age at diagnosis. Variables with high missing rates or a large number of categories were excluded, and patients without recorded follow-up (due to immediate death) were removed. Additionally, we applied filtering criteria to exclude patients with extreme or implausible values. Specifically, we retained only patients with a record number less than or equal to 8, a total tumor size less than or equal to 13, and a survival time greater than or equal to 10 months. The dataset can be downloaded from https://seer.cancer.gov/.

E Comparison of post-processing methods: Figure

In this section, we present the results across real datasets using figures. The error bars represent 95% confidence intervals.

CSD

C-index'

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

0.8425

111

0.020

0.015

0.0125

0.0100

0.20

0.15

0.010

0.200

0.175

0.8350

0.005

0.010

0.000

0.0025

0.0075

0.0050

0.10

0.05

0.005

0.000

0.150

0.125

CSD

iPOT

orated

KSP

CSD

0-iPOT

KSP

CSD

iPOT

KSP

KSP

O-iPOT

KSP

CSD

Non

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

0.816

111

11

0.005

0.004

0.003

0.002

0.001

0.0100

0.0075

0.0050

0.0025

0.12

0.15

0.09

0.06

0.008

0.006

0.004

0.002

0.18

0.16

0.20

0.14

0.000

KSP

CSD

Non-

KSP

CSD

KSP

KSP

Non-

KSP

Non-

KSP

CSD

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

0.84

0.03

0.020

0.03

0.200

0.83

0.02

0.015

0.2

0.02

0.175

WHAS,

0.81

0.01

0.010

0.01

0.150

0.80

0.00

0.005

0.00

+

0.125

KSP

CSD

KSP

CSD

KSP

CSD

-IPOT

KSP

KSP

CSD

KSP

CSD

Non-calirted

Non-calibr

Non-calibr

Non-calibtr

Non-calibr

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

CRPS

0.78

0.80

0.81

0.04

0.02

0.00

0.15

0.10

0.05

0.00

0.3

0.2

0.1

0.06

0.09

0.03

0.00

0.30

0.25

0.15

0.20

KSF

CSD

KSP

Non

KSP

Non-

KSP

CSD

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

0.848

0.0020

0.005

T

0.005

亚

0.200

0.840

0.0015

0.0010

0.0005

0.004

0.003

0.002

0.2

0.1

0.002

0.004

0.003

0.001

0.175

0.150

0.125

CSD

KSP

CSD

KSP

CSD

Non-calibre

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

E

0.8200

0.8150

0.15

0.10

0.05

0.12

0.09

0.06

0.03

0.4

0.6

0.2

0.20

0.15

0.10

0.05

0.3

0.4

0.2

0.00

0.00

0.0

0.00

Non

KSP

CSD

CSD-

0-iPOT

KSP

-iPOT

iPOT

Non

KSP

CSD

CSD-iPOT

CSD

CSD-iPOT

Non-calibr

METABRIC, DeepSurv

C-indext

S-cal(20)

D-cal(20)↓

KS-cal

KM-cal

IBS↓

0.645

0.640

0.650

111

6e-04

4e-04

2e-04

0.0030

0.0025

0.0020

0.0010

0.0015

直

0.05

0.04

0.03

T

5e-04

1e-03

0.18

0.17

0.16

?

KSP

CSD

CSD-IPOT

Non-calibr

orated

KSP

CSD

KSP

CSD

-iPOT

Non-

CSD

CSD

Non-calibn

KSP

CSD

C-index

S-cal(20)

D-cal(20)

KS-cal↓

KM-cal

IBS↓

MTLR

0.625

0.635

11

11

0.00100

0.00075

0.00125

0.00050

0.00025

T

T

0.00250

0.00225

0.00200

0.00175

T

0.045

0.055

0.050

0.040

T

TI

T

0.0025

0.0030

0.0020

0.0015

0.0010

0.18

0.17

0.16

KSP

CSD

CSD-iPOT

KSP

KSP

CSD

KSP

KSP

CSD

CSD-iPOT

KSP

Non-calibrated

Non-

Parametric

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

0.63

0.62

T

T

0.010

0.008

0.006

0.15

0.010

0.185

0.180

METABRIC,

0.60

0.61

0.005

0.000

0.004

0.002

0.10

0.05

0.005

0.165

0.175

0.170

KSP

CSD

CSD-iPOT

Non-calibrated

KSP

CSD

CSD

O-iPOT

Non-calibr

KSP

CSD

rated

KSP

CSD

-iPOT

Non-calibrated

KSP

CSD

METABRIC,CRPS

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

0.60

0.61

0.59

T

T1

T

T

0.075

0.050

0.025

0.000

0.20

0.15

0.10

0.05

0.00

0.4

0.3

0.2

0.1

0.20

0.15

0.10

0.05

0.00

0.40

0.35

0.30

0.25

0.20

KSP

CSD

KSP

CSD

KSP

CSD

KSP

KSP

CSD

KSP

CSD

Non-calibrated

METABRIC. DeepHit

C-index

S-cal(20);

D-cal(20)

KS-cal↓

KM-cal

IBS↓

111

0.645

0.640

0.635

6e-04

7e-04

5e-04

4e-04

0.0025

0.0020

0.050

0.045

0.055

T

0.00150

0.00125

0.00100

L

T

人

0.17

0.18

0.630

3e-04

0.040

0.00075

0.16

SP

CSD

KSP

CSD

CSD-iPOT

KSP

KSP

Non-calibra

Non-calib

Non-calib

Non-calibra

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal*

IBS↓

METABRIC,A

AFT

0.635

0.15

0.15

0.6

0.15

0.30

0.630

0.625

0.10

0.05

0.10

0.05

0.4

0.2

0.10

0.05

0.25

0.20

0.00

0.00

0.00

KSP

CSD

CSD-iPOT

CSD

Non-

CSD

-POT

CSD

Non-

KSP

CSD

CSD-IPOT

CSD

Non-calibr

C-index

S-cal(20)

D-cal(20)↓

KS-cal

KM-cal

IBS↓

0.668

0.676

5e-04

1e-03

0.0030

0.0025

0.0020

0.0010

0.0015

亚

0.06

0.04

0.07

0.05

0.03

0.0020

0.0010

0.0015

0.0005

0.0000

亚

0.200

0.195

0.190

0.185

0.180

0.175

Non-calibn

KSP

CSD

CSD-iPOT

Non-calibr

KSP

CSD

KSP

CSD

O-iPOT

orated

KSP

CSD

KSP

CSD

Non-calibrn

KSP

C-index'

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

0.672

0.660

T

T

0.010

0.015

0.005

0.05

0.04

0.03

0.02

0.01

0.3

0.2

0.1

0.03

0.02

0.01

0.21

0.20

0.19

0.000

0.00

0.00

KSP

CSD

KSP

KSP

KSP

ated

KSP

KSP

Non-

Non-

Non-

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

0.640

0.004

0.005

0.003

0.002

0.001

0.000

0.004

0.003

0.002

0.09

0.05

0.03

0.07

0.000

0.006

0.004

0.002

0.195

0.200

0.190

0.185

0.180

?

orated

KSP

CSD

O-iPOT

Non-calibrated

KSP

CSD

O-IPOT

Non-calib

CSD

Non-calibr

KSP

CSD

KSP

CSD

Non-calibrated

KSP

CSD

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

0.61

T

T

0.06

0.04

0.10

0.3

0.4

0.100

0.075

0.30

0.27

0.02

0.05

0.2

0.1

0.050

0.025

0.24

0.21

0.58

0.00

0.00

0.000

KSP

CSD

Non-calibra

KSP

Non-

KSP

CSD

Non-callbrated

KSP

CSD

Non-

KSP

CSD

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

0.6700

0.010

0.005

0.05

0.03

0.02

0.04

0.01

0.3

0.2

0.1

T

0.025

0.020

0.015

0.010

0.005

0.21

0.20

0.19

0.000

0.00

0.000

KSP

CSD

KSP

CSD

KSP

CSD

rated

KSP

CSD

Non-calib

Non-calibre

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal*

IBS↓

0.668

11

0.15

0.10

0.10

0.6

0.4

0.15

0.10

0.35

0.30

0.05

0.05

0.2

0.05

0.25

0.00

0.00

0.0

0.00

0.20

KSP

CSD

CSD-iPOT

KSF

CSD

O-iPOT

-calibr

KSF

CSD

IPOT

CSD

KSP

CSD

Non-

Non

Non-

CSD-iPOT

C-index'

S-cal(20)*

D-cal(20)

KS-cal

KM-cal

IBS↓

DeepSurv

0.7450

0.7475

111

0.0050

0.0075

0.008

0.006

0.125

0.100

T

0.003

0.004

IIIE

0.17

0.18

NACD,

0.7425

0.0025

Q.00

0.075

0.002

0.16

0.0000

0.050

0.001

0.15

?

?

CSD

KSP

CSD

iPOT

-calibr

KSP

CSD

CSD

KSP

CSD

Non

Non-calibre

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

0

0.7475

.MTLR

.7525

0.7550

11

0.002

0.003

0.001

+000

0.005

0.003

0.002

亚

亚

0.08

0.06

0.07

0.05

0.04

亚_

0.0075

0.0050

0.0025

0.16

0.18

0.17

0.15

亚

KSP

CSD

KSP

KSP

CSD

KSP

CSD

CSD-iPOT

KSP

CSD

KSP

CSD

Non-

Non-calibrated

Non-o

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

NACD,Parametric

0.740

0.003

0.002

0.006

0.10

0.08

0.003

0.002

0.18

0.17

0.735

0.001

0.004

0.002

0.06

0.04

0.001

0.16

0.15

KSP

CSD

KSP

CSD

KSH

CSD

KSP

CSD

KSP

CSD

KSP

CSD

Non-calibr

Non-calib

Non-calibr

Non-calibre

C-index

S-cal(20)*

D-cal(20)

KS-cal

KM-cal

IBS↓

NACD, CRPS

0.73

T

0.04

0.03

0.02

0.06

0°0

0.3

0.2

0.075

0.050

0.250

0.225

0.200

0.72

0.01

0.02

0.1

0.025

0.175

0.00

0.00

0.000

0.150

CSD

KSP

KSP

KSP

CSD

KSP

CSD

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

0.7350

TT

5e-04

1e-03

0.004

0.003

0.002

0.06

0.04

0.05

0.004

0.003

0.002

0.001

立

0.17

0.15

0.18

0.16

CSD

O-POT

KSP

CSD

CSD-POT

Non

Non-calit

Non-calibr

C-index~

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

0.7575

11

0.15

0.10

0.6

0.06

0.08

0.24

0.22

0.7500

0.7475

0.10

0.05

0.00

0.05

0.00

0.4

0.2

0.04

0.02

0.00

0.20

0.18

0.16

0.14

章

Non

-calib

KSP

CSD

KSP

KSP

CSD

Non-

-calibrated

CSD

-iPOT

C-index

S-cal(20)

D-cal(20)↓

KS-cal

KM-cal

IBS↓

0.744

0.740

111

0.00050

0.00025

0.00075

0.004

0.003

0.001

0.002

0.06

0.05

0.04

0.02

0.03

4e-04

2e-04

6e-04

亚

0.065

0.050

0.060

0.055

Non-calibr

KSP

CSD

CSD-iPOT

Non-calibr

KSP

CSD

CSD-iPOT

KSP

CSD

KSF

CSD

-iPOT

Non-calibr

orated

KSP

CSD

CSD

Non-calibr

KSP

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

068

0.680

11

6e-04

4e-04

2e-04

0.005

0.004

0.003

0.001

0.002

車

0.06

0.05

0.04

0.03

6e-04

8e-04

4e-04

2e-04

1

0.066

0.060

0.058

0.056

0.064

0.062

KSP

CSD

-calibr

KSP

CSD

-iPOT

KSP

CSD

KSP

KSP

KSP

Non-

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal*

IBS↓

[I

0.020

0.20

0.04

0.09

0.015

0.04

0.15

0.03

0.08

0.714

0.718

0.005

0.010

0.000

0.02

0.00

0.05

0.10

0.02

0.01

0.00

0.06

0.07

KSP

CSD

KSP

CSD

KSP

CSD

KSP

CSD

CSD

KSP

CSD

Non-calibr

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

.748

0.750

0.744

0.05

0.02

0.01

0.04

0.03

0.08

0.06

0.04

0.02

0.4 [

0.3

0.2

0.1

0.08

0.06

0.04

0.02

0.12

0.10

0.08

0.06

0.00

0.00

0.00

Non-

KSP

CSD

Non-

KSP

CSD

O-iPOT

KSP

CSD

Non-calibrated

KSP

Non-calibrated

KSP

Non-calibrated

KSP

C-index

S-cal(20)*

D-cal(20)

KS-cal

KM-cal

IBS↓

0.7450

0.7475

0.7400

4e-04

3e-04

6e-04

5e-04

2e-04

1e-04

0.005

0.004

0.003

0.002

0.001

0.06

0.05

0.04

0.03

0-89

4e-04

2e-04

1

0.065

0.055

0.050

0.060

CSD

-iPOT

KSP

CSD

O-iPOT

Non-calibr

KSP

CSD

Non-calibr

rated

KSP

CSD

Non-calibrn

KSP

CSD

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

0.20

0.15

0.2

0.3

0.6

0.15

0.25

0.20

0.10

0.4

0.10

0.15

0.05

0.1

0.2

0.05

0.10

0.00

0.0

0.0

0.00

-calib

KSP

CSD

-iPOT

iPOT

KSP

CSD

ated

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

0.614

0.615

111

3e-04

0.0016

亚

0.040

0.035

亚

5e-04

4e-04

0.21

SUPPORT,

0.612

0.610

0.611

II

2e-04

1e-04

0.0012

0.0004

0.0008

0.030

0.025

0.020

亚

0e+00

3e-04

2e-04

1e-04

0.20

0.19

亚

亚

KSP

CSD

KSP

CSD

KSP

CSD

D-iPOT

KSP

CSD

IPOT

KSP

CSD

KSP

Non-calibr

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS

SUPPORT,MTR

0.572

0.574

0.576

T

0.00050

0.00075

0.004

0.003

0.002

0.05

0.06

0.04

0.0125

0.0100

0.0075

0.0050

0.220

0.215

0.570

0.00025

0.001

0.02

0.03

0.0000

0.0025

0.205

0.210

KSP

CSD

Non-

Non-

KSP

Parametic

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

SUPPORT,

0.595

0.605

0.600

0.590

0.0025

0.0020

0.0015

0.0005

0.0010

0.0000

0.004

0.003

0.001

0.002

0.08

0.06

0.04

0.02

0.006

0.004

0.002

0.000

0.21

0.20

KSP

CSD

ated

KSP

CSD

KSP

KSP

CSD

KSP

Non-calib

Non-calibt

C-index

S-cal(20)*

D-cal(20)

KS-cal

KM-cal

IBS↓

CRPS

SUPPORT,(

0.57

0.56

0.55

0.10

0.05

0.4

0.3

0.2

0.1

0.6

0.4

0.2

0.4

0.3

0.2

0.1

0.6

0.5

0.4

0.3

0.00

0.0

0.0

0.0

0.2

KSF

KSP

KSP

CSD

Non-

DeepHit

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

SUPPORT,

0.602

0.604

0.600

0.00100

0.00050

0.00075

0.00025

1

0.003

0.002

0.001

0.06

0.05

0.04

0.03

0.009

0.006

0.003

0.215

0.210

0.205

0.200

T

0.02

0.000

0.195

CSD

CSD

CSD

-iPOT

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

SUPPORT,AFT

0.572

0.570

TI

0.10

0.15

0.100

0.075

0.050

0.6

0.4

0.04

0.26

0.24

0.568

0.05

0.025

0.2

0.02

0.22

0.00

0.000

0.0

0.00

-calibr

KSP

CSD

Non-

KSP

CSD

Non

-iPOT

Non-

KSP

iPOT

CSD

Non-calibn

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

0.959

0.960

111

0.04

0.03

0.02

0.06

0.04

0.2

0.09

0.06

0.3

0.2

0.957

0.956

0.01

0.02

0.00

工

0.1

0.03

0.00

0.1

KSP

CSD

KSP

CSD

KSP

CSD

-IPOT

KSP

CSD

-iPOT

CSD

Non-calibr

Non-calibr

Non

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

,MTLR

0.946

0.015

0.03

0.25

0.20

6e-04

亚

0.04

MIMIC-I,

0.944

0.010

0.02

0.15

4e-04

亚

0.03

0.942

0.005

亚

0.01

0.10

0.05

亚

2e-04

0.02

0.000

0.00

0e+00

=

0.01

KSP

KSP

CSD

ated

KSP

CSD

KSP

KSP

UON

Non-

C-index

S-cal(20)↓

D-cal(20)

KS-cal

KM-cal

IBS↓

0.9630

0.9615

0.9620

0.9610

0.04

0.03

0.02

0.01

0.00

0.12

0.08

0.04

0.00

0.3

0.2

0.1

0.00020

0.00010

0.00015

0.00005

T

0.04

0.03

0.02

0.01

KSP

CSD

Non

KSP

CSD

KSP

CSD

KSP

CSD

KSP

CSD

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

0.956

0.20

0.8

0.020

0.15

0.2

0.6

0.015

0.06

0.952

0.10

0.05

0.00

0.1

0.4

0.2

0.0

0.010

0.005

0.000

0.02

0.04

KSP

CSD

CSD-IPOT

KSF

CSD

KSP

KSP

CSD

KSP

-calibrated

KSP

CSD

Non-

Non-calibrated

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

MIMIC-II,

0.9645

0.9640

0.9635

0.9650

IIII

0.03

0.02

0.01

0.00

0.05

0.04

0.03

0.02

0.01

0.00

0.20

0.15

0.10

0.05

?

0.000100

0.000125

0.000075

0.000050

0.03

0.02

0.04

0.01

CSD

CSD

CSD

-iPOT

CSD

KSP

CSD

Non-calibri

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

0.883

0.882

0.15

0.20

0.4

0.3

0.8

0.6

0.020

0.015

0.06

0.05

0.10

0.2

0.4

0.010

0.04

0.879

0.05

0.1

0.2

0.005

0.03

0.00

0.0

0.0

0.000

0.02

Non

-Calib

KSP

CSD

CSD-iPOT

KSF

CSD

ated

-iPOT

KSP

CSD

-iPOT

Non-

KSP

CSD

CSD-iPOT

SEER-liver, DeepSurv

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

111

0.635

0.00020

0.05

亚

0.155

0.634

0.633

0.632

0.00010

0.00015

0.00005

0.004

0.002

0.000

0.04

0.02

0.03

0.01

0e+00

4e-04

2e-04

0.150

0.145

0.140

亚

亚

KSP

CSD

CSD-iPOT

KSP

CSD

O-iPOT

KSP

CSD

KSP

CSD

KSP

CSD

KSP

CSD

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

MTLR

0.630

0.629

0.632

11

3e-04

5e-04

4e-04

2e-04

0.005

0.004

0.003

0.002

0.001

0.045

0.050

0.040

0.035

0.003

0.002

0.001

0.000

0.150

0.145

0.155

0.140

亚

KSP

CSD

KSP

CSD

KSP

CSD

KSP

KSP

KSP

CSD

-iPOT

Non-

Non-

C-index

S-cal(20)

D-cal(20);

KS-cal

KM-cal*

IBS↓

0.634

0.0012

0.005

0.06

0.0020

0.155

0.632

0.0009

0.004

0.05

0.0015

0.150

SEER-liver,

0.630

0.628

0.0006

0.0003

0.003

0.001

0.002

0.03

0.02

0.04

0.0010

0.0005

0.0000

0.145

0.140

1

亚

亚

rated

CSD

KSP

CSD

KSP

CSD

KSP

CSD

KSP

CSD

KSP

CSD

Non-calibt

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal↓

IBS↓

工

0.15

0.4

0.6

0.4

0.10

0.4

0.2

0.3

0.05

0.2

0.1

0.2

0.00

0.0

0.0

KSP

CSD

KSP

KSP

KSP

KSP

CSD

Non

Non

Non-calibrated

SEER-liver, DeepHit

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal↓

IBS↓

0.635

6e-04

5e-04

0.005

0.004

0.050

事

0.003

0.155

0.634

4e-04

0.003

0.045

0.002

0.150

0.633

0.632

3e-04

2e-04

0.002

0.001

0.035

0.040

0.001

0.000

0.145

0.140

?

CSD

0-iPOT

KSP

CSD

-iPOT

orated

KSP

CSD

-iPOT

KSP

CSD

KSP

KSP

CSD

Non-calibrae

Non-calibr

Non-calit

C-index

S-cal(20)

D-cal(20);

KS-cal

KM-cal

IBS↓

AFT

0.628

0.0015

0.0020

0.0010

0.0025

0.0005

0.0000

0.006

0.008

0.004

0.002

0.08

0.06

0.04

0.004

0.003

0.002

0.001

0.000

0.150

0.155

0.140

0.145

亚

KSP

CSD

Non-

-calibr

KSP

CSD

Non

CSD

CSD

Non-

KSP

CSD

CSD-iPOT

Non-calibr

Non-calibr

DeepSurv

C-index!

S-cal(20);

D-cal(20)↓

KS-cal

KM-cal

IBS↓

SEER-stomach,

0.702

8e-04

6e-04

0.004

0.006

0.04

0.05

8e-04

6e-04

业

0.18

0.17

0.701

4e-04

2e-04

0.002

0.03

0.02

4e-04

2e-04

0.16

0.15

0.700

0e+00

0.000

0.01

0e+00

0.14

KSP

CSD

KSP

CSD

KSP

CSD

-iPOT

Non-calibrated

KSP

CSD

CSD

Non-calibr

orated

KSP

CSD

KSP

CSD

CSD

SEER-stomach,MTLR

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

0.695

0.696

111

6e-04

5e-04

4e-04

3e-04

2e-04

1e-04

0.003

0.004

0.002

0.001

0.052

0.048

0.044

0.040

0.036

0.002

0.001

0.000

0.18

0.17

0.16

0.15

KSP

CSD

KSP

CSD

KSP

IPOT

KSP

CSD

KSP

Non-

Non-

Non-

Non-

Parametric

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

SEER-stomach,

0.699

0.697

0.701

T

?

0.00075

0.00050

0.00100

0.00025

亚

0.003

0.002

0.001

0.04

0.02

0.05

0.03

亚

0.0015

0.0010

0.0005

0.0000

0.14

0.18

0.17

0.16

0.15

ated

CSD

ated

KSP

CSD

KSP

-iPOT

KSP

CSD

KSP

CSD

KSP

Non-calib

Non-calibrated

Non-calit

CRPS

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

SEER-stomach,

0.677

0.679

0.675

0.10

0.15

0.4

0

0.2

0.6

0.4

0.2

0.45

0.40

0.35

0.30

0.673

0.05

0.00

0.1

0.0

0.2

0.1

0.0

0.25

0.15

0.20

Non

KSP

CSD

KSP

KSP

CSD

CSD-POT

Non-calibrated

KSP

CSD

Non-calibrated

KSP

DeepHit

C-index

S-cal(20)

D-cal(20)

KS-cal↓

KM-cal

IBS↓

0.704

5e-04

4e-04

0.004

0.003

0.050

0.045

0.003

0.002

0.18

0.17

0.703

3e-04

2e-04

0.002

0.001

0.040

0.001

0.16

0.15

0.702

1e-04

0.000

0.14

CSD

CSD

D-iPOT

KSP

CSD

KSP

CSD

KSP

CSD

KSP

CSD

Non-calibr

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

0.6910

0.0012

0.006

0.004

0.06

0.05

0.002

章

0.18

0.17

0.0008

0°0

0.001

0.16

0.6890

0.0004

0.002

0.03

0.15

0.000

Non

KSP

CSD

CSD-IPOT

KSP

CSD

CSD

Non

KSP

CSD

DeepSurv

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

SEER-lung,

0.6810

0.6805

111

0.00025

0.00020

0.00015

0.00010

0.00005

0.006

0.004

0.000

0.002

0.04

0.03

0.02

0.01

0.05

0e+00

6e-04

4e-04

2e-04

0.120

0.130

0.125

0.115

0.110

KSP

CSD

iPOT

-calibr

CSD

CSD

D-iPOT

KSP

CSD

O-iPOT

Non-calibrated

KSP

CSD

Non-calibrated

KSP

CSD

Non-calibr

KSP

CSD

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

, MTLR

SEER-lung,

0.6764

0.6768

0.6772

T

T

T1

4e-04

3e-04

2e-04

0.005

0.004

0.003

0.002

0.052

0.050

0.048

0.046

6e-04

4e-04

2e-04

0.130

0.125

0.120

0.115

1e-04

0.001

0.044

0e+00

0.110

KSP

CSD

iPOT

KSP

CSD

KSP

CSD

KSP

CSD

KSP

KSP

CSD

Non-

Non-calibr

Parametric

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal

IBS↓

8e-04

0.681

6e-04

0.004

0.045

0.050

1e-03

0.125

0.130

0.679

4e-04

2e-04

0.001

0.003

0.002

0.040

0.035

0.030

0.025

0e+00

5e-04

0.120

0.105

0.115

0.110

KSP

CSD

ated

KSP

CSD

KSP

CSD

KSP

CSD

ated

KSP

CSD

KSP

CSD

Non-calibt

Non-calibr

Non-calibr

C-index

S-cal(20)

D-cal(20)

KS-cal

KM-cal*

IBS↓

0.6600

0.6625

工

0.20

0.15

0.10

0.05

0.5

0.6

0.4

0.2

0.3

0.2

0.1

0.3

0.4

0.2

0.00

0.0

0.0

0.1

KSP

KSP

KSP

CSD

KSP

KSP

CSD

Non-calibrated

C-index

S-cal(20)

D-cal(20)

KS-cal↓

KM-cal

IBS↓

SEER-lung,

0.6820

0.6815

0.6810

0.6825

T

3e-04

2e-04

4e-04

1e-04

0.005

0.004

0.003

0.002

0.001

0.0500

0.0475

0.0450

章

0e+00

6e-04

4e-04

2e-04

0.125

0.130

0.120

0.115

0.110

CSD

LIPOT

KSP

CSD

O-iPOT

KSP

CSD

orated

KSP

CSD

KSP

KSP

CSD

Non-calibr

Non-calibt

C-index

S-cal(20)↓

D-cal(20)

KS-cal~

KM-cal

IBS↓

SEER-lung, AFT

0.008

0.08

0.6740

0.6735

11

一

0.0020

0.0015

0.0010

0.0005

0.0000

0.006

0.004

0.002

200

0.03

0.06

0.04

0.05

0.0015

0.0010

0.0005

0.0000

0.125

0.120

0.130

0.115

0.110

KSP

CSD

CSD-IPOT

KSP

CSD

KSP

KSP

CSD

F Comparison of post-processing methods: Table

In this section, we present the results in tabular form, in contrast to Appendix E, where the same results are visualized as figures. Similar to Table 1 in the main text, we summarize the number of cases in which each post-processing method outperforms the others for each model. We implemented X-cal (Goldstein et al., 2020 in the main paper), CSD (Qi et al., 2024a), and CSDiPOT (Qi et al., 2024b) with reference to publicly available code repositories under the MIT License: https://github.com/rajesh-lab/X-CAL , https://github.com/shi-ang/CSD , and https://github.com/shi-ang/MakeSurvivalCalibratedAgain .

G Calibration plot

In this section, we present calibration plots across datasets and models.

DeepSurv

MTLR

Parametric

CRPS

DeepHit

AFT

KSP

KSP

KSP

KSP

KSP

KSP

Non-calibrated

Non-calibrated

Non-calibrated

Observed probabity

Non-calibrated

Non-calibrated

"Predicted probability

“Predicted probability

Predictedprobability

Predictedprobabilty

“Predicted probability”

Predicted probability

DeepSurv

MTLR

Parametric

CRPS

DeepHit

AFT

KSP

KSP

KSP

KSP

KSP

KSP

CSD

CSD

CSD

CSD

CSD

CSD

..........

“Predicted probability"

“Predicted probabity

Predicted probability

Predictedprobability

“Predictedprobabilty

"Predicted probability

DeepSurv

MTLR

Parametric

CRPS

DeepHit

AFT

KSP

KSP

KSP

KSP

KSP

KSP

CSD-iPOT

CSD-iPOT

CSD-iPOT

CSD-POT

CSD-iPOT

iegodaaso

CSD-iPOT

.....t..

.....

Predicted probability

Predicted probabity

“Predicted probability

Predictedprobabilty

“Predicted probability

Predicted probability

DeepSurv

DeepHit

Parametric

CRPS

DeepHit

AFT

KSP

KSP

KSP

KSP

KSP

KSP

Non-calibrated

Non-calibrated

Non-calibrated

·

Non-calibrated

+

Non-calibrated

Non-calibrated

"Predicted probabilty"

Predicted probability

“Predicted probability

Predicted probability

"Predicted probabity"

Predicted probability

DeepSurv

MTLR

Parametric

CRPS

DeepHit

AFT

KSP

KSP

KSP

KSP

KSP

KSP

CSD

CSD

CSD

CSD

CSD

CSD

.....

"Predicted probabilty"

"Predicted probabity"

“Predictedprobability

"Predictedprobabity"

"Predicted probability”

"Predicted probabity”

DeepSurv

MTLR

Parametric

CRPS

DeepHit

AFT

KSP

KSP

KSP

KSP

KSP

KSP

CSD-iPOT

CSD-iPOT

CSD-iPOT

CSD-POT

CSD-iPOT

CSD-iPOT

Predicted probability

"Predicted probabilty"

Predictedprobability

Predictedprobability

Predictedprobability

Predicted probability

DeepSurv

MTLR

Parametric

CRPS

DeepHit

AFT

KSP

KSP

KSP

KSP

Non-calibrated

Non-calibrated

Non-calibrated

Observed probabity

·

Non-calibrated

Non-calibrated

Non-calibrated

.

"Predicted probabilty"

Predicted probability

Predicted probability

"Predictedprobabilty"

“Predicted probability

Predictedprobability

DeepSurv

MTLR

Parametric

CRPS

DeepHit

AFT

KSP

+CSD

KSP

CSD

KSP

CSD

KSP

CSD

KSP

CSD

KSP

CSD

"Predicted probabilty"

"Predicted probabit"

"Predicted probabilty"

"Predicted probabity"

"Predicted probabity"

"Predicted probabity"

DeepSurv

MTLR

Parametric

CRPS

DeepHit

AFT

KSP

KSP

KSP

KSP

KSP

KSP

CSD-iPOT

CSD-iPOT

CSD-iPOT

CSD-POT

CSD-iPOT

CSD-iPOT

"Predicted probabilty"

“Predicted probability

“Predicted probability

"Predicted probabilty

Predicted probability

"Predicted probabity"

DeepSurv

MTLR

Parametric

CRPS

DeepHit

AFT

KSP

KSP

KSP

KSP

KSP

KSP

Non-calibrated

d

Non-calibrated

Non-calibrated

Non-calibrated

Non-calibrated

Non-calibrated

"Predicted probabit"

Predicted probability

"Predictedprobability

"Predictedprobabilty"

“Predicted probability

Predicted probability

DeepSurv

MTLR

Parametric

CRPS

DeepHit

AFT

KSP

CSD

KSP

CSD

KSP

CSD

KSP

CSD

KSP

CSD

KSP

CSD

........

"Predicted probabity"

"Predicted probabity"

"Predicted probabilty"

"Predicted probabity"

"Predicted probabity"

"Predicted probabity"

DeepSurv

MTLR

Parametric

CRPS

DeepHit

AFT

KSP

KSP

KSP

KSP

KSP

KSP

CSD-iPOT

CSD-iPOT

CSD-iPOT

CSD-POT

CSD-iPOT

CSD-iPOT

"Predicted probabilty"

"Predicted probabilty"

"Predicted probabilty"

Predictedprobabilty

“Predicted probability

"Predicted probabity"

DeepSurv

MTLR

Parametric

CRPS

DeepHit

AFT

KSP

KSP

KSP

KSP

KSP

Non-calibrated

Non-calibrated

Non-calibrated

Observed probabity

Non-calibrated

Non-calibrated

Non-calibrated

......

"Predicted probabilty"

Predicted probability

“Predicted probability

“Predictedprobabilit"

“Predicted probability

Predicted probability

DeepSurv

MTLR

Parametric

CRPS

DeepHit

AFT

KSP

KSP

KSP

KSP

KSP

KSP

+CSD

CSD

CSD

CSD

CSD

CSD

....!.

"Predicted probabity"

"Predicted probabity"

"Predicted probabit"

"Predicted probabity"

"Predicted probabity”

"Predicted probabity"

DeepSurv

MTLR

Parametric

CRPS

DeepHit

AFT

KSP

KSP

KSP

KSP

KSP

KSP

CSD-iPOT

CSD-iPOT

CSD-iPOT

CSD-POT

CSD-iPOT

Oobsered probabity

CSD-iPOT

"Predicted probabilty"

“Predicted probability

“Predicted probabilty”

"Predictedprobabilty"

“Predicted probability

"Predicted probabity"

DeepSurv

MTLR

Parametric

CRPS

DeepHit

AFT

KSP

KSP

KSP

KSP

KSP

KSP

Non-calibrated

do

Non-calibrated

Non-calibrated

Observed probabity

Non-calibrated

Non-calibrated

Non-calibrated

"Predicted probabity"

Predicted probability

"Predicted probabity"

Predictedprobabilty"

“Predicted probability

Predicted probability

DeepSurv

MTLR

Parametric

CRPS

DeepHit

AFT

KSP

KSP

KSP

KSP

KSP

KSP

+CSD

CSD

CSD

CSD

CSD

CSD

.......!.

"Predicted probabilty"

"Predicted probabity"

"Predicted probabilty"

Predicted probabit"

"Predicted probabity"

"Predicted probabity"

DeepSurv

MTLR

Parametric

CRPS

DeepHit

AFT

KSP

KSP

KSP

KSP

KSP

KSP

CSD-iPOT

CSD-iPOT

CSD-iPOT

CSD-POT

CSD-iPOT

Oobsered probabity

CSD-iPOT

"Predicted probabilty"

"Predicted probabilty"

“Predicted probabity”

00901

Predicted probability

“Predicted probability

"Predicted probabity”

DeepSurv

MTLR

Parametric

CRPS

DeepHit

AFT

KSP

Non-calibrated

KSP

Non-calibrated

KSP

Observed probabity

KSP

+

Non-calibrated

KSP

Non-calibrated

KSP

—Non-calibrated

............

.....

“Predicted probability"

Predicted probability

......

Predictedprobability

"Predictedprobabilty"

“Predicted probability

Predictedprobability

DeepSurv

MTLR

Parametric

CRPS

DeepHit

AFT

KSP

KSP

KSP

KSP

KSP

KSP

+CSD

CSD

CSD

CSD

CSD

CSD

........

"Predicted probabilty"

"Predicted probabity"

"Predicted probabilty"

"Predicted probabity"

"Predicted probabity"

"Predicted probabity"

DeepSurv

MTLR

Parametric

CRPS

DeepHit

AFT

KSP

CSD-iPOT

KSP

CSD-iPOT

KSP

CSD-iPOT

KSP

CSD-POT

KSP

CSD-iPOT

KSP

CSD-iPOT

..a......

.............

"Predicted probabilty"

“Predicted probability

“Predicted probability

Predicted probability

“Predicted probability”

"Predicted probabity"

DeepSurv

MTLR

Parametric

CRPS

DeepHit

AFT

KSP

KSP

KSP

KSP

KSP

KSP

Non-calibrated

do

Non-calibrated

Non-calibrated

Observed probabity

Non-calibrated

Non-calibrated

Non-calibrated

"Predicted probabilty"

Predicted probability

"Predicted probabilty

"Predictedprobabilty"

“Predicted probability

“Predicted probability

DeepSurv

MTLR

Parametric

CRPS

DeepHit

AFT

KSP

KSP

KSP

KSP

KSP

KSP

+CSD

CSD

CSD

CSD

CSD

CSD

"Predicted probabity"

"Predicted probabity"

"Predicted probabilty"

"Predicted probabity"

"Predicted probabity”

"Predicted probabity"

DeepSurv

MTLR

Parametric

CRPS

DeepHit

AFT

KSP

KSP

KSP

KSP

KSP

KSP

CSD-iPOT

CSD-iPOT

CSD-iPOT

CSD-POT

CSD-iPOT

CSD-iPOT

"Predicted probabilty"

"Predicted probabilty"

"Predicted probabilty"

Predictedprobabilty

“Predicted probability

"Predicted probabity"

DeepSurv

MTLR

Parametric

CRPS

DeepHit

AFT

KSP

KSP

KSP

KSP

KSP

KSP

Non-calibrated

Non-calibrated

Non-calibrated

Observed probability

Non-calibrated

Non-calibrated

Non-calibrated

"Predicted probabilty"

Predicted probability

"Predicted probabity"

"Predicted probability"”

“Predicted probability

"Predicted probability”

DeepSurv

MTLR

Parametric

CRPS

DeepHit

AFT

KSP

KSP

KSP

KSP

KSP

KSP

CSD

CSD

CSD

CSD

CSD

CSD

"Predicted probabity"

"Predicted probabity"

"Predicted probabilty"

"Predicted probabity"

"Predicted probabity”

"Predicted probabity"

DeepSurv

MTLR

Parametric

CRPS

DeepHit

AFT

KSP

KSP

KSP

KSP

KSP

KSP

CSD-iPOT

CSD-iPOT

CSD-iPOT

CSD-iPOT

CSD-iPOT

Oobsered probabity

CSD-iPOT

"Predicted probabilty"

?

"Predicted probabilty"

"Predicted probabilty"

"Predicted probabity"

“Predicted probability

"Predicted probabity"

DeepSurv

MTLR

Parametric

CRPS

DeepHit

AFT

KSP

KSP

KSP

KSP

KSP

KSP

Non-calibrated

Non-calibrated

Non-calibrated

Observed probabity

Non-calibrated

Non-calibrated

Non-calibrated

“Predicted probability"

Predicted probability

"Predictedprobabilty

"Predictedprobabilty"

“Predicted probability

“Predicted probability

DeepSurv

MTLR

Parametric

CRPS

DeepHit

AFT

KSP

KSP

KSP

KSP

KSP

KSP

CSD

Aiegod paiaso

CSD

CSD

CSD

CSD

CSD

"Predicted probabity"

"Predicted probabity"

"Predicted probabilty"

"Predicted probabity"

"Predicted probabity”

"Predicted probabity"

DeepSurv

MTLR

Parametric

CRPS

DeepHit

AFT

KSP

KSP

KSP

KSP

KSP

KSP

CSD-iPOT

CSD-iPOT

CSD-iPOT

CSD-POT

CSD-iPOT

CSD-iPOT

"Predicted probabilty"

"Predicted probabilty"

“Predicted probabity”

"Predictedprobabilty"

“Predicted probability

"Predicted probabity"

H Evaluation of calibration errors with 10-bin

In this section, we compare the performance of S-cal and D-cal using evaluations with 10 bins. To determine bin locations, we consider two strategies: uniformly spaced bins and a scheme that allocates denser bins in the tails. The results are reported in a format similar to Table 1 in the main manuscript. For uniformly spaced bins, we use knot points at { 0 . 1 , 0 . 2 , 0 . 3 , 0 . 4 , 0 . 5 , 0 . 6 , 0 . 7 , 0 . 8 , 0 . 9 } , while for tail-focused bins, we use { 0 . 05 , 0 . 10 , 0 . 20 , 0 . 40 , 0 . 50 , 0 . 60 , 0 . 80 , 0 . 90 , 0 . 95 } . We denote the former case as S-cal(10) 1 , D-cal(10) 1 , and the latter case as S-cal(10) 2 , D-cal(10) 2 .

When comparing non-calibrated baselines, KSP consistently outperforms them, regardless of the metric. For CSD, evaluations with S-cal(10) 1 and D-cal(10) 1 slightly favor CSD over KSP, whereas CSD-iPOT outperforms when evaluated using the same metrics. This is natural, since these methods adjust quantile levels exactly at the knot points of S-cal(10) 1 and D-cal(10) 1 . However, when calibration errors are measured with denser bins in the tails, KSP again outperforms all methods, consistent with the results from S-cal(20), D-cal(20), and KS-cal. From this perspective, metrics that depend on the choice of intervals are neither consistent nor fair, highlighting the advantage of bin-free metrics for robust comparison.

I Ablation study on KSP

In this section, we present an ablation study to investigate the effectiveness of KSP. For simplicity, we conduct experiments using the MIMIC-III dataset with three models: DeepSurv, MTLR, and CRPS. This dataset is characterized by a relatively high calibration error, making it a suitable benchmark for evaluating post-processing methods.

I.1 Ablation study 1: link function

We evaluate KSP using various link functions, including the logit, inverse hyperbolic tangent (atanh), three inverse CDFs with range ( -∞ , ∞ ) -probit, inverse Cauchy, and inverse Laplace-and the complementary loglog (cloglog) function. Since atanh has a domain of ( -1 , 1) , we rescale the CDFs using 2 ˆ F θ -1 for compatibility.

Table 23 summarizes the performance across DeepSurv, MTLR, and CRPS. While the logit function does not always yield the best score for every metric, it consistently delivers near-optimal performance across both discrimination and calibration. Atanh and probit show reasonable performance but are prone to numerical instability. The inverse Cauchy and inverse Laplace functions exhibit good performance on some metrics; however, they inherently assume specific parametric forms, making them less suitable for general-purpose use. Cloglog, intended to emphasize tail behavior, results in moderate performance. Considering both empirical stability and robustness, we adopt the logit function as the default link function in KSP.

I.2 Ablation study 2: impact of using top-k deviations

The proposed KSP in the main text minimizes the KS-cal, which corresponds to the maximum deviation between the predicted and empirical CDFs. In this ablation study, we evaluate an alternative formulation that minimizes the sum of the k largest deviations instead of just the maximum alone. We report the performance of KSP under different values of k to examine whether aggregating multiple top deviations yields improved calibration or stability.

As shown in Table 24, increasing k leads to marginal improvements in calibration metrics; however, the gains are minimal. Notably, for MTLR, larger k values result in a decline in the C-index and a rise in calibration errors. These findings indicate that minimizing only the maximum deviation ( k = 1 ) is sufficient to ensure effective calibration without sacrificing predictive performance.

I.3 Ablation study 3: Form of the deviation

Originally, the KSP was defined using the absolute value of deviations in Step 4. We also experiment with an alternative formulation using the squared error. Since the quadratic form penalizes large deviations more heavily than the absolute value, it may potentially lead to better calibration. Additionally, we report the optimization time in seconds (mean and standard deviation) to compare computational efficiency.

As shown in Table 25, although the squared objective slightly emphasizes larger deviations, the absolute form achieves nearly identical performance in both discrimination and calibration. More importantly, it is more efficient in training time.

I.4 Ablation study 4: hyperparameters

To assess the contribution of each hyperparameter used in KSP-namely a , b , and α -we conduct an ablation study by selectively choosing each component. This results in a total of 7 configurations, and the performance of each setting is reported to evaluate the individual and joint effects of the hyperparameters.

As shown in Table 26, using all three hyperparameters a , b , and α yields consistently strong performance across models. While the relative importance of each parameter may vary depending on the model, all three appear to contribute comparably to the overall calibration and discrimination quality.

On the next page, Figure 25 illustrates how each hyperparameter affects ˜ F ( x ) .

+

(a) Effect of varying

a

with fixed

b

= 0

,

α

= 1

(b) Effect of varying

b

with fixed

a

= 1

,

α

= 1

(c) Effect of varying

α

with fixed

a

= 1

,

b

= 0

a=1,b=0,α=1

a =0.5, b=0,α=1

a=2.0,b=0,α=1

1.0

F（x)

1.0

F(x)

1.0

F(x)

Perfect Calibration

Perfect Calibration

Perfect Calibration

0.8

0.8

0.8

0.6

0.6

F(x)

0.6

0.4

0.4

0.4

0.2

0.2

0.2

F

0.0

0.0

0.0

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

X

X

X

a=1,b=0,α=1

a=1, b=-1,α=1

a=1, b=1,α=1

1.0

F(x)

1.0

F（x)

1.0

F(x)

Perfect Calibration

Perfect Calibration

Perfect Calibration

0.8

0.8

0.8

0.6

0.6

0.6

(x)

F(x)

0.4

0.4

0.4

0.2

0.2

0.2

0.0

0.0

0.0

0.0

0.2

0.4

0.6

80

1.0

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

X

X

a=1,b=0,α=1

a=1,b=0,α=0.5

a=1,b=0,α=2

1.0

F（x)

1.0

F(x)

1.0

F(（x)

--Perfect Calibration

Perfect Calibration

- Perfect Calibration

0.8

0.8

0.8

F(x)

0.6

0.6

0.6-

0.4

0.4

0.4

0.2

0.2

0.2

0.0

0.0

0.0

0.0

0.2

0.4

0.6

0.8

1.0

0:0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

X

X

J Adaptability of KS-cal to in-processing procedures

As discussed in the main paper, the original KS-cal can be adapted for in-processing by reformulating it as a penalty term:

Here, k determines how many of the largest deviations are penalized, controlling the trade-off between discrimination and calibration. Using only the maximum deviation ( k = 1 ) can be unstable, as the evaluation point q j varies during training with changes in ˆ F θ . To reduce this instability, we aggregate the topk deviations. The squared form improves gradient behavior and allows for smoother optimization.

We evaluate k ∈ { 1 , 5 , 10 } on the MIMIC-III dataset using DeepSurv, MTLR, and CRPS, and compare the results to X-cal with 20 bins, denoted as X-cal(20). To assess each method's ability to manage the trade-off between calibration and discrimination, we plot calibration error against the C-index for various values of λ and compute the area under the resulting curve. A smaller area indicates a better trade-off.

As shown in Figure 26, increasing λ generally reduces calibration error but also lowers the C-index. Larger values of k tend to yield more favorable trade-offs. While S-cal(20), D-cal(20), and KS-cal show consistent decreases with increasing λ , KM-cal and IBS do not, particularly for models other than DeepSurv.

Although KS-cal(10) clearly outperforms X-cal(20) in terms of area, the comparison may be misleading, as it includes points with low C-index values that are rarely acceptable in practice. Moreover, computing the full curve requires densely sampling λ , which incurs a high computational cost. Thus, we turn to selecting a single optimal λ per method.

MIMIC-I, DeepSurv

MIMIC-II, MTLR

MIMIC-I, CRPS

*KS-cal(1),0.000345(-16.82%)

KS-cal(5),0.000299(-27.82%)

0.015KS-cal(1),0.000456(+0.28%)

+KS-ca(5),0.000410(-9.76%)

0.20-

*KS-cal(5),0.002193(-50.69%)

KS-cal(1),0.006159(+38.48%)

+KS-cal(10), 0.000397 (-12.62%)

*KS-cal(10). 0.001512(-66.01%)

-X-cal(20), 0.000415

-X-cal(20),'0.000455

-X-cal(20),0.004448

00.010

0.15

-cal(20

0.01-

0.05

0.000

0.00--

0.6

0.7

0.8

0.9

0.80

0.85

0.90

0.9!

0.875

0.900

0.925

0.950

C-index

C-index

C-index

MIMIC-IlI, DeepSurv

MIMIC-II, MTLR

MIMIC-III, CRPS

0.06

KS-cal(5).0.001113(+25.65%)

KS-cal(1),0.001225（+38.30%)

+KS-cal(5).0.001084(+4.28%)

KS-cal（1),0.001205(+15.97%)

*KS-cal(5),0.002198(-35.52%)

KS-cal(1),0.005801(+70.17%)

+KS-cal(10).0.001040 (+17.48%)

0.03-

+KS-cal(10).0.001050 (+1.05%)

KS-cal(10), 0.001611 (-52.75%)

-X-cal(20),0.000886

X-cal(20),0.001039

0.2

X-cal(20),0.003409

2

0.04

0.02

D-cal(20)

D-cal

0.1-

0.01-

0.00

0.00**

0.0-

0.6

0.7

0.8

0.9

0.80

0.85

0.90

0.95

0.875

0.900

0.925

0.950

C-index

C-index

C-index

MIMIC-III, DeepSurv

MIMIC-II, MTLR

MIMIC-III, CRPS

0.25 - KS-ca(1). 0.018127 (-4.40%)

KS-cal(1),0.012560 (-5.75%)

KS-cal(1),0.027667(+15.74%)

*KS-cal(5)，0.016731(-11.77%)

*KS-cal(5),0.011766 (-11.71%)

KS-cal(5).0.013820(-42.19%)

0.20 - X-cal(20), 0.018962

KS-cal(10). 0.015891 (-16.19%)

0.20* KS-cal(10). 0.011540 (-13.40%)

-X-cal(20),0.013326

0.6

X-cal(20),0.023905

KS-cal(10). 0.010990(-54.03%)

g0.15

KS-cal

0.4

0.2

0.05-

0.05

0.00-

0.6

0.7

0.8

0.9

0.80

0.85

0.90

0.9

0.0

0.875

0.900

0.925

0.950

C-index

C-index

C-index

MIMIC-IlI, DeepSurv

MIMIC-II, MTLR

MIMIC-I, CRPS

0.100-±KS-ca(1). 0.002117(-11.33%)

KS-cal(1),8.72e-5（+23.11%)

0.03-

KS-cal(1),0.001540(-10.71%)

KS-cal(5), 0.001476(-38.17%)

*KS-cal(5), 9.17e-5 (+29.49%)

*KS-cal(5).0.001170(-32.16%)

0.075-

X-cal(20),0.002387

+KS-cal(10). 0.001294(-45.81%)

9e-04-

X-cal(20),7.08e-5

KS-cal(10), 9.49e-5 (+33.91%)

0.02

X-cal(20),0.001725

KS-cal(10).0.001188(-31.14%)

KM-cal

KM-cal

KM-cal

0.050

6e-04-

0.01

0.025-

3e-04-

0.000-*

0e+00

0.00-...

0.6

0.7

0.8

60

0.80

0.85

0.90

0.9!

0.875

0060

0.925

0.950

C-index

C-index

C-index

MIMIC-II,DeepSurv

MIMIC-III, MTLR

MIMIC-I, CRPS

*KS-cal(1),0.021569 (-4.82%)

-X-cal(20),0.022662

*KS-cal(10),0.019130 (-15.58%)

KS-cal(5).0.019469(-14.09%)

0.04

KS-ca(1).0.003672(-6.28%)

KS-cal(5),0.003628(-7.42%)

KS-cal(10),0.003602(-8.09%)

0.10

-X-cal(20),0.007246

+KS-cal(10),0.005760 (-20.51%)

KS-cal(5),0.005911(-18.42%)

0.2

X-cal(20),0.003919

IBS

S30.03

60.08

B

0.1-

0.02

0.06-

0.6

0.7

0.8

0.9

0.80

0.85

0.90

0.9

0.04

0.875

0.900

0.925

0.950

C-index

C-index

C-index

Example of入Selection

0.05

Error

0.04

ibration

0.03

0.02

Increasing 入 from 0

三

Optimal 入

0.01

0.00

0.86

0.88

0.90

0.92

0.94

0.96

0.98

1.00

C-index

J.1 Selection of the regularization parameter

The core challenge in in-processing methods is choosing a regularization parameter λ that balances calibration and discrimination. While prior work often prioritizes one over the other (Karandikar et al., 2021), our approach seeks to balance both by minimizing the Euclidean distance to the ideal point (1 , 0) in the C-index-calibration error space:

We use S-cal(20), D-cal(20), and KS-cal to compute calibration error and exclude KM-cal and IBS due to their non-monotonic behavior with respect to λ . Since each metric shows different sensitivity to λ (Figure 26), we average the three and denote this composite as CE. We use the squared KS-cal in this average to account for its magnitude.

To ensure both components contribute equally to the distance, we apply the following min-max normalization:

This normalization brings both metrics to the same scale before computing the distance. An example of the scaled values and computed distances is shown in Table 27.

Using this criterion, we compare the best-performing configurations of KS-cal( k ), X-cal(20), and KSP. The results are summarized in Table 28. In-processing methods can sometimes yield lower calibration error than post-processing approaches, but this often comes at the cost of a reduced C-index. They may be appropriate when calibration is prioritized over predictive accuracy, despite a higher computational cost. Addressing this trade-off remains an important direction for future research.

K Additional information on experiment

Table 29 reports the post-processing time (in seconds) for KSP, CSD, and CSD-iPOT across all real datasets. Table 30 summarizes the hyperparameter settings used for training. All models were optimized using the ADAM optimizer with early stopping (patience = 200).

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a post-processing method (KS metric-based post-processing, or KSP for short) based on the Kolmogorov-Smirnov statistic for improving calibration in survival models. Similar to Platt scaling for calibrated classification, KSP applies a linear transformation to the logits of the predicted survival time CDF with an exponential adjustment to the transformed CDF. The three hyperparameters specifying this transformating can be chosen via gradient descent to minimize an empirical version of the KS metric on a validation set. The authors then conduct extensive experiments across 6 base models and 10 datasets, comparing KSP with other post-processing and in-processing methods for survival calibration. They find that KSP improves competitor methods in most scenarios without sacrificing discriminative performance, as measured by the C-index.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths\n\nThe paper is generally well-written and easy to follow.\n\nThe topic is relevant to the NeurIPS community and the contribution is solid. As the authors discuss, uncertainty quantification is important for the safe and effective application of ML techniques in high-stakes scenarios like healthcare, a domain in which survival analysis is particularly relevant. As the proposed method can be combined with any survival model which estimates a survival function, it is widely applicable.\n\nThe experimental validation is very extensive. The authors considered 60 total test scenarios (6 base models x 10 datasets) which cover a variety of model types and dataset sizes. The results are generally in favor of their method, and most of the time they obtain statistically significant improvements across a variety of calibration metrics as compared to the baseline methods. In scenarios where KSP underperforms, they give insightful discussion as to why this may be the case.\n\nIn addition to improved calibration, the method may also be less computationally expensive than other competitor methods for survival calibration.\n\nI would also like to add that I reviewed an earlier version of this paper, and it is clear that the authors have made a substantial effort to address the weaknesses in the previous version.\n\n---\n# Weaknesses\n\nWhile KSP does frequently rank first in terms of the various calibration metrics as compared to the baselines, the absolute improvement is frequently not very large. For instance, in Table 12, the improvement of KSP over the second-best baseline is usually on the order of 10^-3 or 10^-4. Thus, the significance of the contribution (in the practical sense, not the statistical sense) may be somewhat limited.\n\nThere is a line of related work on proper scoring rules for survival analysis which is only briefly referenced in the paper (Yanigasawa, 2023 in the Related work section). The relationship is that proper scoring rules are loss functions which are minimized only by the ground truth survival function (at the population level), so implicitly the use of such a loss function will lead to calibration. It would strengthen the paper to include a model trained with such a loss function as one of the \"uncalibrated\" baselines to see how much gain is to be had from explicit calibration techniques.\n\nQUESTIONS:\nAre there some situations where KSP provides a larger absolute improvement in calibration as compared to the baselines?\n\nWhat is a typical value of B (i.e., number of iterations for KSP to converge)?\n\nHow does KSP compare to training a survival with a proper scoring rule?\n\nWhy should the KM estimator be considered an empirical lower bound on calibration error (lines 223-224)?\n\nLIMITATIONS:\nSome limitations are discussed, mainly in the Experiments section (Section 5).", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a post-processing method to improve calibration in survival models. Existing approaches often rely on heuristic binning, nonparametric estimators, or incur significant computational cost. Inspired by Platt scaling, the proposed method introduces a transformation function optimized via backpropagation to minimize a Kolmogorov-Smirnov (KS) version of D-calibration. Empirical results demonstrate that the proposed method outperforms existing techniques in terms of both calibration performance and computational efficiency.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The motivation is clearly presented.\n2. The empirical section is extensive and includes useful ablation studies.\n3. The paper is generally well-written and accessible.\n\n\n\nWeaknesses:\n1. The paper claims that the KS test has been underutilized in survival calibration, which is misleading. Numerous adaptations of the KS test for right-censored data already exist and are covered in standard survival analysis textbook and literature [1–3]. Furthermore, the proposed use of a KS-based loss function resembles techniques based on Cox–Snell residuals and their calibration assessments [4, 5]. A more thorough discussion of these related methods and their differences from the proposed approach is essential.\n2. While the motivation and background are well-explained, the main methodological section (Section 4) is surprisingly brief. Although simplicity can be a virtue, the proposed method currently appears heuristic and lacks theoretical depth. Several technical questions remain unanswered:\n   * Can the transformation function in line 5 of the algorithm approximate arbitrary survival functions (i.e., is it identifiable)?\n   * Is the objective function (when combined with the KS loss) convex or well-behaved during optimization?\n   * Why are exactly three trainable parameters sufficient, and under what conditions? Table 25 shows that 3-parameter is better than 0/1/2, but the justification needs to be made more rigorous for 4+ parameters.\n   * Is there any formal guarantee on calibration, as provided in recent conformal-based approaches?\n\n3. The evaluation uses 20 bins, while the competing methods (e.g., CSD and CSD-iPOT) were originally designed or evaluated using 10 bins. This discrepancy could influence the relative performance. A fair comparison using consistent bin counts is necessary.\n\n4. Proposition 4.1 assumes the non-crossing property of survival curves, which limits its applicability. The statement that “this condition is satisfied by many commonly used models” is inaccurate. In practice, only CoxPH and WeibullAFT (and their NN-extensions) satisfy this condition. This limitation should be discussed more.\n\n5. Figures 3 and 15–24 use histograms to compare calibration performance, but these are difficult to interpret and obscure the actual differences between methods. For instance, in Figure 3’s leftmost panel, the CSD method appears slightly better than the proposed one. A QQ plot might better illustrate deviations from perfect calibration. Also, the small size of panels in Figure 4 makes error bars and trends hard to see.\n\n\n6. The transformation function is described as “monotonic,” but this should be refined to “strictly monotonic.” A merely monotonic function may contain flat regions, leading to tied CDF values and potentially compromising evaluation metrics such as the time-dependent concordance index. Since all functions evaluated are strictly monotonic, this is a minor but important clarification.\n\n[1] Chapter 7, Survival Analysis Techniques for Censored and Truncated Data. Klein and Moeschberger\n\n[2] Modified Kolmogorov-Smirnov Test Procedures with Application to Arbitrarily Right-Censored Data. Biometrics, 1980\n\n[3] Two-Sample Tests of Cramér--von Mises- and Kolmogorov--Smirnov-Type for Randomly Censored Data. International Statistical Review. 1984.\n\n[4] An evaluation of the Cox-Snell residuals. Elin Ansin. Master Thesis\n\n[5] Randomized Survival Probability Residual for Assessing Parametric Survival Models. Tingxuan Wu. Master Thesis\n\nQUESTIONS:\n1. Why is Park et al.’s in-processing method not included in the comparison? \n2. What is the purpose of the left panel in Figure 2, which aggregates performance across all datasets and models? It shows little difference among the four methods, including the uncalibrated one. Does this imply that in most settings, the original models are already well-calibrated, and post-processing yields only marginal improvements?\n3. The grid used for tuning the regularization parameter is large: {1, 10, 100, 1000}. With such strong regularization, wouldn’t the parameters shrink toward 0s, effectively yielding little adjustment from the original CDF? Again, this raises the question of whether the original models are already sufficiently calibrated.\n\nLIMITATIONS:\nNo limitation is discussed in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nA new calibration method is presented, which is based on the Kolmogorov-Smirnov statistic. It aims to avoid computational limitations and the reliance on binning of some earlier methods.\n\nSTRENGTHS AND WEAKNESSES:\n- As all approaches considered are basically heuristics, it would have been good to try and understand which method performs best in what situation. Currently, some benchmark datasets are used, but carefully designed synthetic data could provide much better insight into the inner behavior of the various methods.\n\n- The suggested method overcomes limitations of earlier methods due to computational complexity and discretization. It is, however, unclear how those earlier methods would perform, in principle, if we could have (close to) unlimited compute and the binning can be coarsened arbitrarily. Are the reported gains primarily the result of computational improvements and the continuous formulation or is there more?\n\n- I think the part of the paper leading up to Theorem 3.1 isn't clear to me, which probably makes me underappreciate the claim in Theorem 3.1. For one, I don't readily see that \"Eqn. (3) is equivalent to (...)\". Also, the text says \"We begin with the simplified setting without covariates,\" but then the general result still follows. The paragraph at line 133 offers some observations and links to other methods, but it is unclear to me what I should take away. In the actual formulation of the theorem, (and maybe this is a silly question but) what does it mean for \"calibration to hold\"? Ultimately, the theorem is an asymptotic result and doesn't necessarily say anything about the finite sample behavior of KS-cal.\n\n- There are some statements that sound appealing, but remain unsubstantiated, as far as I can see. Some examples:\n - Abstract: \"Existing approaches (...) often rely on heuristic binning or nonparametric estimators, which undermine their adaptability to continuous-time settings and complex model outputs.\" (Besides, KS is also a nonparametric estimator.)\n - Page 1: \"but their reliance on fixed sampling schemes or predefined percentiles may limit adaptability.\"\n - Page 9: \"KSP offers a theoretically grounded framework that avoids the limitations of bin-based or sampling-dependent approaches.\"\n\n- I find related work sections \"after the fact\" not very helpful. Apparently, these are works that did not inspire the current contribution, nor are they important to put the work in context nor are they put in a different context based on the results in this contribution. I believe this whole section should be taken out, or whatever is really relevant should be moved to the front.\n\n- A critical discussion in which the pros and cons of this new method in the context of earlier methods and results is lacking. In addition, none of the related works are discussed in the new context provided by the proposed method\n\nQUESTIONS:\n- How exactly do the theoretical results presented provide guarantees for KSP's performance? It is unclear how the theoretical results presented carry over to the actual finite-sample behavior of the method.\n\n- When can we expect the current method to work best and under what settings will one or more of the other methods probably outperform the current one?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces KSP (Kolmogorov-Smirnov post-hoc calibration), a novel method for calibrating survival models using the Kolmogorov-Smirnov metric. The authors address an important challenge in survival analysis by providing a robust, flexible, and theoretically grounded approach that enhances calibration without significantly compromising predictive performance.\n\nSTRENGTHS AND WEAKNESSES:\nSome of the key strengths of this paper are:\n1. The use of the well-established KS metric ensures a solid theoretical basis for the method, making it credible and reliable. Further, KSP demonstrates effectiveness across various survival models and datasets, showcasing its versatility.\n2. The computational efficiency of KSP, requiring fewer iterations compared to other methods like CSD or CSD-iPOT, makes it practical for handling large datasets and complex models.\n3. The method maintains high predictive accuracy while improving calibration, which is crucial in high-stakes domains such as healthcare and infrastructure.\n\nSome of the aspects where this can be improved upon are:\n1. There are instances where KSP performs less well than other methods across specific dataset conditions or small sample sizes. It is important to analyze these scenarios thoroughly to identify \"operating regions\" of the metric\n2. Continuing on the above, the method may not capture calibration error as effectively in very small sample sizes, which could be a limitation in certain scenarios. \n\n\nOverall, the paper presents an innovative solution to a significant problem in survival analysis, offering a method that is both theoretically sound and empirically validated. It has the potential to significantly impact the development and deployment of survival models in real-world applications. The authors are encouraged to explore further enhancements and validations to address its limitations and expand its applicability.\n\nQUESTIONS:\n1. Why does KSP sometimes underperform compared to other methods in specific datasets? What steps can be taken to enhance its performance across these scenarios?\n2. How can KSP be adapted to effectively handle cases with extremely small sample sizes or high censoring rates? Is this feasible? \n3. What future work is planned to further validate and generalize the method across a broader range of survival analysis scenarios, including extreme cases and challenging data conditions?\n\nLIMITATIONS:\nn/a", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}