Toward a Well-Calibrated Discrimination via Survival Outcome-Aware Contrastive Learning
Authors: Dongjoon Lee
Hyeryn Park
Changhee Lee
Venue: NeurIPS 2024 poster (2024)
Keywords: Survival Analysis
Contrastive Learning
Deep Learning
Healthcare
Primary area: machine_learning_for_healthcare

Abstract: Previous deep learning approaches for survival analysis have primarily relied on ranking losses to improve discrimination performance, which often comes at the expense of calibration performance. 
To address such an issue, we propose a novel contrastive learning approach specifically designed to enhance discrimination without sacrificing calibration. 
Our method employs weighted sampling within a contrastive learning framework, assigning lower penalties to samples with similar survival outcomes. This aligns well with the assumption that patients with similar event times share similar clinical statuses. Consequently, when augmented with the commonly used negative log-likelihood loss, our approach significantly improves discrimination performance without directly manipulating the model outputs, thereby achieving better calibration.
Experiments on multiple real-world clinical datasets demonstrate that our method outperforms state-of-the-art deep survival models in both discrimination and calibration. Through comprehensive ablation studies, we further validate the effectiveness of our approach through quantitative and qualitative analyses.

Toward a Well-Calibrated Diserimination via Survival Outcome-Aware Contrastive Learning
Dongjoon Lee" Chung-Ang University dongzza97@cau. kr
Hyeryn Park Chung-Ang University hyeryn2000@cau.cc ac.kr
Changhee Lee Korea University
Changheelee@korea.eee
Abstract
Previous deep learning approaches for survival analysis have primarily relied on ranking losses to improve discrimination performance, which often comes at the expense of calibration performance. To address such an issue, we propose novel contrastive learning approach specifically designed to enhance discrimi- nation withoul sacrificing calibration. Our method employs weighted sampling within contrastive learning framework, assigning lower penalties to samples with similar survival outcomes. This aligns well with the assumption that patients with similar event times share similar clinical statuses. Consequently, when augmented with the commonly used negative log-likelihood loss, our approach significantly improves discrimination performance without directly manipulating the model outputs, thereby achieving better calibration. Experiments on multiple real-world clinical datasets demonstrate that our method outperforms state-of-the-ard deep survival models in both discrimination and calibration. Through comprehensive ablation studies, we further validate the effectiveness of our approach through quantitative and qualitative analyses.
Introduction
Deep learning models have gained significant attention in survival analysis (also known time-to- event analysis), which focuses on predicting and understanding the occurrence of an adverse event (e.g., death) as function of time. The utility of survival models typically evaluated through two key aspects: discrimination and calibration. Discrimination measures a model's ability to differentiate between patients with varying risks, prioritizing those more likely experience the event. Calibration, on the other hand, assesses how well model's 's predictions align with the observed event distribution. In other words, a well-calibrated model offers predictions that elosely match actual survival outcomes, offering crucial prognostic value to clinicians.
In effort enhance the discriminative power of survival models, ranking loss functions 2 are frequently employed. These functions aim to maximize a relaxed proxy of the concordance index (C-index), well-established metric for evaluating the quality of patient rankings based the risk predictions of survival models 5.. However, a notable improvement discriminative power often comes the expense of calibration performance, which negatively affect the clinical utility of predicted survival outcomes. For example, poorly calibrated model that overestimates risk may lead to unnecessary treatment testing for patients. Conversely, underestimating risk may result in patients with adverse conditions not receiving appropriate care.
Contrastive learning 66 7 is framework that aims leam embedding space where similar samples are mapped to nearby locations, while dissimilar samples are pushed farther apart. Notably, although commonly used in unsupervised fashion, this framework shares fundamental principle
Equual contribution
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

with ranking losses: both focus on the relative ranking of samples for better discrimination. However, unlike ranking losses, which directly modify differences in model outputs, contrastive learning operates on the relative distances of representations in the embedding space. This distinction offers the potential to overcome misalignment issues caused by direct comparisons and modifications of model outputs inherent in the ranking-ban based approaches. To leverage this potential and further enhance survival models, we propose incorporating informed sampling into the contrastive learning framework. By assigning lower penalties to potential false negative pairs based on the similarity of their survival outcomes, we instill an inductive bias in the model, encouraging it to learn that samples with similar event times likely share similar clinical statuses. This allows to achieve strong discrimination without directly comparing or manipulating model outputs, thus potentially improving calibration well.
Contribution. In this paper, we propose a novel contrastive learning approach, specifically designed for survival analysis, to enhance both discrimination and calibration performance?] Motivated by the assumption that patients with similar event times share similar clinical status, we contrast samples in the latent space based on their survival outcomes. However, unlike standard contrastive methods, we leverage the model output solely for maximum likelihood estimation (MLE), thereby promoting better calibration. This is achieved by incorporating importance sampling into our contrastive learning framework, where we assign weights proportional to the differences in survival outcomes. This guides our model to learn appropriate similarity relationships between clinical features relevant to survival analysis. Through experiments on multiple real-world clinical datasets, we demonstrate the superiority of our method in both discrimination and calibration performance, outperforming state-of-the-ar deep survival models. We further provide comprehensive quantitative and qualitative analyses through ablation studies to showcase the impact of our novel contrastive learning approach tailored for survival analysis.
2 Related Works
2.1 Deep Learning Approaches Survival Analysis
Deep learning- Deaming-baseed survival models have garnered significant attention due to their ability to provide non-parametric estimations of the underlying discrete-time survival distribution, particularly conditional hazard survival functions. Negative ligeliiel (NLL) has been widely used in survival analysis to estimate various survival quantities, such as conditional hazard functions 9 [.. probability mass functions of event times 11. survival functions 4 MLE, based on NLL, provides unbiased estimates of these quantities under the assumption of ignorable censoring, where the event and censoring are independent given the input features. Building on this foundation, recent deep learning-based survival models often incorporate auxiliary losses alongside the NLL loss to further enhance model performance.
Ranking Loss. Ranking loss, often augmented with the NLL loss, has been widely employed in recent deep survival models to enhance the discriminative power of survival predictions μ 2 4.
These ranking loss functions, which are differentiable approximations or upper bounds of the negative C-index [12), are typically based exponential 21, log-sigmoid [3), or linear 3 functions. In particular, these approaches directly utilize model outputs, such as conditional hazard or survival functions, to establish pairwise ordering of comparable individual risks for better discrimination. However, since ranking loss primarily focuses on samples with earlier event times (often ignoring censored samples), these models may struggle to capture the full distributional characteristics of time-to-event outcomes. This can adversely affect calibration performance, potentially leading to inaccurate predictions regarding the observed time-to-event outcomes 2
Calibration Loss. In healtheare applications, while achieving high discriminative power through ranking loss is crucial, well-calibrated predictions are equally important for effective clinical decision- making. Notably, recent approaches [1] 2 have prioritized enhancing calibration by minimizing the rank probability score (RPS), prediction error metric specifically tailored for survival analysis. Furthermore, the authors in [15] have introduced novel approach that transforms D-Calibration [16] into a differentiable objective, calculating the squared difference between observed and predicted events across multiple time intervals. In contrast to these methods, our proposed approach does not
Source code for ConSurv is available in httpps//ggttabbcoo///oozzz/////s

---

explicitly incorporate a calibration loss function. Nevertheless, it achieves comparable or even superior calibration performance, demonstrating the effectiveness of our contrastive learning framework in implicitly preserving calibration alongside discrimination.
2.2 Contrastive Learning
Contrastive learning 66 17 is framework that learns embedding space that effectively discriminates among samples, by mapping positive pairs (similar samples) closer together, while pushing negative pairs (dissimilar samples) farther apart. While contrastive learning has made significant progress, our review of related work will focus on methods that explore various strategies for injecting inductive bias to enhance representation learning.
Exploiting Continuous Labels. Inspired by the success of incorporating label information into contrastive learning for classification tasks 8 recent research has extended this concept regression tasks by exploiting continuous label information for constructing positive and negative pairs. Zha et al. [18 employ hard thresholding on label differences to generate positive and negative samples, capitalizing on the idea that samples with similar target values should be mapped closer in the latent space. Kerdabadi et al. [19] apply similar approach in a dynamic time-to-event setting, defining positive pairs as those within a specific time window and weighting negative pairs proportionally to their time difference. However, this weighting scheme's 's heavy reliance on absolute time differences can destabilize the contrastive loss during training.
Weighting Negative Samples. Recent works have significantly improved contrastive learning by incorporating inductive bias in the selection of negative samples. This includes focusing more weights on hard negatives that challenge the model for more effective discrimination [20] 21 22 23] and
avoiding potential false negatives that might mislead the model [24. Many of these studies address negative sample selection by employing informed sampling techniques to identify informative hard negatives or potential false negatives. Notably, Robinson et al. [20] leverage importance sampling by proposing similarity-bar based distribution that prioritizes hard-to-distinn hard-to-distinguish negative samples.
3 Problem Formulation
3.1 Preliminary: Discrete-Time Survival Analysis
Suppose we are given a discrete-time survival dataset comprising N patients, denoted as DD {(x;, Ti, 6,)) Each patient is represented by the input feature x; E X where X' the input space, and the observed survival outcomes, т; E 'T and 6, E 10. Here, and 6; indicate the time elapsed until either an event of our interest (e.g., death) or censoring (e.g., lost to follow-up) occurs and whether the event was observed or not (i.e., right-censored), respectively. Throughout, we treat survival time as discrete and the time horizon as finite such that the set of possible survival times defined as T = ,....,Tmaak with a pre-defined maximum time horizon T Tmax-
The conditional hazard function, A: X' given feature x and is defined as A(t|x) = IP(T = t T 2 x). Then, we can represent the survival function S X
[0, is the instantancous risk of the event at time
[0, 1] as follows:
S(t|x) = P(T x
II (1 (1 x(t(xx) Stt
(1)
which a non-inereasing function of I, indicating the probability of the event occurring after time / given feature x. Equivalently, we could estimate the risk function, R X' x T + [0, 1]. which represents the probability of the event occurring at or before time t given feature x, i.c., R(t(x) PTT t[x) = 1 S(t[x).
Then, we can achieve likelihood-based estimates for the hazard function, X, by minimizing the following negative NLL loss:
N
LNI Σ
& logg(r=[xx) + (1 6:) log S(7a xx)]
(2)
where p(()x) = (( 1|x) represents the estimate for the probability of an event occurring
at time f, i.c., P(T = x/x). Here. ( exploits two pieces of information from the survival data: (i)
3

---

when the event is observed (i.e., 6; = 1), we know that the event occurred at time and (ii) when the event is not observed (i.e., ò, 0). we are aware that the event will occur after time Tj.
3.2 Ranking Loss for Survival Analysis
Ranking loss specifically targets enhancing the discriminative power of survival models, which is crucial for better distinguishing between patients based on their risk. Suppose we are given the risk function, R. that quantifies the risk of a given patient with x at any time t. We aim to penalize instances where the risks assigned to pair of patients are incorrectly ordered (i.e., assigning lower risk to patient who died before patient j). This can be achieved by the ranking loss, which is formally given follows:
R EAs Aig ((((TT)))))) < R(T(xx)) R
Σ A ([((T))x))) R(Tx(x;)), (
(3)
where 17 is function that relaxes the non-differentiable indicator function, II. Here, Ars I(ō,
1, Ti < Tÿ) indicates acceptable pairs whose assigned risks are comparable.
Different deep survival models have employed various types of quantities for implementing the ranking loss. For example, Lee al. [ have applied the risk function, R, and set v) y) exp(-(xr /))x), convex function that both penalizes the wrongly ordered pairs and encourages the correctly ordered pairs. The same convex function has been employed utilizing the survival function, S 21. Similarly. Steck et al. [13] utilizes 771, y) = log o x), which is lower bound of the C-index, with the risk function, Chi et al. [3] employ the hazard function with a linear ranking function (2 y) (- y).
Challenges. While combining NLL with ranking loss shows notable improvement discriminative power, it often comes at the expense of calibration performance, potentially harming the clinical utility of predicted survival outcomes. We suspect this primarily due to how ranking loss directly modifies model outputs to order predicted risks, potentially leading to misalignment with the actual risk distribution. In this paper: we propose a different approach. Our method focuses on increasing discriminative power not by directly utilizing survival outcomes but by exploiting the embedding space through our novel contrastive learning framework. This approach preserves the calibration performance achieved by the NLL loss while enhancing discriminative capabilities.
4 Method
To address the challenges described above, we propose a novel Contrastive learning approach to a deep Survival model, which refer to ConSury. The proposed method consists of three key components as illustrated in Figure 1
Encoder (parameterized by 0), fo X H. takes features X' as input and outputs latent representations, i.e., h = fx(x).
Projection head (parameterized by vb), ge + R.d. maps latent representations to the embedding space where contrastive learning is applied, i.e., 9n (h).
Hazard network (parameterized by o). fo H T - [o. 1]. predicts the hazard rate at each time point t E T given the input latent representation h, i.c., ((t|x) = (b, t) = (x).
Motivated by the core concept of contrastive learning frameworks, we aim to differentiate each sample from other semantically different samples based on their survival outcomes. This allows us to overcome the limitation of ranking loss, which arises from the direct comparison of model outcomes in the form of risk/survival functions. Instead, we contrast samples in the latent space based on their survival outcomes and utilize the model outcome solely for the NLL loss encourage better calibration.
4.1 Contrastive Learning for Survival Analysis
Our novel contrastive learning framework imposes lower penalties on potential false negative samples based on the similarities in their survival outcomes. More specifically, given an anchor sample, we define potential false negatives samples with small difference in the corresponding time-to-event.

---

uncenso ed censored
Strkk)
fg (
ANLL
x
fo
h
PpC-)
2
PaRU
x
LSNCE
mng()
x+
fo
h+
Latent
Figure 1: An illustration of the network architecture for ConSurv.
This inherently aligns with our inductive bias that patients with similar survival outcomes should share similar clinical status, which manifests through similar representations.
For each anchor sample x px. the noise-contrastiig estimation (NCE) objective aims to learn mapping f = Is fo utilizing positive sample with the same semantic meaning, i.e., x* p++ and negative samples with (supposedly) different semantic meanings, i.e., x q, as follows [20
(x.xx))
E xxpx 2 1txp
log
M Ex Bxx-nglox(xxx xx]ex(xx--]]
where M the scaling term which is set to the batch size and 8 x X [-1,1] is the similarity score between two samples. Here, we use cosine similarity in the embedding space, i.e.,
s(x, x') = f(x) f(x')
[fx)]l] f(x')l For notational convenience, we omit the corresponding temperature y and
write (=(xxx*) to denote e=(x,xt)/v.
The key aspect in (4) for obtaining embeddings with discriminative power lies in how select negative samples that the anchor sample should be distinguished from. To reflect the differences in the time-to-events in the embedding space, design novel negative distribution, 9, by utilizing the available information from survival outcomes.
Importance Sampling using Survival Outcomes. To accurately distinguish patients based on their time-to-event outcomes, we fully utilize the time-to-event information for designing y based on following inductive bias: similar patients more likely experience the event at similar time points than the ones who are not. Hence, given an anchor (x, r) and a negative sample (x , TT). we define the weight function follows:
(T 1 -
(5)
where >> 0 is temperature coefficient. From this point forward, we will slightly abuse the notation consistent with the notation used for the negative
and write w<xx;: x) to denote uT(T=T)) distribution. This function is a variant of the Laplacian kernel, where the weight increases as the time difference increases. That is, we assign larger weights for samples with large differences in the time-to-events, and smaller weights for samples with small time differences. Utilizing the weight function in (5). we can define the negative distribution, g. as:
q(xTxx] Z w(x x))(xx"),
(6)
where Z is the normalizing constant. Then, using the importance sampling technique, we can approximate the expectation of the similarity score of negative samples drawn from the negative distribution in (6) as:
Ex my [e((xx)) Ex
qqxx=x :x) p(x-)
((x)))
1
M
w(x; x) - (xxx))
~
Z
M
5

---

where the normalizing constant for the empirical distribution can be given as Z = Overall, the survival outcome-awand aware NCE (SNCE) loss in (8) mitigating the effect of potential false negatives that have similar survival outcomes, can be given the following:
M Σラ w(x;; ;x).
(x(xx))))
SNCE
log 1 +EM Z j== w(x xx;) cx(xxx))
4.2 Handling Right-Censoring
However, in survival dataset, there exist samples that right-censored, offering partial information about the corresponding survival outcomes which indicates that the event will occur sometime after the censoring time. Consequently, not every pair in the survival dataset has comparable survival outcomes. Specifically, there are three different cases of sample pairs to consider:
Case I: Both samples are uncensored (i.e., have observed events). We directly compare the two time-to-events of any given pair.
Case 2: One is uncensored and the other is censored. Similar to the acceptable pairs in ranking loss,
only compare a pair when sample experiences an event while sample is censored after that event time (implying no event has occurred by that time), i.e., Tj < Tj.
Case 3: Both samples are censored. comparison possible as we do not know when the actual events have occurred for both samples.
Considering these cases, we redefine the weight function considering the right-censoring as T((ziT)) = 1i.5 T.) where 45 indicates the comparable cases as follows:
(6, 1. 1) ((6, = 6, = 0) ^ (T. < <T) A (I= - -yl a))
1.5 10 otherwise
where a is hyperparameter that represents a margin ensure minimum time difference. Please see Appendix C.3 for more details about the effect of a.
4.3 Network Description
In this subsection, we provide detailed network description of ConSurv and outline the learning procedure using a discrete-time time survival dataset, = {(x,, Ts, 8,)
Negative Log-likelihood. Based on the encoder and the hazard network, we can provide the estimated hazard function given the input feature x at each time point t E T as \(t]x) ff (x),, f). As our hazard estimate is defined as function of time given an input feature, we can naturally model the time-varying effect of input features (thus, more complex relations) risk/survival functions. Then, can compute the NLL loss, LNLL in 2 by plugging in fa(fo(x), t) into á and Š follows:
p(Tlx) - So(o(()),) II (1- EST-1
S(1 II (1- (1- EST
(9)
Contrastive Learning. To train the proposed method using our contrastive loss introduced in 8), we construct augmented samples based on the state-of-the-ar contrastive learning framework specifically designed for tabular data [26]. For each sample in a batch of M samples, we construct corrupted version of the original sample following the marginal corruption process. So, given x; as an anchor, we set the corrupted sample, i.e., X; positive and all the other corrupted samples, i.e., X; for j i, as negative. By passing the original, positive, and negative samples through f = 9o o Se, we can compute our survival outcome-based contrastive learning loss function defined in (8)
Overall, we can estimate the hazard function by training ConSurv with a loss function that combines the NLL loss and the SNCE loss as the following:
LNL + C
(10)
where balancing coefficient that trade-offs between the two loss terms; the effect of B provided in C3 Please find the pseudo-code of ConSurv in H

---

Table 1: Diserimination and calibration of survival models: mean and standard deviation values for CI, IBS, and DDC, along with the number of successful D-calibration tests
METABRIC
NWTCO
CI
IBS L
DDC
D-CAL
C
IBS 4
DDC
D-CAL
CoxPH
0.662+0.005
0.170
0.108800.om
10
0.712=0.0ss
0.301+00.u
+..67
10
DEEFSURV
0.615+0.004
0. 0.18820.019
0.160+0.co 0G1
10
0.615=0.057
0.0901aamT
0,612=0.057
DEEPHITT
0.69660..00
0, 0.205e0.017
0.28940.cmr
0
0.713=0.00
0.140++004s
0,616ea.050
4
DRSA
1.6351
0. 0,260 po.os
0.199 coT
1.2951
DCS
0.598+0..05
0. 0.243
2o.00
0
0.677
0.0988
0.17340.051
6
X-CAL
0.6581o.cm
0.1891o.0r 1o.01s
0.068_uam
0.1/6_oooozz jooouz
0.133 10.074
0.608
0.244
go.os
0.711 20.052
0.102
0.464
10
Casz & Coex
1.060
0.191 10.014
0.1 10
8
0.716
1.10000.00m
0. 0.28910.013
8
Exsz & Cxss
0.643go.000
0. 0.267.0.0as
0.155 coo
0.723g00002
0.210 000
0.551 20.057
CONSURV
0.688
0. 1..
1.
6
0.729gmmm
0.090_aam
0.22220.029
10
GBSG
FLCHAIN
METHOD
C
TBS L
DDC
D-CAI
CT
IBS 4
DDC
D-CAL
COxPH
0.677
0.177=0.00s
0. 0.220
10
0. 0.800=0.om
0.100
0.30220.016
10
DEEVSURV
0.674400000
0.179go.om 000
0.203
10
0.79820.010
0.084
0.294go.010
10
DREPHIT
0.612+0.008
0.207
0.35-440.022
1
0. 0.79800.0
0.168+0.00s
0,50020.004
0
DRSA
0.67920.000
0. 0.1
0.4022o.cs
0. 0.77920.010
0.226
0.3 0.318go.014
0
DCS
0.693+0.000
0. 18000.0000
0.16640.010
5
0. 78626.012
0.105+0.00s
0.251 +4.6s
0
X-CAL
0.687+0.000
0. e..7000
0.087=wmx
0. 0.791 =a.0ne
0.11000oou0
0.302
3
CxL
0.675+0.018
0.174e0.002
0.12640.c25
0
0.791=0.017
0.10M
0,273e0.015
9
Casz Loce
0.687 t0.00s
0.184
0.193 ous
0
0.797 20.022
0.001
0.302
8
Cxu & Exuc
0.687+0.012
0.3 0.342=0.054
00
O. 0.796=0.014
0.184+0.010
0.334ee0000
CONSURV
0.696
0.180 1..00
0
0.810
0.072
0.291
SUPPORT
SEER
METHOD
CI
IBS &
DDC 1
D.CAL
CI
IBS 4
DDC
D.CAL
CoxPH
0.605g9.000 006
0. 0.0
0.
0
0.857
0.00910.001
0.906 001
10
DEEPSURV
0.50910.001
0.1961a.00
0.258 cos
0
0.760
0.0101o.00
1.000
10
DREPHIT
0.502200007
0.27220.003
0
0.843g0.021
0.020
0.825 000
0
DRSA
1..731
0.26310.019
0.285
0
0.810_o.11
0.0231o.ouz
0.650
0
DCS
0.507 1:0000000 000
0.211
0.1
0
0. 0.8602o.ozz
0.01010.001
0.903
0.007
9
X-CAL
0.6041a.006
0.207 go.00
0.1821o.cm
0.844 _ooocs
0.01110.008
0.907 Lo.02e
9
ONzz
0.607+0.005
0. 196
0.1 011
0
0.853
0.009
0. 0.9022...mm
10
Cau Coce
0.609+0.000
0. 193
0. 0.130 cas
0
0.855=0.018
0.009+0.001
0. 0.965
10
Daz & Cook
0.61020.005
0. 0.2942o.0m
0 002
0. 0.863=0.017 1,000 0.120
0
CONSURV
0.618
0.192
0.144
0
0.865
0.004-:0ms
0.961
10
5
Experiment
In this section, we evaluate the performance of ConSurv and multiple survival models using several real-world clinical datasets. Further details about all the experiments are available in Appendix D
5.1
Experiment Setup
Datasets. We compare our proposed method and the benchmarks with the following four commonly used real-world clinical datasets: METABRIC, NWTCO, GBSG, FLCHAIN, SUPPORT, and SEER. For detailed descriptions of these datasets, please refer to D.I
We compare ConSurv with six survival models that were selected based on their respective loss functions, which are critical for understanding trends in survival analysis performance. The evaluated models are based on i) partial oig-likllhood including CoxPH [27] and DeepSury 28 ii) ranking loss including DiepHit [I] and DRSA (10), and iii) calibration loss including DCS and X-CAL [15]. Detailed methodological descriptions are provided in Appendix[D.3
Benchmarks.
We evaluate the discriminative performance of ConSurv and benchmarks using the integrated time-dependent C-index (CI) (29 across all time points, where higher values indicate better performance. For calibration, we utilize the integrated Brier score (IBS) 30 and distributional divergence for calibration (DDC) (2). where lower values indicate better performance. Additionally, D- D-Calibration (D-CAL) 31 assesses calibration with results reported based on p-values; those exceeding 0.05 are noted as statistically significant. Comprehensive details on IBS, DDC, and D-CAL
Performance Metric.
'Table I presents the performance results obtained using 10 random seeds. For enhanced stability verification. conducted additional experiments with 25 random seeds. Please refer to Table 10 in appendix C.5
7

---

OU0 timo
(a) NLL only 400 epochs
(b) SNCE only 400 epochs
(c) NLL + SNCE 400 epochs
Figure 2: t-SNE visualization for latent representations learned with LNu. only, LsNce only, and ConSurvfor the METABRIC dataset, colored by event times (for uncensored samples).
Socdd
BELL
3.
03
55
0.5
33
37
0
(a) VS DeepSurv
(b) vs DeepHit
(c) vs DRSA
(d) DCS
(e) X-CAL
Figure 3: Calibration plots for ConSurv comparison with benchmarks for the METABRIC dataset.
are provided Appendix D.2 Appendix C.I we further provide time-dependent C-index [5] and
time-dependent Brier score 30 at different time points.
5.2 Quantitative Analysis Results
We report the CI, BS, DDC, and D-CAL of our proposed method and the benchmarks in Table The results in Table demonstrate that our method significantly outperforms all the benchmarks in discriminating among individual risks at the evaluated all time points, consistently across the four weal-world clinical datasets. ConSurv achieves such gains in the discriminative performance while providing the best or comparable calibration performance. Notably, as evidenced by the metries IBS, DDC, and D-CAL, our method yields exceptional calibration performance. Contrarily, ranking loss-based models such as DeepHit and DRSA, which are designed primarily to enhance discriminative power, usually exhibit poor calibration. Even without being specifically designed through the loss function for calibration, our model outperforms benchmarks such DCS and X-CAL, which incorporate loss functions used enhance calibration power. This indicates that our approach inherently balances both discriminative and calibration power effectively.
Ablation Study. We conduct ablation study better understand the contribution of different loss functions within our framework employing the following variants of ConSurv: (i) LNLL only, (ii) LNLL& LNCE, (iii) CNLL & LRank and (iv) ConSurv (i.c., CNUL & LSNCE). Table higghlghhs the performance trade-off's between discrimination and calibration power when different loss functions are utilized in survival models. Using only NLL loss typically leads lower discriminative performance while maintaining reasonably good calibration. Augmenting ranking loss to the NLL significantly enhances discriminative performance but decreases calibration. In contrast, our proposed SNCE loss significantly enhances the model's performance compared to (i) using NLL only and (iii) using NLL with ranking loss. This suggests that the SNCE loss not only boosts discriminative power but also preserves or even improves the model's calibration. Additionally, when compared to (ii) using NLLL with InfoNCE, which pushes all negative pairs away equally, we confirm that ConSury, which assigns weights to negative samples based on their time difference until the event, ensures better performance.

---

Subgroup: ER+
Subgroup:
Survivai Probabilh ity
Table 2:
Wasserstein distances from KM curves across various patient sub- groups for the METABRIC dataset.
DR5A
205
190
122
200
20
205
Subgroup: HER2+
time
Subcroup: HER2-
time
Subgroup
ER
HER2
Survival Probabliity ltt
CoxPH
0.030
0.108
0.063
0.089
DeepSurv
0.033
0.115
0.066
0.101
DeepHit
0.063
0.082
0.146
0.156
Goil
Gooht Despet
DRSA
0.181
0.293
0.233
0.328
DCS
0.130
0.146
0.087
0.178
X-CAL
0.136
0.165
0.105
0.180
ConSurv
0.024
0.077
0.044
0.089
time
time
Figure 4: Comparison the survival curves across various patient subgroups for the METABRIC dataset.
5.3
Qualitative Analysis Results
5.3.1 Effect of Contrastive Learning
To further demonstrate how survival outcome-aware contrastive learning promotes discrimination among samples in the latent space, we compare two-dimensional t-SNE visualizations of latent repre- sentations trained with LNLL only, LSNCE only and ConSurv (i.e., CNLL LSNCE) for the METABRIC dataset in Figure Here, we only display uncensored samples with event times for clarity. Figure 2 shows that when trained with LNLZ alone, latent representations are mixed regardless of their corresponding event times. However, incorporating LSNCE into the loss function significantly im- proves the alignment representations with event time information, enabling the predictor to better discriminate the risks associated with each sample. Notably, even when trained solely with LSNCE, latent representations exhibit a clear alignment with event times. This finding strongly suggests that contrastive learning approach effectively encourages discrimination by capturing and reflecting the underlying event time information. (See Appendix C.2 for more datasets.)
5.3.2 Calibration Plot
Figure shows calibration plots comparing the calibration of ConSurv with the deep learning- bbased survival models. Here, we evaluate calibration by matching predicted cumulative densities to observed event frequencies the quantiles of the predicted cumulative density. In these plots, the x=y line represents the ideal state where predicted probabilities perfectly match the observed outcomes. The evaluated survival models show different trends depending on the loss function used for training. The ranking-based models (i.e., DeepHit and DRSA), which directly manipulate the model outputs, experimentally demonstrated the lowest calibration power. Contrarily, the calibration- -based models (i.c., X-CAL and DCS). designed to improve calibration power, show relatively better calibration but do match the calibration performance of DeepSurv, likely due to the assumption proportional hazards. Compared to these survival models trained based on three different types of loss functions, our proposed method demonstrates calibration performance that is similar or even superior.
5.3.3 Subgroup Analysis
We further validate the calibration performance of the survival models by comparing their survival plots with the Kaplan-Meic (KM) curve, a non- -aramaraettiic estimate of the survival function at population level. For this comparison, we examine two binary hormone receptor statuses in the METABRIC dataset: estrogen receptor (ER) and human epidermal growth factor receptor (HER2), which are crucial for determining hormone therapies and can result in significantly different prognoses for breast cancer patients. Figure 4 we average the survival functions of different survival models based the subgroups of patients with the corresponding feature values (c.g., ER+ vs ER-). The

---

results confirm that our method aligns well with the KM curve compared to other deep Staning-basse survival models, especially those with ranking loss.
To quantitatively assess calibration performance across different subgroups, we compare the survival predictions of each model with KM curves for each subgroup using the Wasserstein distance. This metric is well-suited for comparing survival curves as it considers both the overall shape and the discrepancies in predicted survival probabilities each time point. The results in Table demonstrate that our proposed model achieves well-calibrated predictions within each subgroup. (See Appendix C.4.2 for more subgroups.)
6 Conclusion
In this paper, we propose ConSurv, a novel contrastive learning approach for deep survival analysis. Our method leverages weighted sampling incorporate the intuitive assumption that patients with similar event times likely share similar clinical statuses. Unlike most existing deep survival models, ConSurv does not directly manipulate hazard or survival predictions during contrastive learning. This approach allows our method to maintain well-calibrated predictions based on the negative log ligliike loss. Experiments on multiple real-world datasets demonstrate the superiority of our method over state-of-the- art deep survival models. particularly in achieving superior calibration performance compared ranking loss-based models.
Acknowledgments
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. RS-2024-00358602) and the Institute of Information Communica- tions Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. RS-2019-1119009, Artificial Intelligence Graduate School Program (Korea University).
References
[1] Changhee Lee, William 7ame, Jinsung Yoon, and Mihaela Van Der Schaar. Deephit: A deep learning approach to survival analysis with competing risks. In Proceedings of the 32th AAAI Conference artificial intelligence (AAAI 2018), 2018.
[2] Fahad Kamran and Jenna Wiens. Estimating calibrated individualized survival curves with deep learning. In Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAI 2021). 2021.
[3] Shengqiang Chi, Yu Tian, Feng Wang, Yu Wang. Ming Chen, and Jingsong Li. Deep semisuper- vised multitask learning model and its interpretability for survival analysis. IEEE journal of biomedical and health informatics, (18))3185-31999 2021.
[4] Elconora Giunchiglia, Anton Nemchenko, and Mihaela van der Schaar. Rnn-surv: A deep recurrent model for survival analysis. In Proceedings of the 27th International Conference Artificial Neural Networks and Machine Learning (ICANN 2018), 2018
[5] Hajime Uno, Tianxi Cai, Michael J Pencina, Ralph B D' Agosstnoo, and Lee-Jen Wei. On the c-statistics for evaluating overall adequacy of risk prediction procedures with censored survival data. Statistics medicine, 30(10): 105-1117, 2011.
[6] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In Proceedings of the 37th International Conference on Machine Learning (ICML 2020), 2020.
[7] Kaiming Hc, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020), 2020.
[8] Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. Proceedings of the 34th Conference on Neural Information Peocessing Systems (NeurIPS 2020), 2020.
10

---

[9] Michael F Gensheimer and Balasubramanian Narasimhan. A scalable discrete-ting survival model for neural networks. PeerJ, 7:c6257, 2019.
[10] Kan Ren, Jiarui Qin, Lei Zheng, Zhengyu Yang. Weinan Zhang, Lin Qiu, and Yong Yu. Deep recurrent survival analysis. In Proceedings of the 33th AAAI Conference on Artificial Intelligence (AAAI 2019), 2019.
[11] Alicia Curth, Changhee Lee, and Mihaela van der Schaar. Survite: Learning heterogeneous treatment effects from time-to-event data. In Proceedings of the 35th Conference on Neural Information Processing Systems (NeurIPS 2021), 2021.
[12] Frank E Harrell, Robert M Califf, David B Pryor, Kerry I. Lee, and Robert A Rosati. Evaluating the yield of medical tests. Jama, 247 1982.
[13] Harald Steck, Balaji Krishnapuram, Cary Dehing- Philippe Lambin, and Vikas C Raykar. On ranking in survival analysis: Bounds on the concordance index. In Proceedings of the 20th Conference Neural Information Processing Systems (NeurIPS 2007), 2007.
[14] Anand Avati, Tony Duan, Sharon Zhou, Kenneth Jung, Nigam H. Shah, and Andrew Y. Ng. Countdown regression: Sharp and calibrated survival predictions. In Proceedings of The 35th Uncertainty in Artificial Intelligence Conference (UAI 2020), 2020.
[15] Mark Goldstein, Xintian Han, Aahlad Puli, Adler Perotte, and Rajesh Ranganath. X-cal: Explicit calibration for survival analysis. In Proceedings of the 34th Conference on Neural Information Processing Systems (NeurIPS 2020), 2020.
[16] Humza Haider, Bret Hoehn, Sarah Davis, and Russell Greiner. Effective ways to build and evaluate individual survival distributions. Journal of Machine Learning Research, 21(85):1-63, 2020.
[17] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint 1rXiv:1807.0331 2018
[18] Kaiwen Zha, Peng Cao, Jeany Son, Yuzhe Yang, and Dina Katabi. Rank-n-contrast: Learning continuous representations for regression. In Proceedings of the 34th Conference on Neural Information Processing Systems (NeurIPS 2024), 2024.
[19] Mohsen Nayebi Kerdabadi, Arya Hadizadeh Moghaddam, Bin Liu, Mei Liu, and Zijun Yao. Contrastive learning of temporal distinctiveness for survival analysis in electronic health records. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management (CIKM 2023), pages 1897-1906, 2023.
[20] Joshua Robinson, Ching Yao Chuang, Suvrit Sra, and Stefanie Jegelka. Contrastive learning with hard negative samples. arXiv preprint 0rXiv:2010.04592 2020.
[21] Mike Wu, Milan Mosse, Chengxu Zhuang, Daniel Yamins, and Noah Goodman. Condi- tional negative sampling for contrastive learning of visual representations. arXiv preprint arXiv:2010.02037, 2020.
[22] Feng Wang and Huaping Liu. Understanding the behaviour of contrastive loss. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2021), 2021.
[23] Ruijie Jiang. Thuan Nguyen, Prakash Ishwar, and Shuchin Aeron. Supervised contrastive learning with hard negative samples. arXiv preprini arXiv:2209.0007: 2022.
[24] Zhaoyang Zhang, Xuying Wang. Xiaoming Mei. Chao Tao, and Haifeng Li. False: False negative samples aware contrastive learning for semantic segmentation of high-resolution remote sensing image. IEEE Geoscience and Remote Sensing Letters, 19:1 19:1-5, 2022.
[25] Lingling Xu, Haoran Xie, Fu Lee Wang, Xiaohui Tao, Weiming Wang, and Qing Li. Contrastive sentence representation learning with adaptive false negative cancellation. Information Fusion, 102: 2024.
11

---

[26] Dara Bahri, Heinrich Jiang, Yi Tay. and Donald Metzler. Scarf: Self-supervised contrastive learning using random feature corruption. In Proceedings of the 10th International Conference on Learning Representations (ICLR 2022), 2022.
[27] David R Cox. Regression models and life-tables. Journal of the Royal Statistical Society; Series B (Methodological), 34(2):187-200 1972.
[28] Jared L Katzman, Uri Shaham, Alexander Cloninger, Jonathan Bates, Tingting Jiang, and Yuval Kluger. Deepsurv: personalized treatment recommender system using cox proportional hazards deep neural network. BMC medical research methodology, 18(1):1-12, 2018.
[29] Thomas A Gerds, Michael W Kattan, Martin Schumacher, and Changhong Yu. Estimating a time-dependent concordance index for survival prediction models with covariate dependent censoring. Statistics in medicine, 32(13):2173-21 21844444,, 2013.
[30] Erika Graf, Claudia Schmoor, Willi Sauerbrei, and Martin Schumacher. Assessment and comparison of prognostic classification schemes for survival data. Statistics medicine, 18(17-18):2529-555 1999.
[31] Humza Haider, Bret Hoehn, Sarah Davis, and Russell Greiner. Effective ways to build and evaluate individual survival distributions. Journal of Machine Learning Research, 21(85):1-63. 2020.
[32] Zifeng Wang and Jimeng Sun. Survtrace: Transformers for survival analysis with competing events. In Proceedings of the 13th ACM international conference on bioinformaties, compula- tional biology and health informatics, 2022.
[33] Changhee Lee, William Zame, Ahmed Alaa, and Mihaela Schaar. Temporal quilting for survival analysis. In The 22nd international conference on arlificial intelligence and statistics, 2019.
[34] Chirag Nagpal, Xinyu Li, and Artur Dubrawski. Deep survival machines: Fully parametric survival regression and representation learning for censored data with competing risks. IEEE Journal of Biomedical and Health Informatics, 25, 2021.
[35] Christina Curtis, Sohrab P Shah, Suet-Feung Chin, Gulisa Turashvili, Oscar M Rueda, Mark J Dunning, Doug Speed, Andy G Lynch, Shamith Samarajiwa, Yinyin Yuan, et al. The genomic and transcriptomic architecture of 2,000 breast tumours reveals novel subgroups. Nature, 486(7403):346-3322 2012.
[36] Norman E Breslow and Nilanjan Chatterjee. Design and analysis of two-phase studies with binary outcome applied wilms tumour prognosis. Journal of the Royal Statistical Society: Series C (Applied Statistics), 48(4):457-45) 1999.
[37] M Schumacher, G Bastert, H Bojar, K Hübner, M Olschewski, W Sauerbrei, C Schmoor, Beyerle, RL Neumann, and HF Rauschecker. Randomized 2 x 2 trial evaluating hormonal treatment and the duration of chemotherapy in node-positive breast cancer patients. german breast cancer study group. Journal of Clinical Oncology. 12(10)2006--2933, 1994.
[38] Terry Thernea et al. A package for survival analysis in S. R package version, 2015.
[39] William A Knaus, Frank E Harrell, Joanne Lynn, Lee Goldman, Russell S Phillips, Alfred F Connors, Neal V Dawson, William Fulkerson, Robert M Califf, Norman Desbiens, et al. The support prognostic model: Objective estimates of survival for seriously ill hospitalized adults. Annals of internal medicine, 122(3):191-203, 1995.
[40] Changhee Lee, Alexander Light, Ahmed Alaa, David Thurtle, Mihaela van der Schaar, and Vincent J Gnanapragasam. Application of a novel machine learning framework for predicting non-metastatic prostate cancer-specifie mortality in men using the surveillance, epidemiology, and end results (seer) database. The Lancet Digital Health, 2021.
12

---

A
Notations
The notations used in this work listed in Table 3
Table 3: Table of Notations.
Notation
Description
Ai.j
The indicator for acceptable pairs in the ranking loss The margin ensure minimum time difference
8
The balancing coefficient between the loss function The Event indicator for patient i
6
D
The survival dataset
fo
The hazard network function The encoder function
So
9v
The projection head function The latent representation space
H
The conditional hazard function
ENLL
The negative oggllikelhood loss The ranking loss
DRnnk
SSCCE
The survival outcome- outsome-based contrastive learning loss
M
The number of samples in a batch
R
The risk function
The survival function
The temperature coefficient for the weight function
V
The temperature coefficient for the SNCE loss The set of all possible discrete survival times The observed survival time for patient
T
T
T max
The maximum time horizon
&;
The corrupted version of the original sample x; The weight function
The normalization constant
X;
The input feature vector for patient
X
The input feature space
B Augmentation Method
In our experiments with tabular datasets, we compared augmentation methods, focusing on traditional noise injection (NI) and the state-of-the-ar method, SCARF [26]. For noise injection, we introduced random Gaussian noise into the latent space, determining the optimal corruption rate through hyperparameter search consistent with SCARE SCARF consistently showed performance that was similar to or better than NI. Consequently, we adopted SCARF our primary augmentation method for our experiments.
Table 4: Ablation study on the different tabular augmentation methods.
METABRIC
NWTCO
METHOD
C1
IBS 4
DDC +
D-CAL 1
CI
IBS 4
DDC 4
D-CAL
NI SCARF
0.650 0.688
0.096 0.094
0.052 0.097
0.003 0.089
0.704 0.729
0.059
0.250
0.137
0.078
0.222
0.360
GBSG
FLCHAIN
METHOD
C1
IBS
DDC J
D-CAL
CI
IBS J
DDC. J.
D-CAL
NI
0.681 0.696
0.1.5
0.070
0.000
0.788 0.810
0.088
0.247 0.291
0.096
SCARF
0.138
0.180
0.004
0.084
0.204
SUPPORT IBS
SEER
METHOD
C1
DDC. J.
D-CAL
CI
IBS 1
DDC. J
D-CAL
NI
0.612
0.204
0.149 0.155
0.000 0,000
0.842 0.854
0.014
0.975
0.999
SCARF
0.625
0.195
0.009
0.960
0.999
13

---

C Additional Results
Table 5: Comparison of time-dependent C-index.
METABRIC
NWTCO
Methods
25%
50%
75%
25%
50%
75%
CoxPH
0.653+0.00
0.645=0.011
0.637=0.0z
0.718+0.02
0.711 #0.025 0.25
0.706 0.706+0.00
DeepSurv
0.643.10.034
0..20
0.607 10.099
0.640440.08
0.63441.007 0.606
DeepHit
0.605
0.659
0.603
0.717 ::0
0.723
0. 0.70440.00 021
DRSA
0.6394..088
0.605410.02
0.592 00.02
0.713
0.706
0.70440.001
DCS
0.646..0.000
0.615 10.022
0. 0.606 0..024
0.675:10.026
0.657 10.000
0.643 10.029
X-CAL
0, 0,661+0.00 025
0.63220.00 018
0.619 019
0.687 026
0.650=a 029
0.628
ConSurv
0.686 0.028
2..20
0.645 0.023
0.718 0.025
0.709 10.024
0.705.10.25 100025
GBSG
FLCHAIN
Methods
25%
50%
75%
25%
50%
75%
CoxPH
0.71440.027
0.678+0.02 019
0.66440.0z
0.796
0.793
0.790
DeepSury
0.700++.00m
0.668 017
0.65440.011
0.787 :0
0.783 027
0. 0.780 028
DeepHit
0.63000.045
0.626
0.609 10.028
0.7964..0o
0.793
0.785 10.014
DRSA
0.725++066 687
0.687
0.67 + 00.066
0.776++.022 023
0.776
0.772
DCS
00.20
0.687 10.015
0.67 0.6744.0.017 10.017
0.788
0.783
0.779
X-CAL
0.727 =0.055
0.680 016
0.676 00.077
0.793 021
0,788
0.784+0.01z
ConSurv
00.20
0.689 100019
0.676 0.020
0.796 11966_..000 0.002
0.791 10.017
10.015
SUPPORT
SEER
Methods
25%
50%
75%
25%
50%
75%
CoxPH
0.603.10.008
0.60740.000
0.609.10.006
10.011
0.89010.002
0.855 10.022
DeepSurv
0.6033+.000 009
0.600==0.00
0.608
0.838+0.000
0.835 021
0.836+0.00
DeepHit
00.00
0.489 10.008 008
0.521 10.007
0.847 10.016
0.84 10.020
0.827 10.020
DRSA
0.586+0.00 0099
0.583+0.00 007
0.580 009
0,840 0773
0.839 0.77
0,820 083
DCS
0.604 10.00 000
0.602 1.. 009
0.602 10.009
0.864 1.0019
0.863 0..023
0.858 10.023
X-CAL
0.610+0 006
0. 00. 006
0.608 +0 007
0.8404+0.0m 023
0.839
0.833 010
ConSury
0.61611.000
0.617
0.618 0.006
0.870 0.017
0.866 0.026
0.86211.020
C.1
Performance on Time-dependent C-index and Brier score
While the CI and IBS provide valuable overall assessments of given survival model (over the entire time horizon), these metries cannot fully capture variations in model performance across different time points. To address this, we included dime-dependent performance evaluations, namely the time-dependent C-index (5) and the time-dependent Brier score 30 at three different time points (i.e., 25%, 50%, and 75% percentiles of time-to-event utilizing these dependent metrics may reveal subtle differences that might be obscured when using Cl and IBS alone. As shown in Table and we demonstrate the superiority of our method in discriminative power while preserving calibration. To evaluate the performance of survival models at various time points, select the 25%, 50%, and 55%-erccetele of each dataset.
in 32
33
34
It is worth highlighting that
C.2 Effect of Contrastive Learning
In Figure 5
When using LNuL alone, representations tend to cluster tightly in narrow space regardless of time. However, when applying LSNCE (contrastive learning with a time weight) to LNLL representations of features with similar times more likely to be positioned close each other.
C.3 Sensitive Analysis
Sensitive Analysis on Coefficient a is the margin ensure minimum time difference between samples in acceptable pairs. All the weights for the entire sample are precomputed. For setting the margin, calculate the percentile for only the weights corresponding to Case 2 mentioned in section
14

---

eovent time
only training
only training (a) SUPPORT
training
180
-33
120
ovant pranttimee
event pennttmme
NLL only after training
SNCE only after training
NLL + SNCE after training
(b) FLCHAIN
Figure 5: t-SNE visualization for latent representations learned with LxuL only, SNcc only, and ConSury for the FLCHAIN and SUPPORT datasets, colored by event times (for uncensored samples).
Table 6: Comparison of time-dependent Brier score.
METABRIC
NWTCO
Methods
25%
50%
75%
25%
50%
75%
CoxPH
0.167=2..099
2..14
0.033
0.111=0.009
0.11440.000 008
0. 0.11440.000
DeepSurv
0.17320.010
0.228
0.249 +0.048
0.121
0. 0.124
0.125
DeepHit
0.181::0.009
0.256=0.010
0.263 1263:10.226 026
0.151 1:0000000
0.143 2.00000000 0.008
0.154410.00
DRSA
0.2644+0.41
0.343+0.035
0.337 0.337 00.099
0.281 030
0. 0.29440.00
0.310 0.310+0.051
DCS
0.190
0.265510.00 038
0.293 20.000.... 088
0.119
0.107
0.122 +0.024
X-CAL
00.10
0.23444.022
0.2544
0.113
0.106
0.137 20.036
ConSurv
0.166=0.009
0.221 1 10.015
0.255
0.111
0.113
0.113 10.009
GBSG
FLCHAIN
Methods
25%
50%
75%
25%
50%
75%
CoxPH
0.145.2.010 100100
0.211 0.211:0.000
0.219
0.074
0.112
0.133
DeepSurv
0.146=00.01
0.212 009
0.221 00.013
0.076+4.000
0.1144.0.000
0.137 +0.011
DeepHit
0.168
0.242
0.250
0.096
0.181 0.181 10.1100.0..
0.2444 0.004
DRSA
0.21040.018
0.322 022
0.349 00000
0.141
0.25 027
0.323
DCS
0.153
0.210=0.00m 006
0.218 00.010
0.079+2..0s -0.008
0.119 103=000000 .008
10.014
X-CAL
0.150+0.011
0.207=a.007
0.217 0.002
0.080
0.117
0.144440.012
ConSurv
0.146
0.209 008
0.217 0.010
0.07770..00
0. 0.115
0.136
SUPPORT
SEER
Methods
25%
50%
75%
25%
50%
75%
CoxPH
0.219 00,003
0.197 10.008 00.
0.180+0o.0 009
0.005-00.00
0.009
0.014
DeepSurv
0.220+20.004
0.198+40.009
0.180+20.010
0.005_.0.00
0.010
0.016
DeepHit
0.340++..0g
0.282+0.00 003
0.240
0.016+4.00m
0.020
0.026+0.00
DRSA
0.285
0.269 +0.00000000 022
0.249 +0.021
0.01 1-0.00
0.021 +0.066
0.034-10.026
DCS
0.222 006
0.20944..0 015
0.202 017
0.005 0.001
0.010
0.015 0002
X-CAL
0.220+0.00 oos
0.207 2207:00.116 01.016
0.201
20..0...011 +0.018
0.006
001
0.01 1 002
0.019
ConSurv
0.219 o..00
0.195 0.000
0.177 0.000
0.005 1.05._..000 0..001
0.009 10.001
0.015
4.2
By defining this minimum time difference, the margin aids in reducing the ambiguity in survival outcomes, leading to a more precise comparison of patient risks. Specifically, this enhancement vital for improving the model's discrimination capabilities, as it removes potential false negative samples with little difference in their survival outcomes. Table determine the sensitivity the ConSurv with respect to the value of a, report the performance results four datasets by varying
15

---

5mc sme-to-cnssonng
tme-to-event (groud-truth)
Time Difference
3
22
(a) Censored samples without margin
Time Difference
16
(b) Removed censored samples with margin
Time Difference
10
20
22
Patients Index (with Event)
(c) Censored samples within margin a
Figure 6: Comparison of time differences based on censoring times and those based on the (unob- computed for each sample with event
served) ground-truth event times. Time differences with respect to the censoring time and event time of its corresponding comparable censored samples. (Samples are listed in ascending order of their event times.)
the value of a according to the following percentiles time differences: [5.0, 7.0, 10.0, 12.0, 14.0). Additionally, we compare the performance metries with no margin (i.e.,
Effect of Margin a. In F we compare time differences based on censoring times and the (unobserved) ground-truth event times. For each event sample, we compute the time differences between its event time and: 1) the (observed) censoring time of its corresponding comparable censored sample, and 2) the (unobserved) ground-truth event time of that same censored sample.
To show this effect, we have conducted an experiment using additional synthetic data by adapting the time-to-event generation process in DeepHit [1]
for both event times T; and censoring times C₁:
T: exp ((10x,,)² 52in), C. exp ((10r;,)² + 52
(11)
Here, we randomly generated 1,000 random samples, where a sample is censored when T > C..
Without margin, censored samples with censoring times close to event times may be incorrectly treated as having similar event times, leading very low weight assignments. To avoid this, we introduce a margin to exclude such samples when computing our contrastive loss. Overall, the margin in [1;-;77 7j| helps prevent failures accurately distinguishing risk between comparable censored samples.
Sensitive Analysis On Coefficient B. 3 the balancing coefficient in overall loss function where increased 3 makes the impact of CSNCE more dominant. Table g shows that when 3 (using the optimal for each dataset) relatively small (here. I 0.01), contrastive loss struggles to effectively learn similarities between survival outcomes, leading to poor discrimination performance. However, increasing the 3 ratio consistently improves performance. This suggests that our proposed contrastive learning approach does not prevent the survival model from learning effectively guided by the NLL loss.
16

---

Table 7: Impact of margin (a) on performance metrics.
METABRIC
NWTCO IBS
МETHOD
CI
IBS
DDC
D-CALT f
CI
DDC J
D-CAL
Q 0.695
0.088
0.100
0.256
0.765
0.065
0.481
0.001
0: 5
0.687
0.089
0.098
0.243
0.751
0.068
0.480
0.012
7
0.691
0.088
0.083
0.134
0.773
0.059
0.463
0.503
a 10
0.707
0.084
0.093
0.528
0.756
0.069
0.573
0.061
12
0.692
0.088
0.083
0.671
0.751
0.067
0.497
0.703
a 14
0.692
0.086
0.064
0.230
0.753
0.069
0.506
0.159
GBSG
FLCHAIN
METHOD
CIT
IBS 4
DDC
D-CAL f
CI
IBS 4
DDC J
D-CAL
0
0.691
0.153
0.201
0,000
0.779
0.100
0.306
0.731
0 = 5
0.693
0.153
0.199
0.000
0.771
0.100
0.254
0.929
Q = 7
0.695
0.151
0.182
0.000
0.783
0.090
0.280
0.693
10
0.694
0.154
0.224
0.000
0.779
0.100
0.270
0.963
a 12
0.673
0.157
0,241
0.000
0.778
0.095
0.258
0.919
14
0.670
0.159
0.236
0.000
0.777
0.098
0.310
0.929
SUPPORT
SEER
METHOD
C
IBS +
DDC
D-CAL1 f
CI 1
IBS 4
DDC 4
D-CAL
0: = 0
0.592
0.205
0.150
0.000
0.883
0.008
0.962
0.999
5
0.605
0.198
0.142
0.000
0.886
0.007
0.948
0.999
a = 7
0.604
0.202
0.145
0.000
0.874
0.012
0.984
0.991
a = 10
0.610
0.196
0.144
0.000
0.889
0.004
0.933
0.999
a 12
0.608
0.197
0.147
0.000
0.879
0.008
0.954
0.999
a = 14
0.598
0.203
0.149
0.000
0.877
0.007
0.952
0.999
Table 8: Impact of margin (3) on performance metries.
METABRIC
NWTCO
METHOD
CI
IBS
DDC
D-CAL
CI 1
IBS 4
DDC 4
D-CAL
B = 0.01
0.673
0.089
0.148
0.254
0.747
0.069
0.493
0.043
B = 0.1
0.682
0.124
0.109
0.320
0.759
0.091
0.470
0.095
8 1.0
0.706
0.086
0.093
0.524
0.773
0.078
0.222
0.428
B = 10.0
0.652
0.166
0.096
0.405
0.759
0.093
0.344
0.397
B 100.0
0.743
0.071
0.463
0.352
0.743
0.071
0.463
0.352
GBSG
FLCHAIN
METHOD
C
IBS 4
DDC 4
D-CAL
CI
IBS 4
DDC +
D-CAL
3 = 0.01
0.655
0.149
0.150
0.015
0.746
0.063
0.493
0.353
8 0.1
0.677
0.171
0.156
0.012
0.775
0.108
0.364
0.543
B = 1.0
0.703
0.138
0.180
0.075
0.816
0.096
0.338
0.465
B 10.0
0.686
0.169
0.139
0.002
0.777
0.107
0.354
0.537
B 100.0
0.694
0.170
0.363
0.057
0.743
0.071
0.551
0.057
SUPPORT
SEER
МETHOD
CI
IBS 4
DDC +
D-CAL
CI
IBS 4
DDC 4
D-CAL
8 = 0.01
0.594
0.180
0.134
0.000
0.831
0.016
0.983
0.999
8 0.1
0.603
0.185
0.139
0.000
0.842
0.020
0.982
0.999
8 = 1.0
0.610
0.183
0.132
0.000
0.854
0.010
0.975
0.999
B 10.0
0.605
0.194
0.145
0.000
0.832
0.016
0.984
0.999
8 100.0
0.601
0.179
0.152
0.000
0.821
0.021
0.988
0.999

---

C.4
Calibration
C.4.1
Calibration Plot
Figure shows calibration plots of ConSurv in comparison with the benchmarks on all the evaluated datasets. Here, similar to our observation in Section 5.3.2 ConSury consistently exhibits calibration performance that is similar or better than the benchmarks, effectively balancing discriminative and calibration power.
3.0
1.
1.
3.0
Couff!
68
6.9
Cod't
09
Cosfri
Ce
Coffii
0.7
6.7
0.7
0.7
67
0.6
08
0.5
03
0.5
0/4
0.4
044
02
0.3
03
0
62
0.0
0.
0.0
Cantury
Cansa' Deepgate
CHPRATY peephars
Conturs Deepoury
Camalatie
Predected CDF
Prod sled
Prezkted
Fresecod
1.5
03
08
Obernes Camvatie 2
08
66
R.
04
Contury DffA
Redton
Centary
Condary
ConSurn
CanGury
DRSA
DESA
DMSA
CESA
Camaltree
Prodicted CEF
Predecte
PrdS cTec
Peekttdd
Canmary
Carta'v
Cordurs
Caraury
Obarree Camaletie
Procected
Prezkted
X.CAL
00
K.C.L
09
NCAL
00
X-CAL
0.7
07
07
67
0m
0.
0.7
55
(a) NWTCO
(b) GBSG
(c) FLCHAIN
(d) SUPPORT
(e) SEER
Figure 7: Calibration plots for ConSurv in comparison with benchmarks.
18

---

C.4.2 Subgroup Analysis
In this section, we show that our model learns well about the differences between subgroups on real- world datasets, with good calibration performance by comparing the averaged survival predictions and the observed KM estimation within each subgroup. In Figure & we subgroup patients based on cellularity, which is a categorical variable representing the degree of density of cells the tumor. Similarly, in Figure (a), we partition patients based on the size of the tumor (with indicating 70% or more and indicating 30% or less of the total size), which have a significant impact on the event. In Figure (b), examine binary hormone receptor statuses progesterone receptor (PR).
Our results show that our method closely aligns with the KM curve, outperforming other deep learning- based survival models, especially those with ranking loss. We assess calibration performance using the Wasserstein distance compare survival predictions with KM curves across different subgroups. In Table 9 indicate that our model provides well-calibrated predictions within each subgroup.
Subgroup: cellularity
Subgroup: cellularity 2
Subgroup: cellularity 3
Survival Probabilit
Gurt
Cuatt
Cere
Dopule
Figure 8: Comparison of the survival curves across various patient subgroups in the METABRIC dataset.
Subgroup: size+
Subgroup: size-
Subgroup: PR+
Subgroup: PR-
Survicel Frobabilty
EC5
time
time
(a) size
(b) PR
Figure 9: Comparison of the survival curves across various patient subgroups in the METABRIC dataset.
Table 9: Subgroup-level Wasserstein distances the METABRIC dataset.
cellularity
PR
size
Subgroup
3
CoxPH
0.039
0.044
0.067
0.049
0.067
0.076
0.044
DeepSury
0.044
0.048
0.080
0.054
0.074
0.105
0.089
DeepHit
0.101
0.074
0.020
0.043
0.102
0.233
0.033
DRSA
0.215
0.168
0.203
0.205
0.276
0.118
0.234
DCS
0.125
0.060
0.168
0.154
0.126
0.091
0.124
X-CAL
0.113
0.143
0.184
0.159
0.148
0.060
0.176
ConSurv
0.018
0.044
0.063
0.043
0.042
0.060
0.025
19

---

C.5
Evaluating Performance Across Extended Seeds
For enhanced stability verification, we conducted additional experiments with 25 random seeds in Table TO
Table 10: Discrimination and calibration of survival models: mean and standard deviation values for CL, IBS, and DDC, along with the number of successful D D-calibration tests.
METABRIC
NWTCO
METHOD
CrT
DDC
D-CAI.
CI
IBS
DDC
D-CAL
CoxPH
0.645
0.175 1a,02s
0.111 122
25
0.7161
0.108 10.00s
0.5151o..2
10.022
DEEPSURV
0.625
0.18330.022
0.103
25
0.640
0.1 0.11720.011
0.79210.011
24
DEEPHIT
0. 0.004
ga.uus
0.292 239220.007
0
0,717 10.22s
0.657 20.024
DRSA
0.604
0.249
0.178
0
0.709 10.019
0. 0.281 10.04
0.21810.08s
DCS
0.612 can
0.206
0.054_.9am
2
0.642+0.036
0. 19
0.209 2404
19
X-CAL
0.6322n.c07
1o.025
0.065_o.cor co7
0.62219.057
0.128 10.025
0.191_aar
Cuu
0.642gn.ozz
0.077 co.a
10.24
0.109 gu.00s
0.556
Last, & Larx
0.659 can
0.193
0.080
21
0.71511.021
0.108 10.00
0.563
22
Cia Cras
0.247 ta.aso
0.177 20.70.000 cuo
0
0.717 40.a2
0.137 +0.00s
0.653
CONSURY
0.186+0.021 0.625
0.110
23
0.718
0.107
0.551=0.045
GBSG
FLCHAIN
METHOD
C
IBS 5
DDC L
D-CAL
CI1
IBS
DDC
D-CAL
CoxPH
0 0.662
0.181
0.183 037
25
0.790.0.012
0.103
0.287
25
DEEPSURV
0.653 042
0.182+0.009
0.153=0 ocs
24
0.779
0.106=a0.000
0,287=0.021
DEEPHIT
0.633
0.205
0.342
3
0.795
0.1 g0.00
0.49220.015
DRSA
0. 008
0.278+0..015
0.102=0.co oss
0
0.772+0.012
0 2..02
0.309
DCS
0.677 1am0
0,18110.00s
0.124 _aaas
10
0.779:40.16
03.13000000
0.363+o.136
X-CAL
0.675
0,181
1,166
8
0.783
0.1 1220.009
0.4022o.105
Cus
0. 016
0.1 0.179
0.154 oze
0
0.783
0. 1.106
20.04
19
Exes
0.669
0.179
0.155 o28
0.786
0. 1.10620.000
0,283
Lus & Cest
1.687200.01
0,280
0.263
0
0.78510.014
0.1 1.14611 10.000
0.303
CONSURV
0.677
0.179
0.100
18
0.796
0.105 1u.0s
0.280
23
SUPPORT
SEER
METROD
C
IBS +
DDC L
D-CAL
CI
IBS :
DDC
D-CAL
CoxPH
0.604=0.co
0,191+0.008
0.262
0
0.858
0.009
0.966+0.00z
25
DEEPSURV
0.603 own
0.192 s0.0u7
0.245 cos
0
0.814
0.010
1.000
DEEPHIT
0.503 coo
0.272+0.00s
0.337 oos
0
0.840+0.ass
0.020
0.83620.00s
DRSA
0.570=a.com
0.259 0.259 t0.015
0.486
0
0.834
0.021 e0.015
0.671 0.135
DCS
0.508
0.207 0.207 1o.012
0.175_1o.oo co0
0.860
1o.020
0.010
Lo.0c4
X-CAL
0.603=0.cor
0.204+0.0112
0,181
0.837 0
0.015+a.000
0.900+0.049
18
CULL
0.606
0,193+0.008
0.123
0
0.851
0.009=0.001
0.867
25
CaLL Locz
0. c0.00
0.192 t0.007
0.127 o ous
0
0.850
0.012+a.0us
so.64
EMLL Cyask
0.617 20.007
0.1 0.173+0.008
0.231 =0 024
0.862+0.017
0. 0.139
1.00000.000
CONSURY
0.616game ga.ss
0.1 0.190 20.006
0.148 -00033
0
0.864_9.14
0.009
20.001
0.863
25
D Dataset Description and Experimental Setup
D.1 Real-World Datasets
Table 11: Descriptive statisties of the real-world datasets
Dataset
No. Uncensored
No. Censored
No. Features (real, binary, categorical)
METABRIC
888 (55.2%)
1093 (44.8%)
21(6,0,15)
NWTCO
571 (14.2%)
3457 (85.8%)
6 (1,4,1)
GBSG
1267 (56.8%)
965 (43.2%)
7 (4,2,1)
FLCHAIN
4562 (69.95 (6999%)
1962 (30.0%)
8(4,2,2)
SUPPORT
6036 (68.1%)
2837 (31.9%)
14 (8,3.3)
SEER
604(1.11%)
53940(98.99))
12(4.5,3)
Table 11 shows baseline statistics of the real-world datasets. METABRIC 35 comprises clinical
features and gene expression profiles (overall 21 features) used determine the risk of breast cancer
20

---

subgroups for 1,981 patients. Among the total number of patients, 1,093 (55.2%) are censored, and 888 (44.8%) are uncensored. NWTCO [6] is a study related to Wilms' tumor, containing 4,023 patients with 6 clinical and histology features. Among the total number of patients, 3,457 are censored and (71(14.2%) are uncensored. GBSG [37] investigates the effects of hormone treatment on recurrence-free survival time, with the event of interest being breast cancer recurrence. The dataset consists of 21 clinical and tumor-related features for 2,232 patients. Among them, 965 (43,2%) are censored and 1,267 (56.8%) are uncensored. FLCHAIN [38] contains subjects from a study of the relationship between serum-free light chain and mortality, where each subject is described by features. Among 6,524 subjects, 1,962 (30.1% 1) are censored and 4,562 (69.9%) are uncensored. SUPPORT [39 focuses on the survival time of seriously ill hospitalized adults. The dataset consists of 14 clinical features for 8,873 patients, having 2,837 (31.9%) censored and 6,036 (68.1 1%) uncensored. SEER [40] provides data on prostate cancer patients, containing 12 clinical and cancer-related features for 54,544 patients. Among them, 604 (1.11%)
uncensored and 53,940 (98.9%) are censored. We split the data into train, test, and validation sets with a ratio of 0.64:0.20:0..0 and then apply min-max normalization to the input features.
D.2 Performance Metries
Integrated Brier Score (IBS) is metric to evaluate the mean squared error between the predicted survival curve with step function of the observed event. The IBS is the integration of the Brier score across all time points.
Distributional Divergence for Calibration (DDC) is novel metric introduced in 2 to assess the calibration quality of survival models. This metric computes the divergence between the empirical distribution of estimated survival probabilities at the observed event times and a uniform distribution. Specifically, DDC utilizes the Kullback-Leein Leibler (KL) Divergence between these two distributions.
It first bins the estimated survival probabilities S(ts|zz) at the observed event times into ten equal-width intervals across the [0.1] range.
It then calculates the KL divergence (R Q) between this binned distribution P and theoretical uniform distribution Q.
DDC scaled so that its values range between O and 1, with lower values indicating better calibration. A perfectly calibrated model, where the estimated probabilities perfectly match uniform distribution, would yield a DDC value of O.
DDC serves as more nuanced measure compared traditional metrics like the Brier score C-index because it directly evaluates how closely the distribution of the model's probabilistic predictions matches the expected uniform distribution of true probabilities. This direct assessment helps in determining whether the model's predictions are systematically biased or well-aligned with actual outcomes.
D- D-Calibration (D-CAL) assesses how well predicted survival probabilities align with observed outcomes based on goodness-of-itt test. D-CAL bins the predicted survival probabilities at the true event times into ten equal-width intervals from 0 to I, and performs a chi-squared test to ascertain the uniformity of the distribution, As suggested in [3 we reports p-values from the D-CAL test and the number of datasets with p-values greater than 0.05.
D.3 About Benchmarks
Benchmarks. We compare ConSurv with six survival models including one traditional statistical method and four state-of-the-a deep learning methods: CoxPH [27] is commonly used statistical
method under the proportional hazard assumption") DeepSurv 22 an extension of the Cox model
which utilizes DNN to predict individual hazard rates DeepHit [1 is DNN model which directly
models the joint distribution of the event time DRSA [10] is an RNN-based model which predicts
the likelihood of the true event occurrence and estimates the survival rate over time DCS (2) is
an RNN-based model that estimates calibrated individualized survival curves while maximizing the
Python package scikis---rittttttia htttg::///gthuu...ccaaaaaaa////p
httpg://ggtthbb..om//ccl888/////
httpss//ggttabb.oom//22////////
21

---

discriminative power and X-CAL [1]] is survival model designed to focus on the calibration performance by directly utilizing differentiable version D-CAL part of the objectives. Especially with DCS and X-CAL, we made every effort to faithfully reproduce the code with hyperparameter optimization, as the original code had several errors and we were unable to use it for our experiments.
E
Implementation
We discuss implementation of ConSurv below. The source code for ConSurv is available in Throughout the experiments, training ConSurv and its variants takes approximately 60 minutes to 2 hours on single GPU machine 10
htaps//gghbbb.omm/dooggzzz/////
E.1 Hyperparameter Specification
We perform a random search for hyperparameter optimization including the batch size, hidden dimension, depth, learning rates, corruption rates, and on the validation set and choose the settings with the best performance for ConSurv on each dataset. Table 12 describes the model specifications for the evaluated datasets. The candidates for random search can be given as {32, 64, 128, 256), [16, 32, 64), and (3, 5) for batch size, hidden dim, and depth respectively. We carefully choose the and for LSNce over (0.25, 0.5, 0.75), (5.0, 7.0, 10.0, 12.0, 14.0) and (0.03, 0.05, 0.07). Corruption rates are selected from a range of 0.1 to 0.9. train multiple components in our network simultaneously, we need to find different learning rates for each network Jo, fw. and fo. where the learning rate is chosen from a set { Ie-3, lc-4).
Table 12: Hyperparameter specifications
Dataset
Hidden Dim Batch Size Depth Learning Rate Corruption Rate
0
METABRIC
32
1e-4, le-4
0.5
0.75
10.0
NWTCO
8
32
le-4, le-3
0.7
0.75
7.0
0.05
GBSG
8
32
5
lc-4, 1c-3
0.7
0.75
7.0
0.05
FLCHAIN
16
128
le-3, 1e-3
0.8
0.25
7.0
0.03
SUPPORT
32
64
le-3, 1e-3
0.7
0.25
10.0
0.07
SEER
32
256
le-4, le-4
0.4
0.75
0.0
0.03
F
Broader Impact
In this work, highlight the direct and indirect influence of our approach through experiments real-world clinical datasets. The use these datasets was in accordance with the guidance of the respective data providers and domain experts. We acknowledge the potential for both positive and negative impacts associated with survival analysis techniques. While these methods can provide valuable insights for personalized treatment and risk stratification, emphasize the importance of careful interpretation and validation by domain experts before applying these predictions in clinical practice. Misuse misunderstanding survival analysis could lead unintended consequences for individuals or groups, such as inequitable access to care or discriminatory practices. Therefore, we advocate for transparency, accountability, and ongoing dialogue among researchers, clinicians, and the public to ensure the responsible and ethical use of survival analysis in healthcare.
G Limitation and Future Work
Survival analysis datasets often include features where a clear event time is not available, indicating significant number of censored patients. This makes it challenging to assign weights based on time differences and to compare them against each other. Therefore, utilizing such data for learning
http:://ggichbb..oo////////aaaaa httppe///githuab.om/r/aaaa/a com Latprrajjsssahaaaaaaaaa 1ab
Swweree--r-rvvvvva-vaa-- Analysis
The specification of the machine is CPU: INTEL XEON Gold 6240R, GPU: NVIDIA RTX A6000
22

---

purposes presents difficulties. Overcoming these limitations requires further research. Potential avenues include modifying models to account for such uncertainties developing alternative learning approaches. Additionally, there is need for new evaluation metrics or model developments that consider the characteristics of such data.
Furthermore, the tabular datasets we utilized often lack diverse augmentation techniques. Future research could explore augmentation methods suitable for survival datasets enhance contrastive learning performance. For example, techniques considering temporal variations or reflecting uncertain- ties events could be investigated. Such research endeavors are expected enhance understanding in the fields of survival analysis and contrastive learning, contributing to improved model performance.
H Pseudo-code for ConSurv
Algorithm 1 Pseudo-code for the ConSury
Input:
Dataset D = {xx; Tr, b batch size M, margin a, coefficient 3 encoder 0. projection head v hazard network d, learning rate f/1, 7/2-
i=1+
Output: ConSury parameters (f, R), ()
Initialize parameters (0, vh () repeat
Sample a mini-batch of {(x; Ti, 6:) ~ D for l...., .M do
Draw augmentation function: s~S
Augmentation pair: x,,
x x, X;, s(x;)
Latent representation: h, h fx()),, fo (x())
Map to the embedding space: Z, 2 Io (h:), (h) (h,)) end for
for all = 2M and j 1.... 2M do
Cosine similarity in the embedding space: sx;, xj) = 2. [112:([[zz]l)))
Weight function for negative distribution: w( :i)
Redefined weight function considering the right-censoring: क(, ;T,) = b.g w(T ;T))
end for Update the encoder and projection head parameter 6, v/:
(0, 1b) - (0.4) 7/2 (8.0))
1 2M B.
log e((a)at))
2M 1==
23 (xx xx)) (
for i= = L,,.., M do
Compute latent representations: h, fo(x;)
Compute estimates for survival function: Si(Ti(x;) < Iless (1 (o;,((h;,!)
Compute estimates for probability mass
function: fe(T)(x;) + fa Ti) [eer (1 th;,
end for
Update the encoder and hazard network parameters (0,
(8, σ) + (0, ((e00)
-MM DM 5 (og((()(((;) 5.) log S(r:)x)))
until convergence
23

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility. transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and precede the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justifiicton right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]", ", it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS paper checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification:
The claims in the abstract and introduction correctly describe the paper's
contributions. We provide experiments in Section 5 to support these claims.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
24

---

Justification:
We have provided Limitations after the conclusion.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "I "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model sele--peciiiatinn. asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: In (6), we demonstrate how importance sampling can be used to approximate the negative distribution of our loss function, defined in (8).
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Full experimental details are provided in Appendix D and implementation details are provided Appendix [..] We have included our code in the supplementary material and plan to make it publicly available on GitHub upon acceptance.
25

---

Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that are appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Our code currently included the supplementary material and will be made available on GitHub upon acceptance. Detailed instructions are provided in the readme md file within the supplementary material. Real-world datasets used in our experiments are cited with their respective references; no new datasets were generated for this work.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. ec/ sublic/guidess htte://uddeoooooouiiiisessses
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https //nips cc/ wwwpcc/pplcc....c httpll/c/gu.dess/ddeeebbbiiii
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
26

---

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Complete experimental details, including training procedures and the hyperpa- rameter configurations tested and selected for each dataset, are provided in Appendix [ Train/validation/tes splits and preprocessing steps for each dataset are detailed in Appendix
D.I
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: In Section 5 we report the mean and standard deviation across five random
wraiiin/validatioontestng splits.
Guidelines:
The answer NA means that the paper does include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
27

---

Answer: [Yes]
Justification: We conduct all the experiments on a local machine with CPU: Xeon Intel Xeon Gold 6240R and GPU: NVIDIA RTX A6000. Please refer to Appendix E
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e-g.. preliminary failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics htttps://neurip:: wwtpcccppppccaeeeeeeeeeeeeeeeeee pptpblic//thicssuudelnnnee
Answer: [Yes]
Justification: This paper fully complies with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We have included a Broader Impact statement following the Conclusion. Given that both this research field and our specific work are still the early stages of development and not yet deployed, we do not anticipate any immediate adverse effects.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out, For example, it is legitimate to point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
28

---

11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our research does entail such risks, as elaborated in the Broader Impact statement. We engage in foundational exploration of methodologies and do not proceed with deploying a fully trained model.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
We have appropriately referenced all methodologies and datasets used in our study. Whenever necessary, have provided the licenses for datasets in the Dataset Details (Appendix [.I] and Implementation Details (Appendix E. Furthermore, we specify the date of access for any dataset method utilized.
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, haperswithcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Our novel contribution lies in methodology and its practical application. The code accompanying this paper is available the supplementary materials and will be made publicly accessible GitHub upon acceptance.
Guidelines:
29

---

The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mooel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our research does not rely on crowdsourcing; all datasets utilized are publicly accessible.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation,
or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Since we employed openly accessible datasets that have been deidentified, there is no necessity for IRB approval for our study.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
30


Reviews:
[{"review_id": "HceeoEJAx2", "reviewer": "Reviewer_RNpF", "summary": "This paper presents an approach to handle survival analysis datasets using a method to improve both calibration and discrimination. This adds a bunch of novelties to the field like handling right-censoring and an SNCE loss to handle calibration and ranking simultaneously (NLL + contrastive loss). This paper presents a reliable way to look at survival analysis, write a hazard function which is quite reasonable and also present a ranking loss to take care of the domain-specific problem of stratification by risk-levels using generated patient rank list.", "questions": "I understood why there is censored data, but if the authors could explain with an example of a time series what censored data looks like that would make it easier.", "limitations": "maybe added unstructured data like doctors notes", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "-> novelties added quite a lot: SNCE loss and handling right-censoring\n-> ranking loss introduced can be generally used for any method aiming to generate or classify along ranked lists\n-> well presented math for different loss functions and easy to follow", "weaknesses": "-> lack of using unstructured data\n-> more clarity of censored data", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# [ Response to Reviewer RNpF ]\nWe thank the reviewer for the positive feedback on our work. We have included the details of censoring in the General Response Section. \n## (Missing Experiments on Unstructured Data)\nUnfortunately, despite the potential of the contrastive learning framework utilized in our method, we were unable to find suitable unstructured survival data for immediate application. To the best of our knowledge, the only available dataset is the METABRIC dataset, which contains whole slide images (WSIs) of breast tumors [R4, R5] with time-to-event information for each sample. However, these WSIs present a challenge: each sample contains a varying number of image segments (collected from different locations of a tumor). Addressing this would require either domain expertise to choose the correct tumor region of interest or a deep learning technique (such as multiple instance learning) to handle samples with varying numbers of image segments. As applying augmentation in this setting is not straightforward, it was beyond the scope of our current work and remains an interesting avenue for future research.\n\n---\n### *Reference*\n[R4] H. Xu et al., \"A whole-slide foundation model for digital pathology from real-world data,\" Nature, 2024.\n\n[R5] P. Mobadersany et al., \"Predicting cancer outcomes from histology and genomics using convolutional networks,\" National Academy of Sciences, 2018.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XHtHW9Wqel", "reviewer": "Reviewer_Lfcy", "summary": "The paper proposes a contrastive learning loss to regularize maximum likelihood learning of discretized survival times. The main contribution of this work is the utilization of the Laplace kernel to weigh negative pairs inversely proportional to their time difference from the anchor sample, while also accounting for censoring. Specifically, negative pairs with event times close to that of the anchor sample contribute less to the contrastive loss. Experimental results on four datasets demonstrate that the proposed approach achieves a high Concordance Index (C-Index) without a significant loss in calibration when compared to baselines.", "questions": "- What are the values of $\\alpha$ and $\\beta $ for the reported results?\n- Were alternative weighting kernels considered?\n- Were alternative approaches for constructing positive pairs considered?", "limitations": "- The paper does not discuss modeling assumptions; for example, the assumed independent censoring mechanism could be violated in practice.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "The use of contrastive learning to improve model calibration in survival analysis seems interesting. Experimental results demonstrate the efficacy of this approach in maintaining a high C-index without significant loss in calibration, a trade-off that is difficult to achieve in practice.", "weaknesses": "*Given that the main contribution of this work is the application of contrastive loss to survival analysis, the paper lacks complete details for reproducibility:*\n- There are no details on how the model functions $ f_{\\theta}, f_{\\phi}$, and $g_{\\psi} $ are parametrized.\n- Details on the construction of positive pairs are very sparse. No motivation is provided in terms of why the SCARF approach is chosen and why it should be expected to work with survival data.\n- The paper proposes the Laplace Kernel for weighing samples; it is unclear if alternative kernels were also considered.\n- The notation seems sloppy; symbols ${f}$ and $Z$ are overloaded.\n- I expect calibration and the C-Index should be sensitive to $\\beta $; the paper does not discuss how to choose $\\beta $ and what values resulted in the performance results reported. This phenomenon has been discussed in detail in [1]. I encourage the authors to include a comprehensive discussion on this trade-off.\n- It is unclear how the margin parameter $\\alpha$ was chosen. I encourage the authors to provide visualizations in the main paper to demonstrate sensitivity to both $\\alpha$ and $\\beta $.\n\n*The proposed approach relies on discretizing event times, which is sensitive to the time-binning approach. Additionally, it seems the contrastive loss could be an indirect way to achieve consistency in event times. I encourage the authors to also benchmark with approaches that directly predict event times such as [2].*\n\n*The four datasets used seem to be of low sample size. I encourage the authors to also consider larger datasets such as SUPPORT and SEER.*\n\n**References**\n\n- [1] Qi et al., \"Conformalized Survival Distributions: A Generic Post-Process to Increase Calibration\", ICML 2024\n- [2] Chapfuwa et al., \"Calibration and Uncertainty in Neural Time-to-Event Modeling\", IEEE TNNLS 2020", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# [ Response to Reviewer Lfcy ]\nWe thank the reviewer for the valuable suggestions on our work. We have addressed the reviewer’s comments in our response below.\n## (Sensitive Analysis of the $\\alpha$)\nIntroducing a margin prevents a comparable pair of samples with distant (unobserved) event times from being mistakenly treated as similar samples due to small differences between the observed event time of one sample and the censored time of the other sample within that pair. To show this effect, we have conducted an experiment using additional synthetic data by adapting the time-to-event generation process in DeepHit [R2] for both event times $T_i$ and censoring times $C_i$: \n\n$T_i \\sim \\exp\\left((10 x_{i1})^2 + 5 x_{i3}\\right)$\n\n$C_i \\sim \\exp\\left((10 x_{i2})^2 + 5 x_{i4}\\right)$\n\nHere, we randomly generated 1,000 random samples, where a sample is censored when $T_i > C_i$.\n\nIn Figure 1 of the Global PDF, we compare time differences based on censoring times and the (unobserved) ground-truth event times. For each event sample, we compute the time differences between its event time and: 1) the (observed) censoring time of its corresponding comparable censored sample, and 2) the (unobserved) ground-truth event time of that same censored sample. Without a margin, censored samples with censoring times close to event times may be incorrectly treated as having similar event times, leading to very low weight assignments. To avoid this, we introduce a margin to exclude such samples when computing our contrastive loss. Overall, the margin in $|\\tau_i-\\tau_j| \\geq \\alpha$ helps prevent failures in accurately distinguishing risk between comparable censored samples.\n## (Sensitive Analysis of the $\\beta$)\n*Table D. Impact of balancing coefficient $\\beta$ on performance metrics.*\n|Dataset|M||||N||||G||||F||||\n|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|\n|**Methods**|**CI**|**IBS**|**DDC**|**D-CAL**|**CI**|**IBS**|**DDC**|**D-CAL**|**CI**|**IBS**  |**DDC**|**D-CAL**|**CI**|**IBS**|**DDC**|**D-CAL**|\n|$\\beta$=0.01|0.673|0.089|0.148|*0.254*|0.747|**0.069**|0.493|0.043|0.655|0.149|0.150|0.015|0.746|**0.063**|0.493|*0.353*|\n|$\\beta$=0.1|0.682|0.124|0.109|*0.320*|0.759|0.091|0.470|0.095|0.677|0.171|0.156|0.012|0.775|0.108|0.364|*0.543*|\n|$\\beta$=1.0|**0.706**|0.086|**0.093**|*0.524*|**0.773**|0.078|0.222|*0.428*|**0.703**|**0.138**|0.180|*0.075*|**0.816**|0.096|**0.338**|*0.465*|\n|$\\beta$=10.0|0.652|0.166|0.096|*0.405*|0.759|0.093|0.344|*0.397*|0.686|0.169|**0.139**|0.002|0.777|0.107|0.354|*0.537*|\n|$\\beta$=100.0|0.687|**0.078**|0.127|*0.492*|0.743|0.071|**0.463**|*0.352*|0.694|0.170|0.363|*0.057*|0.743|0.071|0.551|*0.057*|\n\nF. Kamran et al. [R6] demonstrates that while ranking loss functions used in survival analysis focus on discriminative power, they often perform poorly on calibration metrics. Similarly, Qi et al. [R7] points out the independence of discrimination and calibration. They show that models with perfect discrimination can still have significant calibration problems. This suggests that finding a good balance is crucial for improving the model performance.\n\nWe investigated the relationship between discrimination and calibration, focusing on the impact of the balancing coefficient $\\beta$. In Table D, additional experiments with $\\beta=0.1$ and $\\beta=10.0$ show that a very small $\\beta$ (0.01) primarily focuses on NLL, resulting in good calibration. As $\\beta$ increases, discriminative power improves while maintaining calibration performance. However, if $\\beta$ is too large (beyond 1.0), it results in deviating from the concept of employing auxiliary loss, thereby leading to suboptimal discriminative and calibration performance. An optimal balance is achieved at $\\beta=1.0$, performing well on both metrics. This highlights the need for hyperparameter searches to achieve optimal performance, as detailed in Appendix C.3.\n\n## (Details of Generating Positive/Negative Samples)\nWe construct positive and negative samples using the contrastive learning framework for survival data [R8]. SCARF addresses the challenge of tabular data augmentation while maintaining semantic integrity. Augmented versions $\\tilde{x}^{(i)}$ of data points $x^{(i)}$ are created by selectively corrupting feature subsets using their marginal distributions. The augmented reference sample serves as the positive pair, while augmented versions from other mini-batch samples are the negative pairs. This permutation method avoids out-of-distribution issues common with noise injection described in Appendix B and is applicable to various data types. Further details have been provided in the updated manuscript.\n\n## (Laplacian Kernel)\nWe compared several kernel functions, especially the exponential and standard kernels. These kernels assign lower values to closer samples and higher values to farther samples. Our main goal, however, was to identify a kernel that not only maintains this distance-weighting property but also stabilizes at a finite value instead of increasing infinitely. Ultimately, we selected a modified version of the Laplacian Kernel, which has the property of having weights converge rather than diverge as the distance between samples increases.\n## (Clarification on Notations)\nEach model function—$f$ (encoder $f_\\theta$, hazard $f_\\phi$) and $g$ (projection head $g_\\psi$)—uses an MLP. Parameters were optimized using random search as detailed in Appendix E.2. We have maintained the representation as $z$ and updated the normalizing constant to $C$.\n\n---\n### *Reference*\n[R2] C. Lee et al., \"DeepHit: A Deep Learning Approach to Survival Analysis with Competing Risks,\" AAAI, 2018.\n\n[R6] F. Kamran et al., \"Estimating calibrated individualized survival curves with deep learning,\" AAAI, 2021.\n\n[R7] Qi et al., \"Conformalized Survival Distributions: A Generic Post-Process to Increase Calibration\", ICML 2024.\n\n[R8] D. Bahri et al., “Scarf: Self-supervised contrastive learning using random feature corruption,” ICLR, 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wUNhz44TFk", "reviewer": "Reviewer_1ZZS", "summary": "The authors study discrete-time survival analysis, proposing to train models using a loss function that combines the NLL loss and a modified NCE loss. This NCE loss is modified to take the survival outcomes (event times) into account, mitigating the effect of potential false negatives that have small event time differences.\n\nThe method is evaluated on 4 tabular clinical survival datasets, comparing against 6 baseline models and 3 method variations. The proposed method performs well compared to all baselines.", "questions": "- Could you perhaps apply the proposed method also to some non-tabular dataset?\n\n- The visualization in Figure 2 is really neat, could you add this for the three other datasets as well (to the appendix)?\n\n\n\n\nMinor things:\n- 130: \"by minimizing the following negative NLL loss\", remove \"negative\"?\n\n- Perhaps add 1-3 sentences to the start of Section 3, just to describe what is covered here (and why this is covered, why this is relevant for your method)?\n\n- 248: \"We compare our proposed method and the benchmarks with...\", \"benchmarks\" --> \"baselines\"? The same also for line 251?", "limitations": "Yes.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is well-written overall.\n\n- The proposed method is described well in Section 4.\n\n- The proposed method is conceptually quite simple and intuitive. To add a contrastive loss term, and for this use a modified contrastive loss that takes the survival outcomes into account, makes intuitive sense.\n\n- The proposed method seems to perform well compared to baselines, both in terms of discrimination and calibration. In particular, it performs well compared to the baseline of replacing the proposed modified NCE loss term with standard NCE.", "weaknesses": "- The technical novelty/innovation is perhaps somewhat limited (\"just\" adding a contrastive loss term).\n\n- The experimental evaluation could perhaps be more extensive, include more datasets. Would have been nice to see the method being applied also to some non-tabular dataset.\n\n\n\n\nSummary:\n- Somewhat limited technical novelty, but this is a well-written paper with a simple/intuitive method that seems to perform well compared to reasonable baselines. Thus, I am definitely leaning towards accept.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# [ Response to Reviewer 1ZZS ]\nWe thank the reviewer for the positive feedback on our work. We have addressed the comments in the updated manuscript and provided a point-by-point response below.\n\n## (Technical Novelty about ConSurv)\nConSurv enhances the discriminative power of survival models by using a contrastive learning framework combined with a weighted sampling technique based on the similarity of event times. Instead of directly changing risk or survival predictions, this method focuses on distinguishing samples in a latent space according to their survival outcomes. It assigns weights to samples based on differences in their survival outcomes, focusing the learning process more on important samples. While Kerdabadi et al. [R9] uses a contrastive learning framework to time-to-event analysis, ConSurv is the first approach to use hard negative samples specifically to increase discriminative power for survival analysis. In addition, ConSurv achieves excellent calibration performance without using a separate loss function specifically for calibration [R6, R10]. This is achieved as the contrastive learning process inherently preserves good calibration while enhancing discriminative capabilities.\n\n## (Challenges of applying the proposed method to non-tabular data.)\nUnfortunately, despite the potential of the contrastive learning framework utilized in our method, we were unable to find suitable non-tabular survival data for immediate application. To the best of our knowledge, the only available dataset is the METABRIC dataset, which contains whole slide images (WSIs) of breast tumors [R4, R5] with time-to-event information for each sample. However, these WSIs present a challenge: each sample contains a varying number of image segments (collected from different locations of a tumor). Addressing this would require either domain expertise to choose the correct tumor region of interest or a deep learning technique (such as multiple instance learning) to handle samples with varying numbers of image segments. As applying augmentation in this setting is not straightforward, it was beyond the scope of our current work and remains an interesting avenue for future research.\n\n## (Additional Experiment: t-SNE visualization)\nWe have included a t-SNE visualization for each of the real-world datasets we used in our experiments in the Global PDF (Figure 2). We show experimental results for FLCHAIN and SUPPORT, where performance improvements are significant. When using the NLL alone, the representations tend to cluster tightly in a narrow space, regardless of time. However, when applying SNCE (contrastive learning with a time weight) to NLL, representations of features with similar times are more likely to be positioned closer together. This shows that our model achieves superior performance.\t\n\n\n## (Clarification on Notations)\nWe thank the reviewer for pointing out typos and suggestions for better clarification. We have addressed those comments in the updated manuscript.\n\n---\n### *Reference*\n[R4] H. Xu et al., \"A whole-slide foundation model for digital pathology from real-world data,\" Nature, 2024.\n\n[R5] P. Mobadersany et al., \"Predicting cancer outcomes from histology and genomics using convolutional networks,\" National Academy of Sciences, 2018.\n\n[R6] F. Kamran et al., \"Estimating calibrated individualized survival curves with deep learning,\" AAAI, 2021.\n\n[R9] Kerdabadi et al., “Contrastive learning of temporal distinctiveness for survival analysis in electronic health records,” CIKM, 2023.\n\n[R10] M. Goldstein et al., “X-CAL: explicit calibration for survival analysis,” NeurIPS, 2020.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UOP0TAERqD", "reviewer": "Reviewer_VjbS", "summary": "This work concerns deep learning for survival analysis. Deep learning models for survival analysis are typically desired to have good discriminative performance, where a model can differentiate between patients of different risk profile, and calibration performance, where the time-to-event is accurately predicted by the model. Methods which seek to improve discrimination performance have often led to poorer model calibration. Consequently, this paper proposes an auxiliary contrastive loss to improve the discriminative performance of deep survival models, while still achieving high calibration performance. The novelty in this method is adapting the SOTA contrastive learning method for tabular data, SCARF, by weighting the negative pairs according to the difference in time-to-event. Hence, samples with greater difference in time-to-event are considered more important negative pairs, enforcing an inductive bias to differentiate the latent representations of samples with different time-to-event.  After providing a detailed problem formulation, the contrastive loss and generation of positive and negative pairs is described. The results of several experiments are provided which compare the proposed method with current state-of-the-art methods when applied to a number of real world datasets, and demonstrate the proposed model performance under ablation of each term in the loss function. From these experiments it is concluded that the auxiliary contrastive loss achieves superior discriminative and calibration performance.", "questions": "The questions below focus on the details of the experiments that were unclear. Clarity on these questions may change the overall opinion of the paper. \n\nTable 1: In the caption you refer to the standard deviation - are the reported values in this table the mean? \nTable 1: Does the standard deviation refer to variation over different data splits, model initialisations or something else? \nTable 1: What do bold and underlined values represent? \nTable 1: How do you determine that the superior performance of ConSurv is statistically significant? \n\nLine 528: Can you confirm whether min-max normalisation was performed before or after data was split into train, test and validation sets? \n\nLine 236: What is the marginal corruption process? By what augmentation were positive pairs generated?\n\nLine 197: What does the temperature coefficient scale? What impact does this have on model training? \n\nLine 104: Does ref. [18] show that model training is destabilised by the contrastive loss? If not, what supports this statement? Could this model have been included in your benchmarks?\n\n\nNOTE: Due to the discussion in the author-reviewer discussion period, I feel that many of these points were clarified. In particular new results have better demonstrated the statistical significance of the performance of ConSurv. I raise my score from a 4 to a 6 to reflect this.", "limitations": "The authors have addressed the potential negative societal impact of their work, stating that model predictions from such work should be under scrutiny by domain experts before being applied in a clinical setting. They stress that incorrect use could lead to inequitable access or discriminatory practices. While this is welcome, it is not specific to this work, but is a concern of all works proposing machine learning for clinical practice. A potential impact more specific to this work might consider why the inductive bias introduced by the auxiliary contrastive loss would lead to unintended consequences for AI-supported decisions on treatment and intervention. For example, the unsupervised nature of the contrastive learning procedure could align patients based on features which may act as a proxy for protected traits and lead to inequitable model performance amongst different demographics. \n\nAdditionally, a discussion of limitations is provided but is limited. The authors correctly identify that a significant number of patients are censored in real world datasets, and this undermines the importance sampling based on difference in time-to-event, as weights are assigned according to a function in equation (5). It would improve the discussion of the limitations if it was pointed out that seeking alternatives or amendments to (5) would be a specific avenue to improve upon this work. \n\nThe limitation section could benefit from greater specificity. The authors propose “modifying models to account for such uncertainties or developing alternative learning methods”. The uncertainties they refer to are unclear, and a preferred statement might be: future avenues of work would be to develop a model which retains the benefit of contrastive learning with respect to discriminative performance, whilst also being agnostic to the number of censored samples within the training set.\n\nThey then state “there is a need for new evaluation metrics or model developments that consider the characteristics of such data”. The clarity of this proposal might be clearer if stated with more specificity i.e. \"to develop a model that retains high discriminatory and calibration performance for datasets with arbitrary proportions of censored samples, new metrics must be established that can compare the same model applied to different proportions of censored samples.\"", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "The strength of this work lies in its originality, quality and clarity. The paper proposes an interesting adaptation to SCARF, a contrastive method for tabular data, by weighting the negative pairs such that samples with greater difference in time-to-event are considered more important negative pairs. Recent work on contrastive learning and deep survival methods are cited and appropriately discussed. \n\nTechnically, the paper is sound, with a good mathematical foundation for the importance sampling in the weighted contrastive loss term, and a thorough set of experiments which establishes performance against important baselines and provides good support for modelling choices via ablation studies. Furthermore, the clarity of the paper is for the most part good: the description of the background, methodology, and experiments are clear and detailed.", "weaknesses": "While the technical parts of this paper are clearly explained, the organisation and presentation of the paper could be improved. In particular, the introduction uses quite a lot of terminology specific to survival analysis that is unexplained or explained in detail only later in the paper. As a specific example, censoring is first mentioned on line 67, but is not clearly explained. Explaining this terminology, which may not be familiar to all of NeurIPS wide readership, at first use would improve the clarity of the paper. Presentation and clarity of the paper would also be improved if all captions were self-contained. In some cases, for example in Table 1, it is difficult to interpret the results as presented, and this could have been avoided if the captions were more detailed. Specifically, it is not clear what the standard deviations represent in this table i.e. variation across model initialisations or data splits.   \n\nThis work should be praised for providing a list of experiments that should provide the evidence needed to make the claim that the proposed model provides superior discriminative performance while maintaining calibration performance. However, the clarity and quality of these experiments is sometimes below expectations, and this ultimately reduces the overall significance of this work. Table 1 is used to report that the proposed model is the best performing model. However, it seems that the model is within 1 standard deviation of the second best performing model, and hence the statistical significance of this result is unclear. As previously mentioned, while mean and standard deviation of performance metrics are reported, it is not entirely clear what distribution they refer to – whether it be different data splits, or model initialisations via random seed. This makes it difficult to interpret the significance of these results, and also makes it more difficult to reproduce. \n\nThe paper, therefore, addresses an important problem, proposes a novel and well thought out method to address this issue, but the significance is undermined by a lack of clarity and ambiguity surrounding the statistical significance of reported results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# [ Response to Reviewer VjbS ]\nWe thank the reviewer for the valuable suggestions on our work. We have addressed the reviewer’s comments in our response below.\n## (Details of Table 1)\n- Mean and Standard Deviation: The values in Table 1 represent the mean performance metrics, with standard deviations provided in Appendix C.1, Table 5. These standard deviations show the variation across 10 random data splits for each model, ensuring robustness and reliability. We have updated the manuscript to clearly refer to the appendix for details on standard deviation.\n- Meaning of Bold and Underlined values: Bold values indicate the models with the best performance for each dataset. Underlined values indicate where the D-CAL measure is above 0.05, meaning that calibration is guaranteed. \n- Statistical Significance of ConSurv’s Performance: To assess the statistical significance of the performance comparison, we conducted Welch’s t-test. In Table E, we present the findings where the improvement was statistically significant (i.e., p-value < 0.05). We have provided a detailed explanation and statistical analysis to substantiate our claim in the updated manuscript. \n\n*Table E. p-value for Performance Metrics.* \n|Model|Metric|METABRIC|NWTCO|GBSG|FLCHAIN|SUPPORT|SEER|\n|-|-|-|-|-|-|-|-|\n|CoxPH|CI|0.004|0.264|0.047|0.325|0.001|0.047|\n||IBS|0.204|0.022|0.612|0.033|0.315|0.000|\n|DeepSurv|CI|0.031|0.013|0.009|0.115|0.003|0.000|\n||IBS|0.302|0.034|0.487|0.046|0.229|0.000|\n|DeepHit|CI|0.020|0.385|0.010|0.301|0.436|0.404|\n||IBS|0.110|0.000|0.000|0.300|0.000|0.000|\n|DRSA|CI|0.000|0.185|0.385|0.000|0.000|0.050|\n||IBS|0.000|0.000|0.000|0.000|0.000|0.000|\n|DCS|CI|0.000|0.000|0.346|0.040|0.030|0.191|\n||IBS|0.000|0.240|0.312|0.040|0.000|0.000|\n|X-CAL|CI|0.025|0.000|0.491|0.128|0.050|0.050|\n||IBS|0.012|0.025|0.212|0.044|0.060|0.000|\n## (Details of Generating Positive/Negative Samples)\nWe construct the positive and negative samples based on the contrastive learning framework for tabular data [R8]. Specifically, we generate augmented versions $\\tilde{x}^{(i)}$ of each data point $x^{(i)}$ by randomly selecting a subset of features (up to a pre-specified corruption rate) and replacing the value by sampling the corresponding feature's marginal distribution. The augmented version of the reference sample is used as the positive pair, and the augmented versions of the remaining samples in the mini-batch are used as the negative pairs. This corruption process is applied regardless of the feature types. We have included the details of generating the positive and negative samples in the updated manuscript. \n## (Additional Experiments: New Baselines)\nIn Table F, we have included a new baseline SupWcon which is a variant of our method where we replace $\\mathcal{L}_{SNCE}$ with a supervised contrastive loss that directly weights negative samples based on the actual time differences of the comparable pairs, without considering their distributional properties of these weights [R11]. This can make weighting of negative samples sensitive to the scale of the time differences, potentially reducing the robustness of the contrastive learning loss. Notably, while weighted contrastive learning based on the difference in time-to-events show slight performance gains, the improvement is not significant when compared to ours. This suggests the importance of applying good inductive bias when contrasting samples, potentially because it encourages the model to focus on discriminating the survival prediction based on a proper distribution defined based on the differences in time-to-events.\n\n*Table F. Discrimination and calibration of survival models for ConSurv w/ $\\mathcal{L}_{SupWcon}$.*\n|Data|CI|IBS|DDC|D-Cal|\n|-|-|-|-|-|\n|METABRIC|0.637±0.025|0.198±0.032|0.106±0.029|0.098±0.109|\n|NWTCO|0.721±0.042|0.106±0.012|0.558±0.055|0.571±0.483|\n|GBSG|0.675±0.027|0.186±0.018|0.179±0.025|0.000±0.000|\n|FLCHAIN|0.781±0.021|0.105±0.006|0.314±0.053|0.332±0.394|\n## (Temperature coefficient scale)\nWe thank the reviewer for insightful feedback regarding the use of $\\sigma$ in our time-to-event contrastive learning framework. Here, we explain how $\\sigma$ values affect model performance.\nWhen $\\sigma$ is very small, the weight for time differences becomes excessively large, causing even minor time differences to be overstated and all negative samples to be treated equally. This leads to the model ignoring time-to-event information and behaving like standard contrastive learning. Conversely, with a very large $\\sigma$, the weight for time differences becomes negligible, making the model insensitive to these differences. As a result, all samples receive similar weights, which also reduces the framework to standard contrastive learning.\nOur experiments ensure these findings, demonstrating that $\\sigma$ must be chosen carefully. We observed that for contrastive learning to function effectively, $\\sigma$ needs to be in a range where the scale of the weight applied to time differences does not excessively diverge from the model output. If the weight values become too extreme, either too large or too small, the model's performance is adversely affected. Thus, selecting an appropriate $\\sigma$ that balances the impact of time differences is crucial for optimal contrastive learning performance.\n## (Implementation Details)\nMin-max normalization was performed before the data was split into train, test, and validation sets. \n## (Clarification on Limitations)\nWe thank the reviewer for pointing out the need for clarification on limitations. We understand the importance of discussing the potential biases in detail and have incorporated your suggestions to make our proposals clearer in the updated manuscript.\n\n---\n### *Reference*\n[R8] D. Bahri et al., “Scarf: Self-supervised contrastive learning using random feature corruption,” ICLR, 2022.\n\n[R11] Kerdabadi et al., “Contrastive learning of temporal distinctiveness for survival analysis in electronic health records,” CIKM, 2023.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
# [ General Response to the Reviewers ]
We thank the reviewers for taking their valuable time to provide insightful comments and suggestions for the paper. We believe the thoughtful reviews and recommendations have substantially improved the quality of the paper. In this response, we aim to address the common comments raised by the reviewers and outline the major changes made:
- Results on additional baselines including DRAFT [R1] 
- Results on additional large real-world datasets (i.e., SUPPORT and SEER)
- Explanations on censoring in time-to-event analysis

## (Additional Experiment: A New Baseline)
We have included a new baseline, DRAFT [R1], which is a deep learning approach to accelerated failure time model assuming a log-normal distribution for the underlying distribution of event times. This baseline directly models event times and hence does not rely on discretization that many deep survival models (e.g. DeepHit [R2], DRSA [R3], and our method) use.  In Table A, we show that while DRAFT demonstrates comparable performance to other deep learning baselines, our method achieves superior performance in both discrimination and calibration.

*Table A. Discrimination and calibration of survival models for DRAFT.*
| **Data**   | **Model** | **CI**   | **IBS**   | **DDC**   | **D-Cal** |
|-|-|-|-|-|-|
| **METABRIC**   | DRAFT |0.651±0.005 | 0.209±0.023 | 0.176±0.027 | 0.194±0.387 |
| 	              | ConSurv | 0.661±0.040 | 0.189±0.019 | 0.010±0.026 | 0.234±0.234 |
|  **NWTCO** | DRAFT | 0.701±0.002 | 0.125±0.009 | 0.364±0.061 | 0.000±0.000 |
|                      | ConSurv | 0.731±0.037 | 0.100±0.017 | 0.195±0.042 | 0.653±0.454 |
|  **GBSG** | DRAFT | 0.678±0.018 | 0.228±0.006 | 0.356±0.049 | 0.000±0.000 |
|                   |  ConSurv | 0.683±0.026 | 0.178±0.011 | 0.172±0.019 | 0.012±0.027 |
| **FLCHAIN** | DRAFT | 0.788±0.015 | 0.117±0.017 | 0.660±0.018 | 0.000±0.000 |
|                       |  ConSurv | 0.794±0.023 | 0.087±0.045 | 0.293±0.039 | 0.317±0.338 |


## (Additional Experiments: Large Real-world Dataset)
We thank the reviewer for suggesting additional experiments on large real-world datasets. Additionally, we have conducted experiments on the SUPPORT (8873 samples) and the SEER Prostate (54544 samples). As presented in Tables B and C, our proposed method not only significantly outperforms all baseline models in terms of discriminative power, but also demonstrates superior calibration performance across these relatively large datasets.

*Table B. Performance for SUPPORT dataset.*
| **Data**   | **Model**   | **CI**   | **IBS**   | **DDC**   | **D-Cal** |
|-|-|-|-|-|-|
| **SUPPORT**| **CoxPH**   | 0.603±0.006 | 0.196±0.006 | 0.258±0.006 | 0.000±0.000 |
|                        | **DeepSurv**| 0.596±0.014 | 0.198±0.009 | 0.244±0.035  | 0.000±0.000 |
|                        | **DRAFT**| 0.593±0.020 | 0.244±0.024 | 0.377±0.033 | 0.000±0.000 
|                        | **DeepHit** | 0.615±0.007 | 0.275±0.002 | 0.336±0.003  | 0.000±0.000 |
|                        | **DRSA**    | 0.521±0.048 | 0.268±0.022 | 0.523±0.047 | 0.000±0.000 |
|                        | **DCS**     | 0.582±0.045 | 0.216±0.016 | 0.142±0.033  | 0.000±0.000 |
|                        | **X-Cal**   | 0.611±0.007 | 0.212±0.016 | 0.191±0.034 | 0.000±0.000 |
|                        | **ConSurv** | 0.615±0.007 | 0.194±0.006 | 0.142±0.014 | 0.000±0.000|

*Table C. Performance for SEER dataset.*
| **Data**   | **Model**   | **CI**   | **IBS**   | **DDC**   | **D-Cal** |
|-|-|-|-|-|-|
| **SEER** | **CoxPH**   | 0.858±0.016 | 0.009±0.001 | 0.967±0.005 | 1.000±0.000 |
|                  | **DeepSurv**| 0.764±0.037 | 0.014±0.004 | 1.000±0.000  | 1.000±0.000 |
|                  | **DRAFT**| 0.825±0.049| 0.014±0.004 | 0.804±0.017 | 0.126±0.182 |
|                  | **DeepHit** | 0.870±0.014 | 0.100±0.001 | 0.336±0.003  | 0.000±0.000 |
|                  | **DRSA**    | 0.840±0.051 | 0.017±0.001 | 0.523±0.047 | 0.000±0.000 |
|                  | **DCS**     | 0.860±0.014 | 0.010±0.001 | 0.142±0.033  | 0.439±0.513 |
|                  | **X-Cal**   | 0.838±0.041 | 0.009±0.001 | 0.191±0.034 | 0.459±0.425 |
|                  | **ConSurv** | 0.865±0.014 | 0.004±0.003 | 0.142±0.014 | 1.000±0.000|

## (Details of Censoring)
Censoring is a crucial concept in survival analysis (also known as time-to-event analysis). It refers to cases where the event of interest (e.g., death) is not observed for some individuals by the end of the study, either due to the study ending before the event occurs or the individual being lost to follow-up. Since not all events are observed, survival data are frequently right-censored; dealing with censored samples is a critical aspect in survival analysis.

Hence, the survival data for an individual patient $i$ consists of either a time-to-event $T_i \in \mathbb{R}^+$ or a time-to-censoring $C_i \in \mathbb{R}^+$, and an indicator $\Delta_i = \mathbb{I}(T_i < C_i)$; $\Delta_i = 1$ if the patient experienced the event of interest and $\Delta_i = 0$ if the patient was right-censored. Here, in survival analysis, censoring provides the information that the patient had not experienced the event (e.g., was alive) up to time $C_i$.
We will provide a clear explanation of censoring in survival analysis in the updated manuscript for a general audience.

---
### *Reference*
[R1] P. Chapfuwa et al., "Calibration and Uncertainty in Neural Time-to-Event Modeling," IEEE TNNLS, 2020.

[R2] C. Lee et al., "DeepHit: A Deep Learning Approach to Survival Analysis with Competing Risks," AAAI, 2018.

[R3] K. Ren et al., “Deep recurrent survival analysis,” AAAI, 2019.

Meta-review: I would like to thank the authors and reviewers for engaging in a fruitful discussion. The reviewers mostly leaned toward acceptance (1 strong accept, 2 weak accepts, 1 borderline reject). I would consider this paper borderline, especially as the strong accept review was unusually terse, and the borderline reject review asks for additional baselines, which while reasonable to request, I realize can be challenging for the authors to fully address within the author/reviewer discussion period. I have decided to recommend acceptance for this paper, as I view the contribution of this paper as useful for the community to know even if there are very clear limitations (that I think the authors should address either with additional experiments or at least with some discussion in the manuscript text).

Specifically regarding the additional baselines requested by reviewer Lfcy: I agree with the authors that CSD was published so recently that it is fine not to compare against it, and that, furthermore, CSD is quite different in that it can be used to modify different existing survival models. However, I agree with reviewer Lfcy that comparison with DATE is actually possible (one can compute standard survival analysis metrics using DATE), so adding DATE as a baseline would be a helpful comparison.

I would encourage the authors to acknowledge that DATE and CSD exist and could potentially be good at both discrimination and calibration. Even if the authors do not plan on adding experiments, at least discussing these other methods as worthwhile to compare against in the future would be helpful. Specifically for CSD, since it works quite differently, an interesting question would be how much of a performance boost does CSD applied to various models (including the proposed method) yield (with the idea that if the base model being modified already does a great job with calibration and discrimination, then you'd expect applying CSD to be unnecessary/to lead to less of an improvement).
Decision: Accept (poster)