Towards simple time-to-event modeling: optimizing neural networks via rank regression
Authors: Hyunjun Lee
Junhyun Lee
Taehwa Choi
Jaewoo Kang
Sangbum Choi
Venue: ICLR 2022 Submitted (2022)
Keywords: time-to-event analysis
survival analysis
semiparametric method
accelerated failure time

Abstract: Time-to-event analysis, also known as survival analysis, aims to predict the first occurred event time, conditional on a set of features.
However, the presence of censorship brings much complexity in learning algorithms due to data incompleteness.
Hazard-based models (e.g. Cox's proportional hazards) and accelerated failure time (AFT) models are two popular tools in time-to-event modeling, requiring the proportional hazards and linearity assumptions, respectively. 
In addition, AFT models require pre-specified parametric distributional assumptions in most cases. 
To alleviate such strict assumptions and improve predictive performance, there have been many deep learning approaches for hazard-based models in recent years. 
However, compared to hazard-based methods, AFT-based representation learning has received limited attention in neural network literature, despite its model simplicity and interpretability. 
In this work, we introduce a Deep AFT Rank-regression for Time-to-event prediction model (DART), which is a deep learning-based semiparametric AFT model, and propose a $l_1$-type rank loss function that is more suitable for optimizing neural networks. 
Unlike existing neural network-based AFT models, the proposed model is semiparametric in that any distributional assumption is not imposed for the survival time distribution without requiring further hyperparameters or complicated model architectures. 
We verify the usefulness of DART via quantitative analysis upon various benchmark datasets. 
The results show that our method has considerable potential to model high-throughput censored time-to-event data.

Under review as a conference paper at ICLR 2022
TOWARDS SIMPLE TIME-TO-EVEN MODELING: OPTI- MIZING NEURAL NETWORKS VIA RANK REGRESSION
Anonymous authors Paper under double-blind review
ABSTRACT
Time-to-event analysis, also known as survival analysis, aims to predict the first occurred event time, conditional on set of features. However, the presence of censorship brings much complexity in learning algorithms due to data incom- pleteness. Hazard-hased models (e.g- Cox's proportional hazards) and acceler- ated failure time (AFT) models are two popular tools in time-to-event modeling, requiring the proportional hazards and linearity assumptions, respectively. ad- dition, AFT models require pre-specified parametric distributional assumptions in most cases. To alleviate such strict assumptions and improve predictive perfor- mance, there have been many deep learning approaches for hazard-based models in recent years. However, compared to hazard- based methods, AFT-based rep- resentation learning has received limited attention in neural network literature, despite its model simplicity and interpretability. In this work, we introduce a Deep AFT Rank-regression for Time-to-even prediction model (DART), which is a deep learning- based semiparametric AFT model, and propose 41-type rank loss function that is more suitable for optimizing neural networks. Unlike existing neu- ral network-baas based AFT models, the proposed model is semiparametric in that any distributional assumption is not imposed for the survival time distribution with- out requiring further hyperparameters verify the usefulness of DART via quantitative analysis upon various benchmark datasets. The results show that our method has considerable potential to model censored time-to-event data.
complicated model architectures. We
INTRODUCTION
Time-to-event analysis, also known as survival or failure time analysis, is major statistical approach in various fields such as biostatistics, medicine, and economics to estimate either risk scores or the distribution of event time, given a set of features of subjects (Viganò et al., 2000; Cheng et al.) 2016 Dirick et al.] 2017 Li et al. 2021). There are benefits of assessing risk quantifying survival probabilities but, for all that, time-to-event analysis itself is challenging because of the existence of censoring. In real-world studies, subject (e.g. patient in medical research) can drop out before events of interest (c.g. death) happen, so that one can not follow them up (Leung et al. 1997). The presence of censoring survival data can create serious challenge in applying standard statistical modeling strategies. Usually, the censoring process is assumed to be non-informative in that it is irrelevant of the underlying failure process given features, but should be properly accounted for, otherwise leading to biased results.
The most popular and standard approach for modeling time-to-event data is use Cox's propor- tional hazards (CoxPH) model. CoxPH relates a conditional hazard to given features in multi- plicative form between the baseline hazard function and exponentiated regression component, and consequently learns relative risks. It often works the assumptions of proportional hazards and lime-invariant covariate-effects, which are difficult to follow in the real world (Aalen 1994). Statis- tical testing procedures, such as Schoenfeld's test, are usually conducted to examine these underly- ing assumptions since many Cox-based analyses are vulnerable to violation model assumptions. (Aaeen & Gjessing 2001 Kleinbaum & Klein 2010).
The accelerated failure time model (AFT) or accelerated life model relates the logarithm of the fail- time linearly to the features. As result of its direct physical interpretation and the connection

---

Under review as a conference paper at ICLR 2022
with linear models, this model provides attractive alternative to the CoxPH for the regression analysis of censored failure time data. Unlike CoxPH, standard AFT model parametrizes the un- derlying time-to-event distribution up to set of finite- mineeedimeniional parameters such as Weibull and log-normal (Lee & Wang, 2003). However, imposing distributional assumption is too strict in real data analysis and can reduce the attractiveness inherent in the AFT model, mostly underperforming Cox-based analysis (Cox) 20088] Kleinbaum & Klein 2010). Recently, based on statistical theories and the advent of deep learning techniques, various time-to-event models have been explored to circumvent the necessity of assumptions such as linearity, single risk, discrete time, and fixed-time
effect (Katzman et al.] 2018 Lee al. 2018 Ren et al.) 2019 Kvamme Borgan
2019: Avati
et al. 2020 Tarkhan et al. 2021 Rahman et al. 2021).
For example, Cox-Time (Kvamme et al. 2019) and DATE (Chaptuwa et al. 2018) alleviate the most fundamental but strict assumptions of the CoxPH and parametric AFT models by achieving non- proportional hazards and con-parmettric event-time distribution, respectively. Cox-Time exploits the neural network relative risk function to model interactions between time and covariates. The authors also show that the proposed loss function is a good approximation for the Cox partial log- likelihood. DATE is conditional generative adversarial network for implicitly specifying a time- to-event distribution of ATF model. It does not require the pre-specified distribution parametric form, instead, the generator learn it from the data with the adversarial loss function. Incidentally, various deep taarning-baseden approaches have been spotlighted to improve performance by resolving issues such as temporal dynamics and calibration (Lee al., 2019; Nagpal al. 2021, Gao & Cui 2021 Kamran & Wiens 2021; Hu et al.) (021). Therefore. it became important to utilize well- designed objective functions that fit not only statistical backgrounds but also optimization of neural networks.
In this paper, we introduce a Deep AFT Rank-regressson for Time-to Time-to-event prediction model (DART), a deep learning-based semiparametric AFT model trained with an objective function orig- inated from Gehan's rank statistic. The model does not require specifying event time distribution while keeping the advantage of the standard AFT model that directly predicts event time. With a simple form of the loss function, by constructing comparable rank pairs, the optimization of DART is efficient compared to other deep learning-based time-to-event models. Experimental results show that DART' is not only well-calibrated but also competitive in event order prediction performance even compared to muzard-based models. Furthermore, believe that this work be widely ap- plied in the community while giving prominence to advantages of the AFT model that is relatively unexplored.
2 RELATED WORKS
We first overview time-to-event modeling focusing on the loss functions of Cox-Time and DATE models to highlight the difference in concepts before introducing our method. The primary interest of time-to-event amalysis is to estimate survival quantities like survival function S(t) = P(T 2 1) or hazard function h(t) lims-+0 P(t S T S t + 8 2 t)/5, where T E R+ denotes time-to-even random variable. In most cases, due to censored observations, those quantities cannot be directly estimated with standard statistical inference procedure. In the presence of right censoring, Kaplan & Meier (1938) and Aalen (178)) provided consistent nonparametric survival function estimators, exploiting reght-censoring time random variable C E +R. Researchers then can get stable estimates for survival quantities with data tuples {v,, 8,, X. where y, min (T;, C.) is the observed event time with censoring, 5, I(T; < C,) is the censoring indicator, and vector of features X; E R... Here, N and P denote the number of instances and the number of features, respectively. While those nonparametric methods are useful, one can improve predictive power by incorporating feature information in a way of regression modeling. Cox proportional- -hazards (CoxPH) and accelerated- failure-time (AFT) frameworks are the most common approaches in modeling survival quantities utilizing both censoring and features.
2.1 HAZARD-BASE MODELS
A standard CoxPH regression model (Cox 1972) formulates the conditional hazard function as:
h({),,) ho(() exp (8 Xi), (i 1....,N,,
2

---

Under review as a conference paper at ICLR 2022
where (-) is unknown baseline hazard function which has to be estimated honparametrically. and 3 E I. is the regression coefficient vector. It is one of the most celebrated models in statistics in that 3 can be estimated full statistical efficiency while achieving nonparametric flexibility ho under the proportionality assumption. Note the model is semiparametric due to the unspecified underlying baseline hazard function hg- Letting R, be the set of all individuals "at risk", meaning that are not censored and have not experienced the event before T;, statistically efficient estimator for regression coefficients can be obtained minimizing the loss function with respect to
LCoxPH(3) Σ
8g log
Σ JER.
exp | X3 3 xx)
which is equivalent the negative partial log-likelihood function of CoxPH model.
Based on this loss function, Kvamme et al. (2019) proposed a deep-learning algorithm for the hazard-based predictive model, namely Cox-Time, replacing 3 X; and 3 X; with g Xy; 0) and g Xz,0), 0), respectively. Here, g(-) denotes the neural networks parameterized by 0, and R. would be replaced by R,, representing the sampled subset of R, With a simple modification of the standard loss function in Eq. 2 Cox-Time can alleviate the proportionality for relative risk, show- ing empirically remarkable performance against other muzard-based models in both event ordering and survival calibration.
2.2 SEBELERRTEEDFAILLUEE
MODELS
The conventional AFT model relates the log- teggtransformed survival time to set of features in linear form:
log = 3" + Et, (i (i= (N.,)..,,N))
(3)
where 6, is an independent and identically distributed error term with a common distribution function Fo(.) that is often assumed be Weibull, exponential, log-normal, etc. As implied in Eq. (3). AFT model takes a form of linear modeling and provides an intuitive and physical interpretation on event time without detouring via the vague concept of hazard function, making it a powerful alternative to hazard-based analysis. However, imposing a parametric distributional assumption for e, is a critical drawback of the model, for which model in Eq. 3 could be subclass of the hazard-based models.
To alleviate linearity and parametric distributional assumptions, several works brought the concept of generative process and approximated the eгтor distribution via neural networks like generative adversarial network (GAN) (Miscouridou et al. Chapfuuaa et al. (2018). Especially, (Chaffuaa al. (2018) proposed a deep adversarial time-to-event (DATE) model, which specifies the loss function as:
LDATE (a, σ) = E(x.,)=F (o Ex- Do(X, Go(X, 1))1 + +x (max y - 0))
(4)
+ Aa ( 1)]
where 0, 6 denotes the parameter set associated with a generator Go and a discriminator Dø, re- spectively, (A2, As)) are hyperparameters tune censoring trade-off, Fno y) and Fc(X, y) are empirical joint distributions for non-censored cases and censored cases, respectively, and Fe is the simple distribution, such as uniform distribution. The generator Go implicitly defines event time dis- tribution. Despite DATE achieves prominent survival calibration via the geaera-iggnraiingg process, the objective function is quite complicated and the GAN framework is inherently prone mode col- lapse, i.e. the generator learns only a few modes of the true distribution while missing other modes (Srivastava et al., 2017). Also, when optimizing neural networks with multiple loss functions, difficult to balance and there might be conflicts (i.c. tradc-off) with each term (Dosovitskiy & Djolongal 2020). Therefore, their loss function might be difficult to be optimized as intended and requires burdening training time, and consequently not be suitable for large-scale time-to-event analysis.
In the statistical literature, there have been many attempts to directly estimate regression coefficients in the semiparametric AFT model, where the error distribution F, is left unknown, rather than imposing specifie parametric distribution or exploiting generative models. In this work, bridge
3

---

Under review as a conference paper at ICLR 2022
Optimized by
logT
B
L
Feature representation
Error distribution
Loss function
pre-specified distribution (e.g. Weibull, log-normal)
DRAFT
Ae
F
INLL LpR
DATE
Generator 6ç
implicitly defined distribution (by the generator)
Log + Lop + Leenss + List
not requiring pre-specification modeling
DART (ours)
de
LRank
Figure I: Illustration of conceptual differences between deep learning-basee AFT models in terms of their respective contributions and required assumptions with format of the standard AFT. To allevi- ate the parametric distribution assumption, which DRAFT has, DATE exploits the GAN framework and learns the implicit underlying distribution go through the generator parameterized by 0. For DRAFT, LNLL and LpR denote negative log- liglliieldoood and partial ranking likelihood, respectively. DATE basically requires four loss functions: LG, LD, for the generator and the discriminator, Leens for adjusting censoring distribution, and Dist for the distortion penalty. Compared to the others, DART does not require pre-specification or modeling for error distribution and trained with simple loss function supported by statistical theory.
non-linear representation learning an objective function for estimation of semiparametric AFT model, which is originated from Gehan's rank statistic. By extensive quantitative analysis, we have shown the beauty of simplicity and compatibility of rank-based estimation, along with outstanding experimental performance.
METHOD
In this section, introduce the concept of DART, followed by predictive analysis for survival quan- tities. The conceptual differences with the other neural network-based AFT models are illustrated in Figure The semiparametric AFT is distinct from a parametric version in that the error distribution function Fb is left completely unknown like the baseline hazard function in the CoxPH. By further exploiting neural networks, we propose DART model that can be formulated as a generalization of model in Eq.
log T: = ((x; c, 1,.1,...,NN) N),
(5)
where g(X;; f)) denotes arbitrary neural networks with input feature vector X; and a parameter set outputting single scalar value as predicted log-scaled time-to-event variable. With this simple and straightforward modeling, DART entails several attractive characteristics over existing AFT-based models. First, the semiparametric nature DART enables flexible estimation error distribution, allowing improved survival prediction via neural network algorithms for Fo. Second, the restrictive log-linearity assumption of AFT model can be further alleviated by exploiting deep neural networks. Specifically, while standard AFT model relates time-to-event variable to feature variable in linear manner, deep learning is able to approximate any underlying functional relationship. lessening lin- earity restriction. Although DART still requires -egtrranformeed time target variable, its deep neural network redeems the point with powerful representative performance supported by universal approximation theorems, enabling automated non- loneernear feature transformation (Leshno al. 1993.
Schäferr & Zimmermann 2006 Zhou 2020).
3.1 PARAMETER ESTIMATION VIA RANK BASED LOSS FUNCTION
In statistical literature, many different estimating techniques have been proposed for fitting semi- parametric AFT model (Tiatis 1990 Jin et al. 2003 2006 Zeng & Lin 2007). Among them, we shall adopt the lityype rank-based loss function by taking into account the censoring infor- mation, which is efficient and suitable for stably fitting neural networks. We also describe two
4

---

Under review as a conference paper at ICLR 2022
alternative optimization methods in the appendix extensions of DART. Letting residual term e, = ed) = log y [(x;; f). the objective loss function for DART can be formulated as:
1 N
L (0)
Σ de(e, - -,) 2 ee,,,
(6)
where ((-) is the indicator function that has value when the condition is satisfied, otherwise 0. The estimator / can be obtained by minimizing the loss function with respect to model parameter set 0. be conveniently conducted via batched stochastic gradient
Optimization of model parameters descent (SGD). Notice that the loss function in Eq. (6) involves model parameter 0 only, with- concerning estimation of the functional parameter Fo. enabling simple time-to-event regression modeling.
Strength of the loss function theoretical consistency of optimization without requiring any addi- tional settings. Let the neural network be expressed: g(X;; o. () W,, where W E RR is transformed feature vector through hidden layers with parameter set Φ. and IR is a parameter set of linear output layer. Then, it is easy to see that the following estimating function is the negative gradient of the loss function with respect to
1 N
U Rank (3)
(W, W,)o(logv 3 W, log Uj -3- W,) set 0.
Eq. is often called the form of Gchan's 's rank statistic Jiin ct al. 2003). testing whether B is equal to true regression coefficients for linear model log T; WW + e,. and the solution to the estimating equation 3 is equivalent to the minimizer of Eq. (6) with respect to B. This procedure entails nice asymptotic results such as vnn-consistency and asymptotic normality of 8 under the counting processes logic, assuring convergence of / towards true parameter 3 as the number of instances gets larger (Tsiatis) 1990, Jin al., 2003). Although these asymptotic results might not be directly generalized to the non non-linear predictor function, expect that hidden layers would be able to assess effective representations W; with non-linear feature transformation, as evidenced by extensive quantitative studies. Note that, to keep theoretical alignment, it is encouraged to set the last layer as a linear transformation with an output dimension of to mimic the standard linear model following non-linear representation. In addition, robust estimation against outlying instances can be attained, depending rank of residual terms along with their difference.
3.2 PREDICTION OF SURVIVAL QUANTITIES
Predicted output 9 0) from trained DART model represents estimated expectation of log T; con- ditional on X,, i.e. mean log-transformed survival time with given feature information of ith in- stance. However, estimating survival quantities (e.g. conditional hazard function) cannot be directly done for AFT-based models. Instead, we utilize the Relson-Aalen estimator (Aalen 1978), veri-
fied to be consistent under the rank-based semiparametrie AFT model (Park & Wei 2003). De-
fine (t: 0) EN (t) and Y (t) Σn Yt(t), where N.(t) I(e; 6, 1) and
Y,(t) I(e > t) are the counting and the at-risk processes, respectively. Then the Nelson-Aales
estimator of (t) is defined by
Ho(!)= = L (YY > 0} dN(u). Y(u)
(8)
The resulting conditional hazard function given X; is defined by
x(t) = ho ((x;; 8)}1 expl (xp[=9(x,0])
(9)
where no(-)) aff(()) is pre-trained baseline hazard function using Nelson-Aalen estima- tor. Consequently, conditional survival function can be estimated by relationship Š(t]X,) expf- ((tt]X))tt) ).), providing comparable predictions to other time-to-event regression models. In practice, training set is used to get pre-trained Nelson-Aalen estimator.
5

---

Under review as a conference paper at ICLR 2022
4 EVALUATION CRITERIA
In this section, we evaluate models with two metrics for quantitative comparison: concordance index (CI) and integrated Brier score (IBS).
Concordance Index. Concordance of time-to-event regression model represents the proposition:
if target variable of instance i is greater than that of instance j. then the predicted outputs of should be greater than that of j. By letting target variable and predicted outcome y, concordance probability of survival model can be expressed as P(ù: > Ø |/: fj). and concordance index mea- sures the probability with trained model for all possible pairs of datasets (Harrell et al. 1982). With hazards survival regression models like Cox-Time [ee et al. (2018), however, Harrell et al. (1982) cannot be used to measure discriminative performance properly. For fair com- parison survival regression models, dime-dependent concordance index (Antolini et al. 20005) or Od was used for those baseline models proposed by Kvamme et al. (2019) to account for tied events. Cdu E [0, 1] can regarded as AUROC curve for time-to-event regression model, denot- ing better discriminative performance for value close 1. Note that standard concordance index yields identical results with Cdd for AFT-based models.
non-proportional aaaaapoorionnll-aaaaad
Integrated Brier Score. Graf et al. (1999) introduced generalized version of Brier score (Brier 1950) for survival regression model along with inverse probability censoring weight (IPCW), which can be described as:
S(t]X,)>I(H) tt, 1) 1) N (1 S(E(X))) >
BS(() 1
(10)
G(y,)
G(t)
where G(t) P(C > t) is Kaplan-Meie estimator for censoring survival function to assign IPCW. BS(() measures both how well calibrated and discriminative predicted conditional survival function: given time point is greater than y/s then S({]X,) should close to Integrated Brier score (IBS) accumulates BS for certain time grid [y]:
IBS to t1
BS(s)ds.
(11)
If S(( = 0.5 for all instances, then IBS becomes 0.25, thus well-fitted model yields IBS lower than. For experiments, time grids can practically be set minimum and maximum of y, of the test set, equally split into 100 time intervals.
5 EXPERIMENTS
In this section, we describe our experiment design and results to validate performance of DART compared other time-to-event regression models. Experiments are done with four real-world sur- vival datasets, and baseline models provided by Kvamme et al. (2019) and Chapfuwa et al. (2018). using evaluation metrics mentioned in previous section.
Table 1: Summary of survival datasets.
DATASET
SIZE
FEATURES
% CENSORED
WSDM KKBoX
2,646,746
15
0.28
SUPPORT
8,873
14
0.32
FLCHAIN
6,524
0.70
GBSG
2,232
7
0.43
5.1 DATASETS
We choose three common survival datasets and single large-scale dataset provided by Kvamme et al. (2019). The descriptive statistics provided in Table First, three common survival datasets that used in this work are the Study to Understand Prognoses Preferences Outcomes and Risks of Treatment (SUPPORT), the Assay of Serum Free Light Chain (FLCHAIN), and the Rotterdam

---

Under review as a conference paper at ICLR 2022
Table 2: Mean and standard deviation of Ctd The boldface denotes best performance. PMF denotes method parameterizing the probability mass function. HAZ and AFT denote hazard-based and AFT-based methods, repectively.
MODEL
WSDM KKBox
SUPPORT
FL.CHAIN
GBSG
PMF
DeepHit
0.553 (0.002)
0.645 (0.009)
0.797(0.015)
0.684 (0.013)
DeepSurv
0.841 (0.000)
0.619 (0.008)
0.797 (0.013)
0.685 (0.013)
HAZ
Cox-CC
0.836 (0.046)
0.618 (0.009)
0.797(0.013)
0.684 (0.012)
Cox-Time
0.853 (0.049)
0.637 (0.009)
0.800 (0.012)
0.687 (0.012)
DRAFT
0.861 (0.005)
0.599 (0.018)
0.725 (0.057)
0.611 (0.016)
AFT
DATE
0.852 (0.001)
0.608 (0.008)
0.784 (0.009)
0.598 (0.034)
DART (ours)
0.867 (0.001)
0.624 (0.009)
0.797 (0.014)
0.687 (0.014)
tumor bank and German Breast Cancer Study Group (GBSG). In addition, WSDM KKBox from preparation for the 11th ACM International Conference on Web Search and Data Mining is the dataset for customer churn prediction containing millions of instances and 15 covariate variables. With this large-scale dataset, consistency of training procedure and predictive performance would clearly be verified.
5.2 BASELINE MODELS
We select six neural network-based time-to-event regression models as our experimental baselines: DRAFT and DATE (Chapfuwa et al., 2018) as AFT-based models for direct comparison with our model, and DeepSurv (Katzman al., 2018), Cox-CC and Cox-Time (Kvamme et al., 2019) as hazard-based models, DeepHit (Lee et al. 2018) as a PMF-based model for references.
For AFT-based models, DRAFT utilizes neural networks to fit log-normal parametric AFT model in non-linear manner. That is, it might be misspecified if true error variable does not follow log- normal distribution. contrast, DATE exploits generative-adversari networks (GANs) to learn conditional time-to-event distribution and censoring distribution using observed dataset.
In case of hazard-based models, DeepSurv fits Cox regression model whose output is estimated from neural networks. The model outperforms the standard CoxPH model in performance, not clearly exceeding other neural network-based models. Furthermore, the proportional hazards assumption still remains unsolved with DeepSurv. Cox-CC is another neural network-based based Cox regression model, using case-control sampling for efficient estimation. While both DeepSurv and Cox-CC are bounded to proportionality of baseline hazards, Cox-Time relieves this restriction using event-time variable to estimate conditional hazard function.
In addition, we include DeepHit (Lee al., 2018) as reference, which is a survival regression model parameterizing discrete-time hazard rate with neural networks based on survival probabil- ity mass function (PMF), considering its contribution alleviate the fundamental assumption of hazard-based and AFT-based models. Although its prediction performance has been reported promi- nent to others, the training procedure quite unstable which a critical shortcoming for practical application.
Except for neural network-based models, we exclude other machine learning-based models from baselines regarding comparison from previous studies. Some neural network- -ased models are ex- cluded as well in this study since we focus on alleviating fundamental assumptions such as pro- portionality and parametric distribution. Note that comparing hazard-based models and AFT-based models has rarely been studied due to their difference in concepts: modeling hazard function and modeling time-to-event variable. Despite models be evaluated with common metrics, analysis upon numerical experiments has to be cautious, especially between a hazer model and an AFT-based model.
7

---

Under review as a conference paper at ICLR 2022
Table 3: Mean and standard deviation of Integrated Brier Score (IBS).
MODEL
WSDM KKBox
SUPPORT
FLCHAIN
GBSG
PMF
Deepllit
0.124 (0.001)
0.221 (0.034)
0.160 (0.081)
0.183 (0.015)
DeepSurv
0.111 (0.000)
0.190 (0.004)
0.101 (0.006)
0.174 (0.004)
HAZ
Cox-CC
0.115 (0.012)
0.191 (0.003)
0.122 (0.028)
0.177 (0.004)
Cox-Time
0.107 (0.009)
0.194 (0.006)
0.114 (0.016)
0.174 (0.005)
DRAFT
0.147 (0.002)
0.314 (0.043)
0. 0.144 (0.022)
0.310 (0.010)
AFT
DATE
0.131 (0.002)
0.227 (0.004)
0. 1.124 (0.012)
0.204 (0.004)
DART (ours)
0.108 (0.001)
0.176 (0.005)
0.068 (0.007)
0.150 (0.023)
5.3 MODEL SPECIFICATION AND OPTIMIZATION PROCEDURE
For fair comparison, apply neural network architecture used in Kvamme et al. (2019): MLP with dropout and batch-normalization. Every dense blocks are set to have the equal number of nodes, output bias is utilized for output layer, and ReLU function is chosen for non non-linear tivation for layers. Preprocessing procedure has also been set based on Kvamme et al. (2019) including standardization of numerical features, entity embeddings (Guo & Berkhahn 2016) for multi-categorical features. The dimension of entity embeddings is set to half size of the number of categories. In addition, due the fact that parameters of AFT-based models tend to be influenced by scale and location of the target variable, has been standardized and its mean and variance are separately stored to rescaled outputs. For SGD algorithm, Adam WR (Loshchilov & Hutter 2017) is used as implemented by Kvamme et al. (2019) with one epoch of an imitial cycle length. We also set the cycle length to double after each cycle. The details about data split and hyperparameter search are described in the appendix.
5.4 PERFORMANCE EVALCATION
To measure discriminative performance of outputs, we exploit standard C-index (Harrell et al. 1982) for AFT-based models while letting hazard-based models to utilize Cdd since equivalent evaluation is possible for AFT-based models including DART since it outputs a single scalar value to evaluate implement
ranks. In terms of survival calibration, definition, due to the fact that evaluation methods of the conditional survival function and IPCW provided by Kvamme et al. (2019) are not compatible with AFT-based models. Specifically, we first fit Kaplan- Meier estimator upon standardized training set, and subsequently evaluate conditional survival estimates and IPCW utilizing estimated residuals, following the definition of baseline haz- ard function of AFT framework rather than to use time-to-event variable directly. For numerical integration, we follow settings of time grid from Kvamme et al. (2019), and standardize the grid with mean and standard deviation stored with standardization procedure of training set. By doing so, IBS can be compared upon identical timepoints for both sizzard-based models and AFT-based models.
own function to obtain IBS based its
5.5 SUMMARY OF RESULTS
Experiment results are provided in Table 2 and 3 In summary, DART is competitive in both dis- criminative and calibration performance, especially for large-scale survival datasets. Specifically, DART yields consistent results for WSDM KKBox dataset compared to other baselines, maintain- ing competitive performance terms of Crd and IBS. point out that DART is the most powerful and AFT-based time-to-even, model that can be prominent alternative when hazard-basse models might be not working.
ANALYSIS
We provide analysis on experimental results, pointing out strengths of DART model in terms of performance metrics.
8

---

Under review as a conference paper at ICLR 2022
Table 4: Comparison of the training time (seconds) per epoch over the KKBox dataset.
Deepllit
DeepSurv
Cox-CC
Cox-Time
DRAFT
DATE
DART (ours)
Time
37.36
27.81
44.86
42.60
759.04
2024.19
29.93
Characteristic of DART for large-scale dataset. As provided in Table 2 and 3 DART generally yields prominent survival calibration performance with small variance in terms of IBS. Especially for large-scale dataset (KKBox), DART shows state-of-the-art performance with the smallest vari- ance in evaluated metrics. This result comes from the characteristic of rank-based estimation strat- egy. Specifically, on the basis of asymptotic property of Eq. 77) estimated model parameters get stable and close true parameter set, when the size of dataset gets larger. Thus, once the trained model attains effective representation (W; in Eq. ( from hidden layers via stochastic optimization methods, DART is able to provide stable outputs with strong predictive power, without sophisticated manipulation upon time-to-event distribution.
Comparison with baTed models. In case of DRAFT, model does not generally perform well for both Cnd and IBS for most datasets. This is attributed to the fact that DRAFT is a simple extension of the parametric AFT model with lognnormality assumption. Thus, this approach is quite sensitive to true underlying distribution of dataset. On the other hand, DATE yields clearly improved perfor- mance against DRAFT especially for survival calibration in terms of IBS. Unlike DRAFT, DATE utilizes GAN to learn conditional error distribution without parametric assumption, allowing the model to yield more precise survival calibration. However, time-to-event distribution is trained with divided loss functions by optimizing two tuning hyperparameters in Eq. (4) This approach can be significantly affected by well-tuned hyperparameters and heavy computation is required to this end, resulting insufficient performance. Meanwhile, as illustrated in Figure IT DART has advantages of simplicity in theoretical and practical points compared to the other AFT-based models.
Comparison with hazard-based and PMF models. As previously reported by Kvamme et al. (201)), Cox-Time shows competitive performance against other hazard-based models, directly uti- lizing event-time variable to model conditional hazard function. However, we found out that Cox- Time requires precise tuning of additional hyperparameters (A and Log-durations) largely affecting predictive performance. DeepHiit, as a PMF-based model, yields relatively poor performance in our experiments for most datasets especially in terms of IBS, inconsistent with previous study. Note that, however, HeepHit was originally designed to handle the competing-riske problem, thus evalu- ation with predictive power might not be comparable. In contrast, DART shows smaller variance in evaluation metrics as the size of data increases, ensuring stable output for large-scale dataset with asymptotic property which is crucial for practical application.
Comparison of the required time for optimizing each model. To verify the compatibility for large-scale data, measure the training time of each model. We strictly bound the scope of the target process, as from data input to parameter update excluding other extra steps. Also, all models evaluated with the same number of nodes, layers, and batch size. All experiments were run on a single NVIDIA Titan XP GPU. Table 4 shows that the simplicity of DART leads to practical efficiency, while DATE is computationally expensive due to the generator-discrimiiaaoo architecture.
To summarize, we suggest that DART would be a powerful alternative to other time-to-event regres- sion models ensuring stable performance with less time consumption.
CONCLUSION
In this work, we propose simple time-to-event regression model, namely DART, utilizing semipara- metric AFT rank-regression method and deep neural networks to alleviate strict assumptions and to attain practical usefulness in terms of high and stable predictive power. Through experiments, our model was shown to be prominent in discriminative and calibration performance even with the large-scale dataset. Although we do not yet cover more complex censoring data, such as competing risks and interval censoring, our approach might be able to provide a stable baseline to handle those tasks in near future with a simple modification of our loss function.

---

Under review as a conference paper at ICLR 2022
ETHICS STATEMENT
We believe that our model can be widely applied in many fields such as biostatistics and treatment recommendation with contributions to society and human well-being. In our experiments, we only used public datasets without any conflicts of interest and sponsorship. We hope that our model's simplicity and public code bring profits to practitioners all over the place.
REPRODUCIBILITY STATEMENT
We clarify that the reported results based multiple repetitions of experiments to remove the outlier effect and our codes are available in public. To obtain reproducible and reliable results, we repeated experiments with the best configuration and various random seeds for all models over KKBox dataset and conducted five-fold cross-validation for small-size datasets. As described in the paper and appendix, the measurements of the training time under the striet control of the experimental setup with repetitions.
REFERENCES
Odd Aalen. Nonparametric inference for family of counting processes. The Annals of Statisties, pp. 701-726, 1978.
Odd O Aalen. Effects of frailty in survival analysis. Statistical Methods in Medical Research, 3(3): 227-243, 1994.
Odd O Aalen and Häkon K Gjessing. Understanding the shape of the hazard rate: A process point of view (with comments and a rejoinder by the authors). Statistical Science, 16(1):1-22, 2001.
Laura Antolini, Patrizia Boracchi, and Elia Biganzoli. time-dependent discrimination index for survival data. Statistics in Medicine, 24(24):397-33944 2005.
Anand Avati, Tony Duan, Sharon Zhou, Kenneth Jung, Nigam H Shah, and Andrew Y Ng- Countdown regression: sharp and calibrated survival predictions. In Uncertainty in Artificial Intelligence, pp. 145-155. PMLR, 2020.
Glenn W Brier. Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1):1-3, 1950.
Jonathan Buckley and Ian James. Linear regression with censored data. Biometrika, 66(3):429-436 1979.
Paidamoyo Chapfuwa, Chenyang Tao, Chunyuan Li, Courtney Page, Benjamin Goldstein, Lawrence Carin, and Ricardo Henao. Adversarial time-to-event modeling. arXiv preprint 2018.
Jie-Zhi Cheng, Dong Ni, Yi-Hong Chou, Jing Qin, Chui-Mei Tiu, Yeun-Chung Chang, Chiun-Sheng Huang. Dinggang Shen, and Chung-Ming Chen. Computer-aided diagnosis with deep learn- ing architecture: applications to breast lesions in us images and pulmonary nodules in et scans. Scientific Reports, 6(1):1-13, 2016.
Christopher Cox. The generalized f distribution: an umbrella for parametric survival analysis. Statistics in Medicine, 27(21):4301-43) 2008
David R Cox. Regression models and life-tables. Journal of the Royal Statistical Society: Series B (Methodological), 3472):187-202, 1972.
Lore Dirick, Gerda Claeskens, and Bart Baesens. Time to default in credit scoring using survival analysis: a benchmark study. Journal of the Operational Research Society, 68(6):652 665, 2017.
Alexey Dosovitskiy and Josip Djolonga. You only train once: Loss-conditional training of deep networks. In International Conference on Learning Representations, 2020. URL https://
openreview net fortt:/fouumm hd
10

---

Under review as a conference paper at ICLR 2022
Yan Gao and Yan Cui. Multi-ethnic survival analysis: Transfer learning with neural networks. In Survival Prediction- Aaaiotionn,.griimmms. Challenges and Applications, pp. 252-257. PMLR, 2021.
Erika Graf, Claudia Schmoor, Willi Sauerbrei, and Martin Schumacher. Assessment and comparison of prognostic classification schemes for survival data. Statistics in Medicine, 18(17-18):2529- 2545, 1999.
Cheng Guo and Felix Berkhahn. Entity embeddings of categorical variables. arXiv preprint wxiv:1604.06737, 2016.
Frank E Harrell, Robert M Califf, David B Pryor, Kerry L Lee, and Robert Rosati. Evaluating the yield of medical tests. The Journal of the American Medical Association (JAMA), 247(18);
2543-2546 1982.
Shi Hu, Egill Fridgeirsson, Guido van Wingen, and Max Welling. Transformer-basee deep survival analysis. In Survival Prediction-Algorithms Challenges and Applications, pp. 132-148. PMLR, 2021.
Zhezhen Jin, DY Lin, LJ Wei, and Zhiliang Ying. Rank-based inference for the accelerated failure time model. Biometrika, 90(2):341-35 2003.
Zhezhen Jin, DY Lin, and Zhiliang Ying. On least-squares regression with censored data.
Biometrika, 9371):147-161, 2006.
Fahad Kamran and Jenna Wiens. Estimating calibrated individualized survival curves with deep learning. Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 240- 248, 2021.
Edward L Kaplan and Paul Meier. Nonparametric estimation from incomplete observations. Journal of the American Statistical Association, 53(282):457-481, 1958.
Jared L Katzman, Uri Shaham, Alexander Cloninger, Jonathan Bates, Tingting Jiang, and Yuval Kluger. Deepsurv: personalized treatment recommender system using a cox proportional hazards deep neural network. BMC Medical Research Methodology, 18(1):24, 2018.
Sunkyu Kim, Keonwoo Kim, Junseok Choe, Inggeol Lee, and Jaewoo Kang. Improved survival analysis by learning shared genomic information from pan-cancer data. Bioinformatics, 36 ():t:meennn.)::8----:: 2020.
David G Kleinbaum and Mitchel Klein. Survival Analysis, volume 3. Springer, 2010.
Håvard Kvamme and Ørnulf Borgan, Continuous and discrete-time survival prediction with neural networks. arXiv preprint atXiv:1910.06770 2019.
Håvard Kvamme, Ørnul Borgan, and Ida Scheel. Time-to-event prediction with neural networks and cox regression. Journal of Machine Learning Research, 20(129):1-38 2019.
Changhee Lee, William R Zame, Jinsung Yoon, and Mihaela der Schaar. Deephit: A deep learning approach to survival analysis with competing risks. In AAAL, pp. 2314-2321, 2018.
Changhee Lee, Jinsung Yoon, and Mihaela Van Der Schaar. Dynamic-deephit: A deep learning approach for dynamic survival analysis with competing risks based on longitudinal data. IEEE Transactions Biomedical Engincering, (7(1):122-133 2019.
Elisa T Lee and John Wang. Statistical Methods for Survival Data Analysis, volume 476. John
Wiley & Sons, 2003.
Moshe Leshno, Vladimir Ya Lin, Allan Pinkus, and Shimon Schocken. Multilayer feedforward net- works with nonpolynomial activation function can approximate any function. Neural Networks, 6(6):861-867, 1993.
Kwan-Moon Leung, Robert M Elashoff, and Abdelmonem A Afffi. Censoring issues in survival analysis. Annual Review of Public Health, 18(1):83-004, 1997.
11

---

Under review as a conference paper at ICLR 2022
Jiay Li, Hongyu Lu, Chenyang Wang, Weizhi Ma, Min Zhang, Xiangyu Zhao, Wei Qi, Yiqun Liu, and Shaoping Ma. difficulty-wware framework for churm prediction and intervention in games. In Proceedings of the 27th ACM SIGKDD Conference Knowledge Discovery & Data Mining,
pp. 943-952, 2021.
Ilya Loshchilov and Frank Hutter. 1:.1v.1711.05101, 2017.
Decoupled
weight decay regularization. arXiv preprint
Xenia Miscouridou, Adler Perotte, Noémie Elhadad, and Rajesh Ranganath. Deep survival analysis: Nonparametrics and missingness. In Machine Learning for Healthcare Conference, pp. 244-256,
2018.
Chirag Nagpal, Steve Yadlowsky, Negar Rostamzadeh, and Katherine Heller. Deep cox mixtures for survival regression. Machine Learning for Healthcare Conference, 2021.
Yuhyun Park and LJ Wei. Estimating subject-specific survival functions under the accelerated failure time model. Biometrika, 90(3):117-723 2003.
Md Mahmudur Rahman, Koji Matsuo, Shinya Matsuzaki, and Sanjay Purushotham. Deeppseudo: Pseudo value based deep learning models for competing risk analysis. In Proceedings of the AAAI Conference Artificial Intelligence, volume 35, pp. 479-487, 2021.
Kan Ren, Jiarui Qin, Lei Zheng, Zhengyu Yang, Weinan Zhang, Lin Qiu, and Yong Yu. Deep recurrent survival analysis. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 4798 4805, 2019.
Anton Maximilian Schäfer and Hans Georg Zimmermann. Recurrent neural networks are universal approximators. In International Conference on Artificial Neural Networks, pp. 632-640. Springer, 2006.
Leslie N Smith. Cyclical learning rates for training neural networks. 2017 IEEE Winter conference Applications of Computer Vision (WACV), pp. 464 472. IEEE, 2017.
Akash Srivastava, Lazar Valkov, Chris Russell, Michael U Gutmann, and Charles Sutton. Veegan: Reducing mode collapse in gans using implicit variational learning. In Proceedings of the 31 International Conference on Neural Information Processing Systems, pp. 3310-3320, 2017.
Steve Su. Flexible parametric accelerated failure time model. Journal of Biopharmaceutical Statistics, pp. 1-18, 2021.
Aliasghar Tarkhan, Noah Simon. Thomas Bengtsson, Kien Nguyen, and Jian Dai. Survival predic- tion using deep learning. In Survival Prediction-Algoritmms Challenges and Applications, pp. 207-214. PMLR, 2021.
Anastasios A Tsiatis Estimating regression parameters using linear rank tests for censored data. The Annals of Statistics, pp. 354-372, 1990.
Antonio Viganò, Marlene Dorgan, Jeanette Buckingham, Eduardo Bruera, and Maria E Suarez- Almazor. Survival prediction terminal cancer patients: a systematic review of the medical literature. Palliative Medicine, 14(5)3363-37,, 2000.
Donglin Zeng and DY Lin. Efficient estimation for the accelerated failure time model. Journal of the American Statistical Association, 1377419) 2007.
Ding-X Xuan Zhou Universality of deep convolutional neural networks. Applied and Computational Harmonic Analysis, 48(2):7877794, 2020.
12

---

Under review as a conference paper at ICLR 2022
A
APPENDIX
A.1
ADDITIONAL PARAMETER ESTIMATION METHODS
As extensions, introduce two additional parameter estimation methods for DART: error-based method and likelihood-basee methods.
A.1.1 ERROR-BASED ESTIMATION
While rank-based method Ji al. 2003) mainly relies on ranks of residual, Buckley & James (1979) proposed least-squares estimator for right-censored survival dataset, known as Buckley- James estimator. With generality, we set Buckley-James type, or error-based loss function for DART:
N
1/p
Eiror (0) =
log Si - g(X;;0) po
N
where p E {1, 2} and
{ Jee udFF(u)
6, log + (1 5,)
1 1 Poles) af (x:00]
is observed time-to-event variable imputed with conditional expectation of residual.
Specifically, fe, (F)(w)/(1 Fo(e:) denotes E(e,|T C, C.), expected value of residual for censored instance conditional on given featire information X,, where Fo is cumulative hazard func- tion that estimated via nonparametric methods like Nelson-Aalen estimator. Thus, this method is equivalent to least-squares estimation where incomplete information of censored instance has been corrected with conditional probability distribution of residual and predicted values. Lemor becomes meea-absolutee-rroo (MAE) objective function for p = 1 and root-mean-squne squared-errov error (RMSE) ob- jective function for p = 2, that are standard form loss of regression problem.
Error-based estimation provides much realistic predicted values for log of time-to-event variable, while rank-based method stands a chance of underevaluating scale of target variable since rank of residuals controls the loss function rather than magnitude of eтror. One of the shortcomings of error-based method is, however, conditional expectation of residual is neither continuous nor componentwise monotone in parameter set 0 (Jin et al.) 2006), thus optimization through SGD algorithms tends to fluctuating, which is verified through series of experiments. This has to be handled via controlling appropriate weight-decay rank-based methods is relatively robust to those configuration settings.
learning rate in practical applications, while
A.1.2 LIKELIHOOD-BASER ESTIMATION
Maximun-likelihood estimation (MLE) is the core element of statistical inference. providing both strong theoretical background on estimated parameters and accurate estimator itself. ZZng & Lin (2007) suggested kernel-smoothed version of MLE for semiparametric AFT model, claiming that nonparametric negative log-likelihood function can approximately yield minimum via kernel smoothing. With this work, we designed objective function for MLE of DART:
N
N
LMLE(()
5. 0)
d,e;
N
N
1=1
N
N
Pj
Co
log
o,K
(12)
N
Tilly
1=1
N
(e) (es-es)/an
5g
log
K(8 K(s)ds
N
where K(-) is zero-mean symmetric kereel function and on is bandwidth parameter. Common choice of K--) probability density function of standard normal distribution, while optimal band- width parameter an for neural network has not been suggested yet.
13

---

Under review as a conference paper at ICLR 2022
Table 5: Hyperparameter search space for GBSG, FLCHAIN, and SUPPORT datasets.
Hyperparameter
Values
# Layers
21,
# Nodes per layer Dropout
{64, 128, 256, 512]
{0.1, 0.3, 0.4, 0.6, 0.7} {0.4. 0.2, 0.1, 0.05, 0.02, 0.01,0.0011 {64, 128, 256, 512, 1024] [00.01,.20.344,,5550000000000 {0.1, 0.25, 0.5, 2,2,5,5, 5, 10, 100) {50, 100, 200, 4003 {0.1, 0.01, 0.001, 0.03
Weight decay
Batch size
a (DeepHit)
a (DeepHit)
Num. durations (DeepHit)
A (CoxTime and CoxCC)
Table 6: Hyperparameter search space for the WSDM KKBox dataset.
Hyperparameter
Values
Layers
(4.6,8)
Nodes per layer
{128, 256, 512)
Dropout
(0.0. 0.1, 0.5}
Minimizing (0) with respect to 0 yields statistically consistent estimator for 0, meaning that it is possible to get better survival calibration compared to rank- and error-based estimation. This can be significant merit for the case that needs precise evaluation of patient-dependent sur- vival probability. In spite of those theoretical strengths of likelihood based method, optimization via batched SGD does not guarantee stable convergence due to the fact that values of 0 close to negative infinite can get sufficiently small loss. Preventing divergence needs detailed control of hyperparameters; bandwidth, learning rate, weight decay, etc.
A.2 EXPERIMENTAL SETTINGS
A.2.1 DETAILS IN HYPERPARAMETER SEARCH AND OPTIMIZATION
In this section, we describe details on experimental settings including hyperparameter search space and optimization strategy. Our codes are available in publiqu
DeepSurv, DeepHit, Cox-CC, Cox-Time, DART. The PyCos package provides the train- ing codes for these models. For WSDM KKBox dataset, we repeated experiments 30 times with best configurations provided by Kvamme et al. (019). Because train/valid/test split of KKBox dataset is fixed, we didn't 't perform redundant search procedure. For the other datasets (SUPPORT, FLCHAIN, and GBSG), we performed 5-fold cross validation as performed at Kvamme et al. 2019) because the size of datasets is relatively small. At each fold, the best configuration among 300 combinations randomly selected byperparameters which are summarized in Table As described in the paper, we used AdamWR (Loshchilov & Hutter 2017) starting with one epoch of an initial cycle and doubling the cycle length after each cycle. The batch size was set to 1024 and the learning rates were found by |mith (2017) as performed Kvamme et (2019). The early stopping was applied with patience of 10 epochs for all models equally.
selected
DRAFT, DATE. The implementation of DATE by the authors includes the code of DRAFT as well. We utilized their official codes for all datasets. The batch size for KKBox dataset was set 8192 because the experiments were not feasible with the batch size 1024 due to their training time. The best configurations of DRAFT and DATE also were founded by grid search with same hyperparameter search space. We repeated experiments 30 times with the best configuration as mentioned above. For the other datasets, as same with other models, we performed 5-fold cross-
htnps///gthhhccmm/mmmmoooooooson time
14

---

Under review as a conference paper at ICLR 2022
Table 7: Best configurations for WSDM KKBox dataset. and applied to the DeepHit.
MODEL
# Layers
# Nodes
Dropout
Deepllit
$12
0.1
0.001
0.5
DuepSurv
256
Cox-CC
128
0.0
Cox-Time
8
256
0.0
DRAFT
128
0.0
DATE
512
0.5
DART (ours)
256
validation and chose the best configuration among 300 random hyperparameter sets at each fold. The early stopping was applied as done in other models equally.
A.2.2 COMPARISON OF THE TRAINING TIME
For fair comparison, we measured the training time for only the optimizing phase because there various extra steps for each model. We strictly set the range of the target process, from putting data to updating model parameters. The specifications
models were set equally: the number of nodes 256, the number of layers 6, and the batch size 1024. With the consumed time of 1000 iterations, we calculated the training time for a single epoch. We excluded the first iteration that an outlier in general. To obtain reliable results, we repeated five times and reported the average values.
A.2.3 FUTURE WORKS
Even though DART alleviates the fundamental and strict assumptions, there are unresolved assump- tions yet. We assume the non-informative right-censoring most time-to-event models did basic assumptions. Also, AFT models inherently consider features as accelerating or delaying factors of the event time. Addressing these basic assumptions the direction we should go order build more generalized model.
As mentioned in the paper, some of the researches resolved issues of time-to-event modeling such as competing risks, discrete time, and time- varyinarying covariate effect (Katzman et al., 2018 Lee et al., 2018 Ren et 2019 Kvamme & Borgan 2019 Avati et al. 2020] Tarkhan al., [222] Rahman Even though we didn't it focus on these issues at this work, but one future integrate these techniques with DART because such assumptions are important
et al. 2021 Su 2021..
direction can
as well and DART' is compatible with others.
We are also aware of series of researches that use various neural networks such as recurrent neural networks (RNNs), convolutional neural networks (CNNs), variational auto-encoder (VAE), or trans- formers (Lee et al. 2019 Nagpal et al. [202T] Gao Cuil 2021 Kim al. 20200) Kamran Wiens, 2021 Hu et al. 2021). 1). All of these studies complementary to our model rather than competing because we can easily replace our go with RNNs, CNNs, VAE, and transformers.
15


Reviews:
[{"review_id": "XbGtzk1C6uA", "reviewer": "Reviewer_qkV5", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is easy to follow with competitive quantitative results. However, the technical contributions are lacking, there are some misleading statements, and the writing needs improvement. Below are specific examples.\n\n-  Introduction: \"The accelerated failure time model (AFT) or accelerated life model relates the logarithm of the failure time linearly to the features.\" This statement is not necessarily true for parametric AFT other than log-Normal, *e.g.*,  Weibull, Exponential, *, etc.*\n-  The $L_{\\rm CoxPH}$  (Eq. 2) definition is not correctly specified. Note the risk set $\\mathcal{R_i}$ should include censored on non-censored times $T_j$ s.t. $T_j < T_i$\n- Sec 3.2: \"However, estimating survival quantities (e.g. conditional hazard function) cannot be directly done for AFT-based models.\" This statement is not necessarily true for some parametric AFT models, *e.g.*, Weibull, Exponential, *, etc.*\n- Sec 3.2: While paper claims to make time-to-event  predictions $\\hat{T} = \\exp(g(X_i, \\theta))$ it is not clear why those predictions can not be used to directly estimate $\\hat{S}(t| X)$. Instead, the paper proposes a semiparametric conditional hazard transformation similar to CoxPH without providing any justification.\n- Sec 4: Is there evidence that supports this statement  \"Note that standard concordance index yields identical results with ${\\rm C}^{\\rm td}$ for AFT-based models \"? \n- Sec 5.4:  For fair comparisons, the paper should compare AFT and hazard models using a similar metric, either C-Index or ${\\rm C}^{td}$ or both.\n- For a comprehensive evaluation, the paper should provide additional qualitative results, *e.g.*, model predictions against ground truth, and calibration curves.\n\n**Minor Issues**\n-  Missing $\\xi \\sim F_{\\xi}$ in third term (Eq. 4)\n- Introduction: Typo ATF should be AFT\n- Sec 4: \"well-fitted model yields IBS lower than\"  sentence is incomplete\n", "summary_of_the_paper": "The paper extends the previously proposed linear semiparametric AFT model based on Gehan’s rank statistic to a nonlinear setup. The nonlinear model termed Deep AFT Rank-regression for Time-to-event prediction model (DART) is parameterized by a neural network. Experimental results across four datasets show a competitive advantage over baselines per concordance index (C-Index), integrated Brier score (IBS), and training time.", "summary_of_the_review": "Apart from empirical competitive quantitative results against baselines, the technical contributions are not clear. The semiparametric AFT model based on Gehan’s rank statistic objective function has been previously proposed. Additionally, while the paper claims to make time-to-event predictions, the conditional hazard transformation of model predictions seems to indicate the model predicts hazards instead.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "HQ0eU9uYqir", "reviewer": "Reviewer_7ymN", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I found this paper straightforward to read although I have several large concerns.\n\nMy first concern is that the exposition is overly complicated even though the key idea of the paper is extremely simple. The proposed model DART can be derived by just taking the known rank estimator by Jin et al (2003) and swapping out the inner product in their equation (2.1) with a neural net, and then just train using minibatch gradient descent instead of linear programming. In other words, much like how Faraggi and Simon (1995) swapped out the inner product in the semiparametric Cox model to be a neural net to come up with the same neural net model as DeepSurv (Katzman et al 2018), this paper does the same thing with the already existing semiparametric AFT model (which already was known before the Jin et al (2003) paper but the optimization procedures in estimating the regression coefficients were not optimal). From a technical novelty/innovation perspective, there is very little that is conceptually new. Instead, the paper is motivated in a way that spends way too much emphasis/text on describing standard results from survival analysis (e.g., what is a Cox model, what is an AFT model, why one would use standard Cox PH vs AFT, that the Cox model can be made time-dependent, that AFT models can be specified in both parametric or semiparametric forms, etc). I'd suggest perhaps having your background instead focus on existing semiparametric AFT literature, that you're simply doing a straightforward neural net extension (replace inner product with neural net), and comparing your model with DRAFT and DATE (I found Figure 1 very helpful). I'd suggest reducing the amount of text spent on explaining what hazards models are or why one should use hazards models over AFT models as this is a very old debate at this point (from what I can tell it really just depends on the data). Of course, a Weibull regression model is both a hazards model and an AFT model.\n\nMy next concern is that the experimental results are inconsistent with existing literature. As is, the experimental results presented make it seem that DART is not a clear winner over top-performing baselines. However, once we consider that some of the numbers are quite off from literature, how good DART is seems even more suspect. For example, your reported DeepHit $C^{\\text{td}}$ number for the KKBox dataset is dramatically off from Kvamme et al (2019)---basically according to what they get, DeepHit gets 0.888 which is higher than DART 0.867. Some of your IBS numbers across methods also seem off. Some explanation of these discrepancies would be helpful.\n\nThere is another exposition issue: early on in the paper, the text makes it seem like the Cox model does not make a linear assumption whereas AFT does. This isn't true. Cox assumes the partial log likelihood is linear. AFT assumes the log survival time is linear. In other words, both make linearity assumptions, just for different quantities.\n\nReferences:\n- David Faraggi and Richard Simon. A neural network model for survival data. Statistics in Medicine 1995.\n- Zhezhen Jin, D. Y. Lin, L. J. Wei, Zhiliang Ying. Rank-based inference for the accelerated failure time model. Biometrika 2003.\n- Jared L. Katzman, Uri Shaham, Alexander Cloninger, Jonathan Bates, Tingting Jiang, and Yuval Kluger. DeepSurv: personalized treatment recommender system using a Cox proportional hazards deep neural network. BMC Medical Research Methodology 2018.\n- Håvard Kvamme, Ørnulf Borgan, Ida Scheel. Time-to-Event Prediction with Neural Networks and Cox Regression. JMLR 2019.", "summary_of_the_paper": "This paper proposes a (deep) neural net extension to an existing ranking-based estimator used for the semiparametric AFT model (Jin et al 2003). The resulting method achieves time-dependent concordance index and integrated Brier score values that are competitive compared to baseline deep AFT models.", "summary_of_the_review": "This paper has extremely limited technical novelty, spends too much text on explaining existing standard results/definitions from survival analysis, and has experimental results that are suspect (inconsistent with existing literature).", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "5FO0cy4Ywh", "reviewer": "Reviewer_Rqgw", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n - Use of a (for the machine learning literature) novel loss function, based on Gehan’s rank statistic\n - The paper is mostly clearly written and easy to follow\n - The authors put their method into context well\n\n\nWeaknesses:\n - Due to semi-parametric nature, prediction only possible up until the last observed patient time.\n - The loss function in Jin et al (2003) is derived for the linear case. While the authors argue for how it could work in the non-linear case, there is no guarantee that the optimal set of parameters corresponds to a zero rank statistic in the non-linear case\n - The loss function is quadratic in the training samples, which could lead to bad performance for large data-sets. The authors however show very fast performance on a large data-set, which makes this a minor point.\n - The authors introduce both a neural network architecture for AFT models and an L1 ranking loss. It is not entirely clear how much influence either of these have on the benchmark results. In the appendix the authors introduce other loss functions, but do not include the trained models based on those losses in the benchmark results. The addition of these in table 2 and 3 would help disentangle the influence of the model architecture and of the loss, which would be interesting.\n\nOther:\n - The loss function is quadratic in the number of patients/intervals, whereas the full likelihood is linear. Given this, the speed on KKBOX seems very surprising and is probably due to a small constant. It would be interesting to see the scaling with the number of patients N with real data with both the ranking and the maximum likelihood loss.\n - The authors could make the connection between $\\epsilon$ in equation (3) and $h_0$ in (9) more clear as a step before using the Nelson-Aalen estimator to use a semi-parametric AFT.\n - The authors use “calibrated” when talking about Brier score. However traditionally when talking, especially about AFT models, calibration refers to the difference between real and predicted event times and the Brier score is a mix of calibration and classification. The Brier score is the calibration of the survival distribution over time, so calling it calibration is not incorrect, however within the context of the survival literature somewhat misleading. More specific language would make the paper more readable.\n - An introduction or motivation why the loss function is a good loss function would be beneficial for the understanding of the paper. The authors could elaborate on rank statistics and why they are expected to be more stable. Especially, as e.g. DeepHit talks about ranking when talking about concordance, it would be helpful for reading that ranking here is in reference to a weighted log-rank estimating function.\n - It would be interesting to see how well the model does in calibration of the event times (also with the other loss functions), as this is clinically meaningful.\n - The bold facing in table 2 is not consistent (DeepHit should be bold in one case)\n - In the appendix the authors make it seem that there are no convergence guarantees for MLE based estimators. However, also considering the application to AFT models as a special case, there do exist estimators that have some theoretical convergence guarantees (Tang et al 2020, Survival Analysis via Ordinary Differential Equations). A small discussion thereof could be beneficial.", "summary_of_the_paper": "The authors suggest a neural network based accelerated failure time (AFT) model, as well as an L1-type rank loss, which they argue results in an easy and fast to train model, which is state of the art in terms of two standard evaluation metrics, concordance and integrated Brier score. Following the semi-parametric AFT literature, the baseline hazard is estimated from the data and the full hazard function is parametrized in a way that the parametric part corresponds to a prediction of the failure time. The main innovation of the paper is the use of a novel loss function based on a ranking loss using Gehan’s rank statistic, which while being used in the statistical literature, has not been used in a machine learning setting. ", "summary_of_the_review": "While there is a large number of machine learning methods for survival, by considering a (for the machine learning literature) novel loss function in the AFT setting, this paper can be seen as a meaningful contribution to the machine learning for survival literature. The theoretical foundation of the loss function is however not perfect and the influence of loss function and architecture are hard to disentangle. Due to the form of the loss function, a formal scaling analysis would also be needed. I would recommend a weak accept. I could be convinced to improve of the score with a more convincing argument why the loss term in the non-linear setting is a good loss term, maybe through some upper bound on the loss term.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "12W7IIJlpaC", "reviewer": "Reviewer_PtQZ", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This is a somewhat confusing paper for several reasons: i) the problem is set as AFT, however, the likelihood (encoded in the residuals), which is key to AFT is not used for optimization, so one could argue that the proposed approach is not an AFT formulation but rather rank regression with censoring; ii) the rank-based loss function in (6) that is very similar to that in Raykar et al. (NeurIPS 2007: On Ranking in Survival Analysis: Bounds on the Concordance Index), which is used in DRAFT but not discussed in the paper; iii) the experiments show that the proposed model is no better than the standard, linear, Cox proportional hazard in terms of C-index which is the most appropriate performance metric considering the proposed model is optimizing for ordering using a rank statistic; iv) the theoretical results for (7) in the linear case, as the authors discuss are not directly generalizable to nonlinear models.\n\nThe proposed approach seems to outperform Cox in large datasets. The authors may consider presenting results on additional large datasets to confirm that it is indeed the case, which in turn will make the experiments section stronger.\n\nNote that DRAFT in Figure 1 is not introduced or discussed in the paper. ", "summary_of_the_paper": "The authors introduce a deep-learning-based extension to accelerated failure time modeling for survival analysis and introduce a rank-regression-type loss function based on Gehan's rank statistic.", "summary_of_the_review": "The proposed approach presented as a deep-learning extension of AFT optimizes a rank-based loss function very similar to that in Raykar et al. (NeurIPS 2007: On Ranking in Survival Analysis: Bounds on the Concordance Index). The experiments results show that the proposed approach does not outperform standard Cox in 3 out of 4 datasets.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "-xuYZ6bJMU", "reviewer": "Reviewer_6g2w", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength:\n\n1. The idea of combining Gehan’s model with non-linear deep learning model is new\n2. Performance is stable on several benchmarks.\n3. Clear writing\n\nWeaknesses:\n1. The intuition of the approach is not quite clear. Does the proposed model focus on reducing computational time or improving the performance, or both?\n2. The results do not support the conclusion. The improvements seem not significant compared to baselines.\n3. The methodology parts mostly combine another existing model idea into the current AFT model. Hence, the innovation is limited methodological-wise.", "summary_of_the_paper": "This paper proposes to combine the idea of Gehan’s rank statistic idea on fitting the AFT model, as well as using the deep learning model as a non-linear method for replacing the linear method in the original AFT model. The authors argue they connect Gehan's non-parametric technique with deep learning models. Experiments on various benchmark datasets show that the proposed model is competitive to state-of-the-art baselines.", "summary_of_the_review": "While the idea of combining Gehan model with deep learning is new, the results show that the proposed model does not outperform SOTA systems on several benchmarks.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: Most reviewers came to the conclusion, that this work lacks novelty and theoretical depth. Further severe concerns about the validity of some statements and about the experimental setup have been raised. The rebuttal was not perceived as being fully convincing, and nobody wanted to champion this paper. 
I share most of these points of criticism. Although there is certainly some potential in this work, I think it is not ready for publication and would (at least) need a major revision.
Decision: Reject