Deep Extended Hazard Models for Survival Analysis
Authors: Qixian Zhong
Jonas Mueller
Jane-Ling Wang
Venue: NeurIPS 2021 Poster (2021)
Keywords: brier score
convergence rate
curse of dimensionality

Abstract: Unlike standard prediction tasks, survival analysis requires modeling right censored data, which must be treated with care. While deep neural networks excel in traditional supervised learning, it remains unclear how to best utilize these models in survival analysis. A key question asks which data-generating assumptions of traditional survival models should be retained and which should be made more flexible via the function-approximating capabilities of neural networks. Rather than estimating the survival function targeted by most existing methods, we introduce a Deep Extended Hazard (DeepEH) model to provide a flexible and general framework for deep survival analysis. The extended hazard model includes the conventional Cox proportional hazards  and accelerated failure time models as special cases, so DeepEH subsumes the popular Deep Cox proportional hazard (DeepSurv) and Deep Accelerated Failure Time (DeepAFT) models. We additionally provide theoretical support for the proposed DeepEH model by establishing consistency and convergence rate of the survival function estimator, which underscore the attractive feature that deep learning is able to detect low-dimensional structure of data in high-dimensional space. Numerical experiments also provide evidence that the proposed methods outperform existing statistical and deep learning approaches to survival analysis.

Deep Extended Hazard Models for Survival Analysis
Qixian Zhong
Department of Statistics and Data Science
School of Economics and Wang Yanan Institute for Studies in Economics (WISE)
Xiamen University
qxzhong@xmu, edu.
Jonas Mueller
Amazon Web Services
j jonasuueamazonncc com
Jane-Ling Wang Department of Statistics UC Davis janelvang@ucdavis.coo edu
Abstract
Unlike standard prediction tasks, survival analysis requires modeling right cen- sored data, which must be treated with care. While deep neural networks excel in traditional supervised learning, remains unclear how best utilize these models in survival analysis. A key question asks which data-generating assumptions of traditional survival models should retained and which should made more flexible via the dunction-approximatiin than estimating the survival function targeted by most existing methods, we intro- duce Deep Extended Hazard (DeepEH) model to provide flexible and general framework for deep survival analysis. The extended hazard model includes the con- ventional Cox proportional hazards and accelerated failure time models special cases, so DeepEH subsumes the popular Deep Cox proportional hazard (DeepSurv) and Deep Accelerated Failure Time (DeepAFT) models. We additionally provide theoretical support for the proposed DeepEH model by establishing consistency and convergence rate of the survival function estimator, which underscore the attractive feature that deep learning able to detect low-dimensional structure data in nigh-dimensional space. Numerical experiments also provide evidence that the proposed methods outperform existing statistical and deep learning approaches survival analysis.
capabilities of neural networks. Rather
Introduction
Across areas such as biomedical science and reliability engineering, survival data analysis is critically used study the time until certain events occur (e.g. patient death clinical applications, component failure in industrial applications). A key issue stems from the fact that many occurrence-time, fail be recorded due natural data censoring. For example, patient may relocate and drop out of longitudinal clinical study, leading to the loss of follow-up observations for the remaining period of the study. For such right censored data, we only know that the true event time greater than the last available observation time for this individual (but not by how much), and thus standard regression approaches are not applicable to most survival data. naive approach that removes all censored individuals from the data would induce bias, as these tend to be individuals with longer event times.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Learning the distribution of the event time in the presence of censorship is fundamental task in survival analysis. A wide array of statistical methodologies have been developed to study this distribution. Among them, Kaplan and Meier [37 considered a product of successive conditional probabilitiiss to estimate the survival function, which encodes tail probabilities of the event time distribution. As nonparametric approach, the Kaplan-Meier estimator assumes no particular form for either the underlying distribution of event time or the relationship between individuals' observed feature-valus and the outcome event time. Alternatively, the Nelson-Aalen estimator considers the cumulative hazard function from a counting process perspective 48). The classical Nelson-Aales estimator was improved by leveraging more powerful supervised learning methods, in particular random forests, which lead to the random survival forest model 33
While the Kaplan- Meier estimator has been extended to assess the effect of multiple subject fea- tures (covariates) on the associated event time (5), it struggles to work with even moderately high- dimensional features. A cemi-parametric approach, the Cox proportional hazards (CoxPH) model 14. has thus been popularly adopted as a dimension reduction approach. However, the CoxPH model is restrictive as it assumes that the conditional log hazard functions for individuals are parallel to each other. When this assumption fails to be realistic, the accelerated failure time (AFT) model often emerges
alternative, which simply assumes a linear regression model for the log event time against the individuals' feature-values. Like the CoxPH model, the AFT model is also semi parammeretttrcc because the error distribution in the regression model is unspecified. An even simpler approach is to adopt a parametric approach for survival data by assuming that the distribution of event time obeys a certain known law, such as the Weibull or Gamma distribution [4]. While con-parametric methods are more flexible, a semi-parametrie or parametric approach can be more effective when there limited data, if the distribution of event times can be closely approximated by a certain cemi-parametric or parametric form.
35 [
Rapid advances in data collection and analysis have encouraged researchers to explore the potential of deep learning in the field of survival analysis. Faraggi and Simon [00 proposed to replace the linear predictor of the CoxPH model with single hidden-layer neural network, but this extension failed to provide reliable improvements in terms of concordance index 663] Under the same CoxPH framework, Katzman et al. (38) and K Kvamme& et al. [41] applied neural networks of greater complexity to obtain more flexible nonlinear models. Zhu et al. (67] and Tarkhan et al. [57] used convolutional architectures to explore the relationship between event time and unstructured features like images, and domain-specific variants of these models have been developed for particular applications, such as genomic data [10 26), clinical research 449 pedestrian waiting time in urban areas 34 and heart failure rehospitalization [36].
Beyond the CoxPH framework, deep learning has been used to model survival data in various alternative forms, including: neural network versions of the Beta-Logistic model [32], recurrent neural networks applied with -iscretitttiiinonn [22 51), generative adversarial modeling of time- to-event distributions (8), generalized forms of regression [40] 43), and more flexible extensions of
the Kaplan- Meiir estimator [ 56] and the Weibull distribution [50] Among existing deep learning
methods for survival analysis, the distribution free approaches
9
12 28
43
make almost no
assumptions the underlying survival function structure, but these very general models may exhibit poor sample complexity, requiring very large data to accurately estimate the survival function.
In this paper, we adapt neural networks to generalize the extended hazard (EH) model [. which includes both the CoxPH and AFT models as special cases. The proportional hazards assumption are often violated in practice and can lead to misinterpretation of the results, especially when there is heterogeneity among individuals (2). In contrast, the AFT framework directly considers the relationship between feature values and the event time, which results in time-dependen hazard ratio that is more realistic. Keiding et al. (39] also show that, relative to CoxPH. AFT models are more stable when accounting for unobserved features. Other studies of CoxPH and AFT models have however reported conflicting results [46), and thus practitioners may find it difficult to choose which model to use in their applications. The EH model 119 58 59], under which feature values can affect
both the baseline hazard rate and hazard ratio, flexibly combines the CoxPH and AFT models into a more general framework. EH models also subsume a wider range of survival models such as Weibull and Gamma distributions. Compared with the current practice of deep learning under the CoxPH or AFT framework, our deep extensions of EH model retain the advantages of standard EH model over CoxPH and AFT, and achieve better empirical performance on real survival data.
2

---

While deep learning has enjoyed a number of breakthroughs in survival data analysis. mathematical understanding of its success still lagging far behind. Leveraging neural networks' capacity for this paper provides theoretical support for our Deep Extended Hazard (DeepEH) model, which replaces the two linear risk predictors in the standard EH model with two nonparametric functions (cf. h and h2 in 4 Using neural networks to model these nonparametric functions, we show that the resulting estimators of h1, f2 and survival functions are asymptotically consistent and enjoy fast convergence rates. Specially, the convergence rates are determined by the intrinsic dimension of the underlying functions rather than the original high-dimension input features. Our results formally illustrate how neural networks are able to identify low-dimensional structure of the data, hence circumventing the curse dimensionality.
nonparametric function approximation 44 17 29
53
65
2 Background
In survival analysis, we are interested in modeling the time T until some event of interest (i.e. survival time). whose probability distribution typically characterized in one of four equivalent ways: probability density function, survival function. hazard function, or cumulative hazard function. The survival function S(t) the probability of individual surviving beyond time
-(t)= > 1) 1) J. f(sdds.
where f is the probability density function of survival times. The hazard function, denoted by A(t), characterizes the chance that an event occurs in an infinitesimal interval after time t, given it has not yet occurred time t:
P( < T < t <
X(t) lim 6-+0
This results in the relationship: S(t) ext)-A(t)}, where A(t) 5 A(s)ds is the cumulative
huzard function.
In practice, a survival study of n subjects will contain less than n completely observed event times due to right censoring. For instance, the event of some individuals may not have occurred yet the time when the study is concluded or the subject may have dropped out of the study. For the i-th subject in the study, T; and C, denote, respectively, the event time and the potential censoring time; and X; E R2 denotes the observed features (covariates). Thus, the observed data from a typical survival study contain independent observations D {X,, O₁, A, )-)... where the observed time O, = min(T: C2) and the event indicator A, = 1, if the observed time is Tú, i.e. T; S C, otherwise A, = O, the subject censored at C. Given the feature values X,, the event time T, and censoring time Cj are typically assumed to be independent. We aim to estimate the survival function and hazard function using censored data in which the true event times may be unavailable. Conditional on the features X, the overall log- eegllikelihood of the censored survival data D [5] is:
Ln Σ A. log f(O,IX,) + (1 - A,) log S(O,)x.)) 1 A(O,Ii)) -
A(O,Ix,))
(1) where S(4X), X(IX)) and A(t]X) denote the conditional density, survival, hazard, and cumulative hazard function of TIX, respectively, and the last equality is derived from the aforemen- tioned relationship between the (cumulative) hazard function and the density/survival function. To obtain maximum likelihood estimates, one can assume that the conditional distribution of T follows particular parametric distribution. However to avoid restrictive assumptions, it is advantagcous to use more flexible alternatives such as the CoxPH and AFT models.
2.1
Cox Proportional Hazard Model
Given features X, the Cox proportional hazard (CoxPH) model 13 for event time T, is a semi- parametric model whose hazard function has multiplicative form in terms of baseline hazard function (t) and an exponential risk function of X:
X(t|X) Xo( exp() X),
3

---

where the baseline hazard function (t) and parameter 3 are both unknown. The function Au represents the baseline hazard rate when all features take the baseline value of zero. It is difficult to estimate λn and 3 directly from the likelihood function (T) because the optimization with respect to Ao nonparametrie function, would be challenging. In fact, this likelihood is unbounded due to the unconstrained nature of the nonparametric function Ao- Luckily the partial likelihood method [4] has been developed to handle the parametric component of the model without involving the nonparametric component (t). Specifically, the partial likelihood technique first estimates & by maximizing the log-partial likelihood:
(3)
1 T2
A.B X, log( Σ Y,(O,)e9"XX)
where ( 2t. Intriguingly, the log-partial likelihood does not involve the nonparametric baseline hazard function Xo. which is major advantage in accomplishing dimension reduction. Once the parameter| 33 has been estimated through the log partial likelihood, the cumulative baseline hazard
function (t) Ju (s(ds can be estimated through the Breslow estimator [6] Here the simple parameter fgg can estimated directly through the partial likelihood without requiring nonparametric estimation of λo- Thanks to the appeal of this straightforward estimation, the Cox proportional hazards model has dominated the field of survival analysis for nearly fifty years.
2.2 Accelerated Failure Time
The accelerated failure time (AFT) model 622 is an alternative to CoxPH when the proportional hazards (PH) assumption is violated. Given feature values X, the hazard function for an AFT model is of the form:
X(t = \o Xx)x xe
where do is unknown baseline hazard function and 3 is unknown parameter. Unlike the CoxPH, the AFT model allows the feature X to influence the time curve of the baseline hazard function as well. AFT with 3 assumes that subject with larger X value will have double jeopardy through a higher risk factor e x and a faster hazard schedule Ao x) compared to the baseline hazard (t). hence the name "accelerated failure".
2.3 Extended Hazard Model
In practice, may be unclear whether a CoxPH or AFT model is more suitable for the data in hand. The Extended Hazard (EH) model [1I] includes both the CoxPH and AFT model as special cases, thereby providing more flexible and larger class of models to use in such practical applications. While AFT 2 allows for only one feature effect B, the EH model extends this is to allow two different feature effects B1 and 3z:
X(IX) (
Note that the EH model collapses to the CoxPH model when 3 0, and the AFT model when 3 = 82. The EH model can thus flexibly leverage the complementary strengths of the CoxPH and AFT model, adaptively exploiting whichever assumptions are more appropriate for given dataset.
3 Methods
3.1 Deep Extended Hazard Model
Our DeepEH model is derived by extending the standard EH model in 33 to the following hazard function:
X(t|X) (e
(4)
where h2 (X) and hz(X) are unknown functions and λ is an unknown and nen-negative baseline
hazard function. When h 0, collapses to the popular DeepSurv [8. When dnj = h2, we call this the Deep Accelerated Failure Time (DeepAFT) model as it is deep version of the AFT model (2) that has been previously unexplored. All of our theory and methodology for DeepEH extends DeepAFT as well, and we also present empirical results for DeepAFT in this work. Let

---

Ao(t) Sor Ao(s)ds be the cumulative baseline hazard function. By the definition of conditional survival function S(t[X) - exp(- So )ds],, the survival function of DeepEH is expressed as S(IX)= = exp( /'" (e)) ds} - exp(-Ao(te (
(5)
Since the log-likelihood is unbounded, direct maximization of the likelihood function will not work. Instead, pseudo likelihood may be employed instead Tseng and Shu [59]. Specifically, we first use kernel smoothing method to approximate the hazard function with fixed b1 and l22 and then plug the the smoothed hazard function into the log- likellioddo function ① This yields the following pseudo-likelihood function for our DeepEH model:
Cn1 h22 =
1
(x,) - (2(11))
Σ
A
log
Ry(h1) - Rs RR((ha) b
nb j=1
1=1
1=1
1
1 Σ cha(X,,) (X,)
R2(h1) R((h1)
log
i=1
where (-) is a known kernel function satisfying Assumption (A4) below and we adopt the Gaussian kernel in our implementation, (h,) = log(O,, is a positive bandwidth constant, and
((t) J" K(s)ds.
We use two neural networks to approximate the unknown functions h1 (X) and (2()) in (4) More specifically, we learn:
(hr, h2) = arg min L. (2))9)) + PJ (g), (91,92)
(6)
v1-22
where G again denotes a certain family of neural networks and J is a regularization term weighted by 2 0 and it is set to be a L regularization for the weights and biases of 91 and 92 in this paper. We estimate respectively the baseline hazard and cumulative baseline hazard functions via:
1 EL (R, log and Ao(1) = J. )o(s)ds,
So(1)=
(g( R!
(.. -ogg Ko(s)ds
where K2(-) = K-//b)/b. Our implementation employs a Riemann sum of the estimated baseline hazard function do to obtain Åg which leads to estimate of the survival function based on (5)
S(t[X) exp( (x) (x) (x,0)),1(00)
3.2 Theoretical Analysis
To study the asymptotic properties of our estimators, we assume the data follow the survival model specified in (4) and that our multilayer feedforward neural networks obey special structure with parameters d, 8, M and described below. Here a M-layer neural network with parameter d = (do, dds-1) is composition of functions:
g(x) WM1gM-1 РM. (x) (WMM-19M-2(1)) pM-1), .9(()))))))))))))))))))))))))))) = oWWux + (o)
with activation function σ,, weight matrices Wx E dd xd, and bias vectors μr E Rd,, for k 0,.................... M. We consider particular family of neural networks G d) {g g is an M M-larer network with parameters d and max O.
|W V l|meeloo S 1),
where V 6 max(a, b) and |/oo denotes the supremum norm of matrix/vector. Neural networks used in practice often contain many parameters, which can lead to overfitting. To address this, Srivastava et al. [55] proposed dropout regularization, which randomly sets some of network's activations to zero during training. Inspired by dropout, we further consider another neural network family
g(d,s, M, A) = (g (g(M
Σ
We llo llexllo I s, and ligllso $ A},
(8)
5

---

where || llo is the number of non-zero entries of matrix/vector and l oo is the uniform norm of function g.
Our theory also assumes the underlying nonparametric estimands belong to the Hölder class of smooth functions with parameters o, B > 0 and domain TD C IRP defined as:
109(()) 0 09
HID B) (h: D - R:
Σ K:
110
Σ []1A| E.y
sup
B},
|| 111 1110-aa)
where |a]] is the largest integer strictly smaller than a, 0 := ....9.. if 3 (3),- 3p), and [3] = Ek=1 By. Many common functions satisfy the Hölder property, including: polynomials, trigonometric functions, exponential and logarithmic functions, as well as Lipschitz continuous functions when a 1. For some L 2 0 and p .......PPL+))). N we also consider composite Hölder function:
h h1 h where hu (ar, dr) + [a1+1, | 1
h1 ho 0, L. Denote h1 (hn,'.. Mppe D and let pij
be the unique number of features that each hig depends on. Define pf pm V Pipee1 For illustration, if
h(T1, h(I1,...,29 vg)
hu (hoi (x1, 22, 23), ho2 zs, 2o), hoa (27, 2g, z9));
have p pi- 3. Further, we assume each hij E Hop bi] B) and define the underlying function space
H(L.P.P',,, {h =h1 hL-1 h1 ho h4 (hu,, hipp+ )7, h Hpy ([ar, ir B), for some |aı| [bx| s B}:
(10)
where p- = (Pn, ... PI,+)), p² = (pū pi) and a = (ao. .r.)) For this function space, we denote of a: [T (a; 1), arg min. foj (2a} + p ), and
(11)
Here A b min b). Without loss of generality, we assume that each feature has been rescaled to the unit interval, X E [0, 1j.. Our theoretical results stated below also depend on the following formal assumptions (proofs are relegated to the Appendix):
(A1) The underlying functions hn and h2 in (4) belong to the Hölder class
Ho (h H(L,P. a, B): B[AA(X)) 0}.
(A2) The observation times O, lie in finite interval [0, T). and the underlying baseline hazard An in (4 is positive with twice continuous derivative in (),
(A3) The sub-density p(t) dP S 1, A 1)/di is twice differentiable in () and there exists a positive constant с such that P(T 2 x(X) 2 with probability one.
(A4) The kernel function K(-), defined on [-1, 1], is symmetric and twice-comtinuoush) differentiable
in (-,
(A5) M = Ologn)), s Onr, log n), d1 A... A du = O and d2 V... V dy = (n)).
Assumption (A1) is required for the identifiability of lt and l22 Assumptions (A2) and (A3) are common smoothness assumptions for right censored data (58), where the study ends at time т. The second part of (A3) is typically satisfied in practice when the study must end while some subjects are still alive. Assumption (A4) ensures that the likelihood function can well approximated by the pseudo-likelihood likelihood function. Assumption (A5) configures the structure of neural networks in ( to ensure larger networks are used as sample size increases.
Theorem Suppose assumptions (AI)-A5) hold and ps - 0 as n + oo and b = On1-1/5). for any e > 0 there exists sufficiently large A > 0 and an estimator h = (/r,h2) e G s, M, A) x ((d, s, M, A) that optimizes (6) such that
1/5). Then
lim P(d h2 h) 2) =0
n +0000

---

l11 h1 h1]1==((0.1))) ll2 h h2lle=((0.1pp)) Moreover, we have where h = (h1, and d(h, h))
lim P(II((IIX) -(-1 2 -.
n R-000
The estimator f. = f-= h2)) based on the n observations {X,, O,, A,) in Section and we omit its subscript for simplicity throughout the paper. From the proof of Theorem observe that both the approximation error and estimation h. The approximation error is the smallest distance between h and the neural networks space G(d, s, M, A) x G(d, s, M,, A), for which more complex ((d, s, M, A) yields smaller approximation errors. On the other hand, the estimation error the difference between the minimizer h and the optimal approximation of h in ((d, 8, M, A). With increasing sample sizes n, both the approximation error and estimation error become smaller. The next theorem shows the exact convergence rates of the proposed estimators.
contributes
the error h
Theorem
Lel Ot, = |ojj +1. 1. Under assumptions (A1)-(A5), the baseline harzard function Ao is at continuously differentiable, b
and ps (r then we have
d(h, h)) = Op(rn logrroog"m n) and [S((1)) 5(11)) Op(rn log"nlog" n).
Theorem shows that the resulting convergence rates are jointly determined by the smoothness and intrinsic dimension of the true functions, which are similar to Schmidt-Hieber (53] for nonparametric
regression and Zhong et al. [66] for the partially linear Cox model. For example, if the functions hi and ht2 have the form in 9 with twice differentiable functions has, the convergence rate of the proposed method is of the order n 2/7 log n, but classical nonparametric smoothing methods, such as kernel smoothing and B- B-splines, would yield slower convergence rate of order -2/13, This illustrates how deep learning is able alleviate the curse of dimensionality and identify the low dimensional structure the data embedded higher dimensions.
Experiments
This section presents the numerical performance of DeepEH in comparison with ten other survival methods on four real datasets. The Appendix contains additional empirical studies and details regarding implementation.
4.1 Evaluation Criteria
Due to the presence of censoring in survival data, traditional performance measures such as mean squared error cannot be used to evaluate the accuracy of predictions. Instead, the concordance-indes (C-index, (27) one of the most widely used performance measures for survival models. It assesses how good model is by measuring the concordance between the rankings of the predicted event times and the true event times. Specifically, if the predicted event time of the i-th individual is Σç, the C-index defined by = P(, < T,JO, < O,, A, = 1). However, it is difficult to obtain the predicted event time in most survival models, so the following C-index proposed in Antolini et [3] often used in practice:
Concordance Index.
= P(Š(O,IX) S(O, < <0,
(12)
If Oi, Ar and (({]X;) denote observations and predicted conditional probabilities, respec- tively, the C-index in [2) can be estimated empirically by
c- Ei (S X,) {(O,)))))
E= O,)
The range of the C-index is [0, 1], and larger values indicate better performance with random guess leading to a C-index of 0.5.
Integrated Brier Score. Similar to the mean squared error, the Brier score 7 is a measure of performance based the predicted probability for a binary outcome. Graf et al. 24 extended the
7

---

Brier score to right censored data in order to assess the accuracy of an estimated survival function at some time Under right censorship, the Brier score time is formally given by
- Ti Σ{
[1(t) t))
[1 - S(t >1) 1
(t
G(O,)
G(I)
where & is the Kaplan-Meier estimator of G(() P(C 1) for censoring time C. As metric to evaluate the performance of survival estimates, we consider the integrated Brier score (IBS)
IBS [ BS(t) dt.
(13)
IBS is proper scoring rule 23 where smaller values of this metric indicate better performance. We calculate IBS in our experiments via a Riemann sum of the Brier score BS(().
4.2 Baseline Methods
We compare our proposed models with three traditional survival methods, random survival forest and six recent deep learning methods.
CoxPH, AFT, EH represent the standard Cox proportional hazard, accelerated failure time and extended hazard models introduced in Section
RSF leverages the accuracy of random forests for right- survival settings 33 RSF bootstrap samples from the original data and grows a tree for each bootstrapped data set by splitting on nodes that maximize the log-rank difference between their sub-nodes. It then applies the Nelson-Aalen estimator to calculate the cumulative hazard function for each tree and finally averages these estimates.
DeepSurv replaces the linear relationship in the CoxPH model with neural network 38
CoxTime extends CoxPH with a time-dependent hazard ratio, learned using neural network 41.
PCHazard treats the hazard function as piece-wise constant in predefined time-intervals, using neural network to maximize this likeliiel [4)]
DeepHit directly estimates the joint distribution of events in discrete time, using multi-task neural network trained with a likelihood loss and a ranking loss 43
DSM develops a deep learning framework to estimate the survival function that can also handle competing risks in addition to single event survival data [47].
DeepAFT is adapted from the proposed DeepEH when enforcing h1 = fı2, and we implement it as special case of DeepEH.
Among all the methods considered in this paper, DeepHit the most general model, but it may only perform well in applications where data are plentiful (which is not the case in existing survival studies) and no lower dimensional structures exist for the survival times.
4.3 Data Sets
Here evaluate the performance of various survival methods using four real survival datasets: Chemotherapy for Stage B/C colon cancer (COLON), Molecular Taxonomy of Breast Cancer Inter- national Consortium (METABRIC), Rotterdam tumor bank and German Breast Cancer Stuyy Group (RotGBSG) and Worcester Heart Attack Study (WHAS) and
COLON collected from sample of 929 subjects evaluate the effects of the drugs levamisole and fluorouracil on resected colon carcinoma (45. By the end of study, 48.6 percent patients were dead. Twelve features are included, among which two continuous features: age, number of lymph nodes with detectable cancer, and ten are categorical features: treatment, sex, obstruction colon by tumour, perforation of colon, adherence to nearby organs, differentiation of tumour, extent of local spread, time from surgery to registration, more than positive lymph nodes.
METABRIC includes paired DNA-RNA profiles and clinical information of 1,980 breast cancer patients, among which 57.7 percent are recorded to die from the cancer [16]. Based on Immunohisto- chemical 4 score plus Clinical Treatment Score (IHC4+TTS. (2]). DeepSury [38 considered four

---

0.8
C-index
COLON
ME TABRIC
RoGBBG
0.7
CoXPH
AFT
EH
RSF
DeepSurv
CoxTime
PCHazard
DeepHit
DSM
DeepAFT
DeepEH
Methods
Figure 1: Average Concordance index achieved by each survival model (higher is better) over five different train/test splits of each dataset.
0.200
IBS
0.175
COLON
METABRIC
RoGB85
0.150
CoxPH
AFT
EH
RSF
DeepSurv
CoxTime
PCHazard
DeepHit
DSM
DeepAFT
DeepEH
Methods
Figure 2: Average Integrated Brier Score achieved by each survival model (lower is better) over five different train/test splits of each dataset.
genes, four clinical indicators, and each patient's 's age as features to predict survival. We follow the same choices in study. The four genes (MK167, EGFG, PGR and ERBB2) and the patient's 's age are continuous features, while hormone treatment indicator, radiotherapy indicator, chemotherapy indicator, ER-positive indicator are binary features.
RotGBSG is the breast cancer data extracted from the Rotterdam tumor bank (1 and the German Breast Cancer Study Group [54]. The data set consists of 2,232 patients and 56.7 percent of patients have observed death time. During the studies, researchers recorded the standard features: age, tumor size, number of positive lymph nodes, progesterone and oestrogen receptor status, menopausal status and tumor grade for each patient. In our analysis, tumor size, number of positive lymph nodes and tumor grade are coded to be ordinal features, whereas other features are coded to be continuous.
WHAS is data of 1,638 subjects for studying the factors of acute myocardial infarction 33 The features contain sex, age, body mass index, left heart failure complications, and order of myocardial infarction, of which age and body mass index continuous features and others are binary. By the end of the WHAS study, 42.1 percent of the subjects had died of acute myocardial infarction.
4.4 Results
The evaluation for each dataset is done via 5-fold cross- volsddaal,,,,,n,, where randomly split each dataset into 5 equal folds that correspond to different train/test sets. Within each fold, 20% of the training data reserved
validation Performance is gauged via the C-index and IBS.
Figure
D and Figure 2 show the average C-index and IBS (on test data) over five different train/test splits, respectively. More details, including the standard deviations, are displayed in Table I of the

---

Appendix. In terms of the C-index, DeepEH outperforms the other ten survival methods on three datasets: COLON, METABRIC and RotGBSG, For the WHAS dataset, RSF is the best method followed by DeepEH. We also observe that DeepEH improves over its classical predecessor, the EH model. In addition, DeepEH outperforms all its submodels, i.e., CoxPH, AFT, DeepSurv and DeepAFT, on all datasets. In terms of IBS, DeepEH performs the best among all eleven survival methods on COLON, RoGGBSG and WHAS, and it outperformed the other ten methods substantially on RoGGBSG. Similiar to C-index, DeepEH also has better performance in IBS than its submodels (CoxPH, AFT, EH, DeepSury and DeepAFT) on all datasets.
Conclusion
Due to its massive potential, deep learning for survival analysis has attracted considerable attention across epidemiology and clinical research. A theoretical understanding, however, is still at its infancy. Mathematically and empirically, this work studied deep extensions of a flexible survival analysis method, the EH model, which can represent a wider family of survival models. We introduced a special but widely used neural network structure facilitate mathematical study of deep survival analysis. Under the EH model, the neural network estimators of our proposed DecpEH method are asymptotically consistent, as are the corresponding estimators of the survival function. The favorable convergence rates of the new estimators reveal that deep learning is a powerful tool to mitigate the curse of dimensionality with fixed but high dimensional covariates. Empirically, the proposed DeepEH model achieves superior C-index and IBS than competing methods. Our methods thus constitute valuable additions to the practical survival modeling toolkit. Their broader impact will likely be to improve estimation accuracy in existing survival analysis applications.
In the future, practical modifications of the neural network architecture may be investigated adapt the deep survival methods presented here for more complex features such as images [67] or gene expression measurements [10). While all experiments in this paper were limited to time-invariant features, extensions to time-varying features (Iongitudinal data) remain of interest. Longitudinal covariates bring rich information the study but pose challenges to conventional survival models, such as the CoxPH or AFT model, which require continuous measurements of the entire trajectory of the longitudinal covariates that are unavailable in practice [3 61]. The (bio)statistics community has been actively researching how to best model survival data and longitudinal covariate processes jointly [8 [2 64 particular, joint modeling for EH models with longitudinal covariates has been explored by Tseng et al. (58, 60). Thus natural direction for future work is to extend our DeepEH addition, this paper only provides evaluations based on C-index and IBS, where the former metric measures the discriminative ability of model while the latter accounts for both discrimination and calibration. Broader evaluation of deep survival methods remains of interest, including measures such as 1-calibration and D-calibration [25]
models to this setting.
Acknowledgements
We are grateful to anonymous Referees for valuable feedback on this work. The research of Qixian Zhong is supported by National Natural Science Foundation of China grant USFC-11931000 Key Laboratory of Econometrics (Xiamen University), and Ministry of Education and Fujian Key Lab of Statistical Science, Xiamen University. The research of Jane-Ling Wang is supported by the USA NSF grant DMS-1914917
10

---

References
[1] O. Aalen. Nonparametric inference for a family of counting processes. The Annals of Statistics, 6(4):701-726, 1978.
[2] O. Aalen and H. K. Gjessing. Understanding the shape of the hazard rate: process point of view (with comments and a rejoinder by the authors). Statistical Science, 16(1):1-22) 2001.
[3] L. Antolini, P. Boracchi, and E. Biganzoli. A time-dependent discrimination index for survival data. Statistics in Medicine, 24(24):3927-3944 2005.
[4] B. Bauer and M. Kohler. On deep learning as a remedy for the curse of dimensionality in nonparametric regression. The Annals of Statistics, 47(4):2261 2019.
[5] R. Beran. Nonparametric regression with randomly censored survival data. Technical report, University of California, Berkeley, 1981.
[6] N. E. Breslow. Discussion of the paper by DR Cox. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 34(2):216-217, 1972.
[7] G. Brier. Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1):1-3, 1950.
[8] P. Chapfuwa, C. Tao, C. Li, C. Page, B. Goldstein, L., C. Duke, and R. Henao. Adversarial time-to-event modeling. In International Conference on Machine Learning, pages 735-744. Proceedings of Machine Learning Research, 2018.
[9] Chapfuwa, C. Tao, I.. Carin, and R. Henao. Survival function matching for calibrated time-to-event predictions. arXiv preprint arXiv:1905.08833 088888 2019.
[10] T. Ching, X. Zhu, and L. X. Garmire. Cox-nnet: artificial neural network method for prognosis prediction of high-throughput omics data. PLoS Computational Biology, 14(4): c1006076, 2018
[11] A. Ciampi and J. Etezadi-Amoli A general model for testing the proportional hazards and the accelerated failure time hypotheses in the analysis of censored survival data with covariates. Communications in Statistics-Thean and Methods, 14(3):651-6677 1985.
[12] A. Cottin, N. Pécuchet, M. Zulian, A. Guilloux, and S. Katsahian. IDNetwork: A deep illness- death network based on multi-states event history process for versatile disease prognostication. In Survival Irrdiction-Algorithnn Challenges and Applications, pages 1-21. Proceedings of Machine Learning Research, 2021.
[13] D. R. Cox. Regression models and life-tables. Journal of the Royal Statistical Sociery: Series (Statistical Methodology). 34(2):187-202, 1972.
[14] D. R. Cox. Partial likelihood. Biometrika, 62(2):269-276 1975.
[15] D. R. Cox and D. Oakes. Analysis of Survival Data. Monographs on Statistics and Applied Probability. Chapman & Hall, London, 1984.
[16] C. Curtis, S. P. Shah, S.-F. Chin, G. Turashvili, O. M. Rueda, M. Dunning, D. Speed, A. G. Lynch, S. Samarajiwa, and Y. Yuan. The genomic and transcriptomic architecture of 2,000 breast tumours reveals novel subgroups. Nature, 486:346-355 2012.
[17] G. Cybenko. Approximation by superpositions of sigmoidal function. Mathematics of Control, Signals, and Systems, 2(4):303-314, 1989.
[18] R. Flashoff, G. Li, and N. Li. Joint Modeling of Longitudinal and Time-to-event Data. CRC Press, 2016.
[19] J. Etezadi-Amoli and A. Ciampi. Extended hazard regression for censored survival data with covariates: a spline approximation for the baseline hazard function. Biometrics, 43(1):181-192, 1987.

---

[20] D. Faraggi and R. Simon. A neural network model for survival data. Statistics in Medicine, 14 (1):73-82) 1995.
[21] J. A. Foekens, H. A. Peters, M. P. Look, H. Portengen, M. Schmitt, M. D. Kramer, N. Brünner, Jänicke, M. E. Meijer-van Gelder, and S. C. -lenzen-Logmans. The urokinase system of plasminogen activation and prognosis in 2780 breast cancer patients. Cancer Research, 60(3): 636- 643, 2000.
[22] E. Giunchiglia, A. Nemchenko, and M. van der Schaar. RNN-SURV: A deep recurrent model for survival analysis. In International Conference Artificial Neural Networks, pages 23-32. Springer, 2018.
[23] T. Gneiting and A. E. Raftery. Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association, 102(477):35--375 2007.
[24] E. Graf, C. Schmoor, W. Sauerbrei, and M. Schumacher. Assessment and comparison of prognostic classification schemes for survival data. Statistics Medicine, 1817-118) 2545, 1999.
[25] H. Haider, B. Hoehn, S. Davis, and R. Greiner. Effective ways build and evaluate individual survival distributions. Journal of Machine Learning Research, 21:85-63, 2020.
[26] J. Hao, Y. Kim, T. Mallavarapu, J. H. Oh, and M. Kang. Interpretable deep neural network for cancer survival analysis by integrating genomic and clinical data. BMC Medical Genomics, 12 (10):1-13, 2019. 2019.
[27] E. Harrell, R. M. Califf, B. Pryor, L. Lee, and A. Rosati. Evaluating the yield of medical tests. Journal of the American Medical Association, 247(18):2543-2546, 1982.
[28] Heitz, J. Ficek, M. Faltys, T. M. Merz, G. Rätsch, and M. Hüser. WRSE-a non-parametric weighted-ressolution ensemble for predicting individual survival distributions in the ICU. In Survival Prediction-Algoriumms, Challenges and Applications, pages 54-69. Proceedings of Machine Learning Research, 2021.
[29] K. Hornik, M. Stincheombe, and H. White. Multilayer feedforward networks universal approximators. Neural Networks, 2(5):359-366 1989.
[30] D. W. Hosmer Jr, S. Lemeshow, and S. May. Applied Survival Analysis: Regression Modeling of Time-to-event Data. Wiley, 2008
[31] F. Hsieh, Y.-K. Tseng, and J.-L. Wang. Joint modeling of survival and longitudinal data: likelihood approach revisited. Biometrics, 62(4):1037-1043 2006.
[32] D. Hubhard, B. Rostykus, Y. Raimond, and T. Jebara. Beta survival models. In Survival Prediction-Algoritmms Challenges and Applications, pages 22-39. Proceedings of Machine Learning Research, 2021.
[33] H. Ishwaran, U. B. Kogalur, E. H. Blackstone, and M. S. Lauer. Random survival forests. The Annals of Applied Statistics, 2(3):841-866 2008.
[34] A. Kalatian and B. Farooq. Deepsurvival: Pedestrian wait time estimation in mixed traffic conditions using deep survival analysis. arXiv preprint arXiv: 1008, 2019.
[35] J. D. Kalbfleisch and R. L.. Prentice. The Statistical Analysis of Failure Time Data. John Wiley & Sons, 2011.
[36] S. Kalmady, W. Sun, J. Ezekowitz, N. Fine, J. Howlett, A. Savu, R. Greiner, and Kaul. Improving the calibration of long term predictions of heart failure rehospitalizations using medical concept embedding. In Survival Prediction-Alooritmms Challenges and Applications, pages 70-82. Proceedings of Machine Learning Research, 2021.
[37] E. I.. Kaplan and P. Meier. Nonparametric estimation from incomplete observations. Journal of the American Statistical Association, 53(282):457-453 1958.
12

---

[38] J. L. Katzman, U. Shaham, A. Cloninger, J. Bates, T. Jiang, and Y. Kluger. Deepsurv: person alized treatment recommender system using Cox proportional hazards deep neural network. BMC Medical Research Methodology, 18(1):24, 2018.
[39] N. Keiding, K. Andersen, and J. P. Klein. The role of frailty models and accelerated failure time models in describing heterogeneity due to omitted covariates. Statistics in Medicine, 16(2):
215-224, 1997.
[40] H. Kvamme and Ø. Borgan. Continuous and discrete- time survival prediction with neural networks. Lifetime Data Analysis, 27:7 27:710-736, 2021.
[41] H. Kvamme, Ø. Borgan, and Scheel. Time-to-event prediction with neural networks and Cox regression. Journal of Machine Learning Research, 20(129): 1-30, 2019.
[42] R. Lakhanpal, I. Sestak, B. Shadbolt, G. M. Bennett, M. Brown, T. Phillips, Zhang, A. Bull- man, and A. Rezo. Ihe4 score plus clinical treatment score predicts locoregional recurrence in early breast cancer. The Breast, 29:147-152, 2016.
[43] C. Lee, W. R. Zame, J. Yoon, and M. van der Schaar. Deephit: A deep learning approach survival analysis with competing risks. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018.
[44] E. T. Lee and J. Wang. Statistical Methods for Survival Data Analysis. John Wiley & Sons, 2003.
[45] D. Lin. Cox regression analysis of multivariate failure time data: the marginal approach. Statistics in Medicine, 13(21))2233-22) 2247777, 1994.
[46] R. G. Miller Jr. Survival Analysis. John Wiley & Sons, 2011.
[47] C. Nagpal, X. R. Li. and A. Dubrawski. Deep survival machines: Fully parametric survival regression and representation learning for censored data with competing risks. IEEE Journal of Biomedical and Health Informatics, 25(8):3163-317 2021.
[48] W. Nelson. Theory and applications of hazard plotting for censored failure data. Technometrics, 14(4):945 1972.
[49] S. Pölsterl, Sarasua, Kutiérrez-Beckeer and C. Wachinger. A wide and deep neural network for survival analysis from anatomical shape and tabular clinical data. arXiv preprint arXiv:190.0038% 2019.
[50] R. Ranganath, A. Perotte, N. Elhadad, and D. Blei. Deep survival analysis. In Proceedings of the 1st Machine Learning for Healthcare Conference, pages 101-114. Proceedings of Machine Learning Research, 2016.
[51] K. Ren, J. Qin, I.. Zheng, 7. Yang, W. Zhang, L. Qiu, and Y. Yu. Deep recurrent survival analysis. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 4795-4805, 2019.
[52] D. Rizopoulos. Joint Models for Longitudinal and Time-to-evens data: With Applications in Chapman and Hall/CRC, 2012.
[53] Schmidt-Hieber. Nonparametric regression using deep neural networks with ReLU activation function. Annals of Statistics, (8(4))1775-1897 2020.
[54] M. Schumacher, G. Bastert. H. Bojar, K. Huebner, M. Olschewski, W. Sauerbrei, C. Schmoor, C. Beyerle, R. Neumann, and H. Rauschecker. Randomized x trial evaluating hormonal treatment and the duration of chemotherapy in node positive breast cancer patients. german breast cancer study group. Journal of Clinical Oncology, 12(10))0086-093, 1994.
[55] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. Dropout: simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, (929):9929-158 2014.
13

---

[56] T. Sylvain, M. Luck, J. Cohen, H. Cardinal, A. Lodi, and Y. Bengio. Exploring the wasser- stein metric for time-to-evene analysis. In Survival Drediction-Algoritine Challenges and Applications, pages 194-206. Proceedings of Machine Learning Research, 2021.
[57] A. Tarkhan, N. Simon, T. Bengtsson, K. Nguyen, and J. Dai. Survival prediction using deep learning. In Survival PredictionAlloorthhns Challenges and Applications, pages 207-214. Proceedings of Machine Learning Research, 2021.
[58] Y. Tseng, Y. Su, M. Mao, and J. Wang. An extended hazard model with longitudinal covariates. Biometrika, 102(1):135-150, 2015.
[59] Y.-K. Tseng and K.-N. Shu. Efficient estimation for a semiparametric extended hazards model. Communications in Statistics Simulation and Computation, 40(2):258-273, 2011.
[60] Y.-K. Tseng, Hsieh, and J.-L.. Wang. Joint modelling accelerated failure time and longitu- dinal data. Biometrika, 92(3):587-600 2005.
[61] A. Tsiatis, V. Degruttola, and M. Wulfsohn. Modeling the relationship of survival to longitudinal data measured with error. applications to survival and CD4 counts in patients with AIDS. Journal of the American Statistical Association, 90(429):27-37, 1995.
[62] L.-J. Wei. The accelerated failure time model: a useful alternative the Cox regression model in survival analysis. Statistics in Medicine, (((1--55)1871-1183 1992.
[63] A. Xiang, P. Lapuerta, A. Ryutov, J. Buckley, and S. Azen. Comparison of the performance of neural network methods and Cox regression for censored survival data. Computational Statisties & Data Analysis, 34(2):243-25 257- 2000.
[64] C. Xu, P. Z. Hadjipantelis, and J.-L.. Wang. Semi-parametre joint modeling of survival and longitudinal data: The R package JSM. Journal of Statistical Software, 93(2):1-29, 2020.
[65] D. Yarotsky. Error bounds for approximations with deep ReLU networks. Neural Nenworks, 94: 103-114, 2017.
[66] Q. Zhong, J. Mueller, and J.-L. Wang. Deep learning for the partially linear Cox model. 2021. Submitted.
[67] X. Zhu, J. Yao, and J. Huang. Deep convolutional neural network for survival analysis with patho- logical images. In 2016 IEEE International Conference on Bioinformatics and Biomedicine. pages 544-547. IEEE, 2016.
14


Reviews:
[{"review_id": "i-8o1HHvd8", "reviewer": "Reviewer_z47A", "summary": "Substituting two general functions for linear risk predictors in the Extended Hazard (EH) model, the DeepEH approximates these two functions via feed-forward neural networks (FNN). The DeepEH may fall into the category of FNN-assisted survival analysis which utilizes FNNs to estimate the hazard/survival function. The shining point of this paper is the established asymptotic properties of estimators from DeepEH. \n\n\n", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Depending on up-to-date theoretical developments on FNN approximation such as Schmidt-Hieber (2020, Ann. Statist. 48:1875--1897), this submission makes a theoretical contribution to the FNN-assisted survival analysis. We guess this effort is of interest to the community of deep learning research. Also, this paper involves extensive numerical comparison among DeepEH and a dozen competitors.\n\nCorrectness: We believe the theories and proofs are generally correct. \n\nClarity: Most parts are easy to follow.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The extension from the EH model is natural. In addition, there has been literature that proves the power of FNN from a theoretical point of view, whereas this paper fails to make a review in this regard. Among other works, Schmidt-Hieber (2020) gave an exact upper bound of the approximation error for FNNs involving the least-square loss. Since the DeepEH optimizes a likelihood-based loss, this paper builds up its asymptotic properties by following assumptions and proofs of Theorems 1 and 2 in Schmidt-Hieber (2020) as well as theories on empirical processes.\n\nAdditional Feedback: 1) In the manuscript, $\\mathbb{P}$ mostly represents a probability but sometimes for a cumulative distribution function (e.g., Eqs. (3) and (4) and L44, all in Appendix), which leads to confusion. 2). The notation $K$ is abused too: it is used both for a known kernel function (e.g., L166) and the number of layers (e.g., L176). 3). What is $K_b$ in estimating baseline hazard (L172)?", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "1 hour"}, "extra_text": {}}, {"review_id": "dM1WNOaoqQu", "reviewer": "Reviewer_hDJX", "summary": "This paper introduces the Deep Extended Hazard Model as a new approach to learning a survival model, based on learning 2 neural nets, to estimate the 2 terms of the extended hazard function.  It explains how this model extends the EH model, which extends both the Cox proportional hazard model and Accelerated Failure Time.  It then provides first a theoretical analysis, following by an empirical exploration of these models ", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper did a nice job of motivating then explaining the various models considered. However, I had a hard time following the theoretical analysis.  Eg. I wish the authors had provided some details of the derivation of Eq1, or a pointer.  I also wondered which parts of Sec 3.2 were general for all neural nets, and what was specific to survival functions, and what just for variants of nn extensions of EH models? Some parts could have been a bit more precise--e.g I assume that \\hat-h is based on the sample size n (line 225). Also, I did not understand how the theoretical analysis informed the development of the algorithm, or was used in the analysis of the empirical results.  \n\nThey empirically explored many (10) survival models, over 4 different datasets. The authors should have performed some statistical tests, to indicate which of the results were significant.  Given the paper’s concern with high dimensional covariates, it was surprising that the datasets were all low-dimensional.  I wish the authors had empirically explored this issue, by using high dimensional data.\n\nWhile IBS is considered both discriminative and calibration, it would have been useful to see if each dataset was calibrated--perhaps at the median time.  See the analysis in https://jmlr.csail.mit.edu/papers/v21/18-772.html .\n\n\nwrt the IBS:  Li258, \\hat-G is NOT the KM estimator.  It is the estimator of the conditional survival function of the censoring times calculated using the Kaplan-Meier method.  Is the integral Eq12 done analytically for your models, or estimated empirically? If the latter, are there issues related to the intervals considered?  Is \\tau the largest time (censored or not), or the largest uncensored time, or something else? \n\nThe authors should have discussed how they split the data into 5 folds–was this balanced for (censored, death) times? For anything else?\n\nI wish they plotted the results in a figure–perhaps moving Table 1 to the appendix. Similarly, it would be helpful if they had plotted the hazard curves, or the survival curves, to help the reader visualize the results.\n\n \nli55: given the many different ways to evaluate a survival model, what does “improvement” mean?\n\nli77: I assume  “omitted features” means some values are missing for some instances. If so, are these omissions considered  MCAR?\n\nli119: extra β\n\nli277: Why mention competing risks? Isn’t there only a single type of risk?\n\n--------------\nMy assessment would be better if the paper\n\n included statistical conclusions\n\n explicitly connected the theoretical analysis to the algorithm design and/or the empirical results\n\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Predicting a patient's survival time for a given treatment, could help identify the appropriate treatment.  Of course, it could also be used to determine the most toxic poison or disease.  However, I do not think the manuscript needs to discuss this.\n\n", "review_text": "", "extra_scores": {"ethical_concerns": "No ethical concerns.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "N-yPs4i0tBO", "reviewer": "Reviewer_cez1", "summary": "The authors propose to extend the \"Extended Hazards Model\" (EH) with deep learning approaches to model right censored time to event data. EH model is a generalization of both the Accelerated Failure Time and the Cox Model Proportional Hazards Model and thus is more flexible than either of the two approaches. They demonstrate empirically that the extended hazards model improves the prediction of survival across multiple datasets in terms of both discriminative performance (C-Index) and Integrated Brier Score on multiple competitive baselines. \n\n", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\nLearning survival models to model right censored datasets with neural networks is a challenging problem. The authors propose to employ the extended hazards model to model survival data. Their motivation is well justified intuitively. The EH is a generalization of both the Cox Proportional Hazards model, and the Accelerated Failure Time model and thus alleviates challenges associated with these two models. \n\n## Strengths\n\n- Rigorous experimental evaluation across multiple strong datasets and baselines, in terms of both Discriminative Performance and Integrated Brier Score. \n- A major challenge with the CoxPH model is its limitations in handling problems where the PH assumptions are not met. The Extended Hazards approach is well justified intuitively since, it can alleviate these challenges. \n- Some interesting theoretical justification for the use of Deep Learning to learn low dimensional representation to better estimate the time-to-event distribution. \n\n## Weaknesses:\n\n- Although the paper attempts at providing a theoretical justification, in Sec 3.2. The theoretical results are largely a consequence of the convergence rates and results already established in (Y.-K. Tseng and K.-N. Shu., 2011) and (Zeng and Lin, 2007) making this part of the contribution, much less novel.\n- I would thus evaluate this paper as a modeling paper, and not necessary as theoretical contribution. Unfortunately, in terms of modelling the main text of the paper glosses over many important details of the Extended Hazards model which would be required for reproducibility by a reader not deeply entrenched in survival analysis. For example, \n\nA) it is unclear how Eq. (4) -> Eq. (5), since the authors don't explicitly mention the relationship between the hazard function and the survival function.\n\nB) In the estimate of the Pseudo-Likelihood (post line 165),  it is unclear how the Kernel $\\mathcal{K}(\\cdot)$ shows up as it has not been referred to before. In any case, the likelihood $\\mathcal{L}(\\cdot)$ should be explicitly rewritten as a function of the Kernel $\\mathcal{K}_b(\\cdot)$ with bandwidth $b$. \n\nC) It is unclear as to how $\\Phi(\\cdot)$ is computed. (It is an integral over event times) is this computed over a fixed discrete grid or over all observed event times? \n\n- Finally, I have major concerns about reproducibility of this paper. For example, in the main text and the supplement, the authors do not mention the choice of the  kernel function ($\\mathcal{K}(\\cdot)$) they use or how the bandwidth is selected. While the number of baselines considered are extensive; there is no discussion about the hyperparameter grid over which the baselines were tuned. \n\nOverall, the paper proposes an interesting extension to the original idea of extended hazards using Deep Learning, which is welcome. The paper also aims to provide theoretical justification for their choice, however some of the implementation and design details are ignored in the paper calling into question the reproducibility of the proposed approach.\n\n\n## Other Concerns/Recommendations: \n\n- Have the authors evaluated performance in terms of Concordance over truncated event times? If yes, they are encouraged to include these results in the supplement to establish how the discriminative performance varies at different time horizons. \n- An important problem that the model aims to alleviate is the strong assumption of proportional hazards. The authors are recommended to include plots of the estimated survival distributions on real world data in the main text to demonstrate the model is able to capture scenarios where the assumption is violated. (One clear situation of this is when the estimated survival curves of two individuals intersect)\n- The authors are recommended to keep the theorems in the main text but to move exact details of Section 3.2 to the supplement, and use the extra space to address the concerns raised above. \n- Have the authors compared the performance of their approach to stratified Cox/AFT approaches (separate cox/aft models for different strata) or approaches that involve learning mixtures of Cox/aft models (Discrete mixture regression models for heterogenous time-to-event data: Cox assisted clustering, Kevin H. Eng & Bret M. Hanlon, 2012; Deep Cox Mixtures for Survival Regression, Nagpal et al, 2021; )? If not, can the authors comment when should an EH model be preferred over using stratified Cox models? ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors should clearly describe how strong assumptions on the hazard distributions can lead to miss estimation of the risk profile/ trajectory for individuals leading to potentially harmful decision making. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "Co1UmgCPMqJ", "reviewer": "Reviewer_kQKv", "summary": "This work proposes DeepEH, a survival analysis framework that formulates the hazard function as \\lambda(t|X) = \\lambda_0(t.exp(h_1(x))).exp(h_2(x)). This formulation has the Cox PH and AFT as special cases. The authors estimate h_1 and h_2 by neural networks and provide theoretical support for the consistency and convergence rate of the resulting survival function estimator.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "1) I saw a few errors in the English and a few typos: \"adopt a parametric approaches\", \"are asymptotically consistent 90 and enjoys fast convergence rates\", \"advantageousß\", \"aasumes\", \"The data sets consists of\"\n\n2) What do you mean by \"omitted features\" in \"Keiding et al. [33] also show that, relative to CoxPH, AFT models are more stable when accounting for omitted features.\"?\n\n3) When you say \"Specially, the convergence rates are determined by the intrinsic dimension of the underlying functions rather than the original high-dimension input features.\" and then go on to say \"This, to some extent, explains the great success of deep learning and distinguishes our contributions from existing work.\", how is your modeling different from previous survival analysis frameworks that make use of deep learning? In other words, what sets your work apart from other survival analysis algorithms that use deep learning given they are using neural networks too?\n\n4) In section 2.1, when you say \"It is difficult to estimate and directly from the likelihood function\", it would have been beneficial to add a sentence or two explaining why.\n\n5) When you say \"This is the main reason why the Cox proportional hazards model dominates the field of survival analysis for nearly fifty years\" what is the main reason of Cox's domination? Optimizing the parametric and non-parametric components separately?\n\n6) In section 3.1, the pseudo-likelihood can be better justified. The intuition behind the formula of the loss is missing. It seems to me that the pseudo-likelihood and the hazard function that is introduced afterwards can switch places. A better storyline would have been: (according to my understanding of what this section is trying to say) Following [52] the hazard function can be approximated by the following formula because ... which would then give rise to a pseudo-likelihood (why is it \"pseudo\" in the first place?) which is ...\n\n7) In section 4.1, the evaluation criteria can be intuitively more elaborated on. The C-index is a bit explained when you talk about assessing the concordance between rankings but the Integrated Brier Score is not explained much. Giving more intuition to the reader would go a long way in understanding how these criteria are useful in assessing survival estimators.\n\n8) How are you recovering the survival function from the hazard function? Are you using numerical integration methods? \n\n9) In section 4.2, why DeepHit only performs well when no lower dimensional structures exist for the survival times?\n\n10) Table 1 could use rearrangement. It is difficult to read. Could be decluttered.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper has been reviewed by four knowledgeable referees resulting in one accept, two marginal accept and one marginal reject recommendations. Most of the criticism centers on limited novelty and utility of the theoretical portion of this work. Yet, the proposed method appears to stand on its own, yet it has been evaluated on a relatively low dimensional data only. The authors have engaged in discussions with the reviewers that helped resolve some but not all of the stated concerns. All things considered, I recommend this paper for acceptance if space permits its inclusion in the conference agenda. The authors should strive to incorporate all constructive recommendations from the reviewers in the finał camera ready version of their paper.
Decision: Accept (Poster)