Towards simple time-to-event modeling: optimizing neural networks via rank regression

1 INTRODUCTION . Time-to-event analysis , also known as survival or failure time analysis , is a major statistical approach in various fields such as biostatistics , medicine , and economics to estimate either risk scores or the distribution of event time , given a set of features of subjects ( Viganò et al. , 2000 ; Cheng et al. , 2016 ; Dirick et al. , 2017 ; Li et al. , 2021 ) . There are benefits of assessing risk or quantifying survival probabilities but , for all that , time-to-event analysis itself is challenging because of the existence of censoring . In real-world studies , a subject ( e.g . a patient in medical research ) can drop out before events of interest ( e.g . death ) happen , so that one can not follow them up ( Leung et al. , 1997 ) . The presence of censoring in survival data can create a serious challenge in applying standard statistical modeling strategies . Usually , the censoring process is assumed to be non-informative in that it is irrelevant of the underlying failure process given features , but should be properly accounted for , otherwise leading to biased results . The most popular and standard approach for modeling time-to-event data is to use Cox ’ s proportional hazards ( CoxPH ) model . CoxPH relates a conditional hazard to given features in a multiplicative form between the baseline hazard function and exponentiated regression component , and consequently learns relative risks . It often works on the assumptions of proportional hazards and time-invariant covariate-effects , which are difficult to follow in the real world ( Aalen , 1994 ) . Statistical testing procedures , such as Schoenfeld ’ s test , are usually conducted to examine these underlying assumptions since many Cox-based analyses are vulnerable to violation of model assumptions . ( Aalen & Gjessing , 2001 ; Kleinbaum & Klein , 2010 ) . The accelerated failure time model ( AFT ) or accelerated life model relates the logarithm of the failure time linearly to the features . As a result of its direct physical interpretation and the connection with linear models , this model provides an attractive alternative to the CoxPH for the regression analysis of censored failure time data . Unlike CoxPH , standard AFT model parametrizes the underlying time-to-event distribution up to a set of finite-dimensional parameters such as Weibull and log-normal ( Lee & Wang , 2003 ) . However , imposing distributional assumption is too strict in real data analysis and can reduce the attractiveness inherent in the AFT model , mostly underperforming Cox-based analysis ( Cox , 2008 ; Kleinbaum & Klein , 2010 ) . Recently , based on statistical theories and the advent of deep learning techniques , various time-to-event models have been explored to circumvent the necessity of assumptions such as linearity , single risk , discrete time , and fixed-time effect ( Katzman et al. , 2018 ; Lee et al. , 2018 ; Ren et al. , 2019 ; Kvamme & Borgan , 2019 ; Avati et al. , 2020 ; Tarkhan et al. , 2021 ; Rahman et al. , 2021 ) . For example , Cox-Time ( Kvamme et al. , 2019 ) and DATE ( Chapfuwa et al. , 2018 ) alleviate the most fundamental but strict assumptions of the CoxPH and parametric AFT models by achieving nonproportional hazards and non-parametric event-time distribution , respectively . Cox-Time exploits the neural network as a relative risk function to model interactions between time and covariates . The authors also show that the proposed loss function is a good approximation for the Cox partial loglikelihood . DATE is a conditional generative adversarial network for implicitly specifying a timeto-event distribution of ATF model . It does not require the pre-specified distribution in parametric form , instead , the generator can learn it from the data with the adversarial loss function . Incidentally , various deep learning-based approaches have been spotlighted to improve performance by resolving issues such as temporal dynamics and calibration ( Lee et al. , 2019 ; Nagpal et al. , 2021 ; Gao & Cui , 2021 ; Kamran & Wiens , 2021 ; Hu et al. , 2021 ) . Therefore , it became important to utilize welldesigned objective functions that fit not only statistical backgrounds but also optimization of neural networks . In this paper , we introduce a Deep AFT Rank-regression for Time-to-event prediction model ( DART ) , a deep learning-based semiparametric AFT model trained with an objective function originated from Gehan ’ s rank statistic . The model does not require specifying event time distribution while keeping the advantage of the standard AFT model that directly predicts event time . With a simple form of the loss function , by constructing comparable rank pairs , the optimization of DART is efficient compared to other deep learning-based time-to-event models . Experimental results show that DART is not only well-calibrated but also competitive in event order prediction performance even compared to hazard-based models . Furthermore , we believe that this work can be widely applied in the community while giving prominence to advantages of the AFT model that is relatively unexplored . 2 RELATED WORKS . We first overview time-to-event modeling focusing on the loss functions of Cox-Time and DATE models to highlight the difference in concepts before introducing our method . The primary interest of time-to-event analysis is to estimate survival quantities like survival function S ( t ) = P ( T ≥ t ) or hazard function h ( t ) = limδ→0 P ( t ≤ T ≤ t + δ|T ≥ t ) /δ , where T ∈ R+ denotes time-to-event random variable . In most cases , due to censored observations , those quantities can not be directly estimated with standard statistical inference procedure . In the presence of right censoring , Kaplan & Meier ( 1958 ) and Aalen ( 1978 ) provided consistent nonparametric survival function estimators , exploiting right-censoring time random variable C ∈ R+ . Researchers then can get stable estimates for survival quantities with data tuples { yi , δi , Xi } Ni=1 , where yi = min ( Ti , Ci ) is the observed event time with censoring , δi = I ( Ti ≤ Ci ) is the censoring indicator , and a vector of features Xi ∈ RP . Here , N and P denote the number of instances and the number of features , respectively . While those nonparametric methods are useful , one can improve predictive power by incorporating feature information in a way of regression modeling . Cox proportional-hazards ( CoxPH ) and acceleratedfailure-time ( AFT ) frameworks are the most common approaches in modeling survival quantities utilizing both censoring and features . 2.1 HAZARD-BASED MODELS . A standard CoxPH regression model ( Cox , 1972 ) formulates the conditional hazard function as : h ( t|Xi ) = h0 ( t ) exp ( βTXi ) , ( i = 1 , . . . , N ) , ( 1 ) where h0 ( · ) is an unknown baseline hazard function which has to be estimated nonparametrically , and β ∈ RP is the regression coefficient vector . It is one of the most celebrated models in statistics in that β can be estimated at full statistical efficiency while achieving nonparametric flexibility on h0 under the proportionality assumption . Note the model is semiparametric due to the unspecified underlying baseline hazard function h0 . Letting Ri be the set of all individuals “ at risk ” , meaning that are not censored and have not experienced the event before Ti , statistically efficient estimator for regression coefficients can be obtained minimizing the loss function with respect to β : LCoxPH ( β ) = ∑ i δi log ∑ j∈Ri exp [ βTXj − βTXi ] , ( 2 ) which is equivalent to the negative partial log-likelihood function of CoxPH model . Based on this loss function , Kvamme et al . ( 2019 ) proposed a deep-learning algorithm for the hazard-based predictive model , namely Cox-Time , replacing βTXj and βTXi with g ( yj , Xj ; θ ) and g ( yi , Xi ; θ ) , respectively . Here , g ( · ) denotes the neural networks parameterized by θ , and Ri would be replaced by R̃i , representing the sampled subset ofRi . With a simple modification of the standard loss function in Eq . ( 2 ) , Cox-Time can alleviate the proportionality for relative risk , showing empirically remarkable performance against other hazard-based models in both event ordering and survival calibration . 2.2 ACCELERATED-FAILURE-TIME MODELS . The conventional AFT model relates the log-transformed survival time to a set of features in a linear form : log Ti = β TXi + i , ( i = 1 , . . . , N ) , ( 3 ) where i is an independent and identically distributed error term with a common distribution function F0 ( · ) that is often assumed to be Weibull , exponential , log-normal , etc . As implied in Eq . ( 3 ) , AFT model takes a form of linear modeling and provides an intuitive and physical interpretation on event time without detouring via the vague concept of hazard function , making it a powerful alternative to hazard-based analysis . However , imposing a parametric distributional assumption for i is a critical drawback of the model , for which model in Eq . ( 3 ) could be a subclass of the hazard-based models . To alleviate linearity and parametric distributional assumptions , several works brought the concept of generative process and approximated the error distribution via neural networks like generative adversarial network ( GAN ) ( Miscouridou et al. , 2018 ; Chapfuwa et al. , 2018 ) . Especially , Chapfuwa et al . ( 2018 ) proposed a deep adversarial time-to-event ( DATE ) model , which specifies the loss function as : LDATE ( θ , φ ) = E ( X , y ) ∼Fnc [ Dφ ( X , y ) ] + EX∼Fnc , ξ∼Fξ [ 1−Dφ ( X , Gθ ( X , ξ ; δ = 1 ) ) ] + λ2E ( X , y ) ∼Fc , ξ∼Fξ [ max ( 0 , y −Gθ ( X , ξ ; δ = 0 ) ) ] ( 4 ) + λ3E ( X , y ) ∼Fnc [ ‖t−Gθ ( X , ξ ; δ = 1 ) ‖1 ] where θ , φ denotes the parameter set associated with a generator Gθ and a discriminator Dφ , respectively , ( λ2 , λ3 ) are hyperparameters to tune censoring trade-off , Fnc ( X , y ) and Fc ( X , y ) are empirical joint distributions for non-censored cases and censored cases , respectively , and Fξ is the simple distribution , such as uniform distribution . The generatorGθ implicitly defines event time distribution . Despite DATE achieves prominent survival calibration via the sample-generating process , the objective function is quite complicated and the GAN framework is inherently prone to mode collapse , i.e. , the generator learns only a few modes of the true distribution while missing other modes ( Srivastava et al. , 2017 ) . Also , when optimizing neural networks with multiple loss functions , it is difficult to balance and there might be conflicts ( i.e . trade-off ) with each term ( Dosovitskiy & Djolonga , 2020 ) . Therefore , their loss function might be difficult to be optimized as intended and requires a burdening training time , and consequently not be suitable for large-scale time-to-event analysis . In the statistical literature , there have been many attempts to directly estimate regression coefficients in the semiparametric AFT model , where the error distribution F0 is left unknown , rather than imposing specific parametric distribution or exploiting generative models . In this work , we bridge non-linear representation learning and an objective function for estimation of semiparametric AFT model , which is originated from Gehan ’ s rank statistic . By extensive quantitative analysis , we have shown the beauty of simplicity and compatibility of rank-based estimation , along with outstanding experimental performance .

Summary: The authors suggest a neural network based accelerated failure time (AFT) model, as well as an L1-type rank loss, which they argue results in an easy and fast to train model, which is state of the art in terms of two standard evaluation metrics, concordance and integrated Brier score. Following the semi-parametric AFT literature, the baseline hazard is estimated from the data and the full hazard function is parametrized in a way that the parametric part corresponds to a prediction of the failure time. The main innovation of the paper is the use of a novel loss function based on a ranking loss using Gehan’s rank statistic, which while being used in the statistical literature, has not been used in a machine learning setting.