SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Hideaki Kim
NTT Human Informatics Laboratories
NTT Corporation
hideaki.kin@ntt.com
Survival or time-to-event data with time-varying covariates are common in practice, and exploring the non-stationarity in covariates is essential to accurately analyzing the nonlinear dependence of time-to-event outcomes on covariates. Traditional survival analysis methods such as Cox proportional hazards model have been extended to address the time-varying covariates through a counting process formulation, although sophisticated machine learning methods that can accommodate time-varying covariates have been limited. In this paper, we propose a non-parametric Bayesian survival model to analyze the nonlinear dependence of time-to-event outcomes on time-varying covariates. We focus on a computationally feasible Cox process called *permanent* process, which assumes the square root of hazard function to be generated from a Gaussian process, and tailor it for survival data with time-varying covariates. We verify that the proposed model holds with the representer theorem, a beneficial property for functional analysis, which offers us a fast Bayesian estimation algorithm that scales linearly with the number of observed events without relying on Markov Chain Monte Carlo computation. We evaluate our algorithm on synthetic and real-world data, and show that it achieves comparable predictive accuracy while being tens to hundreds of times faster than state-of-the-art methods.
Survival or time-to-event data analysis has been widely applied for analyzing the dependence of survival time, the time until an event occurs, on covariates. They have a wide ranging list of applications in reliability engineering [27], finance [14], marketing [20], and especially, clinical research [4, 5, 28].
An essential part of the survival data analysis is to estimate a hazard function, that is, the instantaneous probability of events occurring at any particular time, as a regression function of covariates. Given a hazard function, we can evaluate the impact of covariates on survival times, and assess the degree of risk of experiencing an event in the future, through the survival function, i.e., the probability of no events occurring during a specified interval. The literature contains a vast number of studies that have proposed survival models to estimate hazard functions, most of which assume that covariates are time-invariant (e.g., gender and age at the time of diagnosis in clinical research): they range from the classical Cox proportional hazards model [5] to modern machine learning models based on generalized boosting machines [38], random forests [17], Gaussian processes [11, 25], and deep neural networks [10, 22, 49]. However, time-varying covariates are common in survival data, and the rapid advances in data collection allow us to access long-term and high temporal resolution covariate data. This is encouraging researchers to explore the non-stationarity in covariates for the accurate estimation/prediction of time-to-event outcomes. An important application of sur
vival analysis with time-varying covariates is what-if analysis: through simulations of future events under various covariate functions over time, we can find the optimal covariate function or policy that would yield a desirable future state. For example, in reliability engineering applications where the events are the failures of machines, time-varying covariates could be the maintenance schedule and the room temperature/humidity controlled by air conditioners. The maintenance manager can optimize the schedules of maintenance and air conditioning by balancing the risk of failure and the costs of maintenance and air conditioning. If survival models for static covariates were applied to nonstationary data, then the static survival models would fail to estimate the underlying dependence of the hazard function on covariates, resulting in unreliable decision-making. Therefore, survival models that accommodate time-varying covariates are needed. This paper assesses survival data with time-varying covariates to estimate a hazard function that takes covariates into account.
The standard survival model is the Cox proportional hazards model [5, 6, 40] (CoxPH), which forms the logarithm of the hazard function as a linear combination of covariates. The original CoxPH assumes that covariates are time-invariant and have a constant log-linear effect over time on the hazard function. It was extended to address time-varying covariates through the technique of counting process formulation [1]. Although the extended CoxPH is very efficient to compute, it fails to address the nonlinear dependence of survival times on covariates, which is often the case in real-world data. To rectify this limitation, two non-parametric machine learning models have been proposed: generalized boosted models [16] and random forest-based models [47, 48], both of which adopt the extended CoxPH as a learner/tree, and construct non-parametric predictors in an ensemble manner. They succeeded in addressing the nonlinear dependence of survival times on time-varying covariates, but the large number of trees needed for accurate predictions tend to make the algorithm too slow and ineffective for real-time applications.
In this paper, we propose a novel Bayesian survival model to estimate a hazard function as a nonlinear regression function of covariates from survival data with time-varying covariates. To construct a scalable algorithm, we utilize **permanental process**, a doubly-stochastic point process which assumes the square root of hazard function to be generated from a Gaussian process; its computational advantages have been highlighted by recent studies [12, 18, 24, 30, 45]. We tailor it for survival data in the counting process format, and show that the tailored process, which we call the **survival permanent process**, holds the representer theorem [44]: the maximum a posteriori estimator of the latent function can be represented as a finite linear combination of a transformed kernel product evaluated on the event time points. The representer theorem offers us a feasible Bayesian estimation algorithm that scales linearly with the number of events observed without relying on Markov Chain Monte Carlo computation. Furthermore, we derive the predictive distribution and the marginal likelihood in one feasible form, enabling us to implement the uncertainty evaluation and the hyper-parameter optimization in a Bayesian manner. To the best of our knowledge, it is the first time to exploit the representer theorem for survival data analysis. We provide python codes to reproduce the results in this paper<sup>1</sup>.
In Section 2, we introduce the survival permanent process (SurvPP) and construct a scalable Bayesian estimation algorithm for it. In Section 3, we outline related work. In Section 4, we compare SurvPP with reference survival models on synthetic and real-world data, and confirm that our algorithm achieves comparable predictive accuracy while being tens to hundreds of times faster than state-of-the-art survival models. Finally, Section 5 states our conclusions.
2.1 Survival Permanental Processes
We assume that a right-censored dataset with time-varying covariates,  $\mathcal{D} = \{(\pmb{y}_u(t), T_u, \eta_u)\}_{u=1}^U$ , is observed, where  $\pmb{y}_u(t) : (0, T_u] \to \mathcal{V} \subset \mathbb{R}^d$ ,  $T_u \in \mathbb{R}$ , and  $\eta_u \in \{0, 1\}$  are the map of  $d$ -dimensional covariate, the end time of observation, and the indicator that represents whether an individual experienced an event ( $\eta_u = 1$ ) or was right-censored ( $\eta_u = 0$ ) at  $T_u$ , for individual  $u \in \{1, 2, \dots, U\}$ , respectively. We consider tailoring a permanent process for the survival data, where latent function,  $x(\pmb{y}) : \mathcal{V} \to \mathbb{R}$ , is generated from a Gaussian process (GP) on covariate space  $\mathcal{V}$ , and an event time for each individual  $u$  is generated from a point process with hazard function,  $\lambda_u(t)$ ,
$^{1}$ Code and data are provided at https://github.com/HidKim/SurvPP.
$$
p (x (\boldsymbol {y}) | \mathcal {D}) = \frac {p (\mathcal {D} | x (\boldsymbol {y})) \mathcal {G P} (x (\boldsymbol {y}) | k)}{\int \mathcal {D} x (\boldsymbol {y}) p (\mathcal {D} | x (\boldsymbol {y})) \mathcal {G P} (x (\boldsymbol {y}) | k)}, \tag {1}
$$
$$
\log p (\mathcal {D} | x (\boldsymbol {y})) = \sum_ {u = 1} ^ {U} \left[ \eta_ {u} \log \lambda_ {u} \left(T _ {u}\right) - \int_ {0} ^ {T _ {u}} \lambda_ {u} (t) d t \right], \quad \lambda_ {u} (t) = x ^ {2} \left(\boldsymbol {y} _ {u} (t)\right), \tag {2}
$$
where  $\mathcal{GP}(x(\pmb{y})|k)$  represents a GP with kernel function  $k(\pmb{y},\pmb{y}^{\prime}), p(\mathcal{D}|x(\pmb{y}))$  is the likelihood function of the point process, while  $\int \mathcal{D}x(\pmb{y})$  in the denominator represents the integral over the function or the infinite-dimensional variable  $x(\pmb{y})$ . We call the model defined by (1-2) the survival permanent process (SurvPP). Hazard function  $\lambda_{u}(t)$  represents an instantaneous probability of events occurring at each point in time, and our goal is to estimate the functional form of the hazard function over covariate domain,  $x^{2}(\pmb{y})$ , from the survival data  $\mathcal{D}$ .
Given a hazard function over covariate domain,  $x^{2}(\pmb{y})$ , and a covariate map for individual  $u$ ,  $\pmb{y}_u(t)$ , we can evaluate the survival function and the probability density distribution of event time at arbitrary time point  $t$ , denoted by  $S(t)$  and  $P_e(t)$ , respectively, as follows:
$$
S (t) = \exp \left[ - \int_ {0} ^ {t} x ^ {2} \left(\boldsymbol {y} _ {u} (s)\right) d s \right], \quad P _ {e} (t) = x ^ {2} \left(\boldsymbol {y} _ {u} (t)\right) \exp \left[ - \int_ {0} ^ {t} x ^ {2} \left(\boldsymbol {y} _ {u} (s)\right) d s \right]. \tag {3}
$$
By using (3), we can perform survival analyses that include the analysis of the expected duration of event time, and the assessment of the degree of risk that an individual will experience the event before a specified period.
2.2 Counting Process Format of Data
Traditionally, survival data with time-varying covariates have taken the counting process format [1]. The format assumes that, for each individual  $u$ , covariates are measured at  $J_{u}$  finite representative time points and can be regarded as constant between successive time points,
$$
\boldsymbol {y} _ {u} (t) = \boldsymbol {y} _ {u} ^ {j}, \quad t \in \left(s _ {u} ^ {j}, s _ {u} ^ {j + 1} \right], \quad j = 0, \dots , J _ {u} - 1, \tag {4}
$$
where  $(s_u^0,s_u^{J_u}) = (0,T_u)$ . It then splits individual  $u$ 's record,  $(\pmb {y}_u(t),T_u,\eta_u)$ , into  $J_{u}$  pseudo-individual records as in
$$
\left\{\left(s _ {u} ^ {j}, s _ {u} ^ {j + 1}, \xi_ {u} ^ {j}, \boldsymbol {y} _ {u} ^ {j}\right) \right\} _ {j = 0} ^ {J _ {u} - 1}, \quad \xi_ {u} ^ {j} = \eta_ {u} \cdot \mathrm {I} (j = J _ {u} - 1), \tag {5}
$$
where  $\mathrm{I}(\cdot)$  represents the indicator. Joining the pseudo-individual records of  $U$  individuals together results in the counting process format of data:
$$
\mathcal {D} = \left\{\left(T _ {j} ^ {0}, T _ {j} ^ {1}, \xi_ {j}, \boldsymbol {y} _ {j}\right) \right\} _ {j = 1} ^ {J}, \quad J = \sum_ {u = 1} ^ {U} J _ {u}. \tag {6}
$$
In this paper, we employ the counting process format of (6), and rewrite the likelihood function of  $\mathrm{SurvPP}(2)$  as
$$
\log p (\mathcal {D} | x (\boldsymbol {y})) = \sum_ {j = 1} ^ {J} \left[ \xi_ {j} \log x ^ {2} (\boldsymbol {y} _ {j}) - \left(T _ {j} ^ {1} - T _ {j} ^ {0}\right) x ^ {2} (\boldsymbol {y} _ {j}) \right] = \sum_ {n = 1} ^ {N} \log x ^ {2} \left(\tilde {\boldsymbol {y}} _ {n}\right) - \sum_ {j = 1} ^ {J} \Delta_ {j} x ^ {2} \left(\boldsymbol {y} _ {j}\right) \tag {7}
$$
where  $\Delta_j$  and  $\{\tilde{y}_n\}_{n=1}^N$  represent the durations of observation and the covariates observed at event times, respectively:
$$
\Delta_ {j} = T _ {j} ^ {1} - T _ {j} ^ {0}, \quad \left\{\tilde {\boldsymbol {y}} _ {n} \right\} _ {n = 1} ^ {N} = \left\{\boldsymbol {y} _ {j} \mid \eta_ {j} = 1, 1 \leq j \leq J \right\}, \quad N = \sum_ {j = 1} ^ {J} \xi_ {j} = \sum_ {u = 1} ^ {U} \eta_ {u}. \tag {8}
$$
Note that the number of events,  $N$ , is usually much smaller than the number of pseudo-individual records:  $N \ll J$ .
In practice, there are two ways of defining the representative time points  $\{s_u^j\}_{j = 0}^{J_u}$ : one is as the points that were observed, which might be sparse over time due to measurement constraints; the other is as the denser points obtained by using interpolation methods (e.g., splines). This paper assumes the latter, and thus various values of  $J_{u}$  were considered in the synthetic data experiment (see Section 4). Although the counting process format of input assumes piecewise stationarity in covariates, this assumption is not so strong because we can adopt the representative time points in any density if necessary.
2.3 Maximum A Posteriori Estimator
We consider the problem of obtaining the maximum a posteriori (MAP) estimator of  $x(\pmb{y})$ , denoted by  $\hat{x}(\pmb{y})$ , to maximize the posterior probability (1). We derive  $\hat{x}(\pmb{y})$  through the approach of using the path integral representation of GP [23],
$$
\mathcal {G P} (x (\boldsymbol {y}) | k) \mathscr {D} x (\boldsymbol {y}) = \sqrt {\frac {1}{| \mathcal {K} |}} \exp \left[ - \frac {1}{2} \iint_ {\mathcal {Y} \times \mathcal {Y}} k ^ {*} (\boldsymbol {y}, \boldsymbol {y} ^ {\prime}) x (\boldsymbol {y}) x \left(\boldsymbol {y} ^ {\prime}\right) d \boldsymbol {y} d \boldsymbol {y} ^ {\prime} \right] \mathscr {D} x (\boldsymbol {y}), \tag {9}
$$
where  $\mathcal{K}$  and  $\int_{\mathcal{V}} k^{*}(\boldsymbol{y}, \boldsymbol{y}^{\prime}) \cdot d\boldsymbol{y}^{\prime}$  are the integral operator with kernel function  $k(\boldsymbol{y}, \boldsymbol{y}^{\prime})$  and its inverse operator, respectively:  $\mathcal{K}^{(*)}x(\boldsymbol{y}) = \int_{\mathcal{V}} k^{(*)}(\boldsymbol{y}, \boldsymbol{y}^{\prime})x(\boldsymbol{y}^{\prime})d\boldsymbol{y}^{\prime}$ ,  $\mathcal{K}k^{*}(\boldsymbol{y}, \boldsymbol{y}^{\prime}) = \delta(\boldsymbol{y} - \boldsymbol{y}^{\prime})$ . Here  $|\mathcal{K}|$  represents the function determinant [15] of  $\mathcal{K}$ , defined by the product of its eigenvalues [23]. Using the representation of (9), we write the posterior of SurvPP (1-2) in the following functional form,
$$
p (x (\boldsymbol {y}) | \mathcal {D}) \mathscr {D} x = \frac {1}{p (\mathcal {D})} \exp \left[ - S \left(x (\boldsymbol {y}), \underline {{x}} (\boldsymbol {y})\right) - \frac {1}{2} \log | \mathcal {K} | \right] \mathscr {D} x, \tag {10}
$$
where  $S\big(x(\pmb {y}),\underline{x} (\pmb {y})\big)$  is the action integral, defined by
$$
S \left(x (\boldsymbol {y}), \underline {{x}} (\boldsymbol {y})\right) = \int_ {\mathcal {Y}} \left[ \frac {1}{2} x (\boldsymbol {y}) \underline {{x}} (\boldsymbol {y}) + \sum_ {j = 1} ^ {J} x ^ {2} (\boldsymbol {y}) \Delta_ {j} \delta (\boldsymbol {y} - \boldsymbol {y} _ {j}) - 2 \sum_ {n = 1} ^ {N} \log | x (\boldsymbol {y}) | \delta (\boldsymbol {y} - \tilde {\boldsymbol {y}} _ {n}) \right] d \boldsymbol {y}, \tag {11}
$$
and  $\underline{x} (\pmb {y}) = \int_{\mathcal{Y}}k^{*}(\pmb {y},\pmb{y}^{\prime})x(\pmb{y}^{\prime})d\pmb{y}^{\prime}$ . Then we apply calculus of variations to the action integral, where the functional derivative of  $S(x(\pmb {y}),\underline{x} (\pmb {y}))$  on the MAP estimator  $\hat{x} (\pmb {y})$  should be equal to zero:  $\frac{\delta S}{\delta\hat{x}(\pmb{y})}\delta \hat{x} (\pmb {y}) + \frac{\delta S}{\delta\hat{\underline{x}}(\pmb{y})}\delta \hat{\underline{x}} (\pmb {y}) = 0$ , which results in the exact MAP estimator,
$$
\hat {x} (\boldsymbol {y}) = 2 \sum_ {n = 1} ^ {N} h (\boldsymbol {y}, \tilde {\boldsymbol {y}} _ {n}) v _ {n}, \quad v _ {n} = \hat {x} \left(\tilde {\boldsymbol {y}} _ {n}\right) ^ {- 1}, \tag {12}
$$
where  $h(\pmb{y}, \pmb{y}')$  is a transformed kernel function that solves a discretized Fredholm integral equation of the second kind [36],
$$
h \left(\boldsymbol {y}, \boldsymbol {y} ^ {\prime}\right) + 2 \sum_ {j = 1} ^ {J} k \left(\boldsymbol {y}, \boldsymbol {y} _ {j}\right) \Delta_ {j} h \left(\boldsymbol {y} _ {j}, \boldsymbol {y} ^ {\prime}\right) = k \left(\boldsymbol {y}, \boldsymbol {y} ^ {\prime}\right). \tag {13}
$$
See Appendix A for the detailed derivations. Equation (12) shows that the MAP estimator of  $\mathrm{SurvPP}$  involves the representer theorem under the transformed kernel function  $h(\pmb {y},\pmb{y}^{\prime})$  , and thus the Bayesian estimation reduces to a finite-dimensional optimization problem. Here, we call  $h(\pmb {y},\pmb{y}^{\prime})$  the equivalent kernel following studies by Flaxman et al. [12], Walder & Bishop [45], and Kim et al. [24]. Given the equivalent kernel, the unknown coefficients  $v_{n}$  in (12) solve the simultaneous quadratic equations derived from (12),
$$
r _ {n} \triangleq 2 v _ {n} \sum_ {n ^ {\prime} = 1} ^ {N} h \left(\tilde {\mathbf {y}} _ {n}, \tilde {\mathbf {y}} _ {n ^ {\prime}}\right) v _ {n ^ {\prime}} - 1 = 0, \quad n = 1, 2, \dots , N. \tag {14}
$$
In this paper, we estimate a set of coefficients,  $\{v_{n}\}_{n = 1}^{N}$ , by solving a minimization problem of the mean of the squared residuals,  $\sum_{n = 1}^{N}|r_n|^2 /N$ , with a popular gradient descent algorithm, Adam [26].
2.4 Equivalent Kernels
The equivalent kernel  $h(\pmb{y}, \pmb{y}')$  solves the discretized Fredholm integral equation (13). When the kernel function of GP has degenerate form with rank  $M(<\infty)$  such that
$$
k (\boldsymbol {y}, \boldsymbol {y} ^ {\prime}) = \sum_ {m = 1} ^ {M} \phi_ {m} (\boldsymbol {y}) \phi_ {m} \left(\boldsymbol {y} ^ {\prime}\right) = \phi (\boldsymbol {y}) ^ {\top} \phi \left(\boldsymbol {y} ^ {\prime}\right), \tag {15}
$$
it is easily shown that the discretized Fredholm integral equation (13) can be solved analytically [2, 24] as,
$$
h (\boldsymbol {y}, \boldsymbol {y} ^ {\prime}) = \phi (\boldsymbol {y}) ^ {\top} \left(\boldsymbol {I} _ {M} + 2 \boldsymbol {A}\right) ^ {- 1} \phi \left(\boldsymbol {y} ^ {\prime}\right), \quad \boldsymbol {A} = \sum_ {j = 1} ^ {J} \Delta_ {j} \phi \left(\boldsymbol {y} _ {j}\right) \phi \left(\boldsymbol {y} _ {j}\right) ^ {\top}, \tag {16}
$$
where  $\mathbf{I}_M$  is the  $M\times M$  identity matrix, and  $\phi (\pmb {y}) = (\phi_1(\pmb {y}),\phi_2(\pmb {y}),\dots ,\phi_M(\pmb {y}))^\top$ . Note that  $\mathbf{A}$  is defined by a sum of outer products, which can be rewritten through a matrix-matrix multiplication as
$$
\boldsymbol {A} = \boldsymbol {B} \boldsymbol {B} ^ {\top}, \quad \boldsymbol {B} = \left[ \sqrt {\Delta_ {1}} \phi (\boldsymbol {y} _ {1}), \dots , \sqrt {\Delta_ {J}} \phi (\boldsymbol {y} _ {J}) \right]. \tag {17}
$$
Empirically, implementation by matrix-matrix multiplication (17) is substantially faster than the sum of outer products when  $J \gg 1$ .
When  $k(\pmb{y}, \pmb{y}')$  has degenerate form with rank  $M$ , the relation (16) shows that the equivalent kernel  $h(\pmb{y}, \pmb{y}')$  also has degenerate form obtained through Cholesky decomposition:
$$
h (\boldsymbol {y}, \boldsymbol {y} ^ {\prime}) = \left(\boldsymbol {L} \phi (\boldsymbol {y})\right) ^ {\top} \left(\boldsymbol {L} \phi (\boldsymbol {y} ^ {\prime})\right), \quad \boldsymbol {L} ^ {\top} \boldsymbol {L} = \left(\boldsymbol {I} _ {M} + 2 \boldsymbol {A}\right) ^ {- 1}. \tag {18}
$$
The degenerate equivalent kernel (18) offers fast Bayesian estimation that scales linearly with  $N$  (see Section 2.6). In this paper, we used the random feature map [37, 42, 43] of a Gaussian kernel to obtain a degenerate form of kernel  $(M = 500)$ .
If the covariate map,  $\pmb{y}_u(t)$ , can be assumed to be smooth enough over time, a more accurate evaluation of  $\mathbf{A}$  than equation (16) is possible (see Appendix D.1), but this was not exploited in the main experiments.
2.5 Predictive Distribution and Marginal Likelihood
One of the advantages of GP models over non-Bayesian approaches is that they can provide predictive distributions and marginal likelihoods, which enable us to perform uncertainty evaluations, hyper-parameter optimization, and model selection in Bayesian manner. Following the methodology with the path integral representation of GP [23, 24], SurvPP (1-2) adopts a Laplace approximation in the functional space, and finds the approximate form of the predictive distribution and the marginal likelihood. We only show the results due to space limitations. For details, see Appendix B.
The marginal likelihood,  $p(\mathcal{D})$ , is given as
$$
\log p (\mathcal {D}) = \log | \boldsymbol {Z} | - \frac {1}{2} \log | \boldsymbol {I} _ {N} + \boldsymbol {Z} ^ {- 1} \boldsymbol {H} | - \frac {1}{2} \log | \boldsymbol {I} _ {M} + 2 \boldsymbol {A} | ^ {- 1} + (\log 2 - 1) N, \tag {19}
$$
where  $\mathbf{I}_N$  is the identity matrix with size  $N$ , and
$$
\boldsymbol {Z} _ {n n ^ {\prime}} = \left(2 v _ {n} ^ {2}\right) ^ {- 1} \delta_ {n n ^ {\prime}}, \quad \boldsymbol {H} _ {n n ^ {\prime}} = h \left(\tilde {\boldsymbol {y}} _ {n}, \tilde {\boldsymbol {y}} _ {n ^ {\prime}}\right), \quad \boldsymbol {h} (\boldsymbol {y}) = \left(h (\boldsymbol {y}, \tilde {\boldsymbol {y}} _ {1}), \dots , h (\boldsymbol {y}, \tilde {\boldsymbol {y}} _ {N})\right) ^ {\top}. \tag {20}
$$
The predictive distribution of intensity function on a covariate value,  $p(\lambda = x^2(\mathbf{y}))$ , is given as
$$
p _ {\mu , \nu} (\lambda) = \frac {1}{\Gamma (\nu) \mu^ {\nu}} \lambda^ {\nu - 1} \exp (- \lambda / \mu), \quad \mu = 2 \sigma \frac {2 \hat {x} ^ {2} + \sigma}{\hat {x} ^ {2} + \sigma}, \quad \nu = \frac {(\hat {x} ^ {2} + \sigma) ^ {2}}{2 \sigma (2 \hat {x} ^ {2} + \sigma)}, \tag {21}
$$
where  $\hat{x}$  is the abbreviation of  $\hat{x}(\pmb{y})$ , and the predictive variance,  $\sigma$ , is defined by  $\sigma = h(\pmb{y}, \pmb{y}) - \pmb{h}(\pmb{y})^\top (\pmb{Z} + \pmb{H})^{-1} \pmb{h}(\pmb{y})$ . (21) represents the predictive distribution of the hazard function as a regression function of covariates. It is worth noting that the posterior process of  $\operatorname{SurvPP}$  is given by a permanent process, and we can evaluate the distribution of the survival function and perform a risk-aware survival analysis by generating random samples of the estimated hazard function of covariates,  $\lambda(\pmb{y})$ .
2.6 Computational Complexity
The computational complexity of evaluating the equivalent kernel (16) for covariate pair  $(\pmb{y},\pmb{y}^{\prime})$  is  $\mathcal{O}(M^3 +dJM + JM^2)$ , where  $\mathcal{O}(dJM)$  and  $\mathcal{O}(JM^2)$  come from the computation of feature maps,  $\phi_{m}(\pmb{y}_{j})$ , and the sum of outer products (17), respectively.
When the equivalent kernel is given in degenerate form with rank  $M (< N)$  such that  $h(\pmb{y},\pmb{y}^{\prime}) = \sum_{m=1}^{M}\varphi_{m}(\pmb{y})\varphi_{m}(\pmb{y}^{\prime})$ , the objective function to be minimized in MAP estimation,  $\sum_{n=1}^{N}|r_n|^2/N$ ,
incurs a linear computation with the number of observed events  $N$ , that is,  $\mathcal{O}(NM)$ , for each evaluation in gradient descent algorithms: The vector of residual,  $\boldsymbol{r} = (r_1, \dots, r_N)^\top$ , can be expressed as  $(\boldsymbol{R}(\boldsymbol{R}^\top \boldsymbol{v})) \odot (2\boldsymbol{v}) - 1$ , where  $\boldsymbol{R}$  is the  $N \times M$  matrix defined by  $\boldsymbol{R}_{nm} = \varphi_m(\boldsymbol{y}_n)$ , and  $\odot$  represents the Hadamard product.  $\boldsymbol{R}$  costs  $\mathcal{O}(dNM)$ .