Title: Survival Permanental Processes for Survival Analysis with Time-Varying Covariates
Venue: NeurIPS 2023
Accepted: yes
Keywords: survival analysis

temporal point process

Bayesian estimation

permanental process

representer theorem

kernel method
Abstract: Survival or time-to-event data with time-varying covariates are common in practice, and exploring the non-stationarity in covariates is essential to accurately analyzing the nonlinear dependence of time-to-event outcomes on covariates. Traditional survival analysis methods such as Cox proportional hazards model have been extended to address the time-varying covariates through a counting process formulation, although sophisticated machine learning methods that can accommodate time-varying covariates have been limited. In this paper, we propose a non-parametric Bayesian survival model to analyze the nonlinear dependence of time-to-event outcomes on time-varying covariates. We focus on a computationally feasible Cox process called permanental process, which assumes the square root of hazard function to be generated from a Gaussian process, and tailor it for survival data with time-varying covariates. We verify that the proposed model holds with the representer theorem, a beneficial property for functional analysis, which offers us a fast Bayesian estimation algorithm that scales linearly with the number of observed events without relying on Markov Chain Monte Carlo computation. We evaluate our algorithm on synthetic and real-world data, and show that it achieves comparable predictive accuracy while being tens to hundreds of times faster than state-of-the-art methods.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes the first coherent probabilistic formulation for survival analysis in scenarios where the covariates depend on time, present all practical details for learning MAP estimates and marginal likelihoods and predictive distributions, and provide a range of synthetic and real-data experiments that demonstrate the approach is superior to earlier heuristic solutions and standard survival models. The limitations are adequately addressed.\n\nSTRENGTHS:\nThe paper is overall strong, providing comprehensive solution to an open problem and delivering it in a clear manner with sufficient empirical validation. I cannot immediately think of any specific way the paper could easily be improved, except perhaps by adding a clear motivational example for readers who cannot immediately understand the setting. It is by no means a perfect paper, but nevertheless represents solid scientific work in the field.\n\nThe problem is highly relevant and rarely studied. Even though some previous attempts have been made, we have been missing a proper formulation and I think that this paper provides one in a form that other can build on. The solution proposed here is definitely sound and covered well in the paper. The technical solution is not extremely novel in the sense that the hazard function is simply a transformation of a GP and standard MAP estimation is used, but this is appropriate and perhaps even ideal for a paper that proposes the formal definition of the process. It makes the paper easier to read for a broad audience, compared to one that would attempt to also use novel methodological elements within the process. The paper as written not encourages further work in improved modelling of the hazard functions and other inference techniques, and some ideas are already outlined in the conclusion.\n\nWEAKNESSES:\nThe main empirical result is that we do not really observe an improvement in accuracy over the previous methods, which is naturally a bit of a disappointment. However, the proposed method is faster.\n\nAnother minor weakness is evaluation only on a fairly simple artificial data. A brief demonstration on the value of properly addressing time-varying covariates in some easy-to-understand application task would have made the paper more interesting also for audiences outside the core NeurIPS readers.\n\nQUESTIONS:\n1) What would be the most interesting motivational example of importance of time-varying context that you could describe in 4-5 lines, with detailed examples of what the time-varying covariates could be and what goes wrong if using standard survival models? Could you fit that in your response and then in the Introduction?\n\nLIMITATIONS:\nNone", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis manuscript proposes a Gaussian process based method for survival modeling (or time-to-event modeling) assuming time-varying covariates.  Authors assume covariates are time-varying and they are treated using the counting process format which assumes covariates remain fixed until to the next measurement time point.  The hazard function (or square root of it) is assumed GP prior. Authors rely on previous work on path integral representation of GPs and derive a maximum a posterior estimator of the latent function that serves the hazard function for the likelihood/point process. The method is evaluated on a simulated data set with the log-linear as well as non-linear dependence on covariates.  On simulated data the proposed method performs on par with previous methods but has some beneficial performance in terms of computational efficiency.\n\nSTRENGTHS:\nAuthors study the important question of time-to-event prediction in the case of time-varying covariates. Model formulation that assumes GP prior for the (square of the) hazard function and development of the methodology using path integration representation of GPs in survival modeling appear novel. Quality and clarity of the work appears very good. Problem setting is important.\n\nWEAKNESSES:\nThe development of the maximum a posterior estimate of the latent function as such relies strongly on the previous work (e.g. ref. [19]) and it is unclear what part of the method development is actually novel. The proposed method is tested only on a single simulated data setting with linear and non-linear dependency on time-varying covariates.  On simulated data, performance of the model (in terms of accuracy) is only comparable relative to previous methods.  The proposed method is tested primarily on a very low-dimensional setting (2-dimensions).  Additional experiments presented in Appendix show that the performance of the method in a real data set is slightly weaker than that of the standard CoxPH model.\n\nQUESTIONS:\nAuthors note in Conclusions section (and in Appendix) that the kernel hyper-parameters are optimised using the grid-search. Is the proposed method sensitive to kernel parameter values?  Is it due to the grid search that you need to assume the same length scales for all input features, and is that limiting the performance?\n\nLIMITATIONS:\nI believe authors have adequately described limitations.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a non-parametric Bayesian survival model called Survival Permanental Process (SurvPP) for analyzing survival data with time-varying covariates. The model is based on a permanental process that defines the latent hazard function on covariate space and can handle right-censored observations in a counting process format. The paper shows that SurvPP holds a representer theorem, which offers a fast Bayesian estimation algorithm that scales linearly with the number of observed events without relying on Markov Chain Monte Carlo computation. The paper evaluates SurvPP on synthetic data and shows that it achieves comparable predictive accuracy while being hundreds of times faster than state-of-the-art methods (specifically, CoxPH and GBMs).\n\nSTRENGTHS:\nThe main strengths of this paper are listed below:\n\n1. The paper introduces a novel non-parametric model, called SurvPP, for estimating hazard functions based on survival data with time-varying covariates. This model addresses the limitation of the Cox proportional hazards model (CoxPH) by allowing for non-linear dependence between survival times and covariates, and also where these covariates can change, after the time when the predictions are made.\n\n2. The paper shows that SurvPP holds a representer theorem, which means it can be trained efficiently using a fast Bayesian estimation algorithm that scales linearly with the number of observed events without relying on Markov Chain Monte Carlo computation.\n\n3. The authors have conducted experiments on two synthetic datasets to demonstrate that their proposed SurvPP outperforms CoxPH and another non-parametric model called Generalized Boosted Regression Models (GBM) in terms of predictive performance. SurvPP achieves significantly better performance than CoxPH, but the performance gaps between SurvPP and GBM are marginal and not significant.\n\n4. The paper shows that SurvPP is computationally efficient, particularly when covariates are measured frequently. SurvPP can be trained faster than GBM and CoxPH in estimating hazard functions when covariates are measured frequently.\n\nWEAKNESSES:\n(See next entry: Questions )\n\nQUESTIONS:\nAlthough this paper is well written and technically sound, the reviewer believes that the paper has very limited practical utility. Below are the points that seem serious limitations, which the authors need to address:\n\n**1:** While the paper presents a novel approach to survival analysis using permanental processes and provides a comparison with conventional models (Cox-PH and GBM), the experimental validation appears to be limited to synthetic datasets with only two-dimensional covariates. The reviewer would like to see how the proposed model performs on real-world datasets, which often have higher dimensionality and more complex structures. Specifically, it would be beneficial to test the model on widely recognized and publicly available datasets such as the MIMIC III (Medical Information Mart for Intensive Care III) or the U.K. Cystic Fibrosis Registry. These datasets contain a rich variety of covariates (that do vary over time) and present real-world challenges that synthetic datasets may not fully capture. Using such datasets would not only provide a more rigorous test of the model's performance but also make the results more relatable to practitioners in the field. It would also allow for a more direct comparison with other models in the literature that have been tested on these datasets.\n\nThe authors should explain why they chose to use these synthetic datasets for their experiments, by motivating the particular covariate functions they used–Equations 26, 27, etc.--explaining why they match a realistic situation. Additionally, they should extend the experimental validation to include real-world datasets, to further demonstrate the applicability and robustness of the proposed model.  This would address some other concerns–eg, how to determine the appropriate kernel to use, etc.\n\n**2:** Additionally, the scalability of the proposed model in the context of high-dimensional, low-sample size problems remains an area that warrants further exploration. While we see the authors have acknowledged this limitation (Section 5, lines 308 onwards), the current treatment of this issue does not fully address the potential implications for the practical application of the proposed model. In real-world scenarios, datasets often exhibit high dimensionality, and it is common to encounter situations where the sample size is relatively small. These conditions pose significant challenges for many machine learning models, and it is crucial to understand how the proposed model would perform under such circumstances.The Bayesian estimation framework proposed in this paper adds another layer of complexity to this issue.. Thus, a more detailed discussion of how the proposed Bayesian estimation framework would scale with data dimensionality would be highly beneficial. Therefore, the authors are advised to expand their discussion on this topic. Specifically, it would be valuable to include theoretical considerations, potential strategies for handling high-dimensional data, and, if possible, empirical results from preliminary experiments with (maybe synthetic) high-dimensional, low-sample size datasets.\n\nIn addition to a discussion about the sample efficiency, the paper should also discuss the learner’s computational efficiency.\n\n**3:** The performance of the proposed model is compared with two benchmark methods, namely the Cox proportional hazards model (CoxPH) and the generalized boosted regression model (GBM). While these comparisons provide valuable insights into the model's performance, the choice of benchmark methods raises some questions. Specifically, the CoxPH and GBM models, although well-established in the field, do not represent the most recent advancements in survival analysis. It would be interesting to see how the proposed model compares with more recent methods that leverage advanced machine learning techniques. Specifically, the authors should compare their approach with the following methods:\n\n*Nagpal, Chirag, Vincent Jeanselme, and Artur Dubrawski. \"Deep parametric time-to-event regression with time-varying covariates.\" Survival Prediction-Algorithms, Challenges and Applications. PMLR, 2021.*\n\n*C. Lee, J. Yoon and M. v. d. Schaar, \"Dynamic-DeepHit: A Deep Learning Approach for Dynamic Survival Analysis With Competing Risks Based on Longitudinal Data,\" in IEEE Transactions on Biomedical Engineering, vol. 67, no. 1, pp. 122-133, Jan. 2020, doi: 10.1109/TBME.2019.2909027.*\n\n*Tomašev, Nenad, et al. \"Use of deep learning to develop continuous-risk models for adverse event prediction from electronic health records.\" Nature Protocols 16.6 (2021): 2765-2787.*\n\nComparing the proposed model with these more recent methods would provide a more comprehensive evaluation of its performance and relevance in the current research landscape. It would also allow for a more direct comparison with state-of-the-art models, enhancing the paper's contribution to the field.\n\n**4:** The authors have presented their experimental results in a cross-validation setting for synthetic datasets. While cross-validation is a widely accepted method for model evaluation, particularly when dealing with limited data, it is not immediately clear why this approach was chosen for synthetic data, where the volume of data can be controlled. Synthetic data offers the unique advantage of being able to generate as much data as needed, which could potentially allow for the creation of independent validation sets. This approach would provide a more stringent test of the model's generalization ability, as it would involve testing on data that the model has not seen during training, even indirectly through cross-validation folds.\n\nCould the authors provide some insights into their decision to use cross-validation for synthetic data? Furthermore, would it be possible to conduct additional experiments using independent validation sets with synthetic data to further assess the model performance?\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses a survival analysis problem with time-varying covariates by proposing a novel Bayesian approach to estimate a hazard function as a nonlinear function of covariates using the kernel trick. MAP, predictive distribution, and computational complexity of the proposed approach were presented. Experiments were done on both synthetic and real-world benchmark datasets.\n\nSTRENGTHS:\n- Well-written and easy to follow. The relationship with the related work is clearly presented.\n\n- Proposed a novel Bayesian approach to tackle the survival analysis with time-varying covariates that allows modeling of a nonlinear relationship between hazard and covariates. The authors presented MAP, predictive distribution, and computational complexity of the proposed approach.\n\n- Experimental results clearly support that SurvPP provides good predictive performance as well as is computationally efficient.\n\nWEAKNESSES:\n- Just as other GP or kernel-based methods, SurvPP might not be ideal for complex high-dimensional data. But the authors have pointed this out, and a potential way to handle this has been clearly discussed.\n\n- In the experimental results on the real-world benchmark dataset, SurvPP showed slightly worse AUC than the other models, unlike in synthetic data. Could the authors kindly give some explanation why? Interestingly, in terms of TLL, SurvPP is doing quite well.\n\nQUESTIONS:\n- The authors might not have enough space to put the experimental results on the real-world benchmark dataset. Would the authors plan to move it to the manuscript?\n\n- Is there any specific reason that the authors used AUC rather than C-index, which is more commonly used in the survival analysis?\n\n- It is interesting that SurvPP, which is a non-parametric model, showed as good TLL as CoxPH, which is a (semi) parametric model, unlike the other nonparametric tree-based approaches. Could authors give some explanation why this happened? Is this something expected?\n\n- I am also curious about what each method's survival curve looks like. Might be interesting if the authors could compare the average prediction curve against the KM survival curve.\n\n- As the proposed method is a Bayesian approach, it can provide more than just better prediction performance. Maybe the authors could add some discussion on that.\n\nLIMITATIONS:\nDid not find any other limitations than the authors discussed in the last section.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a new hazard function in survival analysis. The hazard is a squared Gaussian process, where the index set is the covariates at a given time. They assume that the covariates are piecewise constant, propose a MAP estimator and derive the predictive distribution and marginal likelihood. They use ideas from kernel methods and achieve computational complexities that are lower than alternate approaches, scaling linearly with the number of observed events.\n\nSTRENGTHS:\nThe idea of connecting this to kernel methods to reduce computational complexity is quite clever, and the speedups over GBM and RFM are impressive.\n\nWEAKNESSES:\nThe piecewise constant time-varying covariate assumption is very strong. You should mention this.\n\nThere also needs to be some discussion of exogeneous vs endogeneous time-varying covariates. It looks like you're implicitly assuming that your covariates are exogeneous, but you should make that clear.\n\nYou need a real data experiment in the main paper. In the introduction, you mention 'confirm that our algorithm achieves comparable pre67 dictive accuracy while being hundreds of times faster than state-of-the-art survival models' but saying nothing about real data. You have something in the supplement, but again, there's no reference to how good the results are in the main paper. Given that, one has to give less weight to the supplement.\n\nIn your benchmarks, I assume that all of them are assuming piecewise constant time-varying covariates. You should include something using interpolation: e.g. linear mixed models with splines. They do this a lot in the joint modeling literature.\n\nQUESTIONS:\nAre your benchmarks also assuming piecewise constant time-varying covariates?\n\nLIMITATIONS:\nWhile the limitation they mention are important, some of the limitations that I mentioned in the weaknesses are not addressed.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-08"}