Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: In this paper, the authors present a framework for heterogeneous treatment effect estimation for time-to-event outcomes. In recent times, both areas have been explored independently and this paper proposes an approach for the censored time-to-event setting. SurvITE (individual treatment effect) and SurvHTE (heterogeneous treatment effect). Authors present experimental results for the same comparing against ablations and survival analysis baselines.
main_review: **originality** - the proposed problem setup brings together significant body of work in both areas (survival analysis and heterogeneous/individual treatment effect estimation) together. The proposed formulation in Eq (6) uses ideas similar to those proposed in 8,9,10 and 32 from the references.

**quality and clarity** I do believe the draft appears to be technically sound and well presented in general. My only concern is with the way the IPM is applied in this paper. While authors do claim that this help obtaining shift-invariant representations, it is not directly clear for me on how penalizing IPM for each (a,$\tau$) combination w.r.t the baseline is more better than doing it across the treatments explicitly. It might help to improve the writeup here, as I do think this is critical for the paper.  

In addition, I do think the experimental analysis section does appear incomplete.  I also don't completely agree with authors claim on line 44 that reference 27 is doing plain regression on targets. I think a more comprehensive discussion is needed w.r.t this paper in the experiments also if possible.

**significance** - Making a contribution in this space of HTE for time-to-event survival outcomes is very much needed. 

**Post author feedback**

I have gone through the author response and I have changed my score for this paper. I do believe clarifying these concerns will improve the paper quality.
limitations_and_societal_impact: Yes
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: In this work the authors quickly remind the reader of the pitfalls one encounter in the survival analysis framework, in order to motivate a new loss and corresponding deep model for modelling of the group-specific hazard functions.
The main contribution of this paper resides in Proposition 1: an upper bound of the population risk (unknown) in terms of quantities that can be estimated from the data. From this proposition the authors propose to minimize the upper-bound and then provide an example to motivate their intuition.
main_review: I find this paper interesting and fairly well written (given the constraints of the NeurIPS format). The bound in Proposition 1 is not only interesting from a purely theoretical point of view but also enables practical methods, of which the authors give an example. The bound is similar to previous bounds in the literature but does not rely on invertibility, which makes it much more practical as most models in use today are not invertible and when they are (e.g. Normalizing Flows), are computationally expensive.

The authors correctly point out that it's possible to solve the same problem by reweighting the ERM loss. It would have been worthwhile to study the link between their proposed method and propensity methods as it is known that reweighting and optimal transport plans (present here through the Wasserstein metric) are equivalent and since bounds on the excess risk in the reweighted ERM framework under censorship are known (see e.g. [1], or [2, 3]).

[1] G. Ausset, S. Clémençon, and F. Portier, “Empirical Risk Minimization under Random Censorship: Theory and Practice,” 2019, Available: http://arxiv.org/abs/1906.01908
[2] W. Stute, “The Central Limit Theorem Under Random Censorship,” Ann. Statist., vol. 23, no. 2, pp. 422–439, Apr. 1995, doi: 10.1214/aos/1176324528.
[3] T. A. Gerds, J. Beyersmann, L. Starkopf, S. Frank, J. V. D. Laan, and M. Schumacher, “The Kaplan-Meier Integral in the Presence of Covariates: A Review,” From Statistics to Mathematical Finance, pp. 25–42, 2017, doi: 10.1007/978-3-319-50986-0.


I also find the choice of competing methods in the experimental section to be puzzling and in need of better motivation: the authors exclude a wide range of methods (namely the "deep" Cox extensions like [4] or [5]) that are known to often be state of the art, for reasons I do not fully understand. This is especially puzzling since they later use the vanilla Cox model as a baseline.

[4] J. Katzman, U. Shaham, J. Bates, A. Cloninger, T. Jiang, and Y. Kluger, “DeepSurv: Personalized Treatment Recommender System Using A Cox Proportional Hazards Deep Neural Network,” BMC Med Res Methodol, vol. 18, no. 1, p. 24, Dec. 2018, doi: 10.1186/s12874-018-0482-1.
[5] C. Lee, W. Zame, J. Yoon, and M. V. D. Schaar, “DeepHit: A Deep Learning Approach to Survival Analysis With Competing Risks,” 2018.


Despite those shortcomings I think, the paper is still overall a good fit for NeurIPS.
limitations_and_societal_impact: Yes, the authors are aware of the potential limitations of their work in terms of fairness. There is, however, not much they can do if they want to obtain bounds.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper tackles the problem of individual treatment effect prediction in survival analysis situations, which is the case the outcome variable is the time to event. In this situation, there are two additional bias(covariate shift) problems compared with the standard ITE prediction. They showed the generalization bound for this problem, then applied counter factual regression to minimize it through the IPM term.
main_review: I think the ITE prediction for the survival outcome is an interesting topic. Such a situation occurs not only in clinical studies but also in marketing, retail, and some other business situations.

The main contribution of this work is defining the generalization bound for this problem then minimize it through the CFR scheme. I think their approach technically sounds.

It was quite time-consuming for me to understand this paper. Mainly because of the lack of knowledge about the survival analysis. So I appreciate it if you could add some explanation for the problem this paper tries to solve without using the term for the survival analysis.

For minor issues

- line 156 there are two dots before "Here,"
- line 361 RMST -> RMSE?
- Is there any reason to use ITE and HTE alternately?
limitations_and_societal_impact: They adequately addressed the limitation in their paper.
review_title: null

Meta-review: The authors formalize heterogeneous treatment effect estimation from time-to-event data under empirical risk minimization. This is a natural extension of the prior work done for continuous and binary outcomes with the added challenges from to the time-to-event data such as censoring bias. The paper being a straightforward follow the recipe type of paper doesn't detract from its quality. The main thing I'd suggest would be complete the references in the related work for ML methods to include things like:

@inproceedings{avati2020countdown,
  title={Countdown regression: sharp and calibrated survival predictions},
  author={Avati, Anand and Duan, Tony and Zhou, Sharon and Jung, Kenneth and Shah, Nigam H and Ng, Andrew Y},
  booktitle={Uncertainty in Artificial Intelligence},
  pages={145--155},
  year={2020},
  organization={PMLR}
}

and

@inproceedings{ranganath2016deep,
  title={Deep survival analysis},
  author={Ranganath, Rajesh and Perotte, Adler and Elhadad, No{\'e}mie and Blei, David},
  booktitle={Machine Learning for Healthcare Conference},
  pages={101--114},
  year={2016},
  organization={PMLR}
}



Decision: Accept (Poster)