Paper: STABLE ESTIMATION OF SURVIVAL CAUSAL EFFECTS (ICLR-2024)

Reviewer comment (Experiments):
The proposed method appears to operate on a population level, yet all steps involve conditioning the hazard function on covariates. I expected to see individual-level results, and it's unclear how well the method performs at the individual level.

Author response:
Population vs Individual Level: Our method addresses the average treatment effect (ATE) rather than individual or heterogeneous treatment effects (HTE).  Conditioning on the covariate is necessary to adjust for confounding when estimating both things.

Reviewer comment (Writing):
It's not evident which distribution the expectations are calculated on in the text.

Author response:
Distribution of Expectations: We realize the lack of clarity regarding the distribution of expectations in the text. All our expectations are calculated concerning the observable distribution O=(X,A,E,T). We'll explicitly state this in the paper for better understanding.

Reviewer comment (Evaluation):
If the odds ratio (OR) is already displaying high bias, why did you introduce the concept of relative RMSE? Why not employ RMSE and compare it to the OR in addition?

Author response:
Discrepancy in Bias vs. Outcome Regression (OR) and Relative RMSE: We display Relative RMSE because RMSE itself changes significantly over time, i.e., when we go from estimating the survival curve at time t=2 to time t=10, so we can’t really see the whole story on one set of axes. The choice to compare everything to OR gives us a reasonable scale to work with.

Reviewer comment (Writing):
The notation might not be fully rigorous. For example, P sometimes refers to probability, and sometimes refers to density.

Author response:
The probability P. We could not find in the manuscript where P has been wrongly attributed to density. All of our notations refer to probability because the space of time T, assignment A and event E are all discrete.

Reviewer comment (Theory):
Theorem 3.5 provides asymptotical linearity of $\hat \psi$ with balancing weights estimated by (19). Is the choice of (19) in some sense optimal? Could the authors comment on the variance of $\hat \psi$? Is it optimal/minimal?

Author response:
Optimality of the variance of $\hat \psi$. The variance of $\hat \psi$ is indeed optimal as we have mentioned after Theorem 3.5. To clarify this further, we will discuss more the connection to the influence function of $\hat \psi$. In essence, our method has similar asymptotic properties to other semi-parametric efficient methods for estimating the ATE because our Riesz representer estimator is a consistent estimator of the inverse propensity weight, as pointed out in section 3.

Reviewer comment (Experiments):
A recently proposed causal survival forest (Estimating heterogeneous treatment effects with right-censored data via causal survival forests, Cui et al. 2023 JRSSB) can also be used to evaluate $\psi^{a,t}$. A comparison in simulations would be helpful.

Author response:
Causal Survival Forest. Thank you for pointing out this relevant paper. We will add the new experimental result.

Reviewer comment (Theory):
By covariate balancing, Hirshberg & Wager (2021) are able to handle high dimensional data, can the proposed method deal with high-dimensionality?

Author response:
Hirshberg & Wager (2021) able to handle high dimensional data. We chose to state our asymptotic results in nonparametric terms for simplicity, but high-dimensional asymptotic results analogous to those in Hirshberg & Wager (2021) follow from the same arguments.

Reviewer comment (Presentation):
Introduction: related work (section 4) should be moved to introduction. I also recommend that authors only cite and discuss literature that is truly relevant/important to their work.

Author response:
2. Introduction and Related Work. We will make the following changes:
Paragraph 3 in the introduction which talks about the advantages and the instability problem of previous debiased machine learning methods, will be streamlined, and will  incorporate paragraph 1 and 2 in related works. 
Paragraph 4 in the introduction which talks about covariate balancing will incorporate paragraph 3 in related works.

Reviewer comment (Writing):
Setting: section 2.3 is really confusing. A lot of notation without explanation. It is more natural to introduce notation only when it needs to be used.

Author response:
3. Section 2.3. We understand your concerns about the notation and its complexity in this section. These notations establish the framework for Section 3. We will move this subsection to section 3 and try to offer more contextual explanations for the readers.

Reviewer comment (Writing):
Approach: it reads like a stack of math techniques. Unimportant derivations or formulas can be omitted. More explanation would be helpful.

Author response:
4. Approach. We appreciate your feedback on this critical section. We firmly believe that the core innovation of our paper lies in the novel mathematical framework presented in Section 3. We will provide more contextual explanations and cut back on the theoretical details.

Reviewer comment (Experiments):
Experiment: the settings should be described explicitly instead of citing other papers.

Author response:
4. Experiments: We will bring the data generating process details from the Appendix to the main text.

Reviewer comment (Theory):
Why do you focus on discrete survival analysis? Can your method be generalized to continuous time?

Author response:
5. Discrete time vs Continuous time SA: We chose to focus on discrete survival analysis due to its relevance in various real-world applications [1]. For example, In healthcare, patients may get checked up every few months so that we can not know exactly when the event happens, therefore discretized time is more robust. While our method primarily targets discrete survival analysis, we acknowledge the significance of continuous-time models. Theory-wise, the distinction between the 2 models, more specifically the fact that the discrete hazard is a conditional mean, while the continuous hazard is not, poses challenges in unifying the theoretical framework. We will tackle continuous SA in future work.

Reviewer comment (Experiments):
Is the asymptotic normality of your estimator examined in your experiments?

Author response:
5. Asymptotic normality:
Let Q be (estimate - estimand) / estimated standard error.
The Kolmogorov-Smirnov statistics (the maximum difference between 2 CDFs) of Q produced by the methods vs. a normal distribution on the Synthetic dataset are: for Outcome Regression (OR): 0.93, for Double robust (DR): 0.25, for Balance: 0.06. We will also include a histogram of Q based on [2], which shows that Balance in fact approximates the normal distribution best.  

[1] Multivariate Survival Analysis and Competing Risk.

[2] https://docs.doubleml.org/stable/guide/basics.html#regularization-bias-in-simple-ml-approaches