Venue: ICLR 2018 conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: This paper introduces siamese neural networks to the competing risks framework of Fine and Gray. The authors optimize for the c-index by minimizing a loss function driven by the cumulative risk of competing risk m and correct ordering of comparable pairs. While the idea of optimizing directly for the c-index directly is a good one (with an approximation and with useful complementary loss function terms), the paper leaves something to be desired in quality and clarity.

Related works:
- For your consideration: is multi-task survival analysis effectively a competing risks model, except that these models also estimate risk after the first competing event (i.e. in a competing risks model the rates for other events simply go to 0 or near-zero)? Please discuss. Also, if the claim is that there are not deep learning survival analyses, please see, e.g. Jing and Smola.
- It would be helpful to define t_k explicitly to alleviate determining whether it is the interval time between ordered events or the absolute time since t_0 (it's the latter). Consider calling k a time index instead of t_k a time interval ("subject x experiences cause m occurs [sic] in a time interval t_k")
- Line after eq 8: do you mean accuracy term?
- I would not call Reg a regularization term since it is not shrinking the coefficients. It is a term to minimize a risk not a parameter.
- You claim to adjust for event imbalance and time interval imbalance but this is not mathematically shown nor documented in the experiments.
- The results show only one form of comparison, and the results have confidence intervals that overlap with at least one competing method in all tasks.
review_title: Re: Siamese survival analysis

final_score: rating: 4: Ok but not good enough - rejection
review_content: The authors tackle the problem of estimating risk in a survival analysis setting with competing risks. They propose directly optimizing the time-dependent discrimination index using a siamese survival network. Experiments on several real-world dataset reveal modest gains in comparison with the state of the art.

- The authors should clearly highlight what is their main technical contribution. For example, Eqs. 1-6 appear to be background material since the time-dependent discrimination index is taken from the literature, as the authors point out earlier. However, this is unclear from the writing. 

- One of the main motivations of the authors is to propose a model that is specially design to avoid the nonidentifiability issue in an scenario with competing risks. It is unclear why the authors solution is able to solve such an issue, specially given the modest reported gains in comparison with several competitive baselines. In other words, the authors oversell their own work, specially in comparison with the state of the art.

- The authors use off-the-shelf siamese networks for their settting and thus it is questionable there is any novelty there. The application/setting may be novel, but not the architecture of choice.

- From Eq. 4 to Eq. 5, the authors argue that the denominator does not depend on the model parameters and can be ignored. However, afterwards the objective does combine time-dependent discrimination indices of several competing risks, with different denominator values. This could be problematic if the risks are unbalanced.

- The competitive gain of the authors method in comparison with other competing methods is minor.

- The authors introduce F(t, D | x) as cumulative incidence function (CDF) at the beginning of section 2, however, afterwards they use R^m(t, x), which they define as risk of the subject experiencing event m before t. Is the latter a proxy for the former? How are they related?
review_title: Modest technical contribution, minor gains in performance with competitors

final_score: rating: 4: Ok but not good enough - rejection
review_content: The paper entitled 'Siamese Survival Analysis' reports an application of a deep learning to three cases of competing risk survival analysis. The author follow the reasoning that '... these ideas were not explored in the context of survival analysis', thereby disregarding the significant published literature based on the Concordance Index (CI). 

Besides this deficit, the paper does not present a proper statistical setup (e.g. 'Is censoring assumed to be at random? ...) , and numerical results are only referring to some standard implementations, thereby again neglecting the state-of-the-art solution. That being said, this particular use of deep learning in this context might be novel.
review_title: Brittle application of deep learning to competitive risk analysis.

Meta-review: Reviewers unanimous in assessment that manuscript has merits, but does not satisfy criteria for publication.

Pros:
- Potentially novel application of neural networks to survival analysis with competing risks, where only one terminal event from one risk category may be observed.

Cons:
- Incomplete coverage of other literature.
- Architecture novelty may not be significant.
- Small performance gains (though statistically significant)
Decision: Reject