Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: The authors propose an approach to cluster subjects into K clusters according to their underlying, often unknown, life distribution. The loss function to the neural-network-based approach is based on a tight upper bound to the two-sample Kuiper test. Experiments are conducted on artificial and two real life datasets.

I enjoyed reading the paper, the authors propose an approach to address an important yet relatively under-explored problem in survival analysis.

It is not entirely clear how to handle the case when after the model is trained, H^(u) for a new subject u is larger than t_max when the model is trained. In such case, \Chi^(u) will be negative, thus what happens with \beta^(u)(W_2)?

When the termination signals are not available (friendster data) termination signals are set artificially when training and evaluation, thus, is the model learning artificially set termination signals and thus artificial survival functions? what is the point of doing this? Moreover, if the termination signals are artificial, what is the point of calculating C-Index and Integrated Brier score? I understand the motivation to compare to existing methods, it is really not clear how meaningful they are at evaluating performance.

In general, using C-Index and Brier scores in the context of the presented application may be misleading (or non-applicable) because these metrics are usually applied to survival analysis scenarios where one seeks to estimate likelihood of survival or time to event (over a usually infinite time horizon). Here, \beta^(u)(W_2) is a function of \Chi^(u) which is known (also not comparable to survival analysis), artificially tied to a pre-specified time-horizon t_max, and not dependent on covariates as in survival analysis. Further, SSC-Bair and SSC-Gaynor are clustering models, how are survival estimates obtained for these?

In practice, how is K selected? based on the friendster results for K=2,3,4,5 they all seem to produce distinct clusters, so which one should be used? This raises a question: how are the clusters, their members or the number of clusters informing the use case? In the MIMIC III experiment, where events are observed, the model finds two clusters without strikingly different survival functions (relative to friendster). Can one really consider S_2 as the high-risk group? Can one get higher risk clusters with larger K?

Minor:
- A_i^(u) is missing from the definition in the training data.
- In Section 3.2, if \xi^(u) is shared, why the superscript indicating that is subject specific?
review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: This paper proposes a method to cluster subjects based on the latent lifetime distribution. The proposed model clusters by maximizing the empirical divergence between different clusters. 

The problem setting of this paper is not clear to me. I am not sure which variables are observed and which are not.  For example, in the Friendster experiment, the data of 5 months from joining is used for clustering. However, the termination windows are chosen to be 10 months. Therefore, it is clear that the observed data will not contain the termination signals, and I do not believe the training of the model is possible, without observing any termination signals.  In the paper, do we consider only one type or multiple types of events? Is $M_{k, i}$ a vector that represents the attributes or properties of an event?

Some details of the model are not clear to me. In Equation (2), the input of the neural network differs in length across different subject $u$, because the number of observed events for each subject is different. 
How does the proposed neural network take inputs of different lengths?
How the non-decreasing function $\xi^{(u)}$ is defined in Section 3.2? Is it a function of the observed data for each subject?  

How the empirical distribution $\hat{S}_i$ in Equation (4) is computed is also not clear to me. How $\hat{S}_i$ is a vector? Is it constructed by concatenating $\hat{S}_k(W_1, W_2; D)[t] $ with different $t$? How to normalize $\hat{S}_i$ such that it is a valid probabilistic distribution? Since $\hat{S}_i$ is high dimensional, it looks very challenging to estimate the joint distribution.

The overall objective function is given by Equation (4), is it correct? In Equation (4), why should we compute the minimum values across all possible pairs of clusters rather than the summation of all pairs? If Equation (4) is the overall objective function, then it looks like the model does not contain a component that maximizes a likelihood function. How is it guaranteed that the model will fit the data? It looks like the model will converge to a trivial solution that $\beta$ is a constant such that $\beta = 1$ for one cluster and $\beta = 0$ for another cluster, if the likelihood function is not involved. This will give a maximum divergence between distributions. 

In summary, it seems that numerous technical details are missing and the paper might contain technical flaws. I do not suggest the acceptance of this paper.


review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: This paper proposes a deep learning method for clustering subjects (patients, users of social networks) into clusters of different lifetime (survival time) distributions. Inputs for a subject include covariates (age, gender, etc) and an event sequence. A key difficulty in the problem is that the lifetime of a subject is often unobserved. The proposed method is to learn a termination function that is non-decreasing with time, which essentially treats prolonged inactivity as termination in a probabilistic way. Clustering is done in a discriminative manner where the objective function considers cluster sizes besides between-cluster differences in lifetime distribution. (The inclusion of cluster size information in the objective function avoids degenerate clustering results.)
The paper is strong in technical contents. The objective function is well-motivated and placed on solid theoretical foundation. Empirically, the proposed method performed significantly better than baselines.
The weakness of the paper is that the utility of the work has not been demonstrated. For example, the Friendster dataset (1.1 million social network users) is partitioned into 2-5 clusters. However, are there clusters that a service provider can use to improve their service? It is doubtful whether such “useful” clusters can be found using the proposed algorithm. One might need to obtain a fairly large number of clusters, via conditioning perhaps, before finding some really useful ones. 
•	The paper is strong in technical contents. The objective function is well-motivated and placed on solid theoretical foundation.
•	Empirically, the proposed method performed significantly better than baselines. 
•	The utility of the work has not been demonstrated. Potential impact is an outstanding issue here because the proposed method is very special-purpose. 

Question to the authors: It is observed that the numbers of friends and comments are strongly correlated with the clustering. Were those two covariates included in the analysis? If not, why?


review_title: Official Blind Review #1

Meta-review: The authors propose a clustering algorithm for users in a system based on their lifetime distribution. The reviewers acknowledge the novelty of the proposed clustering algorithm, but one concern left unresolved is how the results of the analysis can be of use in the real world examples used. 
Decision: Reject