Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: The authors propose a clustering approach for time series that encourages instances with similar time profiles to be clustered together. The approach consists of three modules: an encoder, a cluster assigner and a (future outcome) predictor, all specified as neural networks.

The objective of the model is to produce cluster embeddings that are as informative of the outcomes as possible, while not being a direct function of covariates. Note that (2) may look misleading because it indicates that the outcome is a function of the cluster assignment, however, it does not show that the assignment is indeed a function of the covariates.

It is not entirely clear how a patient is assigned to a cluster provided that cluster assignments are a function of time.

It is desirable that performance metrics do not seem affected by the unknown number of clusters, however, this makes for difficult to interpret clusters. More so in practice when the number of identified clusters is a function of the model architecture and hyperparameters (\alpha and \beta). Is the number of clusters selected by cross-validation and if so, what performance metric is used to select the best choice?

In Table 3 for UKCF with 3 comorbidities, how are AUROC and AUPRC evaluated provided these are binary predictions?
review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: This paper proposes a temporal clustering algorithm for the medical domain. The main advantage of the proposed method is that it uses supervised information for temporal clustering. The proposed method is evaluated on two real-world datasets and showed improvements against a few other temporal clustering methods.

Detailed Comments: 

Methodology:
The actor-critic part of the loss is really just a type of policy gradient. There is no estimation of a value function in Eqn. 7 or anywhere to be found in Appendix B. This is very misleading on the part of the authors because policy grad and AC are very different algorithms. AC type of algorithms provide some variance reduction mechanisms for the classical policy gradient (not done here), but they require additional work to estimate a value function for future trajectories that incur bias. If the authors claim a real AC algorithm for the predictive clustering loss, then some justification for bias-variance tradeoffs should be mentioned instead of attributing it to tuning the hyperparameters between the losses.  

Perhaps a bigger issue is that there is a general tradeoff between reconstructing the time-series (the unsupervised learning portion) vs. predictive performance that is common to predictive clustering problems -- it is not addressed here. One can increase the performance in one (in this case prediction accuracy) while sacrificing the other. 

In the extreme case, one can just tune the hyperparameter of the loss function to turn this into a pure supervised learning problem while sacrificing the capacity of the embedding representations to have any meaning (e.g., to reconstruct time-series [SOM-VAE] or predict future ones). The authors did not really propose a systematic way to control this tradeoff, nor did they provide some experiments to show how well the embeddings can be used to recover the temporal patterns. 

Experiments:
The baseline experiments are rather weak. For example, 3 out of the 4 models (DTW, DCN extensions and SOM-VAE) were all unsupervised learning techniques that are not designed to take into account label information. This is especially true for comparison against SOM-VAE (which the authors called state of the art for the proposed problem). SOM-VAE is actually less intended for prognostication than the likes of Baytas et al. (2017) and Madiraju et al. (2018) and provides less informative baselines. The authors should also provide more details regarding how these ``extensions'' are done for things like DCN and K-means clustering on deep neural networks. 
review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: Summary:
This work proposes a method to perform clustering on a time-series data for prediction purposes, unlike the classical clustering where it is done in an unsupervised manner. The authors use an encoder (RNN) to process the time-series medical records, a selector to sample the cluster label for each encoding, and a predictor to predict the labels based on the selected cluster centroids. Since the sampling process prevents the authors from using back-prop, they employ an actor-critic method.

Strengths:
- Although some would argue otherwise, patient similarity has some promise to be useful in clinical practice.
- The proposed method clearly outperformed various baselines in terms of clustering (Table 1).
- Table 3 and Figure 3 show that the proposed method can capture heterogeneous subgroups of the dataset.

Concerns:
- I'm not a clustering expert, but I'm skeptical this is the first work to combine clustering and supervised prediction using an RL technique.
- It is unclear what it means to train the embedding dictionary. Are there trainable parameters in the embedding dictionary? It seems that all it does is calculate the mean of the z_t's (i.e. centroid) in each cluster. Or do you take the centroid embeddings and put that through a feed-forward network of some sort?
- The effect of Embedding Separation Loss (Eq.5) seems quite limited. According to Table 2, it doesn't seem to help much. And contrary to the authors' claim, using \beta increase the number of activated clusters from 8 to 8.4.
- Most importantly, the central theme of this work is combining clustering with prediction labels for the downstream prediction task. But the authors do not compare the prediction performance of the proposed method with other clustering method or "patient similarity" methods, or even simple supervised models. The only prediction performance metric to be found is Table 2 from the ablation study.
review_title: Official Blind Review #2

Meta-review: This paper proposes a reinforcement learning approach to clustering time-series data. The reviewers had several questions related to clarity and concerns related to the novelty of the method, the connection to RL, and experimental results. While the authors were able to address some of these questions and concerns in the rebuttal, the reviewers believe that the paper is not quite ready for publication.
Decision: Reject