Paper:
SOM-VAE: INTERPRETABLE DISCRETE REPRESENTATION LEARNING ON TIME SERIES
High-dimensional time series are common in many domains. Since human cognition is not optimized to work well in high-dimensional spaces, these areas could benefit from interpretable low-dimensional representations. However, most representation learning algorithms for time series data are difficult to interpret. This is due to non-intuitive mappings from data features to salient properties of the representation and non-smoothness over time.
To address this problem, we propose a new representation learning framework building on ideas from interpretable discrete dimensionality reduction and deep generative modeling. This framework allows us to learn discrete representations of time series, which give rise to smooth and interpretable embeddings with superior clustering performance. We introduce a new way to overcome the non-differentiability in discrete representation learning and present a gradient-based version of the traditional self-organizing map algorithm that is more performant than the original. Furthermore, to allow for a probabilistic interpretation of our method, we integrate a Markov model in the representation space. This model uncovers the temporal transition structure, improves clustering performance even further and provides additional explanatory insights as well as a natural representation of uncertainty. We evaluate our model in terms of clustering performance and interpretability on static (Fashion-)MNIST data, a time series of linearly interpolated (Fashion-)MNIST images, a chaotic Lorenz attractor system with two macro states, as well as on a challenging real world medical time series application on the eICU data set. Our learned representations compare favorably with competitor methods and facilitate downstream tasks on the real world data.

Review:
This work addresses the problem of learning latent embeddings of high-dimensional time series data. The paper emphasises the need of interpretable representations accounting for the correlated nature of temporal data. To this scope, the study proposes to cluster the data in a latent space estimated through an auto-encoder. The clustering is obtained by leveraging on the idea of self-organising maps (SOM). Within this setting, the data is mapped into a 2D lattice where each coordinate point represents the center of an inner cluster. 
This construction motivates the formulation of the auto encoder through the definition of several cost terms promoting reconstruction, clustering, and consistency across latent mappings. 
This definition of the problem allows an heuristic for circumventing the non-differentiability of the discrete mapping. The enhance consistency over time, the model is further equipped with an additional cost term enforcing transition smoothness across data points and latent embeddings. 
The experiments are carried out with respect to synthetic 2D time-series, chaotic time-series from dynamical systems, and clinical data. In each case the proposed method shows promising results with respect to the proposed benchmark. 
The study presents some interesting methodological and technical ideas. On the other hand the manuscript presentation is quite convoluted, at the expense of a lacks of clarity in the details about the implementation of the methodology. Moreover, motivated by practical aspects, the model optimisation relies on computational strategies not completely supported from the theoretical point of view (such as the zeroing of the gradient in backpropagation, or the approximation of the clustering function to overcome non-differentiability). The impact of these modeling choices would deserve more investigation and discussion. 
Detailed comments:
- As also stated by the authors, the use of a 2D latent representation is completely arbitrary. It may be true that a 2D embedding provides a simple visualisation, however interpretability can be obtained also with much richer representations in a number of different ways (e.g. sparsity, parametric representations, …). Therefore the feeling is that the proposed structure may be quite ad-hoc, and one may wonder whether the algorithm would still generalise to more complex latent representations.
- Related to the previous comment, the number of latent points seems to be crucial to the performance of the method. However this aspect is not discussed in detail, while it would be beneficial to provide experiment about the sensitivity and accuracy with respect to the choice if this parameters.
- The method relies on several cost terms plugged together. While each of them takes care of specific consistency aspects of the model, their mutual relation and balance may be very critical. This is governed by a series of trade-off parameters whose effect is not discussed  nor explored throughout the study. I guess that the optimisation stability may be also quite sensitive to this trade-off, and it would be important to provide more details about this aspect. 
- Surprisingly, k-means seems to perform quite well in spite of its simplicity. Also, there is no mention about initialisation and choice of the parameter “k”. The authors may want to better discuss the performance of this algorithm, especially compared to its much lower modeling complexity with respect to the proposed method. 
- Still related to the comparison with respect to the state-of-art, interpretability in time series analysis can be achieved with much lesser assumptions and parameters by using standard approaches such as independent component analysis. I would expect this sort of comparison, especially in case of long-term data such as the one provided in the Lorenz system. 
- Clustering of short-term time series, such as the clinical ones, is a challenging task. The feeling is that a highly parametrised model, such as the proposed one,  may still not be superior with respect to classical methods, such as the mixture of linear regressions. This sort of comparison would be quite informative to appreciate the real value of the proposed methodology.