Because the authors did not respond to reviewer feedback, I am maintaining my original review score.  -----  This paper proposes to model relational (i.e., correlated) time series using a deep learning-inspired latent variable approach: they design a flexible parametric (but not generative) model with Gaussian latent factors and fit it using a rich training objective including terms for reconstruction (of observed time series) error, smoothness in the latent state space (via a KL divergence term encouraging neighbor states to be similarly distributed), and a final regularizer that encourages related time series to have similar latent state trajectories. Relations between trajectories are hard coded based on pre-existing knowledge, i.e., latent state trajectories for neighboring (wind speed) base stations should be similar. The model appears to be fit using gradient simple descent. The authors propose several elaborations, including a nonlinear transition function (based on an MLP) and a reconstruction error term that takes variance into account. However, the model is restricted to using a linear decoder. Experimental results are positive but not convincing.  Strengths: - The authors target a worthwhile and challenging problem: incorporating the modeling of uncertainty over hidden states with the power of flexible neural net-like models. - The idea of representing relationships between hidden states using KL divergence between their (distributions over) corresponding hidden states is clever. Combined with the Gaussian distribution over hidden states, the resulting regularization term is simple and differentiable. - This general approach -- focusing on writing down the problem as a neural network-like loss function -- seems robust and flexible and could be combined with other approaches, including variants of variational autoencoders.  Weaknesses: - The presentation is a muddled, especially the model definition in Sec. 3.3. The authors introduce four variants of their model with different combinations of decoder (with and without variance term) and linear vs. MLP transition function. It appears that the 2,2 variant is generally better but not on all metrics and often by small margins. This makes drawing a solid conclusions difficult: what each component of the loss contributes, whether and how the nonlinear transition function helps and how much, how in practice the model should be applied, etc. I would suggest two improvements to the manuscript: (1) focus on the main 2,2 variant in Sec. 3.3 (with the hypothesis that it should perform best) and make the simpler variants additional "baselines" described in a paragraph in Sec. 4.1; (2) perform more thorough experiments with larger data sets to make a stronger case for the superiority of this approach. - The authors only allude to learning (with references to gradient descent and ADAM during model description) in this framework. Inference gets its one subsection but only one sentence that ends in an ellipsis (?). - It's unclear what is the purpose of introducing the inequality in Eq. 9. - Experimental results are not convincing: given the size of the data, the differences vs. the RNN and KF baselines is probably not significant, and these aren't particularly strong baselines (especially if it is in fact an RNN and not an LSTM or GRU). - The position of this paper is unclear with respect to variational autoencoders and related models. Recurrent variants of VAEs (e.g., Krishnan, et al., 2015) seem to achieve most of the same goals as far as uncertainty modeling is concerned. It seems like those could easily be extended to model relationships between time series using the simple regularization strategy used here. Same goes for Johnson, et al., 2016 (mentioned in separate question).  This is a valuable research direction with some intriguing ideas and interesting preliminary results. I would suggest that the authors restructure this manuscript a bit, striving for clarity of model description similar to the papers cited above and providing greater detail about learning and inference. They also need to perform more thorough experiments and present results that tell a clear story about the strengths and weaknesses of this approach.
Output: en