Venue: ICLR 2018 conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: This work proposes an LSTM based model for time-evolving probability densities. The model does not assume an explicit prior over the underlying dynamical systems, instead only uncertainty over observation noise is explicitly considered. Experiments results are good for given synthetic scenarios but less convincing for real data.  

Clarity: The paper is well-written. Some notations in the LSTM section could be better explained for readers who are unfamiliar with LSTMs. Otherwise, the paper is well-structured and easy to follow.

Originality: I'm not familiar with LSTMs, it is hard for me to judge the originality here.

Significance: Average. The work would be stronger if the authors can extend this to higher dimensional time series. There are also many papers on this topic using Gaussian process state-space (GP-SSM) models where an explicit prior is assumed over the underlying dynamical systems. The authors might want to comment on the relative merits between GP-SSMs and DE-RNNs.

The SMC algorithm used is a sequential-importance-sampling (SIS) method. I think it's correct but may not scale well with dimensions.
review_title: Review

final_score: rating: 5: Marginally below acceptance threshold
review_content: The papers proposes a recurrent neural network-based model to learn the temporal evolution of a probability density function. A Monte Carlo method is suggested for approximating the high dimensional integration required for multi-step-ahead prediction.

The approach is tested on two artificially generated datasets and on two real-world datasets, and compared with standard approaches such as the autoregressive model, the Kalman filter, and a regression LSTM.

The paper is quite dense and quite difficult to follow, also due to the complex notation used by the authors.

The comparison with other methods is very week, the authors compare their approach with two very simple alternatives, namely a first-order autoregressive mode and the Kalman filter.  More sophisticated should have been employed.
review_title: Review

final_score: rating: 6: Marginally above acceptance threshold
review_content: Interesting ideas that extend LSTM to produce probabilistic forecasts for univariate time series, experiments are okay. Unclear if this would work at all in higher-dimensional time series. It is also unclear to me what are the sources of the uncertainties captured.


The author proposed to incorporate 2 different discretisation techniques into LSTM, in order to produce probabilistic forecasts of univariate time series. The proposed approach deviates from the Bayesian framework where there are well-defined priors on the model, and the parameter uncertainties are subsequently updated to incorporate information from the observed data, and propagated to the forecasts. Instead, the conditional density p(y_t|y_{1:t-1|, \theta}) was discretised by 1 of the 2 proposed schemes and parameterised by a LSTM. The LSTM was trained using discretised data and cross-entropy loss with regularisations to account for ordering of the discretised labels. Therefore, the uncertainties produced by the model appear to be a black-box. It is probably unlikely that the discretisation method can be generalised to high-dimensional setting?

Quality: The experiments with synthetic data sufficiently showed that the model can produce good forecasts and predictive standard deviations that agree with the ground truth. In the experiments with real data, it's unclear how good the uncertainties produced by the model are. It may be useful to compare to the uncertainty produced by a GP with suitable kernels. In Fig 6c, the 95pct CI looks more or less constant over time. Is there an explanation for that?

Clarity: The paper is well-written. The presentations of the ideas are pretty clear.

Originality: Above average. I think the regularisation techniques proposed to preserve the ordering of the discretised class label are quite clever.

Significance: Average. It would be excellent if the authors can extend this to higher dimensional time series.

I'm unsure about the correctness of Algorithm 1 as I don't have knowledge in SMC.
review_title: Interesting ideas that extend LSTM to produce probabilistic forecasts for univariate time series, experiments are okay. Unclear if this would work at all in higher-dimensional time series. It is also unclear to me what are the sources of the uncertainties captured.

Meta-review: Thank you for submitting you paper to ICLR. Two of the reviewers are concerned that the paper’s contributions are not significant enough —either in terms of the theoretical or experimental contribution -- to warrant publication. The authors have improved the experimental aspect to include a more comprehensive comparison, but this has not moved the reviewers. 
Decision: Reject