Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes LatTe Flows, a novel autoregressive flow model with autoencoders to learn latent embeddings from time series. The authors claim that it can model cross-series dependencies across time while scaling to large dimensions with an end-to-end training process.
main_review: **Strengths**
This paper has done good comparative evaluations of the proposed framework against baselines to demonstrate that it significantly outperforms the state-of-the-art on multi-step forecasting benchmarks. 


**Weaknesses**
1. I'm not convinced that the proposed latent variable estimation framework is completely novel. As pointed out by the authors, the core component of the proposed framework is MAF autoregressive flow model, which has been applied for many sequential tasks already.

2. $L_{\text{Negll}}$ in Eq (1) is different from the definition in Section 3.1. One is defined on observed variables y and the other on latent variables x. It'll be better if the authors derive Eq (1) from a pure MLE perspective, instead of combining reconstruction loss with flow loss. I personally feel the loss function is not theoretically justified because I'm not sure what it is optimizing but I may be wrong.

summary_of_the_review: I suggest the rejection of this paper in its current version. Although the authors have done comparisons of their framework with baselines, the proposed model is not totally novel.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The authors propose a novel approach that scales well for multivariate sequence forecasting tasks. They offer to use conditional normalizing flow to capture the sequence dynamics in the latent space and then use auto-regressive architecture to decode the signal in the original space. The authors clearly explain why they prefer latent forecasting and experimentally verify the benefits of the proposed method on both widely used public datasets and the AH&MS dataset.

main_review: Though the idea is very interesting, and the experimental results are also very encouraging, I have below major/minor concerns on this paper:

-- Major 1:
The idea of modeling the latent representations with normalizing flow is not new. Even sequential normalizing flow has been proposed recently (see [1] and [2]). It is not clear to me what is the major novelty in terms of methodology.

-- Major 2:
Some descriptions are not clear to me. For example, in section 3.2, p_{theta}(y|h) and p_{theta}(x|h) should be distinguished. According to Figure 1, it seems that h_t would affect x_t, but not the whole flow process. Are all the revertible transformations between the low-dimensional sample z_t and x_t included in the parameter set theta? Also, how your “NegLL” term affects the normalizing flow is not illustrated in the draft.

--Minor concerns:
1. The model does not improve in every scenario, but the overall performance does look good.
2. Table two is referred to twice in the draft, but table one is not referred mentioned in the main text. I guess one reference should be corresponding to table one?
3. In Table two, what are the latent dimensions of all the other methods? Did the authors also tune on different approaches?
4. How does parameter lambda affect the learned representation and the reconstruction?
5. The authors do not compare to TLAE, though you claimed this is (one of) the most related work.
6. The authors mention that TLAE is using diagonal Gaussian prior, which is restrictive. According to Figures one and two, it seems the proposed method is also using a restrictive prior?
7. It would be good to provide additional experiments to show that your methods scale well rather than on performance. 
8. There is a typo in line 9 of the page 4.

[1]: https://arxiv.org/pdf/2010.03172.pdf
[2]: https://arxiv.org/pdf/1901.10548.pdf

summary_of_the_review: Right now, I am more inclined not to accept the paper. However, I am happy to discuss and also change my evaluation if I missed anything important.
review_title: null

final_score: rating: 1: strong reject
review_content: summary_of_the_paper: In essense, the proposed method leans some lower-dimensional representation of the frames pertaining to observed sequences, and then postulates an RNN-type of model in the leaned latent space to model temporal dynamics. The whole pipeline is trained end-to-end.
main_review: I have to admit that the paper is a boring read. The combination of some rudimentary ideas such as learning some framewise latent space representations (ideally via some flow model) and then using some sort of RNN to capture temporal dynamics within is far from seemingly  novel, let alone interesting. 

The experimental results are unimpressive; performance differences are marginal, comparisons are not against the SOTA in the field, and the considered datasets are far from honestly high-dimensional; I would like to see tens of thousands of dimensions to be convinced.  
summary_of_the_review: The method is trivial. The experimental results do not support publication of an methodologically boring paper.
review_title: null

Meta-review: This paper proposes a new autoregressive flow model with autoencoders to learn latent embeddings from time series. The authors conducted extensive comparative experiments, and the experimental results are very encouraging. However, the proposed method, as a combination of the encoder/decoder structure and autoregressive flows on the latent space, does not seem novel enough.
Decision: Reject