Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper presents an approach for learning latent season-trend based representations for time-series forecasting, based on variational inference. A sesonal encoder and a trend encoder are used to obtain latent representations. The latent representations are then combined together to produce a forecast. The forecast is done via the predictor module, which makes use of Fourier Transform for the seaosnal representation and a simple MLP for a the trend representation, and then sums them up to obtain the forecast. Reconstruction error cannot be minimized directly, as the true trend and season components are not known, as a result autocorrelation distance is used for seaosnality reconstruction, and the temporal correlation is used for reconstructing the trend reconstruction. For the purpose of disentanglement, the MI between the latent components is minimized. The overall loss function has an ELBO loss, and MI maximization between the original time-series and latent component, and MI minimization between the season and trend latent components. Since the optimization is untractable, lower and upper bounds for MI are derived to help with the optimization. Experiments are performed on standard univariate and multivariate timeseries benchmarks, showing LaST achieves state of the art performance. Visualizations show effective disentanglement.  
strengths_and_weaknesses: Strengths
- good motivation to expand on decomposition of trend seasonality
- novel way to do so compared to existing approaches (e.g. variational inference vs CoST using contrastive learning)
- meaningful loss function (elbo, reconstruction, predictor)
- identified technical challenges in reconstruction, and proposed a way to address it
- identified challenges in MI optimisation and developed bounds to address it 
- good coverage and comprehensive experiments - baselines + datasets
- good ablation / visualizations
- a generally comprehensive paper which is well written

Weakness
- superposition of a lot of complex components to make it work, possibly creating uncertainty on the robustness of the model
questions: - how complex was performing this training and getting it to work? To me, it seemed like a daunting task
limitations: NA
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: In this submission, the authors propose to learn disentangled seasonal and trend representations of seasonal time series. In particular, in contrast to previous methods that use average pooling for the trend feature, the major contribution here is to directly optimize the two representations by variational method. Also, a detailed analysis of the loss function and the optimization bound is provided. Empirical results show strong effectiveness and outperforms several strong baselines. The following analysis also supports the motivation.
strengths_and_weaknesses: Originality: 
The proposed method of representation disentanglement is an extension of other fields such as computer vision. However, given the nature of seasonal time series, the application of this method is non-trivial, as well as the design of a proper predictor. The bound proof is original in this setting.

Quality: 
1. The conduct of empirical analysis is of high quality. The experiments are sufficient to support the claims (great performance boost). One weakness, as also noticed by the author, is the potential degeneration of features caused by narrowing down prior and posterior, and the effect is not discussed in detail. 
2. As the reviewer is not an expert in theoretical analysis, I would leave this part to other reviewers.

Clarity:
1. The overall presentation is clear, including a good schematic pipeline description and results tables.
2. The development of Theorem 2 is a bit confusing. The authors first claim that L rec can be estimated without leveraging Xs and Xt, while eq 6 still has the two variables. Line 143, what is "first difference?"

Significance:
The reviewer thinks this method provides significant advances in this subfield.

questions: 1. When discussing the connection to previous papers, the writing should both include the difference and the inheritance. Which components are extended from the previous studies? Also, what are techniques that can potentially be combined (such as contrastive learning from CoST)?

2. What is the boundary of the method? The drawback of feature degeneration should be discussed in more detail.

3. What is the training and inference time? How it can be compared with the baselines?
limitations: As discussed above, the reviewer encourages the authors to face the general limitation of VI methods and discuss the implications in this setting.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper focuses on the time series forecasting task and presents a decomposition method LaST to learn latent representations. The decomposition method is derived from variational inference and mutual information. Along with a predictor, LaST can achieve competitive performance in many benchmarks.
strengths_and_weaknesses: ### Strengths
1. The loss function for time series decomposition is reasonable and novel.

2. This paper is well-organized and clear.

3. The model performance is competitive and is with detailed analysis.

### Weaknesses
1. The ‘proof’ of the ‘Theorem’ 2 is too intuitive.
- The minimization between autocorrelations of $X$ and $\hat{X}^s$ is not convincing, which will bring the noise to the optimization. And this process also relies on an underlying assumption, that the raw time series is periodic by removing the trend, which is contradictory to the statement in line 89.
- The same problem is also in the design of CORT($X$,$\hat{X}^t$).
2. Comparing baselines:
- I think the N-BEATS is a necessary baseline since you both adopt the decomposition and a similar predictor for forecasting (linear model for trend and sine/cosine functions for seasonal).
- The most important thing of this paper is to explain the necessity of learning ‘latent’ representations. What about using the moving average of Autoformer for decomposition and adopting the same predictor for prediction? I think this vanilla baseline is also necessary.

questions: 1. What is the difference between the autocorrelation calculation used in Equ 6 and the autocorrelation calculation method in Autoformer? Maybe giving a citation is better.
2. The visualization in Figure 2 is not surprising. You can conduct the same visualization to Autoformer. I think the simple moving average block can achieve the representation disentanglement well. You can compare these two decomposition methods.
3. I am a little confused about the predictor (Figure 1 (b)) without the supplementary material. Especially in lines 127-128, the inverse DFT cannot change the sequence length. You should give more details about the “extend” in the main text.
4. It seems the prediction results show the over-smoothing problem (Figure 3). This problem is fatal for details predicting of time series. This can be caused by the design of the predictor. More showcases are required especially the non-stationary time series.

limitations: No, the author has not discussed any limitations of this work. I think the convergence property can be further analyzed. Also, the stochasticity of time series is neglected in the decomposition, which should also be noticed by the authors.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The author proposed VAE-based method, LaST, to disentangle the seasonal-trend representations of sequential data. The LaST uses the trend and seasonal data as the reconstructed targets, which enforce the model to learn representations dedicated to its target hence achieving better disentanglement. Finally, a predictor is introduced to guarantee its performance on the downstream tasks. Results show that LaST can achieve better predicting performance compared to other forecasting baselines.
strengths_and_weaknesses: Pros:
1. The usage of trend and seasonal input Xt and Xs force the model to learn dedicated representations and make us easier to evaluate the disentanglement.
2. The author provides rigorous mathematical proof, including the decomposition of ELBO, and the lower and upper bounds for MI optimization.
3. The author provides extensive results on both the prediction performance for downstream tasks and the disentanglement of the trend and seasonal features.

Cons:
1. The author stated that "existing approaches with a single high-dimensional representation sacrifice the information utilization and explainability". Models like CoST uses different modules to extract trend/seasonal dependencies. The author fails to address the difference between those models that do not use a high-dimensional representation.

2. Other disentangled representation learning methods for sequential data are not referenced as baselines i
questions: 1. Does these trend data Xt and seasonal data Xs are extracted from X? If so, how do you extract them?
2. Can you compare your results with other disentangled representation learning methods for sequential data?

limitations: 1. The generalisability of this LaST might be limited since it requires that the input data are a triplet: Time series data X, trend data Xt and seasonal data Xs. Many sequential datasets, especially the real-world ones, often don't have the complete triplet data or with a lot of missing/noising data. So its generalisability should be further investigated.

2. The author might need to add more baselines (other disentangled representation learning methods for sequential data).
review_title: null

Meta-review: metareview: The paper presents a novel learning approach named LaST for time-series forecasting based on variational inference to disentangle the seasonal-trend representations in the latent space. Empirical results validate the effectiveness of the proposed method in comparison with several strong baselines. Reviewers generally agree the work is technically solid, the idea is novel, the experiments are convincing, and the paper is well presented. Thus, the paper is clearly above the acceptance bar. Authors are encouraged to incorporate all the discussions and the additional results during the rebuttal in the final version. 
confidence: Certain
Decision: Accept