Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 1: strong reject
review_content: summary_of_the_paper: The paper focuses on designing a graph-based algorithm for multi-variate long-term forecasting
strength_and_weaknesses: STRENGTHS

- Interesting and relevant topic

WEAKNESSES

- The paper is presenting a very complex approach, however it is failing to beat simple baselines [1,2]. Therefore, results are not convincing. Comparing results of this work to univariate baselines [1,2], it is clear that the graph based results are not better than the univariate results. Therefore, I make the conclusion that the proposed mechanisms for using graph information are not effective. 

[1] Cristian Challu, Kin G. Olivares, Boris N. Oreshkin, Federico Garza, Max Mergenthaler-Canseco, Artur Dubrawski, N-HiTS: Neural Hierarchical Interpolation for Time Series Forecasting. https://arxiv.org/pdf/2201.12886.pdf

[2] Ailing Zeng, Muxi Chen, Lei Zhang, Qiang Xu, Are Transformers Effective for Time Series Forecasting? https://arxiv.org/pdf/2205.13504.pdf
clarity,_quality,_novelty_and_reproducibility: - Clarity is good
- Originality is marginal, there is a large body of work using GCN approach, for example in the context of multivariate traffic forecasting
- Code is not provided and no timeline is given when it will be released. Reproducibility is low


summary_of_the_review: The empirical results are not state of the art with respect to simple uni-variate baselines and therefore are not convincing. My opinion is that we need to more carefully moderate this flow of work that started with Informer, in which researchers recklessly apply overly complex architectures to the long-term forecasting datasets and never use a proper set of baselines to show that their sophisticated neural architectures are actually effective. I would finally like to see at least one paper on these datasets managing to beat univariate baselines, convincingly proving that there is a multivariate approach that works effectively by extracting information from multiple variables. I also have a strong reproducibility concern - code is not provided.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper propose a graph convolutional network-based framework which construct graph representations )time-, dimension-, and cross-segments-based view. The authors provides substantial motivations for long-term time-series forecasting problem: 1) lack of structural relationships between variables, 2) difficulty of capturing local dynamic changes, and 3) high complexity.  They validate their methods on five public datasets with five baselines and show that the proposed method outperforms its baselines on all tasks.
strength_and_weaknesses: The proposed method is technically simple, but it seems to show outstanding performance, compared to popular time-series forecasting models on the five public datasets. If technically correct, the authors provides a good solution that enables to generate structural information from general time-series datasets and show better accuracy results than its baselines.

However, I believe more description about the entire forecasting network and qualitative analysis (e.g., analysis on the extracted graphs of cross-segment wise, dimension-wise, time-wise) are required to see the effects of proposed methods. 

Q1) The authors should describe how RevIn module works for normalizing process on input sequences?
Q2) What does “multivariate long-time time-series” mean on page 4?
clarity,_quality,_novelty_and_reproducibility: I found most sections are clearly written and easy to understand. It will be great if the code of the proposed method is available to check the entire process for reproducibility. 
summary_of_the_review: The authors provides a good description about their motivation and proposed method seems to address the targeted problem. Although the shown results are quite interesting, I believe the authors need to provide qualitative and quantitative analysis on extracted graphs to address the effects of the suggested methods, which seems to effectively work on long-term forecasting tasks.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper presents a graph and GCN based method for long-term time series forecasting. The method aims to deal with three challenges: 1. modeling relationship between variables, 2. modeling local dynamic changes, 3. improving efficiency. The method constructs three graph views from the input time series, and extracts features from the constructed graphs using GCNs. The features were used for prediction.

strength_and_weaknesses: Strength
1. Modeling correlation between variables in multivariate time series forecasting is important.
2. The proposed investigates several ways of constructing relationships between variables.
3. The experimental results demonstrate the proposed method is effective of modeling multivariate time series to some extent.

Weakness
1. Modeling multivariate time series by graphs and GNN has been studied before in multiple works (e.g., Spectral Temporal Graph Neural Network for Multivariate Time-series Forecasting. In NeurIPS, 2020). The novelty of this paper is limited.
2. The proposed method is straightforward, by constructing correlation graphs form multivariate time series in a feature engineering manner. The choices of the three graph views may be subjective.
3. The method design is incremental upon several existing methods. Some design intuitions are unclear. For example, it is unclear on why to use GCN in Eq 6, 12, and 14 to process features. It seems to encourage smoothing in neighborhood, but the paper doesn't elaborate if how could this smoothing be important. Similarly, the differential operation in Eq 9, the linear layers in Eq 8, 12, and 14 are unclear.
4. The method simplifies the normalization of adjacency matrices in Eq 7. There is no theoretical support for this simplification. The reason for the simplification is to reduce complexity. A complexity analysis should be provided to support Eq 7. Intuitively, because D matrix is diagonal. The complexity may not change much by skipping it.
5. The Fig 1 seems inconsistent with the method description in Section 3. The input to dimension graph generator should be the output of the GCN on cross graph generator, and the input to the time graph generator should be the output of the GCN on the dimension graph generator. The Fig 1 does not reflect this sequential structure.
6. It is not clear using the same method in Eq 11 and Eq 13, why the graph constructed in Eq 13 is a time graph.
7. The training loss and training process of the proposed method are not clear. Some constraint to encourage the diversity of the multi-head attention could be included in the training loss.
8. In the experiments, no graph based baseline methods were compared. Some of them should be included to demonstrate the advantage of the proposed method. It is better to include confidence intervals in the tables in the experiments to demonstrate the statistical difference between different methods.

clarity,_quality,_novelty_and_reproducibility: See strength and weakness.
summary_of_the_review: This paper introduces a graph and GCN based methods for modeling the correlation between different variables in multivariate time series for long-term forecasting. The technical novelty of the proposed method is limited. Some method designs were not well justified. The experiments remain to be improved. See details in the strength and weakness section.
review_title: null

final_score: rating: 1: strong reject
review_content: summary_of_the_paper: This paper mainly studies the problem of multivariate time series forecasting. The authors argue there are three basic challenges and accordingly propose a multi-view time-series graph structure representation method. Extensive results show their improved forecasting performances on some datasets as well as the improved efficiency. 

 
strength_and_weaknesses: This paper is easy to follow. The paper organization is good and well written. However, there are some serious problems with this paper: 

the motivation and the related works. The argument that the lack of a relationship structure between multivariate variables needs to be addressed is inaccurate. There are several related works to study the variable relationships of multivariate time series forecasting, such as MTGNN [1], stemGNN [2], etc. It is a well-studied problem. I don’t understand why the authors don’t cite these important and famous related works and discuss the differences.  For the third motivation, there are several Linear complexity models such as N-BEATS [3]. The authors ignore this related work either. It is not appropriate. 

The experiments are not convincing. By default, we don’t consider variable relationships in benchmarks including ETT, Exchange or Weather coming from Informer and Autoformer. This makes the experimental results not fair when comparing with transformer-based forecasting models. The variable relationships (like graph structures) are always considered in another benchmark shown in MTGNN [1]. Also, some important baselines are missing, such as MTGNN [1], StemGNN [2]. All these points make the results less convincing. 

[1] Wu, Zonghan, et al. "Connecting the dots: Multivariate time series forecasting with graph neural networks." Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining. 2020. 

[2] Cao, Defu, et al. "Spectral temporal graph neural network for multivariate time-series forecasting." Advances in neural information processing systems 33 (2020): 17766-17778. 

[3] Oreshkin, Boris N., et al. "N-BEATS: Neural basis expansion analysis for interpretable time series forecasting." arXiv preprint arXiv:1905.10437 (2019). 
clarity,_quality,_novelty_and_reproducibility: The presentation of this paper is good. 

The novelty is limited. 

The reproducibility is bad. 

The overall quality is not very good. 
summary_of_the_review: This motivation is not convincing. Some important previous works are ignored. The experimental results are not fair. I prefer to reject the paper.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper proposes a multi-view graph structure representation method for multivariate long-term time-series forecasting tasks. It utilizes graph convolutional networks to establish topological connections from three perspectives: time, dimension, and cross-segment.

The paper is well-written and the proposed method has demonstrated better performance compared to transformer-based baselines on multiple benchmark datasets, with improved efficiency. 

However, this paper does not compare its method with graph-based methods (Reviewer PLaJ, 9BLg) and univariate baselines (Reviewer sneL), and further experiments should be conducted on more suitable multivariate datasets (Reviewer 9BLg). Additionally, an analysis of the constructed graphs from different views (cross-segment wise, dimension-wise, time-wise) is needed but currently missing. We recommend the authors to improve this work based on the reviewers’ comments.
justification_for_why_not_higher_score: Insufficient experiments including baseline comparison and the choice of datasets
Decision: Reject