Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: In this paper, the authors proposed a novel time-series dependency modeling framework for the long/short-term forecasting task. Specifically, the time series is downsampled into different sub-sequences and resolutions and jointly modeled to produce an enhanced overall representation for the forecasting task.
strengths_and_weaknesses: Strengths
+ The idea of interactive learning between downsampled sub-sequences is interesting.
+ Overall, the paper is well organized and written. 

Weakness
+ The motivation of this paper is built on the assumption that temporal relations are largely preserved after downsampling into two sub-sequences which may not hold true in many real-world situations such as volatile time series in the financial domain. 
+ In the main result, it's better to include the error bars instead of simply providing the prediction results to report the model performance.
+ The design motivation of some critical points, such as intermediate supervision, is not clearly explained or demonstrated in the experiments. 
+ The overall innovation is a bit incremental. The major contribution may come from the design of the interactive learning of two sub-sequences. .
questions: 1. The paper is based on an assumption: "temporal relations are largely preserved after downsampling into two sub-sequences." However, I don't think this holds true in many real-world scenarios. For example, downsampling may result in two sequences with different trends for a volatile time series, such as stock price. Also, some citations are expected to back this assumption. 
2. The overall contribution is a little bit incremental. Stacked blocks and the tree-like design in the SCINet are not something new, though the interactive learning between two sub-sequences is interesting.
3. Page 4, line 155. Can the authors provide more details about how the elements in all the sub-features are ordered? 
4. Page 4, line 159. It's noticed that the authors used the fully-connected layers as the prediction layer. However, some constraints after the fully-connected layers are necessary to deal with the real-world situation. For example, for traffic forecasting, only a positive value is expected. 
5. Page 5, line 165. Which part of the X is concatenated to $\hat{X}^k$?
6. The main results should include an error bar or the standard deviation. Also, the model should be evaluated by both absolute-value-based and percentage-based metrics. 
7. The motivation for adopting intermediate supervision is not well supported. It would be better if the authors could provide more details or demonstrate it through the ablation study. 
limitations: The authors partially pointed out this work's limitations regarding missing data and irregular time series. Other potential limitations are listed in the questions above.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper proposes to progressively downsampling the sequences into two sub-sequences, and then conduct convolution and interaction operations to study complex temporal dynamics of time series data. Experiments on various time series datasets confirm its significant forecasting accuracy improvements over both existing convolutional and Transformer-based models.
strengths_and_weaknesses: Strengths:
1. The core idea of progressively downsampling the sequences into two sub-sequences to study temporal dynamics is novel.
2. Extensive experiments are performed to confirm the effectiveness.
3. Source code is provided for reproducibility.

Weaknesses:
1. The code (https://anonymous.4open.science/r/SCINet-2588) is not anonymous. The following information is included:
     If you find this repository useful for your work, please consider citing it as follows:
     @article{...author={L**, ...},...
     }.  (checked on 2022/07/10. To keep anonymous, only the first letter of the first author's last name is shown here.)
2. For the experiment results in Table 2 and 4, the N/A and - could be filled by re-implementation or re-training.
3. It seems that SCINet needs more memory usage even compared to Transformer based on Table 3.
questions: Please refer the weaknesses. My biggest concern is on the anonymous.
limitations: N/A
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper devises a new architecture, SCINet, for the time-series forecasting problem. To extract features, SCINet mainly adopts convolutions on sub-series with different resolutions, which is accompanied by additional operations to ensure the interactions among the subsequences. The proposed architecture can also be stacked to improve forecasting accuracy. SCINet is compared with other convolutional models and transformer-based models on short-term/long-term/spatial-temporal time series forecasting tasks. 
strengths_and_weaknesses: Strengths:    
1. The paper is well written and easy to follow.

2. Though convolution is not uncommon in TSF models, the paper utilizes the special properties of time series data and proposes the downsampling and interaction operations, which leads to good empirical performance. 

Weaknesses:  
1. More empirical comparison with other related models would be better to support the superiority of the SCI-Net. (Please refer to Questions 3-4 for details)

2. More explanations on the training strategies adopted would be helpful to prove that the experiment section gives a fair comparison with other works. (Please refer to Questions 5-7 for details)

questions: 1. In line 109, the authors state that “each SCI-Block has both local and global views of the entire time series”. I would appreciate it if the authors could elaborate more on how a single SCI-Block can attain the global view of entire time series.
2. I would also appreciate it if the authors could explain more about the claim – “the proposed downsample-convolve-interact architecture achieves an even larger receptive field at each convolutional layer” in line 139.
3. As SCI-Net is claimed to “capture both short-term and long-term temporal dependencies in the time series”, I think it would be helpful to show its superiority by comparing it with Transformer-XL (which is good at preserving long-term information) empirically. 
4. Besides, as SCI-Net and Pyraformer both adopt hierarchical architecture for time-series forecasting task, I think it’s worth adding the latter in the experiment section for comparison. 
5. In the Table 3&4 of the Supplementary Material, the authors report the hyperparameters used by the SCI-Net. I notice that the batch size and learning rate significantly vary both within a single dataset / across the datasets. I’m wondering whether the performance of SCI-Net is highly sensitive to these training strategies.
6. In Table 3 of the main text, I notice that the paper uses the performance reported in the Autoformer and Informer paper. However, Autoformer and Informer adopt a significantly different training strategies (batch size, learning rate, and training epochs) compared with the SCI-Net. More concretely, Autoformer and Informer use constant batch size and same initial learning rate, respectively; while SCI-Net tunes these hyperparameters for every setting. Moreover, Autoformer and Informer are only trained for 10 and 8 epochs, respectively; while SCI-Net is trained for 150 epochs. Therefore, I’m wondering whether Table 3 offers a fair comparison and whether SCI-Net is efficient in terms of training time and generalizable to different datasets without intensive hyperparameter tuning.
7. It would be helpful if the authors could provide how many times each experiment is repeated and the error bar (e.g., standard deviation) of the reported results.

limitations: The limitations have been covered properly by the authors in the corresponding section.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The article proposes a novel method based on hierarchical downsample-convolve-interact architecture, named SCINet. The article points out that the temporal relation of time-series data is maintained even after downsampling, and argues that multiple convolution filters extract rich features from downsampled sub-sequences. The SCI block, the basic unit of SCINet, utilizes different convolution kernels, an inter-learning approach is applied, has residual connections, and linear enhancement is applied. The proposed method was evaluated on 9 types of popular time-series dataset and compared with the latest methods. Performance degradation occurred in the ETTh1 [37] dataset in the long-term forecasting task, but a significant improvement in overall performance was verified. 
strengths_and_weaknesses: (strength)
The proposed method has achieved improved performance compared to the latest deep learning-based time-series modeling methods. The proposed method is technically sound and has a sufficient amount of validation.
(weakness)
The article mixes several ideas, including SCI-block, hierarchical structure, downsampling, and residual connection. 

questions: 1. Is the improved performance achieved by stacking SCI-blocks, or is it due to the residual connection between SCI-blocks, or is it due to the SCI-block itself? Considering that the comparative methods also handle downsampled features by multiple convolution, it is necessary to highlight the cause of the performance improvement.
limitations: The authors point out that if the time-series obtained in the real-world are noisy or greatly affected by missing data, the method can lead to degradation in performance.
review_title: null

Meta-review: metareview: Reviewers agree that the proposed idea of interactive learning between two down-sampled sub-sequences is interesting and novel in time-series forecasting. Extensive experiments over 9 datasets and against various baseline methods are conducted to confirm the effectiveness of the proposed SCINet architecture. The paper is well organized and written, and the source code is provided for better reproducibility. Therefore, AC recommends acceptance.

It is worth mentioning that several reviewers raised their concerns towards the overall novelty, as tree-structured framework and stacked modules with intermediate supervision are not new in this area. Authors confirm that the main contribution lies in the proposal of SCI-Block design, which successfully captures long-time dependency even with convolution operations (instead of attention mechanism). This is built upon the assumption that temporal relations are largely preserved after down-sampling into two sub-sequences. However, it remains unclear how to verify whether the time-series forecasting task at hand meets such assumption. AC would like to encourage authors to further discuss this topic in their revised manuscript.
confidence: Certain
Decision: Accept