Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes the Multi-scale Isometric Convolution Network, which can efficiently capture
local-global interaction of time series data for long-term forecasting. It uses multi-scaled
convolutions for local information extraction and isometric convolution to capture the global
relationship. In general, it achieves the state-of-the-art accuracy in five time series benchmarks
with linear complexity.
strength_and_weaknesses: Strength:
The overall paper is well written and most of the concepts are concisely explained. As the paper
stated, both local patterns and global dependency are crucial for long-term time series
forecasting. The design is well motivated. The benchmark results are also promising to
demonstrate the effectiveness of the framework.

Weaknesses:
Despite the state-of-the-art accuracy, it seems that the main improvement comes from the
regression-based trend-cyclical prediction. Trend-cyclical analysis in section 4.2 is not enough to
understand how the regression influence the prediction. I suggest evaluating the regression and
the mean prediction separately from the full benchmark performance. Also, the results show that
MICN-mean has comparable performance (sometimes even worse) than FEDformer, while
FEDformer is using mean trend prediction. I can’t help but thinking if FEDformer with regression
trend prediction can surpass MICN. More discussion in trend-cyclical prediction is needed.
Another thing that can be discussed more is the intuition behind isometric convolution. The
motivation of the linear complexity is good, but the paper is unclear about how isometric
convolution is going to capture the correlations between local time region. Besides, there exists
many works that focus on the efficiency of self-attention and many has linear complexity as well
(e.g., Linformer [1], Fastformer [2], …). If the primary motivation behind isometric convolution is
the complexity then these works should be discussed as well.
I also have concerns over the explainability of the local-global structure. Self-attention is good at
making interpretable prediction (can be crucial for real-life application, as the paper mentioned),
whereas I don’t see how MICN can be interpreted at a first glance. It is sad when the paper
argues “the framework can deeply mine the intricate temporal patterns” but fails to interpret them.
This is also one of the reasons I am curious about how linear self-attention can perform instead of
isometric convolution.
Other minor suggestion:
1. The abstract says that the structure can model patterns separately and “purposefully”. The
wording is vague and confusing. Does it indicate that we can incorporate external knowledge
into the model?
2. In section 3.1, briefly mentions that the term (e.g., “AvgPool”, “kernel”) is from the convolution
perspective. I first thought the “kernel” represents the kernel function.
3. The difference between ablation table 5 and table 13 can be explained more.
[1] Wang, Sinong, et al. “Linformer: Self-Attention with Linear Complexity.” ArXiv.org, 14 June
2020, https://arxiv.org/abs/2006.04768.
[2] Wu, Chuhan, et al. “Fastformer: Additive Attention Can Be All You Need.” ArXiv.org, 5 Sept.
2021, https://arxiv.org/abs/2108.09084.
clarity,_quality,_novelty_and_reproducibility: The paper is well written with detailed appendix. The general idea of using a hierarchy alike
structure for capturing local-global interaction, plus solving the self-attention complexity issue, are
not completely novel (many works in vision transformer field), but the authors manage to design a
structure specifically for time series forecasting.
summary_of_the_review: As I mentioned above, the overall method is well motivated and the benchmark results are
promising, whereas the paper can be further improved by more ablation studies and discussions.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper focuses on long term forecasting. The problem to be solved is to predict values of a variable for a future period. The authors start by providing a comprehensive review of the related papers and end with the conclusion that a good forecasting method should have the  ability to extract local features to measure short-term changes, and the ability to model the global
correlations to measure the long-term trend. Based on that, they propose a Multi-scale Isometric Convolution Network (MICN). They use multiple branches of different convolution kernels to model different potential pattern information of the sequence separately.
strength_and_weaknesses: The topic of this paper address the problem of long-term series forecasting. The topic is very interesting and the paper is well written. The authors provide a comprehensive review of the related papers. Mathematical models and figures are proposed to understand the problem and the solution provided by the authors. Extensive experiments are provided to demonstrate the effectiveness of the proposed method.
clarity,_quality,_novelty_and_reproducibility: The paper is well written overall and easy to understand
summary_of_the_review: This paper proposes a Multi-scale Isometric Convolution Network (MICN)  based on convolution structure to efficiently replace the self-attention,and it achieves linear computational complexity and memory cost. 
The  empirical studies show that the proposed model improves the performance of state-ofthe-art methods by 18.2% and 24.5% for multivariate and univariate forecasting, respectively
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a method named MICN to replace the self-attention module and achieves linear computational complexity. This method is based on convolution structure and proposes Multi-scale Isometric Convolution using multiple branches of different convolution kernels to model different potential pattern information of time series. The MICN has a local-global structure where the local features of the sequence are extracted by a local module based on downsampling convolution and the global correlation is modeled by a global module based on isometric convolution. The results show that MICN achieves state-of-the-art accuracy and isometric convolution is superior in both complexity and effectiveness to self-attention.
strength_and_weaknesses: Pros:
- The results show that MICN achieves state-of-the-art accuracy on five real-world benchmarks. MICN yields impressive 18.2% and 24.5% relative improvements for multivariate and univariate time series, respectively.
- This paper uses a lot of ablation experiments to verify that the proposed Multi-scale Isometric Convolution outperforms the self-attention family and Auto-correlation mechanism.
- This paper gives a detailed mathematical description of their method, showing how to extract local features to measure short-term changes and to model the global correlations to measure the long-term trend using a convolution structure.

Cons:
- Complexity analysis is needed. One of the motivations of this paper is the high complexity of self-attention and this paper tries to reduce the complexity. However, the full text doesn't analyze the complexity of the proposed method, just saying the complexity of the method is O(L), but 'L' is not mentioned in the full text. And the complexity of the method seems to be N*logI.
- Isometric convolution is not explained in detail as the core part of the article. How is isometric reflected? Why do convolutions in this form?
- The paper proves that the proposed Multi-scale Isometric Convolution is superior to self-attention through experiments. However, for the short length of 92, the isometric convolution performance is all lower than self-attention in multivariate forecasting, which is inconsistent with the description of the paper, and there are no further explanations and analysis.

Questions：
- In Section 3.2, what conditions are satisfied with the kernels to ensure that the time series length is constant? How is each kernel determined? 
- In Section 3.3, is 'mean' here an average operation? How to average a time series? How to get the result of o-dimension from i-dimension time series through mean operation?
- In section B.3, why is the replacement of Isometric convolution and self-attention in MICN-regre instead of the Seasonal Prediction Block?
clarity,_quality,_novelty_and_reproducibility: Good
summary_of_the_review: The idea is interesting and somewhat novel. More theoretical analysis would be better.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper presents a transformer model based method for efficient capture of local-global interaction in time series data for long-term forecasting. It proposes a multi-scaled convolutions for extracting short term dependencies and isometric convolution for long term dependencies. The paper has a clear motivation, the algorithm is clearly explained and the experiments are sound. In general, the paper is well-written and is clearly explained. 

Authors bold claims on the interpretation have not been sufficiently substantiated. Although authors have indicated that this is a scope of future work, recommend to include these texts subtly without emphasis.

The paper should be revised to include complexity analysis of the proposed method.
justification_for_why_not_higher_score: The manuscript should be revised to include complexity analysis.
justification_for_why_not_lower_score: The paper presents a transformer based method to capture long term temporal dependencies, while also preserving short-term sequences for long term time-series predictions. It has potential for interpretable time-series forecasting. The paper is clearly written.
Decision: Accept: notable-top-5%