Paper: iTransformer: Inverted Transformers Are Effective for Time Series Forecasting (ICLR-2024)

Reviewer comment (Experiments):
Experiments are conducted only on selected datasets compared to some major baselines. The paper does not report on datasets Exchange, ILI and ETTm1, ETTm2, ETTh1 and ETTh2, which are different from the experiments reported in TimesNet. This makes it hard to determine if the proposed method really outperforms the baselines consistently or just on the selected datasets.

Author response:
Thanks for your scientific rigor. We provide the additional experiments in our revised paper:

  * Report the standard deviations of iTransformer performance in $\underline{\text{Table 5 of the revised paper}}$ with five random seeds,  **exhibiting stable performance**. 
  * Update the main results in $\underline{\text{Figure 1 and Table 1 of the revised paper}}$. We report the averaged results from four subsets of ETT and PEMS. We also add the Exchange dataset. The detailed results are updated in  $\underline{\text{Table 9 and 10 of the revised paper}}$.
  * Full results of the inverted Transformers on more datasets in $\underline{\text{Appendix E and F of the revised paper}}$ to further support the conclusions of our model analysis.

Reviewer comment (Reproducibility):
Experimental results for the close baseline PatchTST vary from published results. PatchTST consistently reports better results, e.g., for Electricity with horizon 96 they report an MSE of 0.129, whereas this paper reports 0.195. The source of this discrepancy is unclear.

Author response:
We carefully read through the forecasting settings of the PatchTST paper and its official repo. There's a lot of difference here:

  * **Enlarged lookback window**: PatchTST paper adopts tunable lookback length in $\{336, 512\}$, while ours adopts the unified length of $96$, following the unified long-term forecasting protocol of TimesNet.
  * **More epochs to train**: PatchTST paper trains the PatchTST model with $100$ epochs, while we train all models with $10$ epochs.
  * **Learning rate**: PatchTST paper adapts a carefully designed learning rate strategy.

Reviewer comment (Evaluation):
No standard deviations are reported. Standard deviations would help assess which differences might be significant and which might be spurious.

Author response:
Thanks for your scientific rigor. We provide the additional experiments in our revised paper:

  * Report the standard deviations of iTransformer performance in $\underline{\text{Table 5 of the revised paper}}$ with five random seeds,  **exhibiting stable performance**. 
  * Update the main results in $\underline{\text{Figure 1 and Table 1 of the revised paper}}$. We report the averaged results from four subsets of ETT and PEMS. We also add the Exchange dataset. The detailed results are updated in  $\underline{\text{Table 9 and 10 of the revised paper}}$.
  * Full results of the inverted Transformers on more datasets in $\underline{\text{Appendix E and F of the revised paper}}$ to further support the conclusions of our model analysis.

Reviewer comment (Writing):
The writing style sometimes does not go directly to the point and could be improved to better convey the idea and have greater impact.

Author response:
Thanks for your valuable suggestions. We conduct the following revisions of our paper:

  * Misspelling: hardy -> hardly; Soloar-Energy -> Solar Energy.
  * Abstract: "inverts the duties of the attention mechanism and the feed-forward network" -> "applies the attention and feed-forward network on the inverted dimensions". The detailed duties are elaborated in $\underline{\text{Section 3.2}}$. 
  * Figure 8: add the label of the x-axis.
  * Abbreviate the Electricity dataset as ECL.

Reviewer comment (Experiments):
Authors fail to include some important SOTA baselines and properly synthesize/discuss CD/CI/distribution shift comparing all the baselines (including the missing ones) and iTransformer.

Author response:
**Q6:** Discussions on the Channal-Independence/Dependence, Instance Normalization, and the linear models.

Thanks for your valuable suggestion. **We newly added RLinear to our baseline**, which is very competitive on several datasets, such as ETT and Weather.

As the reviewer insightfully mentioned the CI/CD and RevIN[1], it is important to discuss on these techiques: 

  - Previous works[2]\[3] have discussed the CI/CD tradeoff on model capacity and robustness. For example, CI can greatly increase the performance when data is scarce, while CD benefits from a higher capacity ideally. In the context of our work, **we think it is essential to make the variates independent**, especially when we know there are potential risks of delayed series and loosely organized measurements. Still, no matter CI/CD, **multivariate correlations** can not be explicitly utilized. Perpendicular to these works, iTransformer repurposes an architecture with the native Transformer modules to cope with it.
  - RevIN or Series Stationarization[4] has been widely applied for the distribution shift (or non-stationarity) for real-world time series. They can boost the performance of both linear models and Transformers. There are Transformers in our baseline already equipped with this technique (PatchTST and Stationary), and we newly include the linear one, RLinear. These works strive for modeling **temporal dependencies**. In iTransformer, it is modeled naturally by the FFN and layernorm, and still leaves further improvement for us to tackle the distribution shift.
  - Compared with linear forecasters, **iTransformer is good at forecasting high-dimensional time series** (that is, numerous variates, and complicated correlations). Under the univariate forecasting scenarios, iTransformer becomes in fact a stackable linear forecaster (FFN with the laynorm). For variate correlating, iTransformer keeps the variate independent and the attention module is on duty.

[1] RevIN: Reversible Instance Normalization For Accurate Time-series Forecasting Against Distribution Shift.

[2] The Capacity and Robustness Trade-off: Revisiting the Channel Independent Strategy for Multivariate Time Series Forecasting.

[3] Is Channel Independent strategy optimal for Time Series Forecasting?

[4] Non-stationary Transformers: Exploring the Stationarity in Time Series Forecasting.

Reviewer comment (Writing):
In the abstract, “the duties of the attention mechanism” should be “the following duties of the […]” to make reading smoother and avoid readers questioning what duties means here.

Author response:
Thanks for your valuable suggestions. We conduct the following revisions of our paper:

  * Misspelling: hardy -> hardly; Soloar-Energy -> Solar Energy.
  * Abstract: "inverts the duties of the attention mechanism and the feed-forward network" -> "applies the attention and feed-forward network on the inverted dimensions". The detailed duties are elaborated in $\underline{\text{Section 3.2}}$. 
  * Figure 8: add the label of the x-axis.
  * Abbreviate the Electricity dataset as ECL.

Reviewer comment (Experiments):
The claim “with potentially unaligned timestamps” needs to be referenced or proven. Authors should clarify which datasets have unaligned timestamps, or explain if by “unaligned” they mean delays between values from variates. Authors should demonstrate this for datasets where relevant, as in Weather this is clearly not the case.

Author response:
**Q3:** The proof supports the potential risks of unaligned timestamps and measurements on specific data sets.

We provide a detailed analysis of the Traffic dataset in $\underline{\text{Appendix E.3 of the revised paper}}$. We find **there are systematical delays in the road occupancy that each series describes**, because the sensors are installed in different areas (imagine how a traffic jam influences the road occupancy of different areas). 

Delay detection is the essential work that the MTSF model should cope with. However, it is because the vanilla Transformer embeds the localized time points that the obvious decay will be fused with noise.

Another support comes from the declined Traffic performance of the second and third rows in $\underline{\text{Table 3}}$, where attention is applied to the tokens of simultaneous time points. Since they do not reflect the same event, the performance can degrade a lot because of meaningless attention maps, unless the model has an enlarged respective field to capture the decay.

As for the unaligned measurements, it is common in time series forecasting, such as organizing together the different meteorological indicators (temperature and rainfall in Weather), or the quantity and proportion of the same observation (ILI dataset).

Reviewer comment (Writing):
It is more appropriate to call the Electricity dataset as ECL.

Author response:
Thanks for your valuable suggestions. We conduct the following revisions of our paper:

  * Misspelling: hardy -> hardly; Soloar-Energy -> Solar Energy.
  * Abstract: "inverts the duties of the attention mechanism and the feed-forward network" -> "applies the attention and feed-forward network on the inverted dimensions". The detailed duties are elaborated in $\underline{\text{Section 3.2}}$. 
  * Figure 8: add the label of the x-axis.
  * Abbreviate the Electricity dataset as ECL.

Reviewer comment (Reproducibility):
For reproducibility, authors should avoid confusion and specify which ETT is used. It appears to be ETTm according to the appendix (15min frequency), but is it m1 or m2? Informer results resemble ETTh.

Author response:
**Q2:** Avoid dataset confusion and be precise on ETT and PEMS datasets.

Thanks for your scientific rigor. To avoid confusion, we extensively test the performance of all four ETT subsets and the same four PEMS subsets as the SCINet. **We report the averaged results of these subsets** in $\underline{\text{Figure 1 and Table 1 of the revised paper}}$ to  The detailed dataset descriptions are updated in $\underline{\text{Appendix A.1 of the revised paper}}$.

To be more precise and reproducible, we also **report the standard deviations of iTransformer performance in five runs, indicating stable results**, which are updated in $\underline{\text{Table 5 of the revised paper}}$.

Reviewer comment (Reproducibility):
For reproducibility, authors should also specify which PEMS dataset is used — e.g., PEMS-Bay (occupancy ratio or speed in San Francisco Bay?) or another variant.

Author response:
**Q2:** Avoid dataset confusion and be precise on ETT and PEMS datasets.

Thanks for your scientific rigor. To avoid confusion, we extensively test the performance of all four ETT subsets and the same four PEMS subsets as the SCINet. **We report the averaged results of these subsets** in $\underline{\text{Figure 1 and Table 1 of the revised paper}}$ to  The detailed dataset descriptions are updated in $\underline{\text{Appendix A.1 of the revised paper}}$.

To be more precise and reproducible, we also **report the standard deviations of iTransformer performance in five runs, indicating stable results**, which are updated in $\underline{\text{Table 5 of the revised paper}}$.

Reviewer comment (Presentation):
In Figure 7 (b? right part), the content is difficult to understand without an explanation of x and y axes, the differences between Score Map and Correlations, the important aspects of the figure, how readers should interpret the lines, and why it is better than a usual Transformer.

Author response:
**Q4:** More detailed explanations about $\underline{\text{Figure 7(b)}}$ about the multivariate correlations.

We newly add detailed explanations about multivariate correlations in $\underline{\text{Appendix E.1 of the revised paper}}$. In a nutshell, the x- and y-axis place the variates, and **the attention map can thus exhibit the Pearson Correlations among variates**. We want to show that the attention maps can reflect the correlations of raw series (comparing the subplots of the same column), benefiting Transformer with enhanced interpretability. Besides, the correlations, which vary from the lookback window to forecast window, will be reflected by the attention maps that change accordingly (comparing the subplots of the same row).

Reviewer comment (Presentation):
Figure 8 needs a label for the x axis.

Author response:
Thanks for your valuable suggestions. We conduct the following revisions of our paper:

  * Misspelling: hardy -> hardly; Soloar-Energy -> Solar Energy.
  * Abstract: "inverts the duties of the attention mechanism and the feed-forward network" -> "applies the attention and feed-forward network on the inverted dimensions". The detailed duties are elaborated in $\underline{\text{Section 3.2}}$. 
  * Figure 8: add the label of the x-axis.
  * Abbreviate the Electricity dataset as ECL.

Reviewer comment (Experiments):
The claim “which can be attributed to the extremely fluctuating series of the dataset, and the patching mechanism of PatchTST may lose focus on specific locality to handle rapid fluctuation. By contrast, our proposed method aggregating the whole series variations for series representations can better cope with this situation” requires proof. Authors should show this situation and provide learning weight evidence for iTransformer demonstrating it can cope with the situation.

Author response:
**Q5:** The proof supports the fluctuation of PEMS.

We provide PEMS prediction visualization in $\underline{\text{Appendix E.2 of the revised paper}}$. **It shows that PEMS has more fluctuating series variations compared with others**. We highlight "PatchTST may lose focus on specific locality to handle rapid fluctuation" for that the patch length should be adaptively tuned for the series (such as frequency). Instead, iTransformer embeds the whole series as a token to describe the underlying process globally.

Reviewer comment (Experiments):
Authors are missing the opportunity to have a greater impact in the MTSF community by positioning their proposal more clearly in the discussion about channel dependence (CD) or channel independence (CI) and projection types. Despite comparisons to PatchTST and CD/CI discussion, methods like RLinear or RMLP (possibly outperforming PatchTST) are missing, and these use RevIN which helps handle distribution shift and could be applied to Transformer-based models.

Author response:
**Q6:** Discussions on the Channal-Independence/Dependence, Instance Normalization, and the linear models.

Thanks for your valuable suggestion. **We newly added RLinear to our baseline**, which is very competitive on several datasets, such as ETT and Weather.

As the reviewer insightfully mentioned the CI/CD and RevIN[1], it is important to discuss on these techiques: 

  - Previous works[2]\[3] have discussed the CI/CD tradeoff on model capacity and robustness. For example, CI can greatly increase the performance when data is scarce, while CD benefits from a higher capacity ideally. In the context of our work, **we think it is essential to make the variates independent**, especially when we know there are potential risks of delayed series and loosely organized measurements. Still, no matter CI/CD, **multivariate correlations** can not be explicitly utilized. Perpendicular to these works, iTransformer repurposes an architecture with the native Transformer modules to cope with it.
  - RevIN or Series Stationarization[4] has been widely applied for the distribution shift (or non-stationarity) for real-world time series. They can boost the performance of both linear models and Transformers. There are Transformers in our baseline already equipped with this technique (PatchTST and Stationary), and we newly include the linear one, RLinear. These works strive for modeling **temporal dependencies**. In iTransformer, it is modeled naturally by the FFN and layernorm, and still leaves further improvement for us to tackle the distribution shift.
  - Compared with linear forecasters, **iTransformer is good at forecasting high-dimensional time series** (that is, numerous variates, and complicated correlations). Under the univariate forecasting scenarios, iTransformer becomes in fact a stackable linear forecaster (FFN with the laynorm). For variate correlating, iTransformer keeps the variate independent and the attention module is on duty.

[1] RevIN: Reversible Instance Normalization For Accurate Time-series Forecasting Against Distribution Shift.

[2] The Capacity and Robustness Trade-off: Revisiting the Channel Independent Strategy for Multivariate Time Series Forecasting.

[3] Is Channel Independent strategy optimal for Time Series Forecasting?

[4] Non-stationary Transformers: Exploring the Stationarity in Time Series Forecasting.

Reviewer comment (Experiments):
The paper’s impact would increase if authors compared results to revInformer, RLinear, and RMLP (and corresponding papers) and discussed results in the context of inverse vs. CI vs. distribution shift handling. This would better inform the community about the broader context.

Author response:
**Q6:** Discussions on the Channal-Independence/Dependence, Instance Normalization, and the linear models.

Thanks for your valuable suggestion. **We newly added RLinear to our baseline**, which is very competitive on several datasets, such as ETT and Weather.

As the reviewer insightfully mentioned the CI/CD and RevIN[1], it is important to discuss on these techiques: 

  - Previous works[2]\[3] have discussed the CI/CD tradeoff on model capacity and robustness. For example, CI can greatly increase the performance when data is scarce, while CD benefits from a higher capacity ideally. In the context of our work, **we think it is essential to make the variates independent**, especially when we know there are potential risks of delayed series and loosely organized measurements. Still, no matter CI/CD, **multivariate correlations** can not be explicitly utilized. Perpendicular to these works, iTransformer repurposes an architecture with the native Transformer modules to cope with it.
  - RevIN or Series Stationarization[4] has been widely applied for the distribution shift (or non-stationarity) for real-world time series. They can boost the performance of both linear models and Transformers. There are Transformers in our baseline already equipped with this technique (PatchTST and Stationary), and we newly include the linear one, RLinear. These works strive for modeling **temporal dependencies**. In iTransformer, it is modeled naturally by the FFN and layernorm, and still leaves further improvement for us to tackle the distribution shift.
  - Compared with linear forecasters, **iTransformer is good at forecasting high-dimensional time series** (that is, numerous variates, and complicated correlations). Under the univariate forecasting scenarios, iTransformer becomes in fact a stackable linear forecaster (FFN with the laynorm). For variate correlating, iTransformer keeps the variate independent and the attention module is on duty.

[1] RevIN: Reversible Instance Normalization For Accurate Time-series Forecasting Against Distribution Shift.

[2] The Capacity and Robustness Trade-off: Revisiting the Channel Independent Strategy for Multivariate Time Series Forecasting.

[3] Is Channel Independent strategy optimal for Time Series Forecasting?

[4] Non-stationary Transformers: Exploring the Stationarity in Time Series Forecasting.

Reviewer comment (Experiments):
For experiments where only P% of variates are used for training with random selection, it would be good to run the experiment with different random sets and plot error or box plots to show generality. For example, for Figure 8 (a? left), show min, max, average for each % of variates, and in Figure 5 use a boxplot. Authors should also clarify that iTransformer uses the same variate set as CI transformer in these comparisons.

Author response:
**Q3:** Try various configurations for the experiments with partial variates.

Thanks for your scientific rigor. We newly conduct $60$ experiments on partial variates, which list the variate generalization results of all five folds with 20% partial variates, as well as the average (**Avg**) and standard deviation of the five runs. We list the comprehensive generalization error as **Comp** by calculating the increasing ratio of MSE from **Full** (full-variate training) to **Avg**.

It seems that taking the first 20% variates instead of the last 20% results in slightly better transfer on ECL. But the conclusion can still hold that our model can feasibly generalize on unseen variates with a small increase in forecasting error on Traffic and Solar Energy.

As you rigorously point out, it is important to perform the experiment with different sets of variates and plot the averaged results. Due to the limited time, we are unable to complete Channel-Indpendent experiments since it is really time-consuming on the dataset with numerous variates. We promise to update the Figures with the averaged results from all folders in the final version.

| MSE (iTransformer) | ECL                  | Traffic             | Solar Energy         |
| ------------------ | -------------------- | ------------------- | -------------------- |
| Full (0-100%)      | 0.162                | 0.443               | 0.202                |
| 0-20%              | 0.192                | 0.444               | 0.214                |
| 20%-40%            | 0.210                | 0.448               | 0.210                |
| 40%-60%            | 0.210                | 0.448               | 0.242                |
| 60%-80%            | 0.222                | 0.445               | 0.239                |
| 80%-100%           | 0.222                | 0.445               | 0.225                |
| Avg ± Std  (Comp)  | 0.211±0.011 (+30.2%) | 0.445±0.001 (+0.6%) | 0.226±0.013 (+11.8%) |


| MSE (iInformer)   | ECL                  | Traffic             | Solar Energy         |
| ----------------- | -------------------- | ------------------- | -------------------- |
| Full (0-100%)     | 0.169                | 0.459               | 0.200                |
| 0-20%             | 0.204                | 0.466               | 0.233                |
| 20%-40%           | 0.211                | 0.465               | 0.228                |
| 40%-60%           | 0.218                | 0.465               | 0.237                |
| 60%-80%           | 0.217                | 0.477               | 0.238                |
| 80%-100%          | 0.215                | 0.468               | 0.230                |
| Avg ± Std  (Comp) | 0.213±0.005 (+26.0%) | 0.468±0.005 (+2.0%) | 0.233±0.004 (+16.4%) |


| MSE (iReformer)  | ECL                  | Traffic             | Solar Energy        |
| ---------------- | -------------------- | ------------------- | ------------------- |
| Full (0-100%)    | 0.170                | 0.459               | 0.198               |
| 0-20%            | 0.190                | 0.463               | 0.204               |
| 20%-40%          | 0.202                | 0.462               | 0.203               |
| 40%-60%          | 0.203                | 0.463               | 0.207               |
| 60%-80%          | 0.221                | 0.463               | 0.215               |
| 80%-100%         | 0.202                | 0.466               | 0.208               |
| Avg ± Std (Comp) | 0.204±0.010 (+20.0%) | 0.463±0.001 (+1.0%) | 0.207±0.004 (+4.8%) |


| MSE (iFlowformer) | ECL                  | Traffic             | Solar Energy        |
| ----------------- | -------------------- | ------------------- | ------------------- |
| Full (0-100%)     | 0.168                | 0.449               | 0.204               |
| 0-20%             | 0.191                | 0.456               | 0.214               |
| 20%-40%           | 0.208                | 0.449               | 0.217               |
| 40%-60%           | 0.234                | 0.454               | 0.229               |
| 60%-80%           | 0.217                | 0.450               | 0.222               |
| 80%-100%          | 0.234                | 0.448               | 0.229               |
| Avg ± Std  (Comp) | 0.216±0.016 (+28.5%) | 0.451±0.003 (+0.4%) | 0.222±0.006 (+8.6%) |

Reviewer comment (Experiments):
To improve reproducibility and impact, authors should add results and visualizations for other datasets (similar to Table 2, Figure 5, etc.) in the appendix, to demonstrate whether the findings generalize.

Author response:
**Q7:** Complete experiments to enhance the robustness of the results and reproducibility.

We extensively include the following experiments to our paper:

  * To support $\underline{\text{Table 2}}$: promotion of the inverted Transformer on all datasets in $\underline{\text{Appendix F.1 of the revised paper}}$.

  * To support $\underline{\text{Figure 5}}$: additional results of PEMS to show iTransformer can generalize on unseen variates in $\underline{\text{Appendix F.3 of the revised paper}}$.

Additional results of increasing lookback on Solar-Energy: increasing the lookback length lead to an U-curve of performance (change point in $336$ and $720$) on this dataset. We also provide the explanations in $\underline{\text{Q4 of Reviewer yopU}}$.

Reviewer comment (Evaluation):
Figure 6 results for datasets like Solar Energy may not be as strong due to high seasonality and dependence on weather, so increasing lookback might not help much. Authors should address this.

Author response:
**Q7:** Complete experiments to enhance the robustness of the results and reproducibility.

We extensively include the following experiments to our paper:

  * To support $\underline{\text{Table 2}}$: promotion of the inverted Transformer on all datasets in $\underline{\text{Appendix F.1 of the revised paper}}$.

  * To support $\underline{\text{Figure 5}}$: additional results of PEMS to show iTransformer can generalize on unseen variates in $\underline{\text{Appendix F.3 of the revised paper}}$.

Additional results of increasing lookback on Solar-Energy: increasing the lookback length lead to an U-curve of performance (change point in $336$ and $720$) on this dataset. We also provide the explanations in $\underline{\text{Q4 of Reviewer yopU}}$.

Reviewer comment (Evaluation):
The paper is relatively short of explanation/justification about the effectiveness of such an approach.

Author response:
**Q5:** Explanations about the effectiveness of our approach.

We add new sections to analyze the effects of the inverting:

  * Analysis of multivariate correlations in $\underline{\text{Appendix E.1 of the revised paper}}$.
  * Potential risks of the vanilla Transformer embedding in $\underline{\text{Appendix E.3 of the revised paper}}$.
  * Full results of the iTransformers in $\underline{\text{Appendix F of the revised paper}}$ to support that our method can boost the Transformers' performance, and generalize on unseen variates.

Reviewer comment (Experiments):
It would be better to describe the train-validation-test split in experiments, like training in past years and predicting in the next year, as it could be tricky for data pre-processing in time series forecasting and cause data leakage issues.

Author response:
**Q1:** Describe the train-validation-test split in experiments.

We adopt the same train-validation-test split protocol as previous works, where the train, validation, and test datasets are **strictly divided according to chronological order to make sure there are no data leakage issues**. The data processing and split protocols are also included in $\underline{\text{Appendix A.1 of the revised paper}}$.

Reviewer comment (Theory):
After reading this paper, does the author implicitly assume the heterogeneity of variates is more important than temporal dependency in terms of forecasting?

Author response:
**Q2:** Do we implicitly assume that variate heterogeneity is more important than temporal dependency for forecasting?

We think they are both important to achieve good MTSF performance. **However, the problem is that the heterogeneity of variates can hardly be considered in the vanilla Transformer**. After embedding, the variates are projected into the channels of embedding. It ignores the problem of inconsistent physical measurements and can fail to maintain the independence of variates, let alone capture and utilize the multivariate correlation, which is essential for forecasting with numerous variates, and complicated systems driven by the latent physical process (such as meteorological systems). 

In addition, even if the FFN and layernorm seem simpler than attention blocks, **they are efficient and competent in learning the temporal dependency of a series**, which can be traced back to statistical forecasters such as ARIMA and Holt-Winter. They also have no problems with inconsistent measurements since they work on the time points of the same variate, and have an enlarged respective field as the whole lookback series can be embedded as the variate token.

Reviewer comment (Evaluation):
There are some interesting results in Table 3. Could the author comment on the not-so-good performance of the second row (both attention)?

Author response:
**Q3:** The reason for the not-so-good performance when applying attention to both temporal and variate dimensions.

As we observed the same declined performance of Traffic in the third row, the reason can be that **the designs both apply the attention module to the temporal dimension**. In our opinion, capturing temporal dependencies by attention is not a big problem. But it is **based on the fact that the time points of each timestamp are essentially aligned**.

We analyze the Traffic dataset in $\underline{\text{Appendix E.3 of the revised paper}}$, and we find there are potential risks of systematical delay among the road occupancy that each series describes, since the sensors are installed in different areas of the highway.

Consequently, applying attention to the time points of the same timestamp on such a dataset can make the model learn meaningless attention maps, unless the model has an enlarged respective field to capture the decay or causal process. It is one of the reasons that we advocate embedding the whole series as the variate token.

Reviewer comment (Evaluation):
A minor one: in the left panel of Figure 6, there is a little jump on the red line from 336 to 720. Any reason why? Do multiple runs help?

Author response:
**Q4:** Explanations about the error jump when the input length increases from $336$ to $720$ in $\underline{\text{Figure 6}}$.

We have also noticed the problem that our approach embedding a large lookback series simply by MLP can be simple and coarse-grained. As we increase the lookback length by a large step, the input series inevitably contains more nonbeneficial information for future prediction, and the model capacity should also increase accordingly. It is instructive for us to develop fine-grained variate tokenization and well-designed embedding mechanisms.

Reviewer comment (Evaluation):
According to Table 3 and Table 7, most of the result values are relatively small. This suggests that some marginal improvement may be susceptible to random factors (e.g., iTransformer v.s. PatchTST on ETT and Weather dataset, iTransformer v.s. SCINet on PEMS dataset). Therefore, I recommend reporting the standard deviation under different random runs and adding a significance test to provide further insights.

Author response:
Thanks for your scientific rigor. We repeat each experiment five times with different random seeds. The standard deviations of iTransformer performance are updated in $\underline{\text{Table 5 of the revised paper}}$, which shows **the performance is stable**. 

We also conducted the significance tests to compare iTransformer and the mentioned previous SOTA. We include all subsets of ETT and PEMS. The performance is also averaged from four prediction lengths with each containing five runs of random seeds. The standard deviations and test statistics are listed below, showing that **the performance is on par with the previous SOTA PatchTST and SCINet within the margin of error**.

| MSE     | iTransformer | Previous SOTA | test statistic |
| ------- | ------------ | ------------- | -------------- |
| ETT     | 0.383±0.001  | 0.381±0.001   | 2.124          |
| Weather | 0.258±0.001  | 0.259±0.001   | -2.913         |
| PEMS    | 0.119±0.001  | 0.121±0.002   | -0.419         |

To our best knowledge, the benchmark of time series forecasting has been excavated for a long time. And PatchTST and SCINet as the SOTA on specific datasets has surpassed concurrent works by a large margin, **iTransformer can be on par with on their advantaged datasets while further go ahead on other chanllenging datasets** (such as ECL and Traffic), and thus achieves the comprehensive SOTA.

Reviewer comment (Reproducibility):
Can you please explain why the TiDE results are so different from those reported in their original paper?

Author response:
We carefully read through the forecasting settings of the TiDE paper and its official repo. The differences are listed here:

  * **Enlarged lookback window**: TiDE paper adopts the lookback length of $720$, while ours uses the length of $96$, following the unified long-term forecasting protocol of TimesNet.

  * **More epochs to train**: TiDE paper trains the TiDE model with $100$ Epochs, while we reproduce the results of all models with $10$ epochs.

We try our best to keep hyperparameters comparable for the fairness of experiments, such as adopting the same hidden dimension and the number of layers. **We also compare their performance under the same lookback length of TiDE, where iTransformer still achieves better results**, since our model can also benefit from an enlarged lookback window.

| MSE          | ECL       | ETTh2     | Exchange  | Traffic   | Weather   | Solar-Energy | PEMS03    |
| ------------ | :-------- | :-------- | :-------- | :-------- | :-------- | :----------- | :-------- |
| TiDE         | 0.160     | 0.309     | 0.092     | 0.445     | 0.171     | 0.303        | 0.216     |
| iTransformer | **0.129** | **0.301** | **0.086** | **0.349** | **0.158** | **0.191**    | **0.071** |