TimeBridge: Non-Stationarity Matters for Long-term Time Series Forecasting

Authors: Peiyuan Liu, Beiliang Wu, Yifan Hu, Naiqi Li, Tao Dai, Jigang Bao, Shu-Tao Xia
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords: Long-term time series forecasting, Non-stationarity, Dependency Modeling

Abstract:
Non-stationarity poses significant challenges for multivariate time series forecasting due to the inherent short-term fluctuations and long-term trends that can lead to spurious regressions or obscure essential long-term relationships. Most existing methods either eliminate or retain non-stationarity without adequately addressing its distinct impacts on short-term and long-term modeling. Eliminating non-stationarity is essential for avoiding spurious regressions and capturing local dependencies in short-term modeling, while preserving it is crucial for revealing long-term cointegration across variates. In this paper, we propose TimeBridge, a novel framework designed to bridge the gap between non-stationarity and dependency modeling in long-term time series forecasting. By segmenting input series into smaller patches, TimeBridge applies Integrated Attention to mitigate short-term non-stationarity and capture stable dependencies within each variate, while Cointegrated Attention preserves non-stationarity to model long-term cointegration across variates. Extensive experiments show that TimeBridge consistently achieves state-of-the-art performance in both short-term and long-term forecasting. Additionally, TimeBridge demonstrates exceptional performance in financial forecasting on the CSI 500 and S\&P 500 indices, further validating its robustness and effectiveness. The code is available in the supplementary material.

Decision: reject
Reviewer scores: 8, 3, 3

Reviews and discussion:
<thread>
creation date:
1730520235904

thread_type:
official_review

<thread_object>
creator: reviewer_ns1f

<content>
summary:
This paper introduces TimeBridge for predicting non-stationary time series, aiming to resolve the contradiction between short-term fluctuations and long-term trends. By dividing the input sequence into small segments and applying an integrated attention mechanism to reduce short-term instability while utilizing joint attention mechanisms to preserve non-stability to capture long-term co-integration relationships across variables, TimeBridge effectively captures stable dependencies without introducing pseudo-regression risks. Experimental results demonstrate that TimeBridge performs well on multiple tasks, particularly outperforming current state-of-the-art methods in financial time series prediction. Additionally, through a series of ablation studies, the importance of removing or preserving non-stability and the order of integrated and joint attention modules is further validated, revealing optimal model configurations under different dataset backgrounds. This work not only enhances the accuracy and robustness of time series prediction but also provides new perspectives and methods for handling non-stationary data in the future.

strengths:
- Resolving the conflict between short-term fluctuations and long-term trends: TimeBridge can effectively capture stable dependencies by reducing short-term non-stability through dividing the input sequence into small fragments and applying integrated attention mechanisms, while utilizing joint attention mechanisms to preserve non-stability to capture cross-variable long-term co-integration relationships.

- Improving the accuracy and robustness of time and sequence prediction: TimeBridge performs well on multiple tasks, particularly outperforming current state-of-the-art methods in financial time series prediction.

- Providing new perspectives and methods for handling non-stationary data in the future: Through a series of ablation studies, this article further verifies the importance of removing or retaining non-stability, as well as the sequence of integrated and joint attention modules, and reveals the optimal model configuration under different datasets, providing new perspectives and methods for handling non-stationary data in the future.

weaknesses:
- The paper lacks sufficient originality, as it primarily builds upon existing methods without presenting a clear, novel contribution to the field. This limits the manuscript's potential impact and reduces its value as an advancement in research.

- The lack of experiments with varying input/output (I/O) ratios raises concerns about the fairness of the comparisons. Different I/O settings can affect model outcomes substantially, and without exploring these variations, the paper provides an incomplete evaluation of model performance.

- The authors have only considered MSE and MAE as metrics for their long-term forecasting task, which is insufficient. This limited evaluation may obscure important aspects of model accuracy, particularly in cases where relative error size is a crucial factor for assessing model effectiveness.

- The paper lacks a clear description of the hyperparameter tuning process for the comparative algorithms, which raises concerns about the comparisons. Without adequate tuning, it is difficult to ascertain if the reported performance differences are truly reflective of each model's capabilities.

questions:
- The experimental settings are not reasonable. Why input length is set to 720, output length O is set to 96, 192, 336, 720 and not something more practical, like 1 month of data? 

- The paper does not account for the impact of varying input/output (I/O) ratios, which can significantly influence model performance. Fixing different input length for baselines without comprehensive experiments may lead to biased comparisons and limit the robustness of the results. The author needs to report the results of the baseline under the same experimental setup.

- The paper lacks an evaluation of the algorithm's complexity, particularly with respect to its theoretical complexity, GPU resource cost, and runtime efficiency. Without this assessment, it is difficult to gauge the algorithm's practicality and scalability, especially in resource-constrained environments.

- The absence of detailed hyperparameter optimization for the baseline algorithms undermines the fairness of the comparisons presented in the paper. Without this tuning, the results may not accurately represent the optimal performance of the comparative models, potentially leading to biased conclusions.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730709823153

thread_type:
official_review

<thread_object>
creator: reviewer_ac7s

<content>
summary:
The paper presents TimeBridge for non-stationary time series forecasting. TimeBridge utilizes Integrated Attention to mitigate short-term non-stationarity and Cointegration Attention for modeling long-term non-stationarity. Experiment results on benchmarking time series demonstrate the effectiveness of TimeBridge.

strengths:
+ Non-stationarity is a major challenge in time series forecasting. The paper aims to address this important task
+ Experiment results are encouraging, with a comprehensive ablation study.

weaknesses:
- The idea of Integrated Attention and Cointegration Attention is not new. The model in general lacks novelty. 

- The improvement by TimeBridge is not as much as claimed in the paper (over 10%). On most of the datasets, TimeBridge achieves similar results or marginally better results (Table 1 and 2). In general, the popular benchmarking time series are relatively easier tasks. On finance applications, TimeBridge's performance is similar to TSmixer.

questions:
What is TimeBridge's performance on bigger time series dataset, such as New York Taxi or Climate Data?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730734735787

thread_type:
official_review

<thread_object>
creator: reviewer_5kbc

<content>
summary:
The paper introduces TimeBridge, a framework for multivariate time series forecasting that addresses the challenge of non-stationarity by differentiating between its impacts on short-term and long-term modeling. TimeBridge utilizes Integrated Attention to manage short-term fluctuations in batches, reducing spurious regressions and capturing local dependencies. Cointegrated Attention is introduced to preserve long-term non-stationarity across variates, enabling effective long-term dependency capture. Experiments on the CSI 500 and S&P 500 indices verify the short- and long-term forecasting performance. It is generally a method for handling nuanced non-stationarity effects in complex multivariate scenarios.

strengths:
1. This paper is well written. The notations are clear.

2. It provides up-to-date literature on learning techniques for capturing stationary/non-stationary and dependency in sequential data. It combines the treatments of non-stationarity and dependency modeling in one shoot and delivers convincing performances.

3. The notion of cointegration of time series has been missing or forgotten in recent time series forecasting literature; this paper showed how cointegration could help reveal the non-stationary part when multiple time series evolve simultaneously. 
In Operations Research, cointegration is a well-established technique, and it has been introduced to machine learning literature for long:

Marco Cuturi, Alexandre D’Aspremont (ICML, 2013). https://proceedings.mlr.press/v28/cuturi13.html

The authors may refer to basic cointegration techniques to resolve the computational challenges or benchmark the extraction of stationary/non-stationary movements.

4. The experiments are comprehensive and cover recent state-of-the-art competing methods. The results are convincing, as shown by a solid ablation study showing the contribution of the building blocks, e.g., Integrated Attention and Cointegrated Attention.

weaknesses:
1. According to Figure 3, the building blocks of the proposed methods are streamlined with no conjugation. In some sense, this is brute force and remains room for improvement or further technical development, speaking of the systematic organic treatment of the non-stationarity and dependency modeling in long-term time-series forecasting.

questions:
Q. As mentioned in Strength 3, it would be interesting to recap traditional techniques for cointegration in operations research and benchmark the stationary or non-stationary time series before discussing the impact of an attention-powered module.
</content>
</thread_object>

</thread>


Metareview:
metareview:
**(a) Summary**

This paper addresses the task of long-term time series prediction by tackling the challenge of incorporating both short-term and long-term dependencies. The proposed method, called TimeBridge, combines two types of attention mechanisms: integrated attention on patches of time series for short-term modeling, followed by cointegrated attention to capture long-term dependencies. The empirical effectiveness of TimeBridge has been evaluated on multiple real-world datasets.

**(b) Strengths**

- **Relevance and motivation:** The paper is well-motivated, as the challenge of effectively modeling both short-term and long-term dependencies, particularly considering the non-stationarity of time series, is a critical problem in time series prediction.
- **Architecture design:** While the individual components of the proposed architecture are not entirely novel, their combination into the overall framework is unique and can be a valuable contribution to the community.

**(c) Weaknesses**

- **Hyperparameter tuning**: Hyperparameter optimization lacks sufficient analysis and discussion despite its importance to empirical performance. The explanation provided (e.g., Lines 905–906) is insufficient. Given the inherent challenges of hyperparameter tuning in time series prediction, a more thorough exploration of the process and sensitivity analysis with respect to key parameters is essential.
- **Datasets and experimental settings:** There is a notable error, which is the claim that 15 minutes * 96 equals one month for the Weather dataset, while in fact this corresponds to one day. This issue was raised during the AC-reviewer discussion phase and highlights the need for more rigorous verification of the dataset domains and associated details.
- **Empirical performance:** The empirical results are not convincing compared to existing approaches, as noted by multiple reviewers. Further evidence is needed to justify the superiority of the proposed method. Please also see my comment on (d).
- **Presentation:** There are several issues in paper’s readability and overall quality, which I strongly recommend to address throughout the paper. For example, "Patching(X)" is defined in Equation (1) but never used in the paper, and the matrix $\mathbf{P}'\_{c,:}$ is defined as $\\{\mathbf{p}\_1', \dots, \mathbf{p}\_N'\\}$ in Equation (2) but it means a set, where duplication is not allowed by definition and the size can be smaller than $N$, hence it is not appropriate to use it to define a matrix. In Equation (3), the function $\text{Attention}(\mathbf{P}\_{c,:}', \mathbf{P}\_{c,:}', \mathbf{P}\_{c,:})$ is not mathematically defined in the paper (although it can be inferred). While each issue may seem minor, collectively they significantly impact the paper’s clarity and quality.


**(d) Decision Reasoning**

I have carefully reviewed the rebuttals and the paper. While I acknowledge that the architecture design demonstrates a novel combination of existing components, the key question remains whether this offers new theoretical or practical insights to the community. Since there is no theoretical analysis in the paper, the practical impact becomes central to its value. Unfortunately, unresolved concerns regarding hyperparameter tuning, empirical performance, and presentation diminish the paper’s strength as a standalone practical contribution.

I believe this paper has potential but requires a major revision to address these issues. A carefully revised and resubmitted version could make a stronger contribution to the field.