Fast and Slow Streams for Online Time Series Forecasting Without Information Leakage

Authors: Ying-yee Ava Lau, Zhiwen Shao, Dit-Yan Yeung
Primary area: learning on time series and dynamical systems
Keywords: online time series forecasting, concept drift, online learning

Abstract:
Current research in online time series forecasting (OTSF) faces two significant issues. The first is information leakage, where models make predictions and are then evaluated on historical time steps that have already been used in backpropagation for parameter updates. The second is practicality: while forecasting in real-world applications typically emphasizes looking ahead and anticipating future uncertainties, prediction sequences in this setting include only one future step with the remaining being observed time points. This necessitates a redefinition of the OTSF setting, focusing on predicting unknown future steps and evaluating unobserved data points.  
Following this new setting, challenges arise in leveraging incomplete pairs of ground truth and predictions for backpropagation, as well as in generalizing accurate information without overfitting to noise from recent data streams. To address these challenges, we propose a novel dual-stream framework for online forecasting (DSOF): a slow stream that updates with complete data using experience replay, and a fast stream that adapts to recent data through temporal difference learning. This dual-stream approach updates a teacher-student model learned through a residual learning strategy, generating predictions in a coarse-to-fine manner. Extensive experiments demonstrate its improvement in forecasting performance in changing environments. Our code is publicly available at https://github.com/yyalau/iclr2025_dsof.

Decision: accept (poster)
Reviewer scores: 8, 6, 6, 8

Reviews and discussion:
<thread>
creation date:
1729164317360

thread_type:
official_review

<thread_object>
creator: reviewer_oqzm

<content>
summary:
The paper corrects a fundamental bug in previous papers for Online Time Series Forecasting, proposing a definition of the OTSF task without data leakage. To solve arising challenges of this new scenario, the authors then propose a dual-stream framework (DSOF) for this task, which updates a teacher-student model through residual learning.

In detail, there are two data streams: slow and fast.
- The slow data streams are stored into a buffer in an FIFO basis for updating teacher and student residual model. This requires complete ground truth data for all time steps in data stream. This is called Experience Replay (ER) Update in the paper.
- The fast data stream, designed to utilize data from timestamps where the ground truth for predictions is not yet fully available, updates the student residual model only, according to discord of (the prediction of the teacher-student model in the previous step) and (the ground truth of this step along with the pseudo-label generated by the prediction of the teacher-student model at this step). This is called Temporal Difference (TD) Update in the paper.

strengths:
This work demonstrates several notable strengths.

- Most significantly, the authors point out a significant problem of several works for Online Time Series Forecasting related to information leakage. The bug pointed out here is such fundamental that I did not even believe it existed before I checked the code of several previous works for OTSF and verified the claim of the authors. The redefinition of OTSF problem would definitely make it clearer for the time series community to develop online models that are not only good on benchmarks, but also practical and useful.

In this data-leakage-free setting, there are data points in timestamps whose ground truth for prediction is currently not fully available, which is an arising problem.

- The authors propose the DSOF framework as a logical solution to this emerging problem. This framework will serve as a baseline for future comparisons. The proposed framework provides insights related to differential learning in reinforcement learning, making it more explainable.



- The authors evaluate their proposed framework with extensive experiments conducted across multiple Models (DLinear, FITS, FSNet, OneNet, iTransformer, PatchTST, NSTransformer, etc) and multiple Datasets (ECL, ETT, Traffic, Weather, Exchange, etc) on multiple settings (different forecasting horizons, different learning rates, etc), demonstrating the effectiveness of their method.



The significance of this work is apparent, especially in its correction to the OTSF problem practice, and also in its contribution to point out the arising problem, and propose a method to (partially) deal with this problem.

weaknesses:
A fixed look-back length of 96 may not be entirely convincing. Some models may show a significant advantage at a certain look-back length, but less advantage or even underperformance at longer look-back lengths. It would be better if the authors could follow the practice in the paper of FITS, PatchTST, etc (which are cited by the authors in their paper) to do grid search for the optimal lookback length for each dataset for each forecasting model, and to compare this optimal value against each other.

The authors' approach is understandable, given that (1) some previous work accepted at top conferences for time series forecasting uses similar settings of fixed look-back length and (2) this approach would require much larger computation resource. However, I believe it would benefit the time series community if new papers utilized grid search and comparing with baselines across different look-back lengths, though this would not affect the contribution and significance of this work.

questions:
- Could you conduct a grid search for different look-back horizons on selected datasets, models, and forecasting horizons to demonstrate robustness with respect to look-back horizon? I'm not suggesting that you repeat all experiments. Conducting grid searches for a subset, particularly with smaller models and datasets (e.g., DLinear, iTransformers on ETTh1, Exchange; H=24), would be computationally feasible and enhance the robustness of your results.

- I noted the experiments on different buffer-size in appendix. Would it be possible that these experiments could partially represent or replace experiments about different look-back horizon?

- Recent work has proposed Foundation Models for Time Series Forecasting, and there are papers discussing Test Time Adaptation for Time Series Forecasting. Would it be possible that the predictions of base models can be improved by the DSOF framework?
</content>
</thread_object>

</thread>
<thread>
creation date:
1729518590086

thread_type:
official_review

<thread_object>
creator: reviewer_7cqr

<content>
summary:
The paper addresses challenges in online time series forecasting (OTSF), particularly the issue of information leakage, where models use historical data for both prediction and evaluation, leading to biased performance assessments. The authors redefine OTSF to focus exclusively on predicting future unknown time steps without using backpropagated data for evaluation, aligning the framework more closely with real-world forecasting tasks.

strengths:
The paper presents a novel redefinition of online time series forecasting (OTSF), specifically addressing the issue of information leakage, which has not been adequately tackled in prior research. By focusing on predicting future time steps without evaluating on previously backpropagated data, the authors set a new standard for OTSF, ensuring a more realistic and reliable forecasting setting. The introduction of the dual-stream framework, DSOF, which combines experience replay (ER) and temporal difference (TD) learning, is an innovative approach that allows the model to adapt effectively to temporal shifts and generalize better to real-world scenarios.


The study is well-executed, with thorough experimentation that covers multiple datasets and a variety of model architectures. The results clearly demonstrate the advantages of the proposed DSOF framework over traditional batch learning methods, showing significant improvements across diverse real-world scenarios. The experiments also include ablation studies, highlighting the importance of core components such as ER and TD loss, thus providing a detailed understanding of how different design choices contribute to the model’s performance. The methodological framework is backed by rigorous empirical evidence, lending credibility to the findings.

weaknesses:
The results of the baselines, especially the comparison between OneNet and FSNet, are unexpected and raise concerns about the experimental setup. Specifically, OneNet, which is theoretically more robust and effective, shows worse performance compared to FSNet, especially on datasets like ECL, which have many variables. This discrepancy is puzzling because OneNet should typically outperform FSNet due to its more comprehensive handling of cross-variable and cross-time dependencies. The authors should clarify the hyperparameter settings used for baseline models and ensure that they have been adequately optimized for each baseline. A detailed explanation of why OneNet underperforms would be valuable, as it could indicate either a misconfiguration or potential issues in the experimental protocol. Additionally, including a sensitivity analysis for hyperparameters could help identify whether the results are consistent or if specific settings unduly favor certain models


While the paper argues that existing OTSF settings introduce information leakage, it overlooks that many practical use cases update the model immediately with each new data point rather than accumulating a sequence. Although the authors highlight the risks of information leakage, real-world scenarios often prioritize fast adaptation to incoming data, even if it means some overlap in training and evaluation periods. This practice, while not perfect, remains a common approach because it ensures the model quickly adapts to recent changes. The authors should address why the proposed method of avoiding information leakage offers a practical advantage, despite differing from common real-world practices. It would also help to show more use cases where avoiding this overlap leads to clear benefits, thus making a stronger case for why users should adopt the new setting. 

The paper does not provide a thorough analysis of the computational cost associated with running DSOF in real-time environments. Given the dual-stream approach and reliance on replay buffers, the method could potentially introduce higher computational demands, especially when dealing with large datasets or long lookback windows. Include a detailed assessment of the computational overhead introduced by the dual-stream approach. Specifically, measure the latency, memory usage, and training time, comparing them with baseline batch learning and other online learning frameworks. This would provide a clearer understanding of the trade-offs involved and help identify scenarios where DSOF is more efficient or where further optimizations are needed.

In Figure 3, the combined use of ER and TD loss leads to a significant performance boost, but the individual contributions of each are less apparent. This pattern is confusing because it suggests that the benefits of each component alone are minimal, raising questions about their necessity if they do not independently improve performance. Without a clearer understanding of how these components interact, it is difficult to assess their true value within the framework.   The authors should provide a more detailed analysis explaining why ER and TD loss, when used together, show such a marked improvement over their separate contributions. This might include examining how these methods complement each other and identifying specific scenarios where one is more effective than the other. Additional ablation studies or visualizations showing how each component contributes to different aspects of the model’s learning could offer deeper insights into their interactions.

questions:
.see weakness
</content>
</thread_object>

</thread>
<thread>
creation date:
1730448911243

thread_type:
official_review

<thread_object>
creator: reviewer_qe4w

<content>
summary:
This paper discusses a dual-stream framework for online forecasting (DSOF). Two modules separate the learning process into two streams: a “slow stream” which updates with complete data, and a “fast stream” which adapts to the most recent data. Using a teacher-student framework， the proposed model aims to improve forecasting performance by balancing adaptation to new data with robustness against noise. Numerical experiments are provided.

strengths:
This paper tries to address an important topic in time series forecasting via online model adjusting to hedge the distribution drifting issue.

weaknesses:
1. The presentation of this paper could be improved. According to the current draft, the novelty appears limited. The way to utilize teacher-student style models and experience replay seems quite standard. I suggest authors include some toy examples or a theoretical analysis to motivate the necessity proposed method from a more principled way.

2. Sample code is not provided to assist the reviewer in verifying reproducibility.

3. Can authors elaborate more on how the proposed model addresses the information leakage? It seems not being well discussed in methodology section (section 3).

questions:
Please see the weakness part.

At the current stage, the novelty is insufficient for acceptance at a top-tier machine learning conference, and I tend to recommend rejection. However, I am open to reevaluating this work based further discussion.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730671046732

thread_type:
official_review

<thread_object>
creator: reviewer_nzdp

<content>
summary:
The paper spots a problem in the training-evaluation pipeline of existing online time series forecasting studies. The problem is that  parts of the future sequence has already been used as labels for updating the model at previous time-steps. This results in information leakage in training. This information leakage results in an overly optimistic training loss and consequently a gap between training and causal evaluation (the latter is what we really care about). The main contribution of the paper is identifying this shortcoming of the previous works. The paper also presents a fast-slow dual network algorithm for OTSF in a setting with no information leakage. They also present comprehensive experiments to showcase the performance of their algorithm.

strengths:
The identified information leakage problem is important, and I think would be influential for the field.

weaknesses:
The fast learning part (Section 3.3.2) motivates by the necessity of fast updates to stay up-to-date with the recent data. For this purpose they update the student model using labels generated by the teacher. The teacher model is however out-dated. This seems like a contradiction, and it is not clear how it may resolve the problem of being up-to-date, except for the fact that the immediate next label, x_{t+1}, is used in updating the student. This seems very small updates 

Alternatively, student updates could leverage k next real data for some k>1, at the cost of a delay of k time-steps. The student of the current work uses k=1. Alternatively, one could consider a whole spectrum (or hierarchy) of models with different values of k (for example powers of 2). 

Computational complexity: the DSOF algorithm involves two networks. Moreover, the teacher model requires to generate a long sequence of samples at each time-step, which can be very computationally costly in certain architectures like decoder transformers. I wonder if the reported outperformance could be achieved under the same computational budget as the baselines?


Comments on the presentation:

The presentation could be much better. The text involves quite long sentences that reduce fluency, resulting in a tiring read. Fig. 1 does not help much in clarifying the information leakage problem. Despite of extensive use of indices, the arguments are vague in several places; for example in line 45 “the whole sequence” could be replaced by the indices you defined in the same sentence. 
Appendices are missing.

The claimed new OTSF framework is not rigorously formalized. For example what data can or cannot be used at each time of the training to prevent information leakage. 

The connection to RL and TD learning in section 3.3 seems very loose. It would be better not to use TD terminology. 

The Appendices are missing from the pdf.

questions:
Please see the comments under the Weakness section.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732274367164

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Summary of Paper Revision

comment:
The following amendments have been made to the manuscript:

1. The appendix is now included at the end of the main text. Previously, it was separated into the supplementary materials, which is inconvenient due to broken hyperlinks.

1. Purple text indicates revisions made to the paper.

1. Description of information leakage problem in OTSF
    1. Using concrete examples instead of mathematical notation to describe the problem for easier understanding [*Section 1.1, line 041*]
    1. New figure for illustrating the information leak problem [*Section 1.1, line 062*]
    1. Moving the formal definition of the redefined setting from introduction to methodology [*Section 3.1, line 191*]
    1. Moving the formulation of the existing setting used in previous OTSF works from introduction to appendix [*Section C, line 1237*]

1. MSE Comparison of Batch learning and DSOF
    1. Updated Results of OneNet [*Section 4.1, line 378*]
    1. Reporting both mean and standard deviation of repeated experiments [*Section E.1, line 1289*]

1. MSE comparison of DSOF with other online learning frameworks
    1. Change of name: the online learning framework used by FSNet and OneNet is renamed from "Prev." to "DGrad".  [*Section 4.2, line 411*]
    2. Included other online learning frameworks, TFCL and DER++, as baselines [*Section 4.2, line 419*]
    1. Runtime (wall time and processor time) comparison of DSOF [*Section A.1.1, line 686*] 
    1. GPU memory usage comparison [*Section A.1.2, line 701*]

1. Ablation studies for the main components in DSOF
    1. The baseline for the ablation study, i.e. the case where both components are not used, is modified. Previously the baseline was batch learning, but now is replaced with DGrad. [*Section A.2.1, line 725* and *Section A.2.2, line 751*]
    1. Included case studies for the interaction of ER and TD loss components [*Section A.2.3, line 772*]
    1. Included runtime (wall time and processor time) comparison [*Section A.2.4, line 794*]

1. Added experiments for testing different batch sizes used in ER procedure. [*Section A.5.2, line 969*]
1. Introducing a parameter $k$ for constructing pseudo labels [*Section A.6.2, line 1100*]
1. Experiments on different look-back window lengths [*Section A.8, line 1173*]


Pending revisions for the upcoming versions of the paper:
1. About the connection of TD in RL and DSOF:
    1.  In the introduction section of the paper [*Section 1.2, line 109*], mention that TD learning serves as an inspiration.
    1. In the methodology section [*Section 3.3, line 230*], clarify that the use of the "TD" terminology is intended to highlight the process of intermediate updates.
    1. Update the introduction of [*Section B, line 1208*] to highlight the common challenges encountered by both TD learning and OTSF, and to explain how the principles of TD learning can be connected to OTSF.
    1. At the conclusion of [*Section B, line 1236*], add a final remark to emphasize that this section does not provide an exact formulation of the OTSF problem as an RL problem, due to the inherently different objectives of the two fields, but rather seeks to enhance OTSF performance with techniques in TD learning.

2. Mention the introduction of parameter $k$ in pseudo label construction in [*Section 3.3.2, line 264*].

3. In [*Section A.6.2, line 1101*], insert a subsection to demonstrate the effects of altering values of $k$.

4. To maintain the formality of the prediction setting as described in [*Section 1.1, line 041*], integrate both mathematical notations and a practical example.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper identifies a common bug for evaluating online time series forecasting in which future sequence data used to evaluate predictions has been used to update the model at previous time steps. The authors also introduce a teacher-student model framework to update a model with real-time data, thus preventing data leakage during evaluation while allowing models to update incoming data.

Overall, this paper is quite significant because it identifies a very prevalent bug affecting the evaluation in a subfield. This contribution is unambiguously significant and presented well in the paper. The primary concerns about this paper relate to the student-teacher framework. The paper did not present the method clearly, and it was not obvious how the proposed method addressed the other identified problem of information leakage.

The authors made significant revisions to improve the paper's readability. The results evaluate online time series forecasting without information leakage, which is an important contribution in and of itself. For this reason, as well as the significance listed above, I vote that this paper be accepted.