ODEStream: A Buffer-Free Online Learning Framework with ODE-based Adaptor for Streaming Time Series Forecasting
Authors: Futoon M. Abushaqra
Hao Xue
Yongli Ren
Flora D. Salim
Venue: Accepted by TMLR ()

Abstract: Addressing the challenges of irregularity and concept drift in streaming time series is crucial for real-world predictive modelling. Previous studies in time series continual learning often propose models that require buffering long sequences, potentially restricting the responsiveness of the inference system. Moreover, these models are typically designed for regularly sampled data, an unrealistic assumption in real-world scenarios. This paper introduces ODEStream, a novel buffer-free continual learning framework that incorporates a temporal isolation layer to capture temporal dependencies within the data. Simultaneously, it leverages the capability of neural ordinary differential equations to process irregular sequences and generate a continuous data representation, enabling seamless adaptation to changing dynamics in a data streaming scenario. Our approach focuses on learning how the dynamics and distribution of historical data change over time, facilitating direct processing of streaming sequences. Evaluations on benchmark real-world datasets demonstrate that ODEStream outperforms the state-of-the-art online learning and streaming analysis baseline models, providing accurate predictions over extended periods while minimising performance degradation over time by learning how the sequence dynamics change. The implementation of ODEStream is available at: \url{https://github.com/FtoonAbushaqra/ODEStream.git}.

Published in Transactions on Machine Learning Research (03/2025)
ODEStream: A Buffer-Free Online Learning Framework with ODE- Adaptor for Streaming Time Series Forecasting
Futoon M.Abushaqra
School of Computing Technologies
RMIT University
Hao Xue
hao.zucl Grunsw. edu.an
School of Computer Science and Engineering University of New South, Wales
Yongli Ren
yongli.ren@rmiit edu.
School of Computing Technologies
RMIT University
Flora D.Salim
flora.salim Sunsw. edu..ox
School of Computer Science and Engineering
Universily of New South Wales
Reviewed on OpenReview: https: // openreview. net/ forum? id= 7WDTKhuU5m
Abstract
Addressing the challenges of irregularity and concept drift in streaming time sories cru- cial for real-world predictive modelling. Previous studies in time series continual learning often propose models that require buffering long sequences, potentially restricting the re- sponsiveness of the inference system. Moreover, these models are typically designed for regularly sampled data, an unrealistic assumption in real-world scenarios. This paper in- troduces ODEStream, novel buffer-free continual learning framework that incorporates temporal isolation layer to capture temporal dependencies within the data. Simultanc- ously, it leverages the capability of neural ordinary differential equations to process irregular sequences and generate a continuous data representation, enabling seamless adaptation to changing dynamics data streaming scenario. Our approach focuses on learning how the dynamics and distribution of historical data change over time, facilitating direct process- ing of reaming sequences. Evaluations benchmark real-world datasets demonstrate that ODEStream outperforms the state-of-the-art online learning and streaming analysis baseline models, providing accurate predictions over extended periods while minimising performance degradation over time by learning how the sequence dynamics change. The implement a aton of ODEStream available at: hhtpp:://// http://hhhh...///oooooouo//////e git
1 Introduction
In today's 's data-driven world, the rapid growth of interconnected devices and systems has led to explosion of time series data generated in real time. These data streams offer invaluable insights across diverse domains, including finance (Kovacs el al.) 2021 Sezer et al.] 2020), healthcare (Zhangla et al. 2021 Kaushik et al. 2020), environmental monitoring (Deuur et al. 2011), and industrial processes (Maschler et al., 2020). Being able analyse the strcaming data in real-time provides more accurate and up-to-date information. allowing us react changes as they happen and adding a significant value the outputs (Almeida al. 2023).

---

Published in Transactions on Machine Learning Research (03/2025)
Handling streaming analysis requires an effective continual online learning approach. Recently few studies have addressed streaming analysis for regression and forecasting (Cossu et al. 2021), by relying on buffer memory to apply a replay method for continual learning (Matteoni et al. 2022 Kwon et al. 2021 Kiyasseh ct al. 2021: Xiao et al. 2022 Chen et al.] 2021). Using the replay method, a training set initialises the model's 8 paramelers, and a selected subset of this data, as well as future incoming data samples, is saved to be used later when the model is retrained (Hoi et al. 2021 Fekri et al.) 2021 He 2021). This process focuses on avoiding catastrophie forgetting issues by continuously updates the model knowledge while retraining on both incoming and historical data. However, since this method is effective at preserving previously learned information, its performance on time series data is uncertain. By focusing mainly on avoiding forgetting. it may fail to adequately capture new and evolving patterns in the data over time (Cossu et al.) 2021 Pham et al. 2023)). Additionally, these replay-based frameworks introduce significant complexity, as they must carefully determine which data samples to store in the buffer, when to update importance weights, and how to allocate memory and computational resources efficiently. These challenges highlight the neod for a more streamlined and efficient approach to managing streaming time series data. Furthermore, the impact of catastrophic forgetting differs significantly in time series data and predictive modelling compared to non-temporal data and classification tasks, where class- and cask-incremental learning are more common scenarios (Ao & Fayek 2023; Pham et al. 2023). In such scenarios, the model's objective is to learn new tasks or classes while ensuring that previously learned ones are not forgotten. On the other hand, when developing a lifelong learning model for streaming time series data, a key consideration emerges: to what extent should we prioritize preserving historical information, especially when that data may have evolved or changed over time (Cossu et al. 2021 Pham et al. 2023).
Strcaming time scries forccasting faces significant challenges, including (1) Concept Drift Phenomenon, where underlying patterns or relationships within the data change over time, posing a substantial challenge for predictive modelling &Fekri et al., 2021). This issue extends beyond online learning models to impact batch learning approaches, as models initially proven accurate can quickly become obsolete as the data con- tinues to evolve. (2) Temporal Irregularity, which is common in streaminn data, especially in real-world applications where the data lacks a specific time frame. (3) Balanced Adaptation, where maintaining the right balance between historical and recent data is important while effectively adapting to the dynamic nature of streaming time series data. Therefore building robust framework that can adeptly address these challenges is essential for maintaining model accuracy over time.
preventing biases towards outdated data
This paper aims to overcome the restrictive boundaries of streaming time series processing by introducing a novel online learning model (ODEStream). Our focus lics on enabling the model to adapt to evolving data patterns without relying on a complex framework while maintaining good performance over time. Specifically, we investigate how the novel neural ordinary differential equation (ODE) methods can be leveraged for memory-free online forecasting for streaming time series data. The key contributions of this paper are as follows:
Unique Perspective: We redefine the context of contimuous time series forecasting problem by introducing novel perspective that prioritises learning the evolving dynamics and data distribu- tion, rather than relying on the preservation of historical samples. This shift enables our model to adapt more effectively to changes over time, offering improved performance compared to traditional approaches.
To the best of our knowledge, this work is the first to leverage neural Ordinary Differential Equation (ODE) models for continuous learning and online forecasting. Due to their ability to dynamically adapt and provide continuous hidden state, ODEs offer a more flexible approach compared to traditional discrete-time models, especially in handling irregularly sampled data.
Utilising Neural ODEs:
Streamlined Buffer-Free Framework: We introduce ODEStream, a buffer-fres continual learn- ing framework designed to have a straightforward and efficient training process. Unlike existing frameworks that require complex decision-making for upcoming data; such as defining thresholds, trigger values, managing buffered samples, ODEStream eliminates these requirements entirely.
2

---

Published in Transactions on Machine Learning Research (03/2025)
This streamlined design reduces overhead, simplifies training, and improves the model's usability in real-world streaming scenarios.
Comprehensive Evaluation: We evaluate ODEStream against state-of-the-art continual learning models and demonstrate that it effectively mitigates the impact of irregularities, adapts to data drift, and maintains stable performance over extended streaming periods.
2 Related Work and Background
2.1 Continual Learning
Continual learning, also known as incremental learning or lifelong learning in several articles (Ao & Fayek 2023), is the process of training a model on a sequence of tasks over time without forgetting previously learned tasks (Lopez-Paz & Ranzato 2017). The method should maintain a balance between preserving the knowl- edge acquired from prior tasks and acquiring new knowledge for future tasks while having restricted access to historical experiences (Grossberg 2013). There are several socnarios for continual lcarning systems, includ- ing Iettance-Incrmeentie Learning (IIL); Task-Incremental Learning (TIL), and Online Continual Learning (OCL), among others (Wang al. 2023). Regardless of the scenario in which continual learning is applied, the strategies and methods that have been used are summarised as Regularization, Replay, Optimisation, and Representation waieeentttiioobeeeeeee approaches (Wang et al. 2023
Replay- based, also known as Memory-based approaches, the most common for various fields and has proven to be highly effective. In these approaches, a subset of past examples is saved and can be replayed when
the model trained in the future, relying on dynamic external memory. An example of this approach is Experience Replay (ER), which relies on fixed-sized replay buffer (Lopez-Paz & Ranzato 2017 Chaudhry
et al.
2019a). ER methods are commonly used in reinforcement learning (Lin) 1992 Rolnick et al. 2019
Focrster et al. 2017), but they have also been applied to supervised lcarning (Chaudhry et al. 2019b Riemer
et al. 2019 Buzzega el al. 2020
Regularization approaches (Li & Hoiem 2017 Rebuffi et al. 2017 Kirkpatrick et al.) 2017 Aljundi et al,
2018) add explicit regularization terms balance the learning of the old and new tasks. They work by regulating the learning process through either penalising feature drift on previously learned tasks or reducing changes in previously learned parameters (Li & Hoiem 2017 Rebuffi et al. 2017 Kirkpatrick et al. 2017] Aljundi et al.] 2018). Regularization methods often require storing a static version of the old model for reference (Wang et al. 2023). Finally, Model-based methods are applied by changing network structure or applying multiple models to respond to different tasks (Fernando et al. 2017 Mallya & Lazebnikk 2018). This method faces a challenge with the potential growth of network parameters. A comprchensive survey on general continual learning methods is provided (Wang el al. 2023).
While extensive work has been conducted on continual learning, there has been limited focus on its application to time series. Recently, In (Sun et al. 2022 novel concept known Continuous Classification of Time Series (CCTS) was introduced, driven by the necessity for early classification of time series, as highlighted by Gupta et al. (2020). Following this, recent article has addressed the issues of CCTS [Sun et al. 2023). To evaluate the continuous learning methods on time series data, Maschler et al. 20221 2022) put several regularization methods undergoing testing for time series anomaly detection and classification; both studies' finding indicates that Online Elastic Weight (OEW) outperformed other regularization methods. In (Matteoni et al. 2002). ; the authors proposed new benchmark datasets for time series continual learning for classification human state monitoring; using the proposed data, the authors tested several continual learning methods and found that experience replay proves to be the most efficient approach, A similar conclusion was reached in (Kwon et al. 2021), where the main goal of the study is to check if methods developed for image-based continual learning classification will work on time series. The authors explored different replay and regularization- based methods on six datasets. The study found that storage is not an issue, then the replay method gives the best performance. Another comparison of continual lcarning methods was applied in (Cossu et al. 2021] where RNN's performance for continual learning was explored across various scenarios.
3

---

Published in Transactions on Machine Learning Research (03/2025)
In (Maschler et al. 2020), the authors presented a model for continuous learning fault prediction. The method was applied to the NASA turbofan engine dataset, and the results demonstrated the model's 's effective performance with distributed datasets without the need for centralised data storage. At the same time, two
replay mothods have rocently been presented for time series (Kiyassch et al.] 2021 Xiao et al. 2022). In (Kiyasseh et al.] 202T), CLOPS was proposed for clinical temporal data, (Xiao el al. 2022) proposed met hod for streaming traffic flow sensor data. Finally, the work (He & Sick [021] was the first to describe continual learning and catastrophic forgetting for regression tasks. The model utilised a neural network with buffers and was deployed with OEW consolidation. Then, the same authors extended their work and focused on power forecasting. In (Gupta et al., 2021), the author focused, for the first time, on multivariate time series and multi-task continual learning. They addressed the issues using a memory-based method for the seen data and proposed an RNN-GNN model.
2.2 Time Series Streaming Data
In general, online time series forecasting has primarily been studied using machine learning models (Jiménez-
Herrera et al.] 2023 Li et al.
2019 Chen et al.
2020
Pham et al. 2023
Melgar-García et al.] 2023 Cossu
ct al. 2021 Fekri ct al. 2021: ; Ao Fayck 2023)
and statistical forocasting techniques (Alborg Last 2018).
either using online-oflline learning (Chen et al.
2020
P Pham et al.; 2023
Melgar-García et al.] 2023]. or by
applying incremental learning (Alberg & Last
2018 Li et al.] 2019 However, using machine learning is
not sufficient for multivariate real-world data, where the temporal dependence and pattern are complex and hard to discover. Recent works have focused on online time series forecasting using deep learning. The work in (Wambura et al.] 2020) introduced a model named OFAT, which stands for One Sketch Fits All-Time Series. The method was designed for long-range forecasting in feature-evolving data streams, which involve data that changes over time and has varying features. The model employs deep neural networks to learn from a small sample of data streams and subsequently applies the learned palterns to new data streams. For that work, CNN is used to extract features from the input data, while the LSTM is used to predict future values of the output data.
In another work (Pham et al. 2023 the authors introduced a novel framework for online time series forecasting that combines the strengths of deep neural networks and Complementary Learning Systems (CLS). The framework, named FSNet, consists of two components: a slowly-learned backbone and a fast adapter. The backbone is responsible for learning long-term temporal dependencies, while the adapter is responsible for learning short-term changes. The two components are dynamically balanced to ensure that the model can adapt to both new and recurring patterns. When the data distribution changes, the adapter is updated to learn the new patterns. When the data distribution is stable, the backbone is updated to improve the accuracy of the prediction, while memory is used to store the historical data that the model has Iearned. Nevertheless, these works continue to face challenges when dealing with irrogular samples.
More Recent studies have focused on the temporal distribution shift problem. In (Du et al., 2021), the concept of Temporal Covariate Shift (TCS) was introduced, and the AdaRNN framework was proposed. AdaRNN combines Temporal Distribution Characterisation (TDC) and Temporal Distribution Matching (TDM) to adaptively manage distribution changes. While the experimental results show that AdaRNN significantly improves forecasting accuracy, it relies heavily on correctly segmenting the data into diverse periods. Bai et al. (2022) introduced the DRAIN framework, which also focuses on adapting to changing data distributions over time by addressing the challenge of temporal domain generalization. DRAIN uses a Bayesian approach to jointly model data and neural network dynamics. Tt employs a recurrent graph gencration sccnario to encode and decode dynamic neural networks, capturing temporal drifts in both data and model parameters. A major drawback of the model is its requirement to encode and decode the entire network parameters, which adds significant complexity to the model, making it computationally intensive.
Another online learning algorithm for sequence data is Real Time Recurrent Learning (RTRL), which was recently revisited by (Irie el al., 2023) to improve its computational and memory requirements. The authors highlight RTRL's conceptual advantages over Backpropagation Through Time (BPTT) along with the time and space complexity challenges. The paper explores approximation thcorics and tests RTRL realistic settings and proposes a Real-Time Recurrent Actor-Critic (R2AC) framework that uses RTRL instead of standard Backpropagation Through Time (BPTT) for RNNs
4

---

Published in Transactions on Machine Learning Research (03/2025)
Recent work has also focused online variational methods for state estimation in dynamic systems (Dwwling et al. 2023 Campbell et al. 2021). In (Dowling et al. 2023), the authors introduce the exponential family variational Kalman filter (eV ()VKF), an online recursive Bayesian method that infers latent neural trajectories while learning the underlying dynamical system. The cVKF effectively handles arbitrary likelihoods and offers a closed-form variational analogue to the Kalman filter's prediction step, resulting in lighter ELBO bounds. Furthermore, in (Campbell et al., (202T) a method for online state estimation state-space models (SSMs) was presented. The model operates by utilizing backward decompositions of the joint posterior distribution and Bellman-type recursions to maintain constant update costs.
2.3 Neural ODEs
Neural Ordinary Differential Equations (Neural ODEs) (Chen et al. 2018 represent a significant advance- ment doep lcarning by integrating differential equations and neural networks. They provide a flexible framework for modelling complex dynamical systems, treating neural networks continuous processes. Re- cent. research in this field has made remarkable progress, showcasing the effectiveness of neural ODEs in various applications, especially for modelling irregularly sampled time series and partially observed data such as ODE-RNN, Latent-ODE, NJ-ODE and GRU-ODE models (Rubanova et al. 2019 Herrera et al., 2020: De Brouwer et al. 2019: Abushaqra et. al.; 2024). These models leverage the ODE's capability to pro- vide continuous hidden states, unlike the traditional discrete approach used in traditional sequence models such as RNN.
In recent years, several models based on Differential Equations (DF) have been introduced to address different
issues, including modifying hidden trajectories (Kidger et al.] 2020 Morrill et al.) 2021 Schirmer et al., 2022)
reducing computational overhead (Habiba & Pearlmutter 2020). However, these models have typically been evaluated and applied in batch learning and fixed datasets; hence, we do not consider them in this work. Despite the progress in applying neural ODEs to various domains, a critical gap exists in their application to strcaming time series data. Existing works primarily focus on batch learning and fixed datasets, lacking exploration of the dynamic and evolving nature of streaming data. The intricacies of adapting neural ODFs real-time processing of continuously changing data remain unexplored. Our research aims bridge this gap by introducing ODEStrcam, novel framework for continual learning in streaming time scries, effectively utilising the strengths of neural ODEs in this challenging context. Moreover, while recent work (Gannnone et al. 2020) proposed real-time classification from event-camera streams using neural ODEs (INODE), its focus on high-frequency event data and real-time classification does not comprehensively address the broader spectrum of challenges associated with streaming time series prediction, which we aim to tackle in this study.
3 Preliminaries
In this section, we introduce the fundamental principles of ODEs-based models and ODE solvers, exploring the theory behind neural ODEs and their role in representing irregular time series.
ODE (Ordinary Differential Equation): An ODE is a mathematical equation that describes the rate of change of variable with respect to an independent variable, typically time. In the context of time series data, ODEs represent the dynamics and interactions between variables over time. A simple example of an da(t) f(((t),)) t), where stt) denotes the state of the system at time t, and f(x((), specifies the ODE is: dt
rate of change at that time.
Solving an ODE involves determining the function æ(t) that satisfies the given differential equation. ODE solvers are numerical methods that approximate the solution over a defined time interval by numerically integrating the ODE using discrete time steps. Starting from an initial condition a(1o), an ODE solver estimates the values of (t) at subsequent time points.
ODE Solvers:
Neural ODEs: Neural ODEs extend traditional ODEs by parameterise the function f( t) with a neural network. This allows for modelling complex, continuous-time dynamics in data-driven manner. The
5

---

Published in Transactions on Machine Learning Research (03/2025)
Transfer
parameters
Sequence data for initial training
Encoder
Decoder
Streaming signals
Dynamics adaptive network (Fine-tuning)
Update
parameters
Latent distribution:
Temporal isolation Layer
Py
NeuraL-DE
KNH
Fully connected layer
P
Sampled representation
Prediction
E 10,
p ~ (Z Zo)
Pre-trained model
Figure 1: ODEStream Framework: which leverages Neural ODEs to encode prior knowledge during the initial training phase. Subsequently, the learned model parameters are transferred and complemented by the temporal isolation layer for continuous online learning. In real-time, streaming data samples are used predict future values, while the model simultaneously updates its parameters adapt newly observed data.
formulation of neural ODE is:
dh(t) So(h()))), where h((l)) ho d,
(1)
ho...., hnp ODESolve(fo. hos (to...., tn)),
(2)
where fo is neural network function with parameters 8, and h(t) represents the system's state at time t. Given the function Sa and an initial condition ho, the ODESolve function computes the values of h at the sequence of time points to...., ln-
4 Methodology
Problem statement: Let consider dynamic environment where streaming time series data is contin- uously generated as X = (2.,,, to develop continual lcarning system for time scries forecasting that adapts to the evolving nature of the data stream. Each observation Ie represents the input at time / for univariate data, while & represents a vector of inputs at time for multivariate data, such as 2, = [1e1, Ft2; ..., Ital, where Ztd the value of the 4t feature time The objective predict the future values the time series. The challenge lies the continual learning setting, where the model must adapt and learn from the continuously incoming data without access to past observations.
with observations arriving in sequence time. The goal is

---

Published in Transactions on Machine Learning Research (03/2025)
4.1 Neural ODEs for Continual Learning
Our goal is to design training method for memory-free continual learning models. In the context of online learning, we aim for our model to keep learning and adapting as new data arrives without forgetting previously learned knowledge. Traditionally, complex models with buffers are used to store a subset of past data to tackle this issue. As discussed, time series continual learning studies have shown that hemory-based approaches often achieve the best performance, particularly in classification tasks. These models continuuuly monitor incoming data and retrain on both old and new samples to maintain accuracy in dynamic environments, as demonst.ratdd by Fekri et al. (2021). However, When these models are applied to continuous data streams, particularly those with gradually evolving distributions, they face significant challenges. The main issue is that the model only has one opportunity to train on each data sequence, unlike batch learning. Furthermore, in the presence of evolving temporal patterns, it remains unclear how the information stored in the buffer can contribute to future prediction tasks.
After studying the neural ODEs method, we found that it can provide a foundational network for online learning due to the following characteristics (Chen et al.) 2018): (i) Continuous-depth representation: Neural ODEs allow the modelling of flexible continuous-depth neural networks. Unlike traditional discrete- depth neural networks, neural ODEs define the network architecture as continuous function, making them suitable for lasks where the depth of the network is not predefined. This means they can adapt their depth and complexity as new data points become available. (ii) Memory efficiency: Neural ODEs do not require storing a large number of intermediate activations, which is valuable when processing streaming data with limited memory resources. (fi)) Handling irregular sampling: Neural ODEs can handle irregularly sampled time series data, which is common in streaming scenarios where data arrive at uneven time intervals or when the features have different temporal representations. This flexibility ensures that the model can adapt to the timing of incoming data points.
4.2 Overview of ODEStream
We proposed a streamlined model for continual learning and forecasting of time series, consisting of two main phases. (i) an offline learning phase, commonly referred to as model warm-up, designed to initialise the model parameters. (di) online learning and forecasting, which generate prodictions based on new sequence flow while adopting and learning from the new changes in the data. The first phase is instrumental in obtaining prior knowledge of the available time sequences. For this part, the model utilises variational autoencoders to capture the distribution, and temporal dependencies of historical data, and an ODE is employed to model evolving dynamics. The trained model is then transitioned to online learning, functioning as a dynamic adaptive network for continuous analysis of streaming sequences. In this primary phase, we leverage pre- trained Neural ODE weights. The model continuously adapts to changes in the data distribution by updating its understanding of the underlying dynamics in real time. As shown in Figure [ It noteworthy that the model franework lacks external memory for storing samples, and it does not include a decision node or threshold values to control the flow of the learning process. In the following sections, we provide a detailed description of the model framework.
4.3 Obtaining Prior Knowledge (Model warm-up)
Online learning is more complex than batch learning, where the model learns feature representations from sample data and applies them to unseen data. The primary objective of the streaming model is to forecast future values using specific horizontal windows over an extended period. Since the data is unavailable during training, the model does not have the opportunity to process and learn from the sequences over many iterations. In this case, when the data distribution eventually changes, and the sequences evolve over time, the learned features from former historical soquences may no longer be helpful in predicting future values. In general, during the initial training phase of a continual learning model, often referred lo as the warm-up phase, the model is pre-trained on historical data from the entire previous period before online learning commences. However, as discussed carlier, this data may have alrcady changed or may change in the near future, so whatever the pre-trained model learns may not remain very relevant in the future. Therefore, relying solely on past data may not fully capture the evolving nature of the streaming sequences.
7

---

Published in Transactions on Machine Learning Research (03/2025)
For the ODEStream model warming-up phase, we do not warm up and train the model on forecasting task with the goal of learning the relationship between historical data and future time steps. Instead, we follow a new approach. Since the prior data contains information about the distribution, dynamics, and temporal dependencies, our focus lies in initialising our model by utilising the available data to understand how this data evolves and how the dynamics change over time. To achieve this, we leverage the variational autoencoders (VAEs) (Liu et al., 2020) model as shown the model framework in Figure 1 In contrast to traditional autoencoders, where an encoder layer maps the data x to a latent vector Z = =(x), in VAE, samples are encoded into distribution g( over the latent variable 2, rather than being transformed into single latent representation. The main idea behind VAE involves two key components; first, an encoder encodes the imput data x into a latent variable 2, capturing its representation distribution through a probabilistic approach parametrized with 0 (Kingma & Welling 2014). next, & decoder reconstructs the original imput data using the latent variable 2.
In detail, the encoder e, which parametrised by 0, produces the outputs: the mean Zμ and log variance Ze of normal distribution, typically assuming a prior p=z) a standard multivariate Gaussian distribution, denoted as NO0, 1). The following equation represents the probability function of the Gaussian distribution:
<(=|>) = (N Zo),
(3)
here, ((=/z) is the conditional distribution of the data a given the latent variable 2, and N is the Gaus- sian distribution with mean Zμ and standard deviation Zo. The latent variable 2 is sampled from the reparameterized distribution as:
Z = 2z + exp(0.. 2a),
(4)
where is a random sample drawn from a standard normal distribution (N 1). This sampled latent variable 2 then fod into the decoder, which generates the final output y by expressing p(x|2) as:
p(a|z) Nuy, og),
(5)
where My and of are the parameters of the decoder.
To obtain the dynamics and learn how the sequence distribution changes over time, we use neural ODEs. As ODEs can learn the dynamics over time, they are suitable for tracking changes in time series data. The RNN encoder processes the input sequence x to obtain the hidden state hg- The hidden state is then mapped to the mean Zp and log variance Zo discussed above using a linear layer. The latent variable a is sampled from a Gaussian distribution as: 200 ~ (2)x 2o)
Given sequential data sequence {.1, 22, .....,7 .r) with sequence length of T, neural RNN encoder processes each time step / to produce a hidden state h. The encoder estimates the parameters Zμ and Z of the variational posterior distribution (<2))1)) for the latent variable 2, Subsequently, fully connected layer maps the hidden state to ho, and samples 21 using the following equation:
240 ~ ((20 2...,,, to....................... tr: 0p) = (2200l 2
(6)
The latent trajectories, which represent the evolution of the latent variable 2, over time, are computed using ODE solver as Equation (using the Euler method in our case). This solver is employed to model how the latent space changes and adapts the sequential data is processed using the initial latent state 240- The dz = f(z, 1; 0y), is mathematical tool that describes how the latent variable changes concerning time t and is parameterised by 0,. Therefore, it captures the dynamic behaviour of the latent space over the entire sequence (the data for the initial training) before the online learning commences.
ODE solver, represented as ar
241, 242," ". 2T = ODESolve(2o. f,0, to. ,tr
4.4 Online Learning
Following the initial training phase, during which the neural ODE model learns from historical data, the model transitions into online learning. In this phase, the trained model is deployed as adaptive dynamic
8

---

Published in Transactions on Machine Learning Research (03/2025)
Time line of stream
ja
fn
ODEStremm
Temporal ieslation layer
Tempural ixolation loyer
Temporal insltioon layer
Dynamics adaptive network
Dynamies adaptive nerwork
Dynamics adaptive network
Model
Prediction
Prediction
Prediction
Figure 2: The continual learning process of ODEStream. The model processes the incoming observa- tion using the pre-trained VAE-ODE and continuously changes its parameters based on new dynamics.
network, enabling it to continuously process and analyse new streaming sequences they become available (as illustrated in Figure 2 For simplicity, denote the sequence X (xi+1, -. as X ). }, where represents the time point following i, assuming i denotes the last data point utilized the initial training phase.
{21, 23, Lj....
The main role of the adaptive network is twofold: first, to generate forecasts based on the recent available data Zjj and second, to adapt to changes in the data distribution. This adaptation is driven not only by prediction accuracy but also by the evolution of the data distribution, which will be discussed in Section 4.5)
The model architecture is intentionally designed to support continual understanding of how the underlying data dynamics evolve over time. This achieved by continuously integrating new data sequences through the neural ODE framework, allowing the model to detect and adjust to shifts in data distribution. Specifically, the pre-trained neural ODE weights and structure are reloaded to model the latent space dynamics =(j) of the streaming sample Zy, where j denotes the most recent value in the data stream. The latent space representation z(j) then linearly mapped to hidden space (hz), which subsequently maps linearly to the output space (3/) as defined by:
2(j) NeuralODE(2o..))
(8)
h Linear(z(f))
(9)
Yj Lincar(h,)
(10)
Based on the learned dynamics, the model parameters 0 are updated iteratively as Ouew = Bold 7 VeLoss
Temporal Isolation Layer: To further enhance the model's adaptability and responsiveness, we incorpo- rated it with specialised component known the temporal isolation layer. This neural network serves purposes: (i) Focus new information, it resembles dynamic "window" into the most recent data sam- ples. Its primary objective is to extract and learn from the unique temporal patterns and features embedded within the recent data. By closcly monitoring the look-back windows and forecast horizon of the data, this layer ensures that the model elfectively captures emerging Irends and patterns. (ii) Avoid historical bias, where, addition to the focus on new information, we also manage the influence past patterns learned by the pre-trained model that may have become obsolete or inaccurate. By independently processing and learning from the most recent data samples, this layer reduces the risk of the model being unduly guided by outdated historical data, ensuring its predictions remain relevant and reliable.

---

Published in Transactions on Machine Learning Research (03/2025)
The imput of this layer is the current stream sequence with specific look back window (lag) (Zy-iags..... ..... ,,.,,)))
where j is the most recent time point of a stream. As shown in Equation 11 and 12 the temporal isolation layer (f) generates the latent representation of that sample h2, and a concatenate layer (g) combines the latent reprosentation of the samples h2 along with the neural ODE-based representation hy of that sample to generale the final prediction. Ej is a cell state as we employ an LSTM for this layer
14 + (1 - (bg
(11)
(12)
Incorporating the temporal isolation layer allows the model to maintain a delicate balance between preserving valuable insights from historical data and staying attuned to the evolving nature of the data stream. This adaptive approach positions the model as a robust and adaptable tool for real-time forecasting and concept adaptation in dynamic environments.
4.5
Model Regularisation
To ensure that the model comprehensively grasps the underlying data distribution, we used Kullback-Leibler (KL) divergence loss as Equation 13 Csiszár 1975) to guide the pre-trained model and encourage the con- tinual learning process to adapt to distribution changes. KL divergence measures the disparity between two probability distributions, which facilitates the transformation of data into a latent space while encouraging the latent variables adhere hensively captures the data distribution, unlike the Mean Squared Error (MSE) loss, which typically focuses solely optimising output quality. However, the final output is future prediction, it is still important to evaluate the prediction performance of the model while learning new dynamics. Hence, combination of KL and MSE loss was utilised in the continual learning process.
specific distribution. In other words, it ensures that the model compre-
N LKL = -0.5 * Σ 1 + log - Z2g - -Za,
(13)
1=1
Furthermore, to control overfitting, we used regularisation methods that play a significant role in enhancing the robustness and generalisation of machine learning models and encouraging the development of simpler, more adaplable models. We used L1 regularisation in Equation 14 as an additional protection for enhanced model simplicity, L1 regularisation penalises the absolute values of neural network weights.
L1 = 0.01
2
(14)
1=1
Algorithm outlines the ODEStream online learning algorithm, in summary, the algorithm employs pre-trained Neural VAE-ODE model within an online learning framework, processing the streaming data sequence X = (2,, [;} and updating the model parameters as new data becomes available. The ODEStream algorithm operates iteratively for each incoming data sample Ij. integrating both the neural ODE representation and the temporal isolation layer to generate predictions and update model parameter. Initially, the model calculates latent representation =(j) of the streaming sample using the NeuralODE function, followed by a linear mapping to hidden and output spaces h,. Simultanconsly, the temporal isolation layer generates independent latent representation h/ for the new sample, isolating recent patterns and minimizing reliance on potentially outdated historical data. At each time step, the algorithm combines the representations h; and Ni using à concatenation layer to produce an updated hidden state h", yielding the final prediction 4/+1- The model parameters are then updated via gradient descent, optimizing with a composite loss function incorporating mean squared error (MSE), KL-divergence LKL, and L1 regularization CLI. This procedure ensures that the model adapts to new data patterns while preserving relevant insights from previous training.
10

---

Published in Transactions on Machine Learning Research (03/2025)
Algorithm 1: ODEStream Online Learning
Input: Pre-trained NeuralODE model, streaming sequence X = {I1 22, ,z4h previously learned
parameters 0
for each time step j in X {x1, f21,22,........... ;2, do:
=()) = NeuralODE(2o.))
h Linear(z(j))
h Cj t hf. Cy-1)
by = n(Kg.hi))
Uj+1 = Linear(1g)
Update model parameters
LKL = 00.5 Σ 1= (1 log( - Z2 %%,.)
LL = 0.01 Σ |zd
10
Loss CMSE + LKL LL
Onew = fold 1 VoLoss
13 end for
14 Output: Updated model parameters 0 and final prediction yj+1
5 Experiments
In our experiments, we aim to address three primary questions that collectively offer a comprehensive as- sessment of the model's performance:
Does ODEStream outperform other recent methods, including ER and online forecasting techniques?
How adaptable is the model to incoming data? How does it evolve over extended periods of streaming data, and how does concept drift in the data influence its performance?
Will ODEStream preserve ils performance even when applied to irregular and sparse streaming data?
5.1
Experimental Setup
5.1.1
Datasets
To evaluate the performance of our proposed model, employed several benchmark real-world forecasting datasets. We used (1) Electricity Consumption Load (ECL) H which is a public dataset describes the electricity consumption (Kwh) of 321 clients. (2) Electricity Transformer Temperature (ETT) (Zhou et al. 2021 ? which is multivariate time series records two years of data, focusing on predicting the electrical transformers' oil temperature based on load capacity. The dataset includes six features, and we utilised both the available hourly (h) datasel (two sites separately) and minute (m) dataset. (3) Weather Data (WTH which describes local climatological conditions for various US sites. It includes 11 climate features and the target value (a wet-bulb temperature).
5.1.2 Baseline
In addition to a non-continual learning baseline using a basic RNN layer, we evaluated a set baselines for continual lcarning time series forccasting. As FSNET (Pham al. 2023) is the most recent statc-of-the-ar model, we consider it our main baseline along with its variant naive that showed better performance for step ahead forecasting. We also include basic buffer-based model (Chaudhry et al. 2019b); Experience
Dingrams20112114
http:///gthubbboo////oooo/////// http://hhaaoy////aaaataaett Daaaastt
S httpp://www.nocl.nnee htw.:///ww.aaaaallaaeaaattttoeoe http://ccllliimmttoogoaaallaaaaa
11

---

Published in Transactions on Machine Learning Research (03/2025)
Replay (ER). ER enhances the OnlineTCN by incorporating episodic memory that preserves past samples, subsequently interleaving them during the learning of more recent ones. Additionally, we consider recent variant of ER, Dark experience replay + (Buzzega et al. 2020), that Enhances the conventional ER by introducing a knowledge distillation loss on the previous logits.
5.1.3 Setup and Experimental details
In the encoder part of the VAE, we utilize RNN model consisting of one GRU layer with hidden dimension of 64, followed by a linear layer that outputs a latent dimension of 64. The decoder comprises three main components: a Neural-ODE that initializes the latent state 2, a linear layer that maps the latent state to the hidden dimension of 64, and a second linear layer that maps the hidden representation to the final output. The dynamics of the system are integrated over time using the Euler method, with the stop size determined based on the time duration from the initial time to the final time 4, constrained by maximum step size of 0.05.
For all the experiments, we applied non-shuffled splitting to divide the data into a training set (25% ) for initialising the model (warm-up training) and a streaming dataset (75%) for online testing and training. We rescaled (normalised) the features using a standard normal distribution that fits on the training set only. We used a look- window of 24 for our model and the baselines, while the learning rate was set to 0.001, and we employed the Adam optimiser for model optimisation. During the warm-up phase, we utilized a batch size of 64 and implemented early stopping after 10 epochs. To simulate streaming environment. for online learning, the batch size and epoch were set to one.
Table 1: Cumulative MSE results of several datasets using ODEstream and the baseline models for different lasks, including: (+) univariate predict univariate, (+) multivariate predict univariale, (*) multivariate predict multivariate.
Method
ECL
ECL
ETTh1*
ETTh1
ETTh2*
ETTh2
ETTml
ETTml
WTH
WTHH"
RNN
0.576
33.84
43.36
1.2653
37.35
6.3711
41.08
0.470
0.1636
0.4616
ER
2.8142
2.8359
1.9785
0.2349
6.7558
0.5044
3.055
0.082
0.3138
0.1788
DER+
2.8107
2.81
1.9712
0.24
6.738
0.5042
3.0467
0.0808
0.3097
0.1717
NaiueNaavv
2.9943
3.0533
2.001
0.2296
6.7749
0.5033
3.0595
0.1143
0.3843
0.2462
FSnet
2.8048
3.6002
1.9342
0,2814
6.681
0.4388
3.0467
0.0866
0.3096
0.1633
ODEStream
0.1173
4.095
0.0594
0.105
0.164
0.1879
0.0625
0.2178
0.0441
0.222
5.2 Evaluation Results
The results of the ODEStream performance compared to the bascline models are presented in Table 1 Our experimental encompassed multiple lasks, including univariate and mullivariate forecasting lasks, denoted as and (+) respectively in the table, along with a aaltivariaae-multivaiiaa forecasting task (represented where the model predicts the future target value with the future feature list applied in (Pham al.
2023
In the case of the ECL dataset, which is naturally univariate data, we conducted univariate forecasting by analysing the historical data of a specific variable (a single client). All other datasets are multivariate data, where we leverage multiple features to predict future values of a single target variable as multivariate fore- casting tasks. For all datasets, we used all available features to predict a set of future values for these features for inaliivariatemmulivvriin and the proposed model described in (Pham et al. 2023). we observed that the reported performance metrics were primarily for the However, we extended our analysis to include the prediction performance for other tasks as well. Regarding the non-continual learning baseline (RNN model) the performance is suboptimal due to the fact that these models are not designed for continual learning and are unsuitable for handling streaming environments ef- fectively. Upon closer examination, the performance shown by the other bascline models was not as high as anticipated in tasks other than multivariate-multivrraaa forecasting. For this task, the performance of our method closely approximated that of the baseline models. However, we achieved substantial en ennneceent
forecasting. It is important to note that while reviewing the baseline models
forecasting task where a multivariate predict multivariate.
12

---

Published in Transactions on Machine Learning Research (03/2025)
in the performance of univariate forecasting tasks when compared to the baseline models. Figure 3 and illustrate the performance of ODEStream, FSNet and DER++ over massive period of streaming, and we can see how well ODEStream and the baseline models adapt to gradual changes in the datasets through continuous learning. is evident that FSNet and +ER+ struggle to adapt to all the data changes over an exlended period, resulting in predictions that diverge significantly from ground truth values for most of the dataset by the end of the streaming sequence. Conversely, ODEStream continuously adapts to new dynamics across the entire duration, which explains why it achieved lower cumulative MSE values. It is also worth mentioning that DER++ outperforms in multivariate forecasting due to the complex and non-stationary nature of the relationships between different variables. DER++ leverages replay buffer that stores a small subset of past data points, allowing the model to rehearse past experiences. This mechanism helps maintain performance on previous data distributions while adapting to new palterns, which crucial for handling the intricacies of multivariate time series data.
In addition to the main experiment, Table 2 presents the results of the multi-horizon predictions for 7 and 24 steps across five datasets. The performance remains stable longer prediction horizons, with significant degradation in accuracy. Notably, even at 7 and 24 steps, the model consistently outperforms the bascline models from previous experiments.
Table Multi-horizon prediction results.
steps
ECL
ETTH
ETTH2
ETTM1
WTH
0.1173
0.0594
0.164
0.0625
0.0141
0.403
0.1006
0.4009
0.5997
0.1089
24
1.003
0.1909
0.3751
0.09186
0.133
Gisendmath
082sowam
Ground ELEEN
00e50ream
thiee aodite
stream poince
(a) WTH
(b) ETTH1
Groand coss2ram
Ginundiuuhh Fshe:
15NU
C01Scwam
4008
50:00
50303
2000
8000
12000
ETTH2
(d) ETTml
Figure 3: ODEStream against FSNet: Adapting dynamics evolution for the entire stream sequence of several datasets.
5.3 Performance on Irregular Data
In this section, we assess the model performance on irregular sparse data, essentially evaluating its ability contimuussy learn from samples with uneven time intervals. To simulate irregular sparse data, we apply cut-out function to the benchmark dataset, following the work in (Rubanova et. al. 2019). Specifically, we
13

---

Published in Transactions on Machine Learning Research (03/2025)
Ground zuth
Observations volues
CDEStram
Dorst
5000
15600
20033
25600
2003
4003
6000
0000
18500
stream time points
stream time points
(a) WTH
(b) ETTh1
Oboeevations values
Ground 00Estroam
Geound muth c0EStrcom
Dare
Dere
2008
4000
5000
10302
2000
GOOD
0000
12000
stream time points
stream time points
(c) ETT12
(d) ETTml
Figure ODEStream against DER++: Adapting dynamics evolution for the entire stream sequence of several datasots.
remove 30% of the data points for each attribute the sequences. Thus we ensure that there is no fixed time interval between observations, and not all samples fully observed every time point. Figure 5 illustrates the model performance with regularly sampled data compared to irregularly sampled data. We observe that the error drops by average of less than 0.020 score While most cases experience minor increase in error, this could imply that the model adapts well to irregularities in the sampling pattern.
0.25
0.236
MSE Error
0.2
0.164
0.1289
0.15
0.1173
0.1
0.0619
0.0625
0.0582
0.0594
0.0484 0.0441
0.05
0
ECL
ETTh1
ETTh2
ETTm1
WTH
Regular data
Irregular
Sparse data
Figure 5: Model performance on irregularly sampled data.
5.4 ODEStream Adaptability
In order to assess the robustness of our proposed model in the presence of concept drift, we employed the Adaptive Windowing (ADWIN) method for concept drift detection to the testing set all datasets under considcration and evaluated the performance of our model in adapting these drifts. Figure 0 provided a visual representation of how our model responded changing data patterns compared baseline model. The red vertical lines represent drift in the data detected by ADWIN. The predictions generated by
14

---

Published in Transactions on Machine Learning Research (03/2025)
ODEStream shed light on the model's ability to maintain effectiveness and accuracy in dynamic and evolving data environments.
3.0
CDESTISi
-15
25
200
158
280
300
divan peinns
(a) BCL
(b) WIT
Drovnrd trath
molll
100
200
302
200
500
700
Strees time aeents
Strem peirds
(c) BTTH1
(d) ETTH2
2.0
1.5
POTAR
1.0
3.5
-1.0
straam time polits
(e) ETTml
Figure 6: Predictions by ODEStream compared to the baseline model FSNet amid concept drift (indicated by red vertical line).
6
Ablation study
For the ablation study, we systematically examined the performance of three model configurations: (1) the full framework with both the temporal isolation layer and the combination of MSE and KL loss for convergence, (2) the model without the temporal isolation layer, but still utilising the MSE and KL loss, and (3) the model without the temporal isolation layer, with only the MSE loss for convergence. By comparing these configurations, we gained insights into the effectiveness and necessity of the temporal isolation layer and the impact of diffcrent loss functions the model's performance in handling concept drift and dynamic time series data. The results of the ablation study are shown in Table 3 demonstrating the significant effectivencss of the temporal isolation layer, which improved performance by more than 50% on average across all datasets.
15

---

Published in Transactions on Machine Learning Research (03/2025)
Cumulaatve MSE results of the ablation study with /without a temporal isolation layer and with using different loss functions (1) MSE, (2) MSE-KL.
Table 3:
Method
ECL
ETTHI
ETTH2
ETTMI
WTH
W/O TIL:
0.299
0.127
0.372
0.10
0.200
W/O T1L2
0.369
0.084
0.347
0.098
0.178
W/ TIL1 ODEStream
0.119
0.054
0.176
0.09
0.060
0.117
0.059
0.164
0.062
0.044
7 Model Robustness
7.1 MSE Convergence
Figure 7 illustrates the convergence of MSE values during streaming prediction on all datasets for both ODEStream and FSNet. It is evident that ODEStream maintains lower MSE values even after extended period of time. This indicates the model's ability to continuously learn from new observations as they become available over time. Additionally, in certain cases, ODEStream's 's MSE values decrease over time, in contrast to the baseline model, where the error values tend to increase with time.
FSHNs cosSeram MS2
2.5
ESMI col Man
2.0
2000 5303
1500
10000
12000
15000
27500
20000
2002
4003
5300
8003
20000
BCL
(b) ETTh1
09:5/ream Mix
2.0
2.5
2000
4000
5200
0008
10000
2000
4000
0000
5000
10000
(c) ETTh2
(d) ETTml
Figure 7: MSE convergence using ODEStream compared FSNet for several datasets.
7.2 Computational Resources
In Figure we show the required resources in terms of time and memory. In the first three plots, we present the required time needed to process a stream of sequences for each dataset. We have added the FSnet required time as a reference to show the improvement and provide insights into the efficiency of ODEStream. For example, FigureBaa]hove shows the average time per second required to process one stream sequence. ODEStream requires 88% less time on average compared with the bascline model. In Figure 3d the average memory usage in megabytes is calculated using the "psudi" library for each process.
16

---

Published in Transactions on Machine Learning Research (03/2025)
160000
2.64
142361
Time Seconds Iteration
140000
2.5
Total Time Seconds
120000
ECL
100000
ETTh1
ETTh1
1.5
80000
ETTh2
ETTh2
0.8767
0.9297
ETTm1
FSNet
40000
0.5
18255
17619
FSNet
20000
Dataset
FSNet
Dataset
FSNet
(a) Execution time in seconds per one sequence.
(b) Execution time seconds for the whole stream- ing period.
45
272.244
247.898
35
Memory Usage (megabytr)
Total Time in Hours
ECL
204.657
ETTh1
200
ECL
25
ETTh2
ETTh1
150
ETTm1
ETThZ
15
WTH
100
82.643
5.03
FSNet
3.37
Dataset
FSNet
Dataset
(c) Execution time hours for the whole streaming (d) The average memory usage for each sequence processing.
period.
Figure 8: ODEStream computational resources in terms of time and memory. FSNet is used reference.
8 Conclusion
In this article, we introduce new approach buffer-free online forecasting from streaming time series data, addressing the critical need for real-time adaptability to changes. The proposed model, ODEStream, leverages the capabilities of neural ordinary differential equations (ODEs) to dynamically adapt to irregular data, and concept drift, climinating the necessity for complex frameworks. Through continuous learning and incorporating a temporal isolation layer, ODEStream stands out in processing streaming data with irreg- ular timestamps and efficiently handles temporal dependencies and changes. Our evaluation real-world benchmark time series datasets a streaming environment demonstrates that ODEStream consistently out- performs a range of online learning and streaming analysis baselines, confirming its effectiveness providing accurate predictions over extended periods while reducing the challenges associated with st.eemm time series data processing, This research marks significant stride in advancing the capabilities of online forecasting, offering a promising solution for real-time analysis of dynamic time series data. Looking ahcad, our future work aims lo exlend the applicability of the proposed methooooogyy to various time series continual learning scenarios. This includes exploring task incremental learning scenarios where a sequence of m tasks arrive, each task consisting of instances labelled time series data. For each task, ODEStream will be employed learn solver for each task with no access the data of previous tasks.
References
Futoon M. Abushagra, Hao Xue, Yongli Ren, and Flora D. Salim. Seqlink: robust neural-ode architecture for modelling partially observed time series. Transactions Machine Learning Research, 2024. URL
17

---

Published in Transactions on Machine Learning Research (03/2025)
Dima Alberg and Mark Last. Short-term load forocasting in smart meters with sliding window-based arima algorithms. Vielnam Journal of Compuler Science, 5:241 249, 2018.
Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In Proceedings of the European conference on computer vision (ECCV), pp. 139 154, 2018.
Ana Almeida, Susana Brás, Susana Sargento, and Filipe Cabral Pinto. Time series big data: a survey on data stream frameworks, analysis and algorithms. Journal of Big Data, 10(1):83, 2023.
Sio-Tong Ao and Haytham Fayek. Continual deep learning for time series modeling. Sensors, (31): 2023.
Guangji Bai, Chen Ling, and Liang Zhao. Temporal domain generalization with drift-aware dynamic neural networks. arXiv preprint arXic:2205.1060 2022.
Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: strong, simple basclinc. Advances in newral information processing systems, 33:15920-15580 2020.
Andrew Campbell, Yuyang Shi, Thomas Rainforth, and Arnaud Doucet. Online variational filtering and parameter learning. Advances in Neural Information Processing Systems, 0018663-186444 2021.
Arslan Chaudhry, Marc' Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with A-GEM. In ICLR (Poster). OpenReview.net, 2019a.
Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhosciny, Thalaiyasingam Ajanthan, Punect K Dokania, Philip HS Torr, and Marc' Aurelio Ranzato. On tiny episodic memories in continual learning. arXiv preprint ar xix::1902.. 10486, 2019b.
Chen, T Lange, M Andjelkovic, Simevski, and M Krstic Prediction of solar particle events with sram- based soft error rate monitor and supervised machine learning. Microelectronics Reliability, 114:113799, 2020.
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
Xu Chen, Junshan Wang, and Kunqing Xic. Trafficstream: streaming traffic flow forocasting framework based on graph neural networks and continual learning. arXiv preprinl arxiv:2166.00 06273, 2021.
Andrea Cossu, Antonio Carta, Vincenzo Lomonaco, and Davide Bacciu. Continual learning for recurrent neural networks: an empirical evaluation. Newral Networks, 143:607-622 2021.
Imre Csiszár. I-divergence geometry of probability distributions and minimization problems. The annals of probability, pp. 146-158, 1975.
Edward De Brouwer, Jaak Simm, Adam Arany, and Yves Moreau. Gru-ode-bayes Continuous modeling of sporadicalll--osseeved time series. Advances in neural information processing systems, 32, 2019.
Soma Shekara Sreenadh Reddy Depuru, Lingfeng Wang, Vijay Devabhaktuni, and Nikhil Gudi. Smart meters for power grid challenges, issues, advantages and status. In 2011 TEEE/PES Power Systems Conference and Exposition, pp. 1 7. IEEE, 2011.
Matthew Dowling, Yuan Zhao, and I1 Memming Park. Real-time variational method for learning neural trajectory and its dynamics. ArXiv, 2023.
Yuntao Du, Jindong Wang, Wenjic Feng, Sinno Pan, Tao Qin, Renjun Xu, and Chongjun Wang. Adarım: Adaptive learning and forecasting of time series. In Proceedings of the 30th ACM international conference on information & knowledge management, pp. 402-411, 2021.
18

---

Published in Transactions on Machine Learning Research (03/2025)
Mohammad Navid Fekri, Harsh Patel, Katarina Grolinger, and Vinay Sharma. Deep learning for load forecasting with smart meter data: Online adaptive recurrent neural network. Applied Energy. 282:116177, 2021.
Chrisantha Fernando, Dylan Banarse, Charles Blundell, Yori Zwols, David Ha, Andrei A Rusu, Alexander Pritzel, and Daan Wierstra. Pathnet: Evolution channels gradient descent super neural networks. arXiv preprint Xite:70010889/ 2017.
Jakob Foerster, Nantas Nardelli, Gregory Farquhar, Triantafyllos Afouras, Philip HS Torr, Pushmeet Kohli, and Shimon Whiteson. Stabilising experience replay for deep multi-agent reinforcement learning, In International conference on machine learning, pp. 1146 1155. PMLR, 2017.
Giorgio Giannone, Asha Anoosheh, Alessio Quaglino, Pierluca D'Oro, Marco Gallieri, and Jonathan Masci. Real-time classification from short event-camera streams using input-filtering neural odes. arXiv preprint arXiv:2004.0311 2020.
Stephen Grossberg. Adaptive resonance theory: How brain learns to consciously attend, Iearn, and recog- nize changing world. Neural networks, 37:1-47, 2013.
Ashish Gupta, Hari Prabhat Gupta, Bhaskar Biswas, and Tanima Dutta. Approaches and applications of early classification of time scries: A review. IEEE Transactions Artificial Intelligence, (11):77-1,, 2020.
Vibhor Gupta, Jyoti Narwariya, Pankaj Malhotra, Lovekesh Vig, and Gautam Shroff. Continual learning for multivariate time series tasks with variable input dimensions. In 2021 IEEE International Conference on Dala Mining (ICDM), pp. 161-170. TEFE, 2021.
Mansura Habiba and Barak Pearlmutter. Neural ordinary differential equation based recurrent neural network model. In 2020 31st Irish Signals and Systems Conference (TSSC), pp. 1-6. IEEE, 2020.
Yujiang He. Adaptive explainable continual learning framework for regression problems with focus on power forecasts. orXiv preprinl arXiv:2108. 10781, 2021.
Yujiang He and Bernhard Sick. Clear: An adaptive continual learning framework for regression tasks. AT Perspectives, 3(1):1-16, 2021.
Calypso Herrera, Florian Krach, and Josef Teichmann. Neural jump ordinary differential equations: Consis- tent continuous-time prediction and filtering. arXiv preprint orXiv:2006.04 2020.
Steven CH Hoi, Doyen Sahoo, Jing Lu, and Peilin Zhao. Online learning: A comprehensive survey. Newro- computing, 459:249 289, 2021.
Kazuki Trie, Anand Gopalakrishnan, and Jürgen Schmidhuber. Exploring the promise and limits of real-time recurrent learning. orXiv preprint arXiv:2305.1904 19044, 2023.
P Jiméncz-Horrera, Melgar-Garcfa, G Asoncio-Cortés, and A Troncoso. Streaming big time scries fore- casting based on nearest similar patterns with application energy consumption. Logic Journal of the IGPL, (1(2):255 270, 2023.
Shruti Kaushik, Abhinav Choudhury, Pankaj Kumar Sheron, Nataraj Dasgupta, Sayee Natarajan, Larry A Pickett, and Varun Dutt. Ai in healthcare: time-series forecasting using statistical, neural, and ensemble archilectures. Frontãers in big dala, 3:4, 2020.
Patrick Kidger, James Morrill, James Foster, and Terry Lyons. Neural controlled differential equations for irregular time series. Advances in Newral Information Processing Systems, 33:6696-6777 2020.
Diederik P. Kingma and Max Welling. Auto-cncoding variational bayes. In Yoshua Bengio and Yann LeCun (ods.), 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014. URL http:://ardwv.rr/abbbs/////s
19

---

Published in Transactions on Machine Learning Research (03/2025)
James Kirkpatrick, Razvan Pascamu, Neil Rabinowitz, Joel Veness, Guillumme Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Gtrassa-BBrwwinske et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521 -326,, 2017.
Dani Kiyassch, Tingting Zhu, and David Clifton. clinical deep learning framework for contimually learning from cardiac signals across diseases, time, modalities, and instilutions. Nature Communications,, 12(1):
4221, 2021.
Adam Kovacs, Bence Bogdandy, and 7solt Toth. Predict stock market prices with recurrent neural net- works using nasdaq data stream. In 2021 IEEE 15th International Symposium on Applied Computational Intelligence and Informalics (SACI), pp. 000449 000454. IEEE, 2021.
Young D Kwon, Jagmohan Chauhan, Abhishek Kumar, Pan Hui HKUST, and Cecilia Mascolo. Explor- ing system performance of continual learning for mobile and embedded sensing applications. In 2021 IBEE/ACM Symposium on Edge Computing (SEC), pp. 319 332. IEEE, 2021.
Jinhua Li, Qun Dai, and Rui Ye. A novel double incremental learning algorithm for time series prediction.
Newral Compuling and Applications, 31:6055 11-6055--677,, 2019.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions on pottern analysis and machine intelligence, 40(12):2935-29M4 2947, 2017.
Long-Ji Lin. Self-improving reactive agents based on reinforcement learning, planning and teaching. Machine learning,
1992.
Wenqian Liu, Runze Li, Meng Zheng, Srikrishna Karanam, Ziyan Wu, Bir Bhanu, Richard Radke, and Octavia Camps. Towards visually explaining variational autoencoders. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp.
2020.
David Lopez-Paz and Mare' Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in neural informalion processing systems, 30, 2017.
Arun Mallya and Svellana Lazebnik. Packnet: Adding multiple tasks to a single network by iterative pruning. In Proceedings of the IEFE conference on Computer Vision and Pattern Recognition, pp- 7765-7773, 2018.
Benjamin Maschler, Hannes Vietz, Nasser Jazdi, and Michael Weyrich. Continual learning of fault prediction for turbofan engines using deep learning with clastic weight consolidation. In 2020 25th IEEE international conference on emerging lechnologies and factory automation (ETFA), volume 1, pp. 959 96696666777 TEEE, 2020.
Benjamin Maschler, Thi Thu Huong Pham, and Michael Weyrich. Regulaaiiaatinnbbase based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP, 104:452-457, 2021.
Benjamin Maschler, Sophia Tatiyosyan, and Michael Weyrich. Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP, 112:513-518, 518, 2022.
Federico Matteoni, Andrea Cossu, Claudio Gallicchio, Vincenzo Lomonaco, and Davide Bacciu. Continual learning for human state monitoring, arXiv preprint arXiv:2207.00010, 2022.
Laura Melgar-Garcfa, David Gutiérrez-Avilss, Cristina Rubio-Fscudero, and Alicia Troncoso. A novel dis- tributed forecasting method based on information fusion and incremental learning for streaming time 2023.
series. Information Fusion,
James Morrill, Cristopher Salvi, Patrick Kidger, and James Foster. Neural rough differential equations for long time scries. In International Conference on Machine Learning, pp. 7820-7838. PMLR, 2021.
Quang Pham, Chenghao Liu, Doyen Sahoo, and Steven Hoi. Learning fast and slow for online time series forecasting. In The Eleventh International Conference on Learning Representations, 2023. URL https: ///ppareevee...ee//ooum//gd/////
20

---

Published in Transactions on Machine Learning Research (03/2025)
Sylvestre-Alvise Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 2001-2010, 2017.
Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Trina Rish, Yuhai Tu, and Gerald Tesauro. Learning to learn without forgetting by maximizing transfer and minimizing interference. In ICLR (Poster). OpenReview.ned 2019.
David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Experience replay for continual learning. Advances in Neural Information Processing Systems, 32, 2019.
Yulia Rubanova, Ricky TQ Chen, and David K Duvenaud. Latent ordinary differential equations for irregularly-sampled time series. Advances in neural information processing systems, 32, 2019.
Mona Schirmer, Maxin Eltayeb, Stefan Lessmann, and Maja Rudolph. Modeling irregular time series with continuous recurrent units. In International Conference on Machine Learning, pp. 19388 19405. PMLR, 2022.
Omer Bcrat Sczer, Mchmet Ugur Gudelek, and Ahmet Murat Ozbayoglu. Financial time series forecasting with deep learning: A systematic literature review: 2005 2019. Applied soft compuling, 90:106181, 2020.
Chenxi Sun, Moxian Song, Derun Cai, Baofeng Zhang, Shenda Hong, and Hongyan Li. Confidence-guided learning process for continuous classification of time series. In Proceedings of the 31st ACM International Conference Information & Knowledge Management, pp. 4525-4529, 2022.
Chenxi Sun, Hongyan Li, Moxian Song, Derun Cai, Baofeng Zhang, and Shenda Hong. Adaptive model training strategy for contimuous classification of time series. Applied Intelligence, pp. 19, 2023.
Stephen Wambura, Jianbin Huang, and He Li. Long-range forecasting in feature-evolving data streams. Knowledge-Based Systems, 206:106405, 2020.
Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu. A comprehensive survey of continual learning: Theory, method and application. arXiv preprint arXiv:230..00487, 2023.
Yanan Xiao, Minyu Liu, Zichen Zhang, Lu Jiang, Minghao Yin, and Jianan Wang. Streaming traffic flow prediction based on continuous reinforcement learning. In 2022 IEEE International Conference on Data Mining Workshops (ICDMW), pp. 16 22. TEFE, 2022.
Qilin Zhangla, Siyuan Sunlb, Bin Yang, Roland Wüchner, Licheng Pan, and Haitao Zhu. Real-time struc- tural health monitoring system based on streaming data. Smart Structures and Systems, 28(2):275 287, 2021.
Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shnai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. In- former: Beyond elficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp. 11106 11106-11115, 2021.
21


Reviews:
[{"review_id": "O257oHwsTa", "reviewer": "Reviewer_hmZ5", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength\n- Comprehensive experiment results\n- Some ablation study to reveal importance of each part.\n\nWeakness:\n- One question is that what is the motivation behind the buffer free idea? In terms of memory efficiency, I can understand the motivation. However the paper also claim that it could benefit the performance, but intuitively, wouldn't it be even better to leverage some buffer, into current method? Would it be even better in terms of performance by adding a memory buffer into this design? Or because of the nature of this design, buffer is not available for it?\n- Regarding the equation 13, would be better to elaborate a bit more, how was that KL divergence calculated, btw which 2 dsitribution?\n- For the ablation study on the KL divergence, seems not all dataset shows significant improvement by adding KL divergence, would be good to have some insight why is this case.(e.g. on ECL, ETTH1 and ETTH2, with ITL, MSE-KL loss is on par, and on ECL, ETTH1 and ETTM1, without ITL, MSE-KL loss is on par or even worse sometimes.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work come up with a novel method by leveraging neural ODE for streaming data prediction.\n- Proposed buffer free method, which is memory efficient.\n- Raised several research question and showed the proposal can effectively answer these questions.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See weakness above", "broader_impact_concerns": "No", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "tOMZMrxJk7", "reviewer": "Reviewer_G8NS", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strengths\n\nFirstly, I very much love the simplicity of the proposed model. The lack of a buffer makes it significantly easier to deploy this in practice as one does not have to tune the buffer configuration. Moreover, I am *very* impressed with the results of the proposed model. The experiments are varied and are all applied on real world data.\n\n# Weaknesses\n\nWhile I like the simplicity of the proposed approach, there are major weaknesses. Firstly, I think the writing needs substantial work. For starters, while I appreciate how the method is described in plain english, it makes it very hard to decipher what is going on. As an example, in the VAE section, the authors spend a paragraph describing the evidence lower bound (ELBO) in plain english, where it would have been significantly more clearer to just write the ELBO. Moreover, it would make it substantially clearer that the authors are only inferring the initial condition and that the subsequent latent states are generated deterministically. It took me quite a few reads of this section to catch this, as only having a stochastic initial condition in a sequential VAE model is generally non-standard, though see [2] for another work that does this. Also, the authors switch between $x$ and $y$ to denote the observations, which makes things confusing.\n\nThe lack of details is extended into the online learning section, as it is not clear at all what is going on. A couple of equations and an algorithm would go quite a long way explaining what is going on here, especially with the inclusion of the temporal isolation layer, the combination of the MSE loss and KL loss, etc.\n\nNext, I think the authors are missing quite a few references. On page 4, the authors state \"In general, online time series forecasting has not been widely studied using deep learning models\" but this is not true. For instance, there have been a large number of works for training recurrent neural networks in a streaming setting [3, 4]. Moreover, there are a variety of works of training deep state-space models on streaming data [5, 6].\n\nLastly, it is almost impossible to reproduce the results of this paper. There are no details of the architecture used in the neural ODE, the step size used in Euler integration, the number of gradient steps used during the warm-up phase, how the model is trained in the streaming setting, etc. This is even more concerning as training neural ODEs can be hairy and there exists a bevy of work on regularization approaches on stabilizing training, e.g., [1]. Moreover, the authors use an RNN for the encoder but don't mention how they handle the irregularly sampled data when it comes to the RNN encoder (for instance, the authors could be using RNNs that have been designed for irregularly sampled data, i.e., [7]). Also, I think a critical baseline is a non-continual learning model as a baseline. For instance, just training an feedforward network or RNN, as an example.\n\n\n## References\n\n[1] How to train your neural ODE: the world of Jacobian and kinetic regularization. https://arxiv.org/abs/2002.02798\n\n[2] LFADS - Latent Factor Analysis via Dynamical Systems. https://arxiv.org/abs/1608.06315\n\n[3] Exploring the Promise and Limits of Real-Time Recurrent Learning. https://arxiv.org/abs/2305.19044\n\n[4] Generating Coherent Patterns of Activity from Chaotic Neural Networks. https://pmc.ncbi.nlm.nih.gov/articles/PMC2756108/pdf/nihms-137176.pdf\n\n[5] Real-Time Variational Method for Learning Neural Trajectory and its Dynamics. https://arxiv.org/abs/2305.11278\n\n[6] Online Variational Filtering and Parameter Learning. https://arxiv.org/abs/2110.13549\n\n[7] Learning Long-Term Dependencies in Irregularly-Sampled Time Series. https://arxiv.org/abs/2006.04418", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In this paper, the authors introduce ODEStream, a continual learning method designed streaming time series. The proposed method is a buffer-free approach to continual learning, where neural ODEs are used to also allow the model handle non-constant data sampling (which is very common in the real world)", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1.In the VAE literature, $ q(z \\mid x)$ is commonly used to denote the encoder while the $p$ is used to denote the prior and the decoder. Thus, it would be better the authors follow this convention. *Not critical*\n\n2. Equations 3 and 5 seem like the same exact thing. *Not critical*\n\n3. Adding equations and algorithm table to make the paper easier to follow and implement. *Critical*\n\n4. Reference works for deep-learning based methods for streaming time series data. *Critical*\n\n5. Experimental details. *Critical*\n\n6. A non-continual learning baseline. *Critical*\n\n7. It is not clear what lift the neural ODE is providing. It would be great if a version parameterized by an RNN was included in the ablation study. *Critical*", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "5r1v8YjtFk", "reviewer": "Reviewer_UhMa", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n- Comprehensive literature review.\n- Tackles a very important and interesting problem.\n- Delivers impressive experimental results.\n- Experiments cover performance on both regular and irregular time series, as well as robustness and computational efficiency.\n- Being able to deal with irregular time series is indeed very important for real life applications.\n\n**Weaknesses:**\n- While reviewing related works like FSNet, it would be beneficial to test the proposed method on Traffic and other benchmark datasets.\n- The experiments seem to focus only on one-step ahead forecasting. It would be valuable to include multi-horizon predictions, or is there a reason this wasn’t explored?\n\n**Questions:**\n- Can the authors clarify if they re-ran the experiments for the baselines or changed the experimental settings? If so, why was this approach taken instead of directly using the reported baseline results?\n- For the univariate case, did the authors carefully perform hyperparameter tuning for the baselines? If so, can they report the details in the appendix?\n\n**Minor Question:**\n- For long streaming scenarios, while I understand the proposed method negates the need for retraining, would it still make sense to discard past observations and retrain the model using more recent data, especially when a large amount has been collected over time?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces ODEStream, a novel model for online learning and forecasting in streaming time series data, emphasizing adaptability without the need for complex frameworks. Unlike traditional methods that rely on preserving historical data, ODEStream focuses on learning dynamic patterns and distributions in continuous time series, leveraging neural Ordinary Differential Equation (ODE) models to handle irregularly sampled data. It presents a streamlined, buffer-free approach that simplifies training by eliminating the need for decision-making processes like thresholding or buffering. Comprehensive evaluations show that ODEStream effectively adapts to data drifts and irregularities, maintaining stable performance over long streaming periods and outperforming state-of-the-art continual learning models.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See above", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: While reviewers acknowledge the simplicity and effectiveness of the method and accompanying experiments, several areas require further attention for clarity and completeness. In particular, the writing is sometimes imprecise, and important references on streaming time series forecasting are missing. Please revise the paper accordingly, incorporating the reviewers' comments and suggestions. These refinements will strengthen the paper’s contribution to TMLR. Hence, I recommend Acceptance with the minor revision.
Decision: Accept (poster)