Title: OneNet: Enhancing Time Series Forecasting Models under Concept Drift by Online Ensembling
Venue: NeurIPS 2023
Accepted: yes
Keywords: Time series forecasting

concept drift

online learning

online convex programming
Abstract: Online updating of time series forecasting models aims to address the concept drifting problem by efficiently updating forecasting models based on streaming data. Many algorithms are designed for online time series forecasting, with some exploiting cross-variable dependency while others assume independence among variables. Given every data assumption has its own pros and cons in online time series modeling, we propose **On**line **e**nsembling **Net**work (**OneNet**). It dynamically updates and combines two models, with one focusing on modeling the dependency across the time dimension and the other on cross-variate dependency. Our method incorporates a reinforcement learning-based approach into the traditional online convex programming framework, allowing for the linear combination of the two models with dynamically adjusted weights. OneNet addresses the main shortcoming of classical online learning methods that tend to be slow in adapting to the concept drift. Empirical results show that OneNet reduces online forecasting error by more than $\mathbf{50}\\%$ compared to the State-Of-The-Art (SOTA) method.

OneNet: Enhancing Time Series Forecasting Models under Concept Drift by Online Ensembling

Abstract

Online updating of time series forecasting models aims to address the concept drifting problem by efficiently updating forecasting models based on streaming data. Many algorithms are designed for online time series forecasting, with some exploiting cross-variable dependency while others assume independence among variables. Given every data assumption has its own pros and cons in online time series modeling, we propose On line e nsembling Net work (OneNet). It dynamically updates and combines two models, with one focusing on modeling the dependency across the time dimension and the other on cross-variate dependency. Our method incorporates a reinforcement learning-based approach into the traditional online convex programming framework, allowing for the linear combination of the two models with dynamically adjusted weights. OneNet addresses the main shortcoming of classical online learning methods that tend to be slow in adapting to the concept drift. Empirical results show that OneNet reduces online forecasting error by more than 50 % compared to the State-Of-The-Art (SOTA) method. The code is available at https://github.com/yfzhang114/OneNet .

1 Introduction

In recent years, we have witnessed a significant increase in research efforts that apply deep learning to time series forecasting [Lim and Zohren, 2021, Wen et al., 2022]. Deep models have proven to perform exceptionally well not only in forecasting tasks, but also in representation learning, enabling the extraction of abstract representations that can be effectively transferred to downstream tasks such as classification and anomaly detection. However, existing studies have focused mainly on the batch learning setting, assuming that the entire training dataset is available beforehand, and the relationship between the input and output variables remains constant throughout the learning process. These approaches fall short in real-world applications where concepts are often not stable but change over time, known as concept drift [Tsymbal, 2004], where future data exhibit patterns different from those observed in the past. In such cases, re-training the model from scratch could be time-consuming. Therefore, it is desirable to train the deep forecaster online, incrementally updating the forecasting model with new samples to capture the changing dynamics in the environment.

The real world setting, termed online forecasting, poses challenges such as high noisy gradients compared to offline mini-batch training [Aljundi et al., 2019a], and continuous distribution shifts which can make the model learned from historical data less effective for the current prediction. While some studies have attempted to address the issues by designing advanced updating structures or learning objectives [Pham et al., 2023, You et al., 2021], they all rely on TCN backbones [Bai et al.,































(a) ETTh2.

























(b) ETTm1.





























(c) WTH.

































(d) ECL.

2018], which do not take advantage of more advanced network structures, such as transformer [Nie et al., 2023, Zhou et al., 2022b]. Our studies show that the current transformer-based model, PatchTST [Nie et al., 2023], without any advanced adaption method of online learning, performs better than the SOTA online adaptation model FSNet [Pham et al., 2023], particularly for the challenging ECL task (Table.1). Furthermore, we find that variable independence is crucial for the robustness of PatchTST. Specifically, PatchTST focuses on modeling temporal dependency ( cross-time dependency ) and predicting each variable independently. To validate the effectiveness of the variable independence assumption, we designed Time-TCN, which convolves only on the temporal dimension. Time-TCN is better than FSNet, a state-of-the-art approach for online forecasting, and achieves significant gains compared to the commonly used TCN structure that convolves on variable dimensions.

Although variable independence enhances model robustness, the cross-variable dependency is also critical for forecasting, i.e. for a specific variable, information from associated series in other variables may improve forecasting results. As shown in Table 1 for datasets ETTm1 and ETTh2, cross-time forecasters tend to yield lower performance for datasets with a small number of variables. Surprisingly, existing models that are designed to leverage both cross-variable and cross-time dependencies such as CrossFormer [Zhang and Yan, 2023] and TS-Mixer [Chen et al., 2023], tend to perform worse than a native TCN. To investigate this phenomenon, we visualized the MSE at different time steps during the entire online adaptation process for both a Cross-Time model (Time-TCN) and a Cross-Variable model (TCN) in Figure 1. We observe a large fluctuation in MSE over online adaption, indicating a significant concept drift over time. We also observe that neither of these two methods performs consistently better than the other, indicating that neither of the two data assumptions holds true for the entire time series. This is why relying on a single model like CrossFormer cannot solve this problem. Existing work depends on a simple model, but for online time series forecasting, data preferences for model bias will continuously change with online concept drifts. Therefore, we need a data-dependent strategy to continuously change the model selection policy. In other words, online time series forecasting should go beyond parameter updating .

In this paper, we address the limitation of a single model for online time series forecasting by introducing an ensemble of models that share different data biases. We then learn to dynamically combine the forecasts from individual models for better prediction. By allowing each model to be trained and online updated independently, we can take the best out of each online model; by dynamically

adjusting the combination of different models, we can take the best out of the entire model ensemble. We refer to our approach as Online Ensembling Network or OneNet for short. More concretely, OneNet maintains two online forecasting models, one focused on modeling temporal correlation and one focused on modeling cross-variable dependency. Each model is trained independently using the same set of training data. During testing, a reinforcement learning (RL) based approach is developed to dynamically adjust the weights used to combine the predictions of the two models. Compared to classical online learning methods such as Exponentiated Gradient Descent, our RL-based approach is more efficient in adapting to the changes/drifts in concepts, leading to better performance. The contributions of this paper are:

1. We introduce OneNet, a two-stream architecture for online time series forecasting that integrates the outputs of two models using online convex programming. OneNet leverages the robustness of the variable-independent model in handling concept drift, while also capturing the inter-dependencies among different variables to enhance forecasting accuracy. Furthermore, we propose an RL-based online learning approach to mitigate the limitations of traditional OCP algorithms and demonstrate its efficacy through empirical and theoretical analyses.

2. Our empirical studies with four datasets show that compared with state-of-the-art methods, OneNet reduces the average cumulative mean-squared errors (MSE) by 53 . 1% and mean-absolute errors (MAE) by 34 . 5% . In particular, the performance gain on challenging dataset ECL is superior, where the MSE is reduced by 59 . 2% and MAE is reduced by 63 . 0% .

3. We conducted comprehensive empirical studies to investigate how commonly used design choices for forecasting models, such as instance normalization, variable independence, seasonal-trend decomposition, and frequency domain augmentation, impact the model's robustness. In addition, we systematically compared the robustness of existing Transformer-based models, TCN-based models, and MLP-based models when faced with concept drift.

2 Preliminary and Related Work

Concept drift. Concepts in the real world are often dynamic and can change over time, which is especially true for scenarios like weather prediction and customer preferences. Because of unknown changes in the underlying data distribution, models learned from historical data may become inconsistent with new data, thus requiring regular updates to maintain accuracy. This phenomenon, known as concept drift [Tsymbal, 2004], adds complexity to the process of learning a model from data. In this paper, we focus on online learning for time series forecasting. Unlike most existing studies for online time series forecasting [Li et al., 2022, Qin et al., 2022, Pham et al., 2023] that only focus on how to online update their models, this work goes beyond parameter updating and introduces multiple models and a learnable ensembling weight, yielding rich and flexible hypothesis space. Due to the space limit, more related works about time series forecasting and reinforcement learning are left in the appendix.

Online time series forecasting: streaming data. Traditional time series forecasting tasks have a collection of multivariate time series with a look-back window L : ( x i ) L i =1 , where each x i is M -channel vector x i = ( x j i ) j M =1 . Given a forecast horizon H , the target is to forecast H future values ( x i ) L + H i = L +1 . In real-world applications, the model builds on the historical data needs to forecast the future data, that is, given time offset K ′ > L , and ( x i ) K ′ i = K ′ -L +1 , the model needs to forecast ( x ) K ′ + H i = K ′ +1 . Online time series forecasting [Anava et al., 2013, Liu et al., 2016, Pham et al., 2023] is a widely used technique in real-world due to the sequential nature of the data and the frequent drift of concepts. In this approach, the learning process takes place over a sequence of rounds, where the model receives a look-back window and predicts the forecast window. The true values are then revealed to improve the model's performance in the next rounds. When we perform online adaptation, the model is retrained using the online data stream with the MSE loss over each channel: L = 1 M ∑ M j =1 ∥ ˆ x j K ′ +1: K ′ + H -x j K ′ +1: K ′ + H ∥ .

Variable-independent time series forecasting. The traditional cross-variable strategy used in most structures takes the vector of all time series features as input and projects it into the embedding space to mix the information. On the contrary, PatchTST [Nie et al., 2023] adopts a variable-independent approach, where each input token only contains information from a single channel/variable. Our research demonstrates that variable independence is crucial for boosting model robustness under

Cross-Variable

Forecaster

Cross-Time

Forecaster

Input Multivariate Series

MSE Loss

MSE Loss

OCP

Linear

Linear

Linear

State (Prediction)

Outcome

Action (Short-term weight

)

Offline RL for Short-term Weight

EGD for Long-term Weight

(a) The overall OneNet architecture.

(b) OCP block.

concept drift. For multivariate time series samples ( x j i ) L i =1 , each channel j is fed into the model independently, and the forecaster produces prediction results ( x j i ) L + H i = L +1 accordingly. As shown in Table 1, cross-variable methods tend to overfit when the dataset has a large number of variables, resulting in poor performance. This is evident in the poor performance of the SOTA online adaptation model FSNet [Pham et al., 2023] in the ECL dataset. However, models that lack cross-variable information perform worse on datasets with a small number of variables where cross-variable dependency can be essential. Although some existing work has attempted to incorporate both crossvariable interaction and temporal dependency into a single framework, our experiments show that these models are fragile under concept drift and perform no better than the proposed simple baseline, Time-TCN. To address this, we propose a novel approach that trains two separate branches, each focusing on modeling temporal and cross-variable dependencies, respectively. We then combine the results of these branches to achieve better forecasting performance under concept drift. We first introduce the OCP block for coherence.

3 OneNet: Ensemble Learning for Online Time Series Forecasting

We first examine online learning methods to dynamically adjust combination weights used by ensemble learning. We then present OneNet, an ensemble learning framework for online time series forecasting.

3.1 Learning the best expert by Online Convex Programming (OCP)

For notation clarity, here we denote x ∈ R L × M as the historical data, y ∈ R H × M as the forecast target. Our current method involves the integration of multiple complementary models. Therefore, how to better integrate model predictions in the online learning setting is an important issue. Exponentiated Gradient Descent (EGD) [Hill and Williamson, 2001] is a commonly used method. Specifically, the decision space △ is a d -dimensional simplex, i.e. △ = { w t | w t,i ≥ 0 and ∥ w t ∥ 1 = 1 } , where t is the time step indicator and we omit the subscript t for simplicity when it's not confusing. Given the online data stream x , its forecasting target y , and d forecasting experts with different parameters f = [˜ y i = f i ( x )] d i =1 , the player's goal is to minimize the forecasting error as

According to EGD, choosing w 1 = [ w 1 ,i = 1 /d ] d i =1 as the center point of the simplex and denoting ℓ t,i as the loss of f i at time step t , the updating rule for each w i will be

where Z t = ∑ d i =1 w t,i exp( -ηl t,i ) is the normalizer, and the algorithm has a regret bound:

Proposition 1. ( Online Convex Programming Bound ) For T > 2 log( d ) , denote the regret for time step t = 1 , . . . , T as R ( T ) , set η = √ 2 log( d ) /T , the OCP updating policy have an External regret (See appendix B.1 for proof and analysis.)

That is, the exponentially weighted average forecaster guarantees that the forecaster's cumulative expected loss is not much larger than the cumulative loss of the best decision. However, an exponentially weighted average forecaster is widely known to respond very slowly to drastic changes in the distribution [Cesa-Bianchi and Lugosi, 2006]. This phenomenon is sometimes referred to as the ' slow switch phenomenon ' in online learning literature, and is further illustrated in Figure 3 where the loss for f 1 is 0 for the first 50 trials and 1 for the next 50 trials. The performance of f 2 is the opposite. When the step size η is small (e.g., η = 0 . 01 ), small changes are made to the weights and no clear adaptation takes place. When a large step size η is applied (e.g., η = 1 ), we observe that the EGD algorithm quickly adapts to the environment change for the first 50 trials by increasing weight w 1 to almost 1 in the first few iterations. But it takes many iterations for the EGD algorithm to adapt to the change in the next 50 iterations, where f 2 works much better than f 1 . We finally note that no matter how we adjust the step size η , the EGD algorithm has to suffer from the trade-off between speed of switching and overall good performance throughout the horizon.

Although few algorithms have been developed to address this issue in online learning [Stoltz and Lugosi, 2005, Cesa-Bianchi and Lugosi, 2003, Blum and Mansour, 2007, Foster and Vohra, 1998], the key idea is to find an activation function that maps the original policy w t to a new one based on the recent loss of all experts. Despite the efforts, very limited successes have been achieved, either empirically or theoretically. In this work, we observe in our experiments that the combination weights w generated by the EGD algorithm are based on historical performance over a long period of time and thus cannot adapt quickly to transient environment changes. Hence, it is better to effectively incorporate both long-term historical information and more recent changes in the environment. A straightforward idea is to re-initialize the weight w per K steps. We show that such a simple algorithm can achieve a tighter bound:

0

20

40

60

80

100

Time steps

0.0

0.2

0.4

0.6

0.8

1.0

Weight for

f

1

First regime

t

, 1=0

t

, 2=1

Second regime

t

, 1=1

t

, 2=0

=0.01

=0.05

=0.1

=0.25

=0.5

=1.0

Proposition 2. ( Informal ) Denote I = [ l, · · · , r ] ∈ [1 , · · · , T ] as any period of time. We then have, the K -step re-initialize algorithm has a tighter regret bound compared to EGD at any small interval I , where | I | < T 3 4 . (See appendix B.2 for proof.)

Proposition 2 stresses that, by considering short-term information, we can attain lower regret in short time intervals. Such a simple strategy still struggles with the hyper-parameter choice of K . Besides, discarding long-term information makes the algorithm inferior to EGD for a long period of the online learning process. In this work, we address this challenge of online learning by exploiting offline reinforcement learning [Levine et al., 2020]. At first, we use EGD to maintain long-term weight w . Besides, we introduce a different set of weights b that can better capture the recent performance of individual models. By combining w and b , our approach can effectively incorporate both long-term historical information and more recent changes in the environment.

Specifically, we adopt the RvS [Emmons et al., 2022] framework, which formulates reinforcement learning through supervised learning, as shown in Figure 2(b). At time step t , our target is to learn

a short-term weight conditioned on the long-term weight w and experts' performances during a short period of history I = [ l, t ] . For simplicity and computation efficiency, we just let l = t -1 . The agent then chooses actions using a policy π θ rl ( b t |{{ w t,i ˜ y i } d i =1 } t ∈ I ; y ) parameterized by θ rl . During training, we concatenate the product between each prediction and expert weight ( w t,i ∗ ˜ y i ) with the outcome y as the conditional input. We follow RvS [Emmons et al., 2022] to implement the policy network as a two-layer MLPs f rl : R H × M × ( d +1) → R d . Then the short-term weight and final ensembling weight will be:

However, unlike in RvS, we cannot train the decision network through simple classification tasks since the ground truth target action is inaccessible. Instead, we propose to train the network by minimizing the forecasting error incurred by the new weight, that is, min θ rl L ( ˜ w ) := ∥ ∑ d i =1 ˜ w t,i f i ( x ) -y ∥ 2 . During inference, as concept drift changes gradually, we use w t -1 + b t -1 to generate the prediction and train the networks after the ground truth outcome is observed. We theoretically and empirically verify the effectiveness of the proposed OCP block in appendix B.4.

3.2 OneNet: utilizing the advantages of both structures

The model structure is shown in Figure 2(a) and we introduce the components as follows:

Two-stream forecasters. The input multivariate time series data is fed into two separate forecasters, a cross-time forecaster f 1 and a cross-variable forecaster f 2 . Each forecaster contains an encoder and a prediction head. Assuming that the hidden dimension of the models are all d m , the encoder of f 1 projects the input series to representation z 1 ∈ R M × d m , and the prediction head generates the final forecasting results: ˜ y 1 ∈ R M × H . For the cross-variable forecaster f 2 , the encoder projects x to z 2 ∈ R L × d m . Then, the representation of the last time step z 2 ,L ∈ R d m is selected and fed into the prediction head to generate the final forecasting results ˜ y 2 ∈ R M × H . Compared to f 1 , whose projection head has a parameter of d m × H , the projection head of f 2 has a parameter of d m × M × H , which is heavier, especially when M is large. Additionally, while f 1 ignores variable dependency, f 2 simply selects the representation of the last time step time series, ignoring temporal dependency. These two modules yield different but complementary inductive biases for forecasting tasks. OCP block is then used for learning the best combination weights. Specifically, we use EGD to update a weight w i for each forecaster and use offline-reinforcement learning to learn an additional short-term weight b i , the final combination weight for one forecaster will be w i ← w i + b i . Considering the difference between variables, we further construct different weights for each variable, namely, we will have w ∈ R M × 2 combination weights.

Decoupled training strategy. A straightforward training strategy for OneNet is to minimize L ( w 1 ∗ ˜ y 1 + w 2 ∗ ˜ y 2 , y ) for both the OCP block and the two forecasters, where w i here denotes the weight with the additional bias term. However, the coupled training strategy has a fatal flaw: considering an extreme case where f 1 always performs much better than f 2 , then w 1 will be close to 1 and w 2 → 0 . In this case, ∇ ˜ y 2 L ( w 1 ∗ ˜ y 1 + w 2 ∗ ˜ y 2 , y ) ≈ 0 , that is, f 2 is probably not trained for a long time. Under the context of concept drift, if retraining is not applied, as time goes on, the performance of f 2 will become much inferior. In this paper, therefore, we decouple the training process of the OCP block and the two forecasters. Specifically, the two forecasters is trained by L ( ˜ y 1 , y ) + L ( ˜ y 2 , y ) and the OCP block is trained by L ( w 1 ∗ ˜ y 1 + w 2 ∗ ˜ y 2 , y ) .

Remark Note that OneNet is complementary to advanced architectures for time series forecasting and online adaption methods under concept drift. A stronger backbone or better adaptation strategies/structure can both enhance performance.

4 Experiments

In this section, we will show that (1) the proposed OneNet attains superior forecasting performances with only a simple retraining strategy (reduce more than 50% MSE compared to the previous SOTA model); (2) OneNet achieves faster and better convergence than other methods; (3) we conduct thorough ablation studies and analysis to reveal the importance of each of design choices of current advanced forecasting models. Finally, we introduce a variant of OneNet, called OneNet-, which has significantly fewer parameters but still outperforms the previous SOTA model by a large margin. Due to space limitations, some experimental settings and results are provided in the appendix.

4.1 Experimental setting

Baselines of adaptation methods We evaluate several baselines for our experiments, including methods for continual learning, time series forecasting, and online learning. Our first baseline is OnlineTCN [Zinkevich, 2003], which continuously trains the model without any specific strategy. The second baseline is Experience Replay (ER) [Chaudhry et al., 2019], where previous data is stored in a buffer and interleaved with newer samples during learning. Additionally, we consider three advanced variants of ER: TFCL [Aljundi et al., 2019b], which uses a task-boundary detection mechanism and a knowledge consolidation strategy; MIR [Aljundi et al., 2019a], which selects samples that cause the most forgetting; and DER++ [Buzzega et al., 2020], which incorporates a knowledge distillation strategy. It is worth noting that ER and its variants are strong baselines in the online setting, as we leverage mini-batches during training to reduce noise from single samples and achieve faster and better convergence. Finally, we compare our method to FSNet [Pham et al., 2023], which is the previous state-of-the-art online adaptation method. Considering different model structures, we compare the performance under concept drift of various structures, including TCN [Bai et al., 2018], Informer [Zhou et al., 2021], FEDformer [Zhou et al., 2022b], PatchTST [Nie et al., 2023], Dlinear [Zeng et al., 2023], Nlinear [Zeng et al., 2023], TS-Mixer [Chen et al., 2023].

Strong ensembling baselines. To verify the effectiveness of the proposed OCP block, we compare it with several ensembling baselines. Given the online inputs x , predictions of each expert ˜ y 1 , ˜ y 2 , and the ground truth outcome y , the final outcome ˜ y of different baselines will be as follows: (1) Simple averaging : we simply average the predictions of both experts to get the final prediction, i.e., ˜ y = 1 2 ( ˜ y 1 + ˜ y 2 ) . (2) Gating mechanism Liu et al. [2021]: we learn weights to the output of each forecaster, that is, h = W Concat ( ˜ y 1 , ˜ y 2 ) + b ; w 1 , w 2 = softmax ( h ) , and the final result is given by ˜ y = w 1 ∗ ˜ y 1 + w 2 ∗ ˜ y 2 . (3) Mixture-of-experts Jacobs et al. [1991], Shazeer et al. [2017]: we use the mixture of experts approach, where we first learn the weights w 1 and w 2 by applying a softmax function on a linear combination of the input, i.e., h = Wx + b ; w 1 , w 2 = softmax ( h ) , and then we obtain the final prediction by combining the predictions of both experts as ˜ y = w 1 ∗ ˜ y 1 + w 2 ∗ ˜ y 2 . (4) Linear Regression (LR) : we use a simple linear regression model to obtain the optimal weights, i.e., [ w 1 , w 2 ] = ( X T X ) -1 X T y , where X = [ ˜ y 1 , ˜ y 2 ] and y is the ground truth outcome. (5) Exponentiated Gradient Descent (EGD) : we use EGD to update the weights w 1 and w 2 separately without the additional bias. (6) Reinforcement learning to learn the weight directly (RL-W) : we use the bias term in the OCP block to update the weights based on the predictions of both experts and the ground truth outcome, i.e., the weight is only dependent on ˜ y 1 , ˜ y 2 , and y , but not on the historical performance of each expert. For all baselines with trainable parameters, the training procedure is just the same as the proposed OCP block.

4.2 Online forecasting results

Cumulative performance Table.2 and Table.3 present the cumulative performance of different baselines in terms of mean-squared errors (MSE) and mean-absolute errors (MAE). In particular, Time-TCN and PatchTST exhibit strong performance and outperform the previous state-of-the-art model, FSNet [Pham et al., 2023]. The proposed OneNet-TCN (online ensembling of TCN and Time-

TCN) surpasses most of the competing baselines across various forecasting horizons. Interestingly, if the combined branches are stronger, for example, OneNet combined FSNet and Time-FSNet, achieving much better performance than OneNet-TCN. Namely, OneNet can integrate any advanced online forecasting methods or representation learning structures to enhance the robustness of the model. The average MSE and MAE of OneNet are significantly better than using either branch (FSNet or Time-TCN) alone, which underscores the significance of incorporating online ensembling.

Comparison with strong ensembling baselines is shown in Table 4. The two-branch framework greatly improves performance compared to FSNet with just simple ensembling methods such as averaging. The MOE approach that learns the weight from the input x performs poorly and is even inferior to simply averaging the prediction results. On the other hand, learning the weight from the prediction results in ˜ y 1 and ˜ y 2 (Gating) performing much better than MOE. This indicates that the combination weights should be dependent on the model prediction. However, formulating the learning problem as linear regression and using the closed-form solution is not a good idea due to the scarce nature of the online data stream and the high noise in the learned weight. EGD provides significant benefits compared to the averaging method, which highlights the importance of the cumulative historical performance of each expert. Additionally, we observe that RL-W achieves performance comparable or even better than EGD on some datasets. Therefore, we propose the OCP block that uses EGD to update the long-term weight and offline RL to learn the short-term weight. This design leads to superior performance compared to all the other baselines.

Forecasting results are visualized in Figure 4 . Compared to baselines that struggle to adapt to new concepts and produce poor forecasting results, OneNet can successfully capture the patterns of time series. More visualization results and convergence analysis are presented in Appendix C.7.

4.3 Ablation studies and analysis

The effect of instances normalization and seasonal-trend decomposition is shown in Table. 5. The results show that removing the seasonal-trend decomposition component from PatchTST has limited







































(a)

(b)

(c) Channel

2

.

4

Ground Truth

TCN

0

FSNet

2

OneNet

Value

Value

-2

Ground Truth

TCN

1

0

FSNet

OneNet

2500

2510

2520

2530

2540

2550

2500

2510

2520

2530

2540

2550

#Instances

#Instances

effect, regardless of whether the model is adapted online or not. Instances normalization is commonly used to mitigate the distribution shift between training and testing data, which is crucial for model robustness when online adaptation is impossible. However, when online adaptation is performed, the influence of instance normalization is reduced. Interestingly, our experiments reveal that instance normalization impedes the model adaptation process in ETTH2, ETTm1, and WTH datasets when the forecast horizon is long (24 or 48) . Thus, simply normalizing time series with zero mean and unit standard deviation may not be the optimal approach under concept drift. Ablation studies of the variable independence and frequency domain augmentation are detailed in the appendix.

Delve deep into parameter-efficient online adaptation. Although OneNet significantly reduces the forecasting error, it also increases the number of parameters and inference time due to its twostream framework. We also design a variant of OneNet that may have slightly lower performance than OneNet, but with fewer parameters, making it more suitable for lightweight applications, denoted by OneNet-. Specifically, we ensemble PatchTST and Time-FSNet, which are both variableindependent. In this case, denote z 1 , z 2 as the generated features for one variable from two branches, we concatenate the two features and feed them into the projection head, which further avoids the offline reinforcement learning block for ensembling weight learning and reduces the parameters. For example, in the ECL dataset, the hidden dimension FSNet [Pham et al., 2023] is 320 , and the sequences have 321 channels. When the forecast horizon is 48 , the projection head consists of just one linear layer with 320 × 321 × 48 = 4 , 930 , 560 parameters. On the contrary, the concatenated features of OneNet- are always less than 1024 dimension, resulting in a final projection head with less than 1024 × 48 = 49 , 152 parameters. Figure 4(a) shows a detailed comparison of different methods on the ECL dataset. For small forecast horizons, all methods have a comparable number of parameters. As the forecast horizon increases, the number of parameters of existing adaptation methods increases rapidly. On the contrary, the number of parameters of OneNet- remains insensitive to the forecast horizon and is always less than all baselines. The performance of OneNet- is shown in Table 12, which is much better than FSNet but achieves fewer parameters.

See the appendix for the comparison of different forecasting models and more numerical results such as detailed ablation studies of different hyper-parameters and adaptation results under more settings.

5 Conclusion and Future Work

Through our investigation into the behavior of advanced forecasting models with concept drift, we discover that cross-time models exhibit greater robustness when the number of variables is large, but are inferior to models that can model variable dependency when the number of variables is small. In addition, this problem becomes more challenging due to the occurrence of concept drift, as the data preferences for both model biases are dynamically changing throughout the entire online forecasting process, making it difficult for a single model to overcome. To this end, we propose the OneNet model, which takes advantage of the strengths of both models through OCP. In addition, we propose to learn an additional short-term weight through offline reinforcement learning to mitigate the slow switch phenomenon commonly observed in traditional policy learning algorithms. Our extensive experiments demonstrate that OneNet is able to effectively deal with various types of concept drifts and outperforms previous methods in terms of forecasting performance.

We also discover that instance normalization enhances model robustness under concept drift, but can impede the model's ability to quickly adapt to new distributions in certain scenarios. This prompts further exploration of whether there exists a normalization technique that can mitigate distribution shifts while enabling rapid adaptation to changing concepts. In addition, although we design a lightened version of OneNet to address the problem of introducing additional parameters and inference time, there is potential for more efficient adaptation methods, such as utilizing prompts and efficient tuning methods from the NLP/CV community, to avoid retraining the full model.

Acknowledgments and Disclosure of Funding

This work was supported by the National Key R & D Program of China (2022ZD0117901) and National Natural Science Foundation of China (Grant No. 62373355 and 62236010), and also supported by Alibaba Group through Alibaba Research Intern Program.

References

Rishabh Agarwal, Dale Schuurmans, and Mohammad Norouzi. An optimistic perspective on offline reinforcement learning. In International Conference on Machine Learning , pages 104-114. PMLR, 2020.

Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In Proceedings of the European conference on computer vision (ECCV) , pages 139-154, 2018.

Rahaf Aljundi, Eugene Belilovsky, Tinne Tuytelaars, Laurent Charlin, Massimo Caccia, Min Lin, and Lucas Page-Caccia. Online continual learning with maximal interfered retrieval. In Advances in Neural Information Processing Systems 32 , pages 11849-11860. 2019a.

Rahaf Aljundi, Klaas Kelchtermans, and Tinne Tuytelaars. Task-free continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 11254-11263, 2019b.

Oren Anava, Elad Hazan, Shie Mannor, and Ohad Shamir. Online learning for time series prediction. In Conference on learning theory , pages 172-184. PMLR, 2013.

Shaojie Bai, J Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271 , 2018.

Avrim Blum and Yishay Mansour. From external to internal regret. Journal of Machine Learning Research , 8(6), 2007.

George EP Box and David A Pierce. Distribution of residual autocorrelations in autoregressiveintegrated moving average time series models. Journal of the American statistical Association , 65 (332):1509-1526, 1970.

Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: a strong, simple baseline. Advances in neural information processing systems , 33:15920-15930, 2020.

Nicolo Cesa-Bianchi and Gábor Lugosi. Potential-based algorithms in on-line prediction and game theory. Machine Learning , 51:239-261, 2003.

Nicolo Cesa-Bianchi and Gábor Lugosi. Prediction, learning, and games . Cambridge university press, 2006.

Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet K Dokania, Philip HS Torr, and Marc'Aurelio Ranzato. On tiny episodic memories in continual learning. arXiv preprint arXiv:1902.10486 , 2019.

Si-An Chen, Chun-Liang Li, Nate Yoder, Sercan O Arik, and Tomas Pfister. Tsmixer: An all-mlp architecture for time series forecasting. arXiv preprint arXiv:2303.06053 , 2023.

Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 , 2014.

Abhimanyu Dubey, Vignesh Ramanathan, Alex Pentland, and Dhruv Mahajan. Adaptive methods for real-world domain generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021.

Scott Emmons, Benjamin Eysenbach, Ilya Kostrikov, and Sergey Levine. Rvs: What is essential for offline rl via supervised learning? ICLR , 2022.

Dean P Foster and Rakesh Vohra. Regret in the on-line decision problem. Games and Economic Behavior , 29(1-2):7-35, 1999.

Dean P Foster and Rakesh V Vohra. Asymptotic calibration. Biometrika , 85(2):379-390, 1998.

Scott Fujimoto, David Meger, and Doina Precup. Off-policy deep reinforcement learning without exploration. In International conference on machine learning , pages 2052-2062. PMLR, 2019.

João Gama, Indr˙ e Žliobait˙ e, Albert Bifet, Mykola Pechenizkiy, and Abdelhamid Bouchachia. A survey on concept drift adaptation. ACM computing surveys (CSUR) , 46(4):1-37, 2014.

Udaya Ghai, Elad Hazan, and Yoram Singer. Exponentiated gradient meets gradient descent. In Algorithmic learning theory , pages 386-407. PMLR, 2020.

Alex Graves and Alex Graves. Long short-term memory. Supervised sequence labelling with recurrent neural networks , pages 37-45, 2012.

Roland Hafner and Martin Riedmiller. Reinforcement learning in feedback control: Challenges and benchmarks from technical process control. Machine learning , 84:137-169, 2011.

Simon I Hill and Robert C Williamson. Convergence of exponentiated gradient algorithms. IEEE Transactions on Signal Processing , 49(6):1208-1215, 2001.

Yusuke Iwasawa and Yutaka Matsuo. Test-time classifier adjustment module for model-agnostic domain generalization. Advances in Neural Information Processing Systems , 2021.

Robert A Jacobs, Michael I Jordan, Steven J Nowlan, and Geoffrey E Hinton. Adaptive mixtures of local experts. Neural computation , 3(1):79-87, 1991.

Rahul Kidambi, Aravind Rajeswaran, Praneeth Netrapalli, and Thorsten Joachims. Morel: Modelbased offline reinforcement learning. Advances in neural information processing systems , 33: 21810-21823, 2020.

Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. Offline reinforcement learning: Tutorial, review, and perspectives on open problems. arXiv preprint arXiv:2005.01643 , 2020.

Wendi Li, Xiao Yang, Weiqing Liu, Yingce Xia, and Jiang Bian. Ddg-da: Data distribution generation for predictable concept drift adaptation. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 36, pages 4092-4100, 2022.

Jian Liang, Ran He, and Tieniu Tan. A comprehensive survey on test-time adaptation under distribution shifts. arXiv preprint arXiv:2303.15361 , 2023.

Bryan Lim and Stefan Zohren. Time-series forecasting with deep learning: a survey. Philosophical Transactions of the Royal Society A , 379(2194):20200209, 2021.

Chenghao Liu, Steven CH Hoi, Peilin Zhao, and Jianling Sun. Online arima algorithms for time series prediction. In Proceedings of the AAAI conference on artificial intelligence , volume 30, 2016.

Minghao Liu, Shengqi Ren, Siyuan Ma, Jiahui Jiao, Yizhou Chen, Zhiguang Wang, and Wei Song. Gated transformer networks for multivariate time series classification. arXiv preprint arXiv:2103.14438 , 2021.

Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 , 2017.

Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. ICLR , 2023.

Quang Pham, Chenghao Liu, Doyen Sahoo, and Steven CH Hoi. Learning fast and slow for online time series forecasting. ICLR , 2023.

Rafael Figueiredo Prudencio, Marcos ROA Maximo, and Esther Luna Colombini. A survey on offline reinforcement learning: Taxonomy, review, and open problems. IEEE Transactions on Neural Networks and Learning Systems , 2023.

Tiexin Qin, Shiqi Wang, and Haoliang Li. Generalizing to evolving domains with latent structureaware sequential autoencoder. In International Conference on Machine Learning , pages 1806218082. PMLR, 2022.

Yao Qin, Dongjin Song, Haifeng Chen, Wei Cheng, Guofei Jiang, and Garrison Cottrell. A dual-stage attention-based recurrent neural network for time series prediction. arXiv preprint arXiv:1704.02971 , 2017.

Rajat Sen, Hsiang-Fu Yu, and Inderjit S Dhillon. Think globally, act locally: A deep neural network approach to high-dimensional time series forecasting. Advances in neural information processing systems , 32, 2019.

Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. ICLR , 2017.

David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. Mastering the game of go without human knowledge. nature , 550(7676):354-359, 2017.

Gilles Stoltz and Gábor Lugosi. Internal regret in on-line portfolio selection. Machine Learning , 59: 125-159, 2005.

Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt. Test-time training with self-supervision for generalization under distribution shifts. In International conference on machine learning , pages 9229-9248. PMLR, 2020.

Alexey Tsymbal. The problem of concept drift: definitions and related work. Computer Science Department, Trinity College Dublin , 106(2):58, 2004.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems , 30, 2017.

Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. arXiv preprint arXiv:2006.10726 , 2020.

Qingsong Wen, Tian Zhou, Chaoli Zhang, Weiqi Chen, Ziqing Ma, Junchi Yan, and Liang Sun. Transformers in time series: A survey. arXiv preprint arXiv:2202.07125 , 2022.

Xiaoyu You, Mi Zhang, Daizong Ding, Fuli Feng, and Yuanmin Huang. Learning to learn the future: Modeling concept drifts in time series prediction. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management , pages 2434-2443, 2021.

Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? 2023.

Hanlin Zhang, Yi-Fan Zhang, Weiyang Liu, Adrian Weller, Bernhard Schölkopf, and Eric P Xing. Towards principled disentanglement for domain generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8024-8034, 2022a.

Yang Zhang, Fuli Feng, Chenxu Wang, Xiangnan He, Meng Wang, Yan Li, and Yongdong Zhang. How to retrain recommender system? a sequential meta-learning method. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval , pages 1479-1488, 2020.

Yi-Fan Zhang, Jindong Wang, Jian Liang, Zhang Zhang, Baosheng Yu, Liang Wang, Dacheng Tao, and Xing Xie. Domain-specific risk minimization for out-of-distribution generalization, 2022b.

Yi-Fan Zhang, Zhang Zhang, Da Li, Zhen Jia, Liang Wang, and Tieniu Tan. Learning domain invariant representations for generalizable person re-identification. IEEE Transactions on Image Processing , 32:509-523, 2022c.

Yi-Fan Zhang, Xue Wang, Kexin Jin, Kun Yuan, Zhang Zhang, Liang Wang, Rong Jin, and Tieniu Tan. Adanpc: Exploring non-parametric classifier for test-time adaptation. ICML , 2023a.

Yifan Zhang, Bryan Hooi, Lanqing Hong, and Jiashi Feng. Test-agnostic long-tailed recognition by test-time aggregating diverse experts with self-supervision. arXiv preprint arXiv:2107.09249 , 2021.

YiFan Zhang, Xue Wang, Jian Liang, Zhang Zhang, Liang Wang, Rong Jin, and Tieniu Tan. Free lunch for domain adversarial training: Environment label smoothing. arXiv preprint arXiv:2302.00194 , 2023b.

Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In International Conference on Learning Representations , 2023.

Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence , volume 35, pages 11106-11115, 2021.

Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, and Chen Change Loy. Domain generalization: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022a.

Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International Conference on Machine Learning , pages 27268-27286. PMLR, 2022b.

Martin Zinkevich. Online convex programming and generalized infinitesimal gradient ascent. In Proceedings of the 20th international conference on machine learning (icml-03) , pages 928-936, 2003.

OneNet: Enhancing Time Series Forecasting Models under Concept Drift by Online Ensembling

----Appendix----

Contents

A Extended Related Work

Concept Drift Concepts in the real world are often dynamic and can change over time, which is especially true for scenarios like weather prediction and customer preferences. Because of unknown changes in the underlying data distribution, models learned from historical data may become inconsistent with new data, thus requiring regular updates to maintain accuracy. This phenomenon, known as concept drift [Tsymbal, 2004], adds complexity to the process of learning a model from data. Concept drift poses several challenging subproblems, ranging from fast learning under concept drift [Pham et al., 2023, Zhang et al., 2020, Gama et al., 2014], which involves adjusting the offline model with new observations to recognize recent patterns, to forecasting future data distributions [Li et al., 2022, Qin et al., 2022], which predicts the data distribution of the next time-step sequentially, enabling the model of the downstream learning task to be trained on the data sample from the predicted distribution. In this paper, we focus on the first problem, i.e. online learning for time series forecasting. Unlike most existing studies for online time series forecasting [Li et al., 2022, Qin et al., 2022, Pham et al., 2023] that only focus on how to online update their models, this work goes beyond parameter updating and introduces multiple models and a learnable ensembling weight, yielding rich and flexible hypothesis space.

Test-Time adaptive methods is similar to online time series forecasting but mainly focuses on domain generalization [Zhang et al., 2022c,a, Zhou et al., 2022a], domain adaptation [Zhang et al., 2023b] and other tasks [Liang et al., 2023]. These recently proposed to utilize target samples. TestTime Training methods design proxy tasks during tests such as self-consistence [Zhang et al., 2021], rotation prediction [Sun et al., 2020] and need extra models; Test-Time Adaptation methods adjust model parameters based on unsupervised objectives such as entropy minimization [Wang et al., 2020] or update a prototype for each class [Iwasawa and Matsuo, 2021]. Domain adaptive method [Dubey et al., 2021] needs additional models to adapt to target domains. There are also some methods that do not need test-time tunning, for example, [Zhang et al., 2022b] introduces specific classifiers for different domains and adapts the voting weight for test samples dynamically and [Zhang et al., 2023a] apply non-parametric test-time adaptation.

Time Series Modeling Time series models have been developed for decades and are fundamental in various fields. While autoregressive models like ARIMA [Box and Pierce, 1970] were the first data-driven approaches, they struggle with nonlinearity and non-stationarity. Recurrent neural networks (RNNs) were designed to handle sequential data, with LSTM [Graves and Graves, 2012] and GRU [Chung et al., 2014] using gated structures to address gradient problems. Attention-based RNNs [Qin et al., 2017] use temporal attention to capture long-range dependencies, but are not parallelizable and struggle with long dependencies. Temporal convolutional networks [Sen et al., 2019] are efficient, but have limited reception fields and struggle with long-term dependencies. Recently, transformer-based [Vaswani et al., 2017, Wen et al., 2022] models have been renovated and applied in time series forecasting. Although a large body of work aims to make Transformer models more efficient and powerful [Zhou et al., 2022b, 2021, Nie et al., 2023], we are the first to evaluate the robustness of advanced forecasting models under concept drift, making them more adaptable to new distributions.

Reinforcement learning and offline reinforcement learning. Reinforcement learning is a mathematical framework for learning-based control, which allows us to automatically acquire policies that represent near-optimal behavioral skills to optimize user-defined reward functions [Hafner and Riedmiller, 2011, Silver et al., 2017]. The reward function specifies the objective of the agent, and the reinforcement learning algorithm determines the actions necessary to achieve it. However, the online learning paradigm of reinforcement learning is a major obstacle to its widespread adoption. The iterative process of collecting experience by interacting with the environment is expensive or dangerous in many settings, making offline reinforcement learning a more feasible alternative. Offline RL [Prudencio et al., 2023, Levine et al., 2020] learns exclusively from static datasets of previously collected interactions, enabling the extraction of policies from large and diverse training datasets. Effective offline RL algorithms have a much wider range of applications than online RL. Although there are many types of offline RL algorithms, such as those using value functions [Fujimoto et al., 2019], dynamics estimation [Kidambi et al., 2020], or uncertainty quantification [Agarwal et al., 2020], RvS [Emmons et al., 2022] has shown that simple conditioning with standard feedforward networks can achieve state-of-the-art results. In this work, we draw inspiration from RvS and learn an additional bias term for the OCP block for simplicity.

B Proofs of Theoretical Statements

B.1 Online Convex Programming Regret Bound

Proposition 1. For T > 2 log( d ) , denote the regret for time step t = 1 , . . . , T as R ( T ) , set η = √ 2 log( d ) /T , and the EGD update policy has regret.

Proof. The proof of Exponentiated Gradient Descent is well-studied [Ghai et al., 2020] and here we provide a simple Regret bound for both OCP.

Denote ℓ t,i = ∥ f i ( x ) -y ∥ 2 , recall that the normalizer for OCP-U is Z t +1 = ∑ d i =1 w t,i exp( -ηℓ t,i ) , then we have

where p t,i = w t,i /Z t ≤ 1 . For clarity, we let w t,i be the unnormalized weight and p t,i be the normalized weight. Now, we assume ηℓ t,i ∈ [0 , 1] . Although it is not guaranteed that ℓ t,i will be small under concept shift, it is generally safe to assume that the concept will shift gradually and will not lead to a drastic change in the loss. As a result, the loss will not become arbitrarily large, and we can divide some large constant such that the loss is bounded in a small range. Based on the assumption, we can use the second Taylor expansion of e -x ≤ 1 -x + x 2 / 2 and the inequation log(1 -x ) ≤ -x for x ∈ [0 , 1] . Then we have

Note that w t,i is the unnormalized weight, and then w 1 ,i = 1 and Z 1 = d . Then we can get the lower bound and upper bound of log Z T +1 :

Finally, recall that we set η = √ 2 log( d ) /T , then we have

which completes our proof.

B.2 Theoretical guarantee for the K -step re-initialize algorithm.

The proposed K -step re-initialize algorithm is detailed as follows: at the beginning of the algorithm, we choose w 1 = [ w 1 ,i = 1 /d ] d i =1 as the center point of the simplex and denote ℓ t,i as the loss for f i at time step t , the updating rule for each w i will be w t +1 ,i = w t,i exp( -η [ ∂ L U ( w t )] i ) Z t = w t,i exp( -η ∥ f i ( x ) -y ∥ 2 ) Z t = w t,i exp( -ηℓ t,i ) Z t , where Z t = ∑ d i =1 w t,i exp( -ηl t,i ) is the normalizer.

Different from the native EGD algorithm, we re-initialize the weight w K +1 = [ w K +1 ,i = 1 /d ] d i =1 per K time steps. We call each K step one round. This simple strategy interrupts the influence of the historical information of length K steps on the ensembling weights, which helps the model to quickly adapt to the upcoming environment.

Proposition 2. For T > 2 log( d ) , denote I = [ l, l +1 , · · · , r ] as any period of time of length r -l +1 where l > 1 and r ≤ T . Denote the length of I as a sublinear sequence of T , namely, | I | = T n , where 0 ≤ n ≤ 1 . We choose K = T 2 n 3 . We then have, the K -step re-initialize algorithm has an regret bound R ( I ) ≤ O ( T 2 n/ 3 ) at any I = [ l, l +1 , · · · , r ] . Namely, for any small internal n < 3 4 , we have R ([ l, l +1 , · · · , r ]) < O ( T 1 / 2 )

Proof. We discuss regret in three cases:

At first, according to Proposition 1, if all K steps fall into [ l, l +1 , · · · , r ] , then we have R ( K ) ≤ 2 √ K ln d ( d -1) . There exist L/K rounds that are all contained in [ l, ..., r ] and the regret of these rounds will be O ( L/K ∗ 2 √ K ) .

For the first round, we do not know when the weights are reinitialized ( l may or may not be a multiple of K ) and the performance of the algorithm in historical time. let's think about the worst case, where regret will be less than O ( K ) .

For the last round, we know when the last round begins and the weights are reinitialized. However, some of the future time steps are not in [ l, l +1 , · · · , r ] and we can only treat the case as the first round, which has a regret O ( K ) .

Considering all cases, we have an internal regret that is bounded by

When we choose an small internal length L = T n and K = T m where m ≤ n . To minimize the upper bound, we should choose m = 2 n 3 and the bound will be O ( T 2 n/ 3 ) . Namely, for any small internal n < 3 4 , we have R ([ l, l +1 , · · · , r ]) < O ( T 1 / 2 ) , which is tighter than the regret bound of the EGD algorithm under the whole online sequence. Specifically, when we choose L = T 2 / 3 and K = T 4 / 9 , we have R ([ l, l +1 , · · · , r ]) < O ( T 4 / 9 ) < O ( T 1 / 2 ) . When we choose L = T 1 / 4 and K = T 1 / 6 , then R ([ l, l +1 , · · · , r ]) < O ( T 1 / 6 ) < O ( T 1 / 2 ) .

In other words, the algorithm focuses on short-term information and leads to a better regret bound in any small time interval . However, with increasing length of I , the bound of the simple algorithm will become worse. Consider an extreme case where n = 1 , the bound will be R ([ l, l + 1 , · · · , r ]) < O ( T 2 / 3 ) , which is inferior to the native EGD algorithm.

B.3 Necessary definitions and assumptions for evaluating model adaptation speed to environment changes.

To complete the proofs, we begin by introducing some necessary definitions and assumptions. Given the online data stream x t and its forecasting target y t at time t . Given d forecasting experts with different parameters f t = { f t,i } , denote ℓ as a nonnegative loss function and ℓ t,i := ℓ ( f t,i ( x t ) , y t ) as the loss incurred by f t,i at time t, we define the following notions.

Definition 1. (Weighted average forecaster) . A weighted average forecaster makes predictions by

where w t,i is the weight for expert f i at time t and f t,i is the prediction of f i at time t .

Definition 2. (Cumulative regret and instantaneous regret) . For expert f i , the cumulative regret (or simply regret) on the T steps is defined by

where r t,i is the instantaneous regret of the expert f i at time t , which is the regret that the forecaster feels of not having listened to the advice of the expert f i right after the t th outcome that has been revealed. ˆ L T = ∑ T t =1 ℓ (˜ y t , y t ) is the cumulative loss of the forecaster and L T,i is the cumulative loss of the expert f i .

Definition 3. (Potential function) . We can interpret the weighted average forecaster in an interesting way which allows us to analyze the theoretical properties easier. To do this, we denote r t = ( r t, 1 , . . . , r t,d ) ∈ R d as the instantaneous regret vector, and R T = ∑ T t =1 r t is the corresponding regret vector. Now, we can introduce the potential function Φ : R d → R of the form

where ϕ : R → R is any nonnegative, increasing, and twice differentiable function, and ψ : R → R is nonnegative, strictly increasing, concave, and twice differentiable auxiliary function. With the notion of potential function, the prediction ˜ y t will be

where ∇ Φ( R t -1 ) i = ∂ Φ( R t -1 ) /∂R t -1 ,i . It is easy to prove that the exponentially weighted average forecaster used in Eq. (2) is based on the potential Φ η ( u ) = 1 η ln ( ∑ d i =1 e ηu i ) .

Theorem 1. (Blackwell condition, Lemma 2.1. in [Cesa-Bianchi and Lugosi, 2006].) If the loss function ℓ is convex in its first argument and we use x 1 · x 2 denote the inner product of two vectors, then

The following theorem is applicable to any forecaster that satisfies the Blackwell condition, not limited to weighted average forecasters. Nevertheless, this theorem will lead to several interesting bounds for various variations of the weighted average forecaster.

Theorem 2. (Theorem2.1 in [Cesa-Bianchi and Lugosi, 2006].) Assume that a forecaster satisfies the Blackwell condition for a potential Φ , then for all i = 1 , · · · ,

B.4 Existing theoretical intuition and empirical comparison to the proposed OCP block.

With the help of the two theorems in Section B.3, we now recall that the Internal Regret R in ( t, w ) [Blum and Mansour, 2007] that measures forecaster's expected regret of having taken an action w at step t :

While Proposition 1 ensures that a small external regret can be achieved, ensuring a small internal regret is a more challenging task. This is because any algorithm with a small internal regret also where

has small external regret but the opposite is not true, as demonstrated in [Stoltz and Lugosi, 2005]. The key question now is whether it is possible to define a policy w that attains small (i.e., sublinear in T ) internal regret. For simplicity, we use R as internal regret in this subsection. To develop a forecasting strategy that can guarantee a small internal regret. We define the exponential potential function Φ : R M → R with η > 0 by

where M = d ( d -1) . Here, we denote r t = ( r t, (1 , 1) , r t, (1 , 2) , . . . , r t, ( d,d -1) ) ∈ R d ( d -1) as the instantaneous regret vector and R T = ∑ T t =1 r t is the corresponding regret vector. Then, any forecaster satisfying Blackwell's condition will have a bounded internal regret (Corollary 8 in [CesaBianchi and Lugosi, 2003]) by choosing a proper parameter η :

With the help of the two theorems in Section B.3, we now recall that the Internal Regret R in ( t, w ) [Blum and Mansour, 2007] that measures forecaster's expected regret of having taken an action w at step t :

For simplicity, we use R as internal regret in this subsection. To conduct a forecasting strategy that can guarantee a small internal regret. We define the exponential potential function Φ : R M → R with η > 0 by

where M = d ( d -1) . Here, we denote r t = ( r t, (1 , 1) , r t, (1 , 2) , . . . , r t, ( d,d -1) ) ∈ R d ( d -1) as the instantaneous regret vector and R T = ∑ T t =1 r t is the corresponding regret vector. Then, any forecaster satisfying Blackwell's condition will have a bounded internal regret (Corollary 8 in [CesaBianchi and Lugosi, 2003]) by choosing a proper parameter η :

Now our target is to find a new policy that makes the forecaster satisfy the Blackwell condition.

To ensure that this value is negative or zero, it is sufficient to demand that.































(a) ETTh2.





























(b) ETTm1.

































(c) WTH.





























(d) ECL.

̸

That is, we need to find a new policy vector w t that satisfies w T t A = 0 , where A = { -∇ k,i Φ( R t -1 ) if i = k, ∑ j = i ∇ k,j Φ( R t -1 ) Otherwise. is an d × d matrix. However, determining the existence

̸

of a new policy vector and efficiently calculating its values can be challenging. Even if we assume the new vector exists, the time complexity of calculating the new policy by the Gaussian elimination method[Foster and Vohra, 1999] is O ( d 3 ) , which is expensive, particularly for datasets with a large number of variables such as the ECL dataset with 321 variables and 321*2 policies. To address this issue, we propose the OCP block which utilizes an additional offline reinforcement learning block f rl with parameter θ rl to learn a bias vector b t for the original policy w t . The new policy vector is then defined as ˜ w t = w t + b t . The learned bias pushes the predicted outcomes closer to the ground truth values, that is, we minimize min θ rl L ( ˜ w ) := ∥ ∑ d i =1 ˜ w i f i ( x ) -y ∥ 2 ; s.t. ˜ w ∈ △ to train θ rl . We measure the internal regret max i,j =1 ,...,d w t,i ( ℓ t,i -ℓ t,j ) at each time step empirically. As shown in Figure 5, the proposed method significantly reduces internal regret without the need for constructing and computing a large matrix.

C Additional Experimental Results

C.1 Datasets

We investigate a diverse set of datasets for time series forecasting. ETT [Zhou et al., 2021] 3 logs the target variable of the "oil temperature" and six features of the power load over a two-year period. We also analyze the hourly recorded observations of ETTh2 and the 15-minute intervals of ETTm1 benchmarks. Additionally, we study ECL 4 (Electricity Consuming Load), which gathers electricity

consumption data from 321 clients between 2012 and 2014. The Weather (WTH) 5 dataset contains hourly records of 11 climate features from almost 1,600 locations across the United States.

C.2 Implementation Details

For all benchmarks, we set the look-back window length at 60 and vary the forecast horizon from H = 1 , 24 , 48 . We split the data into two phases: warm-up and online training, with a ratio of 25:75. We follow the optimization details outlined in [Zhou et al., 2021] and utilize the AdamW optimizer [Loshchilov and Hutter, 2017] to minimize the mean squared error (MSE) loss. To ensure a fair comparison, we set the epoch and batch sizes to one, which is consistent with the online learning setting. We make sure that all baseline models based on the TCN backbone use the same total memory budget as FSNet, which includes three times the network sizes: one working model and two exponential moving averages (EMAs) of its gradient. For ER, MIR, and DER++, we allocate an episodic memory to store previous samples to meet this budget. For transformer backbones, we find that a large number of parameters do not benefit the generalization results and always select the hyperparameters such that the number of parameters for transformer baselines is fewer than that for FSNet. In the warm-up phase, we calculate the mean and standard deviation to normalize the online training samples and perform hyperparameter cross-validation. For different structures, we use the optimal hyperparameters that are reported in the corresponding paper.

License. All the assets (i.e., datasets and the codes for baselines) we use include an MIT license containing a copyright notice and this permission notice shall be included in all copies or substantial portions of the software.

Environment. We conduct all the experiments on a machine with an Intel R Xeon (R) Platinum 8163 CPU @ 2.50GHZ, 32G RAM, and four Tesla-V100 (32G) instances. All experiments are repeated 3 times with different seeds.

Metrics Because learning occurs over a sequence of rounds. At each round, the model receives a look-back window and predicts the forecast window. All models are commonly evaluated by their accumulated mean-squared errors (MSE) and mean-absolute errors (MAE), namely the model is evaluated based on its accumulated errors over the entire learning process.

C.3 Baseline details

We present a brief overview of the baselines employed in our experiments.

First, OnlineTCN adopts a conventional TCN backbone [Zinkevich, 2003] consisting of ten hidden layers, each layer containing two stacks of residual convolution filters.

Secondly, ER [Chaudhry et al., 2019] expands on the OnlineTCN baseline by adding an episodic memory that stores previous samples and interleaves them during the learning process with newer ones.

Third, MIR [Aljundi et al., 2019a] replaces the random sampling technique of ER with its MIR sampling approach, which selects the samples in the memory that cause the highest forgetting and applies ER to them.

Fourthly, DER++ [Buzzega et al., 2020] enhances the standard ER method by incorporating a knowledge distillation loss on the previous logits.

Finally, TFCL [Aljundi et al., 2019b] is a task-free, online continual learning method that starts with an ER process and includes a task-free MAS-styled [Aljundi et al., 2018] regularization.

All the ER-based techniques utilize a reservoir sampling buffer, which is identical to that used in [Pham et al., 2023].

C.4 Hyper-parameters

For the hyper-parameters of FSNet and the baselines mentioned in Section C.3, we follow the setting in [Pham et al., 2023]. Besides, we cross-validate the hyper-parameters on the ETTh2 dataset and use them for the remaining ones. In particular, we use the following configuration:

Learning rate 3 e -3 on Traffic and ECL and 1 e -3 for other datasets. Learning rate 1 e -2 for the EGD algorithm and 1 e -3 for the offline reinforcement learning block, where the selection scope is { 1 e -3 , 3 e -3 , 1 e -2 , 3 e -2 } .

Number of hidden layers 10 for both cross-time and cross-variable branches, where the selection scope is { 6 , 8 , 10 , 12 } .

Adapter's EMA coefficient 0 . 9 , Gradient EMA for triggering the memory interaction 0 . 3 , where the selection scope is { 0 . 1 , 0 . 2 , . . . , 1 . 0 } .

Memory triggering threshold 0 . 75 , where the selection scope is { 0 . 6 , 0 . 65 , 0 . 7 . . . , 0 . 9 } .

Episodic memory size: 5000 (for ER, MIR, and DER++), 50 (for TFCL).

C.5 Additional Numerical Results

Additional forecasting results. In this section, we analyze the performance of different forecasting methods on four datasets, ECL, WTH, ETTh2, and ETTm1, with various starting points, as shown in Figure 8, Figure 9, Figure 10, and Figure 11, respectively. For the last three datasets, all methods produce similar results that can capture the underlying time series patterns, and the performance differences are not significant. However, when it comes to the ECL dataset, we observe that almost all baselines exhibit poor forecasting results at the onset of the concept shift (time step 2500). As we provide more instances, the performance of these methods improves, as evidenced by the cumulative loss curves in Figure 7 and Figure 6.

Abltion studies of hyper-parameters. We conduct detailed ablation studies about model layers, learning rate, and model dimension here. Taking into account the learning rate for the two-branch framework, the learning rate for the long-term weight, and the learning rate for the short-term weight: lr, lr w , lr b , as shown in Table 10 (left), the impact of the learning rate on dual-stream

networks is quite significant. The optimal learning rate varies for each dataset, but we can see that for each dataset, the optimal learning rate is generally within the range of [1 e -4 , 1 e -2] . lr w has a relatively small impact on the final performance of the model. On the contrary, the offline-RL module determines whether the weights can quickly adapt to the new distribution, which has a greater impact on the final performance. In terms of model parameters, # Layers, d m , and d head , all three have a significant impact on the performance of the model. A small model may not be able to fit the training data, but a model that is too large increases the risk of overfitting, so each dataset has an optimal model size. However, in this paper, we use the same hyperparameters for all datasets to simplify the complexity of training and model selection. Specifically, we set lr = 1 e -3 , lr w = 1 e -2 , lr b = 1 e -3 , # layers = 10 , d m = 64 , d head = 320 .

C.6 Ensembling more than two networks.

In the main paper, we verify the effectiveness of ensembling two branches with different model biases. Here, we show that the proposed OneNet framework enables us to incorporate more branches and the OCP block can fully utilize the benefit of each branch. As shown in Table 7, incorporating PatchTST to OneNet-TCN will further reduce the forecasting results during online forecasting.

C.7 More visualization results and Convergence Analysis of Different Structures

As shown in Figure 6 and Figure 7, ETTh2 and ECL datasets pose the greatest challenge to all models due to the sharp peaks in their loss curves. When the forecasting window is short, OneNet outperforms all baselines by a significant margin on all datasets. When the forecasting window is extended to H = 48 , FSNet is comparable to OneNet in the first three datasets. However, when concept drift occurs in the ECL dataset, all baselines experience a drastic increase in their cumulative MSE, except OneNet, which maintains a low MSE. Furthermore, the initialized MSE error of OneNet is consistently lower than that of all baselines, thanks to the two-stream structure of OneNet. For instance, in Figure 6(b) and Figure 6(f), OneNet demonstrates a significantly lower MSE than baselines when the number of instances is less than 100.

C.8 Online forecasting results with delayed feedback

As illustrated in Section 2, this paper adopts the same setting as FSNet [Pham et al., 2023], where the true values of each time step are revealed to improve the performance of the model in subsequent rounds. However, in real-world applications, the true values of the forecast horizon H may not be available until H rounds later, which is known as online forecasting with delayed feedback. This setting is more challenging because the model cannot be retrained at each round and we can only train the model per H round. Tables 8 and 9 show the cumulative performance considering MSE and MAE, respectively. As expected, all methods perform worse with delayed feedback than under the traditional online forecasting setting. Notably, the state-of-the-art method FSNet is shown to be sensitive to delayed feedback, particularly when H = 48 , where it is even inferior to a simple TCN baseline on some datasets. In contrast, our proposed method OneNet significantly outperforms all continual learning baselines across different datasets and delayed forecast horizons.

Method

Online

ETTH2

ETTm1

WTH

ECL

Avg

1

24

48

1

24

48

1

24

48

1

24

48

FEDformer-F

%

1.922

3.045

4.016

0.922

1.003

1.821

3.544

2.344

1.179

43.852

37.802

37.377

11.569

!

1.912

3.013

3.951

0.372

0.633

0.586

2.196

0.376

0.562

39.243

35.975

36.092

10.409

FEDformer-W

%

1.816

3.070

3.996

2.275

3.784

2.662

1.220

1.211

1.431

41.791

37.236

37.210

11.475

!

1.798

2.993

1.623

0.235

0.451

0.516

0.717

0.962

0.372

21.387

24.600

27.640

6.941

PatchTST

%

0.427

2.090

3.290

0.083

0.433

0.570

0.163

0.375

0.467

2.030

4.395

5.101

1.619

!

0.362

1.622

2.716

0.083

0.427

0.553

0.162

0.372

0.465

2.022

4.325

5.030

1.512

Crossformer

%

23.270

28.904

29.218

0.400

1.433

1.691

0.146

0.327

0.426

469.260

475.490

478.270

125.736

!

9.873

2.856

5.772

0.096

0.356

0.370

0.149

0.317

0.359

68.300

92.500

94.790

22.978

TCN

%

27.060

27.760

26.320

2.240

12.170

10.880

0.290

0.480

0.580

538.000

546.000

552.000

145.315

!

0.530

0.930

0.910

0.130

0.310

0.250

0.300

0.348

0.348

3.010

11.680

10.800

2.462

Time-TCN

%

4.530

7.840

1.300

0.097

0.800

1.030

0.162

0.344

0.429

47.900

48.660

67.150

15.020

!

0.480

0.780

1.300

0.090

0.280

0.310

0.300

0.310

0.309

4.010

5.220

5.210

1.550

DLinear

%

2.91

10.25

7.53

0.538

1.461

1.233

0.266

0.462

0.542

12.03

51.28

58.46

12.247

!

2.44

9.24

6.91

0.46

1.3

1.12

0.262

0.459

0.541

6.69

27.82

31.54

7.399

NLinear

%

0.424

50.15

49.52

0.09

4.02

4.13

0.171

1.07

1.08

2.14

930

929

164.316

!

0.369

50.24

49.6

0.089

4.035

4.141

0.171

1.053

1.064

2.135

930

930

164.408

TS-Mixer

%

1.968

3.525

4.88

0.335

0.726

0.855

0.255

0.429

0.503

11.16

30.93

44.68

8.354

!

0.78

2.05

3.060

0.219

0.550

0.660

0.237

0.413

0.482

2.798

4.983

5.764

1.833

C.9 The effect of variable independence and frequency domain augmentation

As shown in Table 11, we observe that frequency-enhanced blocks, which use the wavelet transform, offer greater robustness to the Fourier transform. FEDformer outperforms TCN in terms of generalization, but online adaptation has a limited impact on performance, similar to other transformer-based models. Notably, we find that variable independence is crucial for model robustness. By convolving solely on the time dimension, independent of the feature channel, we significantly reduce MSE error compared to convolving on the feature channel, regardless of whether online adaptation is applied.

C.10 Comparison of existing forecasting structures.

Results are shown in Table 12. Considering the average MSE on all four datasets, all transformerbased models and Dlinear are better than TCN and Time-TCN. However, with online adaptation, the forecasting error of TCN structures is reduced by a large margin and is better than DLinear and FEDformer. Specifically, we show that the current transformer-based model (PatchTST [Nie et al., 2023]) demonstrates superior generalization performance than the TCN models even without any online adaptation , particularly in the challenging ECL task. However, we also noticed that PatchTST remains largely unchanged after online retraining. In contrast, the TCN structure can quickly adapt to the shifted distribution, and the online updated TCN model prefers a better forecasting error than the adapted PatchTST on the first three data sets. Therefore, it is promising to combine the strengths of both structures to create a more robust and adaptable model that can handle shifting data distributions better.

Algorithm 1 Training and inference algorithm of OneNet

1: Input: Historical multivariate time series x ∈ R M × L with M variables and length L , the forecast target y ∈ R M × H , where we omit the variable index and time step index for simplicity.

2: Initialize a cross-time forecaster f 1 , a cross-variable forecaster f 2 with corresponding prediction head, long-term weight w = [0 . 5 , 0 . 5] , short term learning block f rl : R H × M × 3 → R 2 , and step size η for long-term weight updating.

3: Get prediction results from two forecasters.

4: ˜ y 1 ∈ R M × H = f 1 ( x ) . // Prediction result from the cross-time forecaster.

5: ˜ y 2 ∈ R M × H = f 2 ( x ) . // Prediction result from the cross-variable forecaster.

6: Get combination weight from the OCP block.

7: b = f rl ( w 1 ˜ y 1 ⊗ w 2 ˜ y 2 ⊗ y ) // Calculate the short term weight.

8: ˜ w i = ( w i + b i ) / ( ∑ d i =1 ( w i + b i ) ) // Calculate the normalized weight.

9: ˜ y = w 1 ∗ ˜ y 1 + w 2 ∗ ˜ y 2 .

// The final prediction result.

10: Update the long/short term weight.

12: f rl ← Adam ( f rl , L ( w 1 ∗ ˜ y 1 + w 2 ∗ ˜ y 2 , y ))

//Parameters such as learning rate are omitted.

13: Update the two forecasters.

14: f 1 ← Adam ( f 1 , L ( ˜ y 1 , y )) , f 2 ← Adam ( f 2 , L ( ˜ y 2 , y ))



































(a) ETTh2.



































(b) ETTm1.





































(c) WTH.



































(d) ECL

































(e) ETTh2.





































(f) ETTm1.









































(g) WTH.

































(h) ECL

































(a) ETTh2.



































(b) ETTm1.





































(c) WTH.



































(d) ECL

































(e) ETTh2.





































(f) ETTm1.









































(g) WTH.





































(h) ECL

D

(a) Channel

1

.

(b) Channel

2

.

(c) Channel

3

.

(d) Channel

1

.

(e) Channel

2

.

(f) Channel

3

.

(g) Channel

1

.

(h) Channel

2

.

(i) Channel

3

.

(j) Channel

1

.

(k) Channel

2

.

(l) Channel

3

.

2

0

0

Value

Value

Value

-1

0

Ground Truth

Ground Truth

-1

GroundTruth

2

TCN

TCN

TCN

-1

FSNet

FSNet

FSNet

-3

OneNet

OneNet

OneNet

2

5000

5010

5020

5030

5040

5050

5000

5010

5020

5030

5040

5050

5000

5010

5020

5030

5040

5050

#Instances

#Instances

#Instances

2

Ground Truth

4

Ground Truth

TCN

TCN

3

1

FSNet

FSNet

0

OneNet

OneNet

Value

Value

2

Value

0

1

Ground Truth

TCN

0

-2

FSNet

OneNet

2

-1

7500

7510

7520

7530

7540

7550

7500

7510

7520

7530

7540

7550

7500

7510

7520

7530

7540

7550

#Instances

#Instances

#Instances

1.0

2.5

Ground Truth

2

GroundTruth

Ground Truth

TCN

TCN

0.5

TCN

2.0

FSNet

FSNet

FSNet

Value

OneNet

alue

OneNet

Value

0.0

OneNet

1.5

0

0.5

1.0

1.0

0.5

-2

1.5

10000

10010

10020

10030

10040

10050

10000

10010

10020

10030

10040

10050

10000

10010

10020

10030

10040

10050

#Instances

#Instances

#Instances

0.0

0

0

0.5

Value

lue

lue

-1

-1.0

Ground Truth

GroundTruth

-2

Ground Truth

1.5

TCN

-2

TCN

TCN

FSNet

FSNet

3

FSNet

2.0

OneNet

OneNet

OneNet

12500

12510

12520

12530

12540

12550

3

12500

12510

12520

12530

12540

12550

12500

12510

12520

12530

12540

12550

#Instances

#Instances

#Instances

(a) Channel

1

.

(b) Channel

2

.

(c) Channel

3

.

(d) Channel

1

.

(e) Channel

2

.

(f) Channel

3

.

(g) Channel

1

.

(h) Channel

2

.

(i) Channel

3

.

(j) Channel

1

.

(k) Channel

2

.

(l) Channel

3

.

1.75

0.5

1.5

1.50

1.25

Ground Truth

Value

0.0

Value

1.0

三

1.00

TCN

FSNet

0.5

Ground Truth

0.5

Ground Truth

0.75

OneNet

TCN

TCN

FSNet

FSNet

0.50

-1.0

OneNet

0.0

OneNet

0.25

5000

5010

5020

5030

5040

5050

5000

5010

5020

5030

5040

5050

5000

5010

5020

5030

5040

5050

#Instances

#Instances

#Instances

1.0

Ground Truth

Ground Truth

0.0

0.8

TCN

0.0

TCN

FSNet

FSNet

0.5

Value

0.6

OneNet

Value

0.5

OneNet

Value

-1.0

0.4

1.0

Ground Truth

1.5

TCN

0.2

FSNet

1.5

0.0

2.0

OneNet

7500

7510

7520

7530

7540

7550

7500

7510

7520

7530

7540

7550

7500

7510

7520

7530

7540

7550

#Instances

#Instances

#Instances

0

0.5

0.5

Value

-1

Value

1.0

alue

1.0

2

Ground Truth

1.5

Ground Truth

1.5

Ground Truth

3

TCN

TCN

TCN

FSNet

2.0

FSNet

2.0

FSNet

OneNet

OneNet

OneNet

10000

10010

10020

10030

10040

10050

10000

10010

10020

10030

10040

10050

10000

10010

10020

10030

10040

10050

#Instances

#Instances

#Instances

1.5

1.5

1.5

1.0

1.0

1.0

Value

Value

Value

0.5

0.5

Ground Truth

GroundTruth

0.5

Ground Truth

TCN

TCN

TCN

0.0

FSNet

0.0

FSNet

FSNet

OneNet

OneNet

0.0

OneNet

12500

12510

12520

12530

12540

12550

12500

12510

12520

12530

12540

12550

12500

12510

12520

12530

12540

12550

#Instances

#Instances

#Instances

(a) Channel

1

.

(b) Channel

2

.

(c) Channel

3

.

(d) Channel

1

.

(e) Channel

2

.

(f) Channel

3

.

(g) Channel

1

.

(h) Channel

2

.

(i) Channel

3

.

(j) Channel

1

.

(k) Channel

2

.

(l) Channel

3

.

Ground Truth

Ground Truth

1.0

TCN

0

1.0

TCN

FSNet

FSNet

Value

0.5

OneNet

Value

-1

alue

0.5

OneNet

-2

GroundTruth

0.0

-3

TCN

0.0

FSNet

0.5

OneNet

0.5

2500

2510

2520

2530

2540

2550

2500

2510

2520

2530

2540

2550

2500

2510

2520

2530

2540

2550

#Instances

#Instances

#Instances

1.0

Ground Truth

1.5

TCN

1.5

2.0

0.5

FSNet

OneNet

Value

Value

2.5

Value

0.0

2.0

Ground Truth

3.0

Ground Truth

TCN

3.5

TCN

0.5

2.5

FSNet

FSNet

OneNet

4.0

OneNet

-1.0

5000

5010

5020

5030

5040

5050

5000

5010

5020

5030

5040

5050

5000

5010

5020

5030

5040

5050

#Instances

#Instances

#Instances

Ground Truth

1.0

3.0

TCN

0.0

FSNet

Value

1.5

Value

3.5

OneNet

Value

0.5

2.0

Ground Truth

4.0

Ground Truth

1.0

2.5

TCN

TCN

FSNet

4.5

FSNet

3.0

OneNet

1.5

OneNet

7500

7510

7520

7530

7540

7550

7500

7510

7520

7530

7540

7550

7500

7510

7520

7530

7540

7550

#Instances

#Instances

#Instances

0.5

Ground Truth

3.25

Ground Truth

1.5

TCN

3.50

TCN

0.0

FSNet

FSNet

Value

0.5

OneNet

lue

3.75

OneNet

Value

1.0

4.00

1.0

0.5

Ground Truth

4.25

TCN

1.5

4.50

0.0

FSNet

2.0

4.75

OneNet

10000

10010

10020

10030

10040

10050

10000

10010

10020

10030

10040

10050

10000

10010

10020

10030

10040

10050

#Instances

#Instances

#Instances

(a) Channel

1

.

(b) Channel

2

.

(c) Channel

3

.

(d) Channel

1

.

(e) Channel

2

.

(f) Channel

3

.

(g) Channel

1

.

(h) Channel

2

.

(i) Channel

3

.

(j) Channel

1

.

(k) Channel

2

.

(l) Channel

3

.

Ground Truth

2.5

Ground Truth

Ground Truth

1.0

TCN

TCN

1.0

TCN

FSNet

2.0

FSNet

FSNet

0.5

OneNet

OneNet

0.5

OneNet

Value

lue

1.5

Val

Val

0.0

0.0

1.0

0.5

0.5

0.5

1.0

0.0

1.0

2500

2510

2520

2530

2540

2550

2500

2510

2520

2530

2540

2550

2500

2510

2520

2530

2540

2550

#Instances

#Instances

#Instances

Ground Truth

1.0

Ground Truth

Ground Truth

0.5

TCN

TCN

0

TCN

FSNet

0.5

FSNet

FSNet

1.0

OneNet

OneNet

OneNet

Value

Value

0.0

lue

-1.5

-1

0.5

-2.0

-1.0

-2

-2.5

1.5

5000

5010

5020

5030

5040

5050

5000

5010

5020

5030

5040

5050

5000

5010

5020

5030

5040

5050

#Instances

#Instances

#Instances

2

Ground Truth

1.0

3

Ground Truth

TCN

TCN

FSNet

0.5

2

FSNet

OneNet

OneNet

Value

0

Value

0.0

Value

1

0.5

Ground Truth

0

TCN

1.0

FSNet

-1

-2

OneNet

-2

7500

7510

7520

7530

7540

7550

7500

7510

7520

7530

7540

7550

7500

7510

7520

7530

7540

7550

#Instances

#Instances

#Instances

Ground Truth

1.0

0.0

TCN

0.0

FSNet

0.5

0.5

Value

OneNet

Value

Value

0.5

-1.0

0.0

Ground Truth

1.0

Ground Truth

TCN

TCN

1.5

0.5

FSNet

1.5

FSNet

OneNet

OneNet

2.0

10000

10010

10020

10030

10040

10050

10000

10010

10020

10030

10040

10050

10000

10010

10020

10030

10040

10050

#Instances

#Instances

#Instances

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a method for online learning based on ensemble learning. The method proposed by the authors leverages cross-variable and cross-time dependency. In addition, the authors propose a reinformed learning method that dynamically adjusts the weights used to combine the models.\n\nSTRENGTHS:\nSee Question section\n\nWEAKNESSES:\nSee Question section\n\nQUESTIONS:\nThe paper is well-presented and well-organized. However, the literature review is suggested to be enriched. The existing studies are hardly investigated. Online learning literature not based on deep learning needs to be further studied, there is a lack of state-of-the-art references. Moreover, cross-variable dependency has already been studied in online learning as well as ensemble learning based methods.\n\nCross-variable dependency reference:\n\n“Minimax classification under concept drift with multidimensional adaptation and performance guarantees”\n\nEnsemble learning references:\n\n“Online Ensemble Multi-kernel Learning Adaptive to Non-stationary and Adversarial Environments”\n\n“Dynamic Weighted Majority: An Ensemble Method for Drifting Concepts”\n\n“Incremental Learning of Concept Drift in Nonstationary Environments”\n\nI think one improvement the authors can consider is to add some discussions about the computational and memory complexity. In online learning, we sometimes need to obtain real-time predictions. \n\nComparing the theoretical results with similar results from the state-of-the-art would help to show the contributions of the presented bounds.\n\nReferences:\n\n“Online Ensemble Multi-kernel Learning Adaptive to Non-stationary and Adversarial Environments”\n\n“An adaptive gradient method for online AUC maximization”\n\n“Large Scale Online Kernel Learning”\n\nThe reviewer suggests making the code available so that the results can be replicated.\n\nLIMITATIONS:\n.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nMotivated by the observation that a single model may not consistently achieve optimal performance in time series forecasting, the author proposes a method, which combines two stream architectures (cross-time and cross-variable models) using online convex programming. However, due to the limitations of online convex programming, which involves a trade-off between switching speed and overall performance, an RL-based algorithm is introduced as a solution. The proposed method is reinforced by a range of experimental results, providing support for its effectiveness.\n\nSTRENGTHS:\n1. Easy to follow:\nThe paper is excellently written and presented in a manner that is easy to comprehend and follow along.\n\n2. Novelty:\nWhile ensemble methods for time series tasks have been previously proposed, the specific approach taken in this paper, such as the adoption of OCP and RL for model ensembling, is novel and has not been introduced before. This novelty serves as a notable advantage of this paper.\n\n3. Concrete experimental results:\nThe paper provides a comprehensive comparison between its proposed method and numerous baselines across various dataset settings. Additionally, to assess the effectiveness of their ensembling approach, the paper includes comparisons with multiple baseline ensembling methods. These extensive and well-designed experimental results serve to reinforce the credibility and robustness of the paper.\n\nWEAKNESSES:\nThere are minor issues in this paper.\n\n1. About the motivation:\nIn the introduction section, the author presents Figure 1 and Table 1 as part of the motivation for their work. The main motivation behind this study is the recognition that a single model is insufficient to address complex time series problems, leading to the proposal of an ensemble method. However, the necessity of Table 1 to explain this motivation is not clear. Regarding Table 1, the author mentions that \"cross-time forecasters tend to exhibit lower performance in certain datasets, and existing models that leverage both cross-variable and cross-time information perform worse than the naive TCN method.\" This sentence may not be directly relevant to the motivation, and it would be beneficial to rephrase it to establish a closer connection between the sentence and the motivation.\n\n2. Ablation Study:\nIt would be valuable to include experimental results from ablating the offline RL for short-term weight. This ablation study can provide concrete evidence for the necessity of this module in a more tangible manner.\n\nQUESTIONS:\nIn addition to the question raised in the weaknesses section, I have another question.\n\nCould you provide a more detailed algorithmic description that illustrates the training or inference process using the \\usepackage{algorithm} package? This would greatly aid in understanding the intricacies of the algorithm.\n\nLIMITATIONS:\nThe paper gives good motivation, novel methods, and concrete experimental results. I think this paper deserves acceptance. \nAddressing these concerns can further enhance the overall quality and strength of the paper. If some minor concerns which posed in weaknesses and questions section, I will raise my points.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-02"}

comments: SUMMARY:
This paper addresses the problem of concept drift in online Time Separate forecasting. A method OneNet is proposed that uses temporal correlations and inter-variables dependencies. OneNet is compared with other state of the art methods and OneNet is demonstrated to be superior. Furthermore, through empirical studies various factors are studies that effect the forecasting

STRENGTHS:
1. Mathematicsl formulation of the problems and OneNet solution
2. Training of 2 independent forecasts: one for tempra and second for the inter-variables dependencies.
3. The use of long term historical and short term variations
4. Ablation study and analysis
5. Comparison with state of art methods

WEAKNESSES:
Does not describe the datasets used for the experiments, just started using abbreviation in tables.

QUESTIONS:
Will it be possible to test the proposed OneNet using generated data, so that all the benefits can be highlighted with controlled experiments

LIMITATIONS:
No Limitations are listed
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-07

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces an approach called Online ensembling Network (OneNet), aimed at online updating of time series forecasting models. This approach is crafted to handle the concept drift problem efficiently using streaming data. The authors note that while some algorithms designed for online time series forecasting exploit cross-variable dependency, others presume independence among variables. Each assumption carries its own strengths and weaknesses when it comes to online time series modeling. To address this, the authors' OneNet dynamically updates and combines two models. One model emphasizes on modeling the dependency across the time dimension, while the other is concentrated on cross-variate dependency. The OneNet method infuses a reinforcement learning-based strategy into the conventional online convex programming framework. This integration allows for the linear combination of the two models with dynamically adjusted weights. By doing so, OneNet addresses a significant drawback of classic online learning methods, which often struggle to adapt swiftly to the concept drift. Empirical findings showcase that OneNet diminishes online forecasting error by over 50% compared to the State-Of-The-Art (SOTA) method.\n\nSTRENGTHS:\n1. This paper is easy to follow and read. The considered problem is important in the real world.\n\n2. The proposed method works well in many experiments, verifying its effectiveness. The experiments are solid.\n\nWEAKNESSES:\n1. This paper's technical novelty is limited. The authors do not analyse why their method works in theory. Updating techniques itself is not novel, thus theoretical analysis is necessary for this paper, showing the proposed method should work well and work better than other methods in theory. The proposed theory of this paper is not specific and the corresponding bound is not tight and is less informative.\n\n2. In the introduction, I do not see a relevant discussion regarding the theory of online learning.\n\n3. In time series, there will be dependence among data. Will this kind of dependency influence the generalization result? Under what kind of assumption, it will not influence the generalization result?\n\nQUESTIONS:\nsee above.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}