OneNet: Enhancing Time Series Forecasting Models under Concept Drift by Online Ensembling
Authors: YiFan Zhang
Qingsong Wen
Xue Wang
Weiqi Chen
Liang Sun
Zhang Zhang
Liang Wang
Rong Jin
Tieniu Tan
Venue: NeurIPS 2023 poster (2023)
Keywords: Time series forecasting
concept drift
online learning
online convex programming
TL;DR: We propose OneNet, which trains models with different inductive biases and uses a data-driven combination weight to ensemble them. OneNet reduces online forecasting error under concept drift by more than $\mathbf{50\%}$ compared to the SOTA model.

Abstract: Online updating of time series forecasting models aims to address the concept drifting problem by efficiently updating forecasting models based on streaming data. Many algorithms are designed for online time series forecasting, with some exploiting cross-variable dependency while others assume independence among variables. Given every data assumption has its own pros and cons in online time series modeling, we propose **On**line **e**nsembling **Net**work (**OneNet**). It dynamically updates and combines two models, with one focusing on modeling the dependency across the time dimension and the other on cross-variate dependency. Our method incorporates a reinforcement learning-based approach into the traditional online convex programming framework, allowing for the linear combination of the two models with dynamically adjusted weights. OneNet addresses the main shortcoming of classical online learning methods that tend to be slow in adapting to the concept drift. Empirical results show that OneNet reduces online forecasting error by more than $\mathbf{50}\\%$ compared to the State-Of-The-Art (SOTA) method.

OneNet: Enhancing Time Series Forecasting Models under Concept Drift by Online Ensembling
Yi-Fan Zhangl Qingsong Wen Xue WangS, Weiqi Chen Liang Sun
Zhang Zhang 1.2, Liang Wang!.?, Rong Jin Tieniu 1.2
School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS)
2 State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation Alibaba Group
Abstract
Online updating of time series forecasting models aims to address the concept drifting problem by efficiently updating forecasting models based streaming data. Many algorithms are designed for online time series forecasting, with some exploiting cross-vaie variable dependency while others assume independence among variables. Given every data assumption has its own pros and cons in online time series modeling, we propose Online ensembling Network (OneNet). It dynamically updates and combines two models, with one focusing on modeling the dependency across the time dimension and the other cross-variate dependency. Our method incorporates a reinforcement learning- based approach into the traditional online convex programming framework, allowing for the linear combination of the two models with dynamically adjusted weights. OneNet addresses the main shortcom- ing of classical online learning methods that tend to be slow in adapting the concept drift. Empirical results show that OneNet reduces online forecasting error by more than 50% compared to the Of (SOTA) method. The code is available at https: : //gi thub. com wwt./ooooaaaaaa yf/.aggfhagggggg 1t.eeNlee..
Introduction
In recent years, we have witnessed significant increase in research efforts that apply deep learning to time series forecasting [Lim and Zohren, 2021, Wen et al., 2022]. Deep models have proven to perform exceptionally well only in forecasting tasks, but also in representation learning, enabling the extraction of abstract representations that can be effectively transferred to downstream tasks such as classification and anomaly detection. However, existing studies have focused mainly on the batch learning setting, assuming that the entire training dataset is available beforehand, and the relationship between the imput and output variables remains constant throughout the learning process. These approaches fall short in real-world applications where concepts are often not stable but change over time, known as concept drift [Tsymbal, 2004], where future data exhibit patterns different from those observed in the past. In such cases, re-training the model from scratch could be time-consuming. Therefore, desirable to train the deep forecaster online, incrementally updating the forecasting model with new samples to capture the changing dynamics in the environment.
The real world setting, termed online forecasting, poses challenges such as high noisy gradients compared to offline mini-batch training [Aljundi et al., 2019a], and continuous distribution shifts which can make the model learned from historical data less effective for the current prediction. While some studies have attempted to address the issues by designing advanced updating structures or learning objectives [Pham et al., 2023, You et al., 2021], they all rely on TCN backbones [Bai et al.,
Cooresponding author I Work done at Alibaba Group, and now affiliated with Meta.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Table I: A motivating example for online ensembling, where the reported metric is MSE and the forecast horizon length is set as 48. Cells are colored on the basis of the MSE value, from low (red) to medium (white) to high (blue). Columns titled cross-variable refer to methods that focus on modeling cross- variable dependence, and columns titled cross-tinn refer to methods that only exploit the temporal dependence and assume independence among covariates, All methods use the same training and online adaptation strategy.
Cross Cross-Variable Variable
Cross-Time
Both
Ours
Dataset
#Variables
TCN
FSNet
Time-TCN
DLinear
PatchTST
CrossFormer
TS-Mixer
Fedformer
ETTh2
0.910
0.846
1.307
6,910
2,716
5.772
3.060
1.620
0.609
ETTml
7
0,250
0.127
0.308
1.120
0.553
0.370
0.660
0,516
0.108
WTH
21
0,348
0,223
0.308
0,541
0.465
0.317
0,482
0.372
0.200
ECL
321
10.800
7.034
5.230
7.388
5,030
94.790
5.764
27.640
2.201
Crans Taer Choas Vantali
Duno Time Ortnn
07
Doso Tine
AM
Oruaii
cs
Tiiss
Tiles stap
(a) ETTh2
(b) ETTml.
WTH
(d) ECL.
Figure I: A motivating example for online ensembling, where the reported metric is MSE and forecast horizon length is set to 48 during online adaptation. Tross-Time refers to a TCN backbone that assumes independence among covariates and only models the temporal dependence, and cross- variable refers to TCN backbone that takes into cross-variable dependence.
2018], which do not take advantage of more advanced network structures, such as transformer [Nie al., 2023, Zhou al., 2022b]. Our studies show that the current transformer-basee based model, PatchTST [Nie et al., 2023], without any advanced adaption method of online learning, performs better than the SOTA online adaptation model FSNet [Pham et al., 2023], particularly for the challenging ECL task (Table.1) 1). Furthermore, we find that variable independence is crucial for the robustness of PatchTST. Specifically, PatchTST focuses on modeling temporal dependency (cross-timg dependency) and predicting each variable independently. To validate the effectiveness of the variable independence assumption, we designed Time-TCK TCN, which convolves only on the temporal dimension. Time-TCh is better than FSNet, state-of of- -hethar-tt approach for online forecasting, and achieves significant gains compared to the commonly used TCN structure that convolves on variable dimensions.
Although variable independence enhances model robustness, the vaossbvariale dependency is also critical for forecasting, i.e. for a specific variable, information from associated series in other variables may improve forecasting results. As shown in Table 1 for datasets ETTml and ETTh2, cross-time forecasters tend to yield lower performance for datasets with a small number of variables. Surprisingly, existing models that are designed to leverage both pross-variable and cross-time dependencies such as CrossFormer [Zhang and Yan, 2023] and TS-Mixer [Chen et al., 2023], tend to perform worse than a native TCN. To investigate this phenomenon, we visualized the MSE at different time steps during the entire online adaptation process for both a Cross-Tine model (Time-TCN) and Cross Variable model (TCN) in Figure 1. We observe a large fluctuation in MSE over online adaption, indicating significant concept drift over time. We also observe that neither of these two methods performs consistently better than the other, indicating that neither of the two data assumptions holds true for the entire time series. This is why relying a single model like CrossFormer cannot solve this problem. Existing work depends on a simple model, but for online time series forecasting, data preferences for model bias will continuously change with online concept drifts. Therefore, we need a data-dependent strategy to continuously change the model selection policy. In other words, online time series forecasting should go beyond parameter updating.
In this paper, we address the limitation of single model for online time series forecasting by intro- ducing an ensemble of models that share different data biases. We then learn to dynamically combine the forecasts from individual models for better prediction. By allowing each model to be trained and online updated independently, we can take the best out of each online model; by dynamically
2

---

adjusting the combination of different models, we can take the best out of the entire model ensemble. We refer to our approach as Online Ensembling Network or OneNet for short. More concretely. OneNet maintains two online forecasting models, one focused on modeling temporal correlation and one focused on modeling cross-variable dependency. Each model is trained independently using the same set of training data. During testing, a reinforcement learning (RL) based approach is developed to dynamically adjust the weights used to combine the predictions of the two models. Compared to classical online learning methods such as Exponentiated Gradient Descent, our RL-based approach is more efficient in adapting to the changes/drifts in concepts, leading to better performance. The contributions of this paper
We introduce OneNet, a two-stream architecture for online time series forecasting that integrates the outputs of two models using online convex programming. OneNet leverages the robustness of the variable-independennt model in handling concept drift, while also capturing the inter-dependencies among different variables to enhance forecasting accuracy. Furthermore, we propose an RL.-based online learning approach to mitigate the limitations of traditional OCP algorithms and demonstrate its efficacy through empirical and theoretical analyses.
Our empirical studies with four datasets show that compared with state-of-the-art methods, OneNet reduces the average cumulative mean-squared errors (MSE) by 53.1% and mean-absolute errors (MAE) by 34.5%. In particular, the performance gain on challenging dataset ECL is superior, where the MSE is reduced by 59.2% and MAE is reduced by 63.0%.
3. We conducted comprehensive empirical studies investigate how commonly used design choices for forecasting models, such as instance normalization, variable independence, seasonal-trend de- composition, and frequency domain augmentation, impact the model's 's robustness. In addition, systematically compared the robustness of existing Transformer-basel models, TCN-based models, and MLP-based models when faced with concept drift.
2 Preliminary and Related Work
Concept drift. Concepts in the real world are often dynamic and can change over time, which is especially true for scenarios like weather prediction and customer preferences. Because of unknown changes in the underlying data distribution, models learned from historical data may become inconsistent with new data, thus requiring regular updates to maintain accuracy. This phenomenon, known as concept drift [Tsymbal, 2004], adds complexity to the process of learning a model from data. In this paper, we focus on online learning for time series forecasting. Unlike most existing studies for online time series forecasting [Li et al., 2022, Qin et al., 2022, Pham et al., 2023] that only focus on how to online update their models, this work goes beyond parameter updating and introduces multiple models and a learnable ensembling weight, yielding rich and flexible hypothesis space. Due to the space limit, more related works about time series forecasting and reinforcement learning are left in the appendix.
Online time series forecasting: streaming data. Traditional time series forecasting tasks have a collection of multivariate time series with a look-back window L: (xx) where each x; is M-channel vector x, = == Given a forecast horizon H, the target is to forecast H future values [=t) In real-world applications, the model builds on the historical data needs to forecast the future data, that is, given time offset K' > L,, and (x the model needs to forecast
(x)*' =x Online time series forecasting [Anava et al., 2013, Liu et al., 2016, Pham et al., 2023] is a widely used technique in real-world due to the sequential nature of the data and the frequent drift of concepts. In this approach, the learning process takes place over a sequence of rounds, where the model receives look-back window and predicts the forecast window. The true values are then revealed to improve the model's performance in the next rounds. When we perform online adaptation, the model is retrained using the online data stream with the MSE loss over each channel: L i Σح & # - ||-
Variable-independent time series forecasting. The traditional cross-variable strategy used in most structures takes the vector of all time series features as input and projects into the embedding space to mix the information. On the contrary, PatchTST [Nie et al., 2023] adopts variable-independentt approach, where each input token only contains information from a single channel/variable. Our research demonstrates that variable independence is crucial for boosting model robustness under
3

---

My + yi
W2 39
EGD for Long term Weight
Wt Wt-1. 186600-9
OCP
W2-1,1
Ma-12 V1200-y1
Offline RL Short-term Weight
MSE Loss
MSE Loss
We My = Was + by Waa + Bg
5% BoM
Linear
Linear
Action (Shortstem weight by b2)
2g E Modd
Cross-Time Time Forecaster A
Cross Variable Forecaster f2
2g E Riidd
Dmop
Linsar
Way
Input Multivariate Series RLXM
(Prodiction)
(a) The overall OneNet architecture.
(b) OCP block.
Figure 2: (a) OneNet processes multivariate data through cross-time and cross-variable branches, each responsible for capturing different aspects. The weights of these two branches are generated by the OCP block, and only the black arrows require execution during training. (b) The OCP block produces ensembling weights by utilizing both the long-term history of exponential gradient descent (EGD) and the short-term history of offline reinforcement learning (RL).
concept drift. For multivariate time series samples (a/) each channel j is fed into the model independently, and the forecaster produces prediction results (1)) accordingly. As shown in Table 1, cross-variable methods tend to overfit when the dataset has a large number of variables, resulting poor performance. This is evident in the poor performance the SOTA online adaptation model FSNet [Pham et al., 2023] in the ECL dataset. However, models that lack cross-variable information perform worse on datasets with a small number of variables where cross-variable dependency can be essential. Although some existing work has attempted to incorporate both cross- variable interaction and temporal dependency into a single framework, our experiments show that these models are fragile under concept drift and perform better than the proposed simple baseline, Time-TCN. To address this, we propose a novel approach that trains two separate branches, each focusing modeling temporal and cross- voosiblri dependencies, respectively. We then combine the results of these branches to achieve better forecasting performance under concept drift. We first introduce the OCP block for coherence.
OneNet: Ensemble Learning for Online Time Series Forecasting
We first examine online learning methods to dynamically adjust combination weights used by ensemble learning. We then present OneNet, an ensemble learning framework for online time series forecasting.
3.1 Learning the best expert by Online Convex Programming (OCP)
For notation clarity, here we denote x E Lx as the historical data, y E H as the forecast target. Our current method involves the integration of multiple complementary models. Therefore, how to better integrate model predictions in the online learning setting is an important issue. Exponen- tiated Gradient Descent (EGD) [Hill and Williamson, 2001] a commonly used method. Specifically, the decision space A is d-dimensional simplex, i.e. A (wr/wr. O and || w, ||== 1). where t the time step indicator and we omit the subscript for simplicity when it's not confusing. Given the online data stream x, its forecasting target y, and d forecasting experts with different parameters = [ = f(())]{= the player's goal to minimize the forecasting error as
min L(w) :- Σ wxf((x) - -y 112: s.t. WEA.
(1)

---

According to EGD, choosing w1 = |w1,$ 1/d];-1 as the center point of the simplex and denoting l, as the loss of f. time step t, the updating rule for each w, will be
W1+1,1 = W1, (x.exp(-1 -n fi(x) y I2) Wr,4 exp( 771,)
Ze
Zz
where Z₁ -Ei i exp(-1)ha.) is the normalizer, and the algorithm has regret bound: Proposition 1. (Online Convex Programming Bound) For T > 2 log(d), denote the regret for time step 1. 1= -....., TT as R(T), set /2log(d)/22 the OCP updating policy have an External regret (See appendix B.1 for proof and analysis.)
T
T ΣΣ Was ll [xx) - y 112 inf Σ C(u) V log()
T
Σ f(w.) - inf Σ ((u)=
S
(3)
That is, the exponentially weighted average forecaster guarantees that the forecaster's cumulative expected loss is not much larger than the cumulative loss of the best decision. However, an expo- nentially weighted average forecaster widely known to respond very slowly to drastic changes in the distribution [Cesa-Bianchi and Lugosi, 2006]. This phenomenon is sometimes referred to as the "slow switch phenomenon" in online learning literature, and is further illustrated in Figure 3 where the loss for fi is O for the first 50 trials and for the next 50 trials. The performance of f2 is the opposite. When the step size rj is small (e.g.. 1/ = 0.01), small changes are made to the weights and no clear adaptation takes place. When large step size is applied (e.g.. 1/ 1), we observe that the EGD algorithm quickly adapts to the environment change for the first 50 trials by increasing weight wi to almost in the first few iterations. But it takes many iterations for the EGD algorithm to adapt to the change in the next 50 iterations, where f2 works much better than f1. We finally note that no matter how we adjust the step size 77, the EGD algorithm has suffer from the trade-off between speed of switching and overall good performance throughout the horizon.
Although few algorithms have been developed to address this issue in online learning [Stoltz and Lugosi, 2005, Cesa-Bianchi and Lugosi, 2003, Blum and Mansour, 2007,
1.0-
Foster and Vohra, 1998], the key idea is to find an acti- vation function that maps the original policy w, one based on the recent loss all experts. Despite the efforts, very limited successes have been achieved, either empirically or thcoretically. In this work, we observe in our experiments that the combination weights generated by the EGD algorithm are based on historical performance over long period of time and thus cannot adapt quickly to transient environment changes. Hence, it is better to ef- fectively incorporate both long-term historical information and more recent changes in the environment. A straightfor- ward idea to re-initialize the weight w per K steps. We show that such a simple algorithm can achieve a tighter bound:
for
0.6-
0.01
0.4-
Fist regine
n-
Second reginse
4.
n- 0.1
R1=1
0.2-
1
0.25
0.5
0.0
n- 1.0
20
60
100
Time steps
The evolution of the weight assigned to fi where the losses for forecasters vary across the first regime [0, 50) and the second regime (50, 100). Proposition 2. (Informal) Denote -= 11, .r] E [1,.... .T] as any period of time. We then have, the K K-step re-initialize algorithm has a fighter regret bound compared to EGD any small interval where 1 < T (See appendix B.2 for proof.)
Figure 3:
Proposition 2 stresses that, by considering short-term information, we can attain lower regret in short time intervals. Such simple strategy still struggles with the hyper-pane pprrmpprrmetter choice of K. Besides, discarding long-term information makes the algorithm inferior to EGD for a long period of the online learning process. In this work, we address this challenge of online learning by exploiting offline reinforcement learning [Levine et al., 2020]. At first, we use EGD to maintain long-term weight w. Besides, we introduce different set of weights b that can better capture the recent performance of individual models. By combining w and b. our approach can effectively incorporate both long-term historical information and more recent changes in the environment.
Specifically, we adopt the RvS [Emmons et al., 2022] framework, which formulates reinforcement learning through supervised learning, shown in Figure 2(b). At time step f, our target is to learn
5

---

a short-term weight conditioned on the long-term weight w and experts' performances during short period of history [2. 8). For simplicity and computation efficiency, we just let / = t 1. The agent then chooses actions using a policy Te,, (b,l [ell([w,,, ( (= parameterized by 0,-- During training, we concatenate the product between each prediction and expert weight (W,,s ý:) with the outcome y as the conditional input. We follow RvS [Emmons et al., 2022] to implement the policy network as a two-layer MLPs fr xx - R Then the Short-term weight and final ensembling weight will be:
b. = fri 1 @ & ar. d @ y) and Whss (we, + by,))/ Σ (we. + bexs)
(4)
However, unlike in RvS, we cannot train the decision network through simple classification tasks since the ground truth target action is inaccessible. Instead, we propose to train the network by minimizing the forecasting error incurred by the new weight, that is, ming,; L(w) : Σ Wt,af((x) y 1|² During inference, as concept drift changes gradually, we use w,-1 + b to generate the prediction and train the networks after the ground truth outcome is observed. We theoretically and empirically verify the effectiveness of the proposed OCP block in appendix B.4.
3.2 OneNet: utilizing the advantages of both structures
The model structure is shown in Figure 2(a) and we introduce the components as follows:
Two-stream forecasters. The input multivariate time series data is fed into two separate forecasters, a cross-time forecaster fi and cross-variable forecaster f2. Each forecaster contains an encoder and a prediction head. Assuming that the hidden dimension of the models are all dm, the encoder of fi projects the input series to representation ž1 E xdm, and the prediction head generates the final forecasting results: y1 IR M For the cross-variable forecaster fz. the encoder projects x to Z2 E [. Then, the representation of the last time step Z E I2 I is selected and fed into the prediction head to generate the final forecasting results y2 E RM*H Compared to f1. whose projection head has a parameter of d,,, x H. the projection head of f2 has a parameter of dm, x M HI, which is heavier, especially when M is large. Additionally, while fi ignores variable dependency, fa simply selects the representation of the last time step time series, ignoring temporal dependency. These two modules yield different but complementary inductive biases for forecasting tasks. OCP block is then used for learning the best combination weights. Specifically, we use EGD update a learning to learn an additional short-term
weight wy for each forecaster and use offline-reinforcenn weight b; the final combination weight for one forecaster will be wt <- wi + b,. Considering the difference between variables, we further construct different weights for each variable, namely, we will have w E I 1g x2 combination weights.
Decoupled training strategy. A straightforward training strategy for OneNet is to minimize [(w] Y1 + W2 Y2, y) for both the OCP block and the two forecasters, where w; here denotes the weight with the additional bias term. However, the coupled training strategy has fatal flaw: considering an extreme case where fi always performs much better than f2, then W1 will be close to and W/ - In this case, Vy, [(w) yi + W2 * y) R O, that is, fo is probably not trained for a long time. Under the context of concept drift, if retraining is not applied, as time goes on, the performance of f2 will become much inferior. In this paper, therefore, we decouple the training process of the OCP block and the two forecasters. Specifically, the two forecasters is trained by L(y,,y) + L(yz, y) and the OCP block is trained by L(w] y1 W2 y).
Remark Note that OneNet is complementary advanced architectures for time series forecast- ing and online adaption methods under concept drift. A stronger backbone or better adaptation strategies/structuuee can both enhance performance.
Experiments
In this section, we will show that (1) the proposed OneNet attains superior forecasting performances with only a simple retraining strategy (reduce more than 50% MSE compared to the previous SOTA model); (2) OneNet achieves faster and better convergence than other methods; (3) we conduct thorough ablation studies and analysis to reveal the importance of each of design choices of current advanced forecasting models. Finally, we introduce variant of OneNet, called OneNet-, which has significantly fewer parameters but still outperforms the previous SOTA model by a large margin. Due to space limitations, some experimental settings and results provided in the appendix.

---

Table 2: MSE of various adaptation methods. H: forecast horizon. OneNet-TCN is the mixture of TCN and Time-TCN, and OneNet is the mixture of FSNet and TSNe-FSNet.
ETTH2
ETTml
WTH
ECL
Method H
24
48
24
48
24
48
24
48
Avg
Informer
7,571
4.629
5.692
0,456
0.478
0.388
0.426
0.380
0.367
2.265
TCNNneTCN
0,502
0.830
1.183
0.214
0.258
0.283
0.206
0.308
0.302
3.309
11.339
11,534
2.522
TFCL.
0,557
0.846
1.208
0.087
0.211
0.236
0.177
0.301
0.323
2.732
12.094
12.110
2.574
ER
0.508
0.808
1.136
0.086
0.202
0.220
0.180
0.293
0.297
2.579
9.327
9.685
2.110
MIR
0.486
0.812
1.103
0.085
0.192
0.210
0.179
0.291
0.297
2.575
9.265
9.411
2.076
DER+
0,508
0.828
1.157
0.083
0.196
0.208
0.174
0.287
0.294
2.657
8.996
9.009
2.033
FSNet
0.466
0.687
0.846
0.085
0.115
0.127
0.162
0.188
0.223
3.143
6.051
7.034
1.594
Time-TCN
0,491
0.779
1.307
0.093
0.281
0.308
0.158
0.311
0.308
4.060
5,260
5.230
1.549
PatchTST
0.362
1.622
2.716
0.083
0.427
0.553
0.162
0.372
0.465
2.022
4.325
5.030
1.512
OnNNet-TTC
0.411
0.772
0.806
0.082
0.212
0.223
0.171
0.293
0.310
2.470
4.713
4.567
1.253
OneNet
0.380
0.532
0.609
0.082
0.098
0.108
0.156
0.175
0.200
2.351
2.074
2.201
0.747
4.1 Experimental setting
Baselines of adaptation methods We evaluate several baselines for our experiments, including methods for continual learning, time series forecasting, and online learning. Our first baseline is OnNNneTTCN [Zinkevich, 2003], which continuously trains the model without any specific strategy. The second baseline is Experience Replay (ER) [Chaudhry et al., 2019], where previous data is stored in buffer and interleaved with newer samples during learning. Additionally, we consider three advanced variants of ER: TFCL [Aljundi et al., 2019b], which uses task-boundary detection mechanism and a knowledge consolidation strategy; MIR [Aljundi et al., 2019a], which selects samples that cause the most forgetting; and DER++ [Buzzega et al., 2020], which incorporates knowledge distillation strategy. It is worth noting that ER and its variants are strong baselines in the online setting, we leverage mini bathee during training to reduce noise from single samples and achieve faster and better convergence. Finally, we compare our method to FSNet [Pham et al., 2023], which is the previous ofate-of-thh-art online adaptation method. Considering different model structures, we compare the performance under concept drift of various structures, including TCN [Bai et al., 2018], Informer [Zhou al., 2021], FEDformer [Zhou et al., 2022b], PatchTST [Nie et al., 2023], Dlinear [Zeng et al., 2023], Nlinear [Zeng et al., 2023], TS-Mixer [Chen et al., 2023].
Strong ensembling baselines. To verify the effectiveness of the proposed OCP block, we compare with several ensembling baselines. Given the online inputs x, predictions of each expert y1, y2, and the ground truth outcome y. the final outcome y of different baselines will be as follows: (1) Simple averaging: we simply average the predictions of both experts to get the final prediction. i.c., = (y y2). (2) Gating mechanismLiu et al. [2021]: we learn weights to the output of each forecaster, that is, h = WConcat (Vi. f22 + b; w1, W2 = so foftmar(h) and the final result is given by ý W1 * f/1 W2 1/2- (3) Mixture-of-experr /acoesssss/asssssooooooooosbosse accoss et al. [1991], Shazeer et al. [2017]: the mixture of experts approach, where we first learn the weights wh and wh by applying a softmax function a linear combination of the input, i.e., h Wx + b; W1, u2 softmar(h) and then we obtain the final prediction by combining the predictions of both experts as y - wi yi + W2 * 52. (4) Linear Regression (LR): we a simple linear regression model to obtain the optimal weights, i.e., [W), [0g] = (X -)- X" y, where X = Y2] and y is the ground truth outcome. (5) Exponentiated Gradient Descent (EGD): we use EGD update the weights Wj and Wa separately without the additional bias. (6) Reinforcement learning to learn the weight directly (RL-W): we use the bias term in the OCP block update the weights based on the predictions of both experts and the ground truth outcome, i.c., the weight is only dependent on y1, Y2, and y, but not on the historical performance of each expert. For all baselines with trainable parameters, the training procedure is just the same as the proposed OCP block.
4.2 Online forecasting results
Cumulative performance Table.2 and Table.3 present the cumulative performance of different baselines in terms of mean-squared errors (MSE) and mean-absolute errors (MAE). In particular, Time-TCN and PatchTST exhibit strong performance and outperform the previous sfate-of-the-an model, FSNet [Pham et al., 2023]. The proposed OneNet-TCN (online ensembling of TCN and Time-
7

---

Table 3: MAE of various adaptation methods. H: forecast horizon. OneNet-TCN is the mixture of TCN and Time-TCN and OneNet is the mixture of FSNet and Time-FSNet.
ETTH2
ETTmm
WTH
ECL
Method H
24
48
24
48
24
48
24
48
Avg
Informer
0.850
0.668
0.752
0.512
0.525
0,460
0.458
0.417
0.419
OCN
0.436
0.547
0.589
0.085
0.381
0.403
0.276
0.367
0.362
0.635
1.196
1.235
0.543
TFCL
0.472
0,548
0,592
0.198
0.341
0,363
0.240
0.363
0,382
0.524
1.256
1.303
0,549
ER
0.376
0,543
0.571
0.197
0.333
0.351
0.244
0.356
0,363
0,506
1,057
1.074
0.498
MIR
0:410
0.541
0.565
0.197
0.325
0.342
0.244
0.355
0.361
0.504
1.066
1.079
0,499
DER+
0.375
0.540
0.577
0.192
0.326
0,340
0.235
0.351
0.359
0.421
1,035
1.048
0.483
FSNet
0.368
0.467
0,515
0.191
0.249
0.263
0,216
0.276
0.301
0.472
0.997
1.061
0,448
Time-TCN
0.425
0.544
0.636
0.211
0.395
0,421
0,204
0.378
0.378
0.332
0.420
0.438
0.399
PatchTST
0.341
0.577
0.672
0.186
0.471
0.549
0.200
0.393
0.459
0.224
0.341
0.375
0.399
OneNet-TCN
0.374
0.511
0.543
0.191
0.319
0.371
0.221
0.345
0.356
0.411
0.513
0.534
0.391
OneNet
0.348
0.407
0.436
0.187
0.225
0.238
0.201
0.255
0.279
0.254
0.333
0.348
0.293
Table 4: Ablation studies of ensembling methods (MSE results).
ETTH2
ETTml
Method
Avg
24
48
24
48
24
Baseline
FSNet
0.466
0.687
0.846
0.085
0.115
0.127
0.162
0.188
0,223
3.143
6.051
7.034
1.594
Average
0.381
0.607
0.595
0,088
0.105
0.111
0.154
0.176
0.197
2.458
2.833
3.309
0.918
Gating
0.476
0.678
0.782
0,089
0.121
0.135
0.161
0.207
0.232
2.474
2.181
2.301
0.820
Ensembling
MOE
0.488
0.565
1,238
0,084
0.107
0.133
0.155
0.179
0.213
3,312
3.086
2.497
1.005
methods
LR
0.741
0.634
0.589
0.153
0.107
0.113
0.229
0.179
0.198
4.376
2.235
2.478
1.003
EGD
0.383
0.614
0.682
0,081
0.113
0.117
0.153
0.183
0.213
2.546
2.102
2.373
0.797
RI.W
0.374
0.634
0.735
0.091
0.099
0.109
0.157
0.174
0.202
2.457
2.115
2.192
0.778
OneNet-
0.381
0.732
0.932
0.154
0.162
2.351
2.322
3.833
0.961
Ours
OneNel
0.380
0.532
0.609
0.747
TCN) surpasses most of the competing baselines across various forecasting horizons. Interestingly, if the combined branches are stronger, for example, OneNet combined FSNet and Time-FSNet. achieving much better performance than OneNet- TCN.. Namely, OneNet can integrate any advanced online forecasting methods or representation learning structures to enhance the robustness of the model. The average MSE and MAE of OneNet are significantly better than using either branch (FSNet or TCMe-TCN) alone, which underscores the significance of incorporating online ensembling.
Comparison with strong ensembling baselines shown in Table 4. The two-branch framework greatly improves performance compared FSNet with just simple ensembling methods such as averaging. The MOE approach that learns the weight from the input x performs poorly and is even inferior simply averaging the prediction results. On the other hand, learning the weight from the prediction results in y1 and y: (Gating) performing much better than MOE. This indicates that the combination weights should be dependent on the model prediction. However, formulating the learning problem as linear regression and using the closed-form solution is not good idea due the scarce nature of the online data stream and the high noise in the learned weight. EGD provides significant benefits compared to the averaging method, which highlights the importance of the cumulative historical performance of each expert. Additionally, we observe that RL-W achieves performance comparable or even better than EGD on some datasets. Therefore, we propose the OCP block that uses EGD to update the long-term weight and offline RL to learn the short-term weight. This design leads to superior performance compared to all the other baselines.
Forecasting results are visualized in Figure 4. Compared to baselines that struggle adapt to new concepts and produce poor forecasting results, OneNet can successfully capture the patterns of time series. More visualization results and convergence analysis are presented in Appendix C.7.
4.3 Ablation studies and analysis
The effect of instances normalization and seasonal-trend decomposition is shown in Table. 5. The results show that removing the seasonal-trend decomposition component from PST has limited
8

---

1.55
Prodition Logil
Groundl Trab:
1.0
TUN
24-0
PSNa
ObaNet
Vile
Ground Tack
ICN
PSNa
1034
2503
2510
2520
2530
2540
2550
2500
25300
2520
2530
2540
2550
*Iastancees
(b)
(c) Channel
Figure 4: Visualizing the model's prediction and parameters during online learning. (a) Number of parameters for different models on the ECL dataset with different forecast horizons. We concentrate on short 50-time step horizon, starting from 2500. (b), and (c) depict the model's prediction results for the first and second channels of the ECL dataset.
Table 5: Ablation studies of the instances normalization (inv) and seasonal-trend decomposition (Decomp) of non-adapted PStchTST and online adapted PatchTST, where the metric is MSE.
Dataset
FTTH2
FTTml
WTH
FCI.
Online
Inv
Docomp
24
48
1
24
48
1
24
48
24
48
Avg
X
0.360
1.625
2,670
0,083
0.436
0.555
0.161
0,370
0.464
1.988
4.345
5,060
1,510
X
V
0.380
1.492
3.060
0.084
0.427
0.463
0.164
0.358
0.421
2.510
5.320
6.280
1.747
V
1.622
2.716
0.083
0.427
0.162
0.372
0.465
2.022
4.325
1,512
X
X
0.392
1.450
2.630
0.084
0.416
0.487
0.163
0.357
0.431
2.617
5.557
5.655
1.687
V
X
0.397
2,090
3.1.6
0.084
0.448
0.553
0.161
0.372
0.467
2,000
4.398
5,100
1.602
x
X
V
0.674
3.100
4.510
0.086
0.462
0.686
0.165
0.362
0.443
3,900
11.340
21.540
3,939
V
2,090
3,290
0.083
0.163
0.375
0.467
2.030
4.395
5.101
1.619
X
X
0.723
3.030
6,300
0.085
0.451
0.559
0.164
0.361
0.439
3.540
14.170
18.680
4.042
effect, regardless of whether the model is adapted online not. Instances normalization is commonly used to mitigate the distribution shift between training and testing data, which is crucial for model robustness when online adaptation is impossible. However, when online adaptation is performed, the influence of instance normalization is reduced. Interestingly, our experiments reveal that instance normalization impedes the model adaptation process in ETTH2, ETTml, and WTH datasets when the forecast horizon is long (24 or 48). Thus, simply normalizing time series with zero mean and unit standard deviation may not be the optimal approach under concept drift. Ablation studies of the variable independence and frequency domain augmentation are detailed in the appendix.
Delve deep into parameter-efficient online adaptation. Although OneNet significantly reduces the forecasting error, also increases the number of parameters and inference time due to its two- stream framework. We also design variant of OneNet that may have slightly lower performance than OneNet, but with fewer parameters, making more suitable for lightweight applications, denoted by OneNet-. Specifically, independent. In this case, denote 21, Z2 as the generated features for one variable from two branches, concatenate the two features and feed them into the projection head, which further avoids the offline reinforcement learning block for ensembling weight learning and reduces the parameters. For example, in the ECL dataset, the hidden dimension FSNet [Pham et al., 2023] is 320, and the sequences have 321 channels. When the forecast horizon 48, the projection head consists of just one linear layer with 320 x 321 x 48 = 4, 930, 560 parameters. On the contrary, the concatenated features of OneNet- are always less than 1024 dimension, resulting in a final projection head with less than 1024 48 49, 152 parameters. Figure 4(a) shows detailed comparison of different methods on the ECL dataset. For small forecast horizons, all methods have a comparable number of parameters. As the forecast horizon increases, the number of parameters of existing adaptation methods increases rapidly. On the contrary, the number of parameters of OneNet- remains insensitive the forecast horizon and is always less than all basclines. The performance of OneNet- is shown in Table 12, which is much better than FSNet but achieves fewer parameters.
ensemble PatchTST and Time-FSNet, which both variable-
See the appendix for the comparison of different forecasting models and more numerical results such as detailed ablation studies of different hyper-paramenn parameters and adaptation results under more settings.

---

Conclusion and Future Work
Through our investigation into the behavior of advanced forecasting models with concept drift, we discover that cross-time models exhibit greater robustness when the number of variables large, but are inferior to models that can model variable dependency when the number of variables is small. In addition, this problem becomes more challenging due to the occurrence of concept drift, as the data preferences for both model biases are dynamically changing throughout the entire online forecasting process, making it difficult for single model to overcome. To this end, we propose the OneNet model, which takes advantage of the strengths of both models through OCP. In addition, to Iearn an additional short-term weight through offline reinforcement learning to mitigate the slow switch phenomenon commonly observed in traditional policy learning algorithmss Our extensive experiments demonstrate that OneNet is able to effectively deal with various types of concept drifts and outperforms previous methods in terms of forecasting performance.
propose
We also discover that instance normalization enhances model robustness under concept drift, but can impede the model's ability to quickly adapt to new distributions in certain scenarios. This prompts further exploration of whether there exists a normalization technique that can mitigate distribution shifts while enabling rapid adaptation to changing concepts. In addition, although we design lightened version of OneNet to address the problem of introducing additional parameters and inference time, there is potential for more efficient adaptation methods, such utilizing prompts and efficient tuning methods from the NLP/CV community, to avoid retraining the full model.
Acknowledgments and Disclosure of Funding
This work was supported by the National Key R&D Program of China (2022ZD0117901) and National Natural Science Foundation of China (Grant No. 62373355 and 62236010), and also supported by Alibaba Group through Alibaba Research Intern Program.
References
Rishabh Agarwal, Dale Schuurmans, and Mohammad Norouzi. An optimistic perspective on offline reinforcement learning. International Conference on Machine Learning, pages 104-114. PMLR, 2020.
Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) forget. In Proceedings of the European conference on computer vision (ECCV), pages 139-154, 2018.
Rahaf Aljundi, Eugene Belilovsky, Tinne Tuytelaars, Laurent Charlin, Massimo Caccia, Min Lin, and Lucas Page-Caccia. Online continual learning with maximal interfered retrieval. In Advances in Neural Information Processing Systems 32, pages 1 11849-118 11860. 2019a
Rahaf Aljundi, Klaas Kelchtermans, and Tinne Tuytelaars. Trsk-free continual learning. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11254-11263. 2019b.
Oren Anava, Elad Hazan, Shie Mannor, and Ohad Shamir, Online learning for time series prediction. In Conference learning theory, pages 172-184. PMLR, 2013.
Shaojie Bai, J Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01221, 2018
Avrim Blum and Yishay Mansour, From external to internal regret. Journal of Machine Learning Research, 8(6), 2007.
George EP Box and David A Pierce. Distribution of residual autocorrelations in autoregressive- integrated moving average time series models. Journal of the American statistical Association, 65 1399):1509-1526, 1970.
Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: a strong, simple baseline. Advances in neural information processing systems, 1el11992-159300 2020.
10

---

Nicolo and Gábor Lugosi. Potential-base algorithms in on-line prediction and game theory. Machine Learning. $1:239-261, 2003.
Nicolo Cesa-Bianchi and Gábor Lugosi. Prediction, learning, and games. Cambridge university press, 2006.
Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet K Dokania, Philip HS Torr, and Mare' Aurelio Ranzato. On tiny episodic memories in continual learning. arXiv preprint 192.::/902.10486, 2019.
Si-An Chen, Chun-Liang Li, Nate Yoder, Sercan O Arik, and Tomas Pfister. Tsmixer: An all-mip architecture for time series forecasting. arXiv preprint arXiv:2303.06053 2023.
Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. Empirical evaluation of gated recurrent neural networks sequence modeling. arXiv preprint arXiv:1442.355 2014.
Abhimanyu Dubey, Vignesh Ramanathan, Alex Pentland, and Dhruv Mahajan. Adaptive methods for real-world domain generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021.
Scott Emmons, Benjamin Eysenbach, Ilya Kostrikov, and Sergey Levine. Rvs: What is essential for offline rl via supervised learning? ICLR, 2022.
Dean P Foster and Rakesh Vohra. Regret in the on-line decision problem. Games and Economic Behavior, (92-7)):73 1999.
Dean P Foster and Rakesh V Vohra. Asymptotic calibration. Biometrika, 85(2):379-390, 1998.
Scott Fujimoto, David Meger, and Doina Precup. Off-policy deep reinforcement learning without PMLR, 2019.
exploration. In International conference on machine learning, pages
João Gama, Indrè Žliobaité, Albert Bifet, Mykola Pechenizkiy, and Abdelhamid Bouchachia. A survey concept drift adaptation. ACM computing surveys (CSUR), 46(4):1-37, 2014.
Udaya Ghai, Elad Hazan, and Yoram Singer. Exponentiated gradient meets gradient descent. In Algorithmic learning theory, pages 386-407. PMLR, 2020.
Alex Graves and Alex Graves. Long short-term memory. Supervised sequence labelling with recurrent neural networks, pages 37-45, 2012.
Roland Hafner and Martin Riedmiller. Reinforcement learning in feedback control: Challenges and benchmarks from technical process control. Machine learning. 84:137-169 2011.
Simon Hill and Robert C Williamson. Convergence of exponentiated gradient algorithms. IEEE Transactions on Signal Processing, 49(6): 1208-1215, 2001.
Yusuke Iwasawa and Yutaka Matsuo. Test-time classifier adjustment module for model-agnostic domain generalization. Advances in Neural Information Processing Systems, 2021.
Robert A Jacobs, Michael Jordan, Steven J Nowlan, and Geoffrey E Hinton. Adaptive mixtures of local experts. Neural computation, 3(1):79-8) 87, 1991.
Rahul Kidambi, Aravind Rajeswaran, Praneeth Netrapalli, and Thorsten Joachims. Morel: Model- based offline reinforcement learning. Advances in neural information processing systems, 33: 21810-21823, 2020.
Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. Offline reinforcement learning: Tutorial, review, and perspectives on open problems. arXiv preprint arXiv:2005.0166 2020.
Wendi Li, Xiao Yang, Weiqing Liu, Yingce Xia, and Jiang Bian. Ddg-da: Data distribution generation for predictable concept drift adaptation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 4092 4100, 2022.
Jian I Liagg, Ran He, and Tieniu Tan. A comprehensive survey on test-time adaptation under distribu- tion shifts. arXiv preprint arXi::2303.1536 2023.

---

Bryan Lim and Stefan Zohren. Time-series forecasting with deep learning: survey. Philosophical Transactions of the Royal Society A, (79992299)):222000099 2021.
Chenghao Liu, Steven CH Hoi, Peilin Zhao, and Jianling Sun. Online arima algorithms for time series prediction. In Proceedings of the AAAI conference on artificial intelligence, volume 30, 2016.
Minghao Liu, Shengqi Ren, Siyuan Ma, Jiahui Jiao, Yizhou Chen, Zhiguang Wang, and Wei Song. Gated transformer networks for multivariate time series classification. arXiv preprint arXiv:2103.14438, 2021.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017.
Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. ICLR, 2023.
Quang Pham, Chenghao Liu, Doyen Sahoo, and Steven CH Hoi. Learning fast and slow for online time series forecasting. ICLR, 2023.
Rafael Figueiredo Prudencio, Marcos ROA Maximo, and Esther Luna Colombini. A survey on offline reinforcement learning: Taxonomy, review, and open problems. IEEE Transactions on Neural Networks and Learning Systems, 2023.
Tiexin Qin, Shiqi Wang, and Haoliang Li. Generalizing to evolving domains with latent structure- aware sequential autoencoder. In International Conference on Machine Learning, pages I 18062- 18082. PMLR, 2022.
Yao Qin, Dongjin Song, Haifeng Chen, Wei Cheng, Guofei Jiang, and Garrison Cottrell. A dual-stage attention-based recurrent neural network for time series prediction. arXiv preprint arXiv:1704.02971 2017.
Rajat Sen, Hsiang-Fu Fu Yu, and Inderjit Dhillon. Think globally, act locally: A deep neural network approach to high-dimensional time series forecasting. Advances in neural information processing systems, 32, 2019.
Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. ICLR, 2017.
David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. Mastering the game of go without human knowledge. nature, 350(7676)3354 359, 2017.
Gilles Stoltz and Gábor Lugosi. Internal regret in on-line portfolio selection. Machine Learning, 59: 125-159, 2005.
Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt. Test-time training with sulf-super for generalization under distribution shifts. In International conference on machine learning, pages 9229-92444 9248. PMLR, 2020.
Alexey Tsymbal. The problem of concept drift: definitions and related work. Computer Science Department, Trinity College Dublin, 106(2):58, 2004.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell, Tent: Fully test-time adaptation by entropy minimization. arXiv preprint arXiv:2006.10722 2020.
Qingsong Wen, Tian Zhou, Chaoli Zhang, Weiqi Chen, Ziqing Ma, Junchi Yan, and Liang Sun. Transformers in time series: A survey. arXiv preprint a7x1::200..07125, 2022.
12

---

Xiaoyu You, Mi Zhang, Daizong Ding, Fuli Feng, and Yuanmin Huang. Learning to lear the future: Modeling concept drifts in time series prediction. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pages 2434 2344,4443, 2021.
Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? 2023.
Hanlin Zhang, Yi-Fan Zhang, Weiyang Liu, Adrian Weller, Bernhard Schölkopf, and Eric P Xing. Towards principled disentanglement for domain generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8024-8034, 2022a.
Yang Zhang, Fuli Feng, Chenxu Wang. Xiangnan He. Meng Wang, Yan Li, and Yongdong Zhang How to retrain recommender system? sequential meta-learning method. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1479- 1488, 2020.
Yi-Fan Zhang, Jindong Wang, Jian Liang. Zhang Zhang, Baosheng Yu, Liang Wang. Dacheng Tao, and Xing Xie. Domain-specific risk minimization for out-of-distribution generalization, 2022b.
Yi-Fan Zhang, Zhang Zhang, Da Li, Zhen Jia, Liang Wang. and Tieniu Tan. Learning domain invariant representations for generalizable person re-identification. IEEE Transactions on Image Processing, 32:509-5233 2022c.
Yi-Fan Zhang, Xue Wang, Kexin Jin, Kun Yuan, Zhang Zhang, Liang Wang, Rong Jin, and Tieniu Tan. Adanpe: Exploring non-parametric classifier for test-time adaptation. ICMI., 2023a.
Yifan Zhang, Bryan Hooi, Lanqing Hong, and Jiashi Feng. Test-agnostic long-tailed recognition by test-time aggregating diverse experts with self-supervision. arXiv preprint Stti:22007.09245 2021.
Yilan Zhang, Xue Wang, Jian Liang, Zhang Zhang, Liang Wang, Rong Jin, and Tieniu Tan. Free lunch for domain adversarial training: Environment label smoothing. arXiv preprint arXiv:2302.00194 2023b.
Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In International Conference on Learning Representations. 2023.
Haoyi Zhou, Shanghang Zhang. Jieqi Peng. Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 11106-11111 2021.
Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, and Chen Change Loy. Domain generalization: survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022a.
Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang. Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International Conference PMLR, 2022b.
on Machine Learning, pages
Martin Zinkevich. Online convex programming and generalized infinitesimal gradient ascent. In Pro Proceedings of the 20th international conference on machine learning (icml-033, pages 928-936, 2003.
13

---

OneNet: Enhancing Time Series Forecasting Models under Concept Drift by Online Ensembling
Appendix
Contents
1
Introduction
2 Preliminary and Related Work
3 OneNet: Ensemble Learning for Online Time Series Forecasting
3.1
Learning the best expert by Online Convex Programming (OCP) OneNet: utilizing the advantages of both structures
3.2
4
Experiments
4.1
Experimental setting
4.2
Online forecasting results
7
4.3
Ablation studies and analysis
5
Conclusion and Future Work
10
A Extended Related Work
15
B Proofs of Theoretical Statements
16
B.1 Online Convex Programming Regret Bound
16
B.2 Theoretical guarantee for the K K-step re-initialize algorithm.
16
B.3 Necessary definitions and assumptions for evaluating model adaptation speed to environment changes.
17
B.4
Existing theoretical intuition and empirical comparison to the proposed OCP block.
18
C Additional Experimental Results
20
C.1 Datasets
20
C.2 Implementation Details
21
C.3 Baseline details
21
C.4 Hyper-parameters
21
C.5 Additional Numerical Results
22
C.6 Ensembling more than two networks.
23
C.7 More visualization results and Convergence Analysis of Different Structures
23
C.8 Online forecasting results with delayed feedback
23
C.9 The effect of variable independence and frequency domain augmentation
25
C.10 Comparison of existing forecasting structures.
25
14

---

A Extended Related Work
Concept Drift Concepts in the real world are often dynamic and can change over time, which is especially true for scenarios like weather prediction and customer preferences. Because of unknown changes in the underlying data distribution, models learned from historical data may become inconsistent with new data, thus requiring regular updates to maintain accuracy. This phenomenon, known as concept drift [Tsymbal, 2004], adds complexity to the process of learning a model from data. Concept drift poses several challenging subproblems, ranging from fast learning under concept drift [Pham et al., 2023, Zhang et al., 2020, Gama et al., 2014], which involves adjusting the offline model with new observations recognize recent patterns, to forecasting future data distributions [Li et al., 2022, Qin et al., 2022], which predicts the data distribution of the next time-step sequentially, enabling the model of the downstream learning task to be trained on the data sample from the predicted distribution. In this paper, we focus on the first problem, i.e. online learning for time series forecasting. Unlike most existing studies for online time series forecasting [Li et al., 2022, Qin et al., 2022, Pham et al., 2023] that only focus on how to online update their models, this work goes beyond parameter updating and introduces multiple models and learnable ensembling weight, yielding rich and flexible hypothesis space.
Test-Time adaptive methods is similar to online time series forecasting but mainly focuses on domain generalization [Zhang et al., 2022c,a, Zhou et al., 2022a], domain adaptation [Zhang et al., 2023b] and other tasks [Liang et al., 2023]. These recently proposed to utilize target samples. Test- Time Training methods design proxy tasks during tests such as self-consistence [Zhang et al., 202 I], rotation prediction [Sun al., 2020] and need extra models; Test-Timm Adaptation methods adjust model parameters based on unsupervised objectives such as entropy minimization [Wang et al., 2020] or update a prototype for each class [Iwasawa and Matsuo, 2021]. Domain adaptive method [Dubey et al., 2021] needs additional models to adapt to target domains. There are also some methods that do not need test-time tunning, for example, [Zhang et al., 2022b] introduces specific classifiers for different domains and adapts the voting weight for test samples dynamically and [Zhang et al., 2023a] apply non-parametric test-time adaptation.
Time Series Modeling Time series models have been developed for decades and are fundamental in various fields. While autoregressive models like ARIMA [Box and Pierce, 1970] were the first data-driven approaches, they struggle with nonlinearity and non-stationarity. Recurrent neural networks (RNNs) were designed to handle sequential data, with LSTM [Graves and Graves, 2012] and GRU [Chung et al., 2014] using gated structures to address gradient problems. Attention-based RNNs [Qin et al., 2017] use temporal attention to capture long-range dependencies, but are not parallelizable and struggle with long dependencies. Temporal convolutional networks [Sen et al., 2019] are efficient, but have limited reception fields and struggle with long-term dependencies. Recently, transformer-based [Vaswani et al., 2017, Wen et al., 2022] models have been renovated and applied in time series forecasting. Although a large body of work aims to make Transformer models more efficient and powerful [Zhou et al., 2022b, 2021, Nie et al., 2023], we are the first to evaluate the robustness of advanced forecasting models under concept drift, making them more adaptable to new distributions.
Reinforcement learning and offline reinforcement learning. Reinforcement learning is math- ematical framework for learning- based control, which allows us to automatically acquire policies that represent near-optimal behavioral skills optimize user-defined reward functions [Hafner and Riedmiller, 2011, Silver et al., 2017]. The reward function specifies the objective of the agent, and the reinforcement learning algorithm determines the actions necessary achieve it. However, the online learning paradigm of reinforcement learning is a major obstacle to its widespread adoption. The iterative process of collecting experience by interacting with the environment is expensive or dangerous in many settings, making offline reinforcement learning more feasible alternative. Offline RL [Prudencio al., 2023, Levine et al., 2020] learns exclusively from static datasets of previously collected interactions, enabling the extraction of policies from large and diverse training datasets. Effective offline RL algorithms have much wider range of applications than online RL. Although there are many types of offline RL algorithms, such as those using value functions [Fujimoto et al., 2019], dynamics estimation [Kidambi et al., 2020], or uncertainty quantification [Agarwal et al., 2020], RvS [Emmons et al., 2022] has shown that simple conditioning with standard feedforward networks can achieve sfate-of-the-ae results. In this work, we draw inspiration from RvS and learn an additional bias term for the OCP block for simplicity.
15

---

B Proofs of Theoretical Statements
B.1 Online Convex Programming Regret Bound
Proposition 1. For T > 2log(d), denote the regret for time step = 1,..., T as R(T), set
logog()) and the EGD update policy has regret.
r R(T) Σ ((w))
T
inf
((u)
ΣΣ t=1
Wi, fx(x) y 11 inf C(u) v 2T log(d)
(5)
Proof. The proof of Exponentiated Gradient Descent well-studied [Ghai et al., 2020] and here we provide simple Regret bound for both OCP.
Denote l,2 =Il fi(x) yy _1 recall that the normalizer for OCP-U Ze+1 then we have
E -im1 We exp(-1nle,).
Ze+1 Z+
Ed log Li=l
We, exp(-nle,,)
log
log
Pt. exp(-x
(6)
Ze
where Pt.: Wr,sZz S 1. For clarity, let W4,s be the unnormalized weight and Pt,i be the normalized weight. Now, we assume 174,, E [0, 1]. Although it is not guaranteed that &, will be small under concept shift, it is generally safe assume that the concept will shift gradually and will not lead to a drastic change the loss. As a result, the loss will not become arbitrarily large, and divide some large constant such that the loss is bounded in small range. Based the
assumption, we can use the second Taylor expansion of e S - + x² /2 and the inequation
log(1 x) x for [0, 1]. Then we have
172
log Σ Pt. exp(- (xp(-))es)) s log 1)
Σ Peile +
Σ
Prall
2
1=1
Pulles +
71
Σ
sl, S -1)
Σ
Piales +
(8)
2
2
1=1
1=1
Note that Wi, is the unnormalized weight, and then wh, and Z₁ d. Then we can get the lower bound and upper bound of log ZT+
Z
log ZT+1
Σ
log
+ loloo (2) 5 -11
Pe,
log(d)
(9)
Zt
1=1
1=1 iml
d
log ZT+1 = log
Σ
Whii expeeppppp (-1pe,,) 2 -1
4,
(10)
1=1
Finally, recall that we set 17
22log (d) then we have
d
d
T
Tn² <
n
Σ
Prale Σ let
in
Σ
Pr. hes
inf
C(u)
+ log(d),
(11)
which completes our proof.
B.2 Theoretical guarantee for the K-step re-initialize algorithm.
The proposed K-step re-initialize algorithm is detailed as follows: at the beginning of the algorithm, choose W1 [1i, 1/d as the center point of the simplex and denote le,t the loss
for fa at time step t, the updating rule for each w, will be W1-1, = Wt, expl ((oCo(
2.
We. expl (xx)
wa exp(-néx. where Ze Ed Wi, exp(-1)4,) is the normalizer.
16

---

Different from the native EGD algorithm, we re-initialize the weight wx+1 [WK+1.5 = 1/d] per K time steps. We call each step one round. This simple strategy interrupts the influence of the historical information of length K steps on the ensembling weights, which helps the model to quickly adapt to the upcoming environment.
Proposition 2. For T > 2 log(d), denote 1 = -1,++,,,--- where ( 1 and r - T. Denote the length of I as a sublinear sequence of namely, |4| T where 0 S n
as any period of time of length - 1+1
then have, the K- K-step re-initialize algorithm has an regret bound R(T)- I ((T2n/3) at any T= [1,1 11,+1,-.. .r). Namely, for any small internal n < 3
1. We choose K =T
we have R(U 1, rl) < O(T2/2)
Proof. We discuss regret in three cases:
At first, according to Proposition if all K steps fall into + 1, .r], then we have R(K) < 2/K In d(d 1). There exist L/K rounds that are all contained in [L,..., r] and the regret of these rounds will be OTL/KK K)
For the first round, do not know when the weights are reinitialized (1 may or may be a multiple of K) and the performance of the algorithm in historical time. let's think about the worst case, where regret will be less than O(K).
For the last round, we know when the last round begins and the weights are reinitialized. However, some of the future time steps not in 11, + 1,...,......... ,r] and we only treat the case as the first round, which has a regret O(K).
Considering all cases, we have an internal regret that is bounded by
R(I) S OLL/K x VK + K) = O(LLKVK+K)
(12)
When we choose an small internal length L = 7 and K = m where m S n. To minimize the 2n and the bound will be O(T2n/3), Namely, for any small upper bound, we should choose m = 3 internal n < 3. we have R([, + 1,... .r]) < OTT which is tighter than the regret bound of the EGD algorithm under the whole online sequence. Specifically, when choose L 2/3 and K = 7/9 we have R([1, 1, -r) < O < O(T1/2). When we choose L T1/1 and K T/6 then R(I,+ +1,1,,,+,,,... 1, .r) < /6 < 0(T
In other words, the algorithm focuses on short-term information and leads to a better regret bound in any small time interval. However, with increasing length of /, the bound of the simple algorithm will become worse. Consider an extreme case where n 1, the bound will be R([, 1, 1. .r)))r)) r]) OTT2/3), which inferior to the native EGD algorithm.
B.3 Necessary definitions and assumptions for evaluating model adaptation speed environment changes.
To complete the proofs, we begin by introducing some necessary definitions and assumptions. Given the online data stream x, and its forecasting target y, at time t. Given d forecasting experts with different parameters f, = {(,r) J. denote & as a nonnegative loss function and b,,, = (x.. the loss incurred by Jt, time define the following notions.
Definition 1. (Weighted average forecaster). A weighted average forecaster makes predictions by
D Wt w.ft.
y.
(13)
Dd W;-1,
where Wt, is the weight for expert fi at time t and fú, is the prediction of at time t.
17

---

Definition 2. (Cumulative regret and instantaneous regret). For expert fü. the cumulative regret (or simply regret) on the T steps is defined by
T RT Σ
T
(1(v..y)) 4.,) Lr Lrr
(14)
where Ta, is the instantaneous vegret of the expert at time t, which is the regret that the forecaster feels of not having listened to the advice of the expert fi right after the tth outcome that has been revealed. = V1) is the cumulative loss of the forecaster and LT, is the cumulative loss of the expert fr.
Definition 3. (Potential function). We can interpret the weighted average forecaster in interesting way which allows us to analyze the theoretical properties easier. To do this, we denote I (Fe.ls...,, .t..d) E Rd as the instantaneous regret vector, and RT = -Et= It is the corresponding regret vector. Now, we can introduce the potential function t : R - IR of the form
(uu) = = Σ o(uz)
(15)
where R + R is any nonnegative, increasing, and twice differentiable functioon and 10 R - R is nonnegative, strictly increasing, concave, and twice differentiable auxiliary function. With the notion of potential function, functioo,,, the prediction ÿt will be
Σ (D(Ra-))ife ÿ.
(16)
E:S VD(RR-1),
where (4:-i: = 04 (R. It is easy to prove that the exponentially weighted average forecaster used in (2. is based on the potential o, = (Ei=)e e
Theorem 1. (Blackwell condition, Lemma 2.1. in
and Lugosi, 2006).) If the loss
function & is convex in its first argument and we X1 X2 denote the inner product of two vectors, then
supr; (
(17)
The following theorem is applicable to any forecaster that satisfies the Blackwell condition, not limited to weighted average forecasters. Nevertheless, this theorem will lead several interesting bounds for various variations of the weighted average forecaster.
Theorem 2. (Theorem2.1 in |Cesa-Bianchi Bianchi and Lugosi, 2006].) Assume that forecaster satisfies the Blackwell condition for a potential 4 then for all 1,...
Φ(RT) < 2(0)
C(r.),
(18)
where
C(r.) sup nERd
Σ (uu))
Σ o" (u))r,
(19)
B.4 Existing theoretical intuition and empirical comparison the proposed OCP block.
With the help of the two theorems in Section B.3, we now recall that the Internal Regret R (t, w) [Blum and Mansour, 2007] that measures forecaster's expected regret of having taken action w at step t:
T
T
Ran (T, w)
max L.j=I....d
max
Σ t=1
wis (le, - le.))
(20)
While Proposition ensures that small external regret can achieved, ensuring a small internal regret is more challenging task. This is because any algorithm with small internal regret also
18

---

has small external regret but the opposite is not true, as demonstrated in [Stoltz and Lugosi, 2005]. The key question now whether possible to define policy w that attains small (i.e., sublinear in 7) internal regret. For simplicity, we use R as internal regret in this subsection. To develop forecasting strategy that can guarantee small internal regret. We define the exponential potential function + IR + R with 0 by
Φ(u) Σel e
(21)
where M d(d 1). Here, we denote r, (..(..I)....1.2).... (...a)):...PP..ddd-))) Iddd-1) as the
instantaneous regret vector and Ry Ei= H₁ is the corresponding regret vector. Then, any forecaster satisfying Blackwell's condition will have a bounded internal regret (Corollary [Cesa- Bianchi and Lugosi, 2003)) by choosing proper parameter y):
max Re, S $2 12
(22)
With the help of the two theorems in Section B.3, we now recall that the Internal Regret Rin (t, w) [Blum and Mansour, 2007] that measures forecaster's expected regret of having taken an action w at step L:
Rin (t, ())
Σ Te.ii))
max
Σ wis (le, lej))
(23)
<J=
+J=
For simplicity, we use R internal regret in this subsection. conduct forecasting strategy that can guarantee small internal regret. We define the exponential potential function 4 RR - R with ? > by
M Σ
Φuu) In
(24)
where M = d(d - 1). Here, we denote = (A.(1.)) -F.dddd RRd(d-1 as the
instantaneous regret vector and R Σ rt the corresponding regret vector. Then, any forecaster satisfying Blackwell's s condition will have a bounded internal regret (Corollary 8 in [Cesa- Bianchi and Lugosi, 2003]) by choosing a proper parameter yp:
max Re, S I2/tldd(dd1) b
(25)
Now our target is to find new policy that makes the forecaster satisfy the Blackwell condition.
V V
rt
(1a) ( (R, 1)r (le,s les5)
(ij) T (R, (RR-))Weeee
(i.) 1 (R (RR-1)We,
(26)
(LJ) 2 R1 (RR--)Wu,
(1x)
1=1 i=1
d
d
(1.)) (R:
V ( (Re
To ensure that this value negative or zero, is sufficient to demand that.
d
(i.z) & (R, (R,-1)
(1. (R, = (27)
19

---

2000
BGD
BGD
OCP
150
OCP
Internal Rogret
1500
Internal Regret
100
1000
50
0-
0
2000
4000
6000
8000
10000
2000
4000
6000
8000
10000
Time step
Time stop
(a) ETTh2.
(b) ETTml.
500
8000
EGD
EGD
400
OCP
OCP
Internal Regret
Internal Regret
6000
300
4000
200
2000
100
01
0
5000
10000
20000 Time step
15000
25000
5000
10000
15000
20000
Time step
(c) WTH.
(d) ECL.
Figure 5: Empirical verification of the proposed OCP block can significantly reduce the internal regret compared vanilla EGD, where the forecasting window H 48.
That is, we need to find a new policy vector we that satisfies wTT = 0, where A
((R,)( if k Ejæi Ve, Otherwise.
d matrix. However, determining the existence
of new policy vector and efficiently calculating its values can be challenging. Even if we assume the new vector exists, the time complexity of calculating the new policy by the Gaussian elimination method[Foster and Vohra, 1999] is O(d"), which is expensive, particularly for datasets with large number of variables such as the ECI. dataset with 321 variables and 321 +2 policies. To address this issue, we propose the OCP block which utilizes an additional offline reinforcement learning block frt with parameter 0,2 to learn bias vector b, for the original policy w.- The new policy vector is then defined as W. truth values, that is, minimize ming,, L(W) :=1 Σd= w. fi()) - y 1|2; s.l. W A train 0,1. We measure the internal regret max;,j=1....d W4.i le.j) at each time step empirically. As shown in Figure 5, the proposed method significantly reduces internal regret without the need for constructing and computing a large matrix.
b. The learned bias pushes the predicted outcomes closer to the ground
C
Additional Experimental Results
C.1
Datasets
We investigate diverse set of datasets for time series forecasting, ETT [Zhou et al., 2021] logs the target variable of the "oil temperature" and six features the power load over two-year period. We also analyze the hourly recorded observations of ETTh2 and the 15-minute intervals of ETTml benchmarks. Additionally, we study ECL. (Electricity Consuming Load), which gathers electricity
20

---

consumption data from 321 clients between 2012 and 2014. The Weather (WTH)² dataset contains hourly records of 11 climate features from almost 1,600 locations across the United States.
C.2 Implementation Details
For all benchmarks, we set the look-back window length at 60 and vary the forecast horizon from H 1, 1424, 48. We split the data into phases: warm-up and online training, with a ratio of 25:75. We follow the optimization details outlined in [Zhou et al., 2021] and utilize the A optimizer [Loshchilov and Hutter, 2017] minimize the mean squared eπror (MSE) loss. To ensure fair comparison, we set the epoch and batch sizes to one, which is consistent with the online learning setting. We make sure that all baseline models based on the TCN backbone use the same total memory budget FSNet, which includes three times the network sizes: one working model and two exponential moving averages (EMAs) of its gradient. For ER, MIR, and DER++, we allocate an episodic memory to store previous samples to meet this budget. For transformer backbones, we find that large number of parameters do not benefit the generalization results and always select the hyperparameters such that the number of parameters for transformer baselines is fewer than that for FSNet. In the warm-up phase, we calculate the mean and standard deviation to normalize the online training samples and perform hyperparameter crossvvlidation. For different structures, optimal hyperparameters that are reported in the corresponding paper.
the
License. All the assets (i.e., datasets and the codes for baselines) we use include an MIT license containing copyright notice and this permission notice shall be included in all copies or substantial portions of the software.
Environment. We conduct all the experiments on a machine with Intel Xeon (R) Platinum 8163 CPU @ 2.50GHZ, 32G RAM, and four Tesla-V100 100 (32G) instances. All experiments are repeated 3 times with different seeds.
Metrics Because learning occurs over a sequence of rounds. At each round, the model receives a look-back window and predicts the forecast window. All models commonly evaluated by their accumulated mean-squared errors (MSE) and mean-absolute errors (MAE), namely the model is evaluated based on its accumulated errors the entire learning process.
C.3 Baseline details
present a brief overview of the baselines employed our experiments.
First, OnlineTCN adopts a conventional TCN backbone [Zinkevich, 2003] consisting of ten hidden layers, each layer containing two stacks of residual convolution filters.
Secondly, ER [Chaudhry et al., 2019] expands on the OnlineTCN baseline by adding an episodic memory that stores previous samples and interleaves them during the learning process with newer ones.
Third, MIR [Aljundi et al., 2019a] replaces the random sampling technique of ER with its MIR sampling approach, which selects the samples the memory that cause the highest forgetting and applies ER to them.
Fourthly, DER++ [Buzzega et al., 2020] enhances the standard ER method by incorporating knowledge distillation loss on the previous logits.
Finally, TFCL [Aljundi et al., 2019b] is task-free, online continual learning method that starts with an ER process and includes a task-free MAS-styled [Aljundi et al., 2018] regularization.
All the ER-based techniques utilize a reservoir sampling buffer, which is identical to that used [Pham et al., 2023].
C.4 Hyper-parameters
For the in [Pham et al., 2023]. Besides, we cross-validate the hyper--aaameter on the ETTh2 dataset and use them for the remaining ones. In particular, we use the following configuration:
of FSNet and the baselines mentioned in Section C.3, we follow the setting
21

---

Table 6: Standard deviations of the metrics in Table. 2 and Table. 3.
MSE
HTTH2
ETTml
WTH
FCT.
Method
24
48
24
48
24
48
24
48
Informer
1,370
2.254
2,088
0.088
0,035
0.020
0.005
0.003
0.009
OnlineTCN
0.011
0.017
0.148
0.003
0.002
0.002
0.001
0.001
0.001
0.019
0.077
0.122
TFCL
0.030
0,005
0.279
0.004
0,006
0.010
0,002
0.001
0.004
0.047
0.338
0.253
0.018
0.007
0.141
0.005
0.003
0.004
0.001
0.001
0.009
0.034
0.236
0.320
MIR
0.019
0.017
0.130
0.005
0.005
0.006
0.002
0.001
0.009
0.037
0.261
0.143
DER++
0.022
0.024
0.143
0.003
0.002
0.003
0.001
0.001
0.011
0.027
0.072
0.146
FSNet
0.018
0,014
0.128
0.003
0.002
0.003
0.001
0.001
0,001
0.021
0.096
0.105
Time-TCN
0.020
0.010
0.189
0.004
0.004
0.005
0.001
0.001
0.001
0.033
0.232
PatchTST
0.022
0,010
0.183
0.005
0.005
0.007
0.002
0.001
0.007
0.039
0.167
0.239
OneNet-TCN
0.015
0.012
0.104
0.003
0.003
0.003
0.001
0.001
0.007
0.025
0.114
0.152
OneNet
0.015
0.014
0.100
0.003
0.002
0.003
0.001
0.001
0.005
0.021
0.086
0.099
MAE
ETTH2
ETTml I
WTH
ECL
Method
24
48
24
48
48
Informer
0.043
0.102
0.091
0.060
0.023
0.014
0.005
0.003
0.008
OnlineTCCN
0.007
0,002
0.016
0.002
0.002
0.003
0.002
0.077
0.122
0.002
0.009
0.011
TFCL
0.003
0.003
0.024
0.008
0.005
0.008
0.002
0.001
0.006
0.011
0.019
ER
0.017
0,006
0.013
0.009
0.002
0.004
0,002
0.001
0,005
0.011
0.017
0,014
0.018
0.005
0.012
0.009
0.005
0.002
0.001
0.005
0.013
0.013
0.012
DER++
0.015
0.004
0.015
0.007
0.002
0.002
0.002
0.001
0.007
0.002
0.013
0,014
FSNet
0.009
0.005
0.012
0.004
0.002
0.002
0.001
0.001
0.001
0.001
0.011
Time-TCN
0.009
0.004
0.018
0.006
0.003
0.005
0.002
0.026
0.044
0.008
0.009
0.011
PatchTST
0.013
0.005
0.016
0.009
0.004
0.006
0.002
0.001
0.005
0.012
0.010
0.011
OneNet-TCN
0.017
0,005
0.013
0.008
0.003
0.004
0.002
0.001
0.006
0.009
0.009
0,013
OneNet
0.014
0.005
0.013
0.007
0.003
0.003
0.002
0.001
0.004
0.005
0.012
Learning rate 3e - 3 on Traffic and ECI. and 1e 3 for other datasets. Learning rate le - 2
for the EGD algorithm and le - 3 for the offline reinforcement learning block, where the selection scope is {le 3, 3e - 3, le 2, 3e 2}.
Number of hidden layers 10 for both cross-time and cross-variable branches, where the selection scope {6, 8, 10, 12).
Adapter's EMA coefficient 0.9, Gradient EMA for triggering the memory interaction 0.3, where the selection scope is {0.1, 0.2..... 1.0).
Memory triggering threshold 0.75, where the selection scope {0.6, 0.65, 0.7....0.99
Episodic memory size: 5000 (for ER, MIR, and DER++), 50 (for TFCL).
C.5 Additional Numerical Results
Additional forecasting results. In this section, we analyze the performance of different forecasting methods on four datasets, ECL., WTH, ETTh2, and ETTml, with various starting points, as shown in Figure 8, Figure 9, Figure 10, and Figure 11, respectively. For the last three datasets, all methods produce similar results that can capture the underlying time series patterns, and the performance differences are not significant. However, when comes to the ECL dataset, we observe that almost all baselines exhibit poor forecasting results at the onset of the concept shift (time step 2500). As we provide more instances, the performance of these methods improves, as evidenced by the cumulative loss curves in Figure 7 and Figure 6.
Abition studies of hyree-parameters. We conduct detailed ablation studies about model layers, learning rate, and model dimension here. Taking into account the learning rate for the two-branch framework, the learning rate for the long-term weight, and the learning rate for the short-term weight: Ir, (T lrb, shown in Table 10 (left), the impact of the learning rate on dual-stream
22

---

Table 7: MSE and MAE of various adaptation methods. H: forecast horizon. OneNet-TCNN+Pttc the mixture of TCN, TCme-TCN, and PatchTST.
Metric
Method
ETTH2
ETTml
WTH
ECL
Avg
24
24
TCN
0,502
0.830
L.883
0.214
0.258
0.283
0.308
3.309
11.339
11.534
Time-TC
0.491
0.779
1.307
0.093
0.281
0.308
0.158
0.311
0.308
4.060
5.260
5.230
1.549
MSE
PatchTST
0.362
1.622
2,716
0.083
0.427
0.553
0.162
0.372
0,465
2.023
4.325
5.030
1.512
ObNNot-TCN
0.411
0.772
0,806
0.082
0.212
0.223
0.171
0.293
0.310
2.470
4.713
4.567
1.253
OneNee-TCN-Pauiee
0.355
0.844
1.120
0.079
0.239
0.255
0.163
0.298
0.314
2.172
4.142
4.149
1.178
TCN
0.436
0.547
0.589
0.085
0.381
0.403
0.276
0.367
0.362
0.635
1.196
1.235
0.543
Time-TC3
0.425
0.544
0.636
0.211
0.395
0.421
0.204
0.378
0.378
0.332
0.420
0.488
0.399
PatchTST
0,341
0.672
0.186
0.549
0.393
0.224
0.341
OneNes-TCN
0.374
0.511
0.543
0.191
0.319
0.371
0.221
0.345
0.356
0.411
0.513
0.534
0.391
OneNee-TCN-Pach
0.338
0.513
0.552
0.184
0.360
0.381
0.217
0.351
0.381
0.297
0.423
0.457
0.371
networks is quite significant. The optimal learning rate varies for each dataset, but we can see that for each dataset, the optimal learning rate is generally within the range of [le - 4, le 2]. Ir has a relatively small impact on the final performance of the model. On the contrary, the offlineRRL module determines whether the weights can quickly adapt the new distribution, which has greater impact on the final performance. In terms of model parameters, # Layers, dm. and dheod, all three have significant impact on the performance of the model. A small model may not be able to fit the training data, but a model that is too large increases the risk of overfitting, so each dataset has an optimal model size. However, in this paper, we use the same hyperparameters for all datasets to simplify the complexity of training and model selection. Specifically, we set Ir le Ir le 2, 1r, le 3, #layers 10, dn 64, dhead 320.
C.6 Ensembling more than two networks
In the main paper, we verify the effectiveness of ensembling two branches with different model biases. Here, we show that the proposed OneNet framework enables us to incorporate more branches and the OCP block can fully utilize the benefit of each branch. As shown in Table 7, incorporating Patch TST to OneNet-TCN will further reduce the forecasting results during online forecasting.
C.7 More visualization results and Convergence Analysis of Different Structures
As shown in Figure 6 and Figure 7, ETTh2 and ECL datasets pose the greatest challenge to all models due to the sharp peaks in their loss curves. When the forecasting window is short, OneNet outperforms all baselines by significant margin on all datasets. When the forecasting window is extended to H = 48, FSNet is comparable to OneNet in the first three datasets. However, when concept drift occurs in the ECL dataset, all baselines experience drastic inerease in their cumulative MSE, except OneNet, which maintains a low MSE. Furthermore, the initialized MSE error OneNet is consistently lower than that of all baselines, thanks to the two-stream structure of OneNet. For instance, Figure 6(b) and Figure 6(f), OneNet demonstrates a significantly lower MSE than baselines when the number of instances is less than 100.
C.8 Online forecasting results with delayed feedback
As illustrated in Section 2, this paper adopts the same setting as FSNet [Pham et al., 2023], where the true values of each time step are revealed to improve the performance of the model in subsequent rounds. However, in real-world applications, the true values of the forecast horizon H may not be available until H rounds later, which is known as online forecasting with delayed feedback. This setting is more challenging because the model cannot be retrained at each round and we can only train the model per H round. Tables 8 and show the cumulative performance considering MSE and MAE, respectively. As expected, all methods perform worse with delayed feedback than under the traditional online forecasting setting. Notably, the state-of-the-arr method FSNet is shown to be sensitive to delayed feedback, particularly when H 48, where it is even inferior to a simple TCN baseline on some datasets. In contrast, our proposed method OneNet significantly outperforms all continual learning basclines across different datasets and delayed forecast horizons.
23

---

Table 8: MSE of various adaptation methods with delayed feedback. H: forecast horizon. OneNet- TCN is the mixture of TCN and Time-TCN, and OneNet the mixture of FSNet and TSee-FSNet.
ETTH2
ETTml
WTH
ECL
Method H
24
48
24
48
24
48
24
48
Avg
OnlineTCN
0.502
5.871
11,074
0.214
0.410
0.535
0.206
0.429
0.504
3.309
9.621
24.159
4.736
ER
0,508
5.461
17.329
0.086
0.367
0.498
0.180
0.373
0.435
2.579
8.091
17.700
4,467
DER++
0,508
5.387
17,334
0.083
0.347
0,465
0.174
0.369
0.431
2.657
7.878
17.692
4.444
FSNNt
0.466
5.765
11,907
0.085
0.383
0.502
0.162
0.335
0.411
3.143
8.722
27.150
4.919
OneNet-TCN
0.411
2.639
4.995
0.082
0.287
0.382
0.171
0.341
0.433
2.470
4.809
6.252
1.939
OneNet
0,380
2.064
4,952
0.082
0.332
0.351
0.156
0.323
0.394
2.351
4.984
6.226
1.883
Table 9: MAE of various adaptation methods with delayed feedback. H: forecast horizon. OneNet-TCN the mixture of TCN and TCMe-TCN, and OneNet the mixture of FSNet and Time-FSNet.
ETTH2
ETTml
WTH
ECL
Method /H
24
48
48
24
48
24
Avg
OnlineTCN
0.436
1,109
1,348
0.085
0.511
0,548
0.276
0.459
0.508
0.635
0.783
1.076
0.648
ER
0.376
0,976
1,651
0.197
0.456
0.525
0.244
0.421
0,459
0.506
0.595
0.772
0.598
DER++
0.375
0.967
1.644
0.192
0.443
0.508
0.235
0.415
0.456
0.421
0.591
0.758
0,584
FSNet
0.368
0.983
1.494
0.191
0.468
0.502
0.216
0.394
0.453
0.472
0.827
1.391
0.554
OneNet-TCCN
0.374
0.772
0.951
0.191
0.387
0.417
0.221
0.389
0.461
0.411
0.381
0.451
0.451
OneNet
0.348
0.684
0.916
0.187
0.428
0.430
0.201
0.381
0.436
0.254
0.387
0.444
0.425
Table 10: Results of different OneNet 's hyper-parameter configurations on the benchmarks
(H 48). Ir, (Tw, (rb are the learning rate for the two-branch framework, the learning rate for the long-term weight, and the learning rate for the short-term weight. # Layers is the number of layers of the two branches of OneNet. dm,, dhead is the hidden dimension and the output dimension of the encoders, respectively.
MSE
MSE
Hyper Parameter
Hyper-Parameterr Parameter
Value
ETTh2
ETTml
WTH
ECL.
FTTh2
ETTml
WTH
ECL.
1.00E-01
0.632
0.114
0.203
2.402
1,00E. 02
0.585
0.152
0.171
3.128
0.661
0.101
0,201
2.289
Layers
1.00E-03
0.656
0.111
0.196
2,516
10
0.609
0,108
0,200
2.201
1.005-04
2,994
0.464
0.331
4.949
12
0.652
0,115
0,200
2.328
1.00-001
0.619
0.108
0.202
2.177
16
0.679
0.122
0.223
2.201
1.00E-02
0.108
0.205
2.184
0.612
0.116
0.210
2.810
Irw
dn
1.00-003
0.608
0.108
0.201
2.197
0.609
0.108
0,200
2,311
1.00-004
0.607
0.108
0.201
2.197
160
0.619
0.108
0.200
2.141
L.00E001
0.134
0.741
L.00E-02
0.112
0.197
2.372
0.600
0.112
2.364
Inb
dheus
1.005-03
0.111
0.196
0.609
0.201
1.00E-04
0.111
0.196
2.362
0.571
Table 11: Ablation studies of the variable independence and frequency domain augmentation, where the metric is MSE. FEDformer-F uses frequency-enhance blocks with Fourier transform, and FEDformer-V W uses frequency-enhancee blocks with Wavelet transform. Time-TCN the variable independence version of TCN.
ETTH2
ETTml
WTH
ECL
Method
Online
Avg
24
48
24
48
24
48
24
48
FEDformer-F
1.922
3.045
4.016
0.922
1.003
1.821
3.511
2.344
1.179
43.852
37.802
37.377
11,569
1.912
3,013
3,951
0.372
0.633
0.566
2.196
0.376
0.562
39.243
35.975
36.092
10.409
FEDfomee-W W
1.816
3,070
3.996
2.275
3,784
2.662
1.220
1.211
1.431
41.791
37.236
37,210
11,475
1.798
2.993
1.623
0.238
0.717
0.962
21.387
24.600
27.640
TCN
X
27.060
27.760
26.320
2.240
10.880
0.290
0.450
0.580
$38,000
$46.00 0000
0.530
0.930
0.910
0.130
0.310
0.250
0.300
0.348
0.348
3.010
11,680
10.800
2.462
Time-TNN
X
4.530
7,840
11:017
0.097
0.300
11.017
0.162
0.344
0.429
47.900
48 66.66
67.150
15,020
0.480
0.780
0.867
0.090
0,250
0.310
0.300
0.310
0.309
4.010
5.220
5,210
1.550
24

---

Table 12: Comparison of existing forecasting structures, including TCN [Bai et al., 2018) 8), FED- former [Zhou et al., 2022b], PatchTST [Nie al., 2023], Dlinear [Zeng al., 2023], Nlinear [Zeng et al., 2023], TS-Mixer [Chen et al., 2023], and CrossFormer [Zhang and Yan, 2023].
ETTH2
ETTml
WTH
ECL
Mechod
Online
Avg
24
45
24
48
24
48
24
48
PEDformer.F
1.922
3,045
4.016
0.922
1,003
1.821
3,544
2,344
1.179
43.852
37,802
37.377
11,569
1.912
3.013
3,951
0.372
0.633
0.586
2.196
0.376
0.562
39,213
35.975
36.092
10.109
FEDIonner-W
1.816
3,070
3,996
2.275
3.784
2,662
1,211
1.411
41.791
37.236
37.210
11,475
1.798
2,993
0.717
0.962
PaccTTTT
X
0.427
2,090
3,700
0.083
0.163
0:375
0.467
2,030
1.619
0.362
1.622
2.716
0.083
0.427
0.553
0.162
0.372
0.465
2.022
4.325
5,030
1.512
Crossformer
X
23,270
28.904
29.218
0,400
1.433
1.691
0.146
0.327
0.426
469,260
475.490
478.270
125.736
9.873
2.856
5.772
0.096
0.356
0.370
0.149
0.317
0.359
68,300
92,500
94,790
22.978
27.060
27,760
26.320
2,240
12.170
10.880
0.290
0.480
0.580
538,000
546.000
552.000
145.315
TCN
0.530
0.930
0.910
0.130
0.310
0.250
0.300
0.348
0.348
3,010
11.680
10.800
2,462
Time-TCN
4.530
7,840
0.097
1,030
0.162
0.344
0.429
47,900
67.150
15,020
0.180
0.780
0.300
x
2.91
10.25
7.53
0.538
1.461
1,233
0.266
0.462
0.542
12.03
51.28
58.46
12.247
DLinear
2:44
9,21
6.91
0.16
1.12
0.262
0.159
0.511
6.69
27.82
31.51
7.399
NLinear
X
0.424
50:15
49.52
0.09
4.02
4.13
0.171
1.07
1.08
2.14
930
929
164.316
0.369
50.24
49.6
0.089
4.035
4,141
0.171
1.053
1.064
2.135
930
930
164 A08
Mixie
X
1.968
3.525
4.88
0.335
0.726
0.855
0.255
0.429
0.503
11.16
30.93
44.68
8.354
0.78
2,05
3.060
0.219
0.550
0.660
0.237
0.413
0.482
2.798
4.983
5.764
1.833
C.9 The effect of variable independence and frequency domain augmentation
As shown in Table 11, we observe that frequency-enhanced blocks, which use the wavelet transform, offer greater robustness to the Fourier transform. FEDformer outperforms TCN in terms of general- ization, but online adaptation has limited impact on performance, similar to other transformer- models. Notably, we find that variable independence crucial for model robustness. By convolving solely on the time dimension, independent of the feature channel, we significantly reduce MSE error compared to convolving on the feature channel, regardless of whether online adaptation is applied.
C.10 Comparison of existing forecasting structures.
Results are shown in Table 12. Considering the average MSE on all four datasets, all transformer- based models and Dlinear are better than TCN and Time-TCN, However, with online adaptation, the forecasting error of TCN structures is reduced by large margin and is better than DLinear and FEDłormer. Specifically, we show that the current transformer-bic based model (PatchTST [Nie et al., 2023]) demonstrates superior generalization performance than the TCN models even without any online adaptation. particularly in the challenging ECL task. However, we also noticed that PatchTST remains largely unchanged after online retraining. In contrast, the TCN structure can quickly adapt to the shifted distribution, and the online updated TCN model prefers a better forecasting error than the adapted PS on the first three data sets. Therefore, it is promising to combine the strengths of both structures to create a more robust and adaptable model that can handle shifting data distributions better.
25

---

Algorithm Training and inference algorithm of OneNet
1: Input:
Historical multivariate time series x E RMx. with M variables and length L, the forecast y E R RM* where omit the variable index and time step index for simplicity.
2: Initialize a cross-time forecaster fi, a cross-variable forecaster f2 with corresponding prediction
head, long-term weight [0.5, 0.5], short term learning block fr1 H R and step size for long-term weight updating.
3: Get prediction results from two forecasters.
4: y1 E M = (x).
// Prediction result from the cross-time forecaster.
5: Y2 R f2(x).
// Prediction result from the cross-variable forecaster.
6: Get combination weight from the OCP block.
7: b Jri (w1yı & w2 y))
// Calculate the short term weight.
8: ih (w; + b:)/ Ei= (w; b:) // Calculate the normalized weight.
9: y = w yi + W2 y2.
// The final prediction result.
Update the long/short term weight.
11:
exp(-1) 8: y / E ex y
12: fr + Adam (frs, [(w] y1 + w2 y2,
//Parameters such as learning rate are omitted.
Update the two forecasters.
14: fi + Adam ((F1. fa + Adam x(),
26

---

0.4
0.8
TON
Time-TCN
0.3
FSNet
MSB
MSE
OneNet
0.2
0.4
TCN
Timo-TNN FSNet
0.2
0.1
OneNet
0.0
2000
4000
6000
8000
10000
2000
4000
6000
8000
10000
# Instances
# Instances
ETTh2.
(b) ETTml.
57
TCN
0.4
Time-TCN FSNel
MSE
0.3
OneNet
MSE
3-
0.2
2-
TCN
Time-TCN
0.1
1
FSNet
OneNet
0.0
5000
10000
15000
20000
25000
0
5000
10000
15000
20000
# Instances
Inslances
(c) WTH.
(d) ECL
TCN
TON
Time-TCN
0.4
Time-TCN
FSNet
FSNet
1.5
MSE
OneNet
MSE
0.3
OneNet
0.2
0.5
0.1
0.0
2000
4000
6000
8000
10000
0
2000
4000
6000
8000
10000
# Instances
# Instances
(e) ETTh2.
(I) ETTml.
0.8
20
TCN
TCN
0.7
Time-TCN
Time-TCN
0.6
FSNet
15.
FSNet
0.5
OneNet
MSF
OneNet
10
0.4
0.3
0.2
0.1
5000
10000 Instances
15000
20000
25000
0
5000
10000
15000
20000
Instances
WTH.
(h) ECL
Figure 6: Evolution of the cumulative MSE loss during training with forecast window H = (a, d) and H 48 (e, g, h).
27

---

0.4
0.3
0.4
MAE
MAE
0.3
TCN
TCN
Timc-TCN
Time-TCN
0.1
FSNel
FSNel OneNet
OneNet
0.0
2000
4000
6000
8000
10000
2000
4000 # Instances
6000
8000
10000
# Instances
(a) ETTh2.
(b) ETTml.
TCN
TCN
0.4
Time-TCN
0.6
Time-TCN
FSNet OneNel
FSNet OneNet
0.5
MAE
0.3
MAE
0.2
0.3
0.1
0.2
0.0
5000
10000
15000
20000
25000
5000
10000
15000
20000
# Instances
Inslances
(c) WTH.
(d) ECL
0.7
TCN
Time-TCN
0.4
FSNet
MAE
OneNet
MAE
0.3
0.5
0.2
TCN
Time-TCN
0.1
FSNet
OneNet
0.0
2000
4000
6000
8000
10000
0
2000
4000
6000
8000
10000
# Instances
# Instances
(e) ETTh2.
(I) ETTml.
0.8
1.4
TCN
0.7
Time-TCN
1.2
0.6
FSNet
TCN
MAE
0.5
OneNet
Timc-TCN
FSNei
0.4
OneNet
0.3
0.4
0.2
0.2
0.1
5000
10000 Instances
15000
20000
25000
5000
10000
15000
20000
# Instances
WTH.
(h) ECL
Figure 7: Evolution of the cumulative MAE loss during training with forecasting window /l 1 (d,b and H 48 (...
28

---

Ground Truit
Groind
Ground Truth
TCN
ICN
TCN
TNNet
PSNu
T&Nel
ObeNVet
Instanoes
Imslanoce
Instancer
(a) Channel
(b) Channel 2.
(c) Channel 3.
Ground Trunt:
Ground
TCN
TCN
OvuNaa
OnaNier
Ground Truth
PSNel OzeNet
-2
7500
7510
7000
7530
7540
7550
7500
7530
7520
7530
7540
7550
7500
7530
7520
7530
7540
7550
8 Instannes
0 linstances
Instancer
(d) Channel
(e) Channel
(I) Channel 3.
2.5
Geound Truth
ICN PSNa Checkiit
TCN
0.5
TEN
FSNel
2.0
OneNet
0.0
Qacheet
1.5
0.5
-1.0-
0.5
1.5
10000
100.0
10020
J0000
10000
10050
10000
10010
10020
10030
10040
10050
10000
100L0
30020
10030
100400
10059
5 Inatinees
Iistanccs
gg) Channel
(h) Channel
(i) Channel 3.
5abe
Groand Truth
12
Ground Truth
DCN
DSNet
FSNet
2.0
ObeNeer
OpeNet
CurNet
12500
12516
12530
12530
12540
12550
12500
12510
12520
12550
12510
12500
12500
12510
12520
12550
12560
12550
Finiannss
Iistancce
Tratunces
(i) Channel 1.
(k) Channel
(1) Channel 3.
Figure 8: Visualization of the model's prediction throughout the online learning process in the ECL dataset. We focus on short horizon of 50 time steps and the start prediction time is from 5000 (a,b,c), 7500 (d.e.f), 10000 (g.h.i), and 12500 (j.k,I) respectively.
29

---

1.75
0.5
1.5
1,50
1,0
TUN
0.5
Ground 1muth
Ground Truth
0.75
DrNNi
TCN
PSNet
OncNe::
00
OnoNet
50,0
9000
5040
5050
5020
Instances
Instarces
IIintances
(a) Channel
(b) Channel 2.
(c) Channel 3.
Gooand
Ground Truth
0.5
TCN
0.
PSNel
-0.5
0.0
OseNlet
0.5
-1.0
-1.0
Ground Trath
DCN
0.2
PSNC:
-1.5
2.0
Obccet
7510
7530
7540
7550
7510
7530
7540
Instacces
Instunces
Iisttances
(d) Channel
(e) Channel 2.
(f) Channel 3.
-0.5
1.0
Grannd Tarn
Ground Tnnti
Groudd Tritt
TCN
TCN
TUN
PSNel
PSNNer
-2.0
ObeNiet
OneNet
OneNet
Inssamzss
(g) Channel 1.
(h) Channel 2.
(i) Channel 3.
2.5
1.5
1.0
6.5
Grannd
Giuand
0.5
Gioand
TCN
TCN
TCM
PNNet
PSNet
PSNet
Onelvet
OnoNeet
OnoNet
1250
135
17500
12520
12480
1359
1250
13510
17500
17530
17440
17590
12503
12510
12500
17810
17440
125%0
Instunces
Instunces
#astannees
(i) Channel 1.
(k) Channel 2.
(1) Channel 3.
Figure 9: Visualization of the model's prediction throughout the online learning process the WTH dataset. We focus on a short horizon of 50 time steps and the start prediction time is from 5000 (a,,.)), 7500 (d.c.f). 10000 (g.h.), and 12500 (j.k.l), respectively.
30

---

LU
TCN
1.2
TCN
UneNie
CueNNat
Vahe
0.5
0.5
-2
Ground Trri:
0.0
TCN
0.0
-0.5
OueNet
-0.5
2500
25:00
2520
25:0
2540
2550
2500
2510
2520
2530
2540
255g
2500
2510
2520
25.50
2540
2550
Instances
(a) Channel 1.
(b) Channel 2.
(c) Channel 3.
3.0
Gecrad Truhh
-1.5
TCN
2.01
0.5
PSNet
Cucckii
0.0
2.0
Girladd Truth
Ground Truth
0.3
-3.5
PSNNer
PSNet
41
1.0
(d) Channel 1.
(e) Channel 2.
(f) Channel 3.
Ground Truth
-3.0
PSNel
-3.5
OneNNat
Valte
-1.5
-2.0
Group Truth
Grotan Trani
ITN
TCN
PSNet
OgeNNet
CacNNt
7510
0520
7540
7550
2510
7520
7540
7550
7510
7500
7540
Instances
Instrnccss
Instoncce
g Channel
(h) Channel
Channel 3.
0.5
Giouud Truth
-3.25
Trush
1.5
-3.50
IUN
0.0
USSNet
-0.5
OpeNNit
-3.75
ObeNNet
1.0
-4.00
-1.0
0.5
Grozad Trust
-1.25
TCN
+3.5
4.50
0.0
PSNer
-2.0
-6.5
Initin
119500
( Channel 1.
(k) Channel 2.
(1) Channel 3.
Figure 10: Visualization of the model's prediction throughout the online learning process in the ETTh2 data set. We focus on a short horizon of 50 time steps and the start prediction time is from 2500 (a,,,c), 5000 (d,e.f), 7500 (g,h.i), and 10000 (j.kl) respectively.
31

---

Oround Iruhh
2.5
TCN
170
TCN
FSNet
OreNa:
ObeNet
0.5
OnoNai
1.5
0.0
-0.5
0.0
-1.
2510
2530
2540
2550
1500
2510
2530
2510
2540
2540
2510
2520
2540
2550
Instaaces
(a) Channel 1.
(b) Channel 2.
(c) Channel 3.
Giuund Truth
1.0
Girund Truth
Ground Truth
-0.5
TCN
PiNiet
0.5
PSNia
PSNe
-1.0
OraNie:
OraNii
Ozeblel
-1.5
-0.5
-2.0
-2.5
-1.5
50:30
9050
(d) Channel 1.
(e) Channel 2.
(f) Channel 3.
Ground
TCN
TCN
OuzNeer
OnaNiel
-0.5
DCN
1.0
Oueciic
7500
3510
7530
7530
7540
7550
7500
7510
75C20
75300
7540
7550
7500
75.00
75200
75300
7540
7550
+ Instannes
Iramnacess
Iisstances
(g) Channel
(h) Channel
i1) Channel 3.
Giruad
0.0
TUN
-0.5
ChaeNict
0.5
-0.5
Vile
Gevuzd Truis
Grrvazdd Truit TUN
ICN
TSNai
PSNat OreNiit
Orchiict
3.0
100000
10020
10K20
20070
100000
10020
100000
10010
10020
20070
JUG00
10000
10000
ISULD
J0020
10030
100.00
19059
Instannes
Ilustanzes
Bastancess
(j) Channel 1.
(k) Channel
()) Channel 3,
Figure 11: Visualization the model's prediction throughout the online learning process on the ETTml dataset. We focus on short horizon of 50 time steps and the start prediction time is from 2500 (a, b, c), 5000 (d, e, f), 7500 (g. h, i) and 10000 (j. k, 1), respectively.
32


Reviews:
[{"review_id": "9z914GGNBH", "reviewer": "Reviewer_r9Pc", "summary": "The paper introduces an approach called Online ensembling Network (OneNet), aimed at online updating of time series forecasting models. This approach is crafted to handle the concept drift problem efficiently using streaming data. The authors note that while some algorithms designed for online time series forecasting exploit cross-variable dependency, others presume independence among variables. Each assumption carries its own strengths and weaknesses when it comes to online time series modeling. To address this, the authors' OneNet dynamically updates and combines two models. One model emphasizes on modeling the dependency across the time dimension, while the other is concentrated on cross-variate dependency. The OneNet method infuses a reinforcement learning-based strategy into the conventional online convex programming framework. This integration allows for the linear combination of the two models with dynamically adjusted weights. By doing so, OneNet addresses a significant drawback of classic online learning methods, which often struggle to adapt swiftly to the concept drift. Empirical findings showcase that OneNet diminishes online forecasting error by over 50% compared to the State-Of-The-Art (SOTA) method.\n\n\n\n\n", "questions": "see above.", "limitations": "N/A", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. This paper is easy to follow and read. The considered problem is important in the real world.\n\n2. The proposed method works well in many experiments, verifying its effectiveness. The experiments are solid.", "weaknesses": "1. This paper's technical novelty is limited. The authors do not analyse why their method works in theory. Updating techniques itself is not novel, thus theoretical analysis is necessary for this paper, showing the proposed method should work well and work better than other methods in theory. The proposed theory of this paper is not specific and the corresponding bound is not tight and is less informative.\n\n2. In the introduction, I do not see a relevant discussion regarding the theory of online learning.\n\n3. In time series, there will be dependence among data. Will this kind of dependency influence the generalization result? Under what kind of assumption, it will not influence the generalization result?\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable feedback and comments on our paper. We appreciate the positive feedback on the readability and the importance of the considered problem in the real world. We also acknowledge the points you raised regarding the technical novelty and the need for theoretical analysis in the paper.\n\n**Concern 1: this paper's technical novelty**\n\nIn deep learning domain, developing a generalization bound under changing environments for forecasting tasks is indeed a challenging task but also beyond the primary focus of this paper. We will take your suggestion into account and work towards providing a more comprehensive theoretical analysis, including tighter bounds and more informative insights into the generalization results of the proposed OneNet approach.\n\nHowever, note that **our work is primarily empirical in nature**, with a focus on building a stronger online time series forecaster. The theoretical framework presented in the manuscript is employed not as the central theme, but rather as a means to substantiate the rationality of our proposed methodology and to elucidate the motivation behind it.\n\nFurthermore, the previous work closest to ours should be [1], which also extensively discusses the gap between online learning and our specific setting. In our experiments, we have also included various online learning baselines for comparison. We find that traditional online learning algorithms do not perform exceptionally well in this setting. This highlights the need for designing algorithms or models that are better suited for this specific issue of concept drift. We believe that our empirical validation is a valuable contribution to the field of robust time series forecasting. \n\nAdditionally, we conclude the technical novelties as follows for your convenience:\n\n1. As far as we know, this is the first work that evaluates almost all advanced structures/design choices of time series forecasting under concept drift settings. Our experiments shed light on the architectural design of robust forecasting models.\n2. We observe the data preference for cross-variable dependency and cross-time biased models and point out why a single model cannot perform well under all concept-drift settings. Thus we propose OneNet to utilize the strengths of both kinds of models.\n3. OneNet addresses a significant drawback of classic online learning methods and allows for the linear combination of the two kinds of models with data-driven adjusted weights. OneNet reduces the average cumulative mean-squared errors (MSE) by 53.1% and mean-absolute errors (MAE) by 34.5%. In particular, the performance gain on the challenging dataset ECL is superior, where the MSE is reduced by 59.2% and MAE is reduced by 63.0% compared to the SOTA model FSNet[1].\n\n\n[1] Pham, Quang, et al. \"Learning fast and slow for online time series forecasting.\" ICLR , 2023\n\n\n----\n\n**Concern 2: a relevant discussion regarding the theory of online learning**\n\nOur focus is online time series forecasting, which is mainly for real-world applications, therefore, the theoretical analysis in this paper is just trying to explain the motivation of OneNet and make the design choice easy to understand. So we just introduce some basic online learning theory works in section 3. According to your suggestion, in our revised version, we will include a separate subsection in the related work to discuss relevant online learning theory works more comprehensively. This will help readers to have a deeper understanding of the theoretical foundations and the relationship between OneNet and existing online learning approaches.\n\n----\n\n**Concern 3: will dependence among data influence the generalization result?**\n \nYour question may encompass two distinct forms of data dependence: cross-variable dependency and cross-time dependency. However, we would appreciate further clarification on the specific type of dependence you are referring to, as the implications for generalization may differ based on the context.\n\n**Cross-variable dependency:** In time series data, cross-variable dependencies widely exist, meaning that different variables in time series may be correlated or influenced by each other over time. However, it is not accurate to say that cross-variable dependency directly affects generalization ability. The impact on generalization ability depends more on the model and the presence of distribution drift. For example, in usage settings with a large number of variables (ECL dataset in Table.1), the models that heavily focus on cross-variable dependency may overfit the training instances, leading to poor generalization performance when facing distribution drift in a changing environment.  If there is no significant distribution shift between the training and test sets, the impact of cross-variable dependency on generalization ability will not be pronounced.\n\n**Cross-time dependency:** Another type of dependence among data would be cross-time dependency. In literature, people usually assume a stationary beta-mixing autoregressive process to control the time series’s “volatility” or “variance” and derive the generalization error bound. (e.g., [1] and [2]).  However, in our setting, the underlying model evolves over time, i.e., a non-stationary autoregressive process, which makes the analysis even harder. We totally agree with reviewer that a rigorous generalization bound would be very important. Due the scope of this work, we would like to hold it for future research.\n\n[1] McDonald, Daniel J., Cosma Rohilla Shalizi, and Mark Schervish. \"Generalization error bounds for stationary autoregressive models.\" arXiv preprint arXiv:1103.0942 (2011).\n\n[2] Vankadara, Leena Chennuru, et al. \"Causal forecasting: generalization bounds for autoregressive models.\" Uncertainty in Artificial Intelligence. PMLR, 2022.\n\n----\n\nIf you have any further suggestions or comments, we welcome them and will incorporate any necessary improvements."}, {"review_id": "nGmrO56olt", "reviewer": "Reviewer_7dRB", "summary": "This paper addresses the problem of concept drift in online Time Separate forecasting. A method OneNet is proposed that uses temporal correlations and inter-variables dependencies. OneNet is compared with other state of the art methods and OneNet is demonstrated to be superior. Furthermore, through empirical studies various factors are studies that effect the forecasting\n", "questions": "Will it be possible to test the proposed OneNet using generated data, so that all the benefits can be highlighted with controlled experiments", "limitations": "No Limitations are listed", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Mathematicsl formulation of the problems and OneNet solution\n2. Training of 2 independent forecasts: one for tempra and second for the inter-variables dependencies.\n3. The use of long term historical and short term variations\n4. Ablation study and analysis\n5. Comparison with state of art methods\n", "weaknesses": "Does not describe the datasets used for the experiments, just started using abbreviation in tables.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your feedback and positive comments. We appreciate your thorough evaluation on our paper. We are glad that you find the mathematical formulation, the training of two independent forecasts, and the use of long-term historical and short-term variations as strengths of our work.\n\nWe agree with your observation about the lack of description of the datasets used in the experiments. In the revised version of the paper, we will ensure to first provide simple information about the datasets, including their sources and characteristics. \n\nRegarding your question, yes, it is indeed possible to test the proposed OneNet using synthesized data. However, generating meaningful and realistic multivariate simulation data can be hard. Besides, **controlling the data preference to cross-channel bias and cross-time bias is more challenging**. To ensure that the data used in our experiments is representative and practical, we primarily chose real-world datasets to validate our algorithm. These datasets encompass different numbers of variables (from 7 to 321) and varying degrees of concept drift, and OneNet consistently demonstrates **remarkable performance improvements** across various scenarios. \n\nWe will consider conducting meaningful synthetic datasets in future work to further demonstrate the power of OneNet. Your feedback is quite valuable, and we are committed to addressing your comments to enhance the quality and clarity of our work. Thank you for your valuable comment.\n"}, {"review_id": "xV0SDPWmjx", "reviewer": "Reviewer_QKSm", "summary": "Motivated by the observation that a single model may not consistently achieve optimal performance in time series forecasting, the author proposes a method, which combines two stream architectures (cross-time and cross-variable models) using online convex programming. However, due to the limitations of online convex programming, which involves a trade-off between switching speed and overall performance, an RL-based algorithm is introduced as a solution. The proposed method is reinforced by a range of experimental results, providing support for its effectiveness.", "questions": "In addition to the question raised in the weaknesses section, I have another question.\n\nCould you provide a more detailed algorithmic description that illustrates the training or inference process using the \\usepackage{algorithm} package? This would greatly aid in understanding the intricacies of the algorithm.", "limitations": "The paper gives good motivation, novel methods, and concrete experimental results. I think this paper deserves acceptance. \nAddressing these concerns can further enhance the overall quality and strength of the paper. If some minor concerns which posed in weaknesses and questions section, I will raise my points.\n", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. Easy to follow:\nThe paper is excellently written and presented in a manner that is easy to comprehend and follow along.\n\n2. Novelty:\nWhile ensemble methods for time series tasks have been previously proposed, the specific approach taken in this paper, such as the adoption of OCP and RL for model ensembling, is novel and has not been introduced before. This novelty serves as a notable advantage of this paper.\n\n3. Concrete experimental results:\nThe paper provides a comprehensive comparison between its proposed method and numerous baselines across various dataset settings. Additionally, to assess the effectiveness of their ensembling approach, the paper includes comparisons with multiple baseline ensembling methods. These extensive and well-designed experimental results serve to reinforce the credibility and robustness of the paper.", "weaknesses": "There are minor issues in this paper.\n\n1. About the motivation:\nIn the introduction section, the author presents Figure 1 and Table 1 as part of the motivation for their work. The main motivation behind this study is the recognition that a single model is insufficient to address complex time series problems, leading to the proposal of an ensemble method. However, the necessity of Table 1 to explain this motivation is not clear. Regarding Table 1, the author mentions that \"cross-time forecasters tend to exhibit lower performance in certain datasets, and existing models that leverage both cross-variable and cross-time information perform worse than the naive TCN method.\" This sentence may not be directly relevant to the motivation, and it would be beneficial to rephrase it to establish a closer connection between the sentence and the motivation.\n\n2. Ablation Study:\nIt would be valuable to include experimental results from ablating the offline RL for short-term weight. This ablation study can provide concrete evidence for the necessity of this module in a more tangible manner.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable feedback and positive evaluation of our paper. We appreciate your thorough review and are glad that you find the paper easy to follow and well-presented.\n\n----\n\n**Concern 1 About the motivation**\n\nRegarding the motivation, we understand your concern about the clarity of Table 1 in establishing the motivation. We would like to emphasize that the statement \"cross-time forecasters tend to exhibit lower performance in certain datasets, and existing models that leverage both cross-variable and cross-time information perform worse than the naive TCN method\" aims to highlight specific limitations of existing methods when dealing with complex time series forecasting problems.\n\nCross-time forecasters, due to their lack of modeling inter-variable relationships, may **exhibit lower performance on datasets with a small number of variables**. On the other hand, the models that heavily leverage cross-variable information (cross-variable forecasters) may easily **overfit to the current pattern in training data when dealing with datasets with a large number of variables**, leading to poor generalization in the presence of concept drift, as particularly observed in the ECL dataset.\n\nAlthough some existing methods claim to capture both cross-variable and cross-time dependencies, we find that the performance under concept drift is even worse than the naive TCN method. This observation serves as a catalyst for us to propose a new framework, OneNet, which effectively balances the cross-variable and cross-time dependencies under concept drift scenarios, leading to improved generalization performance. \n\n\nWe hope this clarification provides a more explicit connection between Table 1 and the motivation behind our work. We will ensure that the revised version of the paper clearly establishes the rationale for introducing OneNet to address the limitations of existing methods in handling complex time series problems with concept drift. If you have any further questions or suggestions, please feel free to let us know.\n\n----\n\n**Concern 2 Ablation Study**\n\nWe did conduct a comprehensive ablation study on the OCP block in Table 4. Specifically, the Exponentiated Gradient Descent (EGD) serves as the baseline without offline RL for short-term weight, while the Reinforcement Learning to learn the weight directly (RL-W) serves as the baseline without long-term weight. From the table, we can observe that solely using short-term weight already yields better results compared to EGD. Additionally, combining both short-term and long-term weight further enhances the performance of our algorithm.\n\nThe results from this ablation study indeed provide concrete evidence of the necessity of incorporating both short-term and long-term weights using reinforcement learning in the OneNet framework. It demonstrates that the RL-based algorithm plays a crucial role in achieving improved forecasting performance, especially under concept drift scenarios.\n\n----\n\n**Concern 3 a more detailed algorithmic description** \n\nThank you for your suggestion. We have included a more detailed algorithmic description in the uploaded pdf file. This algorithmic description provides a step-by-step illustration of the training and inference process of the OneNet method using the \\usepackage{algorithm} package. We believe that this additional information will aid in understanding the intricacies of our algorithm and enhance the clarity of our paper.\n\n----\n\nIf you have any further suggestions or comments, we welcome them and will incorporate any necessary improvements to enhance the overall quality and reliability of the paper. Your feedback is valuable to us, and we are committed to producing a high-quality and impactful contribution."}, {"review_id": "cJg7rOinoQ", "reviewer": "Reviewer_UB5d", "summary": "The paper proposes a method for online learning based on ensemble learning. The method proposed by the authors leverages cross-variable and cross-time dependency. In addition, the authors propose a reinformed learning method that dynamically adjusts the weights used to combine the models.", "questions": "The paper is well-presented and well-organized. However, the literature review is suggested to be enriched. The existing studies are hardly investigated. Online learning literature not based on deep learning needs to be further studied, there is a lack of state-of-the-art references. Moreover, cross-variable dependency has already been studied in online learning as well as ensemble learning based methods.\n\nCross-variable dependency reference:\n\n“Minimax classification under concept drift with multidimensional adaptation and performance guarantees”\n\nEnsemble learning references:\n\n“Online Ensemble Multi-kernel Learning Adaptive to Non-stationary and Adversarial Environments”\n\n“Dynamic Weighted Majority: An Ensemble Method for Drifting Concepts”\n\n“Incremental Learning of Concept Drift in Nonstationary Environments”\n\nI think one improvement the authors can consider is to add some discussions about the computational and memory complexity. In online learning, we sometimes need to obtain real-time predictions. \n\nComparing the theoretical results with similar results from the state-of-the-art would help to show the contributions of the presented bounds.\n\nReferences:\n\n“Online Ensemble Multi-kernel Learning Adaptive to Non-stationary and Adversarial Environments”\n\n“An adaptive gradient method for online AUC maximization”\n\n“Large Scale Online Kernel Learning”\n\nThe reviewer suggests making the code available so that the results can be replicated.\n", "limitations": ".", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "See Question section", "weaknesses": "See Question section", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely apologize for any confusion caused by the lack of clarity in our paper. We appreciate your feedback and would like to address your comments to provide a clearer understanding of our work.\n\n----\n\n**Concern 1 Online learning literature not based on deep learning**\n\nRegarding the literature review, we acknowledge your suggestion and will enrich our work with more references from online learning not based on deep learning. However, we would like to clarify that our paper's main focus is on online time series forecasting, which is a specific and challenging problem (**See general response for detailed discussion**). \n\nAs the primary goal of this paper is to focus on model architecture and empirical performance, the theoretical results only serve to illustrate the rationality of our design. Although in the current draft, the order of theoretical bounds is similar to the standard result in the literature, it is sufficient to justify the design philosophy behind our proposed components: two-stream forecasters and decoupled training strategy. Due to the scope of this paper, we will hold the in-depth theoretical analysis for future research.\n\n\n----\n\n**Concern 2 cross-variable dependency has already been studied in online learning**\n\nThank you for pointing out the reference [1], which is a statistical learning method designed for supervised classification, and it requires instance-label pairs, making it fundamentally different from our task of time series forecasting.\n\nIn our context, when we mention cross-variable dependency, we refer to the correlation and interactions among different input variables within a time series forecasting problem, which is not the same as [1]. We acknowledge the importance of clarifying the differences between the terminologies used in different contexts. In the revised version of the paper, we will provide more explicit definitions and explanations to distinguish our usage of cross-variable dependency from that of [1] and ensure a better understanding of the specific challenges and solutions in time series forecasting with our proposed OneNet approach.\n\n[1] V. Alvarez, et al., Minimax classification under concept drift with multidimensional adaptation and performance guarantees, ICML 2022.\n\n----\n\n**Concern 3 ensemble learning based methods**\n\nFirst and foremost, we would like to emphasize that our method is not a simple ensemble approach. Our first core contribution lies in **recognizing the limitations of a single model in handling concept drift and devising a way to train two complementary prediction models**. This motivates us to explore ensemble learning algorithms. As shown in Table 4, we find that by using the two complementary models we proposed, even with the simplest ensemble strategy (averaging), significant improvements can be achieved compared to the SOTA method.\n\nMoreover, we introduce advanced ensemble learning baselines, such as Mixture-of-experts [1] and Gating mechanism [2], which are more relevant and real-time in the context of our work. As for the baselines you mentioned, [4] is suitable for classification tasks as it requires judging the correctness of the classification results to update expert weights and create/delete experts. The voting strategy in [3.5] can be adapted for our task with some modifications. We implement it and report the results in the Table.1 of the attached pdf.\n\nAs shown in the table, the method designed for online multiple kernel learning [3] performs poorly in this setting and is even outperformed by our simple baseline (averaging the prediction results of two forecasters). The strategy in [5] yields slightly better results than simple averaging, but in our scenario, it still falls short compared to our ensemble baseline (Gating) and especially our proposed OneNet method. \n\n[1] Outrageously large neural networks: The sparsely-gated mixture-of-experts layer.\n\n[2] Gated transformer networks for multivariate time series classification\n\n[3] Online ensemble multi-kernel learning adaptive to non-stationary and adversarial environments\n\n[4]  Dynamic weighted majority: An ensemble method for drifting concepts\n\n[5] Incremental learning of concept drift in nonstationary environments\n\n----\n\n**Concern 4 computational and memory complexity**\n\nWe understand the importance of computational and memory complexity in online learning, and we agree that discussing these aspects in our paper would be beneficial. Using ensemble techniques would naturally increase the model size and training time. The statistics are shown in the table below, and it can be observed that our method (OneNet) does not sacrifice too much in terms of inference time compared to previous SOTA solution [1]. However, it is inevitable that there will be some memory overhead. (Table.2 in the attached pdf)\n\nNevertheless, it is worth mentioning that in this paper, we have explored ways to mitigate this issue. Starting from line 278, we propose a parameter-efficient version called OneNet-, which significantly reduces the number of parameters and speeds up the inference process compared to the traditional framework. Moreover, its performance is only slightly lower than OneNet. This allows us to strike a better balance between computational complexity and forecasting accuracy.\n\n[1]  Learning fast and slow for online time series forecasting, ICLR, 2023\n\n----\n\n**Concern 5 Comparing the theoretical results with similar results**\n\nAccording to your suggestion, in our revised version, we will include a separate subsection in the related work to discuss relevant online learning theory works more comprehensively.\n\n----\n\n**Concern 6 making the code available**\n\nWe have uploaded the Anonymous GitHub link.\n\nhttps://anonymous.4open.science/r/OneNet-58B3\n\n----\n\nOnce again, we thank you for your valuable feedback, and we will take your suggestions into account while revising the paper. If you have any further questions or need more information, please feel free to let us know. "}]

Author rebuttal:
We thank the Reviewers for the insightful comments and detailed feedback. We are delighted that reviewers find our paper has **good motivation** (QKSm), **easy to follow** (QKSm, r9Pc), **excellently -presented and organized** (UB5d, QKSm), **enough technical novelty** (QKSm, 7dRB), and **Impressive empirical results** (r9Pc, QKSm, 7dRB). 

There is a shared concern from Reviewers r9Pc and UB5d regarding the novelty of our work in comparison to the theory of traditional online learning literature and some ensembling methods designed for classification tasks or online kernel learning. We would like to emphasize that our work is primarily empirical in nature, with a focus on building a stronger online time series forecaster. *The theoretical framework presented in the manuscript is employed not as the central theme, but rather as a means to substantiate the rationality of our proposed methodology and to elucidate the motivation behind it*. Our key contribution is a novel solution to online time series forecasting inspired by the different data preferences to model biases during the concept drift. It is crucial to note that directly applying common online learning frameworks for image data to time series is not straightforward due to the differences between these two types of data, such as discrete vs. continuous labels and distinct label differences among tasks. Besides, we have included various online learning and ensembling baselines in the main paper and the rebuttal process to address these concerns, hoping that these experiments demonstrate the significance of our proposed approach. We refer the Reviewers to our responses to **Concern #1 of Reviewer r9Pc** for a detailed discussion regarding the novelty of our work.

In the revised version, we have made substantial improvements based on the Reviewers' comments. We include a separate subsection in the related work to discuss relevant online learning theory and online ensembling learning works more comprehensively. Additionally, we provide a more detailed algorithmic description to illustrate the training or inference process, addressing Reviewer QKSm's concern about clarity. Furthermore, we incorporate more ensembling baselines and add discussions about computational and memory complexity to address concerns raised by reviewer UB5d.



Meta-review: This paper deals with multivariate forecasting in the online context, where data streams in, and where concept drift potentially occurs. The authors argue that both dependencies between variables, and along time (independent between variables) are important for good practical performance in this setting. They propose a dynamic combination of two models, each representing one such mode of dependencies. This is not an ensemble, since weights change over time. They incorporate a reinforcement learning algorithm into online convex programming in order to adapt the weights. Their experiments on a number of realistic benchmarks shows a significant improvement over the SotA.

This works addresses an important problem. Reviewers agree the method is novel (esp the use of RL), and the experiments are very well done, the writing is clear. Different to many other online learning papers (also on forecasting), the paper does not present strong theoretical results, but the authors convincingly argue that the main focus of the paper is on good empirical performance in the real world. There are also ablation experiments, showing that RL dynamic weighting is essential for good performance. One possibility of improvement is to include baselines into the comparison which also handle concept drift.
Decision: Accept (poster)