Learning Fast and Slow for Online Time Series Forecasting
Authors: Quang Pham
Chenghao Liu
Doyen Sahoo
Steven Hoi
Venue: ICLR 2023 poster (2023)
Keywords: online time series forecasting
continual learning

Abstract: Despite the recent success of deep learning for time series forecasting, these methods are not scalable for many real-world applications where data arrives sequentially. Training deep neural forecasters on the fly is notoriously challenging because of their limited ability to adapt to non-stationary environments and remember old knowledge. We argue that the fast adaptation capability of deep neural networks is critical and successful solutions require handling changes to both new and recurring patterns effectively. In this work, inspired by the Complementary Learning Systems (CLS) theory, we propose Fast and Slow learning Network (FSNet) as a novel framework to address the challenges of online forecasting. Particularly, FSNet improves the slowly-learned backbone by dynamically balancing fast adaptation to recent changes and retrieving similar old knowledge. FSNet achieves this mechanism via an interaction between two novel complementary components: (i) a per-layer adapter to support fast learning from individual layers, and (ii) an associative memory to support remembering, updating, and recalling repeating events. Extensive experiments on real and synthetic datasets validate FSNet's efficacy and robustness to both new and recurring patterns. Our code is publicly available at: \url{https://github.com/salesforce/fsnet/}.

Published as a conference paper at ICLR 2023
LEARNING FAST AND SLOW FOR ONLINE TIME SERIES FORECASTING
Quang Pham Chenghao Liu², Doyen Sahoo Steven C.H. Hoi 8-1
Steven
$ Salesforce Research Asia
f Institute for Infocomm Research RI² Agency for Science, Technology and Research (A (A*STAR),
Fusionopolis Way, #21-01, Connexis South Tower, Singapore 138632, Republic of Singapore
: School of Computing and Information Systems, Singapore Management University
ABSTRACT
Despite the recent success of deep learning for time series forecasting, these meth- ods are not scalable for many real-world applications where data arrives sequen- tially. Training deep neural forecasters cause of their limited ability to adapt to non-stationary environments and remem- ber old knowledge. We argue that the fast adaptation capability of deep neural net- works critical and successful solutions require handling changes to both new and recurring patterns effectively. In this work, inspired by the Complementary Learn- ing Systems (CLS) theory, we propose Fast and Slow learning Network (FSNet) novel framework address the challenges of online forecasting. Particularly, FSNet improves the slowly-learned backbone by dynamically balancing fast adap- tation recent changes and retrieving similar old knowledge. FSNet achieves this mechanism via an interaction between two novel complementary components: (i) a per-layer adapter to support fast learning from individual layers, and (ii) an asso- ciative memory to support remembering, updating, and recalling repeating events. Extensive experiments on real and synthetic datasets validate FSNet's efficacy and robustness to both new and recurring patterns. Our code is publicly available at: https::/ //gs::///iie thub com/ salesforce fsnet/.
the fly is notoriously challenging be-
INTRODUCTION
Time series forecasting plays an important role in both research and industries. Correctly forecast time series can greatly benefit various business sectors such as traffic management and electricity consumption (Hyndman & Athanasopoulos, 2018). As a result, tremendous efforts have been de- voted to develop better forecasting models (Petropoulos et al., 2020; Bhatnagar et al., 2021; Triebe et al., 2021), with a recent success deep neural networks (Li et al., 2019; Xu et al., 2021; Yue et al., 2021; Zhou et al., 2021) thanks to their impressive capabilities to discover hierarchical la- tent representations and complex dependencies. However, such studies focus on the batch learning setting which requires the whole training dataset to be made available priori and implies the rela- tionship between the input and outputs remains static throughout. This assumption is restrictive in real-world applications, where data arrives in stream and the input-output relationship can change over time (Gama et al., 2014). In such cases, re-training the model from scratch could be time con- suming. Therefore, is desirable to train the deep forecaster online (Anava et al., 2013; Liu al., 2016) using only new samples to capture the changing dynamic of the environment.
Despite the ubiquitous of online learning in many real-world applications. training deep forecasters online remains challenging for two reaons. First, naively train deep neural networks on data streams requires many samples converge (Sahoo et al., 2018: Aljundi al.. 2019a) because the offline training benefits such as mini-batches or training for multiple epochs are not available. Therefore, when distribution shift happens (Gama et al., 2014), such cumbersome models would require many samples to learn new concepts with satisfactory results. Overall, deep neural networks, al- though possess strong representation learning capabilities, lack a mechanism to facilitate successful learning on data streams. Second, time series data often exhibit recurrent patterns where one pattern
* Part of this work was done during Quang Pham's 's internship at Salesforce Research Asia. The final version was prepared I'R, A*STAR. Correspondence to: phamhq@ 2r a a-star .eaar.dduss.

---

Published as a conference paper at ICLR 2023
could become inactive and re-emerge in the future. Since deep networks suffer from catastrophic forgetting (McCloskey & Cohen, 1989), they cannot retain prior knowledge and result in inefficient leamning of recurring patterns, which further hinders the overall performance. Consequently, online time series forecasting with deep models presents a promising yet challenging problem.
To address the above limitations, we redically formulate online time series forecasting line, task-free continual learning problem (Aljundi et al., 201 20) Particularly, continual learning requires balancing two objectives: (i) utilizing past knowledge to facilitate fast learning of current patterns; and (ii) maintaining and updating the already acquired knowledge. These two objectives closely match the aforementioned challenges and are usually referred to as the sraillitypllasticn dilemma (Grossberg, 1982). With this connection, we develop an effective online time series fore- casting framework motivated by the Complementary Learning Systems (CI.S) theory (McClelland et al., 1995; Kumaran et al., 201 6), neuroscience framework for human continual learning. Specif- ically, the CLS theory suggests that humans can continually learn thanks to the interactions between the hippocampus and the neocortex, which supports the consolidation, recall, and update such expe- riences to form a more general representation, which supports generalization to new experiences.
on-
This work develops FSNet (Fast-and-Slow learning Network) to enhance the sample efficiency of deep networks when dealing with distribution shifts or recurring concepts in online time series fore- casting. FSNet's key idea for fast learning is its ability to always improve the learning at current steps instead of explicitly detecting changes in the environment. To do so, FSNet employs a per- layer adapter to model the temporal consistency in time series and adjust each intermediate layer to learn better, which in turn improve the learning of the whole deep network. In addition, FSNet further equip each adapter with an associative memory (Kaiser al., 2017) to store important, re- curring patterns observed. When encountering such events, the adapter interacts with its memory to retrieve and update the previous actions further facilitate fast learning. Consequently, the adapter can model the temporal smoothness in time series to facilitate learning while its interactions with the associative memories support remembering and improving the learning of recurring patterns.
In summary, our work makes the following contributions. First, we radically formulate learning fast in online time series forecasting with deep models as a continual learning problem. Second, motivated by the CLS theory, we propose fast-and-slow learning paradigm of FSNet to handle both the fast changing and long-term knowledge in time series. Lastly, we conduct extensive experiments with both real and synthetic datasets to demonstrate FSNet's efficacy and robustness.
PRELIMINARY AND RELATED WORK
This section provides the necessary background of time series forecasting and continual learning.
2.1 TIME SERIES FORECASTING SETTINGS
Let x= =x1.....,, PT) E RT be a time series of T observations, each has n dimensions. The goal of time series forecasting is that given look-back window of length e, ending time i: Xi, (z.-et1,..., 2;). predict the next H steps of the time series as fu (X..H) = (I.+1...... #++H), ) where w denotes the parameter of the forecasting model. We refer to a pair of look-back and forecast windows as a sample. For multiple-step forecasting (H > 1) we follow the standard approach of employing linear regressor forecast all // steps in the horizon simultaneously (Zhou et al., 2021).
Online Time Series Forecasting is ubiquitous is many real-world scenarios (Anava et al., 2013; Liu et al., 2016: Gultekin & Paisley, 2018: Aydore et al., 2019) due the sequential nature of data. In this setting, there is no separation of training and evaluation. Instead, learning occurs over a sequence of rounds. At each round, the model receives look-back window and predicts the forecast window. Then, the true answer is revealed to improve the model's predictions of the incoming rounds (Hazan, 2019). The model is commonly evaluated by its accumulated errors throughout learning (Sahoo et al., 2018). Due to its challenging nature, online time series forecasting exhibits several challenging sub-problems, ranging from learning under concept drifts (Gama et al., 2014), to dealing with missing values because of the irreullarl-sampler data (Li & Marlin, 2020; Gupta et al., 1). In this work, we focus on the problem of fast learning (in terms of sample efficiency) under concept drifts by improving the deep network's 's architecture and recalling relevant past knowledge. There is also a rich literature of Bayesian continual learning to address regression problems (Smola et al., 2003; Kurle et al., 2019; Gupta et al., 2021). However, such formulation follow the Bayesian
2

---

Published as a conference paper at ICLR 2023
Model's forecast
Sparsely triggered connections
Fast learning
Slow learning
Block Output
Output
Regressor
Block L
C
Dilated convolution
Memory
Block
Dilated convolution
Adapter
Convolution Filters
Block
EMA(V))
Block Input
Look-back window
Input
a) TCN backbone
b) Convolution block
c) aSNee-augmented filters
Figure I: An overview of FSNet. A standard TCN backbone (a) of L dilated convolution stacks (b). Each convolution filter FSNet is equipped with an adapter and associative memory to facilitate fast learning by monitoring the backbone's gradient EMA. Best viewed in colors.
framework, which allows for forgetting of past knowledge and does not have an explicit mechanism for fast learning (Huszár, 2017; Kirkpatrick et al., 2018). Moreover, such studies did not focus on deep neural networks and it is non-trivial extend to the setting of our study.
2.2 CONTINUAL LEARNING
Continual learning (Kirkpatrick et al., 2017) an emerging topic aiming to build intelligent agents that learn series of tasks sequentially, with only limited access to past experiences. A contin- ual learner must achieve good trade-off between maintaining the acquired knowledge of previous tasks and facilitating the learning of future tasks, which also known as the stability-plastiin dilemma (Grossberg, 1982; 2013). Due to its connections to humans' learning, several neuroscience frameworks have motivated the development of various continual learning algorithms. One popular framework is the complementary learning systems theory for dual learning system (McClelland al., 1995; Kumaran et al., 2016). Continual learning methods inspired from the CLS theory augments the slow, deep networks with the ability to quickly learn data streams, either via the experience replay mechanism (Lin, 1992: Riemer et al., 2019; Rolnick et al., 2019; Aljundi et al., 2019a; Buzzega et al., 2020) via explicit modeling of each of the fast and slow learning compo- nents (Pham et al., 2021a; Arani et al., 2021). Such methods have demonstrated promising results on controlled vision language benchmarks. In contrast, work addresses the online time series forecasting challenges by formulating them as continual learning problem.
PROPOSED FRAMEWORK
This section formulates the online time series forecasting as task-free online continual learning problem and details the proposed FSNet framework.
3.1 ONLINE TIME SERIES FORECASTING AS A CONTINUAL LEARNING PROBLEM
Our formulation is motivated by the locally stationary stochastic processes observation, where a time series can be split into sequence of stationary segments (Vogt, 2012: Dahlhaus, 2012: Das & Nason, 2016). Since the same underlying process generates samples from a stationary segment, we refer to forecasting each stationary segment as learning task for continual learning. We note that this formulation is general and encompasses existing learning paradigms. For example, splitting into only one segment indicates no concept drifts, and learning reduces to online learning in sta- tionary environments (Hazan, 2019). Online continual learning (Aljundi et al., 2019a) corresponds to the case of there are at least two segments. Moreover, we also do not assume that the points of task switch given to the model, which is common setting in many continual learning stud- ies (Kirkpatrick et al., 2017; Lopez-Paz & Ranzato, 2017). Manually obtaining such information in real-world data can be expensive because of the missing irregularly sampled data (Li & Mar-
3

---

Published as a conference paper at ICLR 2023
lin, 2020; Farnoosh al., 2021). Therefore, formulation corresponds to the online, task-free continual learning formulation (Aljundi et al., 2019a;b; Hu et al., 2020; Cai et al., 2021).
We now discuss the differences between our formulation with existing studies. First, most existing task-free continual learning frameworks (Aljundi ct al., 2019b; Pham et al., 2021a) are developed for image data, which vastly differs from time series. The input and label spaces of images different (continuous vs discrete) while time series' input and output share the same real-valued space. Additionally, the image's label changes significantly across tasks while time series data changes gradually over time with no clear boundary. Moreover, time series exhibits strong temporal information among consecutive samples, which does not exist in image data. Therefore, non- trivial to simply apply existing continual learning methods to time series and successful solutions requires carefully handling unique characteristies from time series data.
Second, time series evolves and old patterns may not reappear exactly in the future. Thus, we are not interested in remembering old patterns precisely but predicting how they will evolve. For example, we do not need to predict the electricity consumption over the last winter. But it is more important to predict the electricity consumption this winter, assuming that it is likely to have a similar pattern as the last one. Therefore, we do not need separate test set for evaluation, but training follows the online learning setting where a model is evaluated by its accumulated errors throughout learning.
3.2 FAST AND SLOW LEARNING NETWORKS (FSNET)
FSNet always leverages past knowledge to improve the learning in the future (Section 3.2.1), which akin facilitating forward transfer in continual learning (Lopez-Paz & Ranzato. 2017). Addi- tionally, FSNet remembers repeating events and continue to learn them when they reappear (Sec- tion 3.2.2, which is akin to preventing catastrophic forgetting (Kirkpatrick et al., 2017). We consider Temporal Convolutional Network (TCN) (Bai et al., 2018) as the backbone deep neural network to extract a time series feature representation due to the simple forward architecture and promising results (Yue et al., 2021). The backbone has L layer with parameters 0- {0.)- FSNet improves the TCN backbone with two complementary components: a per-layer adapter 4 and a per-layer associative memory M,. Thus, the total trainable parameters is w (0). Φ 1/21 and the total asso-
ciative memory is M- Hf We also use h; and h; to denote the original feature and adapter feature map of the l-yer Figure provides an illustration of FSNet.
3.2.1 FAST LEARNING MECHANISM
The key observation allowing for fast learning is to facilitate the learning of each intermediate layer via the following observation: the partial derivative Vo, characterizes the contribution of layer 0 to the forecasting loss f. Traditional training schemes simply move the parameters along this gradient direction, which results in ineffective online learning (Sahoo et al., 2018; Phuong & Lampert, 2019). Moreover, time series data exhibits strong temporal consistency across consecutive samples, which is not captured by existing training frameworks. Putting these observations together, argue that exponential moving average (EMA) of the partial derivative can provide meaningful information about the temporal smoothness in time series. Consequently, leveraging this knowledge can improve the learning of each layer, which in turn improves the whole network's 's performance.
To utilize the gradient EMA, we propose to treat it as a context to support fast learning via the ceature-wise transformation framework (Perez et al., 2018; Dumoulin et al., 2018: Pham et al., 2021b; Yin et al., 2021). Particularly, we propose to equip each layer with an adapter to map the layer's gradient EMA to a set of smaller, more compact transformation coefficients. These coeff- cients are applied on the corresponding layer's parameters and feature so that they can leverage the temporal consistency to learn better. We first define the EMA of the I-layer's partial derivative as:
9: 79: + (1 - 77)gi
where of denotes the gradient of the 1-th layer at time and ğ: denotes the EMA. The adapter takes gi as input and maps it to the adaptation coefficients uy Then, an adapter for the 1-th layer is a linear layer that maps the context 9i set of transformation coefficients u, [a Bt). In this work, we consider a two-stage transformations (Yin et al., 2021) which involve a weight and bias transformation coefficients oy and feature transformation coefficients Bi.
4

---

Published as a conference paper at ICLR 2023
The adaptation process for layer 0 is summarized as:
[ax, Bi] =u,, where un = (g Φi)
(2)
Weight adaptation: 0: -tile(a)) @ 0,, and
(3)
Feature adaptation: h4 =tile(BB)) hi, where h4 0: h4-1.
(4)
Here, h; is stack of I features maps with C' channels and length Z. hs is the adapted feature, 0 denotes the adapted weight, @ denotes the element-wise multiplication, and tile(a); denotes that the weight adaptor is applied per-channel on all filters via tile function that repeats a vector along the new axes. A naive implementation of Equation directly maps the model's gradient the adaptation coefficients and results in very high dimensional mapping. Therefore, we implement the chunking operation (Ha et al., 2016) to split the gradient into equal size chunks and then maps each chunk to an element of the adaptation coefficients. We denote this chunking operator as Q( Φz) and provide the detailed description in Appendix C.
3.2.2 REMEMBERING RECURRING EVENTS WITH AN ASSOCIATIVE MEMORY
In time series, old patterns may reappear and it is imperative to leverage our past actions to improve the learning outcomes. In FSNet, an adaptation to a pattern is represented by the coefficients u, which we argue be useful to learn repeating events. Specifically, u represents how we adapted to particular pattern in the past; thus, storing and retrieving the appropriate u may facilitate learn- ing the corresponding pattern when they reappear. Therefore, as the second key element in FSNet, we implement an associative memory to store the adaptation coefficients of repeating events en- countered during learning. In summary, beside the adapter, we equip each layer with an additional associative memory Mi E NR- xd where d denotes the dimension of uy and N denotes the number of elements, which we fix as = 32 by default.
Sparse Adapter-Memory Interactions Interacting with the memory at every step is expensive and susceptible to noises. Thus, we propose to trigger this interaction subject to a substantial representa- tion change. Interference between the current and past representations can be characterized in terms of a dot product between the gradients (Lopez-Paz end, together with the gradient EMA in Equation 2, deploy another gradient EMA g' with smaller coefficient 7' < y and measure their cosine similarity to trigger the memory interaction as:
Ranzato, 2017; Riemer et al., 2019). To this
91 g
Trigger if cos(ğ: g
(S)
19 9
where 7 > 0 Os hhper- parameter determining the significant degree of interference. Moreover, we want to set to relatively high value (e.g. 0.75) so that the memory only remembers significant changing patterns, which could be important and may reappear.
The Adapter-Memory Interacting Mechanism Since the current adaptation coefficients may not capture the whole event, which could span over few samples, perform the memory read and write operations using the adaptation coefficients's 's EMA (with coefficient 7') to fully capture the current pattern. The EMA of un is calculated in the same manner as Equation 1. When a memory interaction is triggered, the adapter queries and retrieves the most similar transformations in the past via an attention read operation, which is weighted sum over the memory items:
1. Attention calculation: r = (Mft
2. Top-k selection: ri (k) Trpp(rr);
3. Retrieval: ú = EK D1=1
(k) [1].
where (()(()) denotes the i-th element of ri (k) and Mi[i] denotes the i-th row of Mi. Since the memory could store conflicting patterns, we employ a sparse attention by retrieving the top-k most relevant memory items, which we fix k = 2. The retrieved adaptation coefficient characterizes how the model reacted to the current pattern in the past, which can improve learning at the present by combining with the current parameters u, Tuu + T u, where of r as in Equation 5. Then we perform a write operation to update the knowledge stored in Mi as:
the same value
Mi
Mi M + (1 T) & r(() and Mi max(1, M2
5

---

Published as a conference paper at ICLR 2023
where @ denotes the outer-produce operator, which allows us efficiently write the new knowledge to the most relevant locations indicated by (8) (Rae et al., 2016; Kaiser et al., 2017). The memory is then normalized to avoid its values exploding. We provide FSNet's pseudo code in Appendix C.2.
4 EXPERIMENT
Our experiments aim at investigating the following hypotheses: (i) FSNet facilitates faster adaptation to both new and recurring concepts compared to existing strategies with deep models; (ii) FSNet achieves faster and better convergence than other methods; and (iii) modeling the partial derivative is the key ingredients for fast adaptation. Due to space constraints, we provide the key information of the experimental setting in the main paper and provide full details, including memory analyses, additional visualizations and results in the Appendix.
4.1 EXPERIMENTAL SETTINGS
Datasets We explore a wide range of time series forecasting datasets. ETT (Zhou et al., 2021) records the target value of "oil temperature" and 6 power load features over a period of two years. We consider the ETTh2 and ETTml benchmarks where the observations are recorded hourly and in 15-minutes intervals respectively. ECL (Electricty Consuming Load)² collects the electricity consumption of 321 clients from 2012 to 2014. Traffic records the road occupancy rates at San Francisco Bay area freeways. Weather records 11 climate features from nearly 1,600 locations in the U.S in an hour intervals from 2010 to 2013.
We also construct two synthetic datasets to explicitly test the model's ability to deal with new and recurring concept drifts. We synthesize task by sampling 1, 000 samples from a first-order autore- gressive process with coefficient yo: (I), where different tasks correspond to different ip values. The first synthetic data, S-Abrupt (S-A), contains abrupt, and recurrent concepts where the sam- ples abruptly switch from one AR process to another by the following order: 1 A ARo.o(1), i (I), AR). The second data, S-Gradual (S-G) contains gradual, in- cremental shifts, where the shift starts at the last 20% of each task. In this scenario, the last 20% samples of a task is an averaged of two AR process with the order as above. Note that we randomly chose the values of ip so that these datasets do not give unfair ad advantages to any methods.
Baselines We consider suite of baselines from continual learning, time series forecasting, and online learning. First, the OnlineTCN strategy that simply trains continuously (Zinkevich, 2003). Second, we consider the Experience Replay (ER) (Lin, 1992; Chaudhry et al., 2019) strategy where a buffer is employed to store previous samples and interleave them during the learning of newer ones. We also include three recent advanced variants of ER. First, TFCL (Aljundi et al., 2019b) introduces a mask-boundarier detection mechanism and knowledge consolidation strategy by regularizing the networks' outputs (Aljundi et al., 2018). Second, MIR (Aljundi et al., 2019a) replace the random sampling in ER by selecting samples that cause the most forgetting. Lastly, DER++ (Buzzega al., 2020) augments the standard ER with a knowledge distillation strategy (Hinton et al., 2015). We emphasize that ER and and its variants are strong baselines in the online setting since they enjoy the benefits of training on mini-batches, which greatly reduce noises from singe samples and offer faster, better convergence (Bottou al., 1998). While the aforementioned baselines use a TCN backbone, we also include Informer (Zhou et al., 2021), recent time series forecasting method based on the transformer architecture (Vaswani et al., 2017). We remind the readers that online time series forecasting have not been widely studied with deep models, therefore, we include general strategies from related fields that we inspired from. Such baselines are competitive and yet general enough to extend to our problem.
Implementation Details We split the data into warm-up and online training phases by the ratio of 25:75. We follow the optimization details in (Zhou et al., 2021) by optimizing the &2 (MSE) loss with the AdamW optimizer (Loshchilov & Hutter, 2017). In the warm-up phase, we calculate the statistics to normalize online training samples, perform
cross validatii,,,,,,,, and pre-
thttps://pems.dot.co ca....a.goo ca.to// www.aaaa..a...aaa.a.a
"https://www.ner noaa gow wateiaaaaaaccaaaaaaa lcc.ccalllllllcccccoo

---

Published as a conference paper at ICLR 2023
Table 1: Final cumulative MSE and MAE of different methods, 4_" indicates the model did not converge. S-A: S-Abrupt, S-G: S-Gradual. Best results are in bold.
Method
FSNet
DER++
MIR
ER
TFCL
OnlineTCN
Informer
MSE MAE MSE MAE MSE MAE MSE
MAE
MSE
MAE
MSE
MAE
MSE MAE
WTH Traffic ECL ET1ml ETTh2
0.466 0.368 0.508 0.375 0.486 0.410 0.508 0.376
0.557
0.472 0.502 0.436 7.571 0.850
0.687 0.467 0.828 0,540 0.812 0,541 0.808 0.543
0.846
0.548
0.830
0.547 4.629 0.668
48
0.846 0.515 1.157 0.577 1.103 0.565 1.136 0.571
1,208
0.592
1.183
0.589 5.692 0.752
0.105
0.188 0.098 0.183 0.099 0,184 0.099 0.184 0.099
0.185
0.109
0.204 0.456 0.392
24
0.136
0.248 0.239 0.329 0.242 0.335 0.259 0.346 0.239
0.335
0.272
0.361 0,827 0,551
48
0.129
0.245 0.264 0.355 0.271 0.362 0.288 0.372
0.242
0.344
0.280
0.371 0.853 0.533
3.143
0,472 2.657 0,421
2.575
0.504
2.579 0,506
2.732
0,524
3.309
0.635
24
6.051
0.997 8.996 1.035 9.265 1,066 9.327 1.057 12.094
1.256
11.339 1.196
48
7.034
1.061 9.009
1.048
9.411 1.079 9.685
1.074
12.110
1.303
11.534 1.235
0.288
0.253
0.289
0.248
0.290
0.251
0.291
0.252
0.323
0.273
0.315
0.283
0.795 0.507
0.362
0.288 0.387 0.295 0.391 0.302 0.391 0.302
0.553
0.383
0.452
0.363
1.267 0.750
0.162
0.216 0.174 0.235 0.179 0.244 0.180 0.244 0.177 0.240
0.206
0.276
0,426 0.458
0.188
0.276 0.287 0.351 0.291 0.355 0.293 0.356
0.301 0.363
0.308
0.367 0.370 0.417
48
0.223
0.301 0.294 0.359 0.297 0.361 0.297 0.363
0.323
0,382
0,302
0.362 0.367 0.419
1.391
0.929
2.334
1.181
2.482
1.213
2.372
1.157
2.321
1.144
2.668
1.216
3.690 1.410
24
1.299 0.904 3.598
1.439
3.662
1,450
3.375
1,360
3,415
1.366
3.904
1.491
3.657 1.426
G
1.760
1.038
2.335
1.181
2.482
1.213
2.476
1,212
2.428
1.199
2.927
1.304
4.024
1.501
24
1.299
0.904
3.598
1.439
3.662
1,450
3.667
1.489
3.829
1.479
3.904
1.491
3.657
1.426
FSNet
DER++
MIR
ER
TFCL
OnlineTCN
MSE
MSE
0.2
MSE
20-
0.5
0.1
2000
4000
5000
8000
10000
0
10000
20000
30000
40000
5000
10000 #instances
15000
20000
#instances
#instances
(a) ETTh2
(b) ETTml
(c) ECL
0.5
0.5
MSE
0.4
MSE
0.3
0.3
2500
5000 7500 10000
12500
5000 19000 15000
20000 25000 1000 2000 3000 4000 5000 5000
#instances
#instances
# instances
(d) Traffic
WTH
(0 S-A
Figure 2: Evolution of the cumulative MSE loss during training with forecasting window H/ = 24.
In Figure 2.f, each color region denotes a data generating distribution. Best viewed in color.
train the models for the few methods. During online learning, both the epoch and batch size are set one to follow the online learning setting. These configurations are applied all baselines. We implement a fair comparison by making sure that all baselines use the same total memory budget as our FSNet, which includes three-times the network sizes: one working model and two EMA of its gradient. Thus, we set the buffer size of ER, MIR, and DER++ to meet this budget while increasing the backbone size of the remaining baselines. Lastly, for all benchmarks, we set the look-back window length to be 60 and vary the forecast horizon as H E (1,24,455 24.
4.2 ONLINE FORECASTING RESULTS
Cumulative Performance Table I reports the cumulative mean- squared errors (MSE) and mean- absolute errors (MAE) of deep models (TCN and Informer) at the end of training. The reported num-
7

---

Published as a conference paper at ICLR 2023
OnlineTCN
DER++
FSNet
Ground truth
0-
value
value
3000
3005
3010
3015
3020
3100
3105
3110
3115
3120
3200
3205
3210
3215
3220
(a) Prediction from t 3000
(b) Prediction from t = 3100
(c) Prediction from t = 3200
5.0
2-
value
2.5
0.0
-5
2.55
4000
4005
4010
4015
4020
4100
4105
4110
4115
4120
4200
4205
4210
4215
4220
(d) Prediction from t = 4000
(c) Prediction from = 4100
(1) Prediction from t 4200
value
value
5000
5005
5010
5015
5020
5100
5105
5110
s11s
5120
5200
5205
5210
5215
5220
(g) Prediction from f 5000
(h) Prediction from = 5100
(i) Prediction from t 5200
Figure 3: Visualization of the model's prediction throughout the online learning process. We focus on a short horizon of 200 time steps after a concept drift, which is critical for fast learning.
bers averaged over five runs, and we provide the standard deviations in Table Appendix E. We observe that ER and its variants (MIR, DER++) are strong competitors and can significantly improve over the simple TCN strategies. However, such methods still cannot work well under mul- tiple task switches (S (S-Abrupt). Moreover, no clear task boundaries (S (S-Gradual) presents an even more challenging problem and increases most models' errors. In addition, previous work has ob- served that TCN can outperform Informer in the standard time series forecasting (Woo et al., 2022). Here we also observe similar results that Informer does not perform well in the online setting, and is outperformed by other baselines. On the other hand, our FSNet shows promising results on all datasets and outperforms most competing baselines across different forecasting horizons. Moreover, the significant improvements the synthetic datasets indicate FSNet's ability to quickly adapt to the non-stationary environment and recall previous knowledge, even without clear task boundaries.
Convergent behaviors of Different Learning Strategies Figure 2 reports the convergent behav- iors on the considered methods. We omit the S-Gradual dataset for spaces because we observe the same behavior S-Abrupt. Interestingly, we observe that concept drifts are likely happened in most datasets because of the loss curves' sharp peaks. Moreover, such drifts appear at the early stage of learning, mostly in the first 40% of data, while the remaining half of data are quite stationary. This result shows that the traditional batch training is often too optimistic by only testing the model on the last data segment. The results clearly show the benefits of ER by offering faster convergence during learning compared to Online However, storing the original data may not be applicable in many applications. On S-Abrupt, most baselines demonstrate the inability to quickly recover from con- cept drifts, indicated by the increasing trend in the error curves. We also observe promising results of FSNet on most datasets, with significant improvements over the baselines on the ETT, WTH, and S- S-Abrupt datasets. The remaining datasets are more challenging with missing values (Li al., 2019) and large magnitude varying within and across dimensions, which may require calculating better data normalization statistics. While FSNet achieved encouraging results, handling the above challenges can further improve its performance. Overall, the results shed light on the challenges of online time series forecasting and demonstrate promising results of FSNet.
Visualization We explore the model's prediction quality on the S-Abrupt since it is a univariate time series. The remaining multivariate real-world datasets are more challenging visualize. Par-
8

---

Published as a conference paper at ICLR 2023
Table 2: Final comulative MSE and MAE of different FSNet variants. Best results are in bold.
FSNet
Variant
Method
M=128 (large)
M=32 (original)
No Memory
Naive
Data
H
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
24
0.616
0.456
0.687
0.467
0.689
0.468
0.860
0.555
ETTh2
48
0.846
0.513
0.846
0.515
0.924
0.526
0.973
0.570
1 0.285
0.251
0.288
0.253
0.294
0.252
0.330
0.282
Traffic
24
0.358
0.285
0.362
0.288
0.355
0.284
0.463
0.362
1 1.388
0.928
1,391
0.929
1.734
1.024
3.318
1.416
S-A
24
1.213
0,870
1.299
0.904
1.390
0.933
3.727
1.467
1 1.758
1,040
1.760
1.038
1.734
1.024
3.318
1.414
S-G
24
1.293
0.902
1.299
0.904
1.415
0.940
3.748
1.478
ticularly, we are interested in the models' behaviour when old task's reappear. Therefore, in Figure 3, we plot the model's forecasting at various time points after = 3000. We can see the difficulties of training deep neural networks online in that the model struggles to learn at the early stages, where it only observed a few samples. We focus on the early stages of task switches (e.g. the first 200 samples), which requires the model to quickly adapt to the distribution shifts. With the limited samples per task and the presence of multiple concept drifts, the standard online opti- mization collapsed to naive solution of predicting random noises around zero. However, FSNet can successfully capture the time series' patterns and provide better forecasts as learning progresses. Overall, we can clearly see FSNet can provide better quality forecasts compared to other baselines.
4.3 ABLATION STUDIES OF FSNET'S DESIGN
This experiment analyzes the contribution of each FSNet's component. First, we explore the benefits of using the associative memory (Section 3.2.2) by constructing a No Memory variant that only uses an adapter, without the memory. Second, we further remove the adapter. which results in the Naive variant that directly trains the adaptation coefficients jointly with the backhone. The Naive variant demonstrates the benefits of monitoring the layer's 's gradients, our key idea for fast adaptation (Section 3.2.1). Lastly, we explore FSNet's scalability by increasing the associative memory size from 32 items (original) to a larger scale of 128 items.
We report the results in Table 2. We first observe that FSNet achieves similar results with the No Memory variant on the Traffic and S-Gradual datasets. One possible reason is the insignificant repre- sentation interference in the Traffic dataset and the slowly changing representations in the S- Graduul dataset. In such cases, the representation changes can be casily captured by the adapter alone and may not trigger the memory interactions. In contrast, on ETTh2 and S-Abrupt, which may have sudden drifts, we clearly observe the benefits of storing and recalling the model's past action to fa- cilitate learning of repeating events. Second, the Naive variant does not achieve satisfactory results, indicating the benefits of modeling the temporal smoothness in time series via the use of gradient EMA. Lastly, the large memory variant of FSNet provides improvements most cases, indicating FSNet's 'ss scalability with more budget. Overall, these results demonstrated the complementary of each FSNet's components to deal with different types of concept drift in time series.
CONCLUSION
We have investigated the potentials and limitations of training deep neural networks for online time series forecasting in non-stationary environments, where they lack the capability to adapt new or recurring patterns quickly. We then propose Fast and Slow learning Networks (FSNet) by extending the CLS theory for continual learning to online time series forecasting. FSNet augments a neural network backbone with two key components: (i) adapter for fast learning; and (ii) an associative memory to handle recurrent patterns. Moreover, the adapter sparsely interacts with its memory to store, update, and retrieve important recurring patterns to facilitate learning of such events in the future. Extensive experiments demonstrate the FSNet's capability to deal with various types of concept drifts to achieve promising results in both real-world and synthetic time series data.

---

Published as a conference paper at ICLR 2023
ETHIC STATEMENT
In this work, we used the publicly available datasets for experiments. We did not collect human or animal data during this study. Due to the abstract nature of this work, method does not raise concerns regarding social/gender biases or privacy.
REFERENCES
Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In Proceedings of the European Confer- ence on Computer Vision (ECCV), pp. 139-154, 2018.
Rahaf Aljundi, Eugene Belilovsky, Tinne Tuytelaars, Laurent Charlin, Massimo Caccia, Min Lin, and Lucas Page-Caccia, Online continual learning with maximal interfered retrieval. Advances in Neural Information Processing Systems, 32:1 12:-1884-1188 18660, 2019a.
Rahaf Aljundi, Klaas Kelchtermans, and Tinne Tuytelaars. Task-free continual learning. In Pro- ceedings of the IEEF/CVF Conference on Computer Vision and Patlern Recognition, pp. 11254 11263, 2019b.
Oren Anava, Elad Hazan, Shie Mannor, and Ohad Shamir. Online learning for time series prediction. In Conference on learning theory, pp. 172-184. PMLR, 2013.
Elahe Arani, Fahad Sarfraz, and Bahram Zonooz. Learning fast, learning slow: A general contin- ual learning method based on complementary learning system. In International Conference on Learning Representations, 2021.
Sergul Aydore, Tianhao Zhu, and Dean P Foster. Dynamic local regret for non-convex online fore- casting. Advances in Neural Information Processing Systems, 32:7982-7991, 2019.
Shaojie Bai, J Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprini arXiv: 1803.0121, 2018.
Aadyot Bhatnagar, Paul Kassianik, Chenghao Liu, Tian Lan, Wenzhuo Yang. Rowan Cassius, Doyen Sahoo, Devansh Arpit, Sri Subramanian, Gerald Woo, et al. Merlion: A machine learning library for time series. arXiv preprint arXiv:2109.0922 2021.
Léon Bottou et al. Online learning and stochastic approximations. On-line learning neural networks, 17(9):142, 1998.
George EP Box, Gwilym M Jenkins, Gregory C Reinsel, and Greta M Ljung. Time series analysis: forecasting and control. John Wiley & Sons, 2015.
Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: a strong, simple baseline. In 34th Conference on Neural Information Processing Systems (NeurIPS 2020), 2020.
Zhipeng Cai, Ozan Sener, and Vladlen Koltun. Online continual learning with natural distribu- tion shifts: An empirical study with visual data. In Procedingss of the IEEF/CVF International Conference on Computer Vision, pp. 8281-8290, 2021.
Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet K Dokania, Philip HS Torr, and Marc' Aurelio Ranzato. On tiny episodic memories in continual learning. arXiv preprint arXiv: 10486, 2019.
Rainer Dahlhaus. Locally stationary processes. In Handbook of statistics, volume 30, pp. 351-413.
Elsevier, 2012.
Souray Das and Guy P Nason. Measuring the degree of non-stationarity of a time series. Stat, 5(1): 295-305, 2016.
Vincent Dumoulin, Ethan Perez, Nathan Schucher. Florian Strub, Harm de Vries, Aaron Courville, and Yoshua Bengio. Feature-wise transformations. Distill, 2018. doi: 10,2391 5/1istill.00011. https://distill.ub//22 wttp:///ww///..aaaaaaeeeetttmmmm
10

---

Published as a conference paper at ICLR 2023
Amirreza Farnoosh, Bahar Azari, and Sarah Ostadabbas. Deep switching regreseeressive factoriza- tion: Application to time series forecasting. Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 7394-7403, 2021.
João Gama, Indrè Zliobaité, Albert Bifet, Mykola Pechenizkiy, and Abdelhamid Bouchachia. A survey concept drift adaptation. ACM computing surveys (CSUR), 46(4):1-37, 2014.
Stephen Grossberg. How does brain build cognitive code? Studies of mind and brain, pp. 1-52, 1982.
Stephen Grossberg, Adaptive resonance theory: How brain learns consciously attend, learn, and recognize a changing world. Neural networks, 37:1-47 -4, 2013.
San Gultekin and John Paisley, Online forecasting matrix factorization. IEEE Transactions on Signal Processing, 67(5):1223-12266 2018.
Vibhor Gupta, Jyoti Narwariya, Pankaj Malhotra, Lovekesh Vig, and Gautam Shroff. Continual learning for multivariate time series tasks with variable input dimensions. In 2021 IEEE Interna- tional Conference on Data Mining (ICDM), pp. 161-170. IEEE, 2021.
David Ha, Andrew Dai, and Quoe V Le. Hypernetworks. arXiv preprint 16Xiv: 2016.
Elad Hazan. Introduction to online convex optimization. arXiv preprint arXiv:1909.05207, 2019.
Yujiang He and Bernhard Sick. Clear: An adaptive continual learning framework for regression tasks. Al Perspectives, 3(1):1-16, 2021.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in neural network. arXiv preprint arXiv:1503.0255: 2015.
Charles Holt. Forecasting seasonals and trends by exponentially weighted moving averages. International journal of forecasting, 20(1):5-10, 2004.
Hexiang Hu, Ozan Sener, Fei Sha, and Vladlen Koltun. Drinking from a firehose: Continual learning with web-scale natural language. arXiv preprint arXiv:2007.09335, 2020.
Ferenc Huszár. On quadratic penalties in elastic weight consolidation. 1rXiv:1712.03847, 2017.
arXiv preprint
Rob J Hyndman and George Athanasopoulos. Forecasting: principles and practice. ОTexts, 2018.
Herbert Jaeger. Using conceptors to manage neural long-term memories for temporal patterns. The Journal of Machine Learning Research, 18(1):387-429 2017.
Eukasz Kaiser, Ofir Nachum, Aurko Roy, and Samy Bengio. Learning to remember rare events. 2017.
arXiv preprint
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Bann Barwinska. et al. Overcom- ing catastrophic forgetting neural networks. Proceedings of the national academy of sciences, 114(13):3521-3522 2017.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-BBrwinice et al. Reply to huszár: The elastic weight consolidation penalty is empirically valid. Proceedings of the National Academy of Sciences, (1)(11)
2018.
Dharshan Kumaran, Demis Hassabis, and James L McClelland. What learning systems do intelligent agents need? complementary learning systems theory updated. Trends in cognitive sciences, 20 (7):512-534, 2016.
Richard Kurle, Botond Cseke, Alexej Klushyn, Patrick Van Der Smagt, and Stephan Günnemann. Continual learning with bayesian neural networks for non-stationary data. In International Con- ference Learning Representations, 2019.

---

Published as a conference paper at ICLR 2023
Shiyang Li, Xiaoyong Jin, Yao Xuan, Xiyou Zhou, Wenhu Chen, Yu-Xiang Wang, and Xifeng Yan. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting, Advances in Neural Information Processing Systems, 32:5243-5253 2019.
Steven Cheng-Xiin Xian and Benjamin Marlin. Learning from ireeularly-samplee time series: A missing data perspective. In International Conference Machine Learning, pp. 5937-5946. PMLR, 2020.
Long-Ji Lin. Self-improving reactive agents based reinforcement learning, planning and teaching. Machine learning, 8(3-4))293-321, 1992.
Chenghao Liu, Steven CH Hoi, Peilin Zhao, and Jianling Sun. Online arima algorithms for time series prediction. In Thirtieth AAAI conference on artificial intelligence, 2016.
David Lopez-Paz and Mare' Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30:6467-6476, 2017.
Iya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint
171105 2017.
James L McClelland, Bruce L McNaughton, and Randall C O'RReilly Why there complementary learning systems the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychological review, 102(3):419, 1995.
Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pp. 109-165. Elsevier, 1989.
Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. Feature visualization. Distill, 2(11):e7, 2017.
Boris Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. N-beats: Neural basis expansion analysis for interpretable time series forecasting. arXiv preprint arXiv:1905. 10437, 2019.
Ethan Perez, Florian Strub, Harm De Vries, Vincent Dumoulin, and Aaron Courville. Film: Visual reasoning with a general conditioning layer. Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
Fotios Petropoulos, Daniele Apiletti, Vassilios Assimakopoulos, Mohamed Zied Babai, Devon Barrow, Souhaib Ben Taieb, Christoph Bergmeir, Ricardo J Bessa, Jakub Bijak, John E Boylan, et al. Forecasting: theory and practice. arXiv preprint arXiv:2012.03854 2020.
Quang Pham, Chenghao Liu, and Steven Hoi. Dualnet: Continual learning, fast and slow. Advances in Neural Information Processing Systems, 34, 2021a.
Quang Pham, Chenghao Liu, Doyen Sahoo, and Steven CH Hoi. Contextual transformation net- works for online continual learning. International Conference on Learning Representations (ICLR), 2021b.
Mary Phuong and Christoph Lampert. Distillation-based training for multi-exit architectures. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 1355-1364, 2019.
Jack W Rae, Jonathan J Hunt, Tim Harley, Ivo Danihelka, Andrew Senior, Greg Wayne, Alex Graves, and Timothy Lillicrap. Scaling eeemory-augmentne neural networks with sparse reads and writes. In Proceedings of the 30th International Conference on Neural Information Process- ing Systems, pp. 3628-36666 0636,,6662200 2016.
Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. Learning to learn without forgetting by maximizing transfer and minimizing interfer- ence. International Conference Learning Representations (ICLR), 2019.
12

---

Published as a conference paper at ICLR 2023
David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Experience replay for continual learning. In Advances Neural Information Processing Systems, pp. 348- 358. 2019.
Yulia Rubanova, Ricky TQ Chen, and David Duvenaud. Latent odes for seggulaaly--aapplee time series. In Proceedings of the 33rd International Conference on Neural Information Processing Systems, pp. 5320-5330, 2019.
Doyen Sahoo, Quang Pham, Jing Lu, and Steven CH Hoi. Online deep learning: learning deep neural networks the fly. In Proceedings of the 27th International Joint Conference Artificial Intelligence, pp. 2660-2666, 2018.
David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. Deepar: Probabilistic fore- casting with autoregressive recurrent networks. International Journal of Forecasting, 36(3):1181- 1191, 2020.
Alexander Smola, Vishy Vishwanathan, and Eleazar Eskin. Laplace propagation. In NIPS, pp- 441-448. Citeseer, 2003.
Oskar Triebe, Hansika Hewamalage, Polina Pilyugina, Nikolay Laptev, Christoph Bergmeir, and Ram Rajagopal. Neuralprophet: Explainable forecasting at scale. arXiv preprint arXiv:2111.15397, 2021.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural informa- tion processing systems, 30, 2017.
Michael Vogt. Nonparametric regression for locally stationary time series. The Annals of Statistics, 40(5):2601-285 2012.
Gerald Woo, Chenghao Liu, Doyen Sahoo, Akshat Kumar, and Steven Hoi. Cost: Contrastive learning of disentangled seasonal-trend representations for time series forecasting. arXiv preprint arXiv:2202.01575. 2022.
Jiehui Xu, Jianmin Wang. Mingsheng Long, et al. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. Advances in Neural Information Processing Systems, 34, 2021.
Haiyan Yin, Ping Li, et al. Mitigating forgetting in online continual learning with neuron calibration. Advances in Neural Information Processing Systems, 34, 2021.
Zhihan Yue, Yujing Wang, Juanyong Duan, Tianmeng Yang, Congrui Huang, Yunhai Tong, and Bixiong Xu. Ts2vec: Towards universal representation of time series. arXiv preprint
2rXiv:2106.10466. 2021.
Haoyi Zhou, Shanghang Zhang. Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting, In Proceedings of AAAI, 2021.
Martin Zinkevich. Online convex programming and generalized infinitesimal gradient ascent. In Proceedings of the 20th international conference machine learning (icml-03), pp. 928-936, 2003.
A ORGANIZATION
This Appendix is organized as follow. First, we provide more detailed discussion on time series forecasting and continual learning in Appendix B. Then, in Appendix C, we detail the implementa- tion of FSNet. Appendix D discuss the experiments' details, including the synthetic data generation, the loss function, and baselines' descriptions. Lastly, Appendix E provides additional experiment results, including standard deviations, complexity analyses, additional experiments, and visualiza- tions.
13

---

Published as a conference paper at ICLR 2023
B EXTENDED RELATED WORK
This section is an extended version of Section where we discuss in more details the existing time series forecasting and continual learning studies.
B. TIME SERIES FORECASTING
Time series forecasting is an important problem and has been extensive studied in the literature. Traditional methods such as ARMA, ARIMA (Box et al., 2015), and the Wolt-Winters seasonal methods (Holt, 2004) enjoy theoretical guarantees. However, they lack the model capacity to model more complex interactions of real-world data. As a result, they cannot handle the complex interac- tions among different dimensions of time series, and often achieve inferior performances compared to deep neural networks on multivariate time series data (Zhou et al., 2021; Oreshkin et al., 2019).
Recently, learning good time series representation has shown promising results, and deep learning models have surpassed such traditional methods on large scale benchmarks (Rubanova et al., 2019; Zhou et al., 2021; Oreshkin et al., 2019). Early deep learning based approaches built upon standard MLP models (Oreshkin et al., 2019) or recurrent networks such as LSTMs (Salinas et al., 2020)- Recently, temporal convolution (Yue et al., 2021) and transformer (Li et al., 2019; Xu et al., 2021) networks have shown promising results, achieving promising on a wide range of real-world time series. However, such methods assume a static world and information to forecast the future fully provided in the look-back window. As result, they lack the ability to remember events beyond the look-bace back window and adapt to the changing environments on the fly. In contrast, our FSNet framework addresses these limitation by a novel adapter and memory components.
B.2 CONTINUAL LEARNING
Human learning has inspired the design of several strategies to enable continual learning in neural networks. One successful framework is the Complementary Learning Systems theory (McClelland et al., 1995; Kumaran et al., 2016) which decomposes learning into two processes of learning fast (hippocampus) and slow (neocortex). While the hippocampus current information, possibly with the help of experience replay, the neocortex does not change as fast and only accumulate general knowledge. The two learning systems interacts via knowledge consolidation process, where recent experiences in the hippocampus are transferred to the neocortex form more general representation.
quickly change and capture the
addition, the hippocampus also queries information from the neocortex to facilitate the learning of new and recurring events. The CLS theory serves motivation for several designs in continual learning such as experience replay (Chaudhry et al., 2019), dual learning architectures (Pham et al., 202 la; Arani et al., 2021). In this work, we extend the fast and slow learning framework of the CLS theory to the online time series forecasting problem.
B.3 COMPARISON WITH EXISTING CONTINUAL LEARNING FOR TIME SERIES FORMULATIONS
This section provides a more comprehensive comparison between our formulation of online time series forecasting and existing studies in (He Sick, 2021; Jaeger, 2017: Gupta et al., 2021; Kurle et al., 2019).
We first summarize the scope of our study. We mainly concern the online time series forecasting problem (Liu et al., 2016) and focus on addressing the challenge of fast adaptation to distribution shifts in this scenario. Particularly, when such distribution shifts happen, the model is required to take less training samples to achieve low errors, either by exploiting its representation capabilities or reusing the past knowledge. We focus on the class of deep feedforward neural network, par- ticularly TCN, thanks to its powerful representation capabilities and ubiquitous in sequential data applications (Bai et al., 2018).
CLeaR (He & Sick, 2021) also attempted to model time series forecasting as continual learning problem. However, CLeaR focuses on accumulating knowledge over a data stream without forget- ting and does not concern about fast adaptation under distribution shifts. Particularly, CLeaR's ss online training involves periodically calibrating the pre-trained model on new out-of-distribun samples using continual learning strategy. Moreover, CLeaR only calibrates the model when
14

---

Published as a conference paper at ICLR 2023
buffer of novel samples are filled. As result, when distribution shifts, CLeaR could suffer from arbitrary high errors until accumulates enough samples for calibrating. Therefore, CLeaR is not applicable to the online time series forecasting problem considered in our study.
GR-IG (Gupta et al., 2021) also formulates time series forecasting as a continual learning problem. However, they address the challenging of variable input dimensions through time, which could arise from the introduction of new sensors, or sensor failures. Therefore, by motivating from continual learning, GR-IG can facilitate the learning of new tasks (sensors) for better forecasting. However, GR-IG does not consider shifts in the observed distributions and focus on learning new distributions that appear over time. Consequently, GR-IG is also not a direct comparison our method.
Lastly, we also note Conceptors (Jaeger, 2017) as a potential approach to address the time series forecasting problem. Conceptors are class of neural memory that supports storing and retrieving patterns learned by recurrent network. In this work, we choose to use the associative memory maintain long-term patterns, which is more common for deep foed-forward architectures used in our work. We believe that with necessary adaptation, it is possible integrate Conceptors as the memory mechanism FSNet, which beyond the scope of this work.
C FSNET DETAILS
C.I CHUNKING OPERATION
In this section, we describe the chunking adapter's chunking operation to efficiently compute the adaptation coefficients. For convenient, we denote vec(-) as a vectorizing operation that flattens a tensor into vector; We split(e, B) to denote splitting vector into B segments, each has size dim(() e)/ B. An adapter maps its backbone's 's layer EMA gradient to an adaptation coefficient Rd via the chunking process as:
9i vec(gi)
[b.. be
ba] reshappo@gr.d
[11 hg.................. hal ( by W b
bul
[ui U2 U2,..., udl (w h1 w (2) h2
(2) w
hal.
+
Where we denote w-(1) (1)
& and w as the first and second weight matrix of the adapter. In summary, the chunking process can be summarized by the following steps: (1) flatten the gradient EMA into a vector; (2) split the gradient vector into d chunks; (3) map each chunk to a hidden representation; and (4) map each hidden representation coordinate of the target adaptation parameter u.
C.2 FSNET PSEUDO ALORITHM
Algorithm provides the psedo-code for our FSNet.
15

---

Published as a conference paper at ICLR 2023
Algorithm Fast and Slow learning Networks (FSNet)
Two EMA coefficients 7 < 7, memory interaction threshold т Init: backbone 0, adapter Φ, associative memory M, regressor R. trigger False for t < to T do
Require:
Receive the t- look-bace window æt.
hg = 2;
for j
I. do
I/ Forward computation over T. layers // Initial adaptation parameter
0x Bi] = ut, where uy = ((g;;
if trigger True then
úi t R
Mi Writc(Mr úi)
Memory read and write are defined in Section 3.2.2 // Weighted sum the current and past adaptation parameters
ui - rui (1 7)
0, lile((n)) @ 0:
// Weight adaptation Feature adaptation
h: = tile( @ hi, where h; = 02 h4-1-
11
12
Forecast Dt Rhy
13
Receive the ground-truth Calculate the forecast loss and backpropagate
14
Update the regressor R via SGD for j < to do
16
// Backward to update the model and EMA
17
Update the EMA gl.gi'. u Update Φ1, 0₁ via SGD
18
19
if (g) 91) < - then
trigger <- True
D EXPERIMENT DETAILS
D.1 SYNTHETIC DATA
We use the following first-order auto-regressive process model (R) defined as
X, = Ct
where 6 are random noises and X,-1 are randomly generated. The S-Abrupt data is described by the following equation:
A Ro.1
if 1 < / < 1000
ARo.4
if 1000 < S 1999
A ARo.6
if 2000 < S 2999
X,
(8)
A
if 3000 < < 3999
A ARo.4
if 4000
< < 4999
ARa.6
if 5000 < 5999.
The S-Gradual data is described as
ARo1
if 1 < < 800
0.5 (ARo. A.o.4)
if 800 < 1000
ARo.
1000 < < 1600
0.5 x (ARo.4 + AAR..))
if 1600 < t S 1800
ARo.
if 1800 t < 2400
X,
0.5 (ARo.e + +ARo.)
if 2400 t 2600
AR
if 2600 < 3200
0.5 x (ARo.1 + A.R..)
if 3200 < t 3400
ARo4
if 3400 t 4000
0.5 (ARo.4 A.o.s
if 4000 S 4200
ARo.
if 4200 < S 5000
16

---

Published as a conference paper at ICLR 2023
D.1.1 BASELINE DETAILS
Summary
We provide brief summary of the baselines used in your experiments
Informer (Zhou et al., 2021): transformer-based model for time-series forecasting.
Online uses a standard TCN backbone (Woo et al., 2022) with 10 hidden layers, each of which has two stacks of residual convolution filters.
ER (Chaudhry et al., 2019) augments the OnlineTCN baseline with an episodic memory to store previous samples, which are then interleaved when learning the newer ones.
MIR (Aljundi et al., 2019a) replaces the random sampling strategy ER with its MIR sampling by selecting samples in the memory that cause the highest forgetting and perform ER on these samples.
DER++ (Buzzega et al., 2020) augments the standard ER (Chaudhry et al., 2019) with a & knowledge distillation loss on the previous logits.
TFCL (Aljundi et al., 2019b) is method for online, task-free continual learing. TFCL starts with as ER procedure and also includes a MAS-styled (Aljundi al., 2018) regu- larization that is adapted for the task-free setting.
All ER-based strategies use a reservoir sampling buffer. We also tried with a Ring buffer and did not observe any significant differences.
Loss function All methods in our experiments optimize the & loss function defined as follows. Let æ and y E I be the look-back and ground-truth forecast windows, and ý be the model's prediction of the true forecast windows. The & loss is defined as:
1 H
E(gr, ((Jo(ar))
l
vill
(10)
Experience Replay baselines provide the training details of the ER and DER++ baselines in this section. These baselines deploy an reservoir sampling buffer of 500 samples to store the observed samples (each sample is pair of look-back and forecast window).
Let M be the episodic memory storing previous samples, B, be mini-batch of samples sampled from M. ER minimizes the following loss function:
ER (f y:) AER
Σ (()(
(11)
(o.y)).,
where ((.,-) denotes the MSE loss and λER is the trade-off parameter of current and past examples. DER++ further improves ER by adding a distillation loss (Hinton et al., 2015). For this purpose, DER++ also stores the model's forecast into the memory and minimizes the following loss:
c DER++ = ((fw v.) )+ + ABR.
Σ (K.
e(fw(x)) v) + ADER+ Σ
((f(z))))) 0). (12)
(w.g)
D.2 HYPER-PARAMETERS SETTINGS
We cross-validate the spper-parameters on the ETTh2 dataset and use for the remaining ones. Particularly, we use the following configuration:
Adapter's 's EMA coefficient = 0.9,
Gradient EMA for triggering the memory interaction 7' = 0.3
Memory triggering threshold - 0.75
We found that this hyper-paramente configuration matches the motivation in the development of FSNet. In particular, the adapter's ' EMA coefficient y = 0.9 can capture medium-range information to facilitate the current learning. Second, the gradient EMA for triggering the memory interaction
17

---

Published as a conference paper at ICLR 2023
Table 3: Standard deviations of the metrics in Table 1. "1" indicates transformer backbone, "L"
indicates the model did not converge. S-A: S-Abrupt, S-G: S-Gradual.
Method
FSNet
DER+
MIR
ER
TFCL
OnlineTCN
Informer
MSE MAE MSE MAE MSE MAE MSE
MAE MSE
MAE MSE MAE
MSE MAE
ETTh2
0.018 0.009 0.022 0.015 0.019 0.018 0.018 0.017 0.030 0.003 0.011 0.007 1.370 0.043
24
0.014
0.005 0.024 0.004 0.017 0.005 0.007 0.006 0.005 0.003 0.017 0.002 2.254 0.102
48
0.128
0.012 0.143 0.015 0.130 0.012 0.141 0.013 0.279 0.024 0.147 0.016 2.088 0.091
ECL ETTml
0.003
0.004 0.003 0.007 0.005 0.009 0.005 0.009 0.004 0.008 0.003 0.002 0.088 0.060
24
0,002
0.002 0.002 0.002 0.005 0.004 0.003 0.002 0.006 0.005 0.002 0.002 0.035 0.023
48
0.003
0.002 0.003 0.002 0.006 0.005 0.004 0.004 0.010 0.008 0.002 0.003 0.020 0.014
0.021
0.001
0.027
0.002
0.037 0.013 0.034 0.011 0.047
0.011 0.019 0.002
24
0.096
0.011 0.072 0.013 0.261 0.013 0.236 0.017 0.338 0.019 0.077 0.009
48
0.105
0.011 0.146
0.014
0.143
0.012 0.320 0.014
0.253 0.008
0.122 0.011
WTH Traflic
0.001
0.001
0.001
0.001
0.001
0.001
0.001
0.001
0.004
0.003
0.001 0.001
0.009 0.008
24
0.002 0.002 0.002
0.002 0.002 0.002 0.002 0.002 0.004 0.002 0.002 0.001 0.015 0.008
0.001
0.001 0.001 0.002 0.002 0.002 0.001 0.002 0.002
0.002
0.001 0.001
0.005 0.005
0.001
0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.001 0.003 0.003
48
0.001
0.001 0.011 0.007 0.009 0.005 0.009 0.005 0.004 0.006 0.001 0.001 0.009 0.008
0.112
0.037
0.171
0.041
0.176
0.040
0.159 0.033
0.283
0.061
0.009
0.002
0.149 0.059
24
0.199 0.027 0.202 0.034 0.192 0.032 0.022 0.003 0.011 0.002 0.174 0.017 0.322 0.066
0.166
0.039
0.169 0.041 0.177 0.040 0.171 0.039 0.360 0.083 0.164 0.039 0.277 0.099
24 0.187 0.033 0.200 0.033 0.199 0.035 0.190 0.032 0.010 0.002 0. 1.188 0.033 0.771 0.099
7' = 0.3 results in the gradients accumulated in only a few recent samples. Lastly, relatively high memory triggering threshold 0.75 indicates memory-triggering condition can detect substantial representation change to store in the memory. The hyper-paramene parameter coos-validattion is performed via grid search and the grid provided below.
Experience replay batch size (for ER and DER++): [2,4, 4,
Experience replay coefficient (for ER) AER: [0.1, 0.2, 0.7,
DER++ coefficient (for DER++) 7DER+ 0.2, 0.5,
EMA coefficient for FSNet 7 and 7/: [0.1, 0.2, 0.3, 0,,,05,,066,0,,00880000
Memory triggering threshold 7 [0.6, 0.65, 0.7, 0.75, 0.8,0.8
Number of filters per layer: 64
Episodic memory size: 5000 (for ER, MIR, and DER++), 50 (for TFCL)
The remaining configurations such as data pre-processing and optimizer setting follow exactly
as Zhou et al. (2021).
E ADDITIONAL RESULTS
E. STANDARD DEVIATIONS
We report the standard deviation values of the comparison experiment in Table 1, which were av- eraged over five runs. Overall, we observe that the standard deviation values are quite small for all experiments.
E.2 COMPLEXITY COMPARISON
In this Section, analyze the memory and time complexity of FSNet.
18

---

Published as a conference paper at ICLR 2023
Table Summary of the model complexity on the ETTh2 data with forecasta window 24. We report the number of floating points incurred by the backbone and different types of memory. Gl Gradient Importance (TFCL). G-EMA Gradient Exponential Moving Average (FSNet), AM Associative Memory (FSNet), EM Episodic Memory (ER).
Model
Memory
Method
Total
Backbone
Adapter
GI
G-EMA
AM
EM
FSNet
1,041,288
733,334 N/A
N/A
614,400
1,130,496
N/A
3,519,518
ER OnlineTCN
1,041,288
N/A
N/A
N/A
2,822,400
3,863,688
3,667,208
N/A
N/A
N/A
N/A
N/A 806,400
3,667,208
TFCL
1,041,288
N/A
2,082,576
N/A
N/A
3,930,264
Table 5: Summary of the model and total memory complexity of different methods.NV denotes the number parameters of the convolutional layers, H and E. denotes the look-back and forecast win- dows length
Method
OnlineTCN
ER
MIR
DER++
FSNet
Model Complexity
O(N + H) O(E H) O(N + E+
Memory Complexity
N/A ONN + H)
O(N) O(N + H)
Total Complexity
Asymptotic analysis We consider the TCN forecaster used throughout this work and analyze the model, total memory, and time complexities of the methods considered in work. We let N denotes the number of parameters of the the convolutional layers, E denotes the length of the look- back window, and H denotes the length of the forecast window.
Model and Total complexity We analyze the model and the total memory complexity, which arises from the model and additional memory units.
First, the standard TCN forecaster incur ONN H))+)) memory complexity arising from N parameters of the convolutional layers, and an order of H parameters from the linear regressor.
Second, we consider the replayed-bane based strategies, which also incur the same O(N + H) model complexity as the OnlineTTNN, For the total memory, they use episodic memory to store the pre- vious samples, which costs OEE + H) for both methods. Additionally, TFCL stores the importance of previous parameters while MIR makes a copy of the model for its virtual update, both of which cost (N + H.. Therefore, the total memory complexity of the replay strategies (ER, DER++, MIR, and TFCL) is O(N E + H).
Third, in FSNet, both the per-layer adapters and the associative memory cost similar number of parameters as the convolutional layers because they matrices with number of channels one dimension. Therefore, asymptotically, FSNet also incurs a model and total complexity of O(N+H) where the constant term is small.
Table 5 summarizes the asymptotic memory complexity discussed so far. Table shows the number of parameters used of different strategies the ETTh2 dataset with the forecast window of H 24. We consider the total parameters (model and memory) of FSNet as the total budget and adjust other baselines meet the budget. As we analyzed, for FSNet, its components, including the adapter, associative memory, and gradient EMA, require an order of parameter as the convolutional layers in the backbone network. For the OCNNneTCN strategy, we increases the number of convolutional filters so that it has roughly the same total parameters as FSNet. For ER and TFCL, we change the number of samples stored the episodic memory.
Time Complexity We report the throughput (samples/second) different methods in Table 6. We that ER and DER++ have high throughput (low running time) compared to others thanks to their simplicity. As FSNet introduces additional mechanisms to allow the network to take less samples to adapt the distribution shifts, its throughput is lower than ER and DER++ Neverthe-
19

---

Published as a conference paper at ICLR 2023
Table Throughput (sample/second) of different methods in our experiments with forecast window of H = 1.
Running Time
ETTh2
ETTml
WTH
ECL
Traffic
S-A
ER
46
46
43
42
39
46
DER++
45
45
43
42
38
46
TFCL
29
28
27
27
26
27
MIR
22
22
21
21
30
23
FSNet
28
28
28
27
27
29
Table 7: Results of different FSNet's parameer configurations on the ETTh2 (H = 48) and
S-A (H 24) benchmarks.
Configuration
ETTh2
S-A
Y
MSE
MAE
MSE
MAE
0.9
0.3
0.75
0.846
0.515
1.760
1.038
0.9
0.4
0.8
0.860
0.521
1.816
1.086
0.99
0.4
0.7
0.847
0.512
1.791
1.049
0.99
0.3
0.8
0.845
0.514
1.777
1.042
less, FSNet is more efficient than and MIR comparable to TFCL, which are two common continual learning strategies.
E.3 ROBUSTNESS HYPER-PARAMETER SETTINGS
This experiment explores the robustness of FSNet to different hyper-parameter setting. Particularly, we focus on the configuration of three gradient EMA y': and (iii) the associative memory activation threshold т. In general, we provide two guidelines to reduce the search space these hyper-parameters: (i) setting 7 to a high value (e.g. 0.9) and 7 to small value (e.g. 0.3 or 0.4); (ii) set to be relatively high (e.g. 0.75). We report the results of several hyper-parameter configurations in Table 7. We observe that there are not significant differences among these configurations. It is also worth noting that we use the same configuration for experiments conducted in this work. Therefore, we can conclude that FSNet robust these configurations.
(i) the gradient EMA y: (ii) the short-term
E.4 FSNET AND EXPERIENCE REPLAY
This experiment explore the complementarity between FSNet and experience replay (ER). We hy- pothesize that ER is valuable component when learning on data streams because introduces the benefits of mini-batch training to online learning.
We implement variant of FSNet with an episodic memory for experience replay and report its performance in Table 8. We can see that FSNet+ER outperforms FSNet in all cases, indicating the benefits FR, even to FSNet. However, is important that using ER will introduce additional mem- ory complexity and that scales with the look-back window. Lastly, in many real-world applications, storing previous data samples might be prohibited due to privacy concerns.
E.5 VISUALIZATTIONS
E.5.1
VISUALIZATION OF THE SYNTHETIC DATASETS
We plot the data (before normalization) of the S-Abrupt and S-Gradual datasets in Figure 4.
20

---

Published as a conference paper at ICLR 2023
Table 8: Performance of FSNet with and without experience replay,
FSNet
FSNet+ER
Data
H
MSE
MAE
MSE
MAE
0.466
0.368
0.434
0.361
ETTh2
24
0,687
0.467
0.650
0.462
48
0.846
0.515
0.842
0.511
1
0.321
0.26
0.243
0.248
Traffic
24
0.421
0.312
0.350
0.275
1.000
2000
3000
4000
5000
6000
1000
3000
4000
(a) S-Abrupt
(b) S-Gradual
Figure 4: Visualization of the raw S-Abrupt and S-Gradual datasets before normalization. Colored regions indicate the data generating distribution where we use the same color for the same distri- bution. S Guadual, white color region indicates the gradual transition from one distribution to another. Best viewed in color.
E.5.2 ACTIVATION PATTERN OF FSNET
This experiment explores the associative memory activation patterns of FSNet. For this, we consider the S-Abrupt dataset with H and plot the activation patterns in Figure 5. Note that due the large number of memory slots, only plot the memory slot with the highest attention score each step. We remind that S-Abrupt, the first 3,000 samples belong to three different data distribution and these distribution sequentially reappear in the last 3,000 samples, which are color-coded in Figure 5. First, we observe that not all layers are equally important for the tasks. Particularly, FSNet mostly uses the fourth and sixth layers, and rarely uses the deeper ones.
Second, we note that FSNet memory activations exhibit high specialization as we go to deeper layers. Particularly, only single memory slot is activated in the fourth layer (circle marker) throughout training because shallow layers are responsible for general representations, possibly because it learns generic representations for all patterns. On the other hand, deeper layers are activated according to different distributions: seventh layer memory (triangle marker) is activated by the distribution in pink while the ninth layer memory (square and star markers) activated by the remaining distributions. These observations are consistent with the representation learning patterns in deep networks where shallow layers learn generic representation while deeper layers learn representations that are more specialized to different patterns (Olah et al., 2017).
F DISCUSSION AND FUTURE WORK
We discuss two scenarios where FSNet may not work well. First, we suspect that FSNet may struggle when concept drifts do not happen uniformly all dimensions. This problem arises from the irregularly sampled time series, where each dimension is sampled at a different rate. In this scenario, concept drift in dimension may trigger FSNet's memory interaction and affect the
21

---

Published as a conference paper at ICLR 2023
10
layer
x
0
1000
2000
3000
4000
5000
6000
time step
Figure 5: Activation frequency of the memory slot with the highest attention score for each layer in
FSNet on the S-Abrupt dataset. Same marker indicates the same memory slot. Each color region indicates data generating distribution. Best viewed in color.
learning of the remaining ones. Moreover, if dimension is sampled too sparsely, it might be helpful to leverage the relationship along both the time and spatial dimension for better result.
Second, applications such as finance, which involve many complex repeating patterns, can chal- lenging for FStet. In such cases, the number of repeating patterns may exceed the memory capacity of FSNet, causing catastrophic forgetting. In addition, forecasting complex time series requires the network learn good representation, which may not be achieved by increasing the model com- plexity alone. In such cases, incorporating representation learning component might be helpful.
We now discuss several aspects for further studies. We follow Informer apply the z-normalization per feature, which is a common strategy. This strategy works well in the batch setting because its statistics were estimated using 80% of training data. However, after a concept drift in online learning, it unreliable to use previous statistics (estimated over 25% samples) to normalize samples from a new distribution. In such cases, could be helpful adaptively normalize samples from new distributions (using the new distribution's 's statistics). This could be achieved via an online update of normalization statistics or using a sliding window technique. In addition, while FSNet presents general framework to forecast time series online, adopting to particular application requires incorporating specific domain knowledge to ensure satisfactory performances. In summary, we firmly believe that FSNet is encouraging first step towards general solutions for important, yet challenging problem of online time series forecasting.
22


Reviews:
[{"review_id": "LNrZEKc24E", "reviewer": "Reviewer_dWpT", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes an efficient learning method (slow and fast) for forecasting the time series. The task formulated as a task free online continual problem. To do so, they introduce the FSNET network where each layer equipped with a memory and an adaptor to adapt with the recent chsnges. ", "summary_of_the_review": "The novelty of paper is marginal, and also the paper should be revised to be easy to read. Also, comparing with previous method is required.", "strength_and_weaknesses": "The idea is interesting and novel in forecasting the time series \n\nExperimental shows the good performance of proposed method \n\nAuthors formulated the problem as a continual task where can be considered a novel approch.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "BkS2DmswzD-", "reviewer": "Reviewer_MVEA", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper approaches online time series from the angle of task-free continual learning. This was achieved with a fast adaptation mechanism utilizing a moving average of the model gradient, and an associative memory module to recall the adaptation coefficients of reoccurring events. Experiments were performed on both benchmark data and synthesized datasets designed with explicit task changes, in comparison to a good number of baseline models. \n", "summary_of_the_review": "This paper presents a continual learning approach to online time-series modeling and forecasting, with two interesting components that are intuitive and well described. There were some lingering technical questions to be clarified, but the overall paper seems to be interesting with thorough experimental evaluations", "strength_and_weaknesses": "Strengths:\n\n- The proposed mechanisms of fast adaptation and recalling recurring patterns are intuitive and interesting. \n\n- Discarding the need of task boundaries is interesting and useful, although there are some technical questions regarding how the model may handle lookback window/forecasting horizons that contain \"unknown\" tasks transitions.\n\n- The experiments considered a variety of benchmark data as well as synthetic data with controlled transitions of tasks. The comparisons considered a good number of representative baselines demonstrating margins of improvement. \n\n\n\nWeakness:\n\n- A key question is regarding the size of the memory needed in the different datasets/experiments. How is the necessary size determined, and does the memory expand? In synthetic settings where ground truth of the tasks are known, it also be good to know what were learned in M, whether it corresponds to the four true processes, and whether the correct adaptation coefficients were retrieved when the old pattern reappeared. \n\n\n- How much the key hyperprameters, such as tao that triggers the member association in Eq (3), depends on the underlying datasets and what are their effects\n\n\n- From Fig 2, it was not clear if the presented method indeed achieved \"faster\" adaptation. It seemed that the error trend is more or less the same across methods as learning continues. The presented method was overall better, but when there was an increase of error, it was not clear that the presented method had a \"faster\" drop of errors -- the trend is similar (how fast the error drops); it's just that the presented method had overall lower errors. Please clarify if my understanding was not correct.\n\n- It was not clear how the model may work when the look-back window has a mix of tasks (in between transitions). The S-G setup somewhat reflects this, but even in S-A, such scenario exists (when the look-back window or forecasting horizon contains the task transition boundary), correct? How would the presented method accommodate this? By learning and memorizing many different adaptation coefficients reflecting the transition in between tasks?\n\n- In both Fig 2 and Fig 3, for the data where tasks transition are known (eg. the two synthetic sets), it'd be good to label the boundary in the figure (even though one can track it from the texts, i.e., every 1000 steps, it'd be good to make it more intuitive on the figures).\n\n-The separation of mean and standard deviation in the main text and appendix makes it quite hard to follow and assess whether the obtained margins of improvements were statistically significant. This is an important aspect of model performance, and it'd be good to try to move the std in the main text as well.\n\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "-g9kImwLSZv", "reviewer": "Reviewer_r44V", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this manuscript, time-series forecasting has been studied in an online scenario. For online training of deep neural predictors one has to take into account both the new coming knowledge as well as retaining the learned patterns in the past (the so called stability-plasticity dilemma). The authors reportedly are inspired from the Complementary Learning Systems (CLS) theory,  a fast-and-slow neuroscience learning framework in human. Namely, for information retention, they have used an associative memory and for fast learning  they utilized an adapter per layer, in a Temporal Convolutional Network (TCN). \n", "summary_of_the_review": "Although the studied topic is interesting, the current version needs more work to get into the shape that it could be publish in this conference. \n\nUpdate:  in my opinion, after authors modifications in this phase the manuscript is improved. ", "strength_and_weaknesses": "The authors studied an interesting topic which is indeed critical in many practical cases. \n\nThe adaptation process that you explained in page 4 is somewhat very abstract and confusing (considering as your main contribution) and not even mentioned what for instance \\phi_l or h_l refer to (one could find them by continue reading the paper, but could have been mentioned in their place).\nAs in the online case you want to have fast inference, I think it is necessary to include the exact inference time, for each method. Also a detailed analysis of the overhead memory for each method can be informative.\n\nIn my opinion, the novelty of this study is very limited, as the main aspect of this work has been already studied in the past  (Pham et al., 2021a; (Pham et al., 2021b, Arani et al., 2021). Basically, a continual learning scenario has been translated to an online learning scenario, where as in most cases actually there are a lot in common in these two cases.   ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster