ACCTS: an Adaptive Model Training Policy for Continuous Classification of Time Series
Authors: Chenxi Sun
Moxian Song
Derun Cai
Shenda Hong
Hongyan Li
Venue: ICLR 2022 Submitted (2022)
Keywords: Continuous classification of time series
Deep learning
Model training

Abstract: More and more real-world applications require to classify time series at every time. For example, critical patients should be detected for vital signs and diagnosed at all times to facilitate timely life-saving. For this demand, we propose a new concept, Continuous Classification of Time Series (CCTS), to achieve the high-accuracy classification at every time. Time series always evolves dynamically, changing features introducing the multi-distribution form. Thus, different from the existing one-shot classification, the key of CCTS is to model multiple distributions simultaneously. However, most models are hard to achieve it due to their independent identically distributed premise. If a model learns a new distribution, it will likely forget old ones. And if a model repeatedly learns similar data, it will likely be overfitted. Thus, two main problems are the catastrophic forgetting and the over fitting. In this work, we define CCTS as a continual learning task with the unclear distribution division. But different divisions differently affect two problems and a fixed division rule may become invalid as time series evolves. In order to overcome two main problems and finally achieve CCTS, we propose a novel Adaptive model training policy - ACCTS. Its adaptability represents in two aspects: (1) Adaptive multi-distribution extraction policy. Instead of the fixed rules and the prior knowledge, ACCTS extracts data distributions adaptive to the time series evolution and the model change; (2) Adaptive importance-based replay policy. Instead of reviewing all old distributions, ACCTS only replays the important samples adaptive to the contribution of data to the model. Experiments on four real-world datasets show that our method can classify more accurately than all baselines at every time.

Under review as a conference paper at ICLR 2022
ACCTS: AN ADAPTIVE MODEL TRAINING POLICY FOR CONTINUOUS CLASSIFICATION OF TIME SERIES
Anonymous authors
Paper under double-blind review
ABSTRACT
More and more real-world applications require to classify time series at every time. For example, critical patients should be detected for vital signs and diagnosed all times to facilitate timely life-saving. For this demand, we propose a new concept, Continuous Classification of Time Series (CCTS). achieve the high-accuracy classification at every time. Time series always evolves dynamically, changing features introducing the multi-distribution form. Thus, different from the existing one-shot classification, the key of CCTS is to model multiple distributions simul- taneously. However, most models are hard to achieve due to their independent identically distributed premise. If a model learns new distribution, it will likely forget old ones. And if a model repeatedly learns similar data, it will likely be overfitted. Thus, two main problems are the catastrophic forgetting and the over fitting. In this work, we define CCTS as a continual learning task with the unclear distribution division. But different divisions differently affect two problems and fixed division rule may become invalid as time series evolves, In order to overcome two main problems and finally achieve CCTS, we propose novel Adaptive model training policy ACCTS. Its adaptability represents in two aspects: (1) Adap- tive disti-disttibution extraction policy. Instead of the fixed rules and the prior knowledge, ACCTS extracts data distributions adaptive to the time series evolution and the model change; (2) Adaptive importance-basee replay policy. Instead of reviewing all old distributions, ACCTS only replays the important samples adaptive to the contribution of data to the model. Experiments on four real-world datasets show that our method can classify more accurately than all baselines at every time.
INTRODUCTION
In the real world, many applications need to classify time series data at every time (Gupta et al. (2020)). For example, in the Intensive Care Unit (ICU), most detected vital signs of patients change dynamically according to the development of decease. The status perception is needed at any time as the real-time diagnosis provides more opportunities for doctors rescue lives (Chen al. (2014)). In response to the current demand, we propose new concept Continuous Classification of Time Series (CCTS). It aims to classify as accurately as possible st every time with the data evolvement.
For most time series extracted from practical applications, the development, caused by their changed data characteristics, leads to the evolved data distribution, and finally produces the multi diltiidistribttion form. For example, in Figure I the data distributions of blood pressure of 2,000 sepsis patients vary among early, middle and late time stages during hospitalization, bring a cripe-dissribution
In the background of CCTS, modeling multiple data distributions simultaneously is the requirement: when the data distribution changes, the model performance cannot decrease. However, a single model, like deep neural network, is lack of ability to learn all distributions simultancously as they restricted by the premise of independent identically distributed (i.i.d) data (Shim et al. (202T). 1) If model learns new distribution, will negatively affect its performance old ones.
Therefore, when learning the multi-distribution, a model will face two problems:
Catastrophic forgetting. A time series usually has a large number of time points. For example, the blood pressure of critical patient could be sampled hundreds of times. If a model is trained at all points, will learn hundreds of data distributions continuously. Frequent learning of new knowledge will inevitably lead to the forgetting of old ones (Parisi et al. (2019))

---

Under review as a conference paper at ICLR 2022
Early Stage with 20% data length
Middle Stage
Larc Stage
One-shot: Classification of Time Series (CTS)
Chant
with 80% data length
with 100% data length
One-shot: Early Classification Time Series (ECTS)
Continuous: Continuous Classification of Time Series
(CCTS)
1000
500
Value: (mmmHt)
Blood pressone
in Time Series. The statistics of mean blood pressure of 2,000 sepsis represent the distributions (Kiyassch et al. p2021).
Figure 1:
Figure Continuous Classification Mode. Different from the existing time series classification task, CCTS need the continuous classification.
Over fitting. As the development of time series needs process, the data distributions of adjacent time are always similar as shown in Figure [ And in order to alleviate the catastrophic forgetting, many methods replay old distributions to enhance memory (Kiyasseh al. (202T). Over-learning of similar data will cause the strict function and poor generalization (Saha al. (2021).
Two issues are intertwined. The old knowledge review could avoid the forgetting but worsen the overfitting, while the iteration avoidance could alleviate the overfitting but worsen the forgetting. The additional policy like the selective review, could help to balance the above problems. However, the policy formulation is based on the known data distribution, yet the distribution in CCTS is not clear as its data form is sequential and can be divided by different rules. Therefore, before planning how to should also determine the well-divided nulti-distribution as: Less divided distributions may worsen the catastrophic forgetting problem and omit important data features. If dividing time series less, distributions will be more different. It will make the model change greatly after learning the new distribution and seriously affect the model performance old distributions. Besides, the key distribution may not be included due to the incomplete sampling;
learn the multi-distre multi-distribution
More divided distributions may worsen the over fitting problem and have the low training efficiency. dividing time series more, distributions will have many overlaps. It will make some similar features to be learned repeatedly and over fit the trained model to the recurring dataset. Besides, in the model training phrase, learning more distributions will spend more time.
The optimal multi dulli-distributionn is hard to obtain. Unlike images, the time series is more abstract and its characteristics are not explicit (Xing et al. (201T). Although some methods can describe time series like Shaplets (Liang & Wang (202 I)), they still need the prior knowledge. Most importantly, the artificial rule needs to be determined before training the model and remains the same over time. But because the time series has been evolving dynamically, fixed rule is likely to be outdated.
In this work, instead of the fixed methods, we design an Adaptive model training policy for Continuous Classification of Time Series (ACCTS). It has two data-based adaptive policy:
Adaptive multi- multi-distribution extraction policy. It explores the policy space according to the reward of distribution difference and classification accuracy based on the reinforcement learning strategy, and finally extracts data distributions adaptive to the time series evaluation and the model change; Adaptive imporancce-aased replay policy. It leans the impact of each sample on the model, applying partial replay to balance the result accuracy and the time efficiency. The important samples in each distributions are obtained adaptive the time series evaluation and the model change.
Both polices are dynamic rather than static to trade off the forgetting and the overfitting. Experimental results on real-world datasets show that ACCTS is more accurate than all baselines at every time.
RELATED WORK
The popularity of time series classification has attracted increasing attention in many practical fields (Santos & Kern (0016).. The foundation is Classification of Time Series (CTS), making classification based on the full-length data (Fawaz et al. (2019). But in time-sensitive applications, Early Classification of Time Series (ECTS) is more critical, making classification at an early time (Gupta al. (020)). For example, early diagnosis helps for sepsis outcomes (Liu et al. (2018a)). Both of them give one classification result, while CCTS needs continuous multiple results. Thus, based on the classification mode, the existing work can be summarized into categories.
2

---

Under review as a conference paper at ICLR 2022
2.1 SHE-SHOT CLASSIFICATION
This mode classifies time series at a fixed time: as shown in Figure 2 once the classification is complete, the action will not continue. Both CTS and ECTS are belong to this and always use Deep Learning (DL) models. RNNs-based methods recur in evolution direction to learn the sequential dependency (Fawaz et al. (2019). CNN-based methods use kernels to extract local features (Huang) [Choi et al. (2017) models the long-term dependencies: [Tnn et af. (2020) and [Sun et al. (2021) address the time irregularity; Hsu et al. (2019) learns frequency features; Lai et al. (2015) designs Temporal Convolutional Network (TCN) to integrate RNN and CNN. However, the above DL-based methods just perform well on i.i.d data at fixed time, like early 6 hours sepsis diagnosis
al. (2017)) Choi
(Reyna et al. (2019a)), but fail for classifying old distributions after learning new distributions.
2.2 CONTINUOUS CLASSIFICATION
This mode classifies time series at every time as shown in Figure 2 Most of the existing methods apply multi-models to model multi-distribution, like SR (Mori et al. (2018) and ECEC (Lv et al. (2019). They divide data according to time stages and design different classifiers for different distributions. But they only consider the data division, ignoring the strategic training method. Besides, the operation of classifier selection in multi-moded framework will result in additional losses.
CCTS has the multi-distributed data, leading to the catastrophic forgetting problem. Recently, Continual Learning (CL) (Delange et al. (2021) ) aims to address the issue of static models incapable of adapting their behavior for new knowledge. It learns new task at every new moment and each new task corresponds to a new data distribution. Replay-based methods re-train the model by old data to consolidate memory (Rolnick et al. (2019; Kiyasseh et al. (2021) Isele & Cosgun 2018) Rebuffi eet al. ): Regularization baaad methods restrain parameter update of neural networks to limit forgetting (Kirkpatrick et al. (2016); Lopez-Paz & Ranzato (2017); Liu et al. (2018b), Zhang et al.) (2020); Model-based methods change network structure or apply multiple models to response to different tasks (Fernando al. (2017); Maalya & Lazebnik (2018)). But most of the above methods have the problems of storage limitation, distributions drifts and model overfitting. Most importantly, in CL, the definition of old and new tasks clear and the division of distribution fixed. But in CCTS, the distributions, that is, the tasks in CL, is not determined and need to be defined.
Thus, CCTS is a new proposed concept, there is no systematic analysis and corresponding solutions.
PROBLEM FORMULATION
Definition 1 Continuous Classification. time series X [(I1, having T time points is labeled with a class C E C. Continuous classification tasks aim to classify the time series X al every time t, = 1,..., T with the minimum additive loss E ((x(())))
Note that most of the existing time series classification tasks are belong to the one-shot classification mode, where they optimize the objective with a single minimum loss (X:t))::) C). In continuous classification, the model need to learn multiple distributions, which is firstly proposed in CL field. A
CL issue T = (T), T ..., T has a sequence of N tasks, learning a new task T = (X, C) at every time. The goal is to control the statistical risk of all seen tasks ΣN (xxx,,r) .C") [[F" ((X";)) (), C.)).. Based on this settings, we give the definition of CCTS:
Definition Continuous Classification of Time Series (CCTS). time series {a1, ... has T time points and is labeled with class C E C. CCTS has a sequence of N distributions M = {M², ,M MN. }. Each distribution M is represented by the subsequence set X1a0. The goal is the statistical risk control of classification with all distributions: Σn (M 8), C)] with the loss C, the network function and parameters 0 of the classification model. When the model f is trained by the current distribution M its performance on all observed data cannot degrade:
minC(s" (MM
1n C(f" (X 8t) ),) 1
1 tm
subject to
(X (X1=;8"-1)),
3

---

Under review as a conference paper at ICLR 2022
Time scries data
ACCTS policy
New distribution
Yes
Important
X
ACCTS
Samples B
t
f(+] + f Train the Classifier
Move the time
Figure 3: Adaptive Model Training Process for Continuous Classification of Time Series
ADAPTIVE MODEL TRAINING POLICY
Figure 3) shows the model training process by the policy of ACCTS, When a model is trained by time
series from the initial to the final time, ACCTS gives two decisions:
Whether the current time series segment forms new distribution to be learned. If yes, train the
model by the current time series; Otherwise, do not train and continue to get new data points;
Which old samples need to be replayed and learned again. If the previous decision is yes, train the model by the obtained old samples again after train it by the current time series.
4.1
ADAPTIVE MULTI-DISTRIBUTION EXTRACTION
The first decision is got by the adaptive multi-ddstribuee distribution extraction polity. It is an agent that decides whether to extract the current time series sequence train the model. It solves 3-triple partially- observable Markov decision process (S. A, R. (Kaelbling et al. (1995)), where the observation arrive from state at each time, an action a. is sampled using learned policy, and reward is observed according to the selected action's 's quality. The objective is to optimize long-term rewards.
State S. It is represented by the characteristics of the currently data and the adaptability of the old model to the current data. It is intuitive: First, the model need to be trained by the dataset with different features from the previous data for the comprehensive modeling; Second, the model must be trained again when it performs poorly on the current data for the overall accuracy.
At the current time t. we use the Long Short-Term Memory (LSTM) network as the base model to learn the hidden characteristies of a time series X1:t, generating low-dimensional vector representation h4 We also propose the model gradients Jr to evaluate the adaptability of the model to the current time series. The model gradient can help for the interpretation of DL model by explaining the response of the neural network to input data (Srinivas Fleuret (019)). Large gradient fluctuation reflects the low adaptability of the model the input data. Thus, the state s, of current time series is:
8t Concatenate(LSTTM())))
(2)
ft = 5 (W, + bb),, = oW, [Wa[hh + by), C ft .-1 + (Wa(hh be), 0 = o (W,, (
LSTM
h, = = (n))..
à
= ,
C (X 8,),
MG
Action A. At the current time I, the action a, dictates the decisions of ACCTS agent: a; = continue accept the value point of time series and let LSTM move forward one time step; If a, 1, extract the current time series X1: as a new distribution to be learned. For the action selection, we e-greedy selection avoid abundant exploitation. a, is replaced with random action with the probability e of exponentially decreasing from 1 to during the training process.
at
with probability -- random, with probability
G₂ =
E 10
(3)
Reward R. The agent observes the return which can qualify the parameters of the current policy. The goal of CCTS is the high accurate classification by solving the problems of catastrophic forgetting
4

---

Under review as a conference paper at ICLR 2022
Algorithm 1 Adaptive Multi- Extraction Policy
Input: Data D C));
Vo Ouni
Vea (sc,aa))))
Classifier net f.
Output: Extraction policy (f,, Q).
Vo, Oucic
1: for = 1 to 7 do
2: St <- Equation 2
7:
Soft update 2', H¹ by:
3:
pu(s,, Actor)
l2r =0g + (1 7)0g
4: if de = then
0₂r + 70 + (1 - T ,0
5:
Equation 4 6: Update Q, μ by:
8: Train / by (X1:t; C)
9: end if
10: end for
and over fitting as we analysed in Section Thus, we pursue the higher accuracy of the current classifier on all potential data distributions control the catastrophic forgetting, and we limit the number of extracted distributions by the time span between distributions to control the over-fitting Thus, the current time i, after applying the action at, the reward r; consisted of two components. The first term is for the high accuracy of the current model f on all data, the second term is for less divisions by using the time length between the current time t" and the last data extraction time ("-1
Σ C( C(F", x!:)+ (1 (1-0 -t-1D
Tt T
(4)
When using the transition probability P(s at), the total reward of the trajectory is the is sum of the reward in each time. Thus, the objective is maximize the total reward R ΣT Pt. The policy gradient method (Sutton et al. (1999)) learns the policy ( az) = P(a, (st) for the larger return. The objective is J(B)= E r(s, (o s, a)]. For ACCTS, we apply actor-Critic (Zhang et al. (2019) structure with two components of the main net and the target net. The main net of Actor use the state to generate the action a; The main net of Critic Q judges the action a through reward by Q-function (Watkins & Dayan (192)). The target nets of Actor and Critic μ', Q' put the target Q value stable for period time, making the algorithm performance more stable.
Oattor (0₂) E,,ES e [Ω (st, a (sx (0,) [0o]
(5)
OCriti((fe) -Es (rre 2 (sL+1, 10q @(s al
4.2 ADAPTIVE IMPORTANCE-BASES REPLAY
The replay mechanism help to alleviate the catastrophic forgetting (Borsos et al. (2020)). However, the operation of repeated replay easily causes the over-fitting problem, especially for time series with small difference between the two adjacent times. In CL, many methods only replay the representative data, such as the class means (Rebuffi et al. (2017)) and the class prototype (Mazumder et al. (202T)) each representative is fixed to its distribution. But in CCTS, still need to consider whether all the representative need to be learned again and whether the representative will change over time.
Thus, we focus on the adaptive method to explore wider space, where the replayed data dynamic and determined according to the current state. We introduce an importance-based replay method. In each round, it only re-trained the model by some important samples the model. The importance of each sample is learned from the objective of an additive loss function.
We incorporate the importance parameter 3 of time series X; in the replay buffer 8 as a coefficient of its loss L, The overall loss at the current time 1n is the sum of loss of each sample's loss:
[
Ln
(32 Cn.s + A(Bn, - 1)2)
B
6
i=1
(X lam 8.13n <c}
SC., Thus, if a sample X, is hard to classify, its is learned by the gradient descent Ba, + Ba,i 88m.i"
loss L.s will be larger. In order minimize the loss its 3a, will be smaller. Based this, in each
5

---

Under review as a conference paper at ICLR 2022
Algorithm 2 The Model Training Process by ACCTS Policy
Input: Data D {(X, C)); ACCTS Actor p.
//ADAPTIVE EXTRACTION
Output: Final Classifier net f.
6:
if a, then
Initialize buffer B3 < {X1:1)
B, B, + {X1+}
2: Initialize DL Classifier net f'.
8:
f <- Train f by B with Eq
//TRAVERSE EVERY TIME POINT
/IMPORTANCE STORAGE
3: for = 2 to T do
9:
B [X,]X, E B", , <
4: 8t f {he, ]t3 from Eq 2
10: end
5: dz p(s:))
11: end for
learning phrase, the buffer 8 contains the current time series X1:0" and the important old time series X, who are the first few difficult learning samples (3n-1.2 < E) in the last buffer B=-1, Meanwhile, as [3 is the confidence of loss, if 3 0, the loss are hard to be optimized. Thus, inspired by Kiyasseh et al. (2021), we introduce a regularization term (3 1)² and initialize & = 1 to penalize it when rapidly decaying toward 0. As 3 is re-obtained after each model training process, the important samples X are changed adaptively and the buffer updated iteratively.
4.3 OVERALL TRAINING PROCESS FOR CLASSIFIER MODEL
The adaptive multi-distribution extraction policy, which is achieved by the Actor net μr, is trained before the classifier training process, as shown in Algorithm | First, LSTM calculates the current sate s, (Line 2) and gives the action a, (Line 3). Then, the reward Fi is obtained by the long-term accuracy to update the net (Line 6), where Actor and Critic are updated alternately. Main Critic net is updated by Q value, calculating from both two Critic. Main Actor is updated by the crkeppoopaatiion gradient of main Critic. Target Actor and Critic are learned by the soft update (Line 7).
The adaptive importance- based replay policy is trained along with the classifier training process, as shown in Algorithm 2 First, in each time step, the Actor of ACCTS determines if a new distribution appears (Line 4,5). If yes, train the classifier from f to fa-1 by datasets in the buffer 8 (Line 7,8), and get the important samples according to 3 to form new buffer g=+1 (Line 9); Else, continue to get new values in next time point + 1. At the final time, we can get the well-trained classifier f.
Note that the two processes of the adaptive multi-distribution extraction and the adaptive importance- based replay are relevant rather than independent. The extraction policy is based the feature of the buffer data, and the replay policy selects the important samples based on the extracted data. Both of them are data-based, which helps for adaptive combination. That's why we design the replay- based policy rather than the regularization-base policy after the distribution extraction.
5 EXPERIMENTS
5.1 DATASETS
UCR-EQ dataset (Chen et al. (2015) has 471 earthquake records from UCR time series database archive. It is the univariate time series of seismic feature value. Natural disaster early warning, like earthquake warning, helps to reduce casualties and property losses (Ammon et al. (201)).
USHCN dataset (Menne R. (0166) has the daily meteorological data of 48 states in U.S. from 1887 to 2014. It is the multivariate time series of 5 weather features. Rainfall warning is not only the demand of daily life, but also can help prevent natural disasters (Lee et al. (202T)).
COVID-19 dataset Yan L (2020) has 6,877 blood samples of 485 COVID-19 patients from Tongji Hospital, Wuhan, China. It the multivariate time series of 74 laboratory test features. Mortality prediction helps for the personalized treatment and resource allocation (Sun et al. (2020)).
SEPSIS dataset (Reyna et al. (2019b)) has 30,336 patients' records, including 2,359 diagnosed sepsis. It the multivariate time series of 40 related patient features. Early diagnose of sepsis is critical to improve the outcome of ICU patients (Seymour et al. (2017)).
Not that for each time series in the above four datasets, every time point is tagged with a class label, which is the same its outcome label, such 'mortality,, 'sepsis", 'earthquake" and 'rain'.

---

Under review as a conference paper at ICLR 2022
5.2 BASELINES
CCTS is related to ECTS and CL, we use baselines in these fields. The first is ECTS- methods. All methods use the same base model of LSTM and the same structure fully connected layers.
LSTM (Wiens al. 20122; Choi al. (2017)) It contains single classifier model. For one time series, the classifier model is trained by all subsequences from time 1 to time t, where t = 2, .... T. SR (Mori et al. 2018 8) ) It has multiple base models. All models are trained by the full-length time series. The final classification is the fusion result. It also has a stop rule of classification stop time. ECEC @L et al. (2019)). It has a set of base models. Each model is trained by time series in different time stages. When classifying, the data selects the classifier based on its time stages.
The second type is CL-based methods, including regularization and replay methods:
EWC (Kirkpatrick al. (2016)). It is regularization-bu based method, training model to remember the old tasks by constraining important parameters to stay close to their old values.
GEM (Lopez-Paz & Ranzato (2017)) It is a regularization-basee method, training a model to remember the old tasks by finding the new gradients which are acute angles to the old gradients.
CLEAR (Rolnick et al. (2019). It is replay-based method, using the reservoir sampling to limit the number of stored samples to a fixed budget assuming an i.i.d. data stream.
CLOPS (Kiyasseh et al. (2021)). It is replay-based method, training base model by replaying old tasks with importance-guided buffer storage and uncertainty-basel buffer acquisition.
5.3 EVALUATION METRICS
The accuracy is evaluated by Area Under Curve of Receiver Operating Characteristic (AUC-ROC), The performance of continuous mode is evaluated by Backward Transfer (BWT) and Forward Transfer (FWT), the influence that learning a current has on the old/future. R E TR. MMX[MM] is an accuracy matrix, Rig is the accuracy on M after learning M'. b the accuracy with random initialization.
IMI-1
IMI
BWT M
1
Σ
M - -R,ss
FWT
Σ
R
- b,.
[M|
5.4 RESULTS AND ANALYSTS
We test the baselines from the classification accuracy, analyze our ACCTS methods from ablation study and coefficient test, and show the representation of time series in continuous classification.
5.4.1
CLASSIFICATION ACCURACY
ACCTS has the best performance on classification accuracy. As shown in Table it can classify time series more accurately than all baselines at every time. The average accuracy is about 2% higher.
5.4.2 CATASTROPHIC FORGETTING AND OVER FITTING
ACCTS is the best when solving these two problems with the highest BWT and FWT as shown in Table 2 In Table 3 for CL-based methods (EWC, GEM. CLEAR and CLOPS), the accuracy validation set is much lower than that on training set, shown as 4. And in Table [ in late stages, the accuracy of Cl.-based methods is also lower than that of ECTS-based methods. They all show the over-fitting of CL-based methods caused by training and reviewing at all time. But ACCTS can alleviate this problem by training and reviewing at the selected time so that it accuracy higher.
5.4.3 ABLATION STUDY
Both policies of ACCTS are necessary. As shown in Figure 4 compared with the extraction at all times and random extraction, the adaptive extraction performs best in both overall data and the early distribution; Compared with no-replay and all-replay, the adaptive replay has the best performance. Besides, the accuracy of importance-bel fit between two bata- polices of ACCTS: importance-basee replay and distribution extraction fit better than regularization and distribution extraction. This confirms the reason why we refer to replay-based instead of regularization-base CL methods.
replay is higher than regularization. It demonstrate good
5.4.4 COEFFICIENT TEST
ACCTS has two definable coefficients a and e, belong two policies separately. Larger a review more distribution to learn. Larger e causes more samples to review. As shown in Figure the practice set them the direct ratio: Within reasonable range, more distributions need more review.

---

Under review as a conference paper at ICLR 2022
Table Basclines Classification Accuracy (AUC-ROC1) for Real-world Datasets at Time Steps. *20% means the current classification time is 20% the total time of the full-length time series; Bold font indicates the highest accuracy;
More detailed results are in Appendix.
Dataset
Method 20%*
40%
60%
80%
100%
LSTM
0.711=0.038
0.843=0.019
0.874+0.012
0.909+0.014
0.924=0.012
SR
0.73640.014
0.863=0.015
0.888+0.017
0.928+0.100
0,941#0.104
ECEC
0.73840.018
0.865#0.014
0.890#0.015
0,92940.100 107
0,940#0.009
EWC
0.76840.018
0.87420.016
0.895=0.014
0,92340. 102
0.933+0.00 0003
UCR-BQ
GEM
0.767+0.017
0.876=0.016
.0900=0.015
0.929#0.000 0008
0.934+0.004
CLEAR
0.770#0.015
0.880=0.013
0.904+0.012
0.923+0.004
0.932#0.00 0,05
CLOPS
0.773+0.016
0.878=0.016
0,902+0.015
0,917 0.0
0,925#0.005
ACCTS
0.774+0.023
0.882#0.022
0.906#0.005
0.933+0.010
0.946#0.003
LSTM
0.700#0.022
0.745=0.028
0.820#0.015
0.852+0.00 014
0.891 #0.000002 .002
SR
0.730#0.022
0.761=0.023
0.836#0.016
0.902+0.013
0,933+0.009
ECEC
0.736+0.024
0.760=0.025
0.837#0.016
0.906+0.017
0,931#0.009
EWC
0.736+0.025
O. 0.728
0.834+0.016
0.896400017
0.926*0.007
USHCN
GEM
0.72840.026
0.781
0.83840.013
0.89900.010
0,928
CLEAR
0.73840.025
0.78440.024
0,83740.010
0.87940.012
0,921 0044
CLOPS
0.74040.024 0,781
0.835#0.016
0.87740.011
0.91940.013
ACCTS
0.742+0.017
0.791#0.021
0.841=0.012
0.910=0.015
0.939=0.013
LSTM
0.701#0.033
0.833+0.015
0.888+0.013
0.92540.014
0.944+0.015
SR
0,730+0.024
0.867=0.016
0,900=0.018
0,946+0.00 0.06
0.962#0.00 005
ECEC
0.73240.028
0.870=0.016
0.904+0.014
0,948+0.015
0.963+0.017
EWC
0.769#0.015
0.888+0.028
0.923+0.014
0,940#0.013
0,95420.008
CO
GEM
0.779#0.017
0.885#0.022
0.924+0.018
0.939#0.010
0,953+0.005
CLEAR
0.785+0.019
0.879=0.016
0.926#0.014
0,941
0,952+0.008
CLOPS
0.775+0.013 0.900=0.017
0.925+0.015
0.940#0.007
0.954+0.006
ACCTS
0.790
0.901
0.92720.006
0.960#0.011
0.967#0.008
LSTM
0.629+0.035
0.736=0.064
0,748
0.795+0.027
0.827#0.039
SR
0.65940.015
0.791#0.026
0.82740.037
0.84540.014
0.86640.023
ECEC
0.66940.019
0.793+0.016
0.815+0.014
0.84940.016
0,863+0.01
EWC
0.733+0.023
0.827=0.036
0.838+0.024
0.848+0.015
0.854+0.00 0
SEPSIS
GEM
0.730+0.024
0.826#0.033
0.836+0.028
0.849+0.014
0.853+0.012
CLEAR
0.732+0.024
0.825=0.035
0.839+0.028
0.847000010
0.848+0.016
CLOPS
0.733+0.025
0.824=0.036
0.838+0.026
0,850+0.017
0.857#0.018
ACCTS
0.73420.038
0.828+0.030
0.842+0.034
0.857+0.012
0.872+0.012
Table 2: Continual Learning Performance of Baselines. The left table BWT1 results, the right table is FWT1 results.
Method EWC GEM CLEAR CLOPS ACCTS
Mechod
EWC
GEM
CLEAR CLOPS ACCTS
UCR-EQ +0.039 +0.041 +0.053
+0.052
+0.058
UCR-EQ
+0.321 +0.329 40.312
+0.301
+0.345
USHCN +0.058 +0.054 +0.063
+0.074
+0.084
USHCN +0.312
+0.328
+0.335
+0.301
+0.342
COVID- 19 +0.011 +0.012 +0.009
+0.014
+0.020
COVID-19
+0.426 +0.421
+0.427
+0.439
+0.455
SEPSIS +0.019 +0.017 +0.030
+0.032
+0.035
SEPSIS +0.295 +0.265 +0.401
+0.397
+0.410
Table 3: COVID-19 Classification Accuracy with Non-uniform Training Sets and Validation Sets. ( means the accuracy is greatly reduced; More detailed results are in the appendix.
Subset SR
ECEC
EWC
GEM
CLEAR
CLOPS
ACCTS
Male
0.968+0.014 0.969 0.965#0.012 0.965+0.004 0.978=0.009 0.978+0.014 0.971=0.010
Female
0.945+0.004 0.94740.015 0.939=0.018 0.938+0.003
0.919=0.008., 0.929 0.947=0.002
Age 30- 0.965+0.014 0.967=0.015 0.967+0.013 0.964+0.009 0.977=0.008 0.979:0.012 0.972=0.011
Age 30+ 0.941+0.007 0.943+0.018 0.931=0.008., 0.9203+0.0000
0.902=0.006] 0.914+0.007L, 0.945=0.006
Test
0.964+0.013 0.968=0.015 0.966=0.012
0.962+0.006 0.979=0.000
0.978+0.010 0.970=0.00 007
Valid.
0.962+0.006 0.963+0.014 0.954#0.003
0.953+0.005
0.952=0.009L
0.954+0.0044, 0.967=0.006
8

---

Under review as a conference paper at ICLR 2022
Figure 4: Ablation Study of Two Policies of ACCTS with the Case Study of COVID-19
Average acouracy staee
Accuraayy early stage
Average accuracy stages
Acouracy carly stage
dial bution ustraciion candoniyy disiriutiion estraction ACCIS disritution wer actioo
chai buuion estraction Handam stribution extraction ACCT'S di butten extraction
restey
ACCTS importanceebssed
ACCTS prpottance based
EAC Reguiar ccation
EVC Regular
Figure 5: Parameter Test a, e in ACCTS.
Figure 6: Extracted Six Distributions from SEPSIS Dataset by ACCTS.
Figure 7: Impact of Data Size Distributions.
Dierbau 3 166.79n
Seartoution 132-2
iuprosution 4io
LISHEN COVID-19 SEPS3S
UCRE EQ
0.8
Dkarfauz on 0%
Count
044
C
Systolic slood value
Figure 8: The Important Samples in Four SEPSIS Distribution Buffers (2,3,4,5 in Figure
C2
5.4.5 ANALYSIS
The case study of SEPSIS Figure 6 shows that ACCTS only extracts six distributions and the difference among distributions is relatively large. The extraction is concentrated in 85%-length late stage, which may be because the patient's vital signs change significantly near the outcome time.
5.4.6 ANALYSIS OF IMPORTANT SAMPLES
The important samples includes not only the data hard to learn, but also the representative data as shown Figure g It might because that, the representative data is similar to the most common data, resulting in greater additive loss, therefore leading to smaller coefficients in Equation 6 Through the experimental results, we can redefine the important samples: Important samples are samples that can represent most data of a class and samples that are difficult to distinguish by the model.
5.4.7 IMPACT OF DATA SIZE
ACCTS prefers to divide more distributions for large data size shown in Figure Besides, ACCTS has greater advantages over baselines larger data size, detailed results are in Appendix.
6 CONCLUSION
In this paper, we propose a new concept of Continuous Classification of Time Series (CCTS) to meet the real needs. It has two major difficulties of catastrophic forgetting and over fitting. CCTS, the dultiidisttributonn of time series is not clearly defined, and the distribution division directly affects the above two difficulties. Thus, we design an Adaptive model training policy named ACCTS. contains multi-distribution extraction policy adaptive to the time series evaluation and the model change, and an importance-bast based replay policy adaptive to the data features and final accuracy. We test the methods on four real-world datasets and analyze the method from perspectives of accuracy, continual learning, ablation study and parameter setting. Future work could refer to the distribution extraction policy that fits regularization-basee based methods.

---

Under review as a conference paper at ICLR 2022
REFERENCES
C. J.J.ommn A. A. Velasco, T. Lay, and T. C. Wallace. Earthquake forecassin & early warning. Academic press, 2021.
Zalán Borsos, Mojmir Mutny, and Andreas Krause. Coresets via bilevel optimization for continual learning and streaming. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12. 2020, virtual, 2020.
W. Chen, J. Wang, Q. I.. Fe Ng. S. C. Xu, and I.. Ba. The treatment of severe and multiple injuries in intensive care unit: report of 80 cases. European Review for Medical & Pharmacological Sciences, 18(24):3797, 2014.
Yanping Chen, Eamonn Keogh, Bing Hu, Nurjahan Begum, Anthony Bagnall, Abdullah Mueen, and Gustavo Batista. The uer time series classification archive, July 2015. www. uur edu/
eamonnnnimee sateeessereesseeeee _ariies_dat/////
Edward Choi, Andy Schuetz, Walter Stewart, and Jimeng Sun. Using recurrent neural network models for early detection of heart failure onset. Journal of the American Medical Informatics Association, 24(2):361-377 2017.
Matthias Delange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Greg Slabaugh, and Tinne Tuytelaars. continual learning survey: Defying forgetting classification tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1-1, 2021.
Hassan Ismail Fawaz, Germain Forestier, Jonathan Weber, Lhassane Idoumghar, and Pierre-Alain Muller. Deep learning for time series classification: a review. Data Min. Knowl. Discov, 33(4): 917-963, 2019.
Chrisantha Fernando, Dylan Banarse, Charles Blundell, Yori Zwols, David Ha, Andrei Rusu, Alexander Pritzel, and Daan Wierstra. Pathnet: Evolution channels gradient descent in super neural networks. CoRR, abs/1701.08734, 2017.
Ashish Gupta, Hari Prabhat Gupta, Bhaskar Biswas, and Tanima Dutta. Approaches and applications of early classification of time series: A review. IEEE Trans. Artif. Intell., 1(1):47-61, 2020.
En- Yu Hsu, Shien-Liang Liu, and Vincent S. Tseng. Multivariate time series early classification with interpretability using deep learning and attention mechanism. In Qiang Yang. Zhi-Hua Zhou, Zhiguo Gong, Min-Ling Zhang, and Sheng-Jun Huang (eds.), Advances in Knowledge Discovery and Data Mining 23rd Pacific-Asia Conference, PAKDD 2019, Macau, China, April 14-17, 2019, Proceedings, Part III volume 11441 of Lecture Notes in Computer Science, pp. 541-553. Springer, 2019.
Gao Huang. Zhuang Liu, Laurens Van Der Maaten, and Kilian Weinberger. Densely connected convolutional networks. In CVPR, volume 1, pp. 3, 2017.
David Isele and Akansel Cosgun. Selective experience replay for lifelong learning. In Sheila A. Mcllraith and Kilian Q. Weinberger (eds.), Proceedings of the Thirty-Secone AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI- 18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18). New Orleans, Louisiana, USA, February 2-7, 2018, pp. 3302-3309. AAAI Press, 2018.
Leslie Pack Kaelbling, Michael L. Littman, and Anthony Cassandra. Partially observable markov decision processes for artificial intelligence. In Leo Dorst, Michiel van Lambalgen, and Frans Voorbraak (eds.), Reasoning with Uncertainty in Robotics, International Workshop. RUR '95, Amsterdam, The Netherlands, December 4-6, 1995, Proceedings, volume 1093 of Lecture Notes in Computer Science, pp. 146-163. Springer, 1995.
James Kirkpatrick, Razvan Pascanu, Neil C. Rabinowitz, Joel Veness, Guillaume Desjardins, An- Demis
drei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Overcoming catastrophic forgetting in neural networks. CoRR, abs/161 2200719,....... 2016.
10

---

Under review as a conference paper at ICLR 2022
D Kiyasseh, T. Zhu, and D Clifton. A clinical deep learning framework for continually learning from cardiac signals across diseases, time, modalities, and institutions. Nature Communications, 12(1): 4221, 2021.
Siwei Lai, Liheng Xu, Kang Liu, and Jun Zhao. Recurrent convolutional neural networks for text classification. In Blai Bonet and Sven Koenig (eds.). Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, January 25-30, 2015, Austin, Texas, USA, pp. 2267-2273. AAAI Press, 2015.
W. Y. Lee, S. K. Park, and H. H. Sung. The optimal rainfall thresholds and probabilistic rainfall conditions for a landslide early warning system for chuncheon, republic korea. Landslides, 2021.
Zhiyu Liang and Hongzhi Wang. Efficient class-specific shapelets learning for interpretable time series classification. Inf. Sci., 570:428-450, 2021.
Bin Liu, Ying Li, Zhaonan Sun, Soumya Ghosh, and Kenney Ng. Early prediction of diabetes com- plications from electronic health records: A multi-task survival analysis approach. Proceedings of the Thirty-Second AAAI Conference Artificial Intelligence, pp. 101-108, 2018a.
Xialei Liu, Mare Masana, Luis Herranz, Joost van de Weijer, Antonio M. López, and Andrew D. Bagdanov. Rotate your networks: Better weight consolidation and less catastrophic forgetting, In 24th International Conference on Puttern Recognition, ICPR 2018, Beijing, China, August 20-24, 2018, pp. 2262-2268. IEEE Computer Society, 2018b.
David Lopez-Pate Papez-Pazzene and Mare' Aurelio Ranzato. Gradient episodic memory for continual learning. In Advances Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 7, December 4-9, 2017, Long Beach, CA, USA, pp. 6467-6476, 2017.
Junwei Lv, Xuegang Hu, Lei Li, and Peipei Li, An effective confidence- early classification of time series. IEEE Access, 7:961 13 96124,, 2019.
Arun Mallya and Svetlana Lazebnik. Packnet: Adding multiple tasks to a single network by iterative pruning. In 2018 IEEE Conference on Computer Vision and Puttern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pp. 7765-7773. Computer Vision Foundation / IEEE Computer Society, 2018.
Pratik Mazumder, Pravendra Singh, and Piyush Rai. Few-shot lifelong learning. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applica- tions of Artificial Intelligence, IAAI 2021, The Eleventh Symposium Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, pp.
AAAI Press,
2021.
Williams C. Menne, M. and Vose R. Long-term daily and monthly climate records from stations across the contiguous united states. U.S. Historical Climatology Network), 2016.
Usue Mori, Alexander Mendiburu, Sanjoy Dasgupta, and José Antonio Lozano. Farly classification of time series by simultancously optimizing the accuracy and earliness. IEEE Trans. Neural Networks Learn. Syst., 29(10):4569-45 2018.
German Ignacio Parisi, Ronald Kemker, Jose L. Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: review. Neural Networks, 113:54-71, 2019.
Sylvestre-Alvye Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H. Lampert. icarl: Incremental classifier and representation learning. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HII USA, July 21-26, 2017, pp. 5533-5542, IEEE Computer Society, 2017.
M. A. Reyna, C. S. Josef, R. Jeter, S. P. Shashikumar, and A. Sharma. Early prediction of sepsis from clinical data: The physionet/comppuing in cardiology challenge 2019. Critical Care Medicine, 48 12):1, 2019a.
11

---

Under review as a conference paper at ICLR 2022
Matthew A. Reyna, Christopher Josef, Salman Seyedi, Russell Jeter, Supreeth P. Shashikumar, M. Brandon Westover, Ashish Sharma, Shamim Nemati, and Gari D. Clifford. Early prediction of sepsis from clinical data: the physionet/computing in cardiology challenge 2019. In 46th Computing in Cardiology, CinC 2019, Singapore, September 8- 2019, pp. I 1-4, IEEE, 2019b. doi: 10.23919/CinC4984) ACA.9//CCACA444119999990777 URL htt.h:t////ddot org /tt.://.....39991.
2019.900525
David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy P. Lillicrap. and Gregory Wayne. Experience replay for continual learning. In Hamna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' A'Alché-Buc, Emily B. Fox, and Roman Garnett (eds.). Advances Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurlPS 2019, December 8- 8-14, 2019, Vancouver, BC, Canada, pp. 348-358, 2019.
Gobinda Saha, Isha Garg, and Kaushik Roy. Gradient projection memory for continual learning. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021, 2021.
Tiago Santos and Roman Kern. A literature survey of early time series classification and deep learning. In Proceedings of the 1st International Workshop on Science, Application and Methods in Industry 4.0 co-located with (i-K (i-KNOW 2016), Graz, Austria, October 19, 2016, 2016.
Christopher W Seymour, Foster Gesten, Hallie C Prescott, Marcus E Friedrich, Theodore Iwashyna, Gary S Phillips, Stanley Lemeshow, Tiffany Osborn, Kathleen M Terry, and Mitchell M Levy. Time treatment and mortality during mandated emergency for sepsis. New England Journal of Medicine, 376(23): 2235-2244, 2017.
Dongsub Shim, Zheda Mai, Jihwan Jeong, Scott Sanner, Hyunwoo Kim, and Jongseong Jang. Online glass-incrementa continual learning with adversarial shapley value. In Thirty- AAAI Confer- ence on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, pp. 9630-9638, 2021.
Suraj Srinivas and François Fleuret. Full-gradient representation for neural network visualization. In Advances Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurlPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp. 4136-4135, 2019.
Chenxi Sun, Shenda Hong, Moxian Song, Hongyan Li, and Zhenjie Wang. Predicting covid- 19 disease progression and patient outcomes based on temporal deep learning. BMC Medical Informatccs and Decision Making, 21:45, 2020.
Chenxi Sun, Shenda Hong, Moxian Song, Yen-Hsiu Chou, Yongyue Sun, Derun Cai, and Hongyan Li. Te-esn: Time encoding echo state network for prediction based irregularly sampled time series data. In Zhi-Hua Zhou (ed.), Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, pp. 3010-3016. International Joint Conferences on Artificial Intelligence Organization, 8 2021. doi: 10.24963/jcci.202111
URL httht:://///// org/
10. 10.24963/1jcai.. (tl.::aaa..2221116 2021/414
Main Track.
Richard S. Sutton, David A. McAllester, Satinder P. Singh, and Yishay Mansour. Policy gradient methods for reinforcement learning with function approximation. Advances in Neural Informa- tion Processing Systems 12, [NIPS Conference, Denver, Colorado, USA, November 29 December 4, 1999], pp. 1057-1063, 1999.
Qingxiong Tan, Mang Ye, Baoyao Yang. Siqi Liu, Andy Jinhua Ma, Terry Cheuk-Fung Yip, Grace Lai- Hung Wong, and Pong Chi Yuen. DATA-GRU: dual-attention time-aware gated recurrent unit for irregular multivariate time series. In The Thirty-Fourth Fourth AAAI Conference Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020, pp. 930-937. AAAI Press, 2020.
Christopher JCH Watkins and Peter Dayan. Q-learning. Machine learning, 8(3-4):279-13 1992.
12

---

Under review as a conference paper at ICLR 2022
Jenna Wiens, Eric Horvitz, and John V Guttag. Patient risk stratification for hospital-associatee diff as a time-series classification task. In Advances in Neural Information Processing Systems, pp. 467-475, 2012.
Zhengzheng Xing, Jian Pei, Philip S Yu, and Ke Wang. Extracting interpretable features for early classification on time series. In Proceedings of the 2011 SIAM International Conference on Data Mining, pp. 247-258. SIAM, 2011.
Goncalves J et Yan L, Zhang H T. An interpretable mortality prediction model for covid-19 patients. Nature, Machine intelligence, 2, 2020.
Junting Zhang, Jie Zhang, Shalini Ghosh, Dawei Li, Serafettin Tasci, Larry P. Heck, Heming Zhang, and C.-C. Jay Kuo. Class-incremental learning via deep model consolidation. In IEEE Winter Conference Applications of Computer Vision, WACV 2020, Snowmass Village, CO, USA, March 1-5, 2020, pp. 1120-1122 129. IEEE, 2020.
Shangtong Zhang, Wendelin Bochmer, and Shimon Whiteson. Generalized off-policy actor-critic. In Advances Neural Information Processing Systems 32: Annual Conference Neural Information Processing Systems 2019, NeurlPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp- 1999-2009, 2019.
13

---

Under review as a conference paper at ICLR 2022
A DETAILED EXPERIMENTAL RESULTS
A.1 BASIC CLASSIFICATION ACCURACY
Table 4: Classification Accuracy (AUC-ROCT) for 4 Real-world Datasets at the First 5 Time Steps. *k% means the current classification time k% of the total time of full-length time series; Bold font indicates the highest accuracy.
Dataset
Method
10%
20%
30%
40%
50%
LSTM
0.695=0.044
0.711=00003
0.803=0.024
0.843#0.019
0.854+0.017
SR 0.700=0.015
0.736+0.014
0.830#0.016
0.863#0.015
0.871#0.024
ECEC
0.703+0.013
0.738+0.018
0.828+0.017
0.865#0.014
0.873+0.026
EWC
0.72440.015 0.76840.018
0.848+0.014
0.874#0.016
0.883=0.025
UCR-EQ
GEM
0.723=0.014 0.767+0.017
0.850#0.015
0.876#0.016
0.89010.024
CLEAR
0.72940.015 0.770#0.015
0.852+0.019
0.880#0.013
0.899#0.026
CLOPS
0.728+0.016 0.773+00016
0.855#0.015
0.878#0.016
0.896+0.028
ACCTS
0.730=0.022
0.774
0.85620.015
0.882+0.022
0.900=.0017
LSTM
0.682+0.014 0.70040.028
0.721=0.013
0.745=0.028
0.784+0.023
SR
0.702+0.014
0.73040.022
0.745+0.016
0.809#0.024 0.001
ECEC
0.707=0.017
0.73620.022
0.748+0.015
0.760=0.025
0.806+0.025
EWC
0.727=0.018
0.736+0.025
0.768+0.017
0.798+0.024
0,805#0.022
USHCN
GEM
0.720#0.019 0.728+0.026
0.772+0.015
0.781
0.801#0.026
CLEAR
0.728+0.016 0.73840.025
0.773+0.018
0.784+0.024
0.802+0.027
CLOPS
0.728+0.012 0.74010.024
0.769#0.019
0.781
0.800#0.024
ACCTS
0.730+0.018
0.742:0.017
0.775+0.016
0.791=0.021
0.810#.0133
LSTM
0.605#0.044 0.701
0.793#0.022
0.833=0.015
0.844#0.013
SR
0.636+0.014
0.730+0.024
0.810#0.013
0.86710.016
0.901+0.013
ECEC
0.639=0.013 0.732+0.028
0.829#0.013
0.870=0.016
0.901#0.026
EWC
0.703+0.022
0.76940.015
0.870#0.014
0.888+0.028
0.915+0.017
COVID-19
GEM
0.699=0.025 0.77920.017
0.871=0.015
0.885#0.022
0.91420.019
CLEAR
0.710#0.013 0.785#0.0119
0.870=0.016
0.879#0.016
0.916#0.024
CLOPS
0.709*0.017 0.775+0.013
0.869#0.012
0.900#0.017
0.918+0.026
ACCTS
0.71240.021
0.79040.023
0.872+0.013
0.901#0.022
0.919#0.016
LSTM
0.576+0.003 0.629+0.035
0.735+0.064
0.736*0.064
0.745#0.056
SR
0.626+0.035
0.65940.015
0.768+0.013
0.791#0.026
0.803+0.018
ECEC
0.623+0.024 0.669#0.019
0.761#0.016
0.793=0.016
0.811
EWC
0.671
0.733+0.023
0.799#0.015
0.827#0.036
0.832+0.028
SEPSIS
GEM
0.670=0.026
0.730
0.802#0.018
0.826#0.033
0.834#0.026
CLEAR
0.680=0.028 0.73240.024
0.801
0.825+0.035
0.833+0.025
CLOPS
0.68420.025 0.733+0.025
0.802+0.017
0.824+0.036
0.830+0.023
ACCTS
0.69020.032
0.73420.038
0.81240.022
0.828+0.036
0.835+0.024
14

---

Under review as a conference paper at ICLR 2022
Table Classification Accuracy (AUC-ROCT) for 4 Real-world Datasets at the Last Time Steps. *K% means the current classification time is k% of the total time of the full-length time series; Bold font indicates the highest accuracy.
Dataset
Method
60%
70%
80%
90%
100%
LSTM
0.874#0.012
0.913++.034
0.909#0.014
0.919#0.008
0.924#0.012
SR
0.888=0.017
0.92410.010
0.928=0.10 105
0.936#0.109 103
0.941#0.106
ECEC
0.890=0.015
0.92320.013
0.929#0.100 107
0.936+0.006
0.940=0.009
EWC
0.895=0.014
0.910=0.017
0.923+0.100 102
0.930=0.005
0.933 0003
UCR-EQ
GEM
0,900=0.015 0.920#0.015
0.929#0.008
0,935#0.003
0,934#0.00 004
CLEAR
0.904#0.012 0.918+0.019
0.923#0.004
0.928+0.007
0.932+0.000
CLOPS
0.902+0.015 0.915#0.010
0.917#0.006
0.921
0.925#0.008
ACCTS
0.90620.005
0.928+0.007
0.933+0.010
0.940=0.005
0.946+0.003
LSTM
0.820#0.015 0.837+0.024
0.852+0.014
0.891 0.869#0.025
SR
0.836*0.016
0.886+0.023
0.902+0.013
0.933+0.009 0.926
ECEC
0.837#0.016
0.88710.027
0.906+0.017
0.920#0.028
0.931#0.009
EWC
0.834#0.016
0.867+0.026
0.896#0.017
0.906#0.020
0.926#0.005
USHCN
GEM
0.838+0.013 0.86840.029
0.899#0.010
0.910#0.021
0,92810.005
CLEAR
0.837=0.010 0.867+0.023
0.879#0.012
0.899#0.027
0.921
CLOPS
0.835#0.016 0.861+0.024
0.877+0.011
0.895#0.016
0,919+0.013
ACCTS
0.841#0.012
0.898#0.022
0.910=0.015
0.928+0.013
0.939+0.013
LSTM
0.888+0.013 0.918+0.033
0.925#0.014
0.939#0.005
0.94410.015
SR
0.900=0.018 0.935+0.010
0.946+0.006
0.952+0.017
0.962+0.005
ECEC
0.904+0.014 0.937+0.008
0.948+0.015
0.952+0.008
0.963+0.017
EWC
0.923+0.014 0.935+0.007
0.940=0.013
0.950=0.013
0.954+0.00 008
COVID-19
GEM
0.924#0.018 0.936+1.009
0.939#0.010
0.949#0.017
0.953+0.00
CLEAR
0.926#0.014 0.933+0.011
0.941=0.007
0.948#0.009
0.952+0. 008
CLOPS
0.925+0.015 0.935+0.013
0.940#0.007
0.94720.006
0.95410.006
ACCTS
0.927#0.006
0.955#0.008
0.960=0.011
0.96320.009
0.967
LSTM
0.748+0.043
0.773+0.032
0.795+0.027
0.813+0.025
0.827+0.039
SR
0.82740.037
0.835+0.013
0.845+0.014
0.859#0.022
0.866#0.023
ECEC
0.815+0.014
0.827+0.016
0.849+0.016
0.859#0.017
0.863+0.014
EWC
0.838+0.024 0.842+0.030
0.848+0.017
0.850=0.014
0.854+0.016
SEPSIS
GEM
0.836+0.028 0.841+0.034
0.849+0.014
0.851+0.016
0.853+0.012
CLEAR
0.83940.028 0.842+0.031
0.847+0.010
0.850=0.019
0.848+0.016
CLOPS
0.838+0.026 0.842+0.030
0.850+0.017
0.853+0.010
0.857+0.018
ACCTS
0.842+0.034
0.852+0.023
0.857*0.012
0.866#0.014
0.872+0.012
15

---

Under review as a conference paper at ICLR 2022
A.2 CLASSIFICATION ACCURACY ON SUBSETS WITH DIFFERENT SEMANTICS
Table COVID-19 Classification Accuracy with Non-uniform Training Sets and Validation Sets. 4 means the accuracy is greatly reduced.
Subset
LSTM
SR
ECEC
EWC
Male
0.955#0.013
0.96840.014 0.945#0.004
0.969=0.016
0.965#0.012
Female
0.924+0.013
0.947=0.015
0.939#0.018
Age 30-
0.954+0.013
0.965#0.014
0.967=0.015
0.967+0.013
Age 30+
0.923+0.014
0.941#0.007
0.943=0.018
0.931#0.008
Test
0.950#0.011
0.964=0.011
0.968+0.015
0.966+0.0012
Valid.
0.944+0.014
0.962+0.006
0.963+0.014
0.954+0.003
Subset
GEM
CLEAR
CLOOSS
ACCTS
Male
0.965+0.004
0.978+0.009
0.978+0.014
0.971#0.010
Female
0.938+0.003
0.919#0.0084
0,921#0.0094
0.947+0.002
Age 30-
0.964
0.977 0.08
0.979#0.012
0.97240.010
Age 30+
0.923 0400
0.902 |
0.914
0.945+0.006
Test
0.962+0.006
0.979#0.000
0.978+0.010
0.970#0.007
Valid.
0.953+00.005
0.952+0.009,
0.954#0.004|
0.96740.006
16

---

Under review as a conference paper at ICLR 2022
A.3 CLASSIFICATION ACCURACY ON SUBSETS WITH DIFFERENT DATA SIZE
Table Classification Accuracy (AUC-ROCT) for Subsets with Different Data Size. k% means the volume of sub dataset is k% of the corresponding original dataset;
Bold font indicates the highest accuracy;
means that the accuracy of ACCTS is higher 2% than this method.
Dataset
Method
20%
40%
60%
80%
100%
LSTM
0.724
0.765
0,804*
0,809
0,813
SR
0.758
0.784
0.828*
0.813*
0.831
ECEC
0.790
0.770
0.815*
0.827
0.838
EWC
0.785
0.791
0.833*
0,855
0.862
UCR-EQ
GEM
0.780
0.775
0.840*
0.857
0,863
CLEAR
0.784
0.808
0.859
0.864
0.870
CLOPS
0.792
0.809
0.864
0.871
0,875
ACCTS
0.797
0.817
0.872
0.886
0.896
LSTM
0.701
0.730
0.732*
0.760*
0.763*
SR
0.731
0.769
0.782
0.801 *
0.805
ECEC
0.747
0.774
0.800*
0,807
0,816*
EWC
0.739
0.768
0.810
0.817
0.826
USHCN
GEM
0.737
0.772
0.809
0.811
0.818
CLEAR
0.757
0.780
0.812
0.819
0.823*
CLOPS
0.775
0.785
0.817
0.825
0.839
ACCTS
0.776
0.790
0.821
0.835
0.843
LSTM
0.713
0.730
0.765*
0.819*
0.834
SR
0.751
0.767
0.806
0.822
0.842
ECEC
0.755
0.770
0.796*
0.829*
0.856*
EWC
0.763
0.785
0.794*
0.835
0.849*
COVID-19
GEM
0.769
0.772
0.793*
0.849
0,856
CLEAR
0.776
0.791
0.810
0.856
0.866
CLOPS
0.775
0.789
0.809
0.848
0,874
ACCTS
0.781
0.800
0.821
0.863
0.888
LSTM
0.658
0.669
0.691
0.733
0.747
SR
0.682
0.700
0.725*
0.759*
0.768
ECEC
0.679
0.702
0.719*
0.755
0.770
EWC
0.685
0.708
0.729*
0.768 *
0.772
SEPSIS
GEM
0.693
0.704
0.740*
0.771 *
0.781
CLEAR
0.687
0.705
0.741
0.776
0.789
CLOPS
0.698
0.710
0.745
0.779
0.783*
ACCTS
0.701
0.712
0.760
0.794
0.803
17


Reviews:
[{"review_id": "rNEzv9Vs3iD", "reviewer": "Reviewer_AmKR", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Problem:**\n\nThe authors address the problem of generating a class prediction $c_t$ at each time point $t$, given observations $x_{1:t}$ made thus far. In particular, it is supposed that the true class may change over time, so that this problem is not just equivalent to early classification or time series classification from a subsequence.\n\nThe claim ($\\S 1$, $\\S 2.2$) that this problem setting is novel is not supported and warrants some scrutiny. How is it different from estimating the filtering distribution $p(y_t | x_{1:t})$ in a state-space model with discrete latent states and continuous observations? How is it different from a standard sequence-to-sequence learning problem, in which the target sequence takes values in the label space? Note that the case in which the number of classes is unknown or grows over time is well-known, see e.g. [1]. \n\n**Presentation:**\n\nThere is a lack of clarity in the presentation of this work that goes beyond some superficial writing errors. Several terms are non-standard but not defined; e.g. \"multi-distribution\" and \"multi-model\" ($\\S 2.2$).\n\nThe mathematical notation is inconsistent and unclear. In Definitions 1 and 2, the class $C$ does not vary with time. The \"distributions\" $\\mathcal{M}^n$ in Definition 2 are not defined. The loss function $\\mathcal{L}$ takes different arguments in Eq (1) than elsewhere in the preceding definitions. In Algorithm 1, it is not clear how the \"update\" in line 6 makes use of the gradients, and $\\mathcal{Q}', \\mu'$ are not defined.\n\nSome claims made in the paper are simply incorrect as stated:\n- On page 1, the authors write: \"However, a single model, like deep neural network, is lack of ability to learn all distributions simultaneously as they are restricted by the premise of independent identically distributed (i.i.d) data (Shim et al. (2021)).\" In reality, a large and well-known class of deep neural nets - RNNs - are explicitly designed to model non i.i.d. data, specifically time series. The authors use such models as baselines and as part of their own method.\n- On page 3, the authors write: \"Both CTS and ECTS are belong to this and always use Deep Learning (DL) models.\" Time series classification (as distinguished from class prediction per time point) significantly predates deep learning. There exist, for example, an extensive variety of kernel-based or clustering methods.  \n\n**Results:**\n\nThe empirical results show that the proposed method results in improved class prediction accuracy, in the sense of AUC, at each of several time points across a variety of datasets. In addition, the authors include useful analyses of backward and forward transfer that specifically address the question of successful continual learning, as well as a variety of ablation or parameter sensitivity experiments to study their particular solution. The work here is extensive and the results appear promising. Perhaps due to space constraints, there is relatively little discussion of the results presented in Figures 4-8; they are not even captioned.\n\nA major issue with this section is that there is almost no information provided on how the proposed model was trained, for any experiment. There is no documentation of model architectures, hyperparameter settings, optimization procedures, or convergence criteria. It does not seem possible that a competent scientist could reproduce these results, even with unlimited resources.\n\nThere is also a question of computational complexity. The authors propose an RL-based method to adaptively segment the time series data, and then to repeatedly re-train the classification model on old segments in addition to the latest segment. There is no discussion of how long this takes relative to the baselines, but it is reasonable to suspect that the proposed method may be significantly slower.\n\n**Other comments:**\n- The shapelets reference on p. 2 is not correct; the method is due to Ye and Keogh (2009).\n\n- The accuracy results are stated as $AUC \\pm x$, but it is unclear what $x$ is. Where does the variability in the result come from?\n\n- It seems that all of the data examples are limited to binary classification. Moreover, some seem a little dubious for the \"continuous classification\" setting - for example, mortality is a label that starts at zero, switches to one once, and then stays at one. It would be more reasonable to model this in a time-to-event framework than as continuous classification.\n\nReferences:\n\n[1] Fox, E. B., Sudderth, E. B., Jordan, M. I., & Willsky, A. S. (2011). A sticky HDP-HMM with application to speaker diarization. The Annals of Applied Statistics, 1020-1056.", "summary_of_the_paper": "The authors consider a classification setting in which a class label $c_t$ is to be predicted on the basis of time series observations $(x_1, ..., x_t)$ up to time $t$. The goal is to maintain accurate prediction even as the generating distribution evolves, while retaining good accuracy on (i.e. not \"forgetting\" about) previously observed data. They propose a Markov decision process model to segment the time series, followed by a replay-based procedure in which the classification model is updated according to both the current segment and a collection of past segments. The method is contrasted against some baselines on four time series datasets.", "summary_of_the_review": "This paper proposes a complex solution for what is claimed to be a novel problem, but the problem setting as stated seems well-known and the solution is unclear both in its motivation and its details. The results appear promising but lack discussion, and the experiments are poorly documented. My conclusion is that the paper is not ready for publication at this time.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "dwK9FXdai6w", "reviewer": "Reviewer_Kj5t", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Here are some of my concerns regards to this manuscript.\n\n#### <Problem definition>\nIsn't the continuous classification you are talking about the same as forecasting? \nAny real-world time-series forecasting application handles this problem, and there are many services and implementations to process time-series streams.\n(e.g., https://cloud.google.com/architecture/processing-streaming-time-series-data-overview, https://d1.awsstatic.com/whitepapers/time-series-forecasting-principles-amazon-forecast.pdf, https://github.com/AIStream-Peelout/flow-forecast). If we do the aforementioned early classification (= point forecast) every time we receive new data, that becomes continuous time series classification (=forecasting with data streams). Alternatively, you may want to forecast in a long range, such as weather forecast. There are also lots of works on long-term, or long-range forecasting. So can you tell us a little more about which part is the new problem? Besides, the writing and organization of the paper have a large space for improvement. In its current version, the discussion of basic concept and ideas look fragmented and incoherent. \n\n#### <Presentation>\nIt is not easy to read through the paper to understand how the ACCTS model works. For example, when you apply a method (e.g., Actor-Critic), please justify why this method is used. Another example is that I could not follow the logic in the first paragraph of Section 4.2 on why the importance-based replay method is needed. \nAlthough I did not understand it fully, the idea of determining the right time to train the model and the proposed RL-based solution seem to have some technical merits, so I hope that the authors can spend more effort clarifying their ideas and make their writing coherent so that readers can have a better experience reading it. Also, avoid using vague terms like “we focus on the adaptive method to explore a wider space\" without clearly elaborating on its meaning. \n\n#### <Evaluation>\nPlus, it is insufficient to evaluate the algorithms only on classification accuracy. Survival analysis would be a relevant area to discuss COVID-19 and SEPSIS datasets, so please consider the metrics used in survival analysis. \n\n\n#### <Minor issues>\nEq. 1 RHS: Should summation be done from t=1 to t^{n-1}?\np. 4: What do MG stand for?\np. 4: polity -> policy\np. 5: What do you mean by less divisions?\nEq. 7: Is BWT defined correctly?\n", "summary_of_the_paper": "This paper proposes an adaptive model training policy (ACCTS) for continuous classification of time series (CCTS). CCTS is a problem setup that aims to forecast the target variable every time. To mitigate catastrophic forgetting and overfitting problem in CCTS, this paper defines a continual learning task and introduces ACCTS. ACCTS composes of three steps. The first step aims to decide whether to extract the current time-series sequence to train the model. The second step uses an adaptive method to explore a wider space and uses importance-based replay to alleviate catastrophic forgetting. These two processes are relevant to each other and aim to optimize long-term rewards. ACCTS is compared with seven baselines representing early-classification time series and continual learning on four datasets.", "summary_of_the_review": "This paper proposes an adaptive solution for sustainable time-series forecasting. Although it seems to have some technical merits, the presentation needs to be improved a lot, and comprehensive evaluations will be required.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "bQbY6WzRPPo", "reviewer": "Reviewer_kGRM", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "positives:\n- This paper studies a critical problem in real-world applications. The experimental results look promising and show the effectiveness of their proposed method.\n\nnegatives:\n- Many notations are used without formal definition, making this work hard to follow. For example, in section 4, many notations are confusing to me. \n- The application scope of the proposed method is limited. The proposed method is time-consuming as it retrains on selected replicas of received data at each time and needs a large buffer for data storage. Thus, it is only feasible in some specific applications where the accumulated data is small and not time-demanding for inference. However, in many real-world applications of streaming learning, such as the online product recommendation, a large amount of data are accumulating in a short period, and we need to change the model for inference adaptively.\n- This draft lacks some essential discussions on a related topic of online learning in non-stationary environments, see [1] and follow-up works.\n- The performance of the distribution extraction (segmentation) method seems to lack experimental verification. It is suggested to show the distribution change of the whole data stream, and empirically studies whether the segmentation point identifies the distribution change point.\n\n\n[1] Besbes, Omar, Yonatan Gur, and Assaf Zeevi. \"Non-stationary stochastic optimization.\" Operations research 63.5 (2015): 1227-1244.", "summary_of_the_paper": "This draft attempts a streaming classification approach that adaptively identifies the distribution change and retrains the received data to perform well on the whole time series. The proposed method is based on a time series segmentation mechanism inspired by the Markov decision process and a retraining procedure that replays and retrain the selected samples. The empirical evaluation is performed on several real-world datasets.", "summary_of_the_review": "This draft proposes a heuristic approach for streaming data classification. The method stores some selected replicas of received data and retrains the model at each time, which could hardly handle real-world streaming data.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "3gt1QdJKgDy", "reviewer": "Reviewer_Fizc", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n- I think the application of RL to ECTS is novel and interesting.\n- The experimental results look good and could potentially show the effectiveness of this approach.\n\nCons:\n- First and foremost, in the current form, the draft is hard to read. I believe it's a combination of fragmented sentences within paragraphs, the usage of notations, and the lack of justification for certain modeling choices or evaluation metrics.\n- Although the experimental results look promising, the baselines are quite lacking and I'm not convinced of the result. Since the tasks are downstream classification tasks, I would want to see more SOTA time-series classification methods besides just LSTM, SR, and ECEC. I understand that it might be prohibitive to train time-series models on every timestep, but in the paper, evaluations are carried out at 5 checkpoints (20,40,60,80,100) so I think we should be able to train more methods here. See [1] for a strong time-series baseline and other methods in their paper. \n- Since the model requires retraining, I'm not sure if there's a runtime benefit over training multi-models by time-splitting. Even so, besides \"the numbers look better\", are there any insights on why this approach can better model the changing distributions? More concretely,  it would be nice to have a toy dataset with known distribution change or if the authors can show the distribution changes of the 4 real-world used in the evaluation, and empirically study how the approach identities when the distribution changes.\n\n\n-- Minor --\nDefinition 1, do the label C varies with time or is it just one outcome label for every timestep? p. 4: What is MG? model gradient?\n\n[1] Fawaz, Hassan Ismail, et al. \"Inceptiontime: Finding alexnet for time series classification.\" Data Mining and Knowledge Discovery 34.6 (2020): 1936-1962.", "summary_of_the_paper": "This work proposes to continuously classify time-series at every timestep to provide early-classification capability and address real-world needs. The major challenges of this task are catastrophic forgetting and overfitting. The authors state that the multi-distribution of time-series is not clearly defined and the different distributions cause problems with forgetting or overfitting. They proposed an adaptive model training policy to extract multi-distribution instead of rules and priors and an importance-based reply component to replay important experience. They tested on four real-world datasets.", "summary_of_the_review": "The paper proposed a reinforcement learning method for early classification of time series. Although it seems to have some technical novelty, the presentation and evaluation will need to be improved.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a reinforcement learning algorithm to target variable in every time step.  Although the paper proposes an important problem in many real-world applications, there were various major criticisms raised by reviewers.  Most importantly, technical novelty is not well motivated or justified.  There is also a significant lack of a specific description of the proposed method, discussion of computational complexity, clarity and presentation, and evaluation metrics, which decreased the enthusiasm of the reviewers.
Decision: Reject