CLOPS: Continual Learning of Physiological Signals
Authors: Dani Kiyasseh
Tingting Zhu
David A. Clifton
Venue:  (2021)
Keywords: Continual learning
physiological signals
healthcare

Abstract: Deep learning algorithms are known to experience destructive interference when instances violate the assumption of being independent and identically distributed (i.i.d). This violation, however, is ubiquitous in clinical settings where data are streamed temporally and from a multitude of physiological sensors. To overcome this obstacle, we propose CLOPS, a replay-based continual learning strategy. In three continual learning scenarios based on three publically-available datasets, we show that CLOPS can outperform the state-of-the-art methods, GEM and MIR. Moreover, we propose end-to-end trainable parameters, which we term task-instance parameters, that can be used to quantify task difficulty and similarity. This quantification yields insights into both network interpretability and clinical applications, where task difficulty is poorly quantified.

Under review as a conference paper at ICLR 2021
CLOPS: CONTINUAL LEARNING OF PHYSIOLOGICAL SIGNALS
Anonymous authors Paper under double-blind review
ABSTRACT
Deep learning algorithms are known to experience destructive interference when instances violate the assumption of being independent and identically distributed (i.i.d). This violation, however, ubiquitous in clinical settings where data are streamed temporally and from multitude of physiological sensors. To overcome this obstacle, we propose CLOPS, a replay-based continual learning strategy. In three continual learning scenarios based on three publically-available datasets, we show that CLOPS can outperform the state-of-the-art methods, GEM and MIR. Moreover, we propose end-to-end trainable parameters, which we term task- instance parameters, that can be used to quantify task difficulty and similarity. This quantification yields insights into both network interpretability and clinical applications, where task difficulty is poorly quantified.
INTRODUCTION
Many deep learning algorithms operate under the assumption that instances are independent and identically disttibbte (i.i.d.). The violation of this assumption can be detrimental to the training behaviour and performance of an algorithm. The assumption of independence can be violated, for example, when data are streamed temporally from a sensor. Introducing multiple sensors in changing environment can introduce covariate shift, arguably the 'Achilles heel' of machine learning model deployment (Quionero-Candel. et al. 2009).
plethora of realistic scenarios violate the i.i.d. assumption. This is particularly true in healthcare where the multitude of physiological sensors generate time-series recordings that may vary temporally (due to seasonal diseases; e.g. flu), across patients (due to different hospitals or hospital settings), and in their modality. Tackling the challenges posed by such scenarios is the focus of continual learning (CL.) whereby learner, when exposed to tasks in a sequential manner, is expected to perform well on current tasks without compromising performance on previously seen tasks. The outcome is single algorithm that can reliably solve a multitude of tasks. However, most, if not all, research in this field has been limited
small handful of imaging datasets (Lopez-Paz & Ranzato 2017 Aljundi et al. 2019b a). Although understandable from benchmarking perspective, such research fails to explore the utility of continual learning methodologies in more realistic healthcare scenarios (Farquhar Gal 2018). To the best of our knowledge, we are the first to explore and propose CL approach in the context of physiological signals. The dynamic and chaotic environment that characterizes healthcare necessitates the availability of algorithms that are dynamically reliable; those that can adapt to potential covariate shift without catastrophically forgetting how to perform tasks from the past. Such dynamic reliability implies that algorithms longer needs to be retrained on data or tasks to which it has been exposed in the past, thus improving its data-efficiency. Secondly, algorithms that perform consistently well across a multitude of tasks are more trustworthy, desirable trait sought by medical professionals (Spiegelhalter 2020).
Our Contributions. In this paper, we propose replay-baser continual learning methodology that is based on the following:
1. Importance-guided storage: task-instance parameters, scalar corresponding to each instance in each task, as informative signals for loss-weighting and buffer-storage.
2. Uncertainty-based acquisition: an active learning inspired methodology that determines the degree of informativeness of an instance and thus acts bufser-acquisition mechanism.

---

Under review as a conference paper at ICLR 2021
2 RELATED WORK
Continual learning (CL) approaches have resurfaced in recent years (Parisi et al.
2019). Those
similar to ours comprise memory-based methods such as iCaRL (Rebuffi al. 2017), CLEAR (Rolnick et al. 2019), GEM (Lopez-Paz & Ranzato 2017). and aGEM (Chaudhry et al. 2018). In contrast to our work, the latter two methods naively populate their replay buffer with the last m examples observed for particular task. [[sele & Cosgun (2018) and Aljundi et al. (2019b) employ a more sophisticated buffer-storage strategy where a quadratic programming problem is solved in the absence of task boundaries. [Aljundi et al. (2019a) introduce MIR whereby instances are stored using reservoir sampling and sampled according to whether they incur the greatest change in loss parameters were to be updated on the subsequent task. This approach is computationally expensive, requiring multiple forward and backward passes per batch. The application of CL in the medical domain is limited to that of Lenga et al. (2020) wherein existing methodologies are implemented on chest X-ray datasets. In contrast to previous research that independently investigates buffer-storage and acquisition strategies,
focus on dual storage and acquisition strategy.
Active learning (AL. in healthcare has observed increased interest in recent years, with a review of methodologies provided by |Settles (2009). For example, Gong al. (2019) propose a Bayesian deep latent Gaussian model to acquire important features from electronic health record (EHR) data in MIMIC (Johnson et al. 2016) to improve mortality prediction. In dealing with EHR data, Chen et al.) (2013) use the distance of unlabelled samples from the hyperplane in an SVM to acquire datapoints. Wang et al. (2019) implement an RNN to acquire ECG samples during training. Zhou et al. (2017) perform transfer learning in conjunction with a convolutional neural network to acquire biomedical images in an online manner. |Smailagic et al. 2018: 2019) actively acquire unannotated medical images by measuring their distance in a latent space to images in the training set. Such similarity metrics, however, are sensitive to the amount of available labelled training data. Gal et al. (2017) adopt BALD (Houlsby et al. 2011) with Monte Carlo Dropout to acquire instances that maximize the Maasenn-haanon divergence (JSD) across MC samples. To the best of our knowledge, we are the first employ AL-inspired acquisition functions in the context of CL.
3 BACKGROUND
3.1 CONTINUAL LEARNING
In this work, we consider a learner, fw ZT E RT + yT E IR, parameterized by w, that maps an m-dimensional input, TT, to a c-dimensional output, yr. where is the number of classes, for each task T E [1...N]. This learner is exposed to new tasks in a sequential manner once previously- tackled tasks are mastered. In this paper, we formulate our tasks based on a modification of the three-tier categorization proposed by van de Ven & Tolias (019). In our learning scenarios (see Fig. D. a network is sequentially tasked with solving binary classification problem in response to data from nutually-exclusiv pairs of classes Class Incremental I Learning (Class-IL., multi- class classification problem in response to data collected at different times of the year (c.g., winter
Class-IL
Time-IL
Domain-IL
Os. 1
Class 2 vs. 3
Winter Data
Summer Data
Lead
Lead II
fas
fas
fa
[0.1]
[0,1,2.3]
[0.1,2.3]
Figure 1: Ilustrationn of the three continual learning scenarios. A network is sequentially exposed to tasks (Class-IL) with mutually-exclussie pairs of classes, (Time-IL) with data collected at different times of the year, and (Domain-IL) with data from different input modalities.
2

---

Under review as a conference paper at ICLR 2021
and summer) Time Incremental Learning (Time-IL), and multi-class classification problem in response to inputs with different modality Domain Incremental Learning (Domain-IL). In the aforementioned cases, task identities absent during both training and testing and neural architectures are single-headed.
4 METHODS
The two ideas underlying our proposal are the storage of instances into and the acquisition of instances from a buffer such that destructive interference is mitigated. We describe these in more detail below.
4.1 IMPORTANCE-GUDEA BUFFER STORAGE
We aim to populate a buffer, DR, of finite size. M, with instances from the current task that are considered important. quantify importance, we learn parameters, entitled task-instance parameters, Вi.. associated with each instance, 2,T, in each task, T. These parameters play dual role.
4.1.1 LOss- G MECHANISM
For the current task, k and its associated data, Dx. we incorporate / as a coefficient of the loss, Lik- incurred for each instance, Tuk E Dr.. For a mini-batch of size, B, that consists of BN instances from We can learn the values of Bi via gradient
current task, the objective function is shown in Eq. 1 descent, with some learning rate, 1). as shown in Eq.
0L
(1)
Bu. Bic "1 833
(2)
BR
Note that 83.x L.s > This suggests that instances that are hard to classify (t Lax) will exhibit I Bik. From this perspective, 3i can be viewed as a proxy for instance difficulty. However, as presented, 8ix - 0 as training progresses, observation we confirmed empirically. Since Bix is the coefficient of the loss, Lik, this implies that the network will quickly be unable to learn from the data. To avoid this behaviour, introduce a regularization term to penalize its undesirable and rapid decay toward zero. As a result, modified objective function is:
initialize 8ix in order to emulate standard loss function and
Bu Σ 1=1
Leurrent
BirLis + A(3rx - 1)²
(3)
B
When à: >> 1, we replay instances from previous tasks by using replay buffer (see Sec. 4.2 for replay
mechanism). These replayed instances incur a loss Lij j E [l... 11. these instances, in contrast to what we perform to instances from the current task (see Appendix K
1]. We decide to not weight
h-1
B
Creplay
Cij
(4)
L = Leurrent + Creplay
(5)
B
Bs j=1
4.1.2 BUFFER- BUFFER-STORAGE STORAGE MECHANISM
We leverage 3, as a proxy for instance difficulty, to store instances into the buffer. To describe the intuition behind this process, we illustrate, in Fig. 2 the trajectory of 31x and Bo associated with two instances, F1k and T2k. while training on the current task, k. for T = 20 epochs. In selecting instances for storage into the buffer,
1) retrieve their corresponding 3 values at the conclusion of the task, i.c., at (t = 20). 2) rank all instances based on these 3 values, and 3) acquire the top b fraction of instances. This approach, however, lead erroneous estimates of the relative difficulty of instances, as explained next.
that 82x 3ik for the majority of the training process, indicating that 224 had been easier to classify than Fık- The swap in the ranking of these 3 values that occurs towards the end of training in addition myopically looking 3( 20) would erroneously make believe that the
In Fig.

---

Under review as a conference paper at ICLR 2021
opposite true. Such convergence swapping of / values has also been observed by Saxena et al. (2019). As a result, the reliability of 3 as a proxy of instance difficulty is eroded.
To maintain the reliability of this proxy, pro- pose to track the / values after each training epoch, f, until the final epoch, r, for the task at hand and calculate the area under these tracked values. We do by using the trapezoidal rule as shown in Eq. 6 We explored several variants of the storage function and found the proposed form to work best (see Appendix HD. A т,
Teckinstance Puraneer Vakee
0,890
(s((t)
(2x
0.996
Sax
0.004
Rarking Swap
(ex(C)
we rank the instances in descending order of
0.992
(easy to hard) as we found this preferable to the opposite order (see Appendix D]. select the top b fraction, and store them into the buffer, of which each task is allotted a fixed portion.
Epochs
Figure 2: Trajectory of 31k and 82x task k.
Ranking instances based on 8(1 = 20) leads to erroneous estimates of their relative difficulty. We propose to rank instances based on the area under the trajectory of B, denoted as Sik-
The higher the value of the storage fraction, b, the more likely is that the buffer will con- tain representative instances and thus mitigate forgetting, however this comes at an increased computational cost.
Sik = J (u(()dt Bix(t Ar) Bia(t) At
(6)
2
4.2 UNCERTAINTY-BASEH BUFFER ACQUISITION
The acquisition of instances that learner is uncertain about is likely to benefit training (Zhu) 2005) This is the premise of uncertainty-bast acquisition functions such as BALD (Houlshy al. 201
Gal & Ghahramani, 2016). We now outline how to exploit this premise for buffer acquisition.
At epoch number, Tase referred to as Monte Carlo (MC) epochs, each of the M instances, x DB- is passed through the network and exposed to stochastic binary dropout mask generate output, plylæ, ar, w) C This repeated T times to form a matrix, G MRMxTX"" An acquisition function, such BALDMCD. is thus function F F MxTxC + R.M
BALDMCD JSD(PI,P,, ... PP) H (p(y|=)) - Deran))r [H (p(y)22, &))
where H(p(ulz)) represents the entropy of the network outputs averaged across the MC samples, and a ~ p(w/ Drrain)) as in Gal & Ghahramani (2016). At sample epochs, Ts, we rank instances in descending order of BALD and acquire the top fraction from each task in the buffer. A higher value of this acquisition fraction, a, implics more instances are acquired. Although this may not guarantee improvement in performance, does guarantee increased training overhead. Nonetheless, the intuition is that by acquiring instances, from previous tasks, to which a network is most confused, it be nudged to avoid destructive interference in data-efficient manner. We outline the entire training procedure in Algorithms/ 4 in Appendix
EXPERIMENTAL DESIGN
5.1 DATASETS
We conduct experiments in PyTorch Paszke et al. 2019). Given our emphasis on healthcare, we evaluate our approach on three publcaally-availabe datasets that include physiological time-series data such as the electrocardiogram (ECG) alongside cardiac arrhythmia labels. We use D, Cardiology ECG (Hnnun et al. 2019) (12-way), D2 = Chapman ECG (Zheng et al. 2020) (4-way), and D3 PhysioNet 2020 ECG (Perez Alday et al. 2020) (9-way, multi-label), Further details regarding the
datasets and network architecture can be found in Appendix C
Our code is available https//tiiyunl.oomc...SSSSSmmmis
4

---

Under review as a conference paper at ICLR 2021
5.2 CONTINUAL LEARNING SCENARIOS
Here, we outline the three primary continual learning scenarios we use for our experiments. In Class-
IL, D is split according mutually-eeclusive pairs of classes [0, 1J, [2, 3). [4. 5], [6, 7]. [8, 9]. and [10, 11]. This scenario allows to evaluate the sensitivity of a network to new classes. In Time-IL. D2 is split into three tasks; Term 1, Term 2, and Term corresponding to mutually- excluusve times of the year during which patient data were collected. This scenario allows to evaluate the effect of temporal non-stationarity on a network's performance. Lastly, Domain-II, D3 split according to the 12 leads of an FCG; 12 different projections of the same electrical signal generated by the heart. This scenario allows us to evaluate how robust a network is to the input distribution.
5.3 BASELINE METHIOSS
We compare proposed method to the following. Multi-Task Learning (MTL) (Caruana 1993 strategy whereby all datasets are assumed to be available at the same time and thus can be simultaneously used for training. Although this assumption may not hold in clinical settings due to the nature of data collection, privacy or memory constraints, it is nonetheless a strong baseline. Fine-tuning is strategy that involves updating all parameters when training subsequent tasks as they arrive without explicitly dealing with catastrophic forgetting. We also adapt two replay-based methods for scenarios. GEM (Lopez-Paz & Ranzato 2017) solves quadratic programming problem to generate parameter gradients that do not increase the loss incurred by replayed instances. MIR (Aljundi al. 2019a) replays instances from buffer that incur the greatest change in loss given parameter pseudo-update. Details on how these methods were adapted are found in Appendix
5.4 EVALUATION METRICS
To evaluate our methods, we exploit metrics suggested by Lopez-Paz & Ranzato (0117) such as average AUC and Backward Weight Transfer (BWT). We also propose two additional evaluation metries that provide us with a more fine-grained analysis of learning strategies.
t-Step Backward Weight Transfer. To determine how performance changes 't-steps into the future', we propose BWT, which evaluates the performance of the network on a previously-seen task, after having trained on t-tasks after it.
N N
1
BWT,
R
R
(8)
1=1
Lambda Backward Weight Transfer. We extend BWT, all time-steps, i, generate BWT,, As result, we can identify improvements in methodology at the task-level.
N Σ
BWT
R
R
N
1=1
5.5 HYPERPARAMETERS
Depending on the continual learning scenario, we chose 20 or 40, as we found that achieve strong performance on the respective validation sets. We chose TMC 40 and the sample epochs Ts = 41 + n where n N+ in order to sample data from the buffer at every epoch following the first task. The values must satisfy 7s > TMC
For computational reasons, we chose the storage fraction = 0.1 of the size the training dataset and the acquisition fraction a = 0.25 of the number of samples per task in the buffer. To calculate the acquisition function, chose the number of Monte
Carlo samples, = 20. We chose the regularization coefficient, 10. We also explore the effect of changing these values performance (see Appendices and M).
5

---

Under review as a conference paper at ICLR 2021
6 EXPERIMENTAL RESULTS
6.1 CLASS-IL
Destructive interference is notorious amongst neural networks. In this section, quantify such interference when learners are exposed to tasks involving novel classes. In Fig. Ga we illustrate the AUC achieved sequential binary classification tasks. We find that destructive interference is prevalent. For example, the network quickly forgets how to perform task [0 -0 1] once exposed to data from task [2 3]. This can be seen by the AUC s 0.92 - 0.30. The final performance of the network for that particular task (AUC x 0.78) is also lower than that maximally wawiiiall--acieeed.. In Fig, 3b we show that CLOPS alleviates this interference. This can be seen by the absence of significant drops in AUC and higher final performance for all tasks relative to the fine-tuning strategy.
In Table D we compare the performance of the CL strategies in the Class-IL scenario. We find that CLOPS outperforms MTT. (AUC 0.796 vs. 0.701), which is seemingly non-intuitive finding. We hypothesize that this finding is due to positive weight transfer brought about by curriculum wherein sequential tasks of different levels of difficulty can improve generalization performance (Bengio et al., 2009). We explore this hypothesis further Sec. 6.5 We also find that CLOPS outperforms state-of-the-art methods, GEM and MIR, in terms of generalization performance and exhibits constructive interference. For example, CLOPS and MIR achieve an AUC = 0.796 and 0.753, respectively. Moreover, BWT = 0.053 and 0.009 for these two methods, respectively. Such finding underscores the ability of CLOPS to deal with tasks involving novel classes. We also show that CLOPS is robust task order (see Appendix
4-5
0-9
10-11
60
(a) Fine- sine-tuning
(b) CLOPS
Figure Mean validation AUC of the a) fine-tuning and b) CLOPS strategy 0.25 and 0.50) in the Class-IL scenario. Coloured blocks indicate tasks on which the learner is currently being trained. The shaded area represents one standard deviation across five seeds.
Table 1: Performance of CL strategies in the Class-IL IL scenario. Storage and acquisition fractions b = 0.25 and a = 0.50, respectively. Mean and standard deviation are shown across five seeds.
Method
Average AUC
BWT
BWT,
BWT,
MTL
0.701 + 0.014
Fine-tuning
0.770 + 0.020
0.037 0.037
(0.076) + 0.064
(0.176) 0.080
Replay-based Methods
GEM
0.544 = 0.031
(0.024) = 0.028
(0.046) 0.017
(0.175) 0.021 (0.046) 0.022
MIR CLOPS
0.753 0.014
0.009 0.018
0.001 + 0.025
0.796 = 0.013
0.053 1 0.023
0.018 + 0.010
0.008 = 0.016
6.2 TIME-IL
Environmental changes within healthcare introduce seasonal shift into datasets. In this section, we quantify the effect of such a shift on learners. In Fig, 4al we illustrate the AUC achieved on tasks with seasonally-shited data.

---

Under review as a conference paper at ICLR 2021
Term
Term 2
Term 3
04
(a) Fine-tuning
(b) CLOPS
Figure Mean validation AUC of the (a) inn-tuning and (b) CLOPS strategy in the Time-IL scenario. Coloured blocks indicate tasks on which the learner is currently being trained. The shaded area represents one standard deviation from the mean across five seeds.
In this scenario, we find that CLOPS is capable of achieving forward weight transfer (FWT). For example, in Figs. Am and 4b. CLOPS achieves an AUC = 0.62 after one epoch of training on task Term 3, value that the fine-tuning strategy only achieves after 20 epochs, signalling 20-fold reduction in training time. attribute this FWT to the loss weightiig role played by the task-instance parameters. By placing greater emphasis on more useful instances, the generalization performance of the network is improved. We also find that CLOPS exhibits reduced catastrophic forgetting relative to fine-tuning. For example, performance on tasks Term and Term 2 is maintained at AUC 0.90 when training task Term 3. We do not observe this for the fine-tuning setup.
6.3 DOMAIN-IL
So far, we have shown the potential of CLOPS to alleviate destructive interference and allow for forward weight transfer. In this section, and in Table 2 we illustrate the performance of the CL strategies in the Domain-II. scenario. We show that CL.OPS outperforms sfate-of-the-art methods. For example, CLOPS and MIR achieve an AUC = 0.731 and 0. 016,,, respectively. CLOPS is also better mitigating destructive interference, as shown by BWT (0.011) and (0.022), respectively, We provide an explanation for such performance by conducting ablation studies in the next section.
Table 2: Performance of CL strategies in the Domain-IL. scenario. Storage and acquisition fractions
are 6 = 0.25 and a = 0.50, respectively. Mean and standard deviation are shown across five seeds.
Method
Average AUC
BWT
BWT.
BWT,
MTL Fine-tuning
0.730 = 0.016
0.687 = 0.007
(0.041) = 0.008
(0.047) + 0.004
(0.070) 0.007
Replay-based
Methods
GEM
0.502 + 0.012
(0.025) + 0.008
0.004 0.010
(0.046) + 0.021
MIR
0.716 + 0.011
(0.022) + 0.011
(0.013) + 0.004
(0.019) 0.006 (0.019) 0.009
CLOPS
0.731 = 0.001
(0.011) 0.002
(0.020) 1 0.004
6.4 EFFECT OF TASK-INSTANCE PARAMETERS, 8, AND ACQUISITION FUNCTION,
To better understand the root cause of CTOPS' benefits, we conduct three ablation studies: 1) Random Storage dispenses with task-instance parameters and instead randomly stores instances into the buffer, 2) Random Acquisition dispenses with acquisition functions and instead randomly acquires instances from the buffer, and 3) Random Storage and Acquisition which stores instances into, and acquires instances from, the buffer randomly. In Fig. 5] we illustrate the effect of these strategies on performance as we vary the storage fraction, b. and acquisition fraction, a.
We find that 3, as a loss- weighting mechanism, benefits generalization performance. For example, in Fig. 5 (red rectangle), at b 1, a 0.5, show that simply including the loss-weighting
7

---

Under review as a conference paper at ICLR 2021
mechanism 1 AUC 12%. We hypothesize that this mechanism is analogous attention being placed on instance losses and thus allows the network to learn which instances to exploit further. We also find that uncertainty- based acquisition functions offer significant improvements. In Fig. [ (black rectangles), at a = 0.1, = 0.5, we show that such acquisition 1 AUC x 8%. We arrive at the same conclusion when evaluating backward weight transfer (see Appendix
CLOPS
Random Storage
0.1
75.8
77.0
75.8
75.8
79.6
79.1
79.9
79.2
Fraction, 0.23
Storage Fraction,
78.8
78.9
79.6
80.0
0.25
80
80.6
80.4
81.2
0.5
0.5
1.0
82.1
83.3
82.6
83.2
1.0
80
79.6
77.0
81.1
0.1
0.25
0.5
1.0
0.1
0.25
0.5
1.0
Acaulsition Fraction.
Acquisition Fraction.
Random Acquisition
Random Storage and Acquisition
0.1
74.5
77.6
90.4
75.6
0.1
70.3
68.9
67.2
68.9
Fraction, 0.25
Storage Fraction,
80
Average AUC
77.1
78
80.3
80.0
0.25
71.8
73.0
71.5
71.9
75
0.5
0.5
1.0
81.0
82.9
84.0
83.0
1.0
73.0
72.4
12.3
71.7
0.1
0.25
0.5
1.0
0.1
0.25
0.5
1.0
65
Acquisition Fraction,
Acquisition Fraction,
Figure Mean validation AUC of four different learning strategies in the Class-IL scenario. CLOPS, 2) Random Storage, 3) Random Acquisition, and 4) Random Storage and Acquisition. Results are shown as function of storage fractions, b, and acquisition fractions, a and are an average across five seeds. We highlight the utility of (red rectangle) task-instance parameters as a leishweig mechanism and (black rectangle) uncertainty-buee based acquisition functions for the acquisition of instances from the buffer.
6.5 VALIDATION OF INTERPRETATION OF TASK-INSTANCE PARAMETERS, /
We claimed that instances with lower values of 8, and by extension, relatively more difficult to classify. In this section, we aim to validate this intuition. In Fig. 6 we illustrate the distribution of values corresponding each task.
0-1
4-5
0-9
10-11
Normal Sinus Rhythm
Normal Sinus Rhythm
Mastier theher uennee
55
ST ST-elevation
Lowest 5 value
Highest s value
Vilve of TrGaoed lack PICSDDE PAANWER
Figure 6: Distribution of the values corresponding CLOPS (b 0.25 and 0.50) in the Class-IL scenario. Each colour corresponds to a different task. The ECG recording with the lowest value is labelled normal despite the presence of ST-elevation, feature common in heart attacks.
8

---

Under review as a conference paper at ICLR 2021
We find that tasks differ in their difficulty level. For example, task [6 - 7] is considered more difficult to solve than task [8 - 9] as evidenced by the lower distribution mean of the former relative to the latter (s =e 18.85 and highest 8 values, we find that both belong to the same class, normal sinus rhythm. Upon closer inspection, the recording with the lower value exhibits a feature known ST-elevation. This feature, which is characterized by the elevation of the segment between the S and T waves (deflections) of the ECG signal relative the baseline, is typically associated with heart attacks. Mapping ECG recording with such an abnormal feature to the normal class would have been source of confusion for the network. We provide additional qualitative evidence in Appendix G
18.95). After extracting the two ECG recordings associated with the lowest
We also leverage to learn curriculum (Ben- gio et al. 2009).. First, we fit Gaussian. N N o.)) ). to each of the distributions in Fig. 6 Using this information, we define the difficulty of task T as dr
1.0
01
2.3
_ and the similarity. PT
D
0.6
Tork Simmeer
k)j,k), between task j and i as shown in Eq. 10 In [ig.77] we illustrate the resulting pairwise task similarity matrix.
6.7
F
8.
C
0.2
10-11
B
20001 ooo
4
0-1
2-3
4-5
8-9
10-11
S(, =1
(10)
Figure 7: Similarity of tasks in the Class-II. scen- ario. We create a curriculum by following chaining tasks that are similar to one another.
DH Hellinger Distance
We design curriculum by first selecting the easiest task (1 dr) and then creating a chain of tasks that are similar to one another as shown in Fig. 7 For an anti- anti-curriculum, we start with the hardest task (1 d)). In Table we illustrate the performance of various curricula and find that a curriculum exhibits higher constructive interference than a random one (BWT 0.087 vs. 0.053). Such an outcome aligns well with the expectations of curriculum learning, thus helping to further validate the intuition underlying 3.
Table 3: Performance of CLOPS in the Class-IL scenario with different curricula. Storage and acquisition fractions are b- 0.25 and 0.50, respectively. Results shown across five seeds.
Task Order
Average AUC
BWT
BWT,
BWT,
Random
0.796 + 0.013
0.053 = 0,023
0.018 + 0.010
0.008 = 0.016
Curriculum
0.744 + 0.009
0.087 = 0.011
0.038 + 0.021
0.076 = 0.037
Anti-curriculum
0.783 + 0.022
0.058 + 0.016
(0.013) + 0.013
(0.003) + 0.014
7 DISCUSSION AND FUTURE WORK
In this paper, we introduce a replay- based method applied to physiological signals, entitled CLOPS, to mitigate destructive interference during continual learning. CLOPS consists of an importance-guider buffer-storage and letive-learning inspired auquii-acqqiiition mechanism. We show that CLOPS outperforms the state-of-the-ar methods, GEM and MIR, on both backward and forward weight transfer. Furthermore, we propose learnable parameters, as a proxy for the difficulty with which instances are classified, which can assist with quantifying task difficulty and improving network interpretability. We now elucidate future avenues worth exploring.
Extensions to Task Similarity. The notion of task similarity was explored by Thrun & O'S Sullivan (199)); Silver & Mercer (1996). In this work, we proposed definition of task similarity and used to order the presentation of tasks. The exploration of more robust definitions, their validation through domain knowledge, and their exploitation for generalization is an exciting extension.
Predicting Destructive Interference. Destructive interference is often dealt with in a reactive man- ner. By predicting the degree of forgetting that a network may experience once trained sequentially can help alleviate this problem in proactive manner.

---

Under review as a conference paper at ICLR 2021
REFERENCES
Rahaf Aljundi, Eugene Belilovsky, Tinne Tuytelaars, Laurent Charlin, Massimo Caccia, Min Lin, and Lucas Page-Caccia. Online continual learning with maximal interfered retrieval. In Advances in Neural Information Processing Systems, pp. 1 1849-1191 1 18660, 2019a.
Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio. Gradient based sample selection for online continual learning. In Advances in Neural Information Processing Systems, pp. 11816- 11825, 2019b.
Yoshua Bengio. Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In Proceedings of the 26th Annual International Conference on Machine Learning, pp. 41-48, 2009.
Richard A Caruana. Multitask connectionist learning. In In Proceedings of the 1993 Connectionist Models Summer School. Citeseer, 1993.
Arslan Chaudhry, Marc Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with a-gem. arXiv preprini arXiv:1812.00420, 2018.
Yukun Chen, Robert Carroll, Eugenia R McPeek Hinz, Anushi Shah, Anne Eyler, Joshua C Denny, and Hua Xu. Applying active learning to high-throughput phenotyping algorithms for electronic health records data. Journal of the American Medical Informatics Association, 20(+2): e253-e259, 2013.
Sebastian Farquhar and Yarin Gal. Towards robust evaluations of continual learning. arXiv preprint arXiv:1805.09933, 2018.
Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In International Conference on Machine Learning, pp. 1050-1059, 2016.
Yarin Gal, Riashat Islam, and Zoubin Ghahramani. Deep Bayesian active learning with image data. In Proceedings of the 34th International Conference on Machine Learning-Von Volume 70, pp. 1183-1192. JMLR. org, 2017.
Wenho Gong, Sebastian Tschiatschek, Richard Turner, Sebastian Nowozin, and José Miguel Hernández-LLoatt Icebreaker: element-wise wise active information acquisition with bayesian deep latent gaussian model. arXiv preprint arXiv: 1908. 04537, 2019.
Awni Y Hannun, Pranav Rajpurkar, Masoumeh Haghpanahi, Geoffrey H Tison, Codie Bourn, Mintu P Turakhia, and Andrew Y Ng. Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network. Nature Medicine, 25(1):65, 2019.
Neil Houlsby, Ferene Huszár, Zoubin Ghahramani, and Máté Lengyel. Bayesian active learning for classification and preference learning. arXiv preprint arXiv: 1/2.5745 2011.
David Isele and Akansel Cosgun. Selective experience replay for lifelong learning. In Thirty-second AAAI Conference Artificial Intelligence, 2018.
Alistair EW Johnson, Tom J Pollard, Lu Shen, H Lehman Li-wei, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. Mimic-IlI, freely accessible critical care database. Scientific Data, 3:160035, 2016.
Dani Kiyasseh, Tingting Zhu, and David A Clifton. Alps: Active learning via perturbations. arXiv preprint arXiv:2004. 20w.000004..57.,,...0 2020.
Matthias Lenga, Heinrich Schulz, and Axel Saalbach. Continual learning for domain adaptation in chest x-ray classification. arXiv preprint arXiv:2001.05992,, 2020.
David Lopez-Paz and Mare' Aurelio Ranzato. Gradient episodic memory for continual learning. In Advances in Neural Information Processing Systems, pp. 6467-6476, 2017.
German Parisi, Ronald Kemker, Jose L Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: A review. Neural Networks, 2019.
10

---

Under review as a conference paper at ICLR 2021
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, nigh-performance deep learning library. In Advances in Neural Information Processing Systems, pp. 8024-8035, 2019.
E. A. Perez Alday, A. Gu, A. Shah, C. Liu, A. Sharma, S. Seyedi, A. Bahrami Rad, M. Reyna, and G. Clifford. Classification of 12-lead ECGs: the PhysioNet computing in cardiology challenge 2020 (version 1.0.1). PhysioNet, 2020.
Joaquin Quionero-Candela, Masashi Sugiyama, Anton Schwaighofer, and Neil D Lawrence. Dataset shift in machine learning. The MIT Press, 2009.
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference Computer Vision and Pattern Recognition, pp. 2001-2010, 2017.
David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Experience replay for continual learning. In Advances in Neural Information Processing Systems, pp. 348-358, 2019.
Shreyas Saxena, Oncel Tuzel, and Dennis DeCoste. Data parameters: A new family of parameters for learning differentiable curriculum. In Advances Neural Information Processing Systems, pp. 11199-191100.. 2019.
Burr Settles. Active learning literature survey. Technical report, University of Wisconsin-Madisonn Department of Computer Sciences, 2009.
Daniel L Silver and Robert E Mereer. The parallel transfer of task knowledge using dynamic learning rates based on measure of relatedness. Learning learn, pp. 213-233. Springer, 1996.
Asim Smailagic, Pedro Costa, Hae Young Noh, Devesh Walawalkar, Kartik Khandelwal, Adrian Galdran, Mostafa Mirshekari, Jonathon Fagert, Susu Xu, Pei Zhang, al. Medal: Accurate and robust deep active learning for medical image analysis. In IEEE International Conference on Machine Learning and Applications, pp. 481-488, 2018.
Asim Smailagic, Pedro Costa, Alex Gaudio, Kartik Khandelwal, Mostafa Mirshekari, Jonathon Fagert, Devesh Walawalkar, Susu Xu, Adrian Galdran, Pei Zhang, et al. O-medal: Online active deep learning for medical image analysis. arXiv preprint arXiv: 10508, 2019.
David Spiegelhalter. Should we trust algorithms? Harvard Data Science Review, 2(1), 2020, doi: 10. http:///lssmmiipeees...t.lcuuuup
1162/996081922.c911233 URL http::tts/:hht www.mmiitsss.......mm wwt.sssiiteeeeee wtt.....///pp////..en.....
Sebastian Thrun and Joseph O' Dullliva.. Discovering structure multiple learning tasks: The to algorithm. In ICML, volume 96, pp. 489-497, 1996.
Gido M van de Ven and Andreas Tolias. Three scenarios for continual learning. arXiv preprint arXiv: 077344 2019.
Guijin Wang, Chenshuang Zhang, Yongpan Liu, Huazhong Yang. Dapeng Fu, Haiqing Wang, and Ping Zhang. A global and updatable eog beat classification system based on recurrent neural networks and active learning. Information Sciences, 501:523-542, 2019.
Jianwei Zheng, Jianming Zhang, Sidy Danioko, Hai Yao, Hangyuan Guo, and Cyril Rakovski. A 12-lead electrocardiogram database for arrhythmia research covering more than 10,000 patients. Scientific Data, 7(1):18 2020.
Zongwei Zhou, Jae Shin, Lei Zhang, Suryakanth Gurudu, Michael Gotway, and Jianming Liang. Fine- tuning convolutional neural networks for biomedical image analysis: actively and incrementally. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 7340-7351, 2017.
Xiaojin Jeny Zhu. Semi-supervised learning literature survey. Technical report, University of Wisconsin-Madison Department of Computer Sciences, 2005.
11


Reviews:
[{"review_id": "Vqg_phL9tYC", "reviewer": "AnonReviewer5", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "## Summary\nThis paper proposes a replay-based continual learning method and applies it to physiological data.\nThe proposed method consists of two heuristics to manage the replay memory:\n- Learn a weight parameter (task-instance parameter) for each data point by minimizing a special loss function, and store the data with large weight parameters.\n- Periodically compute epistemic uncertainty for all data in the replay memory, and replay the data with large uncertainty.\n\n---\n\n## Pros\n\nThis paper points out the importance of continual learning in the medical field.\n\n---\n\n## Cons\n\n### Weak contribution\n\nSince this paper deals with the continual learning of a specific type of data, I think there are two directions in which this study can be meaningful:\n- Test various CL algorithms in the new domain, determine which method performs better than others, and analyze the reason.\n- Use domain knowledge to design a new algorithm that performs particularly better on the domain.\n\nHowever, I think this paper belongs to neither of the two. There is no domain-specific component in the algorithm, and only two baselines are compared.\n\nIf the method is general enough to be used in other domains, I think the authors should have also tested standard CL scenarios such as Split-CIFAR10 or Split-CIFAR100.\n\n### Lack of theoretical justification\n\nThe proposed method is mostly heuristic and does not have a theoretical basis. Especially eq. (2) seems too arbitrary. Since this paper's empirical evidence is weak, I think a theoretical analysis is necessary to support the algorithm's efficacy.\n\n### Vague and improper definition of time incremental learning\n\nThe authors seem to propose time incremental learning for the first time, but the definition is too vague:\n> Time Incremental Learning (Task-IL) - the same dataset and prediction problem are used for each task, however the time of year at which the data were collected differs from one task to another. Such seasonality is most common in healthcare applications.\n\nThe authors should specify what changes over time. However, I think there is even a bigger problem:\n- If the input distribution changes, it is not different from domain incremental learning.\n- If the output distribution changes for the same input, it is not continual learning. The model *should* forget old tasks and adapt to new tasks.\n- If both the input and output distributions change, it is class incremental learning.\n\nTherefore, I think time incremental learning is not a novel branch of continual learning. The corresponding experiments should better be reformulated as domain incremental learning or class incremental learning.\n\n---\n\n## Overall evaluation\n\nI do not think this paper proposes a novel idea with either a solid theoretical basis or strong empirical results. Therefore, I recommend rejection.\n\n---\n\n## Post rebuttal\nDuring the rebuttal, the authors failed to handle the issues that I raised. Especially, the authors did not respond to my criticisms about the strange experiments. Therefore, I stick to my initial rating.", "extra_scores": {}, "extra_text": {}}, {"review_id": "M2-p8cWciVL", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors propose a learning methodology designed to offset detriments to algorithm performance that arise when instances are not i.i.d (independent and identically distributed), focusing on cases in continual learning (CL) given physiological signals. They designed a replay-based learning method that handles an instance buffer using Importance-guided Storage and Uncertainty-based Acquisition strategies. They apply their method on Class, Time and Domain types of CL, and they introduce t-Step Backward Weight Transfer and  Lambda Backward Weight Transfer methods by which to evaluate their method. They conclude with two ablation studies to explore an explanation for their method’s performance and attempt to validate their hypotheses based on these studies.\n\nSuggestions: \n* Make more explicit, in 4.1.2 (regarding equation 5), which type of task is being referred to, when comparing between prior and present tasks.\n* If time permits, a figure for Experimental Results section 6.3 (to correspond to figures provided in 6.1 and 6.2) would be nice.\n* A brief explanation should be included for the -1 in the regularization term in equation (2)\n* In section 4.1.2, the need for a storage function was well motivated; however, the particular solution that the authors chose needs more motivation (and comparison with possible alternatives such as \\beta^2. Not suggesting that this be empirically verified.)\n* In Figure 2, SDs were calculated over 5 seeds. Was this low number due to the computational cost of experimentation?\n* For a lay ML practitioner to appreciate the assertion made in section 6.5, \"ST-elevation\" needs to be defined more concretely.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "DKBbtKSwthi", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "**Update**\nI think the manuscript has been further improved now and I improve my score to 7. Also since I think results on public datasets in new domains other than images may be helpful for the wider research community. Keep in mind, as written below, that other reviewers more familiar with this research field may be more  able than me to judge the evaluation quality and whether the evaluation contains too substantial flaws to allow publication. \n\n**Disclaimer**\nUpon reviewing the manuscript, I found I am unfortunately not well-acquainted enough with the research fields (active learning/continual learning) to be able to evaluate very well what this manuscript contributes to these fields. I hope my review can still be useful.\n\n**Summary**\nThe manuscript describes and evaluates a continual learning algorithm that learns per-instance scalars beta that weight the loss of each instance. These betas are optimized together with all the other parameters, using an additional regularization term that prevents all betas to be optimized to zero. The betas averaged over time are used to decide which instances get into the replay buffer (easier ones are used). Additionally, to decide which instances  from the replay buffer to train on, Monte Carlo Dropout is used to estimate the uncertainties of the network on each instance and more uncertain instances are then more likely to be sampled for training. The approach is evaluated on three healthcare datasets, in either a setting where sequentially different tasks are learned by the network, the same task is learned with data sequentially used across time (affected by seasonal changes), or the same task is learned while sequentially shifting the domain/input distribution in a controlled way (different projections of same ECG signal). The proposed approach outperforms two baselines with regard to performance on previously-trained-on tasks. Ablation studies show the role of the storage strategy and the acquisition strategy while validating the interpretation of the betas as instance difficulties.\n\n**Main Impression**\nThe approach is clearly described, it was fairly easy to understand for me, the writing is nice. The evaluation seems interesting; it would be more convincing to me if it was also evaluated on some dataset where published results of other works are available. Ablation studies are nice, the results of them could be discussed more.\n\n**Major Points**\n\nAs written above, I would really like to see an evaluation of this approach where one can directly compare to published results by others. Reimplemented baselines always have a risk of being not tuned well enough etc. And as far as I could understand, there does not seem to be anything so healthcare-specific in the algorithm itself that comparison in other domains would not make sense, in case that is the problem.\n\nI like the ablation studies and would have enjoyed more discussion of their results. \n- It seems for b=0.1 and b=0.25, random storage always outperforms CLOPS? Why? b=0.5 also mostly outperforms. \n- Similar, for a=0.5 and a=1.0, it seems that random acquisition mostly outperforms CLOPS, for a=0.25 it is still close, why?\n- What happens for random acquisition and random storage? Was this tested?\n\n\n **Minor Points**\n\n*Some parts of introduction I didn’t agree or find too broad*\n\n“Deep learning algorithms operate under the assumption that instances are independent and identically-distributed (i.i.d.). “\n-> not really specific to deep learning at all to me. Obviously one can apply deep neural networks also under different assumptions. Would either write: “Many machine learning and deep learning algorithms ...”, Or simply “Many deep learning algorithms…”\n\n“Given the potential impact of designing such an algorithm and the machine learning community’s efforts towards achieving artificial general intelligence, research on continual learning has increased (Parisi et al., 2019).”\n-> a bit too broad stuff to me, could be just removed (AGI etc.)\n\n*Confusing beta zero part*\n\n“In this setup, we discovered β_iT quickly decay to zero, and as a coefficient of the loss, prevents the network from learning the task”\nThis made me a bit confused: As per formula, if you set all betas to 0, the loss would become zero no? So it should be a direct consequence of the formula that yes, gradient descent or any optimization should set all betas to zero without any regularization/constraints? The “discovered” confused me here. Or am I misunderstanding something here?\n\n*Figure improvements*\n\nFigure 2: thicker curves? Hard to see\n\nFigure 5 in appendix: legend would be helpful\n\n\n\nAs written above, I don’t feel well-acquainted enough with the literature on continual/active learning to judge the contribution of this manuscript to these fields with confidence; hope other reviewers can do this :)\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "8h1vP7FyXS3", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper presents, CLOPS, a new replay-based continual learning (CL) strategy for physiological signals.\n\nThe paper has the following merits. It is well-organized. I can understand its method, different settings of continual learning, and more ablation study on the model parameters. It did through experiments on the 3 datasets and get good results showing CLOPS beats naive baselines like multi-tasks learning, Fine-tuning as well as other CL methods like GEM, MIR.  \n\nOn the other side, I have the following concerns about the submission.\n\n1. what is the task exactly?\n\nIs cardiac arrhythmia label also time series? Does it have the same sampling frequency as physiological data?\nIn the Class-IL setting, are you doing multi-class classification or binary class? The notation is confusing. Is task [0,1], task [2,3] meaning binary classifications of class 0 v.s  class 1 and class 2 v.s. class 3? Then what does task [0-2] mean? It appears in the second paragraph of section 6.1.\n\n2. not enough highlight for important parameters (storage and acquisition fractions, b, and a).\n\nStorage and acquisition fractions, \"b\" and \"a\" are important parameters of replay-based CL algorithms. However currently, the definition of them is hidden in section 4 and figure captions. They are important parameters. The author even did an ablation study in section 6.4 on it. However, when I first read the paper, after reading the method section, I had no impression of \"a\" and \"b\".\n\n3. not well explained experimental result. \n\nIn Table 1, it shows Fine-tuning works better than Multi-task learning. It is a bit counter-intuitive to me since MTL has stronger supervision at all the time. Do there exist differences between task difficulty? It seems the task order influences the performance of fine-tuning according to the result in Table 1 and Table 9 in appendix G. Can the author provide any explanation?\nIn section 6.2, the author argues that CLOPS achieves an AUC after one epoch while fine-tuning requires 20 epochs to achieve that. First, I am not sure whether it is true since it seems the performance of the fine-tuning is not converged yet, its performance may still increasing. Second, clearly, CLOPS has a much larger performance variance than fine-tuning.  So I think the current result does not support that CLOPS is better than fine-tuning. \n\n4. unclear usage of task similarity.\n\nIn the appendix, the author shows the task similarity produced by the model. However, I am not sure what we can do with such task similarity. In the abstract, the author claim \"this quantification yields insights into both network interpretability and clinical applications\". However, in the paper, I can not see any arguments about what insight of network interpretability and clinical applications we can get.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Reviewers could not reach consensus here and important concerns from one reviewer on empirical results could not be convincingly addressed. The authors have provided a comprehensive response to the reviews, yet failed to convince them.

Decision: Reject