Simulating Task-Free Continual Learning Streams From Existing Datasets
Authors: Aristotelis Chrysakis
Marie-Francine Moens
Venue: Submitted to ICLR 2023 (2023)
Keywords: Task-Free Continual Learning

Abstract: Task-free continual learning is the subfield of machine learning that focuses on learning online from a stream whose distribution changes continuously over time. However, previous works evaluate task-free continual learning using streams with distributions that change only at a few distinct points in time. In order to address the discrepancy between the definition and evaluation of task-free continual learning, we propose a principled algorithm that can permute any labeled dataset into a stream that is continuously nonstationary. We empirically show that the streams generated by our algorithm are less structured than the ones conventionally used in the literature. Moreover, we use our simulated task-free streams to benchmark multiple methods applicable to the task-free setting. We hope that our work will make it more likely that task-free continual learning methods are able to better generalize to real-world problems.

Under review as a conference paper at ICLR 2023
SIMULATING TASK-FREE CONTINUAL LEARNING STREAMS FROM EXISTING DATASETS
Anonymous authors Paper under double-blind review
ABSTRACT
Task-free continual learning is the subfield of machine learning that focuses on learning online from a stream whose distribution changes continuously over time. However, previous works evaluate task-free continual learning using streams with distributions that change only at a few distinct points in time. In order to address the discrepancy between the definition and evaluation of task-free continual learn- ing, we propose principled algorithm that can permute any labeled dataset into a stream that is continuously nonstationary. We empirically show that the streams generated by our algorithm are less structured than the ones conventionally used in the literature. Moreover, we our simulated task-free streams to benchmark multiple methods applicable to the task-free free setting. We hope that our work will make it more likely that task-free continual learning methods are able to better generalize to real-world problems.
INTRODUCTION
The dominant paradigm in the field of machine learning involves building a model using a static set of pre-collected data (Mitchell, 1997; LeCun et al., 2015). Unfortunately, it might not be always possible to stick to this paradigm. For instance, animals and humans extract knowledge from their observations continually, and under changing circumstances (Parisi et al., 2019). The field of con- tinual learning studies exactly this data provided by nonstationary distribution (Aljundi et al., 2019c; Chrysakis & Moens, 2020).
how to train a machine learning model using
Within the continual learning literature, different underlying assumptions give rise to a number of distinct continual learning settings. Such assumptions might be about whether the data distribution is continuously nonstationary or not, or about whether the model optimization takes place online (with small minibatches of data) or offline (with large batches of data) (De Lange et al., 2021). In this paper, we focus on task-free continual learning, which we consider to be the setting closest to how humans and animals learn. In task-free continual learning, the data distribution is assumed to be continuously nonstationary and the optimization takes place online (Aljundi et al., 2019b).
The observation that motivated this work is that there is large gap between how task-free continual learning is defined and how it is evaluated. In fact, previous works evaluate task-free continual learning using streams with data distributions that are not continuously nonstationary. but change only at a few distinct moments in time and remain stationary otherwise (Aljundi et al., 2019b; Jin al., 2021). With this work, we aim to bridge the gap between the definition and the evaluation of task-free continual learning.
Our contributions are the following. First, we provide a principled algorithm that can reorder any labeled dataset into a simulated task-free (STF) continual learning stream. This algorithm was de- signed with the goal of introducing as little design bias as possible to the streams constructs. Second, perform detailed comparison between STF streams generated by the proposed algo- rithm and the type of streams conventionally used in previous works. Via this comparison, we detail number of different ways the streams conventionally used different our STF streams. Third, we transform four well-known datasets into STF streams, and use them to benchmark a number of methods applicable task-free continual learning.
The remainder of the paper is structured as follows. In Section 2, we provide an introduction to con- tinual learning and online continual learning, and extensively discuss the gap between the definition

---

Under review as a conference paper at ICLR 2023
and evaluation of task-free continual learning. In Section 3, present algorithm for generating STF streams, and motivate its design. In Section 4, we present and discuss our experiments, and, finally, in Section 5, we summarize our work, discuss its limitations, and offer future perspective.
BACKGROUND
2.1 CONTINUAL LEARNING
general, continual learning is defined as learning from data that are generated by a nonstationary distribution, that is to say, a distribution that changes over time (Zenke et al., 2017; Kurle et al., 2019; Chrysakis Moens, 2020). An alternative definition of continual learning the learning of sequence of tasks over time (Van de Ven & Tolias, 2019; Prabhu et al., 2020; De Lange et al., 2021). But, what is task?
In
In the context of continual learning, the term task is generally used to describe collection of data which the model observes an independent and identically distributed (iid) manner. Tasks are often assumed to be class-disjoint, that is, if data from a particular class appear in a task, data from the same class will be present in any other task (Van de Ven & Tolias, 2019; Prabhu et al., 2020). Previous works sometimes assume access to task labels, which explicitly inform the learner to which task each data point belongs (Nguyen et al., 2018; Zenke et al., 2017). The setting of flass-incremental continual learning assumes that task labels only given during training, while the setting of task-incremental continual learning assumes access to task labels both during training and during evaluation (Van de Ven & Tolias, 2019; De Lange al., 2021; Lomonaco & Rish, 2021).
Besides access to task labels, another distinction can be made with regard to whether continual learning takes place online or offline. In the offline setting, the learner has access all data from the present task and can perform multiple passes over these data (De Lange et al., 2021; Prabhu et al., 2020). Conversely, in the online setting, the learner receives data from a nonstationary stream in the form of small minibatches, and only has access to one of those minibatches at a time (Aljundi et al., 2019c; Chrysakis & Moens, 2020; Cai et al., 2021).
2.2 ONLINE CONTINUAL LEARNING SETTINGS
To avoid potential confusion, we offer precise definitions for online, task task-agnostic, and task free continual learning. First, online continual learning has evolved! to be an umbrella term that encom- passes all settings in which a model should be trained online using small minibatches of data that generated by nonstationary stream (Aljundi et al., 2019;; Pham et al., 2020; Yin et al., 2021).
Task- aggostac and task-free continual learning are both types of online continual learning. In task- agnostic continual learning the stream is assumed to be a sequence of tasks but without task labels being available. In other words, the stream consists of a number contiguous iid sub-streams (each one corresponding
task), and the distribution only changes when there is transition from one sub-stream to the next. In this setting, however, it is relatively easy to infer task labels during training (Zeno et al., 2018: Kirichenko al., 2021).
Finally, in task-free continual learning the concept of a data distribution that changes at distinct points during learning. is generalized one that changes constantly over time (Aljundi et al., 2019b). Therefore, in task-free stream, there are no iid sub-streams, hence the concepts of tasks, task labels, and task boundaries cannot be defined.
2.3 TASK-FREE CONTINUAL LEARNING AND ITS EVALUATION
We argue that, in terms of its applicability, task-free continual learning the most general continual learning setting. To understand why, we need to consider the various aforementioned settings in the context of the simplifying assumptions they make. The most widely adopted assumptions are a) the existence of tasks, b) task labels during training, c) concurrent access to all data from the present
Wee write evolved because online continual learning was originally defined to be a nonstationary online learning problem without access task labels (Aljundi al., 2019c). A number of subsequent works, however, study online continual learning and do assume access task lahels (Pham al., 2020; Yin al., 2021).
2

---

Under review as a conference paper at ICLR 2023
task, and d) task labels during evaluation. Generally speaking, the more simplifying assumptions a setting adopts, the more niche this setting is, but also, the less applicable in real-life situations it becomes. The task-incremental setting assumes all four, the class-incremental setting assumes the first three, and the task-agnostic setting, in theory, assumes only the first (but, as we discussed earlier, task labels during training can be inferred). In the task-free setting, however, there are no simplifying assumptions. Put another way, task-free continual learning adopts the most general definition of continual learning.
To reinforce this point, let us consider the four real-life continual learning scenarios identified by Farquhar& Gal (2018): a) a disease-diagnosis system trained incrementally with data different from different populations; b) a wind-turbine-satte the turbine in order to prevent damage from strong winds; c) a recommender system that learns to serve ads tailored to user's needs and interests; d) a exploration rover that learns to navigate the various terrains of the planet Mars. These examples were meant to describe continual learning in general, but, interestingly, three out of four are task-free continual learning problems (the only exception is the first one). Indeed, the latter three problems all involve data distributions that change continuously over time (seasonal and climate changes, changes in trends and individual interests, and terrain changes, respectively), not in distinct steps. If these examples are any indication, many real-life continual learning problems task-free.
system that learns to predict when to deactivate
Unfortunately, there is a significant discrepancy between how task-free continual learning is de- fined and how it is evaluated. Due to the lack of appropriate task-free benchmarks, previous works proposing methods that do not make assumptions about the nature of the input stream, evaluate their performance on streams that are not continuously nonstationary (Aljundi et al., 2019a; Jin et al., 2021). Therefore, we argue that task-free continual learning should be evaluated using task-free streams. One way achieve this goal, would be to build new ordered datasets from real-world task- free continual learning problems. However, this process is slow and, potentially, very expensive. Instead, we developed a principled algorithm that can transpose any labeled dataset into a stream that is continuously nonstationary. We describe and motivate this algorithm in the following section.
3 METHODOLOGY
3.1 PROBLEM FORMULATION
Let D = {< 9i) Ri) be an arbitrary labeled dataset of size n, where æ; are the data instances and y are their corresponding labels. This dataset contains data instances I; of c distinct classes, that is to say, for 1, i.... it is y/ E (l.
Our goal is to permute the order in which the data instances appear within the dataset, so that when the permuted dataset is broken down in small task-free continual learning streams. In intuitive terms, we want the data distribution of the resulting streams be changing throughout the duration of learning, and not just distinct points in time (as is the case in non-task-free settings). Moreover, we want the resulting streams to contain as little design bias as possible, in order for them to serve benchmarks that generalize adequately to real-world task-free continual learning problems. Since such streams are an attempt to simulate the characteristics of real-world task-free continual learning streams, we will call them simulated task-free (STF) streams.
it approximates the characteristics of
Formally, our goal is to assign to each data instance v; a permutation index pt that specifies which position I; will appear in the permutation. In particular, if = k the data instance F;, which was the i-th instance in the dataset's ' original order, will appear as the k-th instance the permuted order.
We break this problem down into two sub-problems First, in Section 3.2, we discuss how to assign to each class j one-dimensional distribution D, (for all j 1,....0, c). Then, in Section 3.3, we explain how to use the assigned distributions D, to generate dataset permutation.
3.2 ASSIGNING A DISTRIBUTION TO EACH CLASS
Let E [o, 1] be the time during which the continual learning takes place, where we assume that learning starts = 0 and ends at 1. We define the class distributions Dj as distributions over
3

---

Under review as a conference paper at ICLR 2023
the random variable t. At high level, the time distribution (t( of class j will determine how early or late in the stream instances of class j are likely to appear compared to instances of the other classes. For instance, if IE (D, (t)) > [t)z that is, the mean of the time distribution of class is greater than the mean of the time distribution of class 2, then instances of class 1 are more likely to appear in the stream later than those of class 2. In addition, the standard deviation of the time distribution of each class will determine whether its instances are likely to appear more concentrated or more dispersed over the stream.
Given the information in the previous paragraph, there are several questions that need to be answered. We start by describing the principle of maximum en- tropy (Jaynes, 1957a;b), and how we ap- ply in order to assign mean H4, and standard deviation jj to each class j. Subsequently, we discuss and motivate which family of distributions we decided to use. Finally, we explain how to derive the parameters of each class's distribu- tion given its mean j4y and its standard deviation oyy.
Algorithm Assign a distribution to each class.
Number of classes c
Desired average standard deviation fl
1: Find rate such that T-e X7 - ス = Mo
2: for class j in 1,..., do
3: Sample the standard deviation: oj ~ 8(0 1,,))
4: Compute ry V of
5: Sample the mean: Mj U(p 0.5 - r, 0.5 0,5
6: Compute a, μ [1 (1((a -1
The principle of maximum entropy states that when selecting what kind of distribution to use to represent current knowledge about system, out of all the distributions consistent with this cur- rent knowledge, one should select the distribution with the maximum entropy (Jaynes, 1957a;b). Intuitively, the maximum-entropy distribution is the most uninformative distribu- tion consistent with current knowledge. Hence, by choosing the maximum-entropy distribution, the user takes into account only what the current knowledge suggests, without adding any unnecessary bias (Jaynes, 1968).
of
7: Compute 33 = (1 - /k)) [,,((-u)) 1]
8: Set DD = B(a
9: end for
In order to use the maximum-entron principle to sample the means μ.. we need to first consider what our current knowledge about them is. Since the class distributions (1) are defined on the interval [0. 1]. their corresponding means By should also be contained in the same interval. Hence are looking for the maximum-entropy distribution defined over the closed interval [0, 1]. This distribution is the uniform (Udwadia, 1989):
U(p 0, { 1, for [0. 1]
(1)
0, elsewhere.
Now we move on to sampling the standard deviations oj. Once again, we need to consider what our current knowledge suggests. Since the class distributions (t) are defined on the interval [o, 1], must hold that 0 S o. S 0.5 for all j.2 Also, in contrast to how we sample the means, here we would like to be able to manually set the average standard deviation Ho over all classes (in Section 4.4 we show that by changing the value of No. the resulting streams can become easier or harder to learn from). In short, we are looking for the maximum-entropy distribution that is defined on the interval a E [0, 0.5), and of which the mean value is paz This distribution is the truncated-exponentiall (Udwadia, 1989), and is defined as
E(o \\,7) = co E [0,
where is the normalizing constant, and is the truncation parameter, which in case is set to 0.5. The parameter À is called the rate of the distribution and is set so that the expected value of the truncated exponential is equal to the desired value plg:
B
Pg.
(3)
"hii result follows directly from Popoviciu's inequality on variances (Popoviciu, 1935).
4

---

Under review as a conference paper at ICLR 2023
We discuss the truncated exponential more extensively in the appendix (including how to compute the normalization constant, how to find the appropriate rate given the desired mean Hax and how to draw samples from it).
For deciding which distribution family to use, we could once again try to make use of the maximum entropy principle. We have sampled mean [4j and standard deviation oj for each class j in our dataset, and now we want to assign to that class a distribution defined on the interval [0, 1]. with the same mean and standard deviation. The maximum-entropy distribution of a specified mean and standard deviation, and also defined on a bounded interval is called the truncated normal (Udwadia,
1989). However, we would also like to be able to easily derive the parameters of each distribution given its mean and standard deviation. In the case of the truncated normal, deriving its parameteri- zation involves solving non-linear system of equations, which does not have an analytical solution and is not guaranteed to be solvable in a numerically stable way. Instead, we argue that the Beta distribution is a more appropriate choice for number reasons. First, deriving the parameters of a Beta given a desired mean and standard deviation is trivial (see Eq. 5). Second, as we empirically show in the appendix, the Beta captures 99.84% the entropy of the truncated normal, on average. Third, the Beta is mathematically convenient for use-case since its support is the interval [0, 1].
The Beta distribution defined as
B(a, 3) cz - x)--1, I E [0.
(4)
and is parameterized by its shape parameters a and B, while is a normalization constant. Given a desired mean μ4j and standard deviation o, the shape parameters ag and By of a Beta with such mean and a standard deviation are computed as follows:
14 14) 03
0j
14
2
Bj = 115)
However, we need to be able to guarantee the existence of a distribution with support [0, 1] given the mean My and the standard deviation ay that we have sampled for each class j. With regard to the Beta distribution, the relevant necessary condition
of < (1,(1-1))
Therefore, we need to make sure this condition holds for every class j. A simple way to ensure that, would be to first sample mean μ4 in 00, 1J as described above, and then to sample the standard deviation σ.. with rejection sampling (Casella et al., 2004), until we find a pair (Hj, oj) ) that satisfies Eq. 6. An alternative would be to first sample the standard deviation oj, and then to shrink the support of the uniform distribution from which μj is sampled, in order to guarantee that Eq. will be satisfied for any choice within the shrunk support. After some algebra, we get the shrunk support:
0.5 Ty, 0.5 + ry], where ry
In our view, using the require repeated sampling steps succeed. The entire sampling process is presented in Algorithm 1.
approach is superior to rejection sampling since does not
3.3 PERMUTING THE DATASET
Now we will describe how to use the class distributions DD = BBa,,,3) to permute the dataset D (see also Algorithm 2). First, we assign timestamp 1 to each instance of the dataset. These timestamps will then be used to produce a permutation p, according to which we will permute the dataset D.
For each data instance (z; Vi), we sample timestamp from the distribution of its class. In other words, we set j Yi. and then sample 1, B(aa..8)) 83). Hence, we see that the timestamps of all data instances of particular class are sampled from the same distribution, namely, B(a,, 8y). Afterwards, we compute the permutation p = (pi.....Pa .P.) as the vector of indexes that would sort the vector (4,,... k.).. In other words, (pI.... .P.) is computed by applying the argsort opera- tion on the vector (t1,..., ).). Finally, we permute the dataset according to p. Intuitively, in the permuted dataset, the data instance with the smallest timestamp will appear first, the with the -mallest timestamp will appear second, while the one with the largest timestamp will appear last. A toy example of dataset permutation is presented in Figure 1.
5

---

Under review as a conference paper at ICLR 2023
Sampled Distributions
Class
Class
Class
0.0
0.2
0.4
0.6
0.8
1.0
Sampled Timestamps
II
0.0
0.2
0.4
0.6
0.8
1.0
Labels els of Permuted Stream
Figure 1: We use a toy dataset with 13, 17, and 10 instances, for class 1, 2. and respectively. A task-free stream is constructed by sorting the timestamps sampled from the class distributions.
4 EXPERIMENTS
4.1 EXPERIMENTAL SETTINGS
Datasets We use four datasets of varying difficulty. EMNIST (Cohen et al., 2017) is a dataset containing approximately 130,000
Algorithm Permute the dataset.
grayscale images of handwritten characters and digits belonging to 47 classes. CIFAR-
Dataset D = {(xi, y: }
10 and CIFAR- 100 (Krizhevsky, 2009)
Class distributions B(ay, for =;;....... I....,c
datasets that contain each 50,000 color im- ages of 10 and 100 classes, respectively. Fi- nally, tiny ImageNet (Iee & Yang. 2015) is the most challenging dataset widely used
for data instance i I...., do
2: Set j = y/
3: Sample timestamp: 4 B(a,,;
in evaluating continual learners. It con- tains 100,000 color images of 200 different classes. We have not used any data aug- mentation in our experiments, since we want to keep our evaluation domain-agnostic, and data augmentation might not be possible or practical for data modalities other than images.
4: end for
Compute permutation: p= argsort(41, .n)
6: Permute dataset D according permutation
Methods Experience replay (ER) (Isele & Cosgun, 2018; Chaudhry et al., 2019) is the most fun- damental continual learning baseline. It performs replay from memory which is populated using reservoir sampling (Vitter, 1985). Maximally-interfere
retrieval (MIR) (Aljundi et al., 2019a) is an extension of ER that replays the instances which would experience the largest loss increases the model were to be updated using only the current mini-batee of observations. Class-bis Class-balancing reservoir sampling (CBRS) (Chrysakis & Moens, 2020) uses a memory population algorithm that dlas-balancedd possible all times. Greedy sampler and dumb learner
maintains the memory (GDUMB) (Prabhu et al., 2020) also uses a blluss-bllancing memory population algorithm and trains the model using only data stored in memory." Gradient-based memory editing (GMED) (Jin al., 2021) edits the data stored in memory in order to make them more challenging to memorize. Fi- nally, asymmetric cross entropy (ACE) (Caccia et al., 2021) employs a modified loss function that improves continual learning performance by reducing representation drift.
'The original formulation of GDUMB (Prabhu et al., 2020) is not directly applicable to task-free continual learning since it only trains model after the stream has heen observed in its entirety (Verwimp al., 2021). Nonetheless, it easily extended for in task-free continual learning (please refer the appendix).

---

Under review as a conference paper at ICLR 2023
Continuum with
Tack-fre Continum
60
0.0
02
0-5
0.3
3.0
69
0.8
42
6.1
0.5 0.6
0.2
3.0
100
Pogrmsite
Figure 2: A conventional CIFAR-10 stream with disjoint tasks (left) and a simulated task-free (STF)
stream of the same dataset (right). Best viewed zoomed-in and in color.
Table I: Benchmarking task-free continual learning methods using STF streams four datasets. We present the final accuracy on the test set after observing the entire stream (Fin. Acc.), and the information retention averaged over the entire stream (Av. IR). All entries are 95%-ccofiidenc intervals over 20 runs.
EMNIST
CIFAR-10
CIFAR-100
tinylmageNet
Fin. Acc.
Av.
Fin. Acc.
Av. IR
Fin. Acc.
Av. IR
Fin. Acc.
80.1 0.5
87.7 1
38.4 88.42.1
56.9 56.9 1.7
14.3 0.7
27.9
1.66
16.3 1 0.5
MIR
79.0 0.4
88.4 0.6
3..
2..
14.1
29.2
7.7
16.4
0.4
CBRS
79.7 0.5
87.0 0.6
38.1 2.0
13
14
26.6
8.7
15.7 0.4
GDUMB
81.0 0.2
88.2 0.6
4.74
1.6
0.5
23.6
6.9
14.9
GMED
80.4 0.6
88.0
0.7
39.3
1.9
14.5
28.2
8.8
16.31006
ACR
80.6 + 0.4
89.5 + 0.6
49.9
64.6 1.6
19.3
+ 0.5
32.6 + 0.8
11.3 0.4
20.9 + 0.5
Hyperparameters Following previous work (Aljundi et al., 2019a; Chrysakis & Moens, 2020; Jin et al., 2021), we use stochastic gradient descent optimization with learning rate of 0.1, and we both the stream and replay batch sizes 10. Method specific hyperparameters are set based on the values provided in their respective papers. We use memory sizes in the range of 1-4% of the size of the stream (2000 for EMNIST, 1000 for CIFAR-10, 2000 for CIFAR-100, and 4000 for tinylmageNet). Please refer to the appendix for information on the architectures used.
Evaluation Metries Following previous work (Aljundi al., 2019a; Chrysakis Moens, 2020; Jin et al., 2021), evaluate all methods by calculating the accuracy the unseen testing data after the end learning. Moreover, order to also evaluate the longitudinal learning performance of each method throughout the continuum, use the information retention metric proposed in Cai et al. (2021) (accuracy over past observations) averaged over the entire stream.
4.2 STREAM COMPARISON
We start by comparing conventional CIFAR-10 distinct-task stream with two instances streams generated by our proposed algorithm (see Figure 2). We split the two streams in 200 chunks, and compute the relative frequency each class in each of the 200 chunks. The conventional stream (left) is split into tasks with distinct task boundaries between them, and that the data distribution remains stationary within each task. Conversely, in the STF stream (right), the distribution changes continuously over time, sometimes more slowly and others more abruptly. Moreover, other interesting characteristics of the STF stream, such as a) variation how dispersed or concen- trated each class appears over the stream; b) class distributions with more than modes (e.g., class 4 on the right); and с) class distributions that are skewed (e.g., class 5 on the right). We expect all these characteristics be present in real-world task-free streams, but, unfortunately, they never present in the conventional distinct-task streams.
observe
4.3 BENCHMARKING
At this point we benchmark six methods applicable to task-free continual learning using our STF streams (see Table 1). We describe in the appendix how the average standard deviation Ho for each dataset. ER and its variants (MIR, CBRS, GMED) perform similarly accross all datasets. GDUMB is different from all the other methods in the sense that it is optimizing model using

---

Under review as a conference paper at ICLR 2023
tiyylmagelie
BR ALE
60-
12
55.
20-
12-
GOUND
22.
18-
10-
11
60
5-
40-
14
8-
7-
3.0 Clens
Nattoker
Clum Dec. Niticter
Class Dew. Nittster
Clece Daw. NiMteller
Figure 3: We evaluate the final accuracy of ER, ACE, and GDUMB using STF streams generated from four datasets with the He values used in Section 4.3 scaled by 0.5, 1, 2, or 4. All results presented as 95%-ccnfidence intervals.
Castimuum wit Digan: Taekss
Task-free nankinuum
0.75
0.75
00
0.
0.5
0.8
10
Figure 4: We use CIFAR-100 to create three disjoint-task streams with 10 classes per task (left) and three STF streams (right). For each of the streams, we plot the relative frequency of the most prevalent class at each moment in time.
only data stored in memory. Such an approach appears to be performing well in EMNIST and CIFAR-10, but not so well in the datasets that contain large number of classes (CIFAR-100) tiny- ImageNet). ACE outperforms all other methods CIFAR-10, CIFAR-100, and linylmageNet, a result which suggests that the use of the asymmetric cross-entropy approach can be applied success- fully in streams with continuously changing data distributions.
4.4 THE EFFECT OF CLASS DISPERSION
Here we examine the effect of the hyperparameter He. which determines how concentrated or spread-out the class distributions are. In Figure of their final accuracy for four different values Par (we use the values of Per that we used in Section 4.3 scaled by 0.5, 1, dispersed classes (with larger standard deviations) are easier to learn, and, conversely, streams with more concentrated classes (with smaller standard deviations) are more difficult. Our interpretation of these results is that when the class distributions on average have higher measure of dispersion, the stream batches are more likely contain larger variety of labels, and the model can learn bet- ter
compare ER, ACE, and GDUMB in terms
4). We observe that for all three methods, streams with more
features. Therefore, we can interpret the value of μr, as a kind of difficulty knob for the resulting STF streams. cases. When we set o, streams with one class per task. On the other hand, when we set σ, = 0.5 for all classes j. the resulting streams are iid (or, alternatively, one task that contains all classes).
also interesting to note what happens in the two extreme for all classes j. the resulting streams become equivalent to disjoint-task
4.5 OTHER CONSIDERATIONS
Finally, we want to note some other ways in which STF streams differ with disjoint-task streams. First, we use the CIFAR- 100 dataset to generate three disjoint-task streams and three STF streams. In Figure 4, we plot, for both the disjoint-task (left) and the STF streams (right), the relative fre- quency of the most prevalent class at each moment in time. Since the disjoint-task streams (left) constructed with 10 classes per task, the resulting relative frequency is constantly 0.1. On the contrary, the relative frequencies for the STF streams vary in all of the three plots on the right, which is evidence of their lack of structure.
8

---

Under review as a conference paper at ICLR 2023
Continuums Digeir Tasks
Thek-Tree
2.3
6.
2.3
10
Figure 5: We use CIFAR-100 1-0 to create three disjoint-task streams with 10 classes per task (left) and three STF streams (right). We run the ER algorithm on each of the six streams, and plot the corresponding loss curves.
In Figure we plot the loss of the ER method on CIFAR-100 100, for three disjoint-task streams (left) and three STF streams (right). We observe that, since the disjoint-task streams have always exactly the same structure, their corresponding loss curves are essentially identical (loss spikes take place every time there is a task transition). In contrast, because the STF streams are more varied in terms of their structure, their loss curves are more dissimilar.
DISCUSSION & CONCLUSION
The general goal of research is to increase our knowledge and our ability to solve complex problems, and task-free continual learning is evidently one of them. Given how generally applicable this problem is (see our arguments Section 2.3), we believe it is critical to have in place evaluation frameworks that are in line with real-world applications. In the opposite case, cannot be confident that the algorithms and methodologies that we design will be able to generalize well when applied in the real world. A relevant metaphor would be transfer learning: the closer the source distribution to the target distribution, the easier it to transfer knowledge. Applied to our problem, the closer the evaluation framework is to task-free continual learning, the more confident we can be that the methods that perform well in the evaluation framework will also perform well in the real world.
Furthermore, since there is inherent uncertainty in what real-world task-free continual learning streams would be like, we argue that we should not be imposing any unnecessary structure on our evaluation frameworks. As we showed in Section 4.5, however, conventional task-disjoint streams are highly structured. We consider a more general evaluation framework to be more appropriate as a benchmark. Indeed, using on streams with various characteristics terms of their underlying data distributions (see Figure 2) is a more robust evaluation, than only using streams with exactly the same structure.
One limitation of our stream-simulating algorithm is that it relies labels, and hence, cannot be readily extended to unlabeled datasets. Future work could examine whether this extension is pos- sible by using unsupervised representations followed by clustering to assign pseudo-labels to each instance of the datasct. Extending our algorithm to multi-label classification problems could be pos- sible by transforming the problem into multi-class problem (Spolaór al., 2013). Moreover, algorithm could be adapted to regression problems by quantizing the output space.
To summarize, work is motivated by the observation that the definition and the evaluation of task-free continual learning are not aligned. In particular, task-free continual learning involves data distributions that change continuously over time, but the evaluation of task-free continual learning is performed using data distributions that change only at discrete steps. To help remedy this issue, we have proposed an algorithm that can transform any labeled dataset into a task-free continual learning stream, that is, a stream whose data distribution changes, not just at distinct steps, but continuously over time. We have demonstrated experimentally that the STF streams generated using our algorithm contain much less structure than the disjoint-tase streams conventionally used in past work. This lack of structure is, in our view, a desirable feature, since the STF streams can better capture the uncertainty of what real-world task-free stream would be like. We hope that our work will make it more likely that task-fre free continual learning contributions proposed future work will be able to better generalize to practical applications of task-free continual learning.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Rahaf Aljundi, Eugene Belilovsky, Tinne Tuytelaars, Laurent Charlin, Massimo Caccia, Min Lin, and Lucas Page-Caccia. Online continual learning with maximally interfered retrieval. Advances in Neural Information Processing Systems, 32, 2019a.
Rahaf Aljundi, Klaas Kelchtermans, and Tinne Tuytelaars. Task-free continual learning. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11254- 11263, 2019b.
Rahaf Aljundi. Min Lin, Baptiste Goujaud, and Yoshua Bengio. Gradient based sample selection for online continual learning. Advances in Neural Information Processing Systems, 32, 2019c.
Lucas Caccia, Rahaf Aljundi, Nader Asadi, Tinne Tuytelaars, Joelle Pineau, and Eugene Belilovsky. New insights on reducing abrupt representation change in online continual learning. Interna- tional Conference on Learning Representations, 2021.
Zhipeng Cai, Ozan Sener, and Vladlen Koltun. Online continual learning with natural distribu- tion shifts: An empirical study with visual data. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8281-8290, 2021.
George Casella, Christian Robert, and Martin T Wells. Generalized accept-reject sampling schemes. Lecture Noles-Monograph Series, pp. 342-347, 2004.
Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet K Dokania, Philip HS Torr, and Marc' Aurelio Ranzato. Continual learning with tiny episodic mem- ories. In Workshop Multi-Task and Lifelong Reinforcement Learning, 2019.
Aristotelis Chrysakis and Marie-Francine Moens. Online continual learning from imbalanced data. In International Conference on Machine Learning, pp. 1952-1961. PMLR, 2020.
Gregory Cohen, Saeed Afshar, Jonathan Tapson, and André Schaik. Emnist: Extending mnist to handwritten letters. In 20/7 International Joint Conference on Neural Networks (IJCNN), pp- 2921-2926, 2017.
Matthias De Lange, Rahaf Aljundi, Mare Masana, Sarah Parisot, Xu Jia, Aleš Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence, (17):3366-3388) 2021.
Sebastian Farquhar and Yarin Gal. Towards robust evaluations of continual learning. In Lifelong Learming: A Reinforcement Learning Approach workshop. ICMI., 2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
David Isele and Akansel Cosgun. Selective experience replay for lifelong learning. In Thirty-Secont AAAI Conference Artificial Intelligence, 2018.
Edwin Jaynes. Information theory and statistical mechanics. Physical review, 106(4):620, 1957a.
Edwin T Jaynes. Information theory and statistical mechanics. II. Physical review, 108(2):171, 1957b.
Edwin T Jaynes. Prior probabilities. IEEE Transactions on systems science and cybernetics, 4(3): 227-241, 1968.
Xisen Jin, Arka Sadhu, Junyi Du, and Xiang Ren. Gradient- editing of memory examples for online fask-free continual learning. Advances Neural Information Processing Systems, 34: 29193-29205. 2021.
Polina Kirichenko. Mehrdad Farajtabar, Dushyant Rao, Balaji Lakshminarayanan, Nir Levine, Ang Li, Huiyi Hu, Andrew Gordon Wilson, and Razvan Pascanu. Task agnostic continual learning with hybrid probabilistic models. arXiv preprint arXiv:2106. 1ot.:222...12.7,,. 2021.
10

---

Under review as a conference paper at ICLR 2023
Alex Krizhevsky. Learning multiple layers of features from tiny images. Master's thesis, University of Toronto, 2009.
Richard Kurle, Botond Cseke, Alexej Klushyn, Patrick Van Der Smagt, and Stephan Günnemann. Continual learning with bayesian neural networks for non-stationary data. In International Con- ference on Learning Representations, 2019.
Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge.
CS 231N, 7(7):3, 2015.
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. Nature, 521(7553):436 444, 2015.
Vincenzo Lomonaco and Irina Rish. Continual learning with deep architectures. Tutorial at the International Conference Machine Leanring, 2021.
Tom M Mitchell. Machine Learning. McGraw-Hill, 1997.
Cuong V Nguyen, Yingzhen Li, Thang D Bui, and Richard Turner. Variational continual learning- In International Conference on Learning Representations, 2018.
Genman Parisi, Ronald Kemker, Jose L Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: A review. Neural Networks, 113:54-71, 2019.
Quang Pham, Chenghao Liu, Doyen Sahoo, and HOI Steven. Contextual transformation networks for online continual learning. In International Conference on Learning Representations, 2020.
Tiberiu Popoviciu. Sur les équations algébriques ayant toutes leurs racines réelles. Mathematica, (129-145):20, 1935.
Ameya Prabhu, Philip HS Torr, and Puneet K Dokania. Gdumb: A simple approach that questions our progress in continual learning. In European conference on computer vision, pp. 524-540. Springer, 2020.
Newton Spolaör, Everton Alvares Cherman, Maria Carolina Monard, and Huei Diana Lee. A com- parison of multi-label feature selection methods using the problem transformation approach. Elec- tronic notes in theoretical computer science, 292:135- 151, 2013.
Firdaus E. Udwadia. Some results on maximum entropy distributions for parameters known to in finite intervals. Siam Review, 3101):003-10, 1989.
Gido M Van de Ven and Andreas S Tolias. Three scenarios for continual learning. arXiv preprint 2019.
Eli Verwimp, Matthias De Lange, and Tinne Tuytelaars. Rehearsal revealed: The limits and mer- its of revisiting samples continual learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9385-9394 2021.
Jeffrey S Vitter. Random sampling with reservoir. ACM Transactions on Mathematical Software (TOMS), 11(1:337-57, 1985.
Haiyan Yin, Ping Li, et al. Mitigating forgetting in online continual learning with neuron calibration. Advances in Neural Information Processing Systems, 34: 102200002222,,,, 2021.
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In International Conference on Machine Learning, pp. 3987-3995. PMLR, 2017.
Chen Zeno, Itay Golan, Elad Hoffer, and Daniel Soudry. Task agnostic continual learning using online variational bayes. arXiv preprint arXiv: 1803, 10123, 2018.
11

---

Under review as a conference paper at ICLR 2023
Continum with
Thsk-Free
25
Smeam Regresiion
Srean Poggassiin
Figure 6: A conventional EMNIST stream with disjoint tasks (left) and simulated task-free (STF) stream of the same dataset (right). Best viewed zoomed-in and in color.
Table 2: (left) A simple convolutional block; (middle) The Convolutional Neural Network (CNN) architecture used the EMNIST experiments. (right) The reduced ResNet-18 architecture used for CIFAR-10, CIFAR- 100, and tinylmageNet, is built using the BasicBlock(n y, n&. n.)) from (He et al., 2016), where n is the number convolutional filters, n, is the number of sub-blocks per block, and n, is the stride of the layer.
ConvBlock
CNN
Reduced ResNet-18 18
Conv2D(nin: Nomr)
ConvBlock(1 (1, ConvBlock 64)
BasicBlock(20, 2, 1)
ReLU
BasicBlock(40,, 2, 2)
BatchNorm22
Linear(64,
BasicBlock(80) 2, 2)
Conv2D (Touut Tlout)
BasicBlock( 160, 2, 2)
ReLU
AveragePooling Linear 160, c)
(atchNorm22D(nccl) Max M2
A REPRODUCIBILITY
order to ensure that our work is not only reproducible but also easily accessible, we commit to open sourcing code upon acceptance. Moreover, in order to increase the reproducibility of our results, wherever we present average performance over many runs, we do so in the form 95%- confidence intervals, in order for the numbers to accurately reflect their inherent uncertainty.
B MORE SIMULATED TASK-FREE STREAMS
In Figure Figure 7, Figure we present a comparison of conventional distinct-task stream and simulated task-free (STF) for EMNIST, CIFAR-100, and tinylmageNet respectively.
C
ARCHITECTURES
Please refer Table 2
12

---

Under review as a conference paper at ICLR 2023
Continuum with Digent Taaks
Tisk Free Corknaum
2.8
255 Regrossion
2.0
0.8
Figure 7: A conventional CIFAR-100 stream with disjoint tasks (left) and simulated task-free (STF) stream of the same dataset (right). Best viewed zoomed-in and color.
Condinumm With Digoint Tacks
Tisk Free Contnuum
23
25
0.8
3.5
12
0.8
Wegmodinn
Figure 8: A conventional tinylmageNet stream with disjoint tasks (left) and simulated task-free (STF) stream of the same dataset (right). Best viewed zoomed-in and color.
13

---

Under review as a conference paper at ICLR 2023
D SETTING THE AVERAGE STANDARD DEVIATION
We set the values of je by associating them to a number of tasks. Previous work typically uses range of 5-20 number of tasks. that case, each class's distribution is essentially a uniform with support the inverse of the number of tasks. If the number of tasks is set to T the standard deviation of a uniform with support 1/T is v /1/12 x 1/T. Here, we have used average standard deviations Pa which correspond to T 12, 10, 20 tasks for EMNIST, CIFAR-10 10, CIFAR-100 100, and tiny ImageNet, respectively.
E BETA AND THE TRUNCATED NORMAL
Since the truncated normal is the maximum-entrope distribution of given mean and standard devi- ation defined on closed interval, we want examine how much its entropy the Beta can capture. We start by sampling a mean E [0, IJ and a standard deviation E [0, 0.5]. and take nor- mal distribution with these moments and truncate in [0, 1J. We evaluate the mean f and standard deviation & of the truncated normal, and define a Beta with the same moments. Afterwards, we can evaluate the ratio of the entropy of the truncated normal that the Beta can capture. Performing find that the Beta captures 99.84% of the entropy of the truncated normal. Hence, we argue that the Beta can serve as a substitute for the truncated normal, since, as mentioned earlier, it is more convenient mathematically.
this simulation 10y times,
F ON THE TRUNCATED EXPONENTIAL
The truncated exponential distribution, with rate parameter + 0 and the truncation parameter > for its support, is defined as
fe(r X, & ce
x E [0, 7].
(8)
where e a/(e>> - 1) is the normalizing constant. The mean of a E(), 7) over its support is computed
" ==c L zod=dr= (N7 1 1)- +1
1 X(e^7 1) 1- X
7
(9)
An appropriate rate can be found by numerically solving the previous equation for specific values of Iu and 7.
The cumulative distribution function (CDF) is
Ax - Fe(x (, exn T
[0, 7).
(10)
hence, we can sample from a TE distribution using inverse-transform sampling (ITS) as follows:
1 x In [(( - -1)u + 1], where U
(11)
14


Reviews:
[{"review_id": "_rKu7jhZRWf", "reviewer": "Reviewer_A6c5", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper develops a pipeline for evaluating task-free continual algorithms using three common machine learning datasets. The core idea is to generate a stream of tasks from a dataset, e.g. mini-Imagenet, and then assign to each data instance a permutation index based on the Beta distribution to determine the time in the stream that it should appear to remove task boundaries. Experiments using four continual learning benchmarks and 6 existing continual learning algorithms are provided in the proposed evaluation protocol.", "summary_of_the_review": "This paper proposes a benchmark dataset for task-free continual learning and it lacks any algorithmic novelty. The benchmark is also somewhat contrived and is only based on operating on existing datasets. As a result, the only novelty is how to process a dataset such that it can be used for task-free evaluation and there is no real connection to a practical application. The paper is also more related to a conference like NeurIPS Dataset and Benchmarks, rather ICLR. I also highly double this paper passes the bar for NeurIPS Dataset and Benchmarks. In conclusion, I think this work does not pass the bar for acceptance in ICLR. To improve the work, I think there are two approaches: (i) propose a new continual learning algorithm that can outperform the existing methods in the proposed protocol (ii) build a more realistic dataset. I will add that the latter still will make this work suitable for NeurIPS Dataset and Benchmarks.", "strength_and_weaknesses": "Strength \n\n1. The writing is clear.\n\n2. Task-free evaluation is a challenge for continual learning and the paper identifies a good research problem.\n\nWeaknesses\n\n1. Novelty is extremely limited.\n\n2. The benchmark is contrived without any connection to a practical application.\n\n3. Experiments are limited and do to convey much understanding because the datasets are on the simple end of continual learning datasets and only a limited number of continual learning algorithms are included.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "_kV1VxOIo3s", "reviewer": "Reviewer_ZSfX", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents a framework for task-free continual learning. The framework consists of a method for converting any dataset to a task-free continual learning problem where information on the class is not required and is not explicit, but rather examples from different classes can be observed at any step of the learning sequence. This method relies on assigning distributions to each class so that these distributions can be later used to select examples from a particular class, or set of classes, for a particular batch for learning. To assign these distributions, the authors propose to sample mean and standard deviations from a Beta distribution. Experiments are run over four different benchmark datasets and a range of baseline continual learning methods.  ", "summary_of_the_review": "As noted in the \"Strengths and Weaknesses\" section, although I appreciate the definition of the task-free continual learning scenario and the attempts at experimentally measuring how different baselines work in this setting, I consider that the contribution of this paper lacks novelty and is not sufficient for ICLR since there is no method being proposed for exploiting the task-free learning scenario. Therefore, my recommendation for this paper is a rejection, but I encourage the authors to improve their contribution by incorporating a method for task-free learning as part of their framework.", "strength_and_weaknesses": "Strengths: \n- The paper pursues an interesting research direction which considers continual learning without assumptions on how classes are organised into tasks. Task information is not required for learning nor for inference purposes. This is certainly a more realistic scenario of continual learning, which as the authors exemplify may be applicable to a variety of domains.\n- The paper is clearly written and well presented.\n\nWeaknesses:\n- Although I appreciate the relevance of pursuing real task-free continual learning, I find the proposed method quite straightforward and not fully sound. For example, although you mention some reasons to select the Beta distribution over some other possible distributions for sampling on each class, these arguments seem quite trivial and do not necessarily imply that other distributions are not entirely applicable. For a framework such as the intended one, I would expect to see a thorough analysis of how different class distribution assignments work for the purpose of task-free continual learning. Another aspect that impacts the soundness of the proposed method is the amount of explanations on section 3.2 around maximum entropy principle and the beta distribution itself, and the lack of intuition about how class distribution assignment actually works. \n- A second weakness that I see in this paper is regarding the experiments: from Figure 1, it seems that the proposed method tends to allocate examples from the same class quite contiguously; with the small batch sizes used in the experiments, it is very likely that examples from a particular class will be all observed on the same batch. I would expect to see more systematic experiments that clearly show that the task-free approach works for scenarios where examples from the same class are observed at just a few, some or many steps over the learning sequence. I would also like to see more systematically how important hyperparameters such as the batch size affect task-free continual learning.\n- Finally, beyond the definitions of task-free continual learning and the reported experiments, I consider the contribution of this paper quite limited. For an ICLR paper, I would that the proposed framework would also include a method that could leverage better this task-free scenario.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "FIp5hUgByd", "reviewer": "Reviewer_pevR", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper focusses on task-free continual learning from an online stream whose distribution changes continuously over time. Within the study, a few proposed methods were used to define the boundaries of the distribution in a few very elegantly written conceptual algorithms. Furthermore, they propose real-time evaluation frameworks that can visualise the impact of the class standard deviation multiplier. Finally, they demonstrate task-free continuums and the associated complexities within this.", "summary_of_the_review": "Overall, this paper shows merit. There are a variety of interesting visualizations given, and those were explained properly within the context of the paper. There are, however, field specific visualisations that could have strengthened the argument, especially from the perspective of continuum disjoint tasks. Although not elaborated on too much, continuum disjoint tasks added a valuable addition to the strength of STF, and perhaps this contrast could have added a much needed justification to the argument.", "strength_and_weaknesses": "This paper is well written, and the logic of the paper flows succinctly from one paradigm to another. \n\nHowever, this paper does not outline the pragmatic pitfalls associated with the approach such as data distributions that might have the same class standard deviations, but are fundamentally different in data shape. Although these anomalies are not as common, their approach does not factor this in. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "BMpUOj7LwWL", "reviewer": "Reviewer_gmDh", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a new benchmark for task-free continual learning. They claim that there is a large gap between how task-free continual learning is defined and how it is evaluated, so they. propose an algorithm to reorder any labeled dataset into a simulated task-free continual learning stream for benchmarking. ", "summary_of_the_review": "I appreciate the proposed benchmark, but the contribution and the novelty is limited for ICLR standards. ", "strength_and_weaknesses": "Strengths:\n- It is a well-written paper that defines the problem clearly. \n- An algorithm that can generate a task-free continual learning stream properly for benchmarking is an important contribution.\n\nWeaknesses:\n- I'm not sure about the technical contribution of the paper. I understand it simulates task-free continual learning better than the existing approaches, but doesn't propose an approach to solve the problem itself.  ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject