Original: O NLINE B IAS C ORRECTION FOR T ASK -F REE C ONTINUAL L EARNING
Revised: O NLINE B IAS C ORRECTION FOR T ASK -F REE C ONTINUAL L EARNING

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Task-free continual learning is the machine-learning setting where a model is trained online with data generated by a non-stationary stream.
Revised: Task-free continual learning is the machine-learning setting where a model is trained online with data generated by a nonstationary stream.

Original: Conventional wisdom suggests that, in this setting, models are trained using an approach called experience replay, where the risk is computed both with respect to current stream observations and to a small subset of past observations.
Revised: Conventional wisdom suggests that, in this setting, models are trained using an approach called experience replay, where the risk is computed both with respect to current stream observations and to a small subset of past observations.

Original: In this work, we show both theoretically and empirically how experience replay biases the outputs of the model towards recent stream observations.
Revised: In this work, we explain both theoretically and empirically how experience replay biases the outputs of the model towards recent stream observations.

Original: Moreover, we propose a simple approach to correct for this bias online, by changing the way the output layer of the model is optimized.
Revised: Moreover, we propose a simple approach to mitigate this bias online, by changing how the output layer of the model is optimized.

Original: We show that our approach improves significantly the learning performance of experience-replay approaches overa number of different datasets.
Revised: We show that our approach improves significantly the learning performance of experience-replay approaches over different datasets.

Original: Our findings suggest that, in contrast to stationary machine-learning problems, the output layer of a model should be optimized separately from its preceding layers when performing experience replay
Revised: Our findings suggest that, when performing experience replay, the output layer of the model should be optimized separately from its preceding layers.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: In broad terms, continual learning is the process of incrementally aggregating knowledge from data that are generated by a non-stationary distribution (Lee et al., 2019; Riemer et al., 2019).
Revised: In broad terms, continual learning is the process of incrementally aggregating knowledge from data that are generated by a nonstationary distribution (Lee et al., 2019; Riemer et al., 2019).

Original: The main motivation for studying continual learning is to give artificial learners the ability to learn as biological learners do—perpetually updating and refining their body of knowledge under changing environmental conditions (Silver et al., 2013).
Revised: The main motivation for studying continual learning is to give artificial learners the ability to learn as biological learners do—perpetually updating and refining their body of knowledge under changing external conditions (Silver et al., 2013).

Original: The inability of artificial learners to learn continually is caused by the fact that, in practice, they overwrite previously learned knowledge whenever they encounter new information.
Revised: The inability of artificial learners to learn continually stems from the fact that they overwrite previously learned knowledge whenever they encounter new information.

Original: This phenomenon is called catastrophic forgetting (McCloskey & Cohen, 1989; French, 1999).
Revised: This phenomenon is called catastrophic forgetting (McCloskey & Cohen, 1989; French, 1999).

Original: In this paper, we focus specifically on task-free continual learning (Aljundi et al., 2019c).
Revised: In this paper, we focus specifically on task-free continual learning (Aljundi et al., 2019c).

Original: In this setting, the data are presented to the learner in small minibatches, and this setting is agnostic to the way the data distribution changes over time.
Revised: In this setting, the data are presented to the learner in small minibatches, and this setting is agnostic to the way the data distribution changes over time.

Original: In other words, we assume no knowledge about whether the distribution is piecewise-stationary (such as when there are distinct tasks being learned), or whether the distribution changes continuously over time (Aljundi et al., 2019c).
Revised: In other words, we assume no knowledge about whether the distribution is piecewise-stationary (that is, when there are distinct tasks being learned), or whether the distribution changes continuously over time (Aljundi et al., 2019c).

Original: Most task-free continual learning approaches make use of a memory which can store a small percentage (typically 10% or less) of all observed data instances.
Revised: Most task-free continual learning approaches make use of a memory which can store a small percentage (typically 10% or less) of all observed data instances.

Original: The data instances stored in memory are  replayed in future steps to mitigate catastrophic forgetting.
Revised: The data instances stored in memory are subsequently replayed in order to mitigate catastrophic forgetting.

Original: This simple paradigm, called replay-based continual learning, is surprisingly effective in task-free settings.
Revised: This simple paradigm, called replay-based continual learning, is surprisingly effective in task-free settings.

Original: Furthermore, it is also supported by findings from the field of neuroscience, in relation to how biological learning takes place (Marr, 1971; Ji & 
Revised: Furthermore, it is also supported by findings from the field of neuroscience, in relation to how biological learning takes place (Marr, 1971; Ji & Wilson, 2007;

Original: Wilson, 2007; Liu et al., 2019).
Revised: Liu et al., 2019).

Original: A number of continual learning methods tend to make predictions that are biased towards recently observed data (Buzzega et al., 2021; Mai et al., 2021).
Revised: A number of continual learning methods tend to make predictions that are biased towards recently observed data (Buzzega et al., 2021; Mai et al., 2021).

Original: Several strategies have been proposed to deal with this prediction bias (also called recency bias).
Revised: Several strategies have been proposed to deal with this prediction bias (also called recency bias).

Original: Unfortunately, most of them are not applicable to task-free continual learning, since they have been designed for continual learning settings that consist of a task sequence, and they require knowledge of which classes the current task consists (Wu et al., 2019; Belouadah & Popescu, 2019; Buzzega et al., 2021).
Revised: Unfortunately, most of them are not applicable to task-free continual learning, since they have been designed for continual learning settings that consist of a task sequence, and they require knowledge of which classes the current task comprises (Wu et al., 2019; Belouadah & Popescu, 2019; Buzzega et al., 2021).

Original: One approach which is applicable in task-free continual learning is proposed in Mai et al.
Revised: One approach which is applicable in task-free continual learning is proposed in Mai et al.

Original: , but it can only be performed after the end of training, hence the learner’s predictions duringthe entirety of training  remain biased.
Revised: , but it can only be performed after the end of training, hence the learner’s predictions during training would remain biased.

Original: In this paper, we propose a simple approach that performs online bias correction for task-free continual learning.
Revised: In this paper, we propose a simple approach that performs online bias correction for task-free continual learning.

Original: Our contributions are as follows: a) We formally illustrate that the conventional paradigm of model training in task-free continual learning overweights the importance of current stream observations (Section 3.2), and we speculate that this overweighting is a cause of prediction bias of continual learners; b) We proposeof a novel metric to quantify prediction bias, and we show that this bias can be effectively mitigated by appropriately modifyingonly the parameters of  the final layer of the model, after the end of training (Section 3.4); c) We propose a novel approach called Online Bias Correction (OBC), which maintains an unbiased model online, throughout the entire duration of learning (see Figure 1 for an illustration); d) We evaluate the performance of OBC extensively, and we show that it significantly improves a number ofother task-free continual learning methods, over multiple datasets.
Revised: Our contributions are as follows: a) We formally illustrate that the conventional paradigm of model training in task-free continual learning overweights the importance of current stream observations (Section 3.2), and we speculate that this overweighting is a cause of prediction bias of continual learners; b) We propose a novel metric to quantify prediction bias (Section 3.3), and we show that this bias can be effectively mitigated by appropriately modifying the parameters of only the final layer of the model, after the end of training (Section 3.4); c) We propose a novel approach called Online Bias Correction (OBC; Section 3.5), which maintains an unbiased model online, throughout the entire duration of learning (see Figure 1 for an illustration); d) We evaluate the performance of OBC extensively, and we show that it significantly improves a number of task-free continual learning methods, over multiple datasets (Section 4).

Original:  B ACKGROUND 2. T ASK -F REE C ONTINUAL L EARNING
Revised:  B ACKGROUND 2. T ASK -F REE C ONTINUAL L EARNING

Original: We define task-free continual learning as the online optimization of a model via small minibatches that are sampled from a nonstationary stream.
Revised: We define task-free continual learning as the online optimization of a model via small minibatches that are sampled from a nonstationary stream.

Original: In task-free continual learning, no strong assumptions are made about the nature of the distributional nonstationarity of the stream (Aljundi et al., 2019c).
Revised: In task-free continual learning, no strong assumptions are made about the nature of the distributional nonstationarity of the stream (Aljundi et al., 2019c).

Original: Other continual learning settings, such as task-incremental and class-incremental continual learning assume a data distribution that is piecewise stationary, hence  only changes at discrete points in time (Van de Ven & Tolias, 2019).
Revised: Other continual learning settings, such as task-incremental and class-incremental continual learning assume a data distribution that is piecewise stationary, hence one that only changes at discrete points in time (Van de Ven & Tolias, 2019).

Original: Thegeneral objective of continual learning is to learn from all observed data despite the nonstationary nature of the distribution (Jin et al., 2021).
Revised: The objective of continual learning is to learn from all observed data despite the nonstationary nature of the distribution (Jin et al., 2021), and, in general, previous work assumes no distributional mismatch between training and evaluation data.

Original: Previous work in task-free continual learningis mostly focused on replay-based methods (Aljundi et al., 2019d; Jin et al., 2021).
Revised: Previous work in task-free continual learning mostly focuses on replay-based methods (Aljundi et al., 2019d; Jin et al., 2021).

Original: The dominant replay paradigm is called experience replay (ER) (Isele & Cosgun, 2018; Chaudhry et al., 2019).
Revised: The prevalent replay paradigm is called experience replay (ER) (Isele & Cosgun, 2018; Chaudhry et al., 2019).

Original: According to the ER paradigm, each minibatch of observations received by the learner is combined with another minibatch of equal size sampled from the memory.
Revised: According to the ER paradigm, each minibatch of observations received by the learner is combined with another minibatch of equal size sampled from the memory.

Original: The model is then trained for one step with the combined stream-and-memory minibatch.
Revised: The model is then trained for one step with the combined stream-and-memory minibatch.

Original: Moreover, the memory is typically maintained by an online memory-population algorithm called reservoir sampling (Vitter, 1985b).
Revised: Moreover, the memory is typically maintained by an online memory-population algorithm called reservoir sampling (Vitter, 1985).

Original: There are multiple variants of the ER method.
Revised: There are multiple variants of the ER method.

Original: For instance, one approach called MaximallyInterfered Retrieval (MIR) replays instances that are going to be interfered the most by the current minibatch of new observations.
Revised: For instance, one approach called MaximallyInterfered Retrieval (MIR) replays instances that are going to be interfered the most by the current minibatch of new observations.

Original: Another approach called Class-Balancing Reservoir Sampling (CBRS) (Chrysakis & Moens, 2020b) modifies the memory population algorithm to ensure that the memory remains balanced.
Revised: Another approach called Class-Balancing Reservoir Sampling (CBRS) (Chrysakis & Moens, 2020) modifies the memory population algorithm to ensure that the memory remains balanced.

Original: There also other approaches that deviate from the ER paradigm, such as
Revised: There also other approaches that deviate from the ER paradigm, such as

Original: Greedy Sampler and Dumb Learner (GDUMB)
Revised: Greedy Sampler and Dumb Learner (GDUMB)

Original: (Prabhu et al., 2020b), which only trains the model using data stored in memory, or Asymmetric Corss-Entropy (ACE) (Caccia et al., 2022), which uses a modified version of the cross-entropy loss to prevent the drift of latent representations.
Revised: (Prabhu et al., 2020), which only trains the model using data stored in memory, or Asymmetric Corss-Entropy (ACE) (Caccia et al., 2022), which uses a modified version of the cross-entropy loss to prevent the drift of latent representations.

Original: 2. C OMPUTATIONAL C OST
Revised: 2. C OMPUTATIONAL C OST

Original: An important matter in task-free continual learning is computational cost.
Revised: An important matter in task-free continual learning is computational cost.

Original: Since practical applications might have to deal with large amounts of data, the design of task-free continual learners should ensure they are tractable.
Revised: Since practical applications will likely involve large amounts of data, the design of task-free continual learners should ensure they are tractable.

Original: In practical terms, let us assume that a model has to learn from a stream ofsize n .
Revised: In practical terms, let us assume that a model has to learn from a stream of n instances.

Original: Moreover, we assume that the larger the stream the larger size m of the memory will be.
Revised: Moreover, we assume that applications with larger streams will likely involve memory storages of larger size m .

Original: In real-world applications, the difference between an O ( n ) learning algorithm, and an O ( mn ) algorithm might be enormous.
Revised: In real-world applications, the difference between an O ( n ) learning algorithm, and an O ( mn ) algorithm could be enormous.

Original: Hence, in this work we will only focus on learning algorithms whose computational cost per incoming batch is independent from the memory size m , so that the computational complexity of learning from the entire stream is O ( n ) .
Revised: Hence, in this work we will only focus on learning algorithms whose computational cost per incoming batch is independent of the memory size m , so that the computational complexity of learning from the entire stream is O ( n ) .

Original: 2. B IAS C ORRECTION IN T ASK -F REE C ONTINUAL L EARNING
Revised: 2. B IAS C ORRECTION IN T ASK -F REE C ONTINUAL L EARNING

Original: To the best of our knowledge, there only one approach explicitly designed to correct for prediction biases.
Revised: To the best of our knowledge, there only one approach explicitly designed to correct for prediction biases.

Original: Mai et al.
Revised: Mai et al.

Original: propose learning a model using conventional experience replay, and after the entire stream has been observed, they replace the final linear layer of the model with a nearestclass-mean (NCM) classifier computed using all data stored in memory.
Revised: propose learning a model using conventional experience replay, and after the entire stream has been observed, they replace the final linear layer of the model with a nearestclass-mean (NCM) classifier computed using all data stored in memory.

Original: Moreover, they demonstrate that this approach is effective in increasing the final accuracy of the model.
Revised: Moreover, they demonstrate that this approach is effective in increasing the final accuracy of the model.

Original: However, in many realworld applications, there is a need for a model that learns and performs inference at the same time, and that model should remain unbiased all the time.
Revised: However, in many realworld applications, there is a need for a model that learns and performs inference at the same time, and that model should remain unbiased all the time.

Original: In such a setting, the NCM approach would have to be applied after every update of the model, and since it needs to make a full pass over the memory, it would be computationally very expensive.
Revised: In such a setting, the NCM approach would have to be applied after every update of the model, and since it needs to make a full pass over the memory, it would be computationally very expensive.

Original:  M ETHODOLOGY 3. N OMENCLATURE
Revised:  M ETHODOLOGY 3. N OMENCLATURE

Original: Previous work in continual learning typically regards the entire neural network as one learning component.
Revised: Previous work in continual learning typically views the entire neural network as one learning component.

Original: In contrast, we adopt a more modular view of a neural network that consists of two learning components.
Revised: In contrast, we adopt a more modular view of a neural network that consists of two learning components.

Original: Specifically, we will call the output layer of a neural network the classifier , and we will denote it as the parameterized function c ( xxx ; θθθ c ) .
Revised: Specifically, we will call the output layer of a neural network the classifier , and we will denote it as the parameterized function c ( x ; θ c ) .

Original: Moreover, we will refer to the set of layers preceding the classifier as the feature extractor , and we will similarly denote it as g ( xxx ; θθθ g ) .
Revised: Moreover, we will refer to the set of layers preceding the classifier as the feature extractor , and we will similarly denote it as g ( x ; θ g ) .

Original: Using this notation, we can represent the full neural network h as the composition of g and c , that is
Revised: Using this notation, we can represent the full neural network h as the composition of g and c , that is

Original: [Equation]
Revised: [Equation]

Original: , where θθθ h ≜ { θθθ g ,θθθ c } .
Revised: , where θ h ≜ { θ g , θ c } .

Original: [Equation]
Revised: [Equation]

Original: An important distinction between the classifier and the feature extractor, is that classifier has low learning capacity because it consists of only one linear layer, while the feature extractor has high learning capacity because it is composed of multiple nonlinear layers.
Revised: An important distinction between the classifier and the feature extractor, is that classifier has low learning capacity because it consists of only one linear layer, while the feature extractor has high learning capacity because it is composed of multiple nonlinear layers.

Original: Hence, given enough data, the feature extractor can learn more complex representations, but, in low-data scenarios, it is also more prone to overfitting, compared to the classifier.
Revised: Hence, given enough data, the feature extractor can learn more complex representations, but, in low-data scenarios, it is also more prone to overfitting, compared to the classifier.

Original: 3. D ATA -S AMPLING B IAS
Revised: 3. D ATA -S AMPLING B IAS

Original: At a high level, the optimization process during task-free continual learning is very simple.
Revised: At a high level, the optimization process during task-free continual learning is very simple.

Original: At each step t , the learner receives a small minibatch of b observations SSS t = { ( xxx i , y i ) } bi =1 from the stream.
Revised: At each step t , the learner receives a small minibatch of b observations S t = { ( x i , y i ) } bi =1 from the stream.

Original: The learner then samples another minibatch  of equal sizeRRR from its memory MMM = { ( xxx i , y i ) } mi =1 , and performs an update step over the model parameters using gradients calculated with respect to
Revised: The learner then samples another minibatch R of equal size from its memory M = { ( x i , y i ) } mi =1 , and performs an update step over the model parameters using the gradient calculated with respect to

Original: SSS t ∪ RRR .
Revised: S t ∪ R .

Original: Finally, the learner updates its memory with respect to SSS t , using reservoir sampling (Vitter, 1985a), or some other memory population algorithm.
Revised: Finally, the learner updates its memory with respect to S t , using reservoir sampling (Vitter, 1985), or another memory population algorithm.

Original: This training paradigm is called experience replay (Isele & Cosgun, 2018; Chaudhry et al., 2019).
Revised: This training paradigm is called experience replay (Isele & Cosgun, 2018; Chaudhry et al., 2019).

Original: We will now explain why this paradigm has a data-sampling bias that favors new observations over the instances stored in memory.
Revised: We will now explain why this paradigm has a data-sampling bias that favors new observations over the instances stored in memory.

Original: An intuitive way to understand this data-sampling bias is to consider a learner with an infinite memory.
Revised: An intuitive way to understand this data-sampling bias is to consider a learner with an infinite memory.

Original: Let us assume that the learner has encountered a data instances in the past—all of which have been stored in the memory for replay—and that the learner now receives a minibatch SSS t of b new observations.
Revised: Let us assume that the learner has encountered a data instances in the past—all of which have been stored in the memory for replay—and that the learner now receives a minibatch S t of b new observations.

Original: 1 Given an Occam’s-razor assumption that all observations, current and past, are equally important, we will say that the data-sampling is unbiased if all a + b observed data instances are equally likely to contribute to the following optimization step of the model.
Revised: 1 Given an Occam’s-razor assumption that all observations, current and past, are equally important, we will say that the data-sampling is unbiased if all a + b observed data instances are equally likely to contribute to the upcoming optimization step of the model.

Original: In other words, the probability of using any instance in the model-optimization step should be the same, regardless of whether that instance is in the current stream minibatch SSS t , or stored in memory.
Revised: In other words, the probability of using any instance in the optimization step should be the same, regardless of whether that instance is in the current stream minibatch S t , or stored in memory.

Original: Accordingly, the model should be updated with a minibatch BBB sampled uniformly-at-random from the set of all a + b observed instances (that is, the concatenation of the memory and the stream minibatch MMM ∪ SSS t ).
Revised: Accordingly, the model should be updated with a minibatch B sampled uniformly-at-random from the set of all a + b observed instances (that is, the concatenation of the memory and the stream minibatch M ∪ S t ).

Original: Now let us contrast this unbiased data sampling with experience replay .
Revised: Now let us contrast this unbiased data sampling with experience replay.

Original: Under experience replay, the b new observations are included in the minibatch BBB with a probability of 1 , and, by definition, an equal number of instances are sampled uniformly-at-random from the memory (which contains a instances).
Revised: Under experience replay, the b new observations are included in the minibatch B with a probability of 1 , and, by definition, an equal number of instances are sampled uniformly-at-random from the memory (which contains a instances).

Original: Hence, each memory instance has a probability of b/a ≪ 1 of being sampled for the model update.
Revised: Hence, each memory instance has a probability of b/a ≪ 1 of being sampled for the model update.

Original: Therefore, unlike in the unbiased case described above, new observations are guaranteed to participate in the model update, but an arbitrary memory instance is very unlikely to.
Revised: Therefore, unlike in the unbiased case described above, new observations are guaranteed to participate in the model update, but an arbitrary memory instance is very unlikely to.

Original: In essence, this is a data-sampling bias that favors current observations over past ones, and, in turn, leads to the predictions of the model being biased towards recent observations.
Revised: In essence, this is a data-sampling bias that favors current observations over past ones, and, in turn, leads to the predictions of the model being biased towards recent observations.

Original: A hypothesis arising from this discussion is thatprediction biases could be neutralized if the learner had a memory of infinite size, and if it constructed the minibatch B B B , used to update themodel, by sampling instances uniformly-at-random from all observed data, current or past. In this case, we would expect that, on average, a ratio of b/ ( a + b ) of the instances in the minibatch BBB will be from
Revised: we generally assume that a ≫ b . Put another way, the instances in memory vastly outnumber the instances in the newly observed minibatch. To further elucidate the concept of data-sampling bias, we include a numerical example in the appendix.

Original: [Table]
Revised: [Table]

Revised: A hypothesis arising from this discussion is that prediction biases could be negated if the learner had a memory of infinite size, and if it constructed the minibatch B , used to update the model, by sampling instances uniformly-at-random from all observed data, current or past.

Revised: In this case, we would expect that, on average, a ratio of b/ ( a + b ) of the instances in the minibatch B will be from

Original: SSS t , and a/ ( a + b ) of the instances will be from MMM .
Revised: S t , and a/ ( a + b ) of the instances will be from M .

Original: Unfortunately, in practical applications, it is not possible to have a memory of infinite size.
Revised: Unfortunately, in practical applications, it is not possible to have a memory of infinite size.

Original: As the learner observes more and more data, it is a ≫ b , which means that a/ ( a + b ) ≃ 1 and b/ ( a + b ) ≃ 0 .
Revised: As the learner observes more and more data, it is a ≫ b , which means that a/ ( a + b ) ≃ 1 and b/ ( a + b ) ≃ 0 .

Original: Therefore, on average, the minibatches BBB used to train the model will contain almost exclusively data from the memory, and almost no data from the current stream batch SSS t .
Revised: Therefore, on average, the minibatches B used to train the model will contain almost exclusively data from the memory, and almost no data from the current stream batch S t .

Original: Since in practical applications the size of the memory is much smaller than the size of the stream, this weighting scheme will arguably lead to overfitting the memory data.
Revised: Since in practical applications the size of the memory is much smaller than the size of the stream, this weighting scheme will arguably lead to overfitting the memory data.

Original: In the following section, we propose a metric to quantify the prediction bias of a model.

Original: 3. Q UANTIFYING P REDICTION B IAS
Revised: 3. Q UANTIFYING P REDICTION B IAS

Original: We propose  to quantify prediction bias with respect to a set of unseen data.
Revised: At this point, we will propose a metric to quantify prediction bias with respect to a set of unseen data.

Original: Let TTT = { ( xxx i , y i ) } i be such a set, and by yyy i , we will denote the one-hot representation of  y i .
Revised: Let T = { ( x i , y i ) } i be this set, and by y i , we will denote the one-hot representation of the label y i .

Original: We define the expected vector of prior class probabilitieswith respect to this test set as ppp = 
Revised: We define the expected vector of prior class probabilities as p = E

Original: E [ yyy i ] .
Revised: [ y i ] .

Original: Moreover, we define thethe expected vector of model predictions as qqq = E (cid:2) h
Revised: Moreover, we define the expected vector of model predictions as q = E (cid:2) h

Original: ( xxx i ; θθθ h ) (cid:3) .
Revised: ( x i ; θ h ) (cid:3) .

Original: Note that, both expectations are computed with respect to the instances of  the unseen test set.
Revised: Note that both expectations are taken with respect to the data distribution of T , and are computed by averaging over the test-set instances.

Original: Wecan quantify the prediction bias of the model by measuring the discrepancy between ppp and qqq .
Revised: We quantify the prediction bias of the model by measuring the discrepancy between the expected ground truth p and the expected prediction q .

Original: Since both ppp and qqq are vectors of probabilities, we propose the use of the Jensen-Shannon divergence (Lin, 1991), which is a symmetric divergence measure between two probability distributions.
Revised: Since both p and q are vectors of probabilities, we propose the use of the Jensen-Shannon divergence (Lin, 1991), which is a symmetric divergence measure between two probability distributions.

Original: It is defined as
Revised: It is defined as

Original: [Equation]
Revised: [Equation]

Original: where µµµ = 1 / 2( ppp + qqq ) is the average between ppp and qqq , and j is the index over the classes 1 , . ..
Revised: where µ = 1 / 2( p + q ) is the average of p and q , and j is the index over the classes 1 , . .

Original: , c , that are present in the test set.
Revised: , c , that are present in the test set.

Original: When the model predictions are not biased, ppp and qqq will be very similar, and the divergence will be close to zero.
Revised: At a high level, when the model predictions are not biased, p and q will be very similar, and the divergence will be close to zero.

Original: In the opposite case, the more dissimilar ppp and qqq are, the more the divergence will increase.
Revised: Conversely, the more dissimilar p and q are, the more the divergence will increase.

Original: 3. P OST -T RAINING B IAS C ORRECTION
Revised: 3. P OST -T RAINING B IAS C ORRECTION

Original: At this point, we will show that we can significantly reduce the prediction bias of a task-free continual learner if we appropriately change the parameter vector θθθ c of the classifier after the entire stream has been observed.
Revised: At this point, we will show that we can significantly reduce the prediction bias of a task-free continual learner if we appropriately change the parameter vector θ c of the classifier after the entire stream has been observed.

Original: We will compare three approaches over four datasets (we follow the experimental settings described in Section 4.1).
Revised: We will compare three approaches over four datasets (we follow the experimental settings described in Section 4.1).

Original: We report both the accuracy and the bias (according to Eq. 2) after the end of the stream.
Revised: We report both the accuracy and the bias (according to Eq. 2) after the end of the stream.

Original: We compare regular experience replay (ER), which weights equally its present and past, and memory-replay only (MRO), which uses only data stored in memory to train the model.
Revised: We compare regular experience replay (ER), which weights equally its present and past, and memory-replay only (MRO), which uses only data stored in memory to train the model.

Original: Moreover, we also evaluate a version of ER with post-training bias correction (ER-BC).
Revised: Moreover, we also evaluate a version of ER with post-training bias correction (ER-BC).

Original: To correct the bias of the model, we use the data stored in memory to train only the classifier of the model until convergence.
Revised: To correct the bias of the model, we use the data stored in memory to train only the classifier of the model until convergence.

Original: The feature extractor of the model remains unchanged.
Revised: The feature extractor of the model remains unchanged.

Original: The results of this comparison are presented in Table 1.
Revised: The results of this comparison are presented in Table 1.

Original: We observe that MRO is less biased than
Revised: We observe that MRO is less biased than

Original: ER in all four datasets.
Revised: ER in all four datasets.

Original: On the other hand, despite being more biased, ER outperforms MRO in
Revised: On the other hand, despite being more biased, ER outperforms MRO in

Original: CIFAR-10, CIFAR-100, and tinyImageNet.
Revised: CIFAR-10, CIFAR-100, and tinyImageNet.

Original: As we hypohtesized at the end of Section 3.2, training a model using only the data stored in memory will reduce the prediction bias of that model, but willalso lead to overfitting.
Revised: As we hypohtesized at the end of Section 3.2, training a model using only the data stored in memory will reduce the prediction bias of that model, but will

Original: On the contrary, ER+BC is less biased than ER, and achieves significantly higher accuracy values on all four datasets.
Revised: [Figure]

Original: These results are evidence that prediction bias can be corrected just by changing the way the classifier is trained.
Revised: These results are evidence that prediction bias can be corrected just by changing the way the classifier is trained.

Original: Moreover, because the classifier consists of just one layer, it has lower learning[Figure] capacity, and, therefore, overfits much less than if we were to train the entire model using only the data in memory (as is the case with MRO).
Revised: Moreover, because the classifier consists of just one layer, it has lower learning capacity, and, therefore, overfits much less than if we were to train the entire model using only the data in memory (as is the case with MRO).

Original: In the following section, we exploit the findings of this section to propose an algorithm that maintains an unbiased model throughout the entire stream.
Revised: In the following section, we exploit the findings of this section to propose an algorithm that maintains an unbiased model throughout the entire stream.

Original: 3. O NLINE B IAS C ORRECTION
Revised: 3. O NLINE B IAS C ORRECTION

Original: Our approach is called Online Bias Correction (OBC) and acts as a wrapper around other task-free continual learning methods that perform experience replay.
Revised: Our approach is called Online Bias Correction (OBC) and acts as a wrapper around other task-free continual learning methods that perform experience replay.

Original: At first, a generic task-free continual learning method performs its training step.
Revised: At first, a generic task-free continual learning method performs its training step.

Original: Such a training step, typically includes receiving a minibatch of observations SSS t from the stream (Line 1), sampling another minibatch RRR from memory (Line 2), and performing a training step using the combination of the two minibatches (Line 3).
Revised: Such a training step, typically includes receiving a minibatch of observations S t from the stream (Line 1), sampling another minibatch R from memory (Line 2), and performing a training step using the combination of the two minibatches (Line 3).

Original: Note that only the parameter vector of the feature extractor (FE), and that of the surrogate classifier (SC) are updated during this training step.
Revised: Note that only the parameter vector of the feature extractor (FE), and that of the surrogate classifier (SC) are updated during this training step.

Original: Next, the learner decides which of the new observations in SSS t to store in memory, and which ones to discard (Line 4).
Revised: Next, the learner decides which of the new observations in S t to store in memory, and which ones to discard (Line 4).

Original: Algorithm

Original: Stream minibatches SSS t , t = 1 , 2 , . . 
Revised: Stream minibatches S t , t = 1 , 2 , . . . ,

Original: Memory MMM , Feature Extractor (FE),
Revised: Memory M , Feature Extractor (FE),

Original: Classifier (C), Surrogate Classifier (SC) 1: for each stream minibatch SSS t do 2:
Revised: Classifier (C), Surrogate Classifier (SC) 1: for each stream minibatch S t do 2:

Original: Sample a memory minibatch RRR ∼ MMM 3:
Revised: Sample a memory minibatch R ∼ M 3:

Original: Train the FE and the SC using RRR ∪ SSS t 4:
Revised: Train the FE and the SC using R ∪ S t 4:

Original: Perform memory population using SSS t 5:
Revised: Perform memory population using S t 5:

Original: Freeze the FE 6:
Revised: Freeze the FE 6:

Original: Sample a memory minibatch RRR ∗ ∼ MMM 7:
Revised: Sample a memory minibatch R ∗ ∼ M 7:

Original: Train the C using RRR ∗ 8:
Revised: Train the C using R ∗ 8:

Original: Unfreeze the FE 9: end for
Revised: Unfreeze the FE 9: end for

Original: The following four steps (i.e., Lines 5-8) are the ones that OBC introduces.
Revised: The following four steps (i.e., Lines 5-8) are the ones that OBC introduces.

Original: First, the feature extractor is frozen (Line 5)—that is, in the update step that follows, there will be no gradients with respect to its parameter vector calculated in the backward pass, and thus its parameter vector will not be updated.
Revised: First, the feature extractor is frozen (Line 5)—that is, in the update step that follows, there will be no gradients with respect to its parameter vector calculated in the backward pass, and thus its parameter vector will not be updated.

Original: Next, a new minibatch RRR ∗ is sampled from the memory (Line 6).
Revised: Next, a new minibatch R ∗ is sampled from the memory (Line 6).

Original: This memory minibatch is feedforwarded through the feature extractor and then the classifier (C), and only the parameter vector of the classifier, is updated with the resulting gradients (Line 7).
Revised: This memory minibatch is feedforwarded through the feature extractor and then the classifier (C), and only the parameter vector of the classifier, is updated with the resulting gradients (Line 7).

Original: Since the classifier is trained using only memory data, in order to reduce overfitting, we make use of label smoothing (Szegedy et al., 2016), which is a technique that adds noise to the ground-truth labels, in order to discourage overconfident predictions.
Revised: Since the classifier is trained using only memory data, in order to reduce overfitting, we make use of label smoothing (Szegedy et al., 2016), which is a technique that adds noise to the ground-truth labels, in order to discourage overconfident predictions.

Original: Finally, the feature extractor is unfrozen, meaning that its parameter vector is trainable again (Line 8).
Revised: Finally, the feature extractor is unfrozen, meaning that its parameter vector is trainable again (Line 8).

Original: The design of OBC was motivated by the preliminary experiments of Section 3.4.
Revised: The design of OBC was motivated by the preliminary experiments of Section 3.4.

Original: In particular, we exploit the fact that, as we saw earlier, it is possible to mitigate prediction bias by just optimizing the weights of the classifier with respect to the data in memory.
Revised: In particular, we exploit the fact that, as we saw earlier, it is possible to mitigate prediction bias by just optimizing the weights of the classifier with respect to the data in memory.

Original: Moreover, the classifier, in the context discussed here, consists of a single layer, hence it is more resistant to overfitting compared to the feature extractor, which is typically a deep neural network of high learning capacity.
Revised: Moreover, the classifier, in the context discussed here, consists of a single layer, hence it is more resistant to overfitting compared to the feature extractor, which is typically a deep neural network of high learning capacity.

Original: Therefore, by training the classifier only using memory data, we mitigate the data-sampling bias, and hence,[Table] its prediction bias.
Revised: Therefore, by training the classifier only using memory data, we mitigate the data-sampling bias, and hence, its prediction bias.

Original: Regarding the feature extractor, as we saw in Section 3.4, it is beneficial for it to be trained via experience replay because this approach leads to less overfitting compared to training the feature extractor only with memory data.
Revised: Regarding the feature extractor, as we saw in Section 3.4, it is beneficial for it to be trained via experience replay, possibly because this approach leads to less overfitting compared to training the feature extractor only with memory data.

Original: Therefore, we introduced a surrogate classifier in order to ensure that the feature extractor is trained in exactly the same way as  experience replay (i.e., in conjunction with a biased classifier).
Revised: Therefore, we introduced a surrogate classifier in order to ensure that the feature extractor is trained in exactly the same way as in experience replay (i.e., in conjunction with a biased classifier).

Original: This design choice is also validated  experimentally in Section 4.4.
Revised: We have also validated this design choice experimentally in Section 4.4.

Original: In summary, OBC attempts to capture the benefits of both experience replay (less feature-extractor overfitting), and training the classifierwith only  memory data (less prediction bias), in a best-of-both-worlds manner.
Revised: In short, OBC attempts to capture the benefits of both experience replay (less feature-extractor overfitting), and training the classifier only with memory data (less prediction bias), in a best-of-bothworlds manner.

Original:  E XPERIMENTS 4. E XPERIMENTAL S ETTINGS
Revised:  E XPERIMENTS 4. E XPERIMENTAL S ETTINGS

Original: Datasets and Experimental Setup
Revised: Datasets and Experimental Setup

Original: We use four datasets of varying difficulty.
Revised: We use four datasets of varying difficulty.

Original: The FashionMNIST dataset (Xiao et al., 2017) contains 60,000 grayscale images of clothing items split in 10 classes.
Revised: The FashionMNIST dataset (Xiao et al., 2017) contains 60,000 grayscale images of clothing items split in 10 classes.

Original: CIFAR-10 and CIFAR-100 (Krizhevsky, 2009) each contain 50,000 color images, with the ones in
Revised: CIFAR-10 and CIFAR-100 (Krizhevsky, 2009) each contain 50,000 color images, with the ones in

Original: CIFAR-10 being divided in 10 classes, and the ones in CIFAR-100 in 100 classes.
Revised: CIFAR-10 being divided in 10 classes, and the ones in CIFAR-100 in 100 classes.

Original: Finally, tinyImageNet (Le & Yang, 2015) contains 100,000 color images of 200 different classes.
Revised: Finally, tinyImageNet (Le & Yang, 2015) contains 100,000 color images of 200 different classes.

Original: The tinyImageNet dataset is the most challenging dataset widely used in task-free continual learning literature, mainly due to its large number of classes (200) and the small number of data instances per class (500).
Revised: The tinyImageNet dataset is the most challenging dataset widely used in task-free continual learning literature, mainly due to its large number of classes (200) and the small number of data instances per class (500).

Original: Our experimental setup closely follows previous work (Aljundi et al., 2019b; Jin et al., 2021; Caccia et al., 2022).
Revised: Our experimental setup closely follows previous work (Aljundi et al., 2019b; Jin et al., 2021; Caccia et al., 2022).

Original: We use class-incremental streams that are presented to the learner online, in small minibatches.
Revised: We use class-incremental streams that are presented to the learner online, in small minibatches.

Original: For FashionMNIST and CIFAR-10, we use five binary tasks; for CIFAR-100 and tinyImageNet, we use ten and twenty tasks, respectively, each containing ten classes.
Revised: For FashionMNIST and CIFAR-10, we use five binary tasks; for CIFAR-100 and tinyImageNet, we use ten and twenty tasks, respectively, each containing ten classes.

Original: We follow the majority of past work (Aljundi et al., 2019e; Chrysakis & Moens, 2020a; Aljundi et al., 2019b) by not using data augmentation in our experiments.
Revised: We follow the majority of past work (Aljundi et al., 2019e; Chrysakis & Moens, 2020; Aljundi et al., 2019b) by not using data augmentation in our experiments.

Original: Methods Experience replay (ER) (Isele & Cosgun, 2018; Chaudhry et al., 2019) is the most fundamental continual learning baseline.
Revised: Methods Experience replay (ER) (Isele & Cosgun, 2018; Chaudhry et al., 2019) is the most fundamental continual learning baseline.

Original: It performs replay using a memory that is populated using reservoir sampling (Vitter, 1985a).
Revised: It performs replay using a memory that is populated using reservoir sampling (Vitter, 1985).

Original: Memory-replay only (MRO) also uses reservoir sampling to populate the memory, but instead of using both stream and memory data, it only trains the model using data from the memory.
Revised: Memory-replay only (MRO) also uses reservoir sampling to populate the memory, but instead of using both stream and memory data, it only trains the model using data from the memory.

Original: Maximally-interfered retrieval (MIR) (Aljundi et al., 2019b) is an extension of ER that replays the instances which would experience the largest loss increases, if the model were to be updated using only the current minibatch of observations.
Revised: Maximally-interfered retrieval (MIR) (Aljundi et al., 2019b) is an extension of ER that replays the instances which would experience the largest loss increases, if the model were to be updated using only the current minibatch of observations.

Original: Class-balancing reservoir sampling (CBRS) (Chrysakis & Moens, 2020a) uses a memory population algorithm that attempts to maintain the memory as class-balanced as possible at all times.
Revised: Class-balancing reservoir sampling (CBRS) (Chrysakis & Moens, 2020) uses a memory population algorithm that attempts to maintain the memory as class-balanced as possible at all times.

Original: Greedy sampler and dumb learner (GDUMB)
Revised: Greedy sampler and dumb learner (GDUMB)

Original: (Prabhu et al., 2020a) also uses a class-balancing memory population algorithm and trains the model using only data stored in memory.
Revised: (Prabhu et al., 2020) also uses a class-balancing memory population algorithm and trains the model using only data stored in memory.

Original: 3 Gradient-based memory editing (GMED) (Jin et al., 2021) edits the data stored in memory in order to increase their loss values and make them more challenging.
Revised: 3 Gradient-based memory editing (GMED) (Jin et al., 2021) edits the data stored in memory in order to increase their loss values and make them more challenging.

Original: Finally, asymmetric cross entropy (ACE) (Caccia et al., 2022) modifies the traditional cross-entropy loss with class-masking that reduces representation drift during continual learning.
Revised: Finally, asymmetric cross entropy (ACE) (Caccia et al., 2022) modifies the traditional cross-entropy loss with class-masking that reduces representation drift during continual learning.

Original: Architectures and Hyperparameters
Revised: Architectures and Hyperparameters

Original: Similar to previous work (Lopez-Paz & Ranzato, 2017;
Revised: Similar to previous work (Lopez-Paz & Ranzato, 2017;

Original: Aljundi et al., 2019a), we use a reduced ResNet-18 architecture (He et al., 2016) for CIFAR-10,
Revised: Aljundi et al., 2019a), we use a reduced ResNet-18 architecture (He et al., 2016) for CIFAR-10,

Original: CIFAR-100, and tinyImageNet.
Revised: CIFAR-100, and tinyImageNet.

Original: For the simpler FashionMNIST, we use a simple convolutional neural network (CNN).
Revised: For the simpler FashionMNIST, we use a simple convolutional neural network (CNN).

Original: For more information on these architectures, please refer to the appendix.
Revised: For more information on these architectures, please refer to the appendix.

Original: Following previous work (Aljundi et al., 2019b; Jin et al., 2021), we use a learning rate of 0.1 when using the reduced ResNet-18 architecture.
Revised: Following previous work (Aljundi et al., 2019b; Jin et al., 2021), we use a learning rate of 0.1 when using the reduced ResNet-18 architecture.

Original: When using the simpler CNN, we use a learning rate of 0.03.
Revised: When using the simpler CNN, we use a learning rate of 0.03.

Original: The stream and replay batch sizes were both set to 10, in accordance with past work.
Revised: The stream and replay batch sizes were both set to 10, in accordance with past work.

Original: The batch size of OBC was set to 50 (please refer to the appendix for a sensitivity analysis.)
Revised: The batch size of OBC was set to 50 (please refer to the appendix for a sensitivity analysis.)

Original: Past work typically  uses memory sizes in the range of 1% – 10% of the size of the stream.
Revised: Past work typically [Table] uses memory sizes in the range of 1% – 10% of the size of the stream.

Original: Hence, unless explicitly mentioned otherwise, we set the memory size to 1000 for both FashionMNIST (approximately 2%) and CIFAR-10 (2%), and to 2500 for CIFAR-100 (5%) and 5000 for tinyImageNet (5%).
Revised: Hence, unless explicitly mentioned otherwise, we set the memory size to 1000 for both FashionMNIST (approximately 2%) and CIFAR-10 (2%), and to 2500 for CIFAR-100 (5%) and 5000 for tinyImageNet (5%).

Original: 4 For OBC, and only when training the classifier, we use a batch size of 50, and a label-smoothing 5 factor of 0.
Revised: 4 For OBC, and only when training the classifier, we use a batch size of 50, and a label-smoothing 5 factor of 0.

Original: (we perform a sensitivity analysis of these two hyperparameters in the appendix).
Revised: (we perform a sensitivity analysis of these two hyperparameters in the appendix).

Original: Method-specific hyperparameters were set to the values given in their respective papers.
Revised: Method-specific hyperparameters were set to the values given in their respective papers.

Original: Evaluation Metrics We calculate the accuracy and the proposed prediction bias metric (please refer to Section 3.4) after the entire stream has been observed.
Revised: Evaluation Metrics We calculate the accuracy and the proposed prediction bias metric we proposed in Section 3.3, after the entire stream has been observed.

Original: Both of these metrics are calculated with respect to the unseen test set.
Revised: Both of these metrics are calculated with respect to the unseen test set.

Original: These metrics inform us about how well the model has learned after the end of learning.
Revised: These metrics inform us about how well the model has learned after the end of learning.

Original: We also propose to use the information retention metric (accuracy computed with respect to past observations) proposed in Cai et al.
Revised: We also propose to use the information retention metric (accuracy computed with respect to past observations) proposed in Cai et al.

Original: (2021), averaged over the entire stream.
Revised: (2021), averaged over the entire stream.

Original: This metric is a form of continual evaluation that evaluates how well each method performs, not only after the end of learning, but during its entire length.
Revised: This metric is a form of continual evaluation that evaluates how well each method performs, not only after the end of learning, but during its entire length.

Original: Continual evaluation is crucial in real-world applications where a model needs to train andperform inference  interchangeably.
Revised: Continual evaluation is crucial for real-world applications where training and inference take place interchangeably.

Original: 4. A PPLYING OBC
Revised: 4. A PPLYING OBC

Original: At first we apply OBC to four state-of-the-art, task-free continual learning methods.
Revised: At first we apply OBC to four state-of-the-art, task-free continual learning methods.

Original: Namely, we compare the performance (final accuracy and information retention averaged over the stream) of
Revised: Namely, we compare the performance (final accuracy and information retention averaged over the stream) of

Original: ER, MIR, CBRS, and GMED, with and without OBC on four different datasets (see Table 2).
Revised: ER, MIR, CBRS, and GMED, with and without OBC on four different datasets (see Table 2).

Original: (Due to of lack of space, we report the prediction bias numbers of these experiments in the appendix.)
Revised: (Due to of lack of space, we report the prediction bias numbers of these experiments in the appendix.)

Original: We observe that OBC improves both the learning performance of each method over the continuum (Av. IR), and their final accuracy, for all four datasets.
Revised: We observe that OBC improves both the learning performance of each method over the continuum (Av. IR), and their final accuracy, for all four datasets.

Original: The performance boost that OBC provides is especially evident in CIFAR-10, CIFAR-100, and tinyImageNet.
Revised: The learning benefit that OBC provides is especially evident in CIFAR-10, CIFAR-100, and tinyImageNet.

Original: We note that methods that perform post-training bias correction (as the one we proposed in Section 3.4, or the NCM approach proposed by Mai et al.
Revised: We note that methods that perform post-training bias correction (as the one we proposed in Section 3.4, or the NCM approach proposed by Mai et al.

Original: (2021)) would only affect the final accuracy of each method, but not the average information retention over the stream, because, by definition, the bias correction takes place after the end of training.
Revised: (2021)) would only affect the final accuracy of each method, but not the average information retention over the stream, because, by definition, the bias correction takes place after the end of training.

Original: Keeping in mind that OBC only changes the way the classifier is trained, we argue that when applying bias correction to various methods, we can essentially compare the quality of the representations that their feature extractor learns.
Revised: Since OBC only modifies the way the classifier is trained, we argue that when applying bias correction to various methods, we can essentially compare the quality of the representations that their feature extractor learns.

Original: In the results presented here, there is no clear best method in that respect, but future work could specifically focus on how to make sure that the feature extractor learns better representations.
Revised: In the results presented here, there is no clear best method in that respect, but future work could specifically focus on improving feature-extractor representation learning. [Table] IR), and the prediction bias (Pred. Bias) of each approach.

Revised: All entries are 95% -confidence intervals over 15 runs.

Original: 4. C OMPARISON WITH O THER B IAS C ORRECTION A PPROACHES
Revised: [Table] [Table] C OMPARISON WITH O THER B IAS C ORRECTION A PPROACHES

Original: At this point, we will compare OBC to three other methods that perform implicit bias correction (in the sense that they were not designed specifically to correct for prediction bias, but they do so anyway).
Revised: At this point [Table] we will compare OBC to three other methods that perform implicit bias correction (in the sense that they were not designed specifically to correct for prediction bias

Original: In particular, we compare OBC to GDUMB, MRO (they are not biased by the stream in the first place since they both train models using memory data only), and ACE (it uses both stream and replay data, but with a masked cross-entropy, which is aimed to reduce representation drift,

Original: [Table]
Revised: [Table]

Revised: but they do so anyway).

Revised: In particular

Original: [Table]
Revised: [Table]

Original:  but also does not lead to biased predictions).
Revised: we compare OBC to GDUMB, MRO (they are not biased by the stream in the first place since they both train models using memory data only), and ACE (it uses both stream and replay data, but with a masked cross-entropy, which is aimed to reduce representation drift, but also does not lead to biased predictions).

Original: We also include ER in this comparison, as a biased baseline.
Revised: We also include ER in this comparison, as a biased baseline.

Original: The comparison takes place in CIFAR-10 (see Table 3) and CIFAR-100 (see Table 4).
Revised: The comparison takes place on CIFAR-10 (see Table 3) and CIFAR-100 (see Table 4).

Original: We use memory sizes that correspond to 1% , 2% , 5% of the size of CIFAR-10, and 2% , 5% , 10% of the size of CIFAR-100.
Revised: We use memory sizes that correspond to 1% , 2% , 5% of the size of CIFAR-10, and 2% , 5% , 10% of the size of CIFAR-100.

Original: For CIFAR-10, we can see that OBC outperforms GDUMB and MRO [Table] and is competitive with ACE for very small memories of 500 instances.
Revised: For CIFAR-10, we can see that OBC outperforms GDUMB and MRO, and is competitive with ACE for very small memories of 500 instances.

Original: For the other two memory sizes,
Revised: For the other two memory sizes,

Original: OBC outperforms GDUMB, MRO, and ACE, with respect to all three metrics used.
Revised: OBC outperforms GDUMB, MRO, and ACE, with respect to all three metrics used.

Original: In CIFAR-100,
Revised: In CIFAR-100,

Original: OBC outperforms GDUMB, MRO, and ACE, for all three memory sizes.
Revised: OBC outperforms GDUMB, MRO, and ACE, for all three memory sizes.

Original: Interestingly, wewant to point out that for both CIFAR-10 and CIFAR-100, the gaps in accuracy and average information retention between OBC and the other three bias-correction methods increase with larger memory sizes. 
Revised: Interestingly, we point out that for both CIFAR-10 and CIFAR-100, the gaps in accuracy and average information retention between OBC and the other three bias-correction methods increase with larger memory sizes. [Table]

Original: 4. T HE N EED FOR A S URROGATE C LASSIFIER
Revised: 4. T HE N EED FOR A S URROGATE C LASSIFIER

Original: Finally, we want to explain why we need to introduce a surrogate classifier in the OBC paradigm.
Revised: Finally, we experimentally motivate the introduction of a surrogate classifier in the OBC paradigm.

Original: We compare the performance of OBC when applied to ER, with and without the use of a surrogate classifier.
Revised: We compare the performance of OBC when applied to ER, with and without the use of a surrogate classifier.

Original: In the latter case, the computational graph in the left part of Figure 1 flows from the feature extractor to the classifier, but the classifier is not updated with the resulting gradients.
Revised: In the latter case, the computational graph in the left part of Figure 1 flows from the feature extractor to the classifier, but the classifier is not updated with the resulting gradients.

Original: The right part of Figure 1 remains unchanged.
Revised: The right part of Figure 1 remains unchanged.

Original: The results are presented in Table 5.
Revised: The results are presented in Table 5.

Original: We observe that both for CIFAR-10 and CIFAR-100, the alternative formulation of OBC without a surrogate classifier  performs worse than OBC with a surrogate classifier.
Revised: We observe that both for CIFAR-10 and CIFAR-100, the alternative formulation of OBC without a surrogate classifier (OBC – SC) performs worse than OBC with a surrogate classifier.

Original: These results validate the need for using a surrogate classifier.
Revised: These results validate the use of the surrogate classifier.

Original: Since OBC is appliedin both cases only for the optimization of the classifier, we can only conclude that not using aWe report the accuracy training the information retention averaged [Table] surrogate classifier somehow harms the feature extractor.
Revised: Since OBC is applied only for the optimization of the classifier in both cases, we can only conclude that not using a surrogate classifier negatively affects the feature extractor.

Original: In other words, it appears that the feature extractor learns better representations when it is trained in combination with the biased surrogate classifier than when it is trained combined with the unbiased classifier.
Revised: In other words, it appears that the feature extractor learns better representations when it is trained in combination with the biased surrogate classifier than when it is trained combined with the unbiased classifier.

Original: Future work focusing on better representation learning in task-free settings could possibly interpret this finding further.
Revised: Future work focusing on better representation learning in task-free settings could further interpret this finding.

Original: 4. D ISCUSSION
Revised: 4. D ISCUSSION

Original: As we saw in Section 4.2, OBC performs reliably for multiple methods and over multiple datasets.
Revised: As we saw in Section 4.2, OBC performs reliably for multiple methods and over multiple datasets.

Original: In Section 4.3, we showed that it also outperforms other methods that perform bias correction implicitly, for different memory sizes, and in two different datasets.
Revised: In Section 4.3, we showed that it also outperforms other methods that perform bias correction implicitly, for different memory sizes, and for two different datasets.

Original: In our view, one important point this work makes is that, in the context of task-free continual learning, we should not necessarily be viewing a neural network as a single black box.
Revised: In our view, one important point this work makes is that, in the context of task-free continual learning, we should not necessarily be viewing a neural network as a single black box.

Original: Instead, we should make a distinction between the feature extractor and the classifier, because of their different learning capacities.
Revised: Instead, we should make a distinction between the feature extractor and the classifier, because of their different learning capacities.

Original: In particular, our OBC approach trains the classifier of the neural network online using only memory data, in order to mitigate its prediction bias.
Revised: In particular, our OBC approach trains the classifier of the neural network online using only memory data, in order to mitigate its prediction bias.

Original: Moreover, the feature extractor and the surrogate classifier are trained via experience replay, in order to prevent the overfitting that would result if the feature extractor were to be trained only using memory data.
Revised: Moreover, the feature extractor and the surrogate classifier are trained via experience replay, in order to prevent the overfitting that would result if the feature extractor were to be trained only using memory data.

Original: An interesting direction for future work would be to investigate the extent to which the data-sampling bias affects the feature extractor of the learner.
Revised: An interesting direction for future work would be to investigate the extent to which the data-sampling bias affects the feature extractor of the learner.

Original: One potential limitation of OBC is that it does not improve the feature extractor of the model.
Revised: One potential limitation of OBC is that it does not improve the feature extractor of the model.

Original: Consequently, the final performance after applying OBC will depend on the quality of the representations learned by the method being wrapped.
Revised: Consequently, the final performance after applying OBC will depend on the quality of the representations learned by the method being wrapped.

Original: Therefore, another interesting avenue for future work, would be to focus on how to improve task-free continual representation learning.
Revised: Therefore, another interesting avenue for future work, would be to focus on how to improve task-free continual representation learning.

Original: Despite this limitation, we argue that OBC can be used to disentangle the confounding factors of prediction bias and representation learning.
Revised: Finally, we hypothesize that OBC can be used to disentangle the confounding factors of prediction bias and representation learning.

Original: In particular, when two task-free continual learning methods use the same memory population algorithm and their classifiers are both trained using OBC, any differences in learning performance can be explained by the quality of the representations  their  feature extractors learn.
Revised: In particular, when two task-free continual learning methods use the same memory population algorithm and their classifiers are both trained using OBC, any differences in learning performance can be explained by the quality of the representations learned by their respective feature extractors.

Original: For example, if a method A learns better representations but is also more biased than approach B, it is possible that method B would achieve higher accuracy.
Revised: For example, if a method A learns better representations but is also more biased than approach B, it is possible that method B would achieve higher accuracy.

Original: If we correct for the bias in both methods using our proposed bias-correction approach, wewould expect method A to outperform method B, because A learns better representations than B.
Revised: But, if we correct for the bias in both methods using our proposed bias-correction approach, we expect method A to outperform method B, since A learns better representations than B.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: To summarize, this work discusses the issue of prediction bias in task-free continual learning.
Revised: To summarize, this work discusses the issue of prediction bias in task-free continual learning.

Original: In particular, we provided a concrete explanation of how this bias is caused—namely, by the experiencereplay paradigm favoring current stream observations over the past data stored in memory.
Revised: In particular, we provided a concrete explanation of how this bias is caused—namely, by the experiencereplay paradigm favoring current stream observations over the past data stored in memory.

Original: Subsequently, we proposed an evaluation metric that quantifies the prediction bias of a model on an unseen test set.
Revised: Subsequently, we proposed an evaluation metric that quantifies the prediction bias of a model on an unseen test set.

Original: More importantly, we proposed a simple approach called Online Bias Correction (OBC) that can correct this prediction bias online throughout the entire duration of learning.
Revised: More importantly, we proposed a simple approach called Online Bias Correction (OBC) that can correct this prediction bias online throughout the entire duration of learning.

Original: Because of its online nature, OBC is especially useful for real-world applications in which a model needs to learn and perform inference at the same time.
Revised: Because of its online nature, OBC is especially useful for real-world applications in which a model needs to learn and perform inference at the same time.

Original: Also, OBC is trivial to implement and can be applied as a wrapper over other task-free continual learning methods in order to increase their accuracy and their information retention, and reduce their prediction bias. 
Revised: Also, OBC is trivial to implement and can be applied as a wrapper over other task-free continual learning methods in order to increase their accuracy and their information retention, and reduce their prediction bias. A CKNOWLEDGEMENTS This work is part of the CALCULUS project ( https://calculus-project.eu/ ). The

Revised: CALCULUS project is funded by the ERC Advanced Grant H2020-ERC-2017-ADG 788506.

Original: R EPRODUCIBILITY
Revised: R EPRODUCIBILITY

Original: In order to ensure that our work is not only reproducible but also easily accessible, we commit to open sourcing our code and any necessary configuration files, upon acceptance. Moreover, in order to increase the reproducibility of our results, whenever we present average performance over many runs, we do so in the form 95% -confidence intervals, so that the numbers reflect their inherent uncertainty.
Revised: Our code can be found at https://github.com/chrysakis/OBC .

Original: A H YPERPARAMETER S ENSITIVITY
Revised: A H YPERPARAMETER S ENSITIVITY

Original: We now examine how sensitive OBC is to its hyperparameters.
Revised: We now examine how sensitive OBC is to its hyperparameters.

Original: (Note that other hyperparameters such as the stream batch size are applicable, not directly to OBC, but only to the method that OBC wraps.) 
Revised: (Note that other hyperparameters such as the stream batch size are applicable, not directly to OBC, but only to the method that OBC wraps.) In Figure

Original: In Figure 2, we present the final accuracy of ER+OBC for various values of the OBC batch size (10, 20, 50, 100, 200), and the label-smoothing factor (0.2, . . . ,0.8).
Revised: [Figure] we present the final accuracy of ER+OBC for various values of the OBC batch size (10, 20, 50, 100, 200) [Figure] and the label-smoothing factor (0.2, . . . ,0.8).

Original: The accuracies correspond to CIFAR-100 for memory sizes of 1000 and 2500, and are presented as 95% -confidence intervals.
Revised: The accuracies correspond to CIFAR-100 for memory sizes of 1000 and [Figure] and are presented as 95% -confidence intervals.

Original: The batch-size curves are very similar for both memory sizes.
Revised: The batch-size curves are very similar for both memory sizes.

Original: Higher batch sizes are correlated with higher accuracies, but with diminishing returns.
Revised: Higher batch sizes are correlated with higher accuracies [Figure] but with diminishing returns.

Original: For the label-smoothing factor, we can see that relatively large values (0.7 or 0.8) lead to lower accuracies for both memory sizes.
Revised: For the label-smoothing factor, we can see that relatively large values (0.7 or 0.8) lead to lower accuracies for both memory sizes.

Original: In addition, only for the smaller memory of 1000 instances, we also observe that a relatively small label-smoothing factor of 0.2 or 0.3 also leads to slightly reduced accuracies.
Revised: In addition, only for the smaller memory of 1000 instances, we also observe that a relatively small label-smoothing factor of 0.2 or 0.3 also leads to slightly reduced accuracies [Figure] n [Figure]

Revised: l

Revised: [Figure]

Original:  B A RCHITECTURES
Revised: y [Figure] [Table] B A RCHITECTURES

Original: In Table 6, we report the architectures used in our experiments.
Revised: In Table [Table] we report the architectures used in our experiments [Table]

Original: C R ESULTS ON COR E
Revised: C R ESULTS ON COR E

Original: In Table 7, we present results on the temporally coherent dataset CORe50 (Lomonaco & Maltoni, 2017).
Revised: In Table [Table] we present results on the temporally coherent dataset CORe50 (Lomonaco & Maltoni, 2017).

Original: This dataset contains sequential video frames of 50 objects filmed during 11 different ses-
Revised: This dataset contains sequential video frames of 50 objects filmed during 11 different ses-

Original: [Table] [Table]
Revised: [Table] [Table]

Original: sions, with each session consisting of approximately 300 frames.
Revised: sions, with each session consisting of approximately 300 frames.

Original: As Lomonaco & Maltoni (2017) suggest, sessions 3, 7, and 10 are used for evaluation purposes (approximately 45,000 images), and the remaining 8 sessions are used to construct the stream (approximately 120,000 images).
Revised: As Lomonaco & Maltoni (2017) suggest, sessions 3, 7, and 10 are used for evaluation purposes (approximately 45,000 images), and the remaining 8 sessions are used to construct the stream (approximately 120,000 images).

Original: The stream consists of ten tasks of five objects each.
Revised: The stream consists of ten tasks of five objects each.

Original: We used the reduced ResNet-18 architecture described earlier, with a learning rate of 0.01 and a memory size of 2400 (2% of the size of the stream).
Revised: We used the reduced ResNet-18 architecture described earlier, with a learning rate of 0.01 and a memory size of 2400 (2% of the size of the stream).

Original: The remaining hyperparameters are identical to the ones described in the main paper.
Revised: The remaining hyperparameters are identical to the ones described in the main paper.

Original: As before, we compare four task-free continual learning methods, with and without OBC, and we report the accuracy and the prediction bias calculated with respect to the unseen test set, and the information retention (accuracy on past observations) averaged over the stream.
Revised: As before, we compare four task-free continual learning methods, with and without OBC, and we report the accuracy and the prediction bias calculated with respect to the unseen test set, and the information retention (accuracy on past observations) averaged over the stream.

Original: We observe that the use of OBC results to higher accuracy and average information retention, and lower prediction bias, for all four methods.
Revised: We observe that the use of OBC results to higher accuracy and average information retention, and lower prediction bias, for all four methods.

Original: D R EDUCTIONS IN P REDICTION B IAS WHEN U SING OBC
Revised: D R EDUCTIONS IN P REDICTION B IAS WHEN U SING OBC

Original: In Table 8, we report the prediction bias results from the experiments of Section 4.2, which were not included in the main text due to lack of space.
Revised: In Table 8, we report the prediction bias results from the experiments of Section 4.2, which were not included in the main text due to lack of space.

Original: These results suggest that the use of OBC leads to significant reductions in prediction bias.
Revised: These results suggest that the use of OBC leads to significant reductions in prediction bias.

Original: These reductions are consistent for all four methods and all four datasets.
Revised: These reductions are consistent for all four methods and all four datasets.

Original: E A N UMERICAL E XAMPLE OF D ATA -S AMPLING B IAS
Revised: E A N UMERICAL E XAMPLE OF D ATA -S AMPLING B IAS

Original: Consider a learner that has previously observed data 990 instances, and these instances are currently stored in the learner’s memory.
Revised: Consider a learner that has previously observed data 990 instances, and these instances are currently stored in the learner’s memory.

Original: A new minibatch of 10 instances is now observed by the learner.
Revised: A new minibatch of 10 instances is now observed by the learner.

Original: If the learner wants to update the model in an unbiased way, all 990 + 10 data instances should be equally likely to participate in this update.
Revised: If the learner wants to update the model in an unbiased way, all 990 + 10 data instances should be equally likely to participate in this update.

Original: Assuming that 20 instances will be used in the update, the probability that any instance participates is therefore 20 / (990 + 10)
Revised: Assuming that 20 instances will be used in the update, the probability that any instance participates is therefore 20 / (990 + 10)

Original: = 0 .
Revised: = 0 .

Original: In experience replay, however, the 10 instances of the new minibatch are guaranteed to participate in the update (therefore with a probability of 1 ), while an equal number of instances would be randomly sampled from memory.
Revised: In experience replay, however, the 10 instances of the new minibatch are guaranteed to participate in the update (therefore with a probability of 1 ), while an equal number of instances would be randomly sampled from memory.

Original: Hence, each memory instance is sampled with a probability of 10 / 990 ≃ 0 .
Revised: Hence, each memory instance is sampled with a probability of 10 / 990 ≃ 0 .

Original: Therefore, in experience replay, the probability that a new instance will participate in the update is 1 / 0 .
Revised: Therefore, in experience replay, the probability that a new instance will participate in the update is 1 / 0 .

Original: 01 = 100 times larger than that of a random memory instance, while in the unbiased case, all 990 + 10 are equally likely to be sampled, regardless of whether they are new observations or stored in memory.
Revised: 01 = 100 times larger than that of a random memory instance, while in the unbiased case, all 990 + 10 are equally likely to be sampled, regardless of whether they are new observations or stored in memory.

Revised: F C ONTINUAL L EARNING , O NLINE L EARNING , AND D OMAIN A DAPTATION

Revised: For the sake of clarity we will provide a high-level comparison of continual learning, online learning, and domain adaptation.

Revised: In continual learning, the data distribution changes over time and, in general, previous work assumes that all observed instances are equally important, and that there is no distributional mismatch between the training and testing data.

Revised: Online learning is similar to continual learning in the sense that the data distribution changes over time.

Revised: Yet, the goal in online learning is for the model to appropriately adapt to the current state of the data distribution, therefore currently observed instances are more important than previous ones.

Revised: Finally, in the problem of domain adaptation , a model is trained offline using a training set of data.

Revised: Subsequently, the model needs to perform inference on data generated by a data distribution that is different (to a degree) from the one that generated the training data.