Original: C LASS -I NCREMENTAL L EARNING WITH R EPETITION
Revised: C LASS -I NCREMENTAL L EARNING WITH R EPETITION

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Real-world data streams naturally include the repetition of previous concepts.
Revised: Real-world data streams naturally include the repetition of previous concepts.

Original: From a Continual Learning (CL) perspective, repetition is a property of the environment and, unlike replay, cannot be controlled by the user.
Revised: From a Continual Learning (CL) perspective, repetition is a property of the environment and, unlike replay, cannot be controlled by the user.

Original: Nowadays, ClassIncremental scenarios represent the leading test-bed for assessing and comparing
Revised: Nowadays, ClassIncremental scenarios represent the leading test-bed for assessing and comparing

Original: CL strategies.
Revised: CL strategies.

Original: This family of scenarios is very easy to use, but it never allows revisiting previously seen classes, thus completely disregarding the role of repetition.
Revised: This family of scenarios is very easy to use, but it never allows revisiting previously seen classes, thus completely disregarding the role of repetition.

Original: We focus on the family of Class-Incremental with Repetition (CIR) scenarios, where repetition is embedded in the definition of the stream.
Revised: We focus on the family of Class-Incremental with Repetition (CIR) scenarios, where repetition is embedded in the definition of the stream.

Original: We propose two stochastic scenario generators that produce a wide range of CIR scenarios starting from a single dataset and a few control parameters.
Revised: We propose two stochastic scenario generators that produce a wide range of CIR scenarios starting from a single dataset and a few control parameters.

Original: We conduct the first comprehensive evaluation of repetition in CL by studying the behavior of existing
Revised: We conduct the first comprehensive evaluation of repetition in CL by studying the behavior of existing

Original: CL strategies under different CIR scenarios.
Revised: CL strategies under different CIR scenarios.

Original: We then present a novel replay strategy that exploits repetition and counteracts the natural imbalance present in the stream.
Revised: We then present a novel replay strategy that exploits repetition and counteracts the natural imbalance present in the stream.

Original: On both CIFAR100 and TinyImageNet, our strategy outperforms other replay approaches, which are not designed for environments with repetition.
Revised: On both CIFAR100 and TinyImageNet, our strategy outperforms other replay approaches, which are not designed for environments with repetition.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Continual Learning (CL) requires a model to learn new information from a stream of experiences presented over time, without forgetting previous knowledge (Parisi et al., 2019; Lesort et al., 2020).
Revised: Continual Learning (CL) requires a model to learn new information from a stream of experiences presented over time, without forgetting previous knowledge (Parisi et al., 2019; Lesort et al., 2020).

Original: The nature and characteristics of the data stream can vary a lot depending on the real-world environment and target application.
Revised: The nature and characteristics of the data stream can vary a lot depending on the real-world environment and target application.

Original: Class-Incremental (CI) scenarios (Rebuffi et al., 2017) are the most popular ones in CL.
Revised: Class-Incremental (CI) scenarios (Rebuffi et al., 2017) are the most popular ones in CL.

Original: CI requires the model to solve a classification problem where new classes appear over time.
Revised: CI requires the model to solve a classification problem where new classes appear over time.

Original: Importantly, when a set of new classes appears, the previous ones are never seen again.
Revised: Importantly, when a set of new classes appears, the previous ones are never seen again.

Original: However, the model still needs to correctly predict them at test time.
Revised: However, the model still needs to correctly predict them at test time.

Original: Conversely, in a
Revised: Conversely, in a

Original: Domain-Incremental (DI) scenario (van de Ven & Tolias, 2019)
Revised: Domain-Incremental (DI) scenario (van de Ven & Tolias, 2019)

Original: the model sees all the classes at the beginning and continue to observe new instances of the classes over time.
Revised: the model sees all the classes at the beginning and continue to observe new instances of the classes over time.

Original: The CI and DI scenarios have been very helpful to promote and drive CL research in the last few years.
Revised: The CI and DI scenarios have been very helpful to promote and drive CL research in the last few years.

Original: However, they strongly constrain the properties of the data stream in a way that it sometimes considered unrealistic or very limiting (Cossu et al., 2021).
Revised: However, they strongly constrain the properties of the data stream in a way that it sometimes considered unrealistic or very limiting (Cossu et al., 2021).

Original: Recently, the idea of Class-Incremental with Repetition (CIR) scenarios has started to gather some attention in CL (Cossu et al., 2021).
Revised: Recently, the idea of Class-Incremental with Repetition (CIR) scenarios has started to gather some attention in CL (Cossu et al., 2021).

Original: CIR scenarios are arguably more flexible in the definition of the stream, since they allow both the introduction of new classes and the repetition of previously seen classes.
Revised: CIR scenarios are arguably more flexible in the definition of the stream, since they allow both the introduction of new classes and the repetition of previously seen classes.

Original: Crucially, repetition is a property of the environment and cannot be controlled by the CL agent.
Revised: Crucially, repetition is a property of the environment and cannot be controlled by the CL agent.

Original: This is very different from
Revised: This is very different from

Original: Replay strategies (Hayes et al., 2021), where the repetition of previous concepts is heavily structured and can be tuned at will.
Revised: Replay strategies (Hayes et al., 2021), where the repetition of previous concepts is heavily structured and can be tuned at will.

Original: CIR defines a family of CL scenarios which ranges from CI (new classes only, without repetition) to
Revised: CIR defines a family of CL scenarios which ranges from CI (new classes only, without repetition) to

Original: DI (full repetition of all seen classes).
Revised: DI (full repetition of all seen classes).

Original: Although appealing, currently there exists neither a quantitative analysis nor an empirical evaluation of CL strategies learning in CIR scenarios.
Revised: Although appealing, currently there exists neither a quantitative analysis nor an empirical evaluation of CL strategies learning in CIR scenarios.

Original: Mainly, because it is not obvious how to build a stream with repetition, given the large amount of variables involved.
Revised: Mainly, because it is not obvious how to build a stream with repetition, given the large amount of variables involved.

Original: How to manage repetition over time? How to decide what to repeat? What data should we use?
Revised: How to manage repetition over time? How to decide what to repeat? What data should we use?

Original: In this paper, we provide two generators for CIR that, starting from a single dataset, allow to build customized streams by only setting few parameters.
Revised: In this paper, we provide two generators for CIR that, starting from a single dataset, allow to build customized streams by only setting few parameters.

Original: The generators are as easy to use as CI or DI ones.
Revised: The generators are as easy to use as CI or DI ones.

Original: We leveraged our generators to run an extensive empirical evaluation of the behavior of CL strategies in CIR scenarios.
Revised: We leveraged our generators to run an extensive empirical evaluation of the behavior of CL strategies in CIR scenarios.

Original: We found out that knowledge accumulation happens naturally in streams with repetition.
Revised: We found out that knowledge accumulation happens naturally in streams with repetition.

Original: Even a naive fine-tuning, subjected to complete forgetting in CI scenarios, is able to accumulate knowledge for classes that are not always present in an experience.
Revised: Even a naive fine-tuning, subjected to complete forgetting in CI scenarios, is able to accumulate knowledge for classes that are not always present in an experience.

Original: We observed that
Revised: We observed that

Original: Replay strategies still provide an advantage in terms of final accuracy, even though they are not crucial to avoid catastrophic forgetting.
Revised: Replay strategies still provide an advantage in terms of final accuracy, even though they are not crucial to avoid catastrophic forgetting.

Original: On one side, distillation-based strategies like LwF (Li &
Revised: On one side, distillation-based strategies like LwF (Li &

Original: Hoiem, 2018) are competitive in streams with a moderate amount of repetition.
Revised: Hoiem, 2018) are competitive in streams with a moderate amount of repetition.

Original: On the other side, existing Replay strategies are not specifically designed for CIR streams.
Revised: On the other side, existing Replay strategies are not specifically designed for CIR streams.

Original: We propose a novel Replay approach, called Frequency-Aware Replay (ER-FA) designed for streams with unbalanced repetition (few classes appear rarely, the other very frequently).
Revised: We propose a novel Replay approach, called Frequency-Aware Replay (ER-FA) designed for streams with unbalanced repetition (few classes appear rarely, the other very frequently).

Original: ER-FA surpasses by a large margin other
Revised: ER-FA surpasses by a large margin other

Original: Replay variants when looking at infrequent classes and it does not lose performance in terms of frequent classes.
Revised: Replay variants when looking at infrequent classes and it does not lose performance in terms of frequent classes.

Original: This leads to a moderate gain in the final accuracy, with a much better robustness and a reduced variance across all classes.
Revised: This leads to a moderate gain in the final accuracy, with a much better robustness and a reduced variance across all classes.

Original: Our main contributions are:
Revised: Our main contributions are:

Original: The design of two CIR generators, able to create streams with repetition by only setting few control parameters.
Revised: The design of two CIR generators, able to create streams with repetition by only setting few control parameters.

Original: We built both generators with Avalanche (Lomonaco et al., 2021) and we will make them publicly available to foster future research.
Revised: We built both generators with Avalanche (Lomonaco et al., 2021) and we will make them publicly available to foster future research.

Original: The generators are general enough to fit any classification dataset and are fully integrated with Avalanche pipeline to run CL experiments.
Revised: The generators are general enough to fit any classification dataset and are fully integrated with Avalanche pipeline to run CL experiments.

Original: We perform an extensive evaluation of the properties of CIR streams and the performance of CL strategies.
Revised: We perform an extensive evaluation of the properties of CIR streams and the performance of CL strategies.

Original: We study knowledge accumulation and we showed that Replay, although still effective, is not crucial for the mitigation of catastrophic forgetting.
Revised: We study knowledge accumulation and we showed that Replay, although still effective, is not crucial for the mitigation of catastrophic forgetting.

Original: Some approaches (e.g., LwF) look more promising than others in CIR scenarios.
Revised: Some approaches (e.g., LwF) look more promising than others in CIR scenarios.

Original: We consolidate our results with an analysis of the CL models over time through Centered Kernel Alignment (CKA) (Kornblith et al., 2019) and weights analysis.
Revised: We consolidate our results with an analysis of the CL models over time through Centered Kernel Alignment (CKA) (Kornblith et al., 2019) and weights analysis.

Original: We propose a novel Replay variant, ER-FA, which is designed based on the properties of CIR scenarios.
Revised: We propose a novel Replay variant, ER-FA, which is designed based on the properties of CIR scenarios.

Original: ER-FA surpasses other Replay strategies in unbalanced streams and provide a more robust performance on infrequent classes without losing accuracy on the frequent ones.
Revised: ER-FA surpasses other Replay strategies in unbalanced streams and provide a more robust performance on infrequent classes without losing accuracy on the frequent ones.

Original:  C LASS -I NCREMENTAL L EARNING WITH R EPETITION G ENERATORS
Revised:  C LASS -I NCREMENTAL L EARNING WITH R EPETITION G ENERATORS

Original: [Equation]
Revised: [Equation]

Revised: [Figure]

Original:  𝑒 " 𝑒 # 𝑒$ … 𝑒 ! 𝑒 " 𝑒 # 
Revised: DI Concepts: 𝑒 " 𝑒 # 𝑒 ! 𝑒 " 𝑒 # 𝑒 $ …

Original: CIR CI DI 𝑒 !
Revised: [Figure]

Original: 𝑒 " 𝑒 # 𝑒 $ …
Revised: 𝑒 " 𝑒 # 𝑒 $ …

Original: [Figure]
Revised: [Figure]

Revised: The shape colors indicate whether instances are new in each episode or can be a mixture of old and new instances.

Original: CL requires a model to learn from a stream of N experiences S
Revised: CL requires a model to learn from a stream of N experiences S

Original: = { e 1 , e 2 , ..., e N } , where each experience e i brings a dataset of examples.
Revised: = { e 1 , e 2 , ..., e N } , where each experience e i brings a dataset of examples D e i =

Original:  Many CL scenarios, like CI or DI, are generated from a fixed dataset D= { ( x, y ); x ∈
Revised: { X i , Y i } . Many CL scenarios, like CI or DI, are generated from a fixed dataset D

Original:  X, y ∈ Y } , where x is the input example, y is the target and Y = [1 , · · · , C ] is the label space (closed-world assumption).
Revised: = { ( x, y ); x ∈ X, y ∈ Y } , where x is the input example, y is the target and Y = { 1 , · · · , C } is the label space (closed-world assumption).

Original: Depending on how classes from the entire dataset D are shown or revisited in the stream, this configuration can lead to CI, CIR or DI scenarios (Figure 1).
Revised: Depending on how classes from the entire dataset D are shown or revisited in the stream, this configuration can lead to CI, CIR or DI scenarios (Figure 1).

Revised: In Table 1, we formally present and compare the properties of the three scenario types.

Original: In CIR, streams with repetition are characterized by multiple occurrences of the same class over time.
Revised: In CIR, streams with repetition are characterized by multiple occurrences of the same class over time.

Original: To study this scenario, we propose two stream generators designed to create a stream from a finite dataset: the Slot-Based
Revised: To study this scenario, we propose two stream generators designed to create a stream from a finite dataset: the Slot-Based

Original: Generator ( G slot ) and the Sampling-Based Generator ( G samp ).
Revised: Generator ( G slot ) and the Sampling-Based Generator ( G samp ).

Original: G slot generate streams by enforcing constraints on the number of occurrences of classes in the stream using only two parameters.
Revised: G slot generate streams by enforcing constraints on the number of occurrences of classes in the stream using only two parameters.

Original: G slot does not repeat already observed samples, therefore the stream length is limited by the number of classes.
Revised: G slot does not repeat already observed samples, therefore the stream length is limited by the number of classes.

Original: However, it guarantees that all samples in the dataset will be observed exactly once during the lifetime of the model.
Revised: However, it guarantees that all samples in the dataset will be observed exactly once during the lifetime of the model.

Original: Instead, G samp generates streams according to several parametric distributions that control the stream properties.
Revised: Instead, G samp generates streams according to several parametric distributions that control the stream properties.

Original: It can generate arbitrarily long streams in which old instances can also re-appear with some probability.
Revised: It can generate arbitrarily long streams in which old instances can also re-appear with some probability.

Original: 2. S LOT -B ASED G ENERATOR
Revised: 2. S LOT -B ASED G ENERATOR

Original: The Slot-Based Generator G slot allows to carefully control the class repetitions in the generated stream with a single parameter K .
Revised: The Slot-Based Generator G slot allows to carefully control the class repetitions in the generated stream with a single parameter K .

Original: G slot takes as input a dataset D , the total number of experiences
Revised: G slot takes as input a dataset D , the total number of experiences

Revised: [Table]

Original: N and the number of slots per experience K .
Revised: N and the number of slots per experience K .

Original: It returns a CIR stream composed by N experiences, where each of the K slots in each experience is filled with samples coming from a single class. 
Revised: It returns a CIR stream composed by N experiences, where each of the K slots in each experience is filled with samples coming from a single class. [Figure]

Original: G slot constrains the slot-class association such that all the samples present in the dataset are seen once and only once in the stream.
Revised: G slot constrains the slot-class association such that all the samples present in the dataset are seen exactly once in the stream.

Original: Therefore, G slot considers repetition at the level of the whole class, while the same sample is never revisited.
Revised: Therefore, G slot considers repetition at the level of concepts.

Original: To implement this logic, G slot first partitions all the samples in the dataset into the target number of slots.
Revised: To implement this logic, G slot first partitions all the samples in the dataset into the target number of slots.

Original: Then, it randomly assigns without replacement K slots per experience.
Revised: Then, it randomly assigns without replacement K slots per experience.

Original: At the end, the N mod K blocks remaining are assigned to the first experience, such that the rest of the stream is not affected by a variable number of slots.
Revised: At the end, the N mod K blocks remaining are assigned to the first experience, such that the rest of the stream is not affected by a variable number of slots.

Original: [Figure]

Original: Algorithm 1 in Appendix B provides a formal description of

Original: The Slot-Based Generator is useful to study the transition from CI scenarios to DI scenarios, obtained by simply changing the parameter K (Figure 2).
Revised: The Slot-Based Generator is useful to study the transition from CI scenarios to DI scenarios, obtained by simply changing the parameter K (Figure 2).

Original: For example, let us consider a dataset with 10 classes such as MNIST.
Revised: For example, let us consider a dataset with 10 classes such as MNIST.

Original: By choosing N = 5 and K = 2 we obtain the popular Split-MNIST, a CI scenario with no repetition and 2 classes for each experience.
Revised: By choosing N = 5 and K = 2 we obtain the popular SplitMNIST, a CI scenario with no repetition and 2 classes for each experience.

Original: Conversely, by setting N = 5 and K = 10 we obtain a DI stream where all the 10 classes appear in each experience with new unseen samples.
Revised: Conversely, by setting N = 5 and K = 10 we obtain a DI stream where all the 10 classes appear in each experience with new unseen samples.

Original: More in general, given a dataset with C classes, we obtain a CI scenario by setting K = CN ( N must divide C ).
Revised: More in general, given a dataset with C classes, we obtain a CI scenario by setting K = CN ( N must divide C ).

Original: We obtain a DI scenario by setting K = C .
Revised: We obtain a DI scenario by setting K = C .

Revised: In Appendix B we illustrate the overall steps of stream generation (Figure 12), and provide a step-by-step formal definition of G slot (Algorithm 2).

Original: 2. S AMPLING -B ASED G ENERATOR
Revised: 2. S AMPLING -B ASED G ENERATOR

Original: The Sampling-Based Generator ( G samp ) generates arbitrarily long streams and controls the repetitions via probability distributions.
Revised: The Sampling-Based Generator ( G samp ) generates arbitrarily long streams and controls the repetitions via probability distributions.

Original: The stream generator allows to control the first occurrence of new classes and the amount of repetitions of old classes.
Revised: The stream generator allows to control the first occurrence of new classes and the amount of repetitions of old classes.

Original: Unlike G slot , it allows to generate infinite and even unbalanced streams.
Revised: Unlike G slot , it allows to generate infinite and even unbalanced streams.

Original: G samp parameters
Revised: G samp parameters:

Original: • N : Stream length, i.e. number of experiences in the stream.
Revised: • N : Stream length, i.e. number of experiences in the stream.

Original: • S : Experience size which defines the number of samples in each experience.
Revised: • S : Experience size which defines the number of samples in each experience.

Original: • P f ( S ) :
Revised: • P f ( S ) :

Original: Probability distribution over the stream S used for sampling the experience ID of the first occurrence in each class.
Revised: Probability distribution over the stream S used for sampling the experience ID of the first occurrence in each class.

Original: • P r : List of repetition probabilities for dataset classes.
Revised: • P r : List of repetition probabilities for dataset classes.

Original: Note that P f is a probability mass function over the stream S which means it sums up to 1 . 0 and determines in which parts of the stream it is more likely to observe new classes for the first time.
Revised: Note that P f is a probability mass function over the stream S which means it sums up to 1 . 0 and determines in which parts of the stream it is more likely to observe new classes for the first time.

Original: However, the list of probabilities { p 1 , p 2 , ..., p C } in P r are independent and each probability value 0 . 

Revised: However, the list of probabilities { p 1 , p 2 , ..., p C } in P r are independent and each probability value 0 .

Original: ≤ p i ≤ 1 . 0 indicates how likely it is for each class to repeat after its first occurrence.
Revised: ≤ p i ≤ 1 . 0 indicates how likely it is for each class to repeat after its first occurrence.

Original: For each experience, G samp samples instances from the original dataset D according to a two step process.
Revised: For each experience, G samp samples instances from the original dataset D according to a two step process.

Original: First, G samp defines a C × N binary matrix T called Occurrence Matrix that determines which classes can appear in each experience.
Revised: First, G samp defines a C × N binary matrix T called Occurrence Matrix that determines which classes can appear in each experience.

Original: Then, for each experience e i , 1 ≤ i ≤ N we use the i -th column of T to sample data points for that experience.
Revised: Then, for each experience e i , 1 ≤ i ≤ N we use the i -th column of T to sample data points for that experience.

Original: The generator uses the inputs N , P f ( S ) and P r to generate T .
Revised: The generator uses the inputs N , P f ( S ) [Figure] Sampler … 𝑒 # 𝑒 !

Revised: 𝑒 " 𝑒 # 𝑒 $ 𝑒 % 𝑐 ! 𝑐 " 𝑐 # 𝑐 $ 𝑐 % 𝑐 & 𝑐 ’ 𝑐 ( 𝑐 ) …………

Revised: ……………… 𝒟 𝒮 $"%&’ 𝒮 $()$ 𝑒 * … 𝑒 # 𝑒 *

Original:  Therefore, it first initializes T as a zero C × N matrix.
Revised: [Figure] and P r to generate T . Therefore, it first initializes T as a zero C × N matrix.

Original: Then for each class c in the dataset, it uses P f ( S ) to sample the index of the experience in which class c appears for the first time.
Revised: Then for each class c in the dataset, it uses P f ( S ) to sample the index of the experience in which class c appears for the first time.

Original: Different probability distributions can be used to either spread the first occurrence along the stream or concentrate them at the beginning, which allows a good flexibility in the stream design.
Revised: Different probability distributions can be used to either spread the first occurrence along the stream or concentrate them at the beginning, which allows a good flexibility in the stream design.

Original: After sampling the first occurrences, the classes are then repeated based on P r probability values to finalize matrix T .
Revised: After sampling the first occurrences, the classes are then repeated based on P r probability values to finalize matrix T .

Original: In the simplest case, P r can be fixed to the same value for all classes to generate a balanced stream.
Revised: In the simplest case, P r can be fixed to the same value for all classes to generate a balanced stream.

Original: Once the matrix T is constructed, a random sampler is used to sample patterns for each experience.
Revised: Once the matrix T is constructed, a random sampler is used to sample patterns for each experience.

Original: Since each experience may contain an arbitrary number of classes, another control parameter that could be added here is the fraction of samples per class in experience size S .
Revised: Since each experience may contain an arbitrary number of classes, another control parameter that could be added here is the fraction of samples per class in experience size S .

Original: For simplicity we keep the fractions equally fixed and thus the number of datapoints sampled from each class in experience [Figure] 𝑁𝐾 CL Scenario ℙ !
Revised: For simplicity we keep the fractions equally fixed and thus the number of datapoints sampled from each class in experience e i is ⌊ S |C i | ⌋ where |C i | indicates the number of classes present in e i .

Original: (𝒮) 𝑃 "

Original: Scenario Matrix 𝐺 !"

Original: #$

Original: Parameters

Original: Sampler … 𝑒 # 𝑒 !

Original: 𝑒 " 𝑒 # 𝑒 $ 𝑒 % 𝑐 ! 𝑐 " 𝑐 # 𝑐 $ 𝑐 % 𝑐 & 𝑐 ’ 𝑐 ( 𝑐 ) …………

Original: ……………… 𝒟 𝒮 $"%&’ 𝒮 $()$ 𝑒 * … 𝑒 # 𝑒 * e i is ⌊ S

Original: |C i | ⌋ where |C i | indicates the number of classes present in e i .

Original: Since the sampler is stochastic, each time we sample from a class, both new and old patterns can appear for that class.
Revised: Since the sampler is stochastic, each time we sample from a class, both new and old patterns can appear for that class.

Original: Given a large enough stream length N , the final stream will cover the whole dataset with a measurable amount of average repetition.
Revised: Given a large enough stream length N , the final stream will cover the whole dataset with a measurable amount of average repetition.

Original: In Figure 3 we demonstrate the schematic of the generator G samp .
Revised: In Figure 3 we demonstrate the schematic of the generator G samp .

Revised: We provide the pseudo-code for G samp in Appendix C (Algorithm 2).

Original: Although we assume a fixed number of instances per class in D , G samp can be easily extended to settings where the number of instances can grow over time.
Revised: Although we assume a fixed number of instances per class in D , G samp can be easily extended to settings where the number of instances can grow over time.

Original: Moreover, the sampler can also be designed arbitrarily depending on the stochasticity type, e.g., irregular or cyclic.
Revised: Moreover, the sampler can also be designed arbitrarily depending on the stochasticity type, e.g., irregular or cyclic.

Original:  F REQUENCY -A WARE R EPLAY
Revised:  F REQUENCY -A WARE R EPLAY

Original:  Experience Replay (ER) is the most popular CL strategy due to its simplicity of use and high performance in class-incremental scenarios.
Revised: [Figure] Experience Replay (ER) is the most popular CL strategy due to its simplicity of use and high performance in classincremental scenarios.

Original: The storage policy, which determines which samples to keep in a limited buffer, is the major component of ER methods.
Revised: The storage policy, which determines which samples to keep in a limited buffer, is the major component of ER methods.

Original: Class-Balanced (CB) and Reservoir Sampling (RS) Vitter (1985) are the most popular policies.
Revised: Class-Balanced (CB) and Reservoir Sampling (RS) Vitter (1985) are the most popular storage policies in ER methods.

Original: CB keeps a fixed quota for each class, while RS samples randomly from the stream, which leads to the class frequency in the buffer being equal to the frequency in the stream.
Revised: CB keeps a fixed quota for each class, while RS samples randomly from the stream, which leads to the class frequency in the buffer being equal to the frequency in the stream.

Original: CB and RS are great choices for balanced streams such as class incremental scenarios, where the number of samples per class is the same over the whole stream.
Revised: CB and RS are great choices for balanced streams such as class incremental scenarios, where the number of samples per class is the same over the whole stream.

Original: However, as in most real-world scenarios, CIR scenarios are naturally unbalanced, and different classes may have completely different repetition frequencies.
Revised: However, as in most real-world scenarios, CIR scenarios are naturally unbalanced, and different classes may have completely different repetition frequencies.

Original: Accordingly, CB and RS storage policies may suffer a big accuracy drop in the infrequent classes of an unbalanced stream.
Revised: Accordingly, CB and RS storage policies may suffer a big accuracy drop in the infrequent classes of an unbalanced stream.

Original: For example, in highly unbalanced streams, RS will store an unbalanced buffer replicating the the original distribution of the stream, which is sub-optimal because the less frequent classes will require more repetition to prevent forgetting, while the frequent classes will be repeated naturally via the stream occurrences.
Revised: For example, in highly unbalanced streams, RS will store an unbalanced buffer replicating the the original distribution of the stream, which is sub-optimal because the less frequent classes will require more repetition to prevent forgetting, while the frequent classes will be repeated naturally via the stream occurrences.

Original: [Figure]

Original: We propose Frequency-Aware (FA) storage policy that addresses the imbalance issue in CIR streams by online adjustment of the buffer slots in accordance with the amount of repetition for each class.
Revised: We propose Frequency-Aware (FA) storage policy that addresses the imbalance issue in CIR streams by online adjustment of the buffer slots in accordance with the amount of repetition for each class.

Original: Given a buffer B with a maximum size of M , a list of previously observed classes P initialized asP = {} with a corresponding list
Revised: Given a buffer B with a maximum size of M , a list of previously observed classes P initialized as

Original:  O indicating the number of observations per class c in C , and a dataset D i from experience e i , the algorithm first checks the present classes
Revised: P = {} with a corresponding list O indicating the number of observations per class c in C , and a dataset D i from experience e i , the algorithm first checks the present classes

Original: P i in D i and adds them to P ( P ← P ∪ P i ).
Revised: P i in D i and adds them to P ( P ← P ∪ P i ).

Original: Then, for each class c in P i it increments the number of observations
Revised: Then, for each class c in P i it increments the number of observations

Original: O [ c ] by  1 if the class was previously seen, otherwise it initializes O [ c ] = 1 .
Revised: O [ c ] by [Figure] 1 if the class was previously seen, otherwise it initializes O [ c ] = 1 .

Original: After updating the number of observations, FA computes the buffer quota Q for all observed classes by inverting the number of observations ( Q = [ 1 O [ c ] ∀ c ∈ C ] ) and normalizes it .
Revised: After updating the number of observations, FA computes the buffer quota Q for all observed classes by inverting the number of observations ( Q = [ 1 O [ c ] ∀ c ∈ C ] ) and normalizes it .

Original: This way, the algorithm offers the less frequent classes a larger quota.
Revised: This way, the algorithm offers the less frequent classes a larger quota.

Original: Finally, a procedure ensures the buffer is used to its maximum capacity by filling unused slots with samples from more frequent classes sorted by their observation times.
Revised: Finally, a procedure ensures the buffer is used to its maximum capacity by filling unused slots with samples from more frequent classes sorted by their observation times.

Original: This is a crucial step since it is possible that an infrequent class which is not present e i will be assigned with a larger quota than its current number of samples in B , and therefore the buffer will remain partially empty.
Revised: This is a crucial step since it is possible that an infrequent class which is not present e i will be assigned with a larger quota than its current number of samples in B , and therefore the buffer will remain partially empty.

Original: In Figure 4 we show how our method assigns higher ratio of samples for infrequent classes to overcome the imbalance issue in the stream.
Revised: In Figure 4 we show how our method assigns higher ratio of samples for infrequent classes to overcome the imbalance issue in the stream.

Original:  We present examples of unbalanced scenarios in Appendix F and the [Figure]
Revised: For further analysis and pseudo-code of FA policy refer to Appendix E. We present examples of unbalanced scenarios in Appendix D.

Original:  E MPIRICAL E VALUATION
Revised:  E MPIRICAL E VALUATION

Original: We study CIR scenarios by leveraging our generators G samp and G slot .
Revised: We study CIR scenarios by leveraging our generators G samp and G slot .

Original: First, by using G slot we provide quantitative results about forgetting in CL strategies when transitioning from CI to DI scenarios (Sec. 4.1).
Revised: First, by using G slot we provide quantitative results about forgetting in CL strategies when transitioning from CI to DI scenarios (Sec. 4.1).

Original: Then, by using G samp we focus on long streams with 500 experiences and study the performance of Replay and Naive (Sec. 4.2).
Revised: Then, by using G samp we focus on long streams with 500 experiences and study the performance of Replay and Naive (Sec. 4.2).

Original: The long streams give us the opportunity to study knowledge accumulation over time in the presence of repetition.
Revised: The long streams give us the opportunity to study knowledge accumulation over time in the presence of repetition.

Original: We also provide an intuitive interpretation of the model dynamics over long streams (Sec. 4.3).
Revised: We also provide an intuitive interpretation of the model dynamics over long streams (Sec. 4.3).

Original: Finally, we show that
Revised: Finally, we show that

Original: our
Revised: our

Original: Frequency-Aware Replay is able to exploit the repetitions present in the stream and to surpass the performance of other replay approaches not specifically designed for CIR scenarios (Sec. 4.4).
Revised: Frequency-Aware Replay is able to exploit the repetitions present in the stream and to surpass the performance of other replay approaches not specifically designed for CIR scenarios (Sec. 4.4).

Original: The experiments were conducted using the CIFAR-100 Krizhevsky et al.
Revised: The experiments were conducted using the CIFAR-100 Krizhevsky et al.

Original: and Tiny-ImageNet
Revised: and Tiny-ImageNet

Original: LeCun et al.
Revised: LeCun et al.

Original: (1998) datasets with the ResNet-18 model.
Revised: (1998) datasets with the ResNet-18 model.

Original: For G slot , we run experiments for Naive (incremental fine tuning), LwF Li & Hoiem (2018), EWC Kirkpatrick et al.
Revised: For G slot , we run experiments for Naive (incremental fine tuning), LwF Li & Hoiem (2018), EWC Kirkpatrick et al.

Original: (2017), Experience Replay with reservoir sampling Kim et al.
Revised: (2017), Experience Replay with reservoir sampling Kim et al.

Original: (2020) (ER-RS) and AGEM Chaudhry et al.
Revised: (2020) (ER-RS) and AGEM Chaudhry et al.

Original: (2018) strategies.
Revised: (2018) strategies.

Original: For G samp we run experiments for Naive and ER (CB/RS/FA) strategies.
Revised: For G samp we run experiments for Naive and ER (CB/RS/FA) strategies.

Original: We set the default buffer size for CIFAR-100 to 2000 , and for Tiny-ImageNet to 4000 in the replay strategies.
Revised: We set the default buffer size for CIFAR-100 to 2000 , and for Tiny-ImageNet to 4000 in the replay strategies.

Original: We evaluate all strategies on the Average Test Accuracy (TA).
Revised: We evaluate all strategies on the Average Test Accuracy (TA).

Original: 4. T RANSITION FROM C LASS -I NCREMENTAL TO D OMAIN I NCREMENTAL
Revised: 4. T RANSITION FROM C LASS -I NCREMENTAL TO D OMAIN I NCREMENTAL

Original: DI and CI scenarios are heavily studied in the CL literature.
Revised: DI and CI scenarios are heavily studied in the CL literature.

Original: However, little is known about what happens to the performance of popular CL strategies when gradually transitioning from one scenario to the other.
Revised: However, little is known about what happens to the performance of popular CL strategies when gradually transitioning from one scenario to the other.

Original: By changing the value of K in G slot , we provide a quantitative analysis of such behavior in CIR scenarios.
Revised: By changing the value of K in G slot , we provide a quantitative analysis of such behavior in CIR scenarios.

Original: Figure 5 shows the Average Test Accuracy over all classes for different
Revised: Figure 5 shows the Average Test Accuracy over all classes for different

Original: CL strategies when transitioning from CI (left-most point of each plot) to DI (right-most point of each plot).
Revised: CL strategies when transitioning from CI (left-most point of each plot) to DI (right-most point of each plot).

Original: Replay is one of the most effective strategies in CI scenarios.
Revised: Replay is one of the most effective strategies in CI scenarios.

Original: As expected, in CIR scenarios the advantage provided by ER-RS with respect to other CL strategies diminishes as the amount of repetition increases.
Revised: As expected, in CIR scenarios the advantage provided by ER-RS with respect to other CL strategies diminishes as the amount of repetition increases.

Original: However, in order for the other strategies to match the performance of ER-RS, the environment needs to provide a large amount of repetition.
Revised: However, in order for the other strategies to match the performance of ER-RS, the environment needs to provide a large amount of repetition.

Original: LwF guarantees a consistent boost in the performance, both in CIFAR-100 and Tiny-ImageNet.
Revised: LwF guarantees a consistent boost in the performance, both in CIFAR-100 and Tiny-ImageNet.

Original: In particular, and quite surprisingly, on Tiny-ImageNet LwF is able to quickly close the gap with
Revised: In particular, and quite surprisingly, on Tiny-ImageNet LwF is able to quickly close the gap with

Original: ER-RS and even surpass it as the amount of repetition increases.
Revised: ER-RS and even surpass it as the amount of repetition increases.

Original: The positive interplay between  distillation and repetition provides an effective way to mitigate forgetting in CIR scenarios, without the need to explicitly store previous samples in an external memory.
Revised: The positive interplay between [Figure] distillation and repetition provides an effective way to mitigate forgetting in CIR scenarios, without the need to explicitly store previous samples in an external memory.

Original: EWC showed different sensitivity to the regularization hyper-parameter λ .
Revised: EWC showed different sensitivity to the regularization hyper-parameter λ .

Original: We experimented with λ = 0 .
Revised: We experimented with λ = 0 .

Original: While on MNIST we did not see any difference in performance, on CIFAR-100 and Tiny-ImageNet large values of λ lead to a dramatic decrease, dropping as low as Naive.
Revised: While on MNIST we did not see any difference in performance, on CIFAR-100 and Tiny-ImageNet large values of λ lead to a dramatic decrease, dropping as low as Naive.

Original: We found 0 .
Revised: We found 0 .

Original: 1 to be the best value on both CIFAR-100 and Tiny-ImageNet.
Revised: 1 to be the best value on both CIFAR-100 and Tiny-ImageNet.

Original: This configuration only provides a low amount of regularization.
Revised: This configuration only provides a low amount of regularization.

Original: Overall, the role played by the natural repetition already guarantees a sufficient stability of the model, which is additionally boosted only in the case of LwF.[Figure]
Revised: Overall, the role played by the natural repetition already guarantees a sufficient stability of the model, which is additionally boosted only in the case of LwF.

Original: 4. I MPACT OF R EPETITION IN L ONG S TREAMS
Revised: 4. I MPACT OF R EPETITION IN L ONG S TREAMS

Original: We investigate the impact of repetition in long streams ( N = 500 ) generated with G samp .
Revised: We investigate the impact of repetition in long streams ( N = 500 ) generated with G samp .

Original: For the long-stream experiments we also report the missing-classes accuracy (MCA) and seen-classes accuracy (SCA).
Revised: For the long-stream experiments we also report the missing-classes accuracy (MCA) and seen-classes accuracy (SCA).

Original: MCA measure the accuracy over the classes that were seen before but are missing in the current experience, and SCA measure the accuracy over all seen classes up to the current experience.
Revised: MCA measure the accuracy over the classes that were seen before but are missing in the current experience, and SCA measure the accuracy over all seen classes up to the current experience.

Original: Missing Class Accuracy Increases Over Time In CI scenarios, a Naive strategy catastrophically forgets each class as soon as it starts learning on new classes.
Revised: Missing Class Accuracy Increases Over Time In CI scenarios, a Naive strategy catastrophically forgets each class as soon as it starts learning on new classes.

Original: Surprisingly, we found that in CIR scenarios there is knowledge accumulation over time for all the classes.
Revised: Surprisingly, we found that in CIR scenarios there is knowledge accumulation over time for all the classes.

Original: Figure 6 shows the accuracy of a single class over time, highlighting whether the class is present or not in the current experience.
Revised: Figure 6 shows the accuracy of a single class over time, highlighting whether the class is present or not in the current experience.

Original: At the beginning of the stream missing classes are completely forgotten, which can be noticed by the instant drop of the accuracy to zero.
Revised: At the beginning of the stream missing classes are completely forgotten, which can be noticed by the instant drop of the accuracy to zero.

Original: However, over time the model accumulate knowledge and the training process stabilizes.
Revised: However, over time the model accumulate knowledge and the training process stabilizes.

Original: As a result, the accuracy of missing classes tends to increase over time, suggesting that the network becomes more resistant to forgetting.
Revised: As a result, the accuracy of missing classes tends to increase over time, suggesting that the network becomes more resistant to forgetting.

Original: Notice that this is an example of continual learning property that is completely ignored when testing on CI scenarios.
Revised: Notice that this is an example of continual learning property that is completely ignored when testing on CI scenarios.

Original: This finding prompts the question, ”What is happening to allow knowledge accumulation even for
Revised: This finding prompts the question, ”What is happening to allow knowledge accumulation even for

Original: Naive finetuning?” .
Revised: Naive finetuning?” .

Original: We investigate this question by analysing the model’s accuracy over time and the properties of the learned model in the next experiments.
Revised: We investigate this question by analysing the model’s accuracy over time and the properties of the learned model in the next experiments.

Original: Accuracy Gap Between Naive and Replay Decreases Over Time To study the impact of long streams with repetitions we monitor the accuracy gap between ER and Naive fine-tuning by comparing their accuracy after each experience.
Revised: Accuracy Gap Between Naive and Replay Decreases Over Time To study the impact of long streams with repetitions we monitor the accuracy gap between ER and Naive fine-tuning by comparing their accuracy after each experience.

Original: For the scenario configuration, we set P f ( S ) as a
Revised: For the scenario configuration, we set P f ( S ) as a

Original: Geometric distribution with a coefficient of 0 .
Revised: Geometric distribution with a coefficient of 0 .

Original: 01 and fix the probability of repetition P r as 0 .
Revised: 01 and fix the probability of repetition P r as 0 .

Original: 2 for all classes.
Revised: 2 for all classes.

Original: For more details and illustrations on distribution types refer to Appendix C. In such scenarios, the majority of classes occur for the first time in the first quarter of the stream, and then repeat with a constant probability of 0 . 2 which makes them appropriate for our experiments since all classes are observed before the middle of the stream and the repetition probability is low enough.
Revised: For more details and illustrations on distribution types refer to Appendix C. In such scenarios, the majority of classes occur for the first time in the first quarter of the stream, and then repeat with a constant probability of 0 . 2 which makes them appropriate for our experiments since all classes are observed before the middle of the stream and the repetition probability is low enough.

Original: [Figure]
Revised: [Figure]

Original: As can be seen in Figure 7, while the accuracy of ER saturates over time, the accuracy of Naive increases, closing the gap between the two strategies from around 25% in experience 100 to 7% in experience 500 .
Revised: As can be seen in Figure 7, while the accuracy of ER saturates over time, the accuracy of Naive increases, closing the gap between the two strategies from around 25% in experience 100 to 7% in experience 500 .

Original: This supports our hypothesis that neural network’s ability to consolidate knowledge is significantly influenced by ”natural” repetition in the environment.
Revised: This supports our hypothesis that neural network’s ability to consolidate knowledge is significantly influenced by ”natural” repetition in the environment.

Original: The Role of Repetition
Revised: The Role of Repetition

Original: The amount of repetition is one of the key aspects of a CIR scenario.
Revised: The amount of repetition is one of the key aspects of a CIR scenario.

Original: To find out how strategies perform under different repetition probabilities, we consider a setting where all components of a scenario are fixed except for P r .
Revised: To find out how strategies perform under different repetition probabilities, we consider a setting where all components of a scenario are fixed except for P r .

Original: For this experiment, we set P f ( S ) as geometric distribution with p = 0 .
Revised: For this experiment, we set P f ( S ) as geometric distribution with p = 0 .

Original: 2 and let P r change.
Revised: 2 and let P r change.

Original: In Figure 8 we demonstrate the seen class accuracy (SCA) for the Naive and ER-CB strategies in CIFAR-100.
Revised: In Figure 8 we demonstrate the seen class accuracy (SCA) for the Naive and ER-CB strategies in CIFAR-100.

Original: It is clear from the plots, that the model’s rate of convergence can be significantly affected by the amount of repetition in the scenario.
Revised: It is clear from the plots, that the model’s rate of convergence can be significantly affected by the amount of repetition in the scenario.

Original: Although, it may seem obvious that higher repetition leads to less forgetting, it is not very intuitive to what extent different strategies may gain from the environment’s repetition.
Revised: Although, it may seem obvious that higher repetition leads to less forgetting, it is not very intuitive to what extent different strategies may gain from the environment’s repetition.

Original: While the
Revised: While the

Original: Naive strategy gains a lot from increased repetition, the replay strategy saturates after some point for higher repetitions and the gaps close between different repetition values.
Revised: Naive strategy gains a lot from increased repetition, the replay strategy saturates after some point for higher repetitions and the gaps close between different repetition values.

Original: [Figure]
Revised: [Figure]

Original: 4. M ODEL S IMILARITY AND W EIGHT S PACE A NALYSIS
Revised: 4. M ODEL S IMILARITY AND W EIGHT S PACE A NALYSIS

Original: Weight Interpolation Based on the ”gradual loss drop” observation in missing classes, we study how the loss surface changes over time if we perturb the weights.
Revised: Weight Interpolation Based on the ”gradual loss drop” observation in missing classes, we study how the loss surface changes over time if we perturb the weights.

Original: We interpolate between the model weights from two consecutive checkpoints with an interval of 10 experiences in various segments of the stream.
Revised: We interpolate between the model weights from two consecutive checkpoints with an interval of 10 experiences in various segments of the stream.

Original: Assuming that w ∗ t and w ∗ t +10 are the obtained solutions for experiences t and t +respectively, we generate eight in-between models w k
Revised: Assuming that w ∗ t and w ∗ t +10 are the obtained solutions for experiences t and t +respectively, we generate eight in-between models w k

Original: =
Revised: =

Original: α ∗ w ∗ t + (1 − α ) ∗ w ∗ t +50 by by increasing α from zero to one, and then compute the accuracy of w k for the data of experience t .
Revised: α ∗ w ∗ t + (1 − α ) ∗ w ∗ t +50 by by increasing α from zero to one, and then compute the accuracy of w k for the data of experience t .

Original: We show the interpolation accuracy for various pairs of experiments in different segments of the stream for the Naive strategy in Figure 9 (left).
Revised: We show the interpolation accuracy for various pairs of experiments in different segments of the stream for the Naive strategy in Figure 9 (left).

Original: In the beginning of the stream, the accuracy of experience t in each pair drops significantly, while we observe a milder loss drop towards the end of the stream.
Revised: In the beginning of the stream, the accuracy of experience t in each pair drops significantly, while we observe a milder loss drop towards the end of the stream.

Original: The findings suggest that, towards the end of the stream, even a relatively big perturbation does not have a large negative effect on the model’s accuracy and the optimal solutions of the consecutive experiments are connected with a linear low-loss path.
Revised: The findings suggest that, towards the end of the stream, even a relatively big perturbation does not have a large negative effect on the model’s accuracy and the optimal solutions of the consecutive experiments are connected with a linear low-loss path.

Original: [Figure]
Revised: [Figure]

Original: [Figure]

Original: [Figure]
Revised: [Figure]

Original: Weight changes Another approach to analyzing the gradual drop of the accuracy is by dissecting how much, when, and where the weight changes occurs.
Revised: Weight changes Another approach to analyzing the gradual drop of the accuracy is by dissecting how much, when, and where the weight changes occurs.

Original: As shown in Figure 9 (right), we can observe that within the first experiences, there is a significant difference for blocks 0, 
Revised: As shown in Figure 9 (right), we can observe that within the first experiences, there is a significant difference for blocks 0, 1, and 2. This

Original: [Figure]
Revised: [Table]

Original: and 2.
Revised: classes having repetition probability of only 10%.

Original: This difference then stalls, showing that as we continue training experiences, the weights of these blocks stabilize.
Revised: difference then stalls, showing that as we continue training experiences, the weights of these blocks stabilize.

Original: On the other hand, blocks 3 and 4 show a linear increase in the difference with the number of experiences.
Revised: On the other hand, blocks 3 and 4 show a linear increase in the difference with the number of experiences.

Original: An explanation for this phenomenon, is that the first layers of the model capture knowledge that can be useful for several classes (more general), so it is unnecessary to change them after several experiences.
Revised: An explanation for this phenomenon, is that the first layers of the model capture knowledge that can be useful for several classes (more general), so it is unnecessary to change them after several experiences.

Original: On the other hand, the last blocks are the ones that memorize or learn more specific patterns, so they adapt to each experience.
Revised: On the other hand, the last blocks are the ones that memorize or learn more specific patterns, so they adapt to each experience.

Original: [Table]

Original: classes having repetition probability of only 10%.

Original: CKA Analysis Finally, we show the CKA Kornblith et al.
Revised: CKA Analysis Finally, we show the CKA Kornblith et al.

Original: (2019) of the model in the beginning, middle and the end of the stream with an interval difference of 50 experiences.
Revised: (2019) of the model in the beginning, middle and the end of the stream with an interval difference of 50 experiences.

Original: As shown in the visualizations in Figure 10, the longer the model is trained on more experiences, the less significant the changes in the representations become especially for the final layers.
Revised: As shown in the visualizations in Figure 10, the longer the model is trained on more experiences, the less significant the changes in the representations become especially for the final layers.

Original: We can see that the diagonal of the CKA becomes sharper propagating forward with more experiments.
Revised: We can see that the diagonal of the CKA becomes sharper propagating forward with more experiments.

Original: This indicates that although the model is trained on different subsets of classes in each experiment, the representations change less after some point in the stream.
Revised: This indicates that although the model is trained on different subsets of classes in each experiment, the representations change less after some point in the stream.

Original: [Figure]
Revised: [Figure]

Original: 4. F REQUENCY -A WARE R EPLAY IN U NBALANCED S CENARIOS
Revised: 4. F REQUENCY -A WARE R EPLAY IN U NBALANCED S CENARIOS

Original: [Figure]
Revised: [Figure]

Original: We conduct experiments for bi-modal unbalanced scenarios where classes can have a high frequency of 1 . 0 or a low frequency of 0 .
Revised: We conduct experiments for bi-modal unbalanced scenarios where classes can have a high frequency of 1 . 0 or a low frequency of 0 .

Original: We use a fraction factor that determines the amount of infrequent classes in the scenario, e.g., Fraction= 0 .
Revised: We use a fraction factor that determines the amount of infrequent classes in the scenario, e.g., Fraction= 0 .

Original: 3 means that 30% of the classes are infrequent.
Revised: 3 means that 30% of the classes are infrequent.

Original: In Table 1 we compare ER-FA with the Naive, ERRS and ER-CB strategies.
Revised: In Table 2 we compare ER-FA with the Naive, ERRS and ER-CB strategies.

Original: The numbers show the MCA and average Test Accuracy (TA) metrics for each strategy in the end of the stream averaged over three runs.
Revised: The numbers show the MCA and average Test Accuracy (TA) metrics for each strategy in the end of the stream averaged over three runs.

Original: Our strategy outperforms all other scenarios in almost all settings in both CIFAR-100 and TinyImageNet datasets in terms of TA, and significantly outperforms other methods in terms of MCA (in the last experience).
Revised: Our strategy outperforms all other scenarios in almost all settings in both CIFAR-100 and TinyImageNet datasets in terms of TA, and significantly outperforms other methods in terms of MCA (in the last experience).

Original: Moreover, we illustrate the accuracy of infrequent classes in CIFAR-100 experiments for Fraction= 0 .
Revised: Moreover, we illustrate the accuracy of infrequent classes in CIFAR-100 experiments for Fraction= 0 .

Original: 3 in Figure 11 where ER-FA achieves considerably higher accuracy in the whole stream by assigning larger quota to infrequent classes without losing its performance on frequent classes (refer to Appendix G for further illustrations).
Revised: 3 in Figure 11 where ER-FA achieves considerably higher accuracy in the whole stream by assigning larger quota to infrequent classes without losing its performance on frequent classes (refer to Appendix G for further illustrations).

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Current CL methods are mainly focused on two types of benchmarks namely, Multi Task (MT) and
Revised: Current CL methods are mainly focused on two types of benchmarks namely, Multi Task (MT) and

Original: Single Incremental Task (SIT) Maltoni & Lomonaco (2019).
Revised: Single Incremental Task (SIT) Maltoni & Lomonaco (2019).

Original: MT divides training data into distinct tasks and labels them during training and inference.
Revised: MT divides training data into distinct tasks and labels them during training and inference.

Original: SIT splits a single task into a sequence of unlabeled experiences.
Revised: SIT splits a single task into a sequence of unlabeled experiences.

Original: SIT can be further divided into Domain-Incremental (DI) where all classes are seen in each experience, and Class-Incremental (CL) where each experience contains only new (unseen) classes van de Ven & Tolias (2019).
Revised: SIT can be further divided into Domain-Incremental (DI) where all classes are seen in each experience, and Class-Incremental (CL) where each experience contains only new (unseen) classes van de Ven & Tolias (2019).

Original: Both DI and CI are extreme cases and are unlikely to hold in real-world environments Cossu et al.
Revised: Both DI and CI are extreme cases and are unlikely to hold in real-world environments Cossu et al.

Original: In a more realistic setting, the role of natural repetition in CL scenarios was studied in the context of New Instances and Classes (NIC) scenario
Revised: In a more realistic setting, the role of natural repetition in CL scenarios was studied in the context of New Instances and Classes (NIC) scenario

Original: Lomonaco et al.
Revised: Lomonaco et al.

Original: (2020) and the CRIB benchmark Stojanov et al.
Revised: (2020) and the CRIB benchmark Stojanov et al.

Original: NIC mainly focuses on small experiences composed of images of the same object, and repetitions in CRIB are adapted to a certain dataset and protocol.
Revised: NIC mainly focuses on small experiences composed of images of the same object, and repetitions in CRIB are adapted to a certain dataset and protocol.

Original: The Class-Incremental with Repetition (CIR) scenario was initially formalized in Cossu et al.
Revised: The Class-Incremental with Repetition (CIR) scenario was initially formalized in Cossu et al.

Original: (2021), however the work lacks a systematic study of CIR scenarios as the wide range of CIR scenarios makes them difficult to study.
Revised: (2021), however the work lacks a systematic study of CIR scenarios as the wide range of CIR scenarios makes them difficult to study.

Original: To counter the lack of repetition in CI, replay has been extensively used as a CL strategy (Rebuffi et al., 2017; Lopez-Paz & Ranzato, 2017; Chaudhry et al., 2018; Wu et al., 2019;
Revised: To counter the lack of repetition in CI, replay has been extensively used as a CL strategy (Rebuffi et al., 2017; Lopez-Paz & Ranzato, 2017; Chaudhry et al., 2018; Wu et al., 2019;

Original: Castro et al., 2018; Belouadah & Popescu, 2019;
Revised: Castro et al., 2018; Belouadah & Popescu, 2019;

Original: Kim et al., 2020; Douillard et al., 2020).
Revised: Kim et al., 2020; Douillard et al., 2020).

Original: In such methods, natural repetition is artificially simulated by storing past data in an external memory, and replaying them alongside the scenario stream data.
Revised: In such methods, natural repetition is artificially simulated by storing past data in an external memory, and replaying them alongside the scenario stream data.

Original: Repetition reduces catastrophic forgetting through implicit regularization of model’s weights Hayes et al.
Revised: Repetition reduces catastrophic forgetting through implicit regularization of model’s weights Hayes et al.

Original: In CI benchmarks, replay seems to be the only working strategy van de Ven et al.
Revised: In CI benchmarks, replay seems to be the only working strategy van de Ven et al.

Original: In other words, replay seems to be a necessity when no natural repetition happens.
Revised: In other words, replay seems to be a necessity when no natural repetition happens.

Original: Although replay can be seen as a method to simulate natural repetitions artificially, the two concepts are fundamentally different.
Revised: Although replay can be seen as a method to simulate natural repetitions artificially, the two concepts are fundamentally different.

Original: Repetition in replay strategies occurs with the same data seen in previous experiences, which is neither realistic nor biologically plausible Gupta et al.
Revised: Repetition in replay strategies occurs with the same data seen in previous experiences, which is neither realistic nor biologically plausible Gupta et al.

Original: On the other hand, natural repetitions of already seen objects occur in different real-world environments, and better fit the CIR scenario studied in this paper.
Revised: On the other hand, natural repetitions of already seen objects occur in different real-world environments, and better fit the CIR scenario studied in this paper.

Original: Recently, Lesort et al.
Revised: Recently, Lesort et al.

Original: (2022) scaled the number of tasks in a finite world setting (Boult et al., 2019; Mundt et al., 2022) where the model has access to a random subset of classes in each experience.
Revised: (2022) scaled the number of tasks in a finite world setting (Boult et al., 2019; Mundt et al., 2022) where the model has access to a random subset of classes in each experience.

Original: The authors proposed naive fine-tuning with masking techniques to improve retained accuracy.
Revised: The authors proposed naive fine-tuning with masking techniques to improve retained accuracy.

Original: Our work is different in the sense that we compare among different strategies and study various types of repetitions with two flexible generators.
Revised: Our work is different in the sense that we compare among different strategies and study various types of repetitions with two flexible generators.

Original:  D ISCUSSION AND C ONCLUSION
Revised:  D ISCUSSION AND C ONCLUSION

Original:  CIR scenarios  represent CL environments where repetition is naturally present in the stream.
Revised: We defined CIR scenarios which represent CL environments where repetition is naturally present in the stream.

Original: Our experiments provide a number of different insights that, for the first time, focus solely on the role of repetition in CL environments.

Original: Although the concept of repetition is quite intuitive, it is not obvious how to realize it in practice for research purposes.
Revised: Although the concept of repetition is quite intuitive, it is not obvious how to realize it in practice for research purposes.

Original: We proposed two CIR generators that  address this issue.
Revised: Therefore, we proposed two CIR generators that can be exploited to address this issue.

Original: The generators are able to create (even unlimited) streams with repetition from a single dataset.

Original: We showed that, unlike CI scenarios, knowledge accumulation happens naturally in CIR streams, even without applying any CL strategy.
Revised: Through empirical evaluations, we showed that, unlike CI scenarios, knowledge accumulation happens naturally in CIR streams, even without applying any CL strategy.

Original: Both the model performance (measured by the MCA) and the model similarity (measured by theCKA) increase over time. The systematic repetition provided by Replay, where each iteration includes examples from all seen experiences, provides an advantage but it is not critical. Even random repetition is sufficient to induce knowledge accumulation.
Revised: This raised the question of whether the systematic repetition provided by Replay is critical in all CIR scenarios. With several experiments in long streams, we demonstrated that although Replay provides an advantage in general, even random repetition in the environment can be sufficient to induce knowledge accumulation given a long enough lifetime.

Original: We found that existing Replay strategies are exclusively designed for classical CI scenarios.
Revised: Moreover, we found that existing Replay strategies are exclusively designed for classical CI scenarios.

Original: Thus, we propose a novel strategy, ER-FA, to exploit the properties of streams with repetition.
Revised: Thus, we proposed a novel strategy, ER-FA, to exploit the properties of CIR scenarios.

Original: ER-FA is able to accumulate knowledge even when the stream is highly unbalanced  in terms of class frequency, with few classes appearing very rarely.
Revised: ER-FA accumulates knowledge even in highly unbalanced stream in terms of class frequency.

Original: Our ER-FA surpasses by a large margin other Replay approaches when monitoring the accuracy for infrequent classes. It also preserves accuracy for the frequent ones.
Revised: ER-FA outperforms by a large margin other Replay approaches when monitoring the accuracy for infrequent classes while preserving accuracy for the frequent ones.

Original: Overall, ER-FA guarantees a more robust performance on a wide range of real-world scenarios, where classes are not homogeneously distributed over time.
Revised: Overall, ER-FA guarantees a more robust performance on a wide range of real-world scenarios where classes are not homogeneously distributed over time.

Original: Our study also indicates other strategies which look particularly promising for CIR scenarios.

Original: Distillation-based strategies, like LwF, seem to benefit from the repetition present in the stream.

Original: When learning in scenarios with a moderate amount of repetition, LwF may even constitute a better solution than Replay.

Original: Characterizing the type and amount of repetition becomes therefore fundamental in order to choose an adequate CL strategy.

Original: The empirical results about CIR provided in this work open new research directions which depart from the existing ones, mainly focused on the mitigation of forgetting in CI scenarios.
Revised: The framework defined in this work opens new research directions which depart from the existing ones, mainly focused on the mitigation of forgetting in CI scenarios.

Original: We hope that our  results will promote the study of CIR scenarios and the development of new CL strategies, able to exploit the inner semantics of repetition, a natural trait of real-world data streams, in many other clever and imaginative ways.
Revised: We hope that our experiments and results will promote the study of CIR scenarios and the development of new CL strategies, able to exploit the inner semantics of repetition, a natural trait of real-world data streams, in many other clever and imaginative ways.

Original:  R EPRODUCIBILITY S TATEMENT
Revised:  R EPRODUCIBILITY S TATEMENT

Original: We will provide an open-source implementation of our generators and algorithms with the scripts needed to reproduce the results reported in the paper
Revised: We will provide an open-source implementation of our generators and algorithms with the scripts needed to reproduce the results reported in the paper

Original: A R ELATED W ORK (C ONTINUED )
Revised: A R ELATED W ORK (C ONTINUED )

Original: Considering benchmark formalization frameworks, De Lange & Tuytelaars (2021) recently proposed a subdivision aimed at framing continual learning setups by categorizing them based on the batch and observable horizon that the learning agent is able to access at each time.
Revised: Considering benchmark formalization frameworks, De Lange & Tuytelaars (2021) recently proposed a subdivision aimed at framing continual learning setups by categorizing them based on the batch and observable horizon that the learning agent is able to access at each time.

Original: With this framework, the authors aim to better formalize the online learning setup.
Revised: With this framework, the authors aim to better formalize the online learning setup.

Original: While the concept of observable horizon may be useful in evaluating the significance and local (in time) usefulness of natural repetition in a training stream, this work does not consider the concept of natural repetition in its framework.
Revised: While the concept of observable horizon may be useful in evaluating the significance and local (in time) usefulness of natural repetition in a training stream, this work does not consider the concept of natural repetition in its framework.

Original: Recently, Koh et al.
Revised: Recently, Koh et al.

Original: (2022) proposed to introduce blurry task boundaries in class incremental benchmarks.
Revised: (2022) proposed to introduce blurry task boundaries in class incremental benchmarks.

Original: Their proposal is based on previous works Bang et al.
Revised: Their proposal is based on previous works Bang et al.

Original: (2021); Aljundi et al.
Revised: (2021); Aljundi et al.

Original: (2019) that tried to produce more realistic benchmarks by blurring the class-incremental scenario, which however resulted in a setup in which no classes are added to new tasks.
Revised: (2019) that tried to produce more realistic benchmarks by blurring the class-incremental scenario, which however resulted in a setup in which no classes are added to new tasks.

Original: They argue that this idea moves the focus too far away from the class-incremental setup and it is still not quite realistic.
Revised: They argue that this idea moves the focus too far away from the class-incremental setup and it is still not quite realistic.

Original: The resulting setup, named i-Blurry, aims at resolving the aforementioned issues and moving toward a more realistic scenario by partitioning the classes available in the source dataset into two groups: Disjoint and Blurry.
Revised: The resulting setup, named i-Blurry, aims at resolving the aforementioned issues and moving toward a more realistic scenario by partitioning the classes available in the source dataset into two groups: Disjoint and Blurry.

Original: Classes of the disjoint group are gradually added in successive experiences while samples of classes from the blurry group always appear in all experiences with their numerosity being controlled through a blur ratio M .
Revised: Classes of the disjoint group are gradually added in successive experiences while samples of classes from the blurry group always appear in all experiences with their numerosity being controlled through a blur ratio M .

Original: The authors show that, based on the degree of disjunction N and blurry M , this framework can produce class-incremental (no blurring), domain-incremental (no disjunction), and blurred setups.
Revised: The authors show that, based on the degree of disjunction N and blurry M , this framework can produce class-incremental (no blurring), domain-incremental (no disjunction), and blurred setups.

Original: This setup is the one that most moved towards the direction of introducing repetition in Continual learning benchmarks in a controlled way so far.
Revised: This setup is the one that most moved towards the direction of introducing repetition in Continual learning benchmarks in a controlled way so far.

Original: However, the proposed blurring mechanism is too coarse-grained to simulate a natural repetition of concepts as the significance of the repetition introduced by blurring relies too much on i) a random (uniform) sampling of the concepts to be repeated, ii) the static subdivision of classes in the Disjoint and Blurry groups.
Revised: However, the proposed blurring mechanism is too coarse-grained to simulate a natural repetition of concepts as the significance of the repetition introduced by blurring relies too much on i) a random (uniform) sampling of the concepts to be repeated, ii) the static subdivision of classes in the Disjoint and Blurry groups.

Original: B S LOT -B ASED G ENERATORB. A LGORITHM
Revised: B S LOT -B ASED G ENERATOR

Revised: Following the properties of CIR scenarios in Section 2, G slot generates a subset of CIR streams that hold the assumptions below in the defined properties:

Revised: • | X i ∩

Revised: X j | = 0 : new instances appear in each experiences • (cid:83) i

Revised: =

Revised: N i =

Original: In Algorithm 1 we present all steps of G slot used to generate arbitrary CIR scenarios given a dataset
Revised: 1 X i = X : all samples are used • | Y i ∩ Y j | ≥ 0 , ∀ 1 ≤ i, j ≤ N where i ̸ = j • (cid:83) i

Revised: =

Original: D , number of experiences N and number of slots K .
Revised: N i =1 Y i = Y : all classes are used • X is constant.

Revised: These assumptions allow transitioning through different CIR scenario types between the two extremes of CI and DI.

Revised: B. A LGORITHM

Revised: The overall steps of G slot are illustrated in Figure 12.

Revised: In Algorithm 2 we present all steps of G slot used to generate arbitrary CIR scenarios given a dataset D , number of experiences N and number of slots K .

Original: The output of the algorithm is a CL stream.
Revised: The output of the algorithm is a CL stream

Original:  B. T RANSITIONING
Revised: . B. T RANSITIONING

Original: Transitioning in G slot for a scenario with a fixed number of experiences can be done by increasing
Revised: Transitioning in G slot for a scenario with a fixed number of experiences can be done by increasing

Original: K .
Revised: K .

Original: When K = 1 the generated scenario will be class-incremental and as K get closer to the total number of classes in D , the scenarios moves towards a domain incremental setting.
Revised: When K = 1 the generated scenario will be class-incremental and as K get closer to the total number of classes in D , the scenarios moves towards a domain incremental setting.

Original: In Figure 12 we show an example of how generated scenarios change by increasing K .
Revised: In Figure 13 we show an example of how generated scenarios change by increasing K .

Original: [Figure]
Revised: [Figure]

Revised: Replicate each concept !

Revised: ×# $ times and shuffle the sequence.

Original:  C S AMPLING -B ASED G ENERATORC. D ISTRIBUTION T YPES
Revised: [Figure] [Figure] C S AMPLING -B ASED G ENERATOR

Revised: Following the properties of CIR scenarios in Section 2, G samp generates a subset of CIR scenarios that hold all defined properties.

Revised: Additionally, for any stream S = { e 1 , e 2 , ..., e N } , G samp defines a probability distribution for the first occurrence of concepts over S and per-class probabilities for each concept c ∈ Y .

Revised: G samp can generate arbitrarily long stream ( N ≥ 1 ) and even from a growing set of samples X where Y remains constant.

Revised: C. A LGORITHM

Revised: The overall steps of the G slot are shows in Algorithm 2.

Original: In this section we show some examples of different discrete distributions that can be used for P f ( S ) and P r .
Revised: In this section we show some examples of different discrete distributions that can be used for P f ( S ) and P r .

Original: For P r we use the unnormalized version of the final distribution.
Revised: For P r we use the unnormalized version of the final distribution.

Original: Distributions used for
Revised: Distributions used for

Original: G samp can be any arbitrary discrete distribution and are not limited to the ones we describe here.
Revised: G samp can be any arbitrary discrete distribution and are not limited to the ones we describe here.

Original: Require: Dataset D = { ( x i , y i ) } i =1 ,...,P with C classes, number of experiences N  ,  number of slots K present in each experience.
Revised: Given the number of elements N and scalar e ≥ 0 , the probability mass function of a Zipfian distribution over a list of N elements is defined in Equation 1. When used for the probability of first occurrence, the distribution can be defined over the experiences of a stream. For example, N can be considered as the number of experiences and i can indicate the i th experience in the stream.

Revised: By increasing e , the distribution over the stream will be skewed towards the beginning.

Revised: In figure 14 we demonstrate some examples of first occurrence probabilities over a stream of length 10 generated with Zipf distribution with increasing values of e .

Revised: Many natural distribution follow Zipf distribution

Revised: Require: Dataset D = { ( x i , y i ) } i =1 ,...,P with C classes, number of experiences N , experience size S present in each experience.

Original: Ensure: K ≤ N
Revised: Ensure: K ≤ N

Original: Ensure: C mod N =
Revised: Ensure: C mod N =

Original: Ensure: NK mod C =cls-idxs = {} ▷ Empty dictionary for y ∈ set ( { y i } i =1 ,...,P ) do cls-idxs
Revised: Ensure: NK mod C =cls-idxs = {} ▷ Empty dictionary for y ∈ set ( { y i } i =1 ,...,P ) do cls-idxs

Original: [ y ] =
Revised: [ y ] =

Original: [] ▷ Empty list init end for for i = 1 , . .
Revised: [] ▷ Empty list init end for for i = 1 , . .

Original: . , P do cls-idxs[ y i ].append
Revised: . , P do cls-idxs[ y i ].append

Original: ( i ) end for slots= {} ▷ Empty dictionary for y ∈ cls-idxs do slots[ y ] =
Revised: ( i ) end for slots= {} ▷ Empty dictionary for y ∈ cls-idxs do slots[ y ] =

Original: [] ksample = int(len(cls-idxs[ y ]) /K )
Revised: [] ksample = int(len(cls-idxs[ y ]) /K )

Original: for k = 1 , . 
Revised: for k = 1 , . . .

Original: , ( NK ) /C do subset-idxs = pop(cls-idxs[ y ], ksample) subset-samples =
Revised: , N × K C do subset-idxs = pop(cls-idxs[ y ], ksample) subset-samples =

Original: [ x idx for idx ∈ subset-idxs ] slots[ y ].append(subset-samples) end for end for stream =
Revised: [ x idx for idx ∈ subset-idxs ] slots[ y ].append(subset-samples) end for end for stream =

Original: [] for n = 1 , . .
Revised: [] for n = 1 , . .

Original: . , N do experience = dataset() seen-classes =
Revised: . , N do experience = dataset() seen-classes =

Original: [] for k=1,...,K do repeat y = sample(slots) until y / ∈ seen-classes seen-classes.append( y )
Revised: [] for k=1,...,K do repeat y = sample slots) until y / ∈ seen-classes seen-classes.append( y )

Original: experience.add(pop(slots
Revised: experience.add(pop(slots

Original: [ y ], 1)) end for stream.append(experience) end forreturn streamC.1. Z IPFIAN
Revised: [ y ], 1)) end for stream.append(experience) end forreturn stream

Original: [Figure]
Revised: [Figure]

Original: Given the number of elements N and scalar e ≥ 0 , the probability mass function of a Zipfian distribution over a list of N elements is defined in Equation 1.

Original: When used  forthe probability of first occurrence, the distribution can be defined over the experiences of a stream.
Revised: and it can be used to generate highly skewed distributions both for first occurrence and repetition probabilities. [Equation]

Original: For example, N can be considered as the number of experiences and i can indicate the i th experience in the stream. By increasing e , the distribution over the stream will be skewed towards the beginning.
Revised: Algorithm

Original: In figure 13 we demonstrate some examples of first occurrence probabilities over a stream of length 10 generated with Zipf distribution with increasing values of e .
Revised: Require: Dataset D = { ( x i , y i ) } i =1 ,...,P with C classes, number of experiences N , number of slots K , probability distribution for first occurrence P f ( S ) , and list of repetition probabilities P r .

Original: Many natural distribution follow Zipf distribution and it can be used to generate highly skewed distributions both for  first occurrence and repetition probabilities. [Equation]
Revised: T = { 0 } C × N ▷ Initialize occurrence matrix with zeros for c ∈ { 0 , 1 , . . . C } do i ∼ P f ( S ) ▷ Samples the first occurrence of class c

Original: C.1. P OISSON
Revised: T [ c, i ] =for j ∈ { i, i + 1 , . . . N } do r ∼ U (0 , 1) ▷ U : uniform distribution over [0 , 1 . 0] if r < P r [ c ] then T [ c, j ] =end if end for end for E = {} for e i ∈ { 1 , 2 , . . .

Revised: N } do

Revised: C i =

Revised: RetrieveClasses ( e i )

Revised: D e i =

Revised: Sample ( D, C i , S )

Revised: ▷ Sample S instances from dataset D for classes

Revised: C

Revised: i

Revised: E ← E ∪ D e i end for

Revised: S train , S test =

Revised: GenerateStream ( E ) ▷ Generate streams using E return S train , S test

Original: The PMF for Poisson distribution is given in Equation 2 where µ ≥ 0 .
Revised: The PMF for Poisson distribution is given in Equation 2 where µ ≥ 0 .

Original: Poisson with larger values of µ can be used for distributions where the probability of occurrence/repetition first rises and then gradually decreases over time.
Revised: Poisson with larger values of µ can be used for distributions where the probability of occurrence/repetition first rises and then gradually decreases over time.

Original: [Equation] [Figure]
Revised: [Equation] [Equation]

Original: [Figure]
Revised: [Figure]

Original:  G EOMETRIC
Revised: C.2. G EOMETRIC

Original: Another useful distribution that can be used for the first occurrence probabilities over a stream is
Revised: Another useful distribution that can be used for the first occurrence probabilities over a stream is

Original: Geometric distribution with its PMF given in Equation 3.
Revised: Geometric distribution with its PMF given in Equation 3.

Original: This distribution is in particular interesting for transitioning from domain incremental to class incremental.
Revised: This distribution is in particular interesting for transitioning from domain incremental to class incremental.

Original: By setting p = 1 , only the probability of experience i = 0 will be equal to 1 . 

Revised: By setting p = 1 , only the probability of experience i = 0 will be equal to 1 .

Original: and the rest will be zero [Figure] and by decreasing p , the probability will spread over the stream.
Revised: and the rest will be zero, and by decreasing p , the probability will spread over the stream.

Original: In figure 16 we show examples for generated scenarios with G samp with Geometric first occurrence and fixed probability of repetition
Revised: In figure 17 we show examples for generated scenarios with G samp with Geometric first occurrence and fixed probability of repetition.

Original: [Figure]
Revised: [Equation]

Revised: P r o b a b ili t y p =0.

Original: [Equation]
Revised: p =0. 10 Experience 0. p =0. p =0.

Original:  [Figure]
Revised: p =1. [Figure]

Revised: D U NBALANCED S CENARIOS

Revised: In this section we present a particular type of unbalanced scenarios where a subset of classes in the stream have a low probability of repetition and the rest repeat very often.

Revised: We refer to such scenarios bi-modal scenarios, where each mode refers to a subset of classes with a distinct repetition probability.

Revised: More specifically, we have a stream of experiences S = {

Revised: e 1 , e 2 , ..., e N } where Y S

Revised: =

Revised: (cid:83) N 1 Y e i indicates the set of all available concepts in S .

Revised: In bi-modal scenarios Y

Revised: = Y

Revised: if ∪ Y fr where

Revised: Y if and Y fr are the set of frequent and infrequent concepts respectively, and Y if ∪ Y fr = Ø .

Revised: In

Revised: Figure 18 we show examples of unbalanced bi-modal scenarios.

Revised: [Figure]

Original: D F REQUENCY -A WARE R EPLAY  A LGORITHM
Revised: E F REQUENCY -A WARE R EPLAY E. A LGORITHM

Original: In Algorithm 2 we present the steps for updating the buffer in FA storage policy.
Revised: In Algorithm 3 we present the steps for updating the buffer in FA storage policy.

Original: E C HANGING P f ( S )
Revised: E. A NALYSIS : VARYING THE FRACTION OF INFREQUENT CLASSES

Original: We conduct experiments to find the differences between situations in which all classes occur early versus those in which new classes also appear late in the  stream in order to analyze the role of first
Revised: In this section, we study the behavior of FA, CB, and RS storage policies by changing the fraction of infrequent classes. In our analysis, we consider an unbalanced stream generated with G samp where N = 100 and the probability of repetition for frequent and infrequent classes are 0 .

Original: [Figure]
Revised: 9 and 0 . 1 , respectively. In such streams, the large probability gap between frequent and infrequent classes

Original: Algorithm 2 Frequency-Aware Buffer.
Revised: Algorithm 3 Frequency-Aware Buffer.

Original: Require: Current Buffer Set B , Maximum buffer size M , List of Seen Classes C , Number of
Revised: Require: Current Buffer Set B , Maximum buffer size M , List of Seen Classes C , Number of

Original: Observation per Seen Class O
Revised: Observation per Seen Class O

Original: D = GetExperienceDataset( e i )
Revised: D = GetExperienceDataset( e i )

Original: P = DetectPresentClasses( D )
Revised: P = DetectPresentClasses( D )

Original: C ← C ∪ P for c ∈ P do ▷ For each present class, increment the number of observations if c ∈
Revised: C ← C ∪ P for c ∈ P do ▷ For each present class, increment the number of observations if c ∈

Original: O then O [ c ]+ =else O [ c ] =end if end for
Revised: O then O [ c ]+ =else O [ c ] =end if end for

Original: Q 
Revised: Q =

Original: [Equation]
Revised: [ 1 O [ c ] ∀ c ∈ C ]

Original: ▷ Calculate quota per class ˆ [Equation] ▷ Normalize quota values
Revised: ▷ Calculate quota per class ˆ Q = Q | Q | ▷ Normalize quota values

Original: [Equation] ▷ Calculate buffer slot size for each class
Revised: S = {⌈ Q [ c ] ∗ M ⌉∀ c ∈ C } ▷ Calculate buffer slot size for each class

Original: UpdateSlots( S )
Revised: UpdateSlots( S )

Original: ▷ Update assigned slots according to the current state of B
Revised: ▷ Update assigned slots according to the current state of B

Original: UpdateBuffer( B , D , S ) return B, M, C, O occurrence type.
Revised: UpdateBuffer( B , D , S ) return B, M, C, O [Figure]

Revised: helps us observe the difference more clearly.

Revised: We report the ratio of samples assigned to infrequent classes in the buffer in the lifetime of the model in the stream for scenarios where the fraction of infrequent classes is equal to { 20% , 40% , 60% , 80% , 100% } .

Revised: For this experiment, we set the buffer size to 500 for all methods.

Revised: As demonstrated in Figure 19, when the fraction of infrequent classes is equal to 20% , i.e. only 20% of classes are infrequent, the ratio is very low for RS policy as it tries to replicate the true distribution of the stream while CB assigns exactly 20% of the buffer space to the infrequent samples.

Original: How early or late in the stream we  observe all classes of a dataset, depends on the  the parameters that control P f ( S ) .
Revised: However, we can observe that FA starts to assign more samples over time to the infrequent classes over time as it adapts the buffer slots based on the frequency of repetition. Moreover, it is evident in the plots that, by increasing the fraction of infrequent classes, the ratio gap between FA and CB gets smaller as the quota for CB stays the same while the number of infrequent classes increases. Eventually, when the fraction of infrequent classes is equal to 100% , i.e. all classes have the same (low) probably of repetition, all buffers have exactly the same ratio since all classes are infrequent. In conclusion, FA buffer slots can be very helpful in highly unbalanced streams where a smaller fraction of classes have a low probability of repetition.

Revised: When the stream moves towards becoming balanced, the FA and CB get closer, and all methods become similar in the extreme case of a fully balanced stream with similar probability of repetition.

Revised: [Equation]

Revised: [Figure]

Revised: We conduct experiments to find the differences between situations in which all classes occur early versus those in which new classes also appear late in the stream in order to analyze the role of first occurrence type.

Revised: How early or late in the stream we observe all classes of a dataset, depends on the the parameters that control P f ( S ) .

Original: In this experiment, we fix the probability of repetition P r and change P f ( S ) ’s parameters.
Revised: In this experiment, we fix the probability of repetition P r and change P f ( S ) ’s parameters.

Original: In particular, we opt the geometric distribution for P f ( S ) and choose the values { 0 . 1 , 0 . 3 , 0 . 7 , 1 . 0 } for its only parameter 0 < p ≤ 1 . 0 .
Revised: In particular, we opt the geometric distribution for P f ( S ) and choose the values { 0 . 1 , 0 . 3 , 0 . 7 , 1 . 0 } for its only parameter 0 < p ≤ 1 . 0 .

Original: Increasing p is inversely proportional to the spread factor in the first occurrence distribution, i.e. when p is close to 0 all classes happen in the first experience and as we move p toward 1 . 

Revised: Increasing p is inversely proportional to the spread factor in the first occurrence distribution, i.e. when p is close to 0 all classes happen in the first experience and as we move p toward 1 .

Original: the classes start to spread along the stream.
Revised: the classes start to spread along the stream.

Original: Figure 17 shows the CIFAR-100 results for the Naive and ER-CB strategies.
Revised: Figure 20 shows the CIFAR-100 results for the Naive and ER-CB strategies.

Original: The results suggest that when the spread factor is low, the model initially has difficulty to learn since there are more classes in the initial experiments and thus the model has to learn from fewer instances.
Revised: The results suggest that when the spread factor is low, the model initially has difficulty to learn since there are more classes in the initial experiments and thus the model has to learn from fewer instances.

Original: However, with more experiences, all first occurrence types, reach almost the same SCA.
Revised: However, with more experiences, all first occurrence types, reach almost the same SCA.

Original: [Figure]

Original: F U NBALANCED S CENARIOS
Revised: G ER-FA R ESULTS

Original: In Figure 18 we show examples of unbalanced bi-modal scenarios with two modes of repetition where a fraction of  classes have high repetition probabilities an the rest have low repetition probabilities. [Figure] G ER-FA R ESULTS
Revised: Results in Figure 21 illustrate the total test accuracy and accuracy of frequent classes over time. Although the discrepancy between the accuracies of frequent classes is very small, the total test accuracy can significantly vary due to the difference in the accuracy of infrequent classes as presented in Section 4.4.

Original: [Figure]
Revised: [Figure]