Original: O NLINE P LACEBOS FOR C LASS - INCREMENTAL L EARNING
Revised: O NLINE P LACEBOS FOR C LASS - INCREMENTAL L EARNING

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Not forgetting old class knowledge is a key challenge for class-incremental learning (CIL) when the model continuously adapts to new coming classes.
Revised: Not forgetting old class knowledge is a key challenge for class-incremental learning (CIL) when the model continuously adapts to new coming classes.

Original: A common technique to address this is knowledge distillation (KD) which penalizes prediction inconsistencies between old and new models.
Revised: A common technique to address this is knowledge distillation (KD) which penalizes prediction inconsistencies between old and new models.

Original: Such prediction is made with almost new class data, as old class data is extremely scarce due to the strict memory limitation in CIL.
Revised: Such prediction is made with almost new class data, as old class data is extremely scarce due to the strict memory limitation in CIL.

Original: In this paper, we take a deep dive into KD losses and find that “using new class data for KD” not only hinders the model adaption (for learning new classes) but also results in low efficiency for preserving old class knowledge.
Revised: In this paper, we take a deep dive into KD losses and find that “using new class data for KD” not only hinders the model adaption (for learning new classes) but also results in low efficiency for preserving old class knowledge.

Original: We address this by “using the placebos of old classes for KD”, where the placebos are chosen from a free image stream, such as Google Images, in an automatical and economical fashion.
Revised: We address this by “using the placebos of old classes for KD”, where the placebos are chosen from a free image stream, such as Google Images, in an automatical and economical fashion.

Original: To this end, we train an online placebo selection policy to quickly evaluate the quality of streaming images (good or bad placebos) and use only good ones for one-time feed-forward computation of KD.
Revised: To this end, we train an online placebo selection policy to quickly evaluate the quality of streaming images (good or bad placebos) and use only good ones for one-time feed-forward computation of KD.

Original: We formulate the policy training process as an online Markov Decision Process (MDP), and introduce an online learning algorithm to solve this MDP problem without causing much computation costs.
Revised: We formulate the policy training process as an online Markov Decision Process (MDP), and introduce an online learning algorithm to solve this MDP problem without causing much computation costs.

Original: In experiments, we show that our method 1) is surprisingly effective even when there is no class overlap between placebos and original old class data, 2) does not require any additional supervision or memory budget, and 3) significantly outperforms a number of top-performing CIL methods, in particular when using lower memory budgets for old class exemplars, e.g., five exemplars per class.
Revised: In experiments, we show that our method 1) is surprisingly effective even when there is no class overlap between placebos and original old class data, 2) does not require any additional supervision or memory budget, and 3) significantly outperforms a number of top-performing CIL methods, in particular when using lower memory budgets for old class exemplars, e.g., five exemplars per class.

Original: The code is available in the supplementary.
Revised: The code is available in the supplementary.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: AI learning systems are expected to learn new concepts while maintaining the ability to recognize old ones.
Revised: AI learning systems are expected to learn new concepts while maintaining the ability to recognize old ones.

Original: In many practical scenarios, they cannot access the past data due to the limitations such as storage or data privacy but are expected to be able to recognize all seen classes.
Revised: In many practical scenarios, they cannot access the past data due to the limitations such as storage or data privacy but are expected to be able to recognize all seen classes.

Original: A pioneer work (Rebuffi et al., 2017) formulated this problem in the class-incremental learning (CIL) pipeline: training samples of different classes are loaded into the memory phase-by-phase, and the model keeps on re-training with new class data (while discarding old class data) and is evaluated on the testing data of both new and old classes.
Revised: A pioneer work (Rebuffi et al., 2017) formulated this problem in the class-incremental learning (CIL) pipeline: training samples of different classes are loaded into the memory phase-by-phase, and the model keeps on re-training with new class data (while discarding old class data) and is evaluated on the testing data of both new and old classes.

Original: The key challenge is that re-training the model on the new class data tends to override the knowledge acquired from the old classes (McCloskey et al., 1989; McRae et al., 1993; Ratcliff, 1990), and the problem is called “catastrophic forgetting”.
Revised: The key challenge is that re-training the model on the new class data tends to override the knowledge acquired from the old classes (McCloskey & Cohen, 1989; McRae & Hetherington, 1993; Ratcliff, 1990), and the problem is called “catastrophic forgetting”.

Original: To alleviate this problem, most of CIL methods (Rebuffi et al., 2017;
Revised: To alleviate this problem, most of CIL methods (Rebuffi et al., 2017;

Original: Hou et al., 2019; Douillard et al., 2020; Liu et al., 2020b; 2021a;
Revised: Hou et al., 2019; Douillard et al., 2020; Liu et al., 2020a; 2021a;

Original: Zhu et al., 2021) are equipped with knowledge distillation (KD) losses that penalize any feature and/or prediction inconsistencies between the models in adjacent phases.
Revised: Zhu et al., 2021) are equipped with knowledge distillation (KD) losses that penalize any feature and/or prediction inconsistencies between the models in adjacent phases.

Original: The ideal KD losses should be computed on old class data since the teacher model (i.e., the model in the last phase) was trained on them.
Revised: The ideal KD losses should be computed on old class data since the teacher model (i.e., the model in the last phase) was trained on them.

Original: This is, however, impossible in the CIL setting, where almost all old class data are inaccessible in the new phase.
Revised: This is, however, impossible in the CIL setting, where almost all old class data are inaccessible in the new phase.

Original: Existing methods have to use new class data as a substitute to compute KD losses.
Revised: Existing methods have to use new class data as a substitute to compute KD losses.

Original: We argue that this 1) hampers the learning of new classes as it distracts the model from fitting the ground truth labels of new classes, and 2) can not achieve the ideal result of KD, as the model can not generate the same soft labels (or features) on new class data as on old class data.
Revised: We argue that this 1) hampers the learning of new classes as it distracts the model from fitting the ground truth labels of new classes, and 2) can not achieve the ideal result of KD, as the model can not generate the same soft labels (or features) on new class data as on old class data.

Original: We justify this from an [Figure] r
Revised: We justify this from an empirical perspective as shown in Figure 1 (a):

Original: [Figure]
Revised: [Figure]

Original: k w a t

Original: e r
Revised: b r

Original: [Figure]
Revised: [Figure]

Original:  v a n d i s h w a s h e r p
Revised: k w a t e r po li ce v a n d i s h w a s h e r p

Original: i a no
Revised: i a no

Original: R o a d
Revised: R o a d

Original: T a b l e
Revised: T a b l e

Original: [Figure]
Revised: [Figure]

Original: The free image stream is ImageNet-1k that does not have class overlap with CIFAR-100. They are selected because their partial image regions contain similar visual cues with old classes.
Revised: with old classes.

Original: empirical perspective as shown in Figure 1 (a): the upper bound of KD is achieved when using “Old class data”, and if compared to it, using “New class data” sees a clear performance drop for recognizing both old and new classes.
Revised: the upper bound of KD is achieved when using “Old class data”, and if compared to it, using “New class data” sees a clear performance drop for recognizing both old and new classes.

Original: In Figure 1 (b), we show the reason by diving into loss computation details: when using new class samples (as substitutes) to compute CE and KD losses simultaneously, these two losses are actually weakening each other, which does not happen in the ideal case of using old class samples.
Revised: In Figure(b), we show the reason by diving into loss computation details: when using new class samples (as substitutes) to compute CE and KD losses simultaneously, these two losses are actually weakening each other, which does not happen in the ideal case of using old class samples.

Original: Using unlabeled external data (called placebos in this paper) has been shown to be practical and effective in solving the above issue.
Revised: Using unlabeled external data (called placebos in this paper) has been shown to be practical and effective in solving the above issue.

Original: Compared to existing works (Lee et al., 2019; Zhang et al., 2020), this paper aims to solve two open questions.
Revised: Compared to existing works (Lee et al., 2019; Zhang et al., 2020), this paper aims to solve two open questions.

Original: Q1 :
Revised: Q1 :

Original: How to adapt the placebo selection process in the non-stationary CIL pipeline.
Revised: How to adapt the placebo selection process in the non-stationary CIL pipeline.

Original: The ideal selection method needs to handle the dynamics of increasing classes in CIL, e.g., in a later incremental phase, it is expected to handle a more complex evaluation on the placebos of more old classes.
Revised: The ideal selection method needs to handle the dynamics of increasing classes in CIL, e.g., in a later incremental phase, it is expected to handle a more complex evaluation on the placebos of more old classes.

Original: Q2 :
Revised: Q2 :

Original: How to control the computational and memory-related costs during the selection and utilization of placebos.
Revised: How to control the computational and memoryrelated costs during the selection and utilization of placebos.

Original: It is not intuitive how to process external data without encroaching the memory allocated for new class data or breaking the strict assumption of memory budget in CIL.
Revised: It is not intuitive how to process external data without encroaching the memory allocated for new class data or breaking the strict assumption of memory budget in CIL.

Original: We solve these questions by proposing a new method called PlacoboCIL that adjusts the policy of selecting placebos for each new incremental phase, in an online and automatic fashion and without needing extra memory.
Revised: We solve these questions by proposing a new method called PlacoboCIL that adjusts the policy of selecting placebos for each new incremental phase, in an online and automatic fashion and without needing extra memory.

Original: Specifically, to tackle Q1 , we formulate the PlaceboCIL as an online Markov Decision Process (MDP) and introduce a novel online learning algorithm to learn a dynamic policy.
Revised: Specifically, to tackle Q1 , we formulate the PlaceboCIL as an online Markov Decision Process (MDP) and introduce a novel online learning algorithm to learn a dynamic policy.

Original: In each new phase, this policy produces a phase-specific function to evaluate the quality of incoming placebos.
Revised: In each new phase, this policy produces a phase-specific function to evaluate the quality of incoming placebos.

Original: The policy itself gets updated before the next phase.
Revised: The policy itself gets updated before the next phase.

Original: For Q2 , we propose a mini-batch-based memory reusing strategy for PlaceboCIL.
Revised: For Q2 , we propose a mini-batchbased memory reusing strategy for PlaceboCIL.

Original: Given a free data stream, we sample a batch of unlabeled data, evaluate their quality by using our phase-specific evaluation function (generated by the learned policy), and keep only the high-quality placebos to compute KD losses.
Revised: Given a free data stream, we sample a batch of unlabeled data, evaluate their quality by using our phase-specific evaluation function (generated by the learned policy), and keep only the high-quality placebos to compute KD losses.

Original: After this, we remove this batch totally from the memory before loading a new batch.
Revised: After this, we remove this batch totally from the memory before loading a new batch.

Original: In our implementation, this batch can be very small, e.g., 200 images.
Revised: In our implementation, this batch can be very small, e.g., 200 images.

Original: We randomly remove the same size of new class data for intaking this batch to keep the strict assumption of memory budget.
Revised: We randomly remove the same size of new class data for intaking this batch to keep the strict assumption of memory budget.

Original: We evaluate PlaceboCIL by incorporating it into multiple strong baselines such as PODNet (Douillard et al., 2020), LUCIR (Hou et al., 2019), AANets (Liu et al., 2021a), and FOSTER (Wang et al., 2022), and conducting a series of ablative studies.
Revised: We evaluate PlaceboCIL by incorporating it into multiple strong baselines such as PODNet (Douillard et al., 2020), LUCIR (Hou et al., 2019), AANets (Liu et al., 2021a), and FOSTER (Wang et al., 2022), and conducting a series of ablative studies.

Original: Our results on three popular CIL benchmarks show the clear and consistent superiority of PlaceboCIL, especially when using a low memory budget for old class exemplars.
Revised: Our results on three popular CIL benchmarks show the clear and consistent superiority of PlaceboCIL, especially when using a low memory budget for old class exemplars.

Original: For example, our method boosts the last-phase accuracy by 6 . 9 percentage points on average when keeping onlyexemplars per old class in the memory.
Revised: For example, our method boosts the last-phase accuracy by 6 . 9 percentage points on average when keeping only 5 exemplars per old class in the memory.

Original: In addition, it is worth mentioning that PlaceboCIL is surprisingly efficient even when there is no class overlap between placebos and original old class data.
Revised: In addition, it is worth mentioning that PlaceboCIL is surprisingly efficient even when there is no class overlap between placebos and original old class data.

Original: The reason is that PlaceboCIL can make use of the local visual cues in placebos, e.g., similar visual cues of “table” are found on the local regions of an “piano” (and “dishwasher”) image as shown in Figure 1 (c).
Revised: The reason is that PlaceboCIL can make use of the local visual cues in placebos, e.g., similar visual cues of “table” are found on the local regions of an “piano” (and “dishwasher”) image as shown in Figure 1 (c).

Original: Our contributions are three-fold. 1)
Revised: Our contributions are three-fold. 1)

Original: A generic PlacoboCIL method that selects placebo images from a free image stream to solve the KD issue in existing methods.
Revised: A generic PlacoboCIL method that selects placebo images from a free image stream to solve the KD issue in existing methods.

Original: A novel online learning algorithm for training a placebo selection policy and a mini-batch-based memory reusing strategy to avoid extra memory usage.3)
Revised: A novel online learning algorithm for training a placebo selection policy and a mini-batch-based memory reusing strategy to avoid extra memory usage.

Original:  Extensive comparisons and visualizations on three CIL benchmarks, taking top-performing CIL models as baselines and with the same strict assumption on memory.
Revised: 3) Extensive comparisons and visualizations on three CIL benchmarks, taking top-performing CIL models as baselines and with the same strict assumption on memory.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Class-incremental learning (CIL) methods can be divided into three categories.
Revised: Class-incremental learning (CIL) methods can be divided into three categories.

Original: Distillation-based methods introduce different knowledge distillation (KD) losses to consolidate previous knowledge.
Revised: Distillation-based methods introduce different knowledge distillation (KD) losses to consolidate previous knowledge.

Original: The key idea is to enforce prediction logits (Li et al., 2016; Rebuffi et al., 2017), feature maps (Douillard et al., 2020), or other essential information (Tao et al., 2020; Wang et al., 2022; Simon et al., 2021; Joseph et al., 2022) to be close to those of the pre-phase model.
Revised: The key idea is to enforce prediction logits (Li & Hoiem, 2016; Rebuffi et al., 2017), feature maps (Douillard et al., 2020), or other essential information (Tao et al., 2020; Wang et al., 2022; Simon et al., 2021; Joseph et al., 2022) to be close to those of the pre-phase model.

Original: Memory-based methods use a small number of preserved old class data (called exemplars) (Rebuffi et al., 2017;Shin et al., 2017;
Revised: Memory-based methods use a small number of preserved old class data (called exemplars) (Rebuffi et al., 2017;

Original:  Liu et al., 2020b; Prabhu et al., 2020) or augmented data (Zhu et al., 2021) to recall the old class knowledge.
Revised: Shin et al., 2017; Liu et al., 2020a; Prabhu et al., 2020) or augmented data (Zhu et al., 2021) to recall the old class knowledge.

Original: Network-architecture-based methods (Rusu et al., 2016; Xu et al., 2018; Abati et al., 2020; Yan et al., 2021) design incremental network architectures by expanding the network capacity for new class data or freezing partial network parameters to keep the old class knowledge.
Revised: Network-architecture-based methods (Rusu et al., 2016; Xu & Zhu, 2018; Abati et al., 2020; Yan et al., 2021) design incremental network architectures by expanding the network capacity for new class data or freezing partial network parameters to keep the old class knowledge.

Original: Our method can be used to improve different Distillation-based CIL methods.
Revised: Our method can be used to improve different Distillation-based CIL methods.

Original: Some prior works used unlabeled external data for class-incremental learning.
Revised: Some prior works used unlabeled external data for class-incremental learning.

Original: Lee et al.
Revised: Lee et al.

Original: (2019) proposed a confidence-based sampling method to select unlabeled external data to compute a specially-designed global distillation loss.
Revised: (2019) proposed a confidence-based sampling method to select unlabeled external data to compute a speciallydesigned global distillation loss.

Original: Zhang et al.
Revised: Zhang et al.

Original: (2020) randomly selected unlabeled samples and used them to compute KD loss for model consolidation.
Revised: (2020) randomly selected unlabeled samples and used them to compute KD loss for model consolidation.

Original: Liu et al.
Revised: Liu et al.

Original: (2020a) used unlabeled data to maximize the classifier discrepancy when integrating an ensemble of auxiliary classifiers.
Revised: (2020b) used unlabeled data to maximize the classifier discrepancy when integrating an ensemble of auxiliary classifiers.

Original: Our method differs from them in two aspects.
Revised: Our method differs from them in two aspects.

Original: Our method use the unlabeled data in a more generic way and can be applied to improve different distillationbased methods (Hou et al., 2019; Rebuffi et al., 2017; Wang et al., 2022), while the existing methods use unlabeled data to assist their specially-designed loss terms or components.
Revised: Our method use the unlabeled data in a more generic way and can be applied to improve different distillation-based methods (Hou et al., 2019; Rebuffi et al., 2017; Wang et al., 2022), while the existing methods use unlabeled data to assist their specially-designed loss terms or components.

Original: We train an online policy to select better unlabeled data to adapt to the non-stationary CIL pipeline while existing methods select unlabeled data by applying fixed (i.e., non-adaptive) rules in all incremental phases.
Revised: We train an online policy to select better unlabeled data to adapt to the non-stationary CIL pipeline while existing methods select unlabeled data by applying fixed (i.e., non-adaptive) rules in all incremental phases.

Original: Online learning observes a stream of samples and makes a prediction for each element in the stream.
Revised: Online learning observes a stream of samples and makes a prediction for each element in the stream.

Original: There are mainly two settings in online learning: full feedback and bandit feedback.
Revised: There are mainly two settings in online learning: full feedback and bandit feedback.

Original: Full feedback means that the full reward function is given at each stage.
Revised: Full feedback means that the full reward function is given at each stage.

Original: It can be solved by Best-Expert algorithms (Even-Dar et al., 2005).
Revised: It can be solved by Best-Expert algorithms (Even-Dar et al., 2005).

Original: Bandit feedback means that only the reward of the implemented decision is revealed.
Revised: Bandit feedback means that only the reward of the implemented decision is revealed.

Original: If the rewards are independently drawn from a fixed and unknown distribution, we may use e.g., Thompson sampling (Agrawal et al., 2012) and UCB (Auer et al., 2010) to solve it.
Revised: If the rewards are independently drawn from a fixed and unknown distribution, we may use e.g., Thompson sampling (Agrawal & Goyal, 2012) and UCB (Auer & Ortner, 2010) to solve it.

Original: If the rewards are generated in a non-stochastic version, we can solve it by e.g., Exp3 (Auer et al., 2002).
Revised: If the rewards are generated in a non-stochastic version, we can solve it by e.g., Exp3 (Auer et al., 2002).

Original: Online MDP is an extension of online learning.
Revised: Online MDP is an extension of online learning.

Original: Many studies (Even-Dar et al., 2009; Li et al., 2019) aim to solve it by converting it to online learning.
Revised: Many studies (Even-Dar et al., 2009; Li et al., 2019) aim to solve it by converting it to online learning.

Original: In our case, we formulate the CIL as an online MDP and convert it into a classic online learning problem.
Revised: In our case, we formulate the CIL as an online MDP and convert it into a classic online learning problem.

Original: The rewards in our MDP are non-stochastic because the training and validation data change in each phase.
Revised: The rewards in our MDP are non-stochastic because the training and validation data change in each phase.

Original: Therefore, we design our algorithm based on Exp3 (Auer et al., 2002).
Revised: Therefore, we design our algorithm based on Exp3 (Auer et al., 2002).

Original:  O NLINE P LACEBOS FOR C LASS - INCREMENTAL LEARNING (P LACEBO CIL)
Revised:  O NLINE P LACEBOS FOR C LASS - INCREMENTAL LEARNING (P LACEBO CIL)

Original: CIL has multiple “training-testing” phases during which the number of classes gradually increases to the maximum.
Revised: CIL has multiple “training-testing” phases during which the number of classes gradually increases to the maximum.

Original: In the 0 -th phase, data D 1: c 0 = {D 1 , ..., D c 0 } , including the training samples of c 0 classes, are used to learn the model Θ 0 .
Revised: In the 0 -th phase, data D 1: c 0 = {D 1 , ..., D c 0 } , including the training samples of cclasses, are used to learn the model Θ 0 .

Original: After this phase, only a small subset of D 1: c 0 (i.e., exemplars denoted as E 1: c 0 = {E 1 , ..., E c 0 } ) can be stored in the memory and used as replay samples in later phases.
Revised: After this phase, only a small subset of D 1: c 0 (i.e., exemplars denoted as E 1: c 0 = {E 1 , ..., E c 0 } ) can be stored in the memory and used as replay samples in later phases.

Original: In the ith phase, we use c i to denote the number of classes we have observed so far.
Revised: In the i -th phase, we use c i to denote the number of classes we have observed so far.

Original: We get new class data 
Revised: We get new class data D c

Original: D c i − 1 +1:
Revised: i − 1 +1:

Original: c i = {D c i − 1 +1 , ..., D c i } of ( c i − c i − 1 ) classes and load exemplars E 1: c i − 1 from the memory.
Revised: c i = {D c i − 1 +1 , ..., D c i } of ( c i − c i − 1 ) classes and load exemplars E 1: c i − 1 from the memory.

Original: Then, we initialize Θ i with Θ i − 1 , and train it using T 1: c i =
Revised: Then, we initialize Θ i with Θ i − 1 , and train it using T 1: c i =

Original: E 1: c i − 1 ∪D c
Revised: E 1: c i − 1 ∪D c

Original: i − 1 +1:
Revised: i − 1 +1:

Original: c i .
Revised: c i .

Original: The model Θ i will be evaluated with a testing set Q 1: c i = {Q 1 , ..., Q c i } for all classes seen so far.
Revised: The model Θ i will be evaluated with a testing set Q 1: c i = {Q 1 , ..., Q c i } for all classes seen so far.

Original: Please note that in any phase of PlaceboCIL, we assume we can access a free image stream, where we can load unlabeled images and select placebos.
Revised: Please note that in any phase of PlaceboCIL, we assume we can access a free image stream, where we can load unlabeled images and select placebos.

Original: PlaceboCIL formulates the CIL task as an online MDP.
Revised: PlaceboCIL formulates the CIL task as an online MDP.

Original: In each phase, we update a policy, for which we sample a class-balanced subset from training data as the testing set, and use the updated policy to produce a phase-specific evaluation function.
Revised: In each phase, we update a policy, for which we sample a class-balanced subset from training data as the testing set, and use the updated policy to produce a phase-specific evaluation function.

Original: During model training, we sample unlabeled images, use the evaluation function to quickly judge the image quality (good or bad placebos), and select the good ones to compute KD loss.
Revised: During model training, we sample unlabeled images, use the evaluation function to quickly judge the image quality (good or bad placebos), and select the good ones to compute KD loss.

Original: In this section, we introduce the formulation of online MDP in Section 3.1, show how to apply the policy to select placebos and compute KD loss in Section 3.2, and provide an online learning algorithm to update the policy in Section 3.3.
Revised: In this section, we introduce the formulation of online MDP in Section 3.1, show how to apply the policy to select placebos and compute KD loss in Section 3.2, and provide an online learning algorithm to update the policy in Section 3.3.

Original: The pseudocode is given in Appendix A (Algorithms 1 and 2).
Revised: The pseudocode is given in Appendix A (Algorithms 1 and 2).

Original: 3. O NLINE MDP F ORMULATION FOR CIL
Revised: 3. O NLINE MDP F ORMULATION FOR CIL

Original: The placebo selection process in CIL should be online inherently: training data (and classes) get updated in each phase, so the placebo selection policy should update accordingly.
Revised: The placebo selection process in CIL should be online inherently: training data (and classes) get updated in each phase, so the placebo selection policy should update accordingly.

Original: Thus, it is intuitive to formulate the CIL as an online MDP.
Revised: Thus, it is intuitive to formulate the CIL as an online MDP.

Original: In the following, we provide detailed formulations.
Revised: In the following, we provide detailed formulations.

Original: Stages.
Revised: Stages.

Original: Each phase in the CIL task can be viewed as a stage in the online MDP.
Revised: Each phase in the CIL task can be viewed as a stage in the online MDP.

Original: States.
Revised: States.

Original: The state should define the current situation of the intelligent agent.
Revised: The state should define the current situation of the intelligent agent.

Original: In CIL, we use the model Θ i as the state of the i -th phase (i.e., stage).
Revised: In CIL, we use the model Θ i as the state of the i -th phase (i.e., stage).

Original: We use S to denote the state space.
Revised: We use S to denote the state space.

Original: Actions.
Revised: Actions.

Original: We define the action as a i =( β
Revised: We define the action as a i =( β

Original: i , γ i ) , consisting of the hyperparameters ( β i and γ i ) used to create an evaluation function.
Revised: i , γ i ) , consisting of the hyperparameters ( β i and γ i ) used to create an evaluation function.

Original: As β i and γ i vary in a continuous range, we discretize them to define a finite action space.
Revised: As β i and γ i vary in a continuous range, we discretize them to define a finite action space.

Original: 1 We will elaborate on how to take an action and deploy the hyperparameters in Section 3.2.
Revised: 1 We will elaborate on how to take an action and deploy the hyperparameters in Section 3.2.

Original: Policy π = { p ( a | Θ i ) } a ∈ A is a probability distribution over the action space A , given the current state Θ i .
Revised: Policy π = { p ( a | Θ i ) } a ∈ A is a probability distribution over the action space A , given the current state Θ i .

Original: We will elaborate how to update the policy using our proposed online learning algorithm in Section 3.3.
Revised: We will elaborate how to update the policy using our proposed online learning algorithm in Section 3.3.

Original: Environments.
Revised: Environments.

Original: We take the training an testing data in each phase as the environment.
Revised: We take the training an testing data in each phase as the environment.

Original: In the i -th phase, the environment is H i =( T 1: c i , Q 1: c i ) , where T 1:
Revised: In the i -th phase, the environment is H i =( T 1: c i , Q 1: c i ) , where T 1:

Original: c i is the training data and Q 1: c i is the testing data.
Revised: c i is the training data and Q 1: c i is the testing data.

Original: The environment is time-varying because we observe different training data (and classes) in each new phase.
Revised: The environment is time-varying because we observe different training data (and classes) in each new phase.

Original: Rewards.
Revised: Rewards.

Original: CIL aims to train a model that is efficient in recognizing all classes seen so far.
Revised: CIL aims to train a model that is efficient in recognizing all classes seen so far.

Original: Therefore, it is intuitive to use the testing accuracy as the reward in each phase.
Revised: Therefore, it is intuitive to use the testing accuracy as the reward in each phase.

Original: We cannot observe any reward (i.e., testing accuracy) directly because the testing data is not accessible during training.
Revised: We cannot observe any reward (i.e., testing accuracy) directly because the testing data is not accessible during training.

Original: We solve this by building a local testing set using a subset of training data (see details in Section 3.3).
Revised: We solve this by building a local testing set using a subset of training data (see details in Section 3.3).

Original: Our objective is to maximize a cumulative reward, i.e., R = (cid:80) Ni =1 r H
Revised: Our objective is to maximize a cumulative reward, i.e., R = (cid:80) Ni =1 r H

Original: i (Θ i , a i ) , where r H i (Θ i , a i ) denotes the i -th phase reward, The reward function r H i changes with H i , so it is time-varying.[Figure]
Revised: i (Θ i , a i ) , where r H i (Θ i , a i ) denotes the i -th phase reward, The reward function r H i changes with H i , so it is time-varying.

Original: 3. P LACEBO S ELECTION WITH M INI - BATCH - BASED M EMORY R EUSING S TRATEGY
Revised: P LACEBO S ELECTION WITH M INI - BATCH - BASED M EMORY R EUSING S TRATEGY

Original: In the following we introduce how to build phase-specific evaluation functions using the policy, select highquality placebos without breaking memory constraints, and compute KD loss with the selected placebos.
Revised: In the following, we introduce how to build phase-specific evaluation functions using the policy, select high-quality placebos without breaking memory constraints, and compute KD loss with the selected placebos.

Original: The computation flow (in each phase) is illustrated in Figure 2.
Revised: The computation flow (in each phase) is illustrated in Figure 2.

Original: Computing prototypes.
Revised: Computing prototypes.

Original: Our placebo selection is based on the distance from placebo to the class prototype, i e., the mean feature of each class (Snell et al., 2017).
Revised: Our placebo selection is based on the distance from placebo to the class prototype, i.e., the mean feature of each class (Snell et al., 2017).

Original: First, we compute the prototypes of all seen classes.
Revised: First, we compute the prototypes of all seen classes.

Original: We use exemplars to compute the prototypes of old classes, and use new class  training data for new class prototypes, as follows,
Revised: We use exemplars to compute the prototypes of old classes, and use new class [Figure] 5) When we use up the selected placebos in P , we repeat the selection steps training data for new class prototypes, as follows,

Original: [Equation]
Revised: [Equation]

Original: where F Θ i ( · ) denotes the encoder (i.e., the feature extractor) of Θ i .
Revised: where F Θ i ( · ) denotes the encoder (i.e., the feature extractor) of Θ i .

Original: Pro( E n ) and Pro( D l ) denote the prototypes of the n -th old class and the l -th new class, respectively.
Revised: Pro( E n ) and Pro( D l ) denote the prototypes of the n -th old class and the l -th new class, respectively.

Original: Building evaluation functions.
Revised: Building evaluation functions.

Original: We argue that high-quality placebos for the m -th old class should meet two requirements: (1) being close to the prototype of the m -th class in the feature space because they will be used to activate the related neurons of the m -th old class in the model; and (2) being far from the prototypes of all the other classes in the feature space so that they will not cause the KD issue (as shown in Figure 1).
Revised: We argue that high-quality placebos for the m -th old class should meet two requirements: (1) being close to the prototype of the m -th class in the feature space because they will be used to activate the related neurons of the m -th old class in the model; and (2) being far from the prototypes of all the other classes in the feature space so that they will not cause the KD issue (as shown in Figure 1).

Original: To achieve these, we design the following evaluation function S m ( x ) for the m -th old class in the i -th phase:
Revised: To achieve these, we design the following evaluation function S m ( x ) for the m -th old class in the i -th phase:

Original: [Equation]
Revised: [Equation]

Original: where x denotes an unlabeled input image, and Sim( · , · ) denotes cosine similarity.
Revised: where x denotes an unlabeled input image, and Sim( · , · ) denotes cosine similarity.

Original: β i and γ i are two hyperparameters from the action
Revised: β i and γ i are two hyperparameters from the action

Original: a
Revised: a

Original: i =( β
Revised: i =( β

Original: i , γ i ) , sampled by the policy π .
Revised: i , γ i ) , sampled by the policy π .

Original: Allocating mini-batch-based memory for placebos.
Revised: Allocating mini-batch-based memory for placebos.

Original: We need to allocate a small amount of memory to store unlabeled images (before evaluating them).
Revised: We need to allocate a small amount of memory to store unlabeled images (before evaluating them).

Original: At the beginning of the i -th phase, we allocate memory buffers U and P respectively for the unlabeled image candidates and the selected placebos.
Revised: At the beginning of the i -th phase, we allocate memory buffers U and P respectively for the unlabeled image candidates and the selected placebos.

Original: In order to not exceed the memory budget, we randomly remove the same number, i.e., |U + P| , of samples from the training data of new classes.
Revised: In order to not exceed the memory budget, we randomly remove the same number, i.e., |U + P| , of samples from the training data of new classes.

Original: Our empirical results show this “remove” does not degrade the model performance on new classes.
Revised: Our empirical results show this “remove” does not degrade the model performance on new classes.

Original: Please kindly refer to Appendix G for detailed results and comparisons.
Revised: Please kindly refer to Appendix G for detailed results and comparisons.

Original: Selecting placebos.
Revised: Selecting placebos.

Original: Whenever the placebo buffer P is empty, we load a batch of unlabeled samples U from the free image stream, and choose K placebos for each old class to add into P , as follows,
Revised: Whenever the placebo buffer P is empty, we load a batch of unlabeled samples U from the free image stream, and choose K placebos for each old class to add into P , as follows,

Original: [Equation]
Revised: [Equation]

Original: Calculating loss with placebos.
Revised: Calculating loss with placebos.

Original: After selecting placebos, we sample a batch of new class data d ⊂ 
Revised: After selecting placebos, we sample a batch of new class data d ⊂ D c

Original: D c i − 1 +1:
Revised: i − 1 +1:

Original: c i , a batch of old class exemplars e ⊂ E 1: c 0 , and a batch of placebos p ⊂ P .
Revised: c i , a batch of old class exemplars e ⊂ E 1: c 0 , and a batch of placebos p ⊂ P .

Original: We calculate the overall loss as follows,
Revised: We calculate the overall loss as follows,

Original: [Equation]
Revised: [Equation]

Original: where L CE and L KD denote the CE loss and KD loss, respectively.
Revised: where L CE and L KD denote the CE loss and KD loss, respectively.

Original: λ is a hyperparameter to balance the two losses (Rebuffi et al., 2017).
Revised: λ is a hyperparameter to balance the two losses (Rebuffi et al., 2017).

Original: To control the memory usage, we delete p from P immediately after calculating the loss.
Revised: To control the memory usage, we delete p from P immediately after calculating the loss.

Original: When P is empty, we repeat the placebo selection operation.
Revised: When P is empty, we repeat the placebo selection operation.

Original: 3. O NLINE P OLICY L EARNING A LGORITHM
Revised: 3. O NLINE P OLICY L EARNING A LGORITHM

Original: A common approach to solving an online MDP is to approximate it as an online learning problem and solve it using online learning algorithms (Even-Dar et al., 2005; Agrawal et al., 2012; Auer et al., 2002).
Revised: A common approach to solving an online MDP is to approximate it as an online learning problem and solve it using online learning algorithms (Even-Dar et al., 2005; Agrawal & Goyal, 2012; Auer et al., 2002).

Original: We also follow this idea in PlaceboCIL, and our approximation follows the work by Even-Dar et al.
Revised: We also follow this idea in PlaceboCIL, and our approximation follows the work by Even-Dar et al.

Original: (2009) that is theoretically proved to have the optimal regret.
Revised: (2009) that is theoretically proved to have the optimal regret.

Original: Specifically, Even-Dar et al.
Revised: Specifically, EvenDar et al.

Original: (2009) relax the Markovian assumption of the MDP by decoupling the cumulative reward function and letting it be time-dependent so that they can solve online MDP by standard online learning algorithms.
Revised: (2009) relax the Markovian assumption of the MDP by decoupling the cumulative reward function and letting it be time-dependent so that they can solve online MDP by standard online learning algorithms.

Original: However, we cannot directly apply the algorithms proposed in (Even-Dar et al., 2009) to our problem.
Revised: However, we cannot directly apply the algorithms proposed in (Even-Dar et al., 2009) to our problem.

Original: It is because they assume full feedback meaning that the model can observe the rewards of all actions in every learning phase (which is also why its online learning problem can be solved by Best Expert algorithms (EvenDar et al., 2005)).
Revised: It is because they assume full feedback meaning that the model can observe the rewards of all actions in every learning phase (which is also why its online learning problem can be solved by Best Expert algorithms (Even-Dar et al., 2005)).

Original: While in CIL, we cannot observe any reward (i.e., the testing accuracies) because the testing data Q 1: c i are not accessible in any phase i .
Revised: While in CIL, we cannot observe any reward (i.e., the testing accuracies) because the testing data Q 1: c i are not accessible in any phase i .

Original: To address this problem, we split the training data we have in each phase into two subsets: one for training and another for validation.
Revised: To address this problem, we split the training data we have in each phase into two subsets: one for training and another for validation.

Original: Once we have a validation set, we can solve our online learning problem based on Exp3 (Auer et al., 2002)—a simple and effective bandit algorithm.
Revised: Once we have a validation set, we can solve our online learning problem based on Exp3 (Auer et al., 2002)—a simple and effective bandit algorithm.

Original: In the following, we elaborate how we do this data splitting in each local dataset (i.e., the entire data we have in each training phase of CIL), compute the decoupled cumulative reward, and learn the policy π with Exp3.
Revised: In the following, we elaborate how we do this data splitting in each local dataset (i.e., the entire data we have in each training phase of CIL), compute the decoupled cumulative reward, and learn the policy π with Exp3.

Original: Rebuilding local datasets.
Revised: Rebuilding local datasets.

Original: To compute reward, we sample a class-balanced subset B 1: c i from the training data T 1: c 1 .
Revised: To compute reward, we sample a class-balanced subset B 1: c i from the training data T 1: c 1 .

Original: B 1: c i contains the same number of samples for both the old and new classes.
Revised: B 1: c i contains the same number of samples for both the old and new classes.

Original: In this way, we rebuild the local training and validate sets, and update the environment from the oracle H i =( T 1: c i , Q 1: c i ) (which is unavailable in CIL) to the local environment
Revised: In this way, we rebuild the local training and validate sets, and update the environment from the oracle H i =( T 1: c i , Q 1: c i ) (which is unavailable in CIL) to the local environment

Original: h
Revised: h

Original: i =( T 1: c i \ B 1: c i , B 1: c i ) .
Revised: i =( T 1: c i \ B 1: c i , B 1: c i ) .

Original: Decoupled cumulative reward.
Revised: Decoupled cumulative reward.

Original: We create the decoupled cumulative reward function R based on the original cumulative reward function (cid:80) Nj =1 r H j (Θ j , a j ) .
Revised: We create the decoupled cumulative reward function R based on the original cumulative reward function (cid:80) Nj =1 r H j (Θ j , a j ) .

Original: In the i -th phase, we compute R as follows,
Revised: In the i -th phase, we compute R as follows,

Original: [Equation]
Revised: [Equation]

Original: where the “constant” denotes the historical rewards from the 1 -st phase to the ( i - 1) -th phase.
Revised: where the “constant” denotes the historical rewards from the 1 -st phase to the ( i - 1) -th phase.

Original: It doesn’t influence policy optimization.
Revised: It doesn’t influence policy optimization.

Original: R ( a i , h i ) is the long-term reward of a time-invariant local MDP based on the[Table] local environment h
Revised: R ( a i , h i ) is the long-term reward of a time-invariant local MDP based on the local environment h

Original: i .
Revised: i .

Original: We use R ( a i , h i ) as an estimation of the final cumulative reward, following (Even-Dar et al., 2009).
Revised: We use R ( a i , h i ) as an estimation of the final cumulative reward, following (Even-Dar et al., 2009).

Original: Because we don’t know the total number of phases N during training, we assume there will be n phases in the future.
Revised: Because we don’t know the total number of phases N during training, we assume there will be n phases in the future.

Original: Furthermore, we fix the action a i to simplify the training process.
Revised: Furthermore, we fix the action a i to simplify the training process.

Original: R ( a i , h i ) is a function of a i and h i .
Revised: R ( a i , h i ) is a function of a i and h i .

Original: Training policy with Exp3.
Revised: Training policy with Exp3.

Original: Exp3 (Auer et al., 2002) introduces an auxiliary variable w = { w ( a ) } a ∈ A .
Revised: Exp3 (Auer et al., 2002) introduces an auxiliary variable w = { w ( a ) } a ∈ A .

Original: It is updated as follows.
Revised: It is updated as follows.

Original: In the 1 -st phase, we initialize w as { 1 , . . .
Revised: In the 1 -st phase, we initialize w as { 1 , . . .

Original: In each phase i ( i ≥ 1 ), we update w for T iterations.
Revised: In each phase i ( i ≥ 1 ), we update w for T iterations.

Original: In the t -th iteration, we sample an action a t ∼ π , apply the action a t to the CIL system, and compute R ( a t , h i ) using Eq.
Revised: In the t -th iteration, we sample an action a t ∼ π , apply the action a t to the CIL system, and compute R ( a t , h i ) using Eq.

Original: After that, we update w ( a t ) in w as, 
Revised: After that, we update w ( a t ) in w as, w

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: |

Original: [Equation]
Revised: [Equation]

Original: )) , (6) where ξ can be regarded as the learning rate.
Revised: where ξ can be regarded as the learning rate.

Original: After updating w , we get the policy π = w / || w || .
Revised: After updating w , we get the policy π = w / || w || .

Original: The pseudocode is available in Appendix A (Algorithms 1 and 2). 
Revised: The pseudocode is available in Appendix A (Algorithms 1 and 2). [Table]

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We evaluate our method on three CIL benchmarks and achieve consistent improvements over multiple baseline methods.
Revised: We evaluate our method on three CIL benchmarks and achieve consistent improvements over multiple baseline methods.

Original: Below we introduce datasets and implementation details, followed by results and analyses, including the comparison to the state-of-the-art, an ablation study, and the visualization of our placebos.
Revised: Below we introduce datasets and implementation details, followed by results and analyses, including the comparison to the state-of-the-art, an ablation study, and the visualization of our placebos.

Original: Datasets and free image streams.
Revised: Datasets and free image streams.

Original: We use three datasets: CIFAR-100 (Krizhevsky et al., 2009), ImageNet100 (Rebuffi et al., 2017), and ImageNet-1k (Russakovsky et al., 2015).
Revised: We use three datasets: CIFAR-100 (Krizhevsky et al., 2009), ImageNet-100 (Rebuffi et al., 2017), and ImageNet-1k (Russakovsky et al., 2015).

Original: ImageNet-100, which containsclasses, is sampled from ImageNet-1k.
Revised: ImageNet-100, which contains 100 classes, is sampled from ImageNet-1k.

Original: We use exactly the same classes or orders as the related works (Rebuffi et al., 2017; Hou et al., 2019).
Revised: We use exactly the same classes or orders as the related works (Rebuffi et al., 2017; Hou et al., 2019).

Original: For CIFAR-100, we use ImageNet-1k as the free image stream.
Revised: For CIFAR-100, we use ImageNet-1k as the free image stream.

Original: For

Original:  ImageNet-100, we use a 900 -class subset of ImageNet-1k, which is the complement of ImageNet-100 in ImageNet-1k.
Revised: For ImageNet-100, we use a 900 -class subset of ImageNet-1k, which is the complement of ImageNet-100 in ImageNet-1k.

Original: For ImageNet-1k, we use a 1 , 000 -class subset of ImageNet-21k (Deng et al., 2009) without any overlapping class (different super-classes from those in ImageNet-1k).
Revised: For ImageNet-1k, we use a 1 , 000 -class subset of ImageNet-21k (Deng et al., 2009) without any overlapping class (different super-classes from those in ImageNet-1k).

Original: Implementation details.
Revised: Implementation details.

Original: Following (Hou et al., 2019; Douillard et al., 2020; Liu et al., 2021a;b), we use a modified 32 -layer ResNet for CIFAR-100 and an 18 -layer ResNet for ImageNet datasets.
Revised: Following (Hou et al., 2019; Douillard et al., 2020; Liu et al., 2021a;b), we use a modified 32 -layer ResNet for CIFAR-100 and an 18 -layer ResNet for ImageNet datasets.

Original: We use the following benchmark protocol: training the 0 -th phase model on half of all classes and observing the remaining classes evenly in the subsequent N phases ( N = 5 / 10 / 25 ).

Original: The number of exemplars for each class isin the default setting.
Revised: The number of exemplars for each class is 20 in the default setting.

Original: The training batch size is 128 .
Revised: The training batch size is 128 .

Revised: On CIFAR-100 (ImageNet-Subset/1k), we train it for 160 ( 90 ) epochs in each phase, and divide the learning rate by 10 after 80 ( 30 ) and then after 120 ( 60 ) epochs.

Revised: If the baseline is POD-AANets (Liu et al., 2021a), we fine-tune the model for 20 epochs using only exemplars.

Revised: We apply different forms of distillation losses on different baselines: (1) if the baselines are LwF and iCaRL, we use the softmax KL divergence loss; (2) if the baselines are LUCIR and AANets, we use the cosine embedding loss (Hou et al., 2019); and (3) if the baseline is POD-AANets, we use pooled outputs distillation loss (Douillard et al., 2020).

Original: For our PlaceboCIL, |U| and |P| are set as 1 ,and 200 , respectively.
Revised: For our PlaceboCIL, |U| and |P| are set as 1 , 000 and 200 , respectively.

Original: All experiments of our PlaceboCIL in the main paper use the “strict budget” setting, i.e., deleting |U + P| samples from training data in order to not exceed the memory budget.
Revised: All experiments of our PlaceboCIL in the main paper use the “strict budget” setting, i.e., deleting |U + P| samples from training data in order to not exceed the memory budget.

Original: An ablation study on the memory budget setting is provided in Appendix G. More implementation details are provided in Appendices C (benchmark protocol), D (network architecture), and E (training configurations).[Table] [Table]
Revised: An ablation study on the memory budget setting is provided in Appendix G. More implementation details are provided in Appendices C (benchmark protocol), D (network architecture), and E (training configurations).

Original: Results on five baselines.
Revised: Results on five baselines.

Original: Table 1 shows the average and last-phase accuracy for five baselines (i [Table]
Revised: Table 1 shows the average and last-phase accuracy for five baselines (i.e.,

Original: e., LwF (Li  et al. , et ), LUCIR (Hou et al., 2019) et al 2021a
Revised: LwF (Li & Hoiem, 2016), iCaRL (Rebuffi et al., 2017), LUCIR (Hou et al., 2019), AANets (Liu et al., 2021a), and FOSTER (Wang et al., 2022)).

Original: et al

Original: From the table [Table] we make the following observations.
Revised: From the table, we make the following observations.

Original: 1) Using our PlaceboCIL boosts the performance of the baselines clearly and consistently in all settings, indicating that our method is generic and efficient.
Revised: 1) Using our PlaceboCIL boosts the performance of the baselines clearly and consistently in all settings, indicating that our method is generic and efficient.

Original: When the number of exemplars decreases, the improvement brought by our method becomes more significant.
Revised: When the number of exemplars decreases, the improvement brought by our method becomes more significant.

Original: For example, the lastphase accuracy improvement of LUCIR increases from 2 .
Revised: For example, the last-phase accuracy improvement of LUCIR increases from 2 .

Original: 52 to 13 .
Revised: 52 to 13 .

Original: 88 percentage points when the number of exemplars per class decreases from 20 to 5 .
Revised: 88 percentage points when the number of exemplars per class decreases from 20 to 5 .

Original: This reveals that the superiority of our method is more obvious when the forgetting problem is more serious (with fewer exemplars) due to a tighter memory budget in CIL.
Revised: This reveals that the superiority of our method is more obvious when the forgetting problem is more serious (with fewer exemplars) due to a tighter memory budget in CIL.

Original: Our PlaceboCIL can boost the performance of all KD terms, i.e.,not only for logits-based
Revised: Our PlaceboCIL can boost the performance of all KD terms, i.e.,

Original: KD (Rebuffi et al., 2017) but also for feature-based KD (Hou et al., 2019; Douillard et al., 2020).
Revised: [Table] for based KD ( al also

Original: Comparisons to the state-of-the-art.
Revised: Comparisons to the state-of-the-art.

Original: Table 2 (Blocks 1&3) shows the results of our best model (taking PlaceboCIL as a plug-in module in the top method (Wang et al., 2022)) and some recent top-performing methods.
Revised: Table 2 (Blocks 1&3) shows the results of our best model (taking PlaceboCIL as a plug-in module in the top method (Wang et al., 2022)) and some recent top-performing methods.

Original: We can see that using our PlaceboCIL outperforms all previous methods.
Revised: We can see that using our PlaceboCIL outperforms all previous methods.

Original: Intriguingly, we find that we can surpass others more when the number of phases is larger—where there are more serious forgetting problems.
Revised: Intriguingly [Table] we find that we can surpass others more when the number of phases is larger—where there are more serious forgetting problems.

Original: For example, when N = 25 , we improve POD-AANets by 4 . 4% on the ImageNet-100, while this number is only 1 . 6% when N = 5 (which is an easier setting with more saturated results).
Revised: For example, when N = [Table] we improve POD-AANets by 4 . 4% on the ImageNet-100, while this number is only 1 . 6% when N = 5 (which is an easier setting with more saturated results).

Original: This reflects the encouraging efficiency of our method for reducing the forgetting of old class knowledge in CIL models.
Revised: This reflects the encouraging efficiency of our method for reducing the forgetting of old class knowledge in CIL models

Original: Comparisons to the CIL methods using unlabeled data.
Revised: [Table]

Original: Table 2 (Blocks 2&3) shows the results of our best model and CIL methods using unlabeled data (GD+ext (Lee et al., 2019) and MUC-LwF (Liu et al., 2020a)).
Revised: Table 2 (Blocks 2&3) shows the results of our best model and CIL methods using unlabeled data GD+ext (Lee et al., 2019) MUC LwF (Liu et al., 2020b)).

Original: We can see that our method consistently performs better than others.
Revised: We can see that our method consistently performs better than others.

Original: For another related work,
Revised: For another related work,

Original: DMC (Zhang et al., 2020), we didn’t find the public code.
Revised: DMC (Zhang et al., 2020), we didn’t find the public code.

Original: So, we compare ours with DMC using their paper’s setting: iCaRL w/ ours achieves 62 .
Revised: So, we compare ours with DMC using their paper’s setting: iCaRL w/ ours achieves 62 .

Original: 3% , while the result of DMC is 59 .
Revised: 3% , while the result of DMC is 59 .

Original: 1% (CIFAR-100, 10 phases, 10 classes/phase).
Revised: 1% (CIFAR-100, 10 phases,classes/phase).

Original: Please kindly refer to Appendix B for more analyses.
Revised: Please kindly refer to Appendix B for more analyses.

Original: Ablation study.
Revised: Ablation study.

Original: Tables 3 (and A1) show the ablation results for free data streams, policy learning methods, and placebo selection strategies
Revised: Tables 3 (and A1) show the ablation results for free data streams, policy learning methods, and placebo selection strategies.

Original:  First 
Revised: 1) First block.

Original: Rows shows the baselines.
Revised: Row 1 shows the baselines.

Original: Row 2 shows our method.[Figure] upright [Figure]
Revised: Row 2 shows our method.

Original: l

Original: [Figure]

Original: ss : t a b l e randomly pick five new classes and five old classes

Original: 2) Second block: different free data streams.
Revised: Second block: different free data streams.

Original: Rows 3-6 show the ablation results for the following free data streams.
Revised: Rows 3-6 show the ablation results for the following free data streams.

Original: (1) “Overlapping” means including samples from the overlapping classes between 
Revised: (1) “Overlapping” means including samples from the overlapping classes between CIFAR-100 and ImageNet. (2) “Non- Siberian husky unicycle C

Original: CIFAR-100 and ImageNet.
Revised: I F A Rc l a ss : r o a d [Figure]

Revised: radio telescope n04515003 - upright terrier n03290653 - entertainment marimba home theater upright entertainment

Original: “Non-overlapping” means using only the samples of non-overlapping classes between CIFAR-100 and ImageNet (more realistic than “Overlapping”).
Revised: [Figure] c l a ss : t a b l e overlapping” means using only the samples of non-overlapping classes between CIFAR-100 and ImageNet (more realistic than “Overlapping”).

Original: “new class data” means using only the current-phase new class data (i.e., without using any free data stream) as candidates to select placebos.
Revised: (3) “New data” means using only the current-phase new class data (i.e., without using any free data stream) as candidates to select placebos.

Original: (4) “Old data” means the original old class data are all accessible when computing KD loss (i.e., upper bound of KD effect).
Revised: (4) “Old data” means the original old class data are all accessible when computing KD loss (i.e., upper bound of KD effect).

Original: Please note that in (1) and (2), two classes are considered “overlapping” if their classes or super-classes overlap.
Revised: Please note that in (1) and (2), two classes are considered “overlapping” if their classes or super-classes overlap.

Original: For example, “n02640242 - sturgeon” in ImageNet-1k is regarded as an overlapping class of the “fish” in CIFAR-100, because they overlap at the level of super-class (i.e., “fish”).
Revised: For example, “n02640242 - sturgeon” in ImageNet-1k is regarded as an overlapping class of the “fish” in CIFAR-100, because they overlap at the level of super-class (i.e., “fish”).

Original: When comparing Row 4 with Row 2, we can find that our method is robust to the change of data streams: even if all overlapping classes are removed, our method can still achieve the same-level performance.
Revised: When comparing Row 4 with Row 2, we can find that our method is robust to the change of data streams: even if all overlapping classes are removed, our method can still achieve the same-level performance.

Original: Comparing Row 5 with Row 2, we can get a clear sense that using additional unlabeled data is definitely helpful.
Revised: Comparing Row 5 with Row 2, we can get a clear sense that using additional unlabeled data is definitely helpful.

Original: Comparing Row 6 with Row 2, we see that our method achieves comparable results to the upper bound.
Revised: Comparing Row 6 with Row 2, we see that our method achieves comparable results to the upper bound.

Original: 3) Third block: different policy learning methods.
Revised: 3) Third block: different policy learning methods.

Original: Row 7 is for using fixed evaluation functions ( β i = γ
Revised: Row 7 is for using fixed evaluation functions ( β i = γ

Original: i = 1 ).
Revised: i = 1 ).

Original: Row 8 uses the offline RL (the REINFORCE algorithm (Liu et al., 2021b)) to train the selection policy.
Revised: Row 8 uses the offline RL (the REINFORCE algorithm (Liu et al., 2021b)) to train the selection policy.

Original: Comparing Row 7 with Row 2 shows that using online learning successfully boosts the model performance.
Revised: Comparing Row 7 with Row 2 shows that using online learning successfully boosts the model performance.

Original: Comparing Row 8 with Row 2, we are happy to see that our online learning method achieves the same-level performance as the offline RL while the training time is much less.
Revised: Comparing Row 8 with Row 2, we are happy to see that our online learning method achieves the same-level performance as the offline RL while the training time is much less.

Original: The training time of the baseline (without learning a policy) is 2 .
Revised: The training time of the baseline (without learning a policy) is 2 .

Original: 7 hours.
Revised: 7 hours.

Original: It becomes around 650 hours if we solve the MDP by offline RL.
Revised: It becomes aroundhours if we solve the MDP by offline RL.

Original: In contrast, using our online method takes only 4 . 5 hours.
Revised: In contrast, using our online method takes only 4 . 5 hours.

Original: Fourth block : different placebo selection strategies.
Revised: Fourth block: different placebo selection strategies.

Original: Row 9 uses unlabeled data with higher confidence following (Lee et al., 2019).
Revised: Row 9 uses unlabeled data with higher confidence following (Lee et al., 2019).

Original: Row 10 uses them randomly following (Zhang et al., 2020).
Revised: Row 10 uses them randomly following (Zhang et al., 2020).

Original: Comparing these results with Row 2 shows our superiority.
Revised: Comparing these results with Row 2 shows our superiority.

Revised: The “mini-batch-based memory reusing strategy” is applied in Rows 9 and 10.

Original: Visualization results.
Revised: Visualization results.

Original: Figure 3 (a) demonstrates the activation maps visualized by Grad-CAM for the placebos of two old classes on CIFAR100 (“road” and “table”).
Revised: Figure 3 (a) demonstrates the activation maps visualized by Grad-CAM for the placebos of two old classes on CIFAR-100 (“road” and “table”).

Original: ImageNet-1k is the free data stream.
Revised: ImageNet-1k is the free data stream.

Original: We can see that the selected placebos contain the parts of “road” and “table” even though their original labels (on ImageNet-1k) are totally different classes.
Revised: We can see that the selected placebos contain the parts of “road” and “table” even though their original labels (on ImageNet-1k) are totally different classes.

Original: While this is not always the case, our method seems to find sufficiently related images to old classes that activate the related neurons for old classes (“road” and “table”).
Revised: While this is not always the case, our method seems to find sufficiently related images to old classes that activate the related neurons for old classes (“road” and “table”).

Original: To illustrate that, Figure 3 (b) shows t-SNE results for placebos, old class data (not visible during training), and new class data.
Revised: To illustrate that, Figure 3 (b) shows t-SNE results for placebos, old class data (not visible during training), and new class data.

Original: We can see that the placebos are located near the old class data and far away from the new class data.
Revised: We can see that the placebos are located near the old class data and far away from the new class data.

Original: This is why placebos can recall the old knowledge without harming the new class learning.
Revised: This is why placebos can recall the old knowledge without harming the new class learning.

Original:  C ONCLUSIONS
Revised:  C ONCLUSIONS

Original: We proposed a novel method, PlaceboCIL, which selects high-quality placebo data from free data streams and uses them to improve the effect of KD in CIL.
Revised: We proposed a novel method, PlaceboCIL, which selects high-quality placebo data from free data streams and uses them to improve the effect of KD in CIL.

Original: We designed an online learning method to make the selection of placebos more adaptive in different phases and a mini-batch-based memory reusing strategy to control memory usage.
Revised: We designed an online learning method to make the selection of placebos more adaptive in different phases and a mini-batch-based memory reusing strategy to control memory usage.

Original: Extensive experiments show that our method is general and efficient.
Revised: Extensive experiments show that our method is general and efficient.

Original: E THICS S TATEMENT
Revised: E THICS S TATEMENT

Original: Computational costs.
Revised: Computational costs.

Original: Deep learning methods require intensive usage of computing resources, which is not climate-friendly.
Revised: Deep learning methods require intensive usage of computing resources, which is not climate-friendly.

Original: It calls for future research into proposing more effective training strategies that can speed up the training.
Revised: It calls for future research into proposing more effective training strategies that can speed up the training.

Original: Privacy issues.
Revised: Privacy issues.

Original: Keeping old class exemplars has the issue of data privacy.
Revised: Keeping old class exemplars has the issue of data privacy.

Original: This calls for future research that explicitly forgets or mitigates the identifiable feature of the data.
Revised: This calls for future research that explicitly forgets or mitigates the identifiable feature of the data.

Original: Licenses.
Revised: Licenses.

Original: We use the open-source code for the following papers: AANets (Liu et al., 2021a), iCaRL (Rebuffi et al., 2017), Mnemonics (Liu et al., 2020b), PODNet (Douillard et al., 2020), FOSTER (Wang et al., 2022).
Revised: We use the open-source code for the following papers: AANets (Liu et al., 2021a), iCaRL (Rebuffi et al., 2017), Mnemonics (Liu et al., 2020a), PODNet (Douillard et al., 2020), FOSTER (Wang et al., 2022).

Original: They are licensed under the MIT License.
Revised: They are licensed under the MIT License.

Original: Datasets.
Revised: Datasets.

Original: We use three datasets in our paper: CIFAR-100 (Krizhevsky et al., 2009), ImageNet-100 (Rebuffi et al., 2017) and ImageNet-1k (Russakovsky et al., 2015).
Revised: We use three datasets in our paper: CIFAR-100 (Krizhevsky et al., 2009), ImageNet100 (Rebuffi et al., 2017) and ImageNet-1k (Russakovsky et al., 2015).

Original: The data for both datasets are downloaded from their official websites and allowed to use for non-commercial research and educational purposes.
Revised: The data for both datasets are downloaded from their official websites and allowed to use for non-commercial research and educational purposes.

Original: R EPRODUCIBILITY S TATEMENT
Revised: R EPRODUCIBILITY S TATEMENT

Original: We include the details of reproducing our experiments in Section 4 (main paper), Appendix C, Appendix D, and Appendix E. We also provide the code in the supplementary.
Revised: We include the details of reproducing our experiments in Section 4 (main paper), Appendix C, Appendix D, and Appendix E. We also provide the code in the supplementary.

Original: A PPENDICES A A LGORITHMS
Revised: A PPENDICES A A LGORITHMS

Original: Supplementary to Section 3.
Revised: Supplementary to Section 3.

Original: Algorithm 1 summarizes the overall training steps of the proposed method in the i -th phase ( i ≥ 1 ).
Revised: Algorithm 1 summarizes the overall training steps of the proposed method in the i -th phase ( i ≥ 1 ).

Original: Algorithm 2 presents the training with placebos for a specific action a .
Revised: Algorithm 2 presents the training with placebos for a specific action a .

Original: Policy
Revised: Policy

Revised: learning i

Revised: = 1 then do training

Original: B C OMPARISONS TO THE CIL METHODS USING UNLABELED DATA .
Revised: B C OMPARISONS TO THE CIL METHODS USING UNLABELED DATA .

Original: This is supplementary to Section 4 Comparisons to the CIL methods using unlabeled data .
Revised: This is supplementary to Section 4 “Comparisons to the CIL methods using unlabeled data.”

Original: Part I: Comparing with GD+ext (Lee et al., 2019) and DMC (Zhang et al., 2020) .
Revised: Part I: Comparing with GD+ext (Lee et al., 2019) and DMC (Zhang et al., 2020) .

Original: Lee et al.
Revised: Lee et al.

Original: (2019) use the unlabeled data to compute the distillation loss in CIL.
Revised: (2019) use the unlabeled data to compute the distillation loss in CIL.

Original: Our work also uses unlabeled data but differs from (Lee et al., 2019) in three aspects:
Revised: Our work also uses unlabeled data but differs from (Lee et al., 2019) in three aspects:

Original: • Our method is focused on the learnable strategies of sampling helpful unlabeled data for CIL.
Revised: • Our method is focused on the learnable strategies of sampling helpful unlabeled data for CIL.

Original: It is generic and can be plugged into different distillation-based frameworks, including both logit distillation (LwF and iCaRL) and feature distillation (LUCIR and PODNet).
Revised: It is generic and can be plugged into different distillation-based frameworks, including both logit distillation (LwF and iCaRL) and feature distillation (LUCIR and PODNet).

Original: In Table 2, we also show the empirical results (comparison) to demonstrate our superiority.
Revised: In Table 2, we also show the empirical results (comparison) to demonstrate our superiority.

Original: • We propose an online learning algorithm.
Revised: • We propose an online learning algorithm.

Original: Its learned policy can adaptively produce phase-specific functions to evaluate the quality of the unlabeled samples.
Revised: Its learned policy can adaptively produce phasespecific functions to evaluate the quality of the unlabeled samples.

Original: So our selection functions change accordingly when the number of phases increases.
Revised: So our selection functions change accordingly when the number of phases increases.

Original: Lee et al.
Revised: Lee et al.

Original: (2019)’s selection strategy remains unchanged when the number of phases increases.
Revised: (2019)’s selection strategy remains unchanged when the number of phases increases.

Original: Zhang et al.
Revised: Zhang et al.

Original: (2020) use the unlabeled data without selection.
Revised: (2020) use the unlabeled data without selection.

Original: In Table 3, we show our learnable selection functions generally perform better than (Lee et al., 2019) and (Zhang et al., 2020).
Revised: In Table 3, we show our learnable selection functions generally perform better than (Lee et al., 2019) and (Zhang et al., 2020).

Original: Please kindly refer to Section 4 “ Ablation study ” for the results.
Revised: Please kindly refer to Section 4 “ Ablation study ” for the results.

Original: • Our employment of unlabeled data is dynamic and takes little memory out of the total memory budget.
Revised: • Our employment of unlabeled data is dynamic and takes little memory out of the total memory budget.

Original: We maintain the memory budget strictly the same as the baselines (LUCIR (Hou et al., 2019),
Revised: We maintain the memory budget strictly the same as the baselines (LUCIR (Hou et al., 2019),

Original: PODNet (Christopoulos et al., 2000), and Mnemonics (Liu et al., 2020b)), by loading fewer samples of new classes.
Revised: PODNet (Christopoulos et al., 2000), and Mnemonics (Liu et al., 2020a)), by loading fewer samples of new classes.

Original: In contrast, (Lee et al., 2019) requires double-size memory compared to the baselines (LUCIR (Hou et al., 2019), PODNet (Christopoulos et al., 2000)), and Mnemonics (Liu et al ., 2020b)).
Revised: In contrast, (Lee et al., 2019) requires double-size memory compared to the baselines (LUCIR (Hou et al., 2019), PODNet (Christopoulos et al., 2000)), and Mnemonics (Liu et al., 2020a)).

Original: Part II: Comparing with MUC (Liu et al., 2020a).
Revised: Part II: Comparing with MUC (Liu et al., 2020b).

Original: Our method and (Liu et al., 2020a) use unlabeled data for different purposes and in different ways.
Revised: Our method and (Liu et al., 2020b) use unlabeled data for different purposes and in different ways.

Original: We use the unlabeled placebo data from a free image stream to improve the positive effect of KD in training CIL models.
Revised: We use the unlabeled placebo data from a free image stream to improve the positive effect of KD in training CIL models.

Original: Liu et al.
Revised: Liu et al.

Original: (2020a) integrate the ensemble of multiple FC classifiers to set regularization-based constraints.
Revised: (2020b) integrate the ensemble of multiple FC classifiers to set regularization-based constraints.

Original: They use the unlabeled data to compute the classifier discrepancy loss by using the predictions of multiple classifiers.
Revised: They use the unlabeled data to compute the classifier discrepancy loss by using the predictions of multiple classifiers.

Original: C B ENCHMARK P ROTOCOLS
Revised: C B ENCHMARK P ROTOCOLS

Original: This is supplementary to Section 4 “ Implementation details .
Revised: This is supplementary to Section 4 “ Implementation details .

Original: We follow the benchmark protocol used in (Douillard et al., 2020; Hou et al., 2019; Liu et al., 2021a; 2020b).
Revised: We follow the benchmark protocol used in (Douillard et al., 2020; Hou et al., 2019; Liu et al., 2021a; 2020a).

Original: Given a dataset, the initial model (in the 0 -th phase) is trained on the data of half of the classes.
Revised: Given a dataset, the initial model (in the 0 -th phase) is trained on the data of half of the classes.

Original: Then, it learns the remaining classes evenly in the subsequent N phases.
Revised: Then, it learns the remaining classes evenly in the subsequent N phases.

Original: For N , there are three options ( 5 , 10 , and 25 ), and the corresponding settings are called “ N -Phase”.
Revised: For N , there are three options ( 5 , 10 , and 25 ), and the corresponding settings are called “ N -Phase”.

Original: The learned model in each phase is evaluated on the test set containing all seen classes.
Revised: The learned model in each phase is evaluated on the test set containing all seen classes.

Original: In the tables, we report average accuracy over all phases and the last-phase accuracy.
Revised: In the tables, we report average accuracy over all phases and the last-phase accuracy.

Original: D N ETWORK A RCHITECTURE D ETAILS
Revised: D N ETWORK A RCHITECTURE D ETAILS

Original: This is supplementary to Section 4 “ Implementation details .”
Revised: This is supplementary to Section 4 “ Implementation details .”

Original: Following (Hou et al., 2019; Liu et al., 2021a; Rebuffi et al., 2017; Wu et al., 2019), we use a modified 32 -layer ResNet (Rebuffi et al., 2017) for CIFARand an 18 -layer ResNet (He et al., 2016) for ImageNet-100 and ImageNet-1k.
Revised: Following (Hou et al., 2019; Liu et al., 2021a; Rebuffi et al., 2017; Wu et al., 2019), we use a modified 32 -layer ResNet (Rebuffi et al., 2017) for CIFAR-100 and an 18 -layer ResNet (He et al., 2016) for ImageNet-100 and ImageNet-1k.

Original: Please note that it is standard to use a shallower ResNet for ImageNet-100 and ImageNet-1k in CIL.
Revised: Please note that it is standard to use a shallower ResNet for ImageNet-100 and ImageNet-1k in CIL.

Original: The 32 -layer ResNet consists ofinitial convolution layer and 3 residual blocks (in a single branch).
Revised: The 32 -layer ResNet consists of 1 initial convolution layer and 3 residual blocks (in a single branch).

Original: Each block has 10 convolution layers with 3 × 3 kernels.
Revised: Each block has 10 convolution layers with 3 × 3 kernels.

Original: The number of filters starts from 16 and is doubled every next block.
Revised: The number of filters starts from 16 and is doubled every next block.

Original: After these 3 blocks, there is an average-pooling layer to compress the output feature maps to a feature embedding.
Revised: After these 3 blocks, there is an average-pooling layer to compress the output feature maps to a feature embedding.

Original: The 18 -layer ResNet follows the standard settings in (He et al., 2016).
Revised: The 18 -layer ResNet follows the standard settings in (He et al., 2016).

Original: We deploy AANets using the same parameters as its original paper (Liu et al., 2021a).
Revised: We deploy AANets using the same parameters as its original paper (Liu et al., 2021a).

Original: E  T RAINING C ONFIGURATIONS
Revised: E M ORE T RAINING C ONFIGURATIONS

Original: This is supplementary to Section 4 “ Implementation details .”
Revised: This is supplementary to Section 4 “ Implementation details .”

Original: The training of the classification model Θ exactly follows the uniform setting in (Douillard et al., 2020; Hou et al., 2019; Liu et al., 2021a; 2020b).
Revised: The training of the classification model Θ exactly follows the uniform setting in (Douillard et al., 2020; Hou et al., 2019; Liu et al., 2021a; 2020a).

Original: On CIFAR-100 (ImageNet-Subset/1k), we train it for 160 ( 90 ) epochs in each phase, and divide the learning rate by 10 after 80 ( 30 ) and then after 120 ( 60 ) epochs.

Original: If the baseline is POD-AANets (Liu et al., 2021a), we fine-tune the model for 20 epochs using only exemplars.

Original: We apply different forms of distillation losses on different baselines: (1) if the baselines are LwF (Li et al., 2016) and iCaRL (Rebuffi et al., 2017), we use the softmax KL divergence loss; (2) if the baselines are LUCIR (Hou et al., 2019) and AANets (Liu et al., 2021a), we use the cosine embedding loss (Hou et al., 2019); and (3) if the baseline is POD-AANets (Liu et al., 2021a), we use pooled outputs distillation loss (Douillard et al., 2020).

Original: F A BLATION STUDY ON MORE BASELINES
Revised: F A BLATION STUDY ON MORE BASELINES

Original: This is supplementary to Section 4 “ Ablation study .”
Revised: This is supplementary to Section 4 “ Ablation study .”

Original: In Table A1, we provide the ablation results on another two baselines, LwF Li et al. (2016) and LUCIR Hou et al.
Revised: In Table A1, we provide the ablation results on another two baselines, LwF Li & Hoiem (2016) and LUCIR Hou et al.

Original: [Table]
Revised: [Table]

Original: G A BLATION STUDY FOR THE MEMORY BUDGET SETTINGS
Revised: G A BLATION STUDY FOR THE MEMORY BUDGET SETTINGS

Original: This is supplementary to Section 4.
Revised: This is supplementary to Section 4.

Original: In Table A2, Row 1 shows baselines.
Revised: In Table A2, Row 1 shows baselines.

Original: Rows 2 and 3 show the baselines with our PlaceboCIL as a plug-in module.
Revised: Rows 2 and 3 show the baselines with our PlaceboCIL as a plug-in module.

Original: Row 2 “budget” means the combined size of episodic memory and the placebo buffer is (strictly) equal to the episodic memory size in baseline methods.
Revised: Row 2 “budget” means the combined size of episodic memory and the placebo buffer is (strictly) equal to the episodic memory size in baseline methods.

Original: To this end, we randomly delete |U + P| samples from the new class data D c i − 1 +1:
Revised: To this end, we randomly delete |U + P| samples from the new class data D c i − 1 +1:

Original: c i to obtain the buffer memory storing the unlabeled data batch U and P .
Revised: c i to obtain the buffer memory storing the unlabeled data batch U and P .

Original: Row 3 “non-budget” means we are allowed to use a little additional memory to save the unlabelled data U and P without deleting other samples.
Revised: Row 3 “non-budget” means we are allowed to use a little additional memory to save the unlabelled data U and P without deleting other samples.

Original: Comparing Row 3 to Row 2, we can see that the performance of our PlaceboCIL only reduces 0 .
Revised: Comparing Row 3 to Row 2, we can see that the performance of our PlaceboCIL only reduces 0 .

Original: 04 percentage points on average under the strict “budget” setting.
Revised: 04 percentage points on average under the strict “budget” setting.

Original: The reason is the amount of new data is greatly larger than that of old samples, and removing a small batch of new data does not harm the performance much, as observed in (Liu et al., 2021b).
Revised: The reason is the amount of new data is greatly larger than that of old samples, and removing a small batch of new data does not harm the performance much, as observed in (Liu et al., 2021b).

Original: [Table]
Revised: [Table]

Original: H E QUAL - SIZE S PLIT R ESULTS
Revised: H E QUAL - SIZE S PLIT R ESULTS

Original: This is supplementary to Section 4 “ Ablation study .”
Revised: This is supplementary to Section 4 “ Ablation study .”

Original: In Table A
Revised: In Table A

Original: [Table]
Revised: [Table]

Original: we supplement the results using the equal-size split on CIFAR 100, 10 -phase ( 10 classes/phase, 20 exemplars/class).
Revised: we supplement the results using the equal-size split on CIFAR 100, 10 -phase ( 10 classes/phase, 20 exemplars/class).

Original: We can observe that using placebos selected by heuristic evaluation functions ( β i = γ
Revised: We can observe that using placebos selected by heuristic evaluation functions ( β i = γ

Original: i = 1 ) consistently improves the results on three baselines.
Revised: i = 1 ) consistently improves the results on three baselines.

Original: Using the online learning algorithm can further boost performance.
Revised: Using the online learning algorithm can further boost performance.

Original: The observations are similar to using the original setting in the main paper
Revised: The observations are similar to using the original setting in the main paper.

Original: [Table]

Original: equal-size split on CIFAR-100, 10 -phase ( 10 classes/phase).

Original: I A BLATION R ESULTS FOR D IFFERENT U NLABELED D ATA S OURCES
Revised: I A BLATION R ESULTS FOR D IFFERENT U NLABELED D ATA S OURCES

Original: This is supplementary to Section 4 “ Ablation study .”
Revised: This is supplementary to Section 4 “ Ablation study .”

Original: We believe that the key to success is the design of our method instead of the choice of unlabeled data sources.
Revised: We believe that the key to success is the design of our method instead of the choice of unlabeled data sources.

Original: To verify this, we provide  the results for a new ablative setting: “using random unlabeled data” (to compute the distillation loss) in Table A4.
Revised: To verify this, we provide [Table] the results for a new ablative setting: “using random unlabeled data” (to compute the distillation loss) in Table A4.

Original: We can observe that no matter what unlabeled data sources we use, our method consistently performs better than using random unlabeled data.
Revised: We can observe that no matter what unlabeled data sources we use, our method consistently performs better than using random unlabeled data

Original: [Table]
Revised: [Table]

Revised: denotes using all data from ImageNet, and “overlapping” means including samples from the overlapping classes between CIFAR-100 and ImageNet.

Revised: J R EHEARSAL - FREE EXPERIMENTS

Revised: This is supplementary to Section 4 “ Ablation study .”

Revised: In Table A5, we provide the “rehearsal-free” results for two variants of our method: 1) PlacboCIL (original) w/o exemplars: we don’t use any exemplars to compute the loss, only storing one exemplar each class to compute the evaluation function (Eq. 4); 2) PlacboCIL (higher confident) w/o exemplars: we don’t use any exemplars, and select the unlabeled data with higher confidence as the placebos.

Revised: We can observe that using placebos is effective in the “rehearsal-free” setting.

Revised: The original PlaceboCIL improves the average accuracy of the baseline by 6 . 30 percentage points.

Revised: The “higher confident” version performs a little worse than the original, but it still improves the average accuracy of the baseline by 3 .

Revised: 40 percentage points.

Revised: [Table]