Prediction Error-based Classification for Class-Incremental Learning
Authors: Michał Zając
Tinne Tuytelaars
Gido M van de Ven
Venue: ICLR 2024 poster (2024)
Keywords: continual learning
class-incremental learning
Primary area: transfer learning, meta learning, and lifelong learning
TL;DR: We introduce Prediction Error-based Classification, an approach for class-incremental learning that differs from traditional discriminative and generative paradigms, and demonstrate its strong performance.

Abstract: Class-incremental learning (CIL) is a particularly challenging variant of continual learning, where the goal is to learn to discriminate between all classes presented in an incremental fashion. Existing approaches often suffer from excessive forgetting and imbalance of the scores assigned to classes that have not been seen together during training. In this study, we introduce a novel approach, Prediction Error-based Classification (PEC), which differs from traditional discriminative and generative classification paradigms. PEC computes a class score by measuring the prediction error of a model trained to replicate the outputs of a frozen random neural network on data from that class. The method can be interpreted as approximating a classification rule based on Gaussian Process posterior variance. PEC offers several practical advantages, including sample efficiency, ease of tuning, and effectiveness even when data are presented one class at a time. Our empirical results show that PEC performs strongly in single-pass-through-data CIL, outperforming other rehearsal-free baselines in all cases and rehearsal-based methods with moderate replay buffer size in most cases across multiple benchmarks.

Published as a conference paper at ICLR 2024
PREDICTION ERROR-BASED CLASSIFICATION FOR CLASS-INCREMENTAL LEARNING
Michal Zajac KU Leuven, ESAT-PSI Jagiellonian University
Tinne Tuytelaars KU Leuven, ESAT-PSI
Gido M. van de Ven KU Leuven, ESAT-PSI
ABSTRACT
Class-incremental learning (CIL) is a particularly challenging variant of continual learning, where the goal learn to discriminate between all classes presented in an incremental fashion. Existing approaches often suffer from excessive forgetting and imbalance of the scores assigned to classes that have not been seen together dur- ing training. In this study, we introduce novel approach, Prediction Error-based Classification (PEC), which differs from traditional discriminative and generative classification paradigms. PEC computes a class score by measuring the prediction error of model trained to replicate the outputs of a frozee random neural network on data from that class. The method can be interpreted as approximating clas- sification rule based on Gaussian Process posterior variance. PEC offers several practical advantages, including sample efficiency, ease of tuning. and effectiveness even when data are presented one class time. Our empirical results show that PEC performs strongly in single-pass saeec-ppsssttotuss mass-through-date CIL, outperforming other rehearsal baselines in all cases and rehearsal-base methods with moderate replay buffer size in most cases across multiple benchmarks
INTRODUCTION
Continual learning addresses the problem of incrementally training machine learning models when the data distribution is changing (Parisi et al. 2019 De Lange al., 2022 van de Ven et al. (002)). Humans excel learning incrementally from real-world data that are often noisy and non-stationary:
embedding similar continual learning capability in machine learning algorithms could make them more robust and efficient (Hadsell et al. 2020 Kudithipudi al. 2022). A particularly challenging
variant of continual learning 18 class- incremenall learning (CIL) (Rebuth et al.] [017]; [Blouadahh
(et al. 2021: van de Ven et al. 2022 Masana et al. 2023), ) which involves jointly discriminating between all classes encountered during sequence of classification tasks.
The most common approach for classification, particularly in deep learning, is to learn discrim- inative classifier, as shown Figure Tal However, in continual learning scenarios, issues such as forgetting relevant features of past classes arise. To address this, discriminative classification can be supplemented with continual learning techniques such as replay (Chaudhry al. 2019b Lopez-Paz Paz & Ranzato 2017) or regularization (Kirkpatrick et al. 2017 Zenke et al., 2017). Nonetheless. these methods typically perform worse with class-
learning than with insseminreeentall
learning (van de Ven et al.] 2022). and they often suffer from imbalances between classes that have not been presented together during training (Wu al., 2019). Recently, an alternative paradigm of generative classification (see Figure Tb) has been explored for CIL, with promising results (van de
Ven et al. 2021 Banayecanzade et al. 1021). This approach involves training a likelihood- based generative model for each class, and using class cansiconniiional likelihoods scores for classification during inference. However, learning decent generative models difficult, especially for complex distributions and with limited training data. In this paper, we put forward hypothesis that the generative modeling task is not necessary and that it is possible to obtain accurate scores from class-specific modules in more efficient ways by forgoing the generative capability.
As a realization of this idea, we propose Prediction Error-based Classification (PEC), depicted in Figure Ic In this new approach to classification, we replace the generative modeling objective with much simpler one, defined by frozen random neural network. More precisely, for each class c, we train neural network gø= exclusively on examples from that class, to replicate the outputs of a frozen target random neural network h. During testing, the prediction error go= on an example x is utilized to generate class score for class c, analogous to logits or likelihoods, used in discriminative and generative classification, respectively.
We release source code for our experiments:
https:://gttubb.commm cha a za jac-ml m1 /pac

---

Published as a conference paper at ICLR 2024
This approach is inspired by Random Network Distillation (Burda et al. 2018), technique used for novelty detection in reinforcement learning. PEC can be viewed as approximation of a clas- sification rule based on Gaussian Process poste- rior variance (Ciosek et al,, 2020), which makes it well-motivated and connected to existing theory. PEC also offers several important practical ad- vantages as method for CIL. Firstly, it is highly sample efficient, as demonstrated by its strong performance with a limited number of iterations. Secondly, is simple and easy-to-tune approach that does not introduce any extra hyperparameters. Thirdly, the absence of forgetting is guaranteed by the separation of modules for different classes. Lastly, PEC can operate even when the data are presented only one class at time.
g(*)
logit g(t)22
ciscsifier
seg max
logit g(=)a
logit c
(a) In discriminative classification, an example is processed by common classifier network, and the class with the largest logit is chosen.
gen. model d1.
Ikalhood go (1()))
gan. model da
Ikeihooo (a(2)
arg
gen. model Ikelhood (a
Since PEC was designed with efficiency in mind, we focus in our evaluations on the challeng- ing saagle-paas-througee
gen. model Ikaihoad go(a)
or online, setting (Chaudhry et al. 2019a b. Aljundi et al., 2019a Soutif-Cormerass al. 2023). This setup em- phasizes being able to learn from a stream of data, with limited samples, memory, and com- pute (Chaudhry et al. (2019)). We conduct com- prehensive experiments on CIL sequences utiliz- ing the MNIST. SVHN, CIFAR-10, CIFAR-100, and minilmageNet datasets.
(b) In generative claxsification, the likelihood of a is evaluated under each class-specific generative model, and the class with the highest one is chosen.
shadent
[9x( A(aa]]
stladent
(9x(=)) M(=a]l
(randomm)
hoacher
ang min
Our contributions include:
student
A(=))l
We introduce Prediction Error-based Classi- fication (PEC), novel approach for class- incremental learning. PEC is motivated as an efficient alternative for generative classifica- tion, and can additionally be interpreted approximation of a rule based on Gaussian Process posterior variance.
student
(oc(#) M[*)]I
(c) In Prediction Error-hased Classification, x is passed through class-specific student networks and shared teacher network, which is fixed with ran- domly initialized weights. The class whose student approximates the teacher most closely is chosen.
Figure I: Comparison of discriminative classification, generative classification, and Prediction Barrorbased Classification.
We demonstrate the strong performance PEC in single-pase throughhhogh CIL. Our method outperforms other rehearsal-free base- lines in all cases, and rehearsal -aased approaches with moderate replay buffer size in most cases.
We provide additional experiments analyzing the erucial factors for PEC's strong performance.
PRELIMINARIES
2.1 CLASS-INCREMENTAL LEARNING
Continual learning concerned with training machine learning models in an incremental man- ner, where the training data are not i.i.d. but their distribution can shift during the learning pro- cess (De Lange et al. 2022 Parisi et al. 2019). Depending on assumptions such the type of non-stationarity and access task identity, different types of continual learning have been distin- guished (van de Ven et al. 2022). Arguably, among the most challenging of these is class-incrementt
learning (CIL) (Masana et al. 2023 Zhou et al. 2023 van de Ven et al. 2022).
For simplicity as well to be comparable to previous work, in evaluations we focus task-based Mlas--nncrementa learning, where the learning process is divided into clearly separated tasks that do not repeat (Aljundi et al. 2019bb van de Ven et al.) (022). However, it is important to note that PEC is not limited to this setting, as it does not make use of this task structure. In task-based CIL, an algorithm is presented with a sequence of classification tasks 7i, Te...... .., T,. dataset D' :- {(x), bi is presented, where each y belongs a set of possible classes C*;, At test time, the algorithm must decide to which class a given input belongs, among all possible classes U1. CT. Task identity is not provided during testing, meaning that the algorithm must dis- criminate not only between classes that have been seen together but also between those that have not.
During each task T7,
2

---

Published as a conference paper at ICLR 2024
2.2 APPROACHES TO CLASS -LAAS--ICRREENTAL LEARNING
Most approaches to CIL rely on discriminative classification (see Figure Ta together with some mechanism to prevent forgetting. Among those, some methods use a memory buffer in which data from past tasks are stored, while others refrain from it. As storing data might be forbidden in some protocols, it often considered a desirable property for CIL methods to not rely on a memory buffer.
When stored data from past tasks are available, the typical approach is to revisit that data when training new data. For instance. Experience Replay (ER) (Chaudhry et al.. 2019b) directly retrains on the buffer data, while A-GEM (Chaudhry et al.) 2019a) uses the stored data for gradient projections to reduce interference between tasks. Dark Experience Replay (DER) (Buzzega et al. 2020) is a form of ER that combines replay with self-distillation. Further variants of DER include DER+ and X-DER (Boschini et al. 2022). ER-ACE (Caceia al. 2021) modifies the loss to only affect the scores corresponding to current classes on non-rehearsal data. iCaRL, (Rebuffi et al. 201 combines replay with a nearest-ment -arresstt--oooo--- of- exxempllass classifier in the feature space, BIC (Wu et al. 2019) additionally uses stored data to calibrate logits for classes coming from different tasks, and thus can be classified as a bias-correcting algorithm.
Among methods that do not store data, sometimes referred to as rehearsal-free there are regularization-bee baaiiioonbbabddd approaches that add an auxiliary term to the loss function, discouraging large changes to parts of the model important for past tasks. EWC (Kirkpatrick al. 2017) relies diagonal approximation of the Fisher matrix to weigh the contributions from different parameters. SI (Zenke et al. (017) maintains and updates per-parameter importance measures in an online manner. LwF (Li & Hoiem (017) uses distillation loss to keep predictions consistent with the ones from historical model. Additionally, there are rehearsal-free bias-correcting algorithms such as the Labels trick (Zeno et al., 2018). which only incorporates classes from the current task in the loss function to prevent negative bias on past tasks. Since the Labels trick often improves the performance when added top of rehearsal-free methods, we also consider EWC + LT, SI LT, LWF LT variants.
Another paradigm for CIL,, which also does not require a memory buffer, generative classification, where class-conditional generative models are leveraged directly for classification, see Figure Tb For example, in an approach described in van de Ven et al. (202T). ). a separate variational autoencoder (VAE) (Kingma & Welling, 2013) is trained for every class; we refer this method as VAE-GC. However, when only a single pass through the data is allowed, VAE models might be too complex. and generative classifiers based on simpler models might be preferred. One option is SLDA (Hayes) & Kanan 2020), which can be interpreted
learning Gaussian distribution for each class with shared covariance matrix (van de Ven et al. (02TT) This can simplified further by using an identity covariance matrix, in which case it becomes the Nearest mean classifier.
3 METHOD
3.1 PEC ALGORITHM
In this section, we describe Prediction Error-based Classification (PEC), which is schematically illustrated in Figure Ic Inspired by Random Network Distillation (Burda et al. (018)), our approach entails creating a distinct student neural network gø= for each class c, which is trained to replicate the outputs of a frozen random teacher network hs R" + Rd for data from that class. Here, n and d represent the input and output dimensions, respectively, and the output dimension can be chosen arbitrarily. The intuition is that after training, ga- will be competent at mimicking ftp for the inputs x that are either from the training inputs set Xx. or close to the examples from X We leverage here the generalization property of neural networks (Kawaguchi et al. 2017 Goodfellow jet al.] 2016), which tend to exhibit low error not only on training data but also on unseen test data from the same distribution. If, on the other hand, some out-of colloo--ditributton input 2 is considered, then the error [|9e- (ū) (z) is unlikely to be small (Ciosek et al. 2020). the network was not trained in that region. To classify using PEC, we feed the input x through all networks gø- corresponding to different classes and select the class with the smallest error || go- [g)(x) -h(())l. The training and inference procedures are outlined in Algorithm [nd Algorithm [ respectively.
As the architectures for g and h, use shallow neural networks with one hidden layer, either convolutional or linear, followed by respectively a ReLU or GELU nonlinearity, and then a final linear layer. Additionally, the teacher network's 's middle layer is typically wider than the student's one. A detailed description of the used architectures is deferred to Appendix B. Note that the student networks from different classes can be implemented as a single module in an automatic differentiation framework, jointly computing all class-specific outputs. This means that during inference only two forward passes are needed to obtain the classification scores one for such a merged student, and one for the teacher network. We use the Kaiming initialization (He al. 2015), the default in PyTorch,
3

---

Published as a conference paper at ICLR 2024
Algorithm 1 Training of PEC
input:
Datasets X and student networks go- for classes {1,2, .... C). teacher network he. number of optimization steps S
output: Trained student networks gos, randomly intialized teacher network hep
Initialize 01, 0²,02... _....... 8 with the same random values ...., Initialize & with random values
for class E (1, 2, ...; C) do
for iteration E (1.2..... S) do
x: f batch of examples from X Update 0 to minimize I|go= (2) - (x))11
end for end for
Algorithm 2 Inference using PEC
input:
Example z, student networks goo for classes E (1, C), teacher network hap output: Predicted class Cp return Cp == arg (11.2... I1go (x) ha(x))l
to initialize both the teacher and student networks. We ablate this choice in Appendix C.7 and find that PEC's performance is relatively stable with respect to the initialization scheme.
This classification paradigm is well-suited for class-incremental learning since each class has its own dedicated model, eliminating interference and preventing forgetting. Furthermore, the performance of PEC does not depend on task boundaries (e.g., blurry task boundaries are no problem) and the approach works well even if each task contains only one class. This unlike most discriminative classification methods, which struggle in such setting since they operate by contrasting classes against one another, in consequence requiring more than one class. Moreover, as PEC works well with a batch size of 1, it is also well suited for streaming settings. Additionally, PEC has the advantage of not introducing any new hyperparameters beyond architecture and learning rate, which are also present in all considered baselines.
PEC can be motivated from two different angles. One perspective is that PEC related to the gener- ative classification paradigm, but it replaces the difficult generative task with a much simpler, random task. As will be shown in the experimental section, this leads to significant efficiency improvements. Alternatively, PEC can be seen as an approximation of a principled classification rule based on the posterior variance of a Gaussian Process. We delve into this perspective in the following section.
3.2 THFORETICAL SUPPORT FOR PEC
In this section, we first explain the classification rule based on Gaussian Process posterior variance, and then we describe how this rule is approximated by PEC.
Gaussian Processes Gaussian Process (GP) is a stochastic process, i.c. collection of random variables {H)} indexed by values x E R for some n, such that every finite sub-collection of the
random variables follows a Gaussian distribution (Rasmussen & Williams 2006). Typically, GP is defined in terms of a mean function f
RR"" n -> I.. and covariance function k R R IR.
Given some dataset (X, y). a given prior GP H can be updated to a posterior GP H (x. Let us
denote by o o(H)) the posterior variance of H(x,y) at point x². A useful property of a (xx (a*)
is that it depends only on the inputs X and not the targets y (Rasmussen & Williams! 2006). The
posterior variance of GP is often interpreted as measure of uncertainty (Ciosek et al. 2020).
Classification rule based on GP posterior variance Let H be a Gaussian Process. Let us assume distributions Xy...., &' for the data from C different classes, and finite datasets X, XI....,Xc Xx Xc. Given some test data point x², consider the following classification rule:
class (ass(x") arg min o (H*.
(1)
13e5C
Let us informally motivate this rule. Assume that belongs to the class ē, for some < E C C. Then, in the dataset X5 for the class ē, there are many examples close to z², which in consequence make the posterior variance
xx (x*) close to 0. In contrast, for any c + ē, examples from X. are more distant from x², which means that the reduction from the prior variance o (H) (a*) to the posterior
variance a (Hx.)(x") (x=)) is smaller. Hence, it is intuitively clear that the rule in Equation 1 should
typically choose the correct class In Appendix D. we provide formal support for this argument.
4

---

Published as a conference paper at ICLR 2024
PEC approximates GP posterior variance classification rule Here, argue that PEC approxi- mates the GP posterior variance classification rule, using tools from Ciosek et al. (2020). For a given GP H, a dataset X, and a data point x², define:
H.X B
(x")
= B Σ
1
H:
(2)
B
(x*) = En-H (l00.l((')))) (x 1x2")11
where B is the number of samples from GP H and gh. is a neural network trained to imitate H.X "..(("") (x"), for B E 2+ U {oo), measures the
on dataset X, using some fixed procedure. That is, $B average prediction error at x¹ of neural network trained with dataset X to approximate a function sampled from H. Now, let us introduce GP-PEC(B), modified version of PEC that we will use in our analyses. Given classes I c C' with datasets X1,..., Xc, GP-PEC(B), for B E Z, U {oo), classifies using the following rule:
class(x*) == arg min $B" IseSC
H.X. (xx
We can connect PEC to GP-PEC(B) by using a sample size and by replacing the GP "teacher" H with a neural network teacher h. Note that our theory does not quantify errors arising from this replacement; however, the replacement is justified since wide neural networks with random initializations approximate Gaussian Processes (Neal 1996; Yang 2019). our experiments, we use a wide hidden layer for the teacher network, a choice that we ablate in Section 4.5)
Next, we introduce propositions connecting GP-PEC(B) with the GP posterior variance classification rule. The proofs for these propositions are provided in Appendix D
Proposition 1 (Proposition from Ciosek et al. (2020), rephrased). Let H be Gaussian Process and X dataset. Then the following inequality holds:
H.X (x") 2 (HHx)(xx)) 5:0
(5)
Proposition 2. Let H be a Gaussian Process, X be a data distribution, and X := [x1,, ".1.....,,N'.. AN) ~ X finite dataset. Assume that class of approximators is such that for any h ~ H, h can be approximated by g so that, for arbitrarily large training sets, the training error is smaller than >> O with probability at least - 7 0. Then, under technical assumptions, for any 5 >> 0, holds with probability at least 8 that
H.X (1x) SEtKN. E ~ 1
(6)
where KN goes to 0 as N co.
Proposition T states that the scores used by GP-PEC((O)) overestimate scores from the GP posterior variance rule. On the other hand, Proposition 2 shows that on average, the score assigned by GP- PEC(1) for the correct class will go 0 as N oc, given appropriately strong approximators and technical assumptions listed in Appendix D Together, these propositions indicate that GP)PEC(B), and by extension PEC, approximate the GP posterior variance classification rule in Equation |
EXPERIMENTS
We present an experimental evaluation of PEC on several dass-incremmnnal learning benchmarks. Our main result is that PEC performs strongly in pangle-pss-through-e
CIL, outperforming other rehearsal-free methods in all cases and rehearsal-based methods with moderate replay buffer size in most cases. This section structured as follows: first, we explain the experimental setup in Section [.]] Then, we present the benchmarking results, with Section [4.2 focusing on the single-pass- through-data setting, and Section 4.3 covering a broader spectrum of regimes. In Section 4.4 we examine the comparability of PEC scores for different classes, Finally, Section 4.5 investigates the impact of various architecture choices on PEC's performance.
4.1 EXPERIMENTAL SETUP
All experiments follow the class-incremental learning scenario, and, unless otherwise noted, for each benchmark we only allow single pass through the data, i.c. each training sample seen only once except if it is stored in the replay buffer. Performance is evaluated as the final test accuracy after training on all classes. We adopt Adam (Kingma & Ba 2014) as the optimizer. We provide additional details about the experimental setup in Appendix
5

---

Published as a conference paper at ICLR 2024
Table I: Empirical evaluation of PEC against suite of CII. haselines in case of one class per task. The
mungle-ass-through-ee setting used. Final average accuracy (in %) is reported, with mean and standard error
from 10 seeds. Parameter counts for all methods are matched, except for those indicated with see Section 4.1
Method
Buffer
MNIST
Balanced SVIIN
CIFAR-10
CIFAR-100
minilmageNee (100/1)
(10/1)
(10/1)
(10/1)
(100/1)
Fine-tuning
10.09 (# 0.00)
10.00 (#: 0.00)
9.99 (#: 0.01)
1.09 (#: 0.04)
1.02 (#: 0.02)
Joint, 1 epoch Joint, 50 epochs
97.16 (== 0.04)
92.69 (#: 0.08)
74.16 (:: 0.13)
34.87 (d: 0.17)
25.29 0.15)
98.26 (= 0.03)
95.88 (# 0.03)
93.30 (# 0.08)
73.68 (# 0.08)
72.26 (# 0.12)
ER
84.42 (+: 0.26)
74.80 (+ 0.69)
(+
5.14 0.21)
5.72 (+ 0.24)
A-GEM
V
59.84 (#: 0.82)
9.93 0.00)
(0.19
1.05 (+ 0.03)
1.11
DER
91.65 (+: 0.11)
76.45 (+: 0.71)
40.03 1.52)
0.07)
0.99 0.01)
DER++
91.88 (#= 0.19)
80.66 (+: 0.48)
35.61 2.43)
(4
1-40
X-DER
82.97 (= 0.14)
70.65 (+: 0.61)
43.16
15.57 (+
8.22
ER-ACE
87.78 (== 0.19)
74.47 (# 0.33)
8.21 (1 0121)
5.73
iCaRL
83.08 (= 0.28)
57.18 (# 0.94)
37.80
8.61 0+.13)
7.52
BiC
86.00 (#: 0.37)
70.69 (1: 0.52)
35.86
(4
1,46 0.05)
Labels trick (LT)
10.93 (= 0.87)
9.93 0.088
(+
1.04 0.05)
0.97 0.04)
EWC +LT LT
9.20 ( 0.91)
9,90 0.09)
(1
101 (1 0.05)
1.03 0.03)
SI+LT
10.75 (= 1.03)
10.01 0.06)
10.01 0.26)
0.93 0.05)
0,94 0.04)
LWF+LT
9.35 (+ 0.72)
9.99 (+ 0.07)
9.57 (+ 0.52)
1.00 (+ 0.09)
0.97 (+ 0.02)
gelenttve
Nearest mean'
82.03 (+ 0.00)
12.46 (+ 0.00)
27.70 0.00)
9.97 (+ 0.00)
7,53 (+ 0.00)
SLDA
88.01 (+: 0.00)
21.30 (t: 0.00)
41.37 (1: 0.01)
18.83 (t 0.01)
12.87 0.01)
VAE-GC
83.98 (# 0.47)
50.71 (+ 0.62)
42.71 (# 1,28)
19.70 (+ 0.12)
12.10 (# 0.07)
PEC (ours)
92.31 (+: 0.13)
68.70 (+: 0.16)
58.94 (+:: 0.09)
26.54 (+: 0.11)
14.90 (t: 0.08)
Tasks and datasets To construct sequences of tasks, we split up popular image classification datasets such that each task contains data from subset of classes and those subsets are disjoint.
Following Masana et al. (2023). if dataset D is split into T tasks with C classes each, the resulting sequence is denoted DT) For each dataset, we consider two splits: a standard one, with multiple classes in each task, and more challenging one, with only one class per task. For MNIST (Deng 2012), Balanced SVHN (Netzer et al. 2011), and CIFAR-10 (Krizhevsky et al. 2009). ) we use [5/2] and (10/1) splits. For CIFAR- 100 (Krizhessky et al. 2009). wC use (10/10) and (100/1) splits. For minilmageNet (Vinyals et al. 2016), we use (2075) and (100/1) splits.
Methods We compare PEC against a comprehensive list of CIL baselines, see Section 2.2 Ad- ditionally, we include Fine-tuning, which trains a discriminative classifier on a sequence of tasks without employing any continual learning strategy, and Joint, which can be seen as an upper target that jointly trains all the classes the same time. For the non-rehearsal approaches, variants with added Labels trick in the main text; we provide vanilla results in Appendix C.2
report the
Architectures Total parameter counts used by models for each method are matched, except for Nearest mean and S For methods based on discriminative classification, we use standard architectures: for MNIST, an MLP with two hidden layers, 100 neurons each; for other datasets, variant of ResNet18 (He et al. (016) with the initial downscaling removed, as in [uzzzega et al.) (2020) For VAE-GC and PEC, which maintain separate modules per class, we use small and shallow neural networks to match their total parameter count with other approaches; in particular, we do not use ResNets for those methods. We describe details in Appendix B.1
Replay buffers Although PEC does not a replay buffer, we also compare against rehearsal- baselines. We grant these a replay buffer of 500 examples (Buzzega et al. 2020 Boschini et al.) 2022). This is additional memory that these methods can use; the rehearsal- -reee methods are not compensated for this. For each batch of data from the current task containing b examples, a rehearsal-basee method can access the replay buffer and use b extra examples from it.
Hyperparameters For each combination of method, dataset, and task split, we perform a separate grid search to select the learning rate, the batch size, whether to use data augmentation, whether to use linear learning rate decay, and method- -deendded hyperparameters. Note that PEC does not introduce the latter. Details on the selected values and search grids are deferred to Appendix B.3
'For the main experiments (Tables I anddd 2). we use Balanced SVHN, a version of SVHN with equalized num- bers of samples per class. We do this to isolate the issue of data imbalance, discussed separately in Section 4.4
3 "As these methods rely on the estimation of data means and covariances, it is not directly possible manipulate the number of parameters used. We report numbers of parameters for all the methods in Appendix H.2
6

---

Published as a conference paper at ICLR 2024
Table 2: Empirical evaluation of PEC against suite of CII. baselines in case of multiple classes per task. The pangle-pas-through--aa setting is used. Final average accuracy (in %) is reported, with mean and standard error from 10 seeds. For joint training and generative classifiers, results are the same as with one class per lask (see [ and omitted. Results for PEC are also the same, but included to ease comparison. Appendix B.2
explains why these results are the same. Parameter counts for all methods are matched.
Method
Buffer
MNIST
Balanced SVHN
CI
CIFAR-100 (10/10)
miniImageNet (20/5)
(5/2)
(5/2)
(5/2)
Fine-tuning
19.57 (#= 0.01)
19.14 (# 0.03)
18.54 (:: 0.17)
6.25 (#: 0.08)
3.51 0.04)
ER
86.55 (+: 0.17)
80.00 (+: 0.54)
$7.49 (+: 0.54)
11.41 0.22)
6.67 0.21)
A-GEM
50.82 (+ 1.36)
19.71 (#: 0.49)
18.21 0.14)
5.25 0.14)
2.47 (= 0.18)
DER
91.64 (# 0.11)
78.22 (#: 0.94)
43.98 (+: 1.20)
7.39 0.06)
3.86 (#: 0.04)
DER++
92.30 (== 0.14)
86.50 0.43)
50.61
0.80)
19.99 (1: 0.28)
10.50 0.42)
dactiiiniis
X-DER
V
81.91 (= 0.53)
73.88 (+: 1.59)
(4.15
28.64 (+: 0.22)
14.65 (= 0.40)
ER-ACE
89,97 (+: 0.29)
83,30 (+: 0.33)
(1.25
21.02 0.32)
8,95 0.35)
iCaRL
80.41 (= 0.45)
73.65 0.57)
54.33
2+
13.46 (# 0.20)
BiC
88.08 0.81)
77.65 0.45)
26.19 (6.
6.90 0.19)
Labels trick (LT)
38.34 (= 2.46)
31.86 0.81)
7,84 (4:40000) 0.15)
4.44 0.08)
EWC + LT
46.41 (1 1.04)
26.04 (1 1.56)
8.64 0.226)
3,64 0.16)
ST+LT
52.45 (== 1.84)
30.81 (+= 0.86)
25
8.45 0.144)
4.76 0.06)
LWF+LT
55.38 (= 1.27)
52.21 (+ 1.22)
40.40 0.91)
14.38 (+ 0.17)
3.74 (= 0.33)
PEC (ours)
92.31 (= 0.13)
68.70 (#: 0.16)
58.94 0.09)
26.54 0.11)
14.90 0.08)
4.2 PERFORMANCE OF PEC
Here present the main benchmarking results, Evaluations for the cases of one class per task and multiple classes per task are presented in Table I and Table respectively, PEC performs strongly in both cases, achieving the highest score in 7 out of 10 considered settings.
The performance of PEC is especially good in the challenging case of one class per task, see Table I] Discriminative classification-base approaches struggle in this setting, whereas PEC does not rely on the task structure and achieves the same results for both task splits. On most datasets, PEC substantially outperforms even recently proposed replay-based methods with buffers of 500 examples.
When only comparing against rehearsal-free free methods, PEC achieves the strongest performance in all considered settings. In particular, we see that PEC outperforms VAE-GC, strong rehearsal-free baseline, as well as the other included generative classifiers. Below we also report that PEC reaches a given performance in smaller number of iterations and with less model parameters compared to VAE-GC, see Figure 2 This fulfills our original goal to develop method inspired by generative classification but more efficient. We provide additional experiments in Appendix C.T. where we use the experimental setups from van de Ven et al. (202T), including multi-epoch learning and parameter counts more suitable for VAE-GC, as well as the usage of pre-trained feature extractors in some cases. We show that PEC prevails also in those setups, which further corroborates its robust performance.
Performance on MNIST
Performance on
90-
50-
85-
85
82-
80
0.1
0.33
10
0.03
0.10
0.30
1.50
Number epachs
Number of parameters
Parformance CIFAR-10
Performance on CIFAR-10
Final accuracy On 407
00
30
40-
30
(161) 10
15:2)
0.33
0.01
0:10
0.30
2.00
Number epochs
Number of parameters
Figure 2: CIL performance for varying number epochs (left) and number of model parameters (right). Plotted are the means (solid or dashed lines) and standard errors (shaded areas) from 10 seeds. For PEC and VAE-GC. results are the same for both task splits (sce Appendix [B.2), and henee single curves are shown. Vertical dotted lines indicate the settings used for the experiments in Tables and
7

---

Published as a conference paper at ICLR 2024
4.3 PERFORMANCE IN VARYING REGIMES
In this section, we present comparison of the performance of representative set of approaches in a wider range of experimental conditions. Specifically, we evaluate ER, DER++ VAE-GC, and PEC while varying two factors: the number of epochs (Figure 2 left) and the number of model parameters (Figure 2 right). We keep the rest of the experimental setup the same as before. We use the hyper- parameters as in Section 4.2 except for the learning rates, which we tune for every setting separately.
We observe good performance of PEC across the board, confirming its robustness to varying exper- imental conditions. PEC outperforms ER and VAE-GC in all considered cases, and DER++ in most cases. Increasing the number of epochs generally benefits all methods, especially for CIFAR-10. Con- versely, scaling the number of parameters beyond the levels used in the main experiments improves the performance of PEC and VAE-GC, but less so for the discriminative approaches, ER and DER++
4.4 COMPARABILITY OF PEC CLASS SCORES
In this section, investigate whether the magnitudes of the scores computed by PEC for different classes are compara- ble. In Section 3.2, we pro- vided support for the use of PEC scores in the limit of data and ensemble members. How- ever, in practice, we use a limited dataset and a single student-teacher pair. Thus, is not clear a priori whether the obtained scores for different classes would be comparable out of the box.
Table 3: Performance of PEC under balanced and imbalanced data, well with different mitigation strategies. Reported is the final average accuracy (in %6), with mean and standard error from 10 seeds.
Method
CIFAR-10
Imbalanced CIFAR-10
SVHN
PEC
58.94 (+: 0.09) 58.94 (# 0.09)
28.96 0.21)
21.73 (+ 0.14)
PEC, Equal budgets
55.84 (# 0.14)
68.65 (# 0.21)
PEC, Oracle balancing
61.01 (#: 0.11)
59.13 (+: 0.47)
68.55 0.09) 66.82 (+ 0.25)
PEC, Buffer halancing
59.02 (+ 0.25)
57.31 (+ 0.60)
To address this question, we examine the performance of PEC on the CIFAR- 10 dataset can be improved by re-scaling the obtained class scores using per-class scalars. For this purpose, we do the following. After the training of PFC,
the CMA-ES (Hansen & Ostermeier 2001 evolutionary optimization algorithm (see Appendix B.4 for details) to find a scalar coefficient 8. for each class e to scale its class score; then, sel190- (x) h (x)]] is used as score for class c. We consider two procedures. Oracle balancing uses the full test set to find the optimal & coefficients, and Buffer balancing uses the replay buffer of 500 samples from the training set to do so. We observe in Table 3 that Oracle balancing is able to improve upon the baseline version of PEC, although not by a wide margin, while Buffer balancing does not bring a significant improvement. This suggests good out-of-the-boo comparability of the PEC class scores for a balanced dataset like CIFAR- 10.
However, in initial experiments we found that an imbalanced training dataset, where classes have different numbers of samples, can severely impair the performance of our method. To study the impact of dataset imbalance, we compare the performance of PEC on CIFAR- and Imbalanced CIFAR- 10. The latter is a modified version of CIFAR-10, where the number of examples per class is doubled for class O and halved for class 1, causing dataset imbalance; the test dataset stays the same. We also study SVHN, which is imbalanced in its original form; in Section 4.2 we reported results for the balanced variant. We observe that in the case of imbalanced datasets, the performance of vanilla PEC drops substantially compared to balanced counterparts, see Table 3] At the same time, we find that this deterioration is mostly mitigated by the balancing strategies that were introduced above, which suggests that dataset imbalance affects the comparability of the PEC class scores.
Note that Buffer balancing is a valid strategy that could be used in practice if the replay buffer is al- lowed. For the setting without buffers, we additionally propose Equal budgets, which is a simple strat- egy that equalizes the numbers of training iterations for different classes, Appendix BB44 details. We observe that Equal budgets also effectively mitigates the effect of dataset imbalance, see Table 3
4.5 IMPACT OF ARCUITECTURAL CHOICES
In this section, we examine various architectural choices for PEC and their impact on performance. Jshows our analysis of the following factors: number of hidden layers (for both the student and the Teacher network), width of the hidden layer of the student network, width of the hidden layer of the teacher network, and output dimension. For every considered setting, the learning rate is tuned.
We observe that increased depth does not seem to enhance PEC's performance. On the other hand, scaling up the other considered variables leads to consistent improvements for both MNIST and CIFAR- 10; note that these correspond to different ways of scaling width. We see that the output dimensions and the width of the teacher's 's hidden layer seem particularly important. The impact of the latter can be understood through the lens of the ba-ba classification rule, where the teacher is
8

---

Published as a conference paper at ICLR 2024
Performance MNIST
Performance MNIST
Performance on MNIST
Perforanne MNIST
90
50
90
90
Rial eccurasy
80-
80
80
90-
70-
70
70
70-
60
60
60
60
100
5000
300
layers
Student
Oulput
Performance CIFAR-10
Performance CIFAR-10
Performance CIFAR-10
Performance en OFAR 10
accurscy 90)
8
58
$
56-
56-
54
54-
52
52
6080
60000
100
3.00
743
1500
Hidden layers
Oulput dimiensions
Figure 3: Impact of architectural choices on PEC's performance. Plotted are the means (solid lines) and standard
errors (shaded areas) from 10 seeds. Vertical dotted lines mark settings used for experiments in Tables [ anddd
treated approximating sample from Gaussian Process, Section 3.2 it is known that neural networks approximate Gaussian Processes as their width increases (Neal 1996).
RELATED WORK
We discuss relevant literature on continual learning and cuss-incrrmennal learning in Section 2
Random Network Distillation An important foundational work to study is Burda al. 201 8 where Random Network Distillation proposed. This method consists of cloning a random Irozen neural network into trained one and then utilizing prediction errors to detect novel states a reinforcement learning environment. While we reuse this principle in our PEC, we note that to the best of our knowledge, this approach has not previously been used for class-incremental learning, or even classification in general. Another work on which we build is Ciosek et al. (2020), which reinterprets Random Network Distillation as a principled way to perform uncertainty estimation for neural networks; use ideas from this work to build our theoretical support for PEC.
OOD detection for task inference The works of Kim et (2022a b) decompose CIL into task- incremental learning and task inference, and then tackle the latter with -uttt--diitriuution (OOD) detection methods. Our PEC scores could be interpreted through similar lens. However, the approaches used in the mentioned works not suitable in cases of paagee--s--hrrugh-hoee or see-class-per-tasse Another related work is [Henning al. (202T), which uses uncertainty- OOD detection for task recognition.
LIMITATIONS AND FUTURE WORK
Although we have shown the effectiveness of PEC, also mindful of some of its limitations:
The current version of PEC completely separates information coming from different classes. While this eliminates forgetting, it also prevents the transfer of information.
In its current form, PEC uses a separate student model for each class, resulting in linear increase in the number of parameters with the number of classes. However, PEC uses small and shallow networks, the severity this issue limited in practice. As indicated in Section 4.4 the performance of PEC drops in the case of data imbalance. However, this deterioration is mostly climinated with the proposed mitigation strategies.
An interesting, and perhaps surprising, design choice of PEC is the use random shallow neural network as a teacher. While this choice has been empirically justified by its strong performance, and there is theoretical support for this choice through the link with Gaussian Processes, it is possible that the performance of PEC could be improved further by considering different teacher functions. Options might be use (pre-) trained neural networks or perceptual hashes (Zauner 2010). We leave exploring the effectiveness of such alternative forms of the teacher function for future work.
7 CONCLUSIONS
In this study, proposed Prediction Error-based Classification (PEC). This novel approach for plass-incremental learning uses the prediction error of networks trained mimic frozen random network to generate a score for each class. We demonstrated the strong empirical performance of PEC in single-pass pass-through-date CIL, outperforming rehearsal-free methods in all cases and athearsal-basee methods with moderate replay buffer size in most cases.
9

---

Published as a conference paper at ICLR 2024
ACKNOW
Michat Zajac thanks Marcin Sendera and Tomasz Kušmierczyk for helpful discussions. This paper is part of a project that has received funding from the European Union under the Horizon 2020 research and innovation program (ERC project KeepOnLearning, grant agreement No. 101021347) and under Horizon Europe (Marie Skłodowska-Cun research was supported by the PI-Grrid Infrastructure. The computations were partially performed in the Poznań Supercomputing and Networking Center. The work of Michal Zajac was supported by the National Centre of Science (Poland) Grant No. 2021/43/B/ST6/0145 Michal Zajac acknowledges support from Warsaw University of Technology within the Excellence Initiative: Research University (IDUB) program.
fellowship, grant agreement No. 101067759). This
REFERENCES
Rahaf Aljundi, Eugene Belilovsky, Tinne Tuytelaars, Laurent Charlin, Massimo Caccia, Min Lin, and Lucas Page-Caccia. Online continual learning with maximal interfered retrieval. In Advances in Neural Information Processing Systems, volume 32, 2019a.
Rahaf Aljundi, Klaas Kelchtermans, and Tinne Tuytelaars. Task-free continual learning. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11254-1122 1263, 2019b.
Mohammadamin Banayeeanzade, Rasoul Mirzaiezafdeh, Hosein Hasani, and Mahdieh Soleymani. Generative vs. discriminative: Rethinking the meta-continual learning. In Advances in Neural Information Processing Systems, volume 34, pp. 21592-216044 2021.
Eden Belouadah, Adrian Popeseu, and loannis Kanellos. A comprehensive study of class incremental learning algorithms for visual tasks. Neural Networks, 135:38-55 -5,, 2021.
Matteo Boschini, Lorenzo Bonicelli, Pietro Buzzega, Angelo Porrello, and Simone Calderara. Class- incremental continual learning into the extended der-verse. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2022.
Yuri Burda, Harrison Edwards, Amos Storkey, and Oleg Klimov. Exploration by random network distillation. arXiv preprint arXiv: 12894, 2018.
Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: strong. simple baseline. In Advances in Neural Information Processing Systems, volume 33, pp. 15920-15930, 2020.
Lucas Caccia, Rahaf Aljundi, Nader Asadi, Tinne Tuytelaars, Joelle Pineau, and Eugene Belilovsky.
New insights on reducing abrupt representation change in online continual learning. In International Conference on Learning Representations, 2021.
Arslan Chaudhry, Marc' Aurelio Ranzato, Mareus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with A-GEM. In International Conference on Learning Representations, 2019a.
Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet K Dokania, Philip HS Torr, and Mare' Aurelio Ranzato. On tiny episodic memories in continual learning. arXiv prepriní arXiv: 10486, 2019b.
Kamil Ciosek, Vincent Fortuin, Ryota Tomioka, Katja Hofmann, and Richard Turner. Conserva- tive uncertainty estimation by fitting prior networks. In International Conference on Learning Representations, 2020.
Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Aleš Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(7):3366-338, 2022.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Feii Imagenet: large-scale hier- archical image database, In 2009 IEEE Conference on Computer Vision and Puttern Recognition, pp.
IEEE, 2009.
Li Deng. The MNIST database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 1416)14 2012.
10

---

Published as a conference paper at ICLR 2024
Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pp. 249-2566 JMLR Workshop and Conference Proceedings, 2010.
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT press, 2016.
Raia Hadsell, Dushyant Rao, Andrei A Rusu, and Razvan Pascanu. Embracing change: Continual learning in deep neural networks. Trends in Cognitive Sciencex, 24(12):1028-100 2020.
Nikolaus Hansen and Andreas Ostermeier. Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation, 9(2):159-199) 2001.
Tyler L Hayes and Christopher Kanan. Lifelong machine learning with deep streaming linear discriminant analysis. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition Workshops, pp. 220-221, 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision, pp.
2015.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778, 2016.
Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (GELUs). arXiv preprint 1XX::1606608415. 2016.
Christian Henning. Maria Cervera, Francesco D' Angelo, Johannes Von Oswald, Regina Traber, Benjamin Ehret, Seijin Kobayashi, Benjamin F Grewe, and João Sacramento. Posterior meta-replay for continual learning. Advances in Neural Information Processing Systems, 34:14135-14149 2021.
Kenji Kawaguchi, Leslie Pack Kaelbling, and Yoshua Bengio. Generalization in deep learning. arXiv preprint arXiv:1710.05468; 2017.
Gyuhak Kim, Sepideh Esmaeilpour, Changnan Xiao, and Bing Liu. Continual learning based on ood detection and task masking. In Proceedings of the IEEF/CVE Conference on Computer Vision and Puttern Recognition Workshops, pp. 3856-3866, 2022a.
Gyuhak Kim, Changnan Xiao, Tatsuya Konishi, Zixuan Ke, and Bing Liu. A theoretical study on solving continual learning. Advances in Neural Information Processing Systems, 35:5065-5079, 2022b.
Diederik P Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv preprint 14Xiv 14tp.6922.66 2014.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv:1312.611e 4, 2013.
arXiv preprint
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka BarbbkkaBarriskka, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 114 (13):3521-3526, 2017.
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
Dhireesha Kudithipudi, Mario Aguilar-Simon, Jonathan Babb, Maxim Bazhenov, Douglas Blackiston, Josh Bongard, Andrew P Brna, Suraj Chakravarthi Raja. Nick Cheney, Jeff Clune, et al. Biological underpinnings for lifelong learning machines. Nature Machine Intelligence, 4(3):196 210, 2022.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(12):2935-2947, 2017.
Vincenzo Lomonaco and Davide Maltoni. CORe50: new dataset and benchmark for continuous object recognition. In Conference Robot Learning, pp. 17-26. PMLR, 2017.
11

---

Published as a conference paper at ICLR 2024
David Lopez-Paz and Mare' Aurelio Ranzato. Gradient episodie memory for continual learning. Advances in Neural Information Processing Systems, 30, 2017.
Davide Maltoni and Vincenzo Lomonaco. Continuous learning in single-incremental-taa scenarios. Neural Networks, 116:56-73, 2019.
Marc Masana, Xialei Liu, Bartlomiej Twardowski, Mikel Menta, Andrew D) Bagdanov, and Joost van de Weijer. Class-incremental learning: survey and performance evaluation on image clas- sification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(5):5513-5533, 2023.
Radford M Neal. Priors for infinite networks. In Bayesian Learning for Neural Networks, pp. 29-53. Springer, 1996.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Ng. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011.
German Ignacio Parisi, Ronald Kemker, Jose L. Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: review. Neural Networks, 113:54-71, 71, 2019.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, al. PyTorch: An imperative style, high-perfrrmance deep learning library. Advances in Neural Information Processing Systems, 32, 2019.
Carl Edward Rasmussen and Christopher KI Williams, Gaussian Processes for Machine Learning. MIT press Cambridge, MA, 2006.
Sylvestre- Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 2001-2010, 2017.
Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. Continual learning with deep generative replay. Advances in Neural Information Processing Systems, 30, 2017.
Albin Noutif-Cormerai, Antonio Carta, Andrea Cossu, Julio Hurtado, Vincenzo Lomonaco, Joost Van de Weijer, and Hamed Hemati. A comprehensive empirical evaluation online continual learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pp. 1--3528, 2023.
Gido M van de Ven, Zhe Li, and Andreas S Tolias. Class-incremental learning with generative classi- fiers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 361 3611-3620, 2021.
Gido M van de Ven, Tinne Tuytelaars, and Andreas S Tolias. Three types of incremental learning. Nature Machine Intelligence, 4:1185-1197, 2022.
Oriol Vinyals, Charles Blundell. Timothy Lillicrap. Daan Wierstra, et al. Matching networks for one shot learning. Advances in Neural Information Processing Systems, 29, 2016.
Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 374-382, 2019.
Han Xiao, Kashif Rasul, and Roland Vollgraf. mashion-mmist: novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv: 077447, 2017.
Greg Yang. Wide feedforward or recurrent neural networks of any architecture are gaussian processes. Advances in Neural Information Processing Systems, 32, 2019.
Christoph Zauner. Implementation and benchmarking of perceptual image hash functions, 2010.
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In International Conference on Machine Learning, pp. 3987-3995. PMLR, 2017.
Chen Zeno, Itay Golan, Elad Hoffer, and Daniel Soudry. Task agnostic continual learning using online variational bayes. arXiv preprint arXiv: 10123, 2018.
DeiWei Zhou, Wei Wang, Zhi-Hong Qi, Han-Jia Ye, De-Chuan Zhan, and Ziwei Liu. Deep class-nncrementll learning: A survey. arXiv preprint a3X 2023.
12

---

Published as a conference paper at ICLR 2024
Table 4: MI. architectee used in PEC experiments.
linear hidden layer, w neurons
layer normalization GELU activation
linear output layer, d neurons
Table 5: Convolutional architecture used PEC experiments.
convolutional hidden layer, channels, 3 filter, padding instance normalization
ReLU activation
average pooling, Tinput X Tinput - flattening layer
linear output layer, d neurons
A AUTHORS CONTRIBUTIONS
Michal came up with the idea for PEC, implemented all experiments, developed the theoretical argument, wrote the initial draft of the paper, contributed the discussions, and proposed some of the experiments.
Tinne co-supervised the project, provided feedback on the experiments and the draft, contributed to the discussions, and proposed some of the experiments.
Gido supervised the project, suggested Michal work on generative classification, provided feedback on the experiments and the draft, contributed to writing, contributed to the discussions, and proposed some of the experiments.
B DETAILS ON EXPERIMENTAL SETUP
We implement experiments using Py Torch Paaskke et al. 2019 and Mammoth BBuzzega et al. 2020; (Boschini et al. 2022), except for the experiments in Appendix C. which are implemented on top of the codebase from van de Ven al. (2021 Note that in PEC, we default initialization from Py PyTorch (i.e., Kaiming initialization (He al. 2015b) for both student and teacher networks.
We present details on architectures Appendix B.I on method implementations in Appendix [B.2 on hyperparameters in Appendix B.3 on balancing strategies in Appendix B.4, and on data augmen- tations in Appendix B.5
ARCHITECTURES
We outline neural network architectures used in PEC experiments. MLP (used for MNIST (Deng 2012)) described in Table and convolutional architecture (used for SVHN (Netzer et al.
2011), CIFAR-10 (Krizhessky et al., 2009), CIFAR-100 100 (Krizhevsky et al.) 2009), and minilm ageNet (Vinyals et al. (0166) is presented in Table
Both architectures have width parameter for the middle layer. As discussed in Section 4.5 PEC benefits from having the teacher network wider than the students, and consequently, we use separate values, wy and wh. for student and teacher width, respectively. Additionally, convolutional architecture has parameter r that describes resolution after average pooling.
Below we list settings used for different datasets:
MNIST: MLP network, wy = 10, w/ = 5000, = 99.
'GitHub http://wwww..............//////a
Gut htppapa............tttoooooo/oor incrementa" -
13

---

Published as a conference paper at ICLR 2024
Table 6: Parameter counts used by the methods.
Method
MNIST
Balanced SVHN
CIFAR-10
CIEAR-100
minilmageNNet
Nearest mean
7.8K
30.7K 10.7K
30.7K
307.0K
2.1M
SLDA
622.0K 89.6K
9.5.M
9.5M
9.7M
9.7M
All other
11.2M
11.2M
11.2M
11.2M
Table 7: Multiply operations (MACs) used in single inference forward pass.
Method
MNIST
Balanced SVHN
CIFAR-10
CIFAR-100
minilmageNet
PEC
4.5M
352M 556M
352M 556M
300M 556M
1.9G
discriminative
90.0K
3.9G'
Balanced SVHN and CIFAR- 10: convolutional network, Wig = 60, w/, = 6000, d = 743,
5.
CIFAR-100 100 and minilmageNet: convolutional network, Wig = 40, Wh = 4000, = 172.
4.
These values vary mostly to accommodate different parameter budgets and input sizes. In particular, values of d not being round numbers is consequence of matching given parameter budget. In our preliminary experiments, we did some tuning of the architectures while looking at validation accuracy. However, this tuming was not performed for all datasets, and we directly reuse architectures from CIFAR-10 Balanced SVHN. and from CIFAR- -100 to minilmageNNt. In the latter case, transfer the architecture despite different input sizes, which possible because of the use of the convolutional layer and average pooling. Moreover, given the form of the architectures from Tables 4 5 and some parameter budget, there is very limited search space for the concrete architecture. Hence, expect that the architectures we found would also perform well for other datasets with compatible input sizes.
B.2 DETAILS ON THE IMPLEMENTATION OF METHODS
Parameter counts We report total parameter counts in Table For all methods except for Nearest mean and SLDA (Hayes Kanan 2020), these are matched. Note that for PEC, we not count the parameters of the frozen random teacher h towards the limit; since these parameters are fixed, the algorithm does not have to remember them, and can instead recreate them using only the remembered random seed.
Number of operations needed We compare the number of multiply-accumulate operations (MACs) needed for single inference forward pass, comparing our PEC versus the discriminative approaches. Results in Table 7 show that PEC typically uses fewer operations. Intuitively, this is because PEC does substantial part of its computation in fully connected layer which has much smaller compute-tor taallettt--pramtteee density compared to convolutional layers that mostly constitute the ResNet architecture utilized in discriminative approaches.
Replay-based methods For the replay-based methods, we allow taking an extra examples from the buffer for each batch of the current data containing b examples. Some of the methods (namely, DER++ (Buzzega et al. 2020) and X-DER (Boschini al., 2022)) sample from the replay buffer multiple times for each batch of current data. We modified implementations so that the algorithm only samples once from the replay buffer, and then reuses those samples to compute the various losses defined in those methods.
A-GEM We use the implementation of A-GEM (Chaudhry et al. 2019a) with reservoir replay buffer from Mammoth.
SLDA To make this algorithm comply with the peggl-ppss--troughhhe batch-wise computation of the covariance matrix on the first task and only rely on online updates, using an identity covariance matrix as a starting point. SLDA uses shrinkage parameter e, which is a coefficient for the added identity matrix in the precision matrix computation. While originally it was proposed to be constant equal to 0.0001, observed in our experiments that setting much higher
setting, we do not start with
14

---

Published as a conference paper at ICLR 2024
values for this parameter act as useful regularizer and yield improved results. We describe search space and selected values for this parameter in Appendix [33.1 Additionally, for SLDA experiments on minilmageNet, we rescale the inputs from 84 x 84 to 32 x 32 to make the covariance matrix computation feasible.
VAE-GC For VAE-GC use architectures from van de Ven al. 202 with the widths adjusted to match parameter counts from other methods. We use 100 importance samples to estimate likelihoods. Furthermore, for VAE-GC experiments on minilmageNet, we rescale the inputs from 84 x 84 to 32 x 32 to make the generative task simpler; we found that this resolution works better than the original one.
Finally, note that for PEC and the considered generative classifiers (Nearest mean, SLDA and VAE- GC), we only report results for the one-class-per-task splits. The reason is that the performance of these methods should not depend the task split, because the class-specific modules of these methods are only updated on examples from one class anyway. slight subtlety is that when using a batch size larger than one, the mini-batches in the cealtipee-caases-pee-eee from multiple classes, thus reducing the effective per-class batch size (and making it variable, each mini-batch does need to contain an equal number of samples from each class in the task). For simplicity, and because for PEC we always use batch size = 1, we ignore this subtlety.
setting contain samples
B.3 HYPERPARAMETRRS
Here, we first explain how the hyperparameter searches were performed. Then, in Appendix B.3.1 we provide the search space and selected values for every method. Finally, in Appendix B.3. we study the sensitivity of PEC and other methods to hyperparameters.
To select hyperparameters for our benchmarking experiments presented in Tables T and 2 for every combination of method, dataset, and task split, we perform a separate grid search. For every considered set of hyperparameters, we perform one experiment with single random seed. We then select the hyperparameter values that yield the highest final validation set accuracy, and we use those values for our benchmarking experiments with ten different random seeds.
While it has been argued that in continual learning, extensive hyperparameter searches are not desired and instead hyperparameters should be decided on the fly (Chaudhry et al., 2019a), we perform the searches to ensure that the baseline methods get as good performance as possible, even if in practice they could not be tuned so well. We note that PHC does not introduce any method-specific hyperparameters. Moreover, for all PEC experiments in Section 4 data augmentations, and use linear learning decay. Only the learning rate varies between experiments. However, below we will show that even if the learning rate is not tuned and the default value of 0.001 is used for all datasets, the performance of PEC is still strong. This indicates that our method can work well off the shelf.
we use batch size = 1, do not use
B.3.1
SEARCH SPACES AND SELECTED VALUES
For all methods except SLDA and Nearest mean, we tune the following hyperparameters:
learning rate (Ir) E {0.00003, 0.0001, 0.0003, 0.001, 0.003, 0.01, 0.03).
batch size (bs) E {1, 10)
whether to data augmentation (aug) E {false, truc).
whether to decay LR from the initial value to linearly (decay) E {allse true].
Additional tuned hyperparameters for all methods, together with search spaces, are listed in Table
g
We provide selected hyperparameter values for all methods in Table 10 for MNIST, in Table for Balanced SVHN, in Table 12 for CIFAR- in Table 13 for CIFAR-100, and in Table 14 minilmageNet.
B.3.2 SENSITIVITY TO HYYPRRPARAMETT VALUES
First, we consider a setting where the learning rate cannot be tuned and we run PEC with the default learning rate value of 0.001 for all datasets. Note that this means that only the architecture is varied between experiments (in order match the proper parameter counts), and hence in such a case PEC can be treated as an off-the-shee performance in this case, and in particular it still outperforms other rehearsal-free methods, whose hyperparameters were tuned with a grid search, by wide margins (compare to Tables and
algorithm. We observe in Table 9 that PEC retains the strong
15

---

Published as a conference paper at ICLR 2024
Table 8: Method-specific hyperparameters together with associated search spaces.
Method
Additional hyperparameters
ER
A-GEM
DER
distillation loss coef. a (0.1,0.2.0.3) 0.5,
DER++
distillation loss a (0.11,0.,00.3
regular replay coef.
(0.1,00.,,05...
X-DER
distillation loss coef. a E 0.66
regular replay loss coef.
{0.8, 0.9) consistency loss coef. À (0.05,0.1]
pas//future logits constraint coef. 7 E (0.001,
constant past/future logits constraint 0..3
ER-ACE
ICaRL
BiC
EWC
regularization coel. {0.1, 1, 100, 1000)
SI
regularization coef. C (0.5, 1,2,5)
damping coefficient E (0.9, 1]
LwF
regularization coef. a E (0.5, 10]
Labels trick
Nearest mean
SLDA
identity matrix mixing coef. {0.0001, 0.0003, 0.001, 0.003, 0.01, 0.03, 0.1, 0.3, 0.5,
VAE-GC
PEC
Table Performance comparison PEC with tuned learning rates versus PEC using default learning rate 0.001. Reported is final average accuracy (in %), with mean and standard error from 10 seeds.
Method
MNIST
Balanced SVHN
CIFAR-10
CIFAR- 100
minilmageNet
PEC
92.31 (# 0.13)
68.70 (= 0.16)
58.94 (#: 0.09)
26.54 (# 0.11)
14.90 (+: 0.08) 14.90 (#: 0.08)
PEC, Ir - 0.001
90.51 (# 0.16)
64.25 (= 0.18)
57.38 (#: 0.12)
26.54 (# 0.11)
Additionally, look at how the performance of ER, DER++ VAE-GC, and PEC varies when manipulate learning rates; the rest of the hyperparameters remain as selected originally. The results in Figure indicate that the dependence of PEC on learning rate tuning is comparable to that of other methods.
Performance on MNIST
Performance on CIFAR-10
60
Final accuracy (in
85
50
80-
VAE-GC
70-
PTC
(10/11 siit
35
65
(5/2)
0.0001
0.0003
0.0010
0.0030
0.0100
0.0001
0.0003
0.0010
0.0030
0.0100
Learning rate
Learning
Figure 4: CIL performance for varying learning rate. Plotted are the means (solid or dashed lines) and standard errors (shaded areas) from 10 seeds. For PEC and VAE-GC, results are the same for hoth task splits, and hence single curves are shown. In case of VAE-GC experiment on CIFAR-10 with learning rate 0.01, runs failed because of numerical instabilities.
16

---

10, aug false Published as a conference paper at ICLR 2024
Table 10: Selected hyperparameters for MNIST dataset.
Method
MNIST (10/1)
MNIST (5/2)
ER
Ir 0.0003, bs 1, decay true
0.0001, bs 1, decay true Ir = 0.001, bs = 10, decay = false
A-GEM
Ir = 0.0003, bs = 32, decay = true
DER
Ir 0.001, bs = I0, decay truc,
Ir 0.001, bs 10, decay true,
Q Ir 0.001, bs = 10, decay = true,
Q
DER++
rr = 0.001, bs - 10, decay = true, 0.5, B =1
0 1,
X-DER
Ir 0.001, bs 10, decay = false.
0.003, bs = 10, decay = = false,
0 = 0.3, B = 0.8, X = 0.05,
Q = 0.3, 8 = 0.9, X = 0.05,
0.01, m = 0.3
0.01, = -0.7
ER-ACE
r = 0.001, bs = 10, decay false
0.001, bs
10, decay = Jalse
iCaRL
r 0.003, bs 10, decay true
Ir 0.003, bs
10, decay = false
BIC
r 0.001,
10, decay false
0.001, bs 10, decay false
Labels trick
r 0.001, bs = 1, decay = Jalse
rr = 0.0001, bs = 32, decay
false
EWC LT
F 0.0003, bs = 32, decay - true,
0.0003, bs = 32, decay
X 1000
false, X = 1000
SI LT
Ir 0.003, bs 10, decay true,
0.003, bs decay true, C 0.5, {
C g 0.9
LWF + LT
r 0.003, bs = 32, decay = true, α 5
r 0.001, bs = 10, decay = truc, 5
Nearest mean
SLDA
0.1
VAE-GC
F 0.003, = 32, decay = Jalse
PEC
I 0.01, bs = 1, decay = true
Table 11: Selected hyperparameters for Balanced SVHN dataset.
Method
Balanced SVHN (10/1)
Balanced SVHN (5/2)
ER
Ir = 0.003, bs 1, aug true,
Ir 0.0001, bs 1, aug true,
decay true
decay lrue
A-GEM
Ir = 0.001, bs = 32, aug = brue,
Ir = 0.01, bs = 10, aug = false, decay false
decay true
DER
Ir == 0.0003, bs 10, aug true.
0.0003,
decay = true, a 0.5
decay = =f false, a =
DER++
Ir 0.0003, bs 10, aug true, decay = true, a 0.2, 3
Ir = 0.0003, bs 10, aug true, decay = true, a 0.5, 3
X-DER
I= 0.0001, bs = I0, aug = false, decay false,
Ir = 0.0001, bs = 10, aug = false, decay true, a 0.6, 8 0.8, 0.05, r = 0.01, m 0.3 I = 0.0003, bs = 10, aug = false, decay true
0.3, 3 0.8.
== 0.1, 0.01, m 0.7
ER-ACE
r= 0.0001, bs I0, aug true,
decay truc
iCaRL
Ir 0.0001, bs = 10, aug = false,
= 0.0001, bs = 10, aug = false,
decay = false
decay false
BIC
Ir 0.0001, bs 10, aug
Jalse,
= 0.0003, bs 10, aug Jasse,
decay = true
decay = true
Labels trick
Ir = 0.003, bs = 32, aug = Jalse,
Ir r= 0.00003, bs = 32, aug = J false,
decay Irue
decay brue
EWC+ + LT
Ir = 0.001, bs = 32, aug = Jalse,
= 0.0001, bs = 32, aug = Jalse, decay f false, x = 0.1
decay = false, ) 1000
SI+
Ir 0.01, bs
32, aug
False,
3e 05, = 100, aug = false, decay = lrue, c c= 2, =
decay = f false, c o= 1,
LWF
Ir == 0.003. bs = 10, aug false, decay true,
Ir = 0.0003, bs 10. aug false,
decay = brue, a 10
Nearest mean
SLDA
0.1
VAE-GC
Ir 0.0003, bs = 32, aug = alse, decay = false
PEC
Ir 0.0001, bs I, aug false, decay- true
17

---

Published as a conference paper at ICLR 2024
Table 12: Selected hyperparameters for CIFAR-10 dataset.
Method
CIFAR-10 10 (10/1)
CIFAR- (5/2)
ER
Ir 0.003, bs 1, aug truc,
Ir 0.0001, bs 1, aug truc,
decay true
decay false
A-GEM
Ir == 0.003, bs = 32, aug = false,
Ir = 0.0003, bs = 10, aug = false,
decay false
decay false
DER
bs = 10, aug - false,
0.0001, bs 10, aug = true,
decay = truc,
0.5
decay false, = 0.5
DER++
Ir 0.0001, bs I0, aug Jalse,
= 0.0003, bs 10, aug Jalse, decay = false, a = 1, 3=
decay = false, a =0.1, 3 0.5
X-DER
Ir = 0.0001, bs 10, aug true, decay false,
Ir = 0.0001, bs 10. aug false, decay false, a 0.6,
0.3, 8 0.8.
0.8,
A = 0.1, = 0.01, m 0.3
0.05, 1/ = 0.001, m = 0.3
ER-ACE
Ir 0.001, bs 10, aug Jalse, decay false
rr 0.0003, bs 10, aug true, decay true
iCaRL
Ir = 0.0001, bs = 10, aug Jalse,
Ir = 0.0003, bs = 10, aug = flss, decay true
decay true
BiC
I= 0.0001, bs 10, aug false,
0.0001,
10, aug false
decay = true
decay = false
Labels trick
Ir = 0.03, bs 1, aug false
Ir 0.00003. bs 10, aug false, decay true
decay f false
EWC+LT
I= 0.0003, bs = 32, aug = false,
I = 0.0001, bs = 10, aug = false
decay truc,
1000
decay true, x 1000
ST+LT
Ir 0.0003, bs 32, aug = Jalse,
3e 05, bs 32, aug = Julse,
decay = false, = 1,
decay = false, =2 -2.5=
LwF
Ir 0.003, bs 32, aug Jalse.
0.0003, 10, aug Jalse,
decay = false, a = 10
decay = true, a 10
Nearest mean
SLDA
0.3
VAE-GC
Ir 0.001, bs = 32, aug = false, decay true
PEC
Ir 0.0003, bs 1, aug false, decay True
18

---

Published as a conference paper at ICLR 2024
Table 13: Selected hyperparameters for CIFAR-100 dataset.
Method
CIFAR- 100 (100/1)
CIFAR- 100 (10/10)
ER
Ir 0.003, bs 1, aug true,
Ir 0.003, bs aug true,
decay false
decay false
A-GEM
I= 0.00003, bs 32, aug false,
Ir = 0.01, bs = 10, aug = false, decay true
decay false
DER
decay false, a 0.1 I= 0.00003,
10, aug = true,
10, aug false, decay f false, =
0.01,
DER++
Ir 0.01, bs I0, aug true, decay = false, a 0.1,1 3 ==
== 0.01, bs 10, aug true, decay = true, a 1, =
X-DER
Ir = 0.01, bs 10, aug true, decay fasse,
Ir 0.01, bs 10, aug false, decay false, a 0.6,
0.3, B 0.9,
0.9.
A 0.1.1.111 11 = 0.01, m 0.7
0.05, // = 0.01, m 0.3
ER-ACE
Ir = 3e 05, bs 32, aug true, decay true
r 0.0003, bs 10, aug
J Jalse,
decay false
iCaRL
Ir = 0.0001, bs = 10, aug true,
Ir = 0.03, bs = 10, aug f false,
decay true
decay false
BiC
Ir 0.01, bs 10, aug true,
0.03. bs 10, aug false
decay = faase
decay = lrue
Labels trick
Ir 0.0001, bs 1, aug true,
Ir 0.03. bs 32, aug false, decay true
decay false
EWC+LT
I == 0.001, bs = 32. aug = false,
Ir = 0.01. bs = 10, aug false 10
decay false, x =
decay false,
ST+
Ir 0.001, bs = 10, aug = Jalse,
3e 05, bs = 32, aug Jalse,
decay = false, c = 2,
decay false, == 5. = 0.9
LwF
Ir 3e 05, bs 32, aug Jalse,
0.0003,
32, aug Jalse,
decay = true, a = 0.5
decay = true, a = 10
Nearest mean
SLDA
0.5
VAE-GC
Ir = 0.003, bs = 32, aug false, decay true
PEC
Ir 0.001, 1, aug Jalse, decay true
19

---

Published as a conference paper at ICLR 2024
Table 14: Selected hyperparameters for minilmageNNt dataset.
Method
minilmageNet (100/1)
minilmageNet (20/5)
ER
Ir = 0.003, bs 1, aug true,
Ir 0.01, bs 1, aug true,
decay false
decay false
A-GEM
Ir = 0.0003, bs 32, aug true,
I = 0.00003, bs 32, aug = false, decay false
decay truc
DER
Ir 0.003, bs = I0, aug brue,
= 0.0001, bs 10, aug = true, decay f-f false, =
decay = truc,
DER++
Ir 0.003, bs I0, aug Jalse,
0.001, bs I0, aug true, decay =
decay = true, a = 0.1, 3
1oayy=faloe false, a = 1, B
X-DER
Ir = 0.001, bs 10, aug true, decay false,
Ir = 0.001, bs 10, aug false, decay false, a 0.3,
0.6, 3 0.9,
0.8,
A 0.1.1.111 0.01, m 0.7
0.1, 1 = 0.001, m 0.3
ER-ACE
Ir 3e 05, bs 32, aug Jalse,
r = 0.0001, bs 10, aug Jalse, decay false
decay true I= 0.00003, bs = 10, aug Jalse,
iCaRL
Ir = 0.0001, bs = 10, aug = true,
decay f aasee
decay false
BiC
Ir 0.00003, bs 10, aug decay = false
false,
0.0001,
10, aug false
decay = false
Labels trick
Ir 0.003. bs 32, aug true,
Ir 0.00003, bs 10, aug truc, decay false
decay false
EWC+LT
I = 0.003, bs = 32, aug = false,
I = 0.003, bs 10, aug = false decay false, X 1
decay truc,
10
ST+LT
Ir 0.03, bs = I, aug
Jalse,
3e 05, bs 10, aug false,
decay = true, c = 0.5, [ = 0.9
decay = true, 2, = 0.9
LwF
Ir 0.0003, bs 32, aug Jalse,
05, bs 32, aug Jalse,
decay = true, a
decay = false, a = 10
Nearest mean
SLDA
0.8
VAE-GC
Ir 0.003, bs = 32, aug false, decay true
PEC
Ir 0.001, = 1, aug Jalse, decay true
20

---

Published as a conference paper at ICLR 2024
B.4 DETAILS ON BALANCING STRATEGIES
In Section 4.4 we discussed the issues of comparability of PEC class scores and the impact of dataset imbalance. We introduced strategies to alleviate the effect of data imbalance, and here we provide additional details about these.
In Oracle balancing and Buffer balancing, we use CMA-ES (Hansen & Ostermeier 2001) to find scalars for re-scaling class scores. This evolutionary optimization algorithm maintains a Gaussian distribution, which is iteratively updated to fit values that yield high fitness scores. We tuned hyperparameters for this algorithm in our initial experiments and then used fixed values everywhere. which we provide here: initial standard deviation For each class c, we parameterize the scalar that we search for as = exp(le) and we search for real values L, since want find positive values for the scalars.
0.001, population size = 100, weight decay = 0.
In Equal budgets, we use the same number of gradient descent updates for each class. we are allowed multiple passes over the dataset, then we can set some fixed value No and use this many updates per class. in the the minimal number of examples for any class.
setting, we cut the number of updates for each class
B.5 DATA AUGMENTATIONS
Here, describe the data augmentations used in our experiments. Note that for every method decide whether to use these data augmentations or not based on grid search; for PEC we never use these.
MNIST: no augmentations are used.
Balanced SVHN: random crop to original size with padding = 4.
CIFAR-10, CIFAR- 100, and minilmageNet: random crop to original size with padding = 4, random horizontal flip.
C ADDITIONAL EXPERIMENTS
C.1 ADDITIONAL COMPARISONS ANOTHER EXPERIMENTAL SETUP
Here, to further test the robustness PEC, we perform additional empirical comparisons using an experimental setting based on van de Ven et al. (2021). First, we describe the experimental setup in Appendix C. C.I.1 and additional baselines Appendix C.1.2 and then we proceed the results in Appendix
C.1.1 EXPERIMENTAL SETUP FOR APPENDIX ONLY
We largely follow the experimental setup from van de Ven et al. (02TT). In the implementation of PEC, we match the parameter counts and the number of optimization steps with those used for VAE-GC in yan de Ven et al. 12021).
We compare PEC against a comprehensive list of CIL methods; we provide explanations for the methods that were not included in our main text experiments in Appendix C.1.2)
For some datasets, namely MNIST (Deng) 2012). Fashion-MNIST (Xiao et al. 2017). and CIFAR- 10 (Krizhevsky et al. 2009), directly raw image data. For others, namely CIFAR-IO 100 Krizhevsky al. 2009) and CORe50 (Lomonaco Maltoni (007)), we use pretrained frozen feature extractors, which are applied on the raw data hefore further processing; we follow van de Ven et al. 2021 using the same feature extractors.
Below briefly summarize the experimental setup for the particular datasets:
MNIST and Fashion-MNIST: we use (5/2) task split. Optimization for 2000 steps per task, or 1000 steps per class, with batch size = 128. ML. is used as an architecture, no feature extractor is used. Parameter count per class for VAE-GC and PEC methods: 150K. CIFAR-10: we use (5/2) task split. Optimization for 5000 steps per task, or 2500 steps per class, with batch size = 256. Simple networks with convolutions and linear layers are used as architectures, no feature extractor is used. Parameter count per class for VAE-GC and PEC methods: 350K.
21

---

Published as a conference paper at ICLR 2024
CIFAR-100: (10/10) task split. Optimization for 5000 steps per task, 500 steps per class, with batch size 256. MLP is used as an architecture, simple convolutional network pre-trained on CIFAR-10 is used as a feature extractor. Parameter count per class for VAE-GC and PEC methods: 180K.
CORe50: we use (5/2) task split. For this dataset, we only perform one pass through the data during training. We use batch size 1. MLP is used as an architecture, ResNet18 (He et al. 2016 pre-trained on ImageNet (Deng et al., 2009) is used as feature extractor. Parameter count per class for VAE-GC and PEC methods: 170K.
In all experiments, we use the Adam (Kingma & Ba 2014) optimizer. We set the learning rate the standard value of 0.001, with the following exceptions:
for the CORe50 experiments, LR is set to 0.0003 for PEC and 0.0001 for other methods; for the CIFAR- 10FAR-100-0 experiments, LR is set to 0.0001 for discriminative approaches and standard 0.001 otherwise.
Additionally, for all PEC experiments, the learning rate decays linearly to through the training.
For VAE-GC we use 10K importance samples to estimate likelihoods.
The method- speciiipeciicccci hyperparameters of SI, EWC and ARI are selected based the grid search described in van Ven et al. (202T). While for the experiments in Tables and 2 the shrinkage parameter e of SLDA was tuned, for the experiments in this section the default value of 0.0001 is used.
For VAE-GC experiments MNIST, additionally include batch normalization in the architecture, since we observed improves performance. Additionally, in order to ensure a fair comparison with VAE-GC, the best- cest-performing baseline in these experiments, we run grid search for this method where incorporate the following elements that used with PEC: linear learning rate decay and GELU activation function (Hendrycks & Gimpel 2016). These did not improve the results for VAE-GC.
C.1.2 ADDITIONAL METHODS
We provide short descriptions and references for baselines used in this section that were not present in our main text experiments:
Bias-correction methods attempt to mitigate imbalances between classes coming from different tasks.
Copy Weights with Re-init (CWR) (Lomonaco & Maltoni 2017) maintains the frozen representation network after the initial task, and for each consecutive task, only updates the part of the final linear layer corresponding to classes from that task.
CWR+ (Maltoni & Lomonaco 2019)) improves over CWR by introducing different initial-
ization and normalization schemes.
AR1 (Maltoni & Lomonaco (2019) builds on CWR+ but enables representations to shift, regularizing them additionally with SI.
Generative replay uses generative models to produce samples for pseudo-rehearsal.
Deep Generative Replay (DGR) (Shin et al. 2017) leverages VAE to produce examples from past tasks and uses them as additional data for training.
1.3 RESULTS
The results are presented in Table 15| Note that the experimental setup used here involves multiple epochs and, in most cases, higher parameter counts than in the main text experiments. As con- sequence, the advantage of PEC over VAE-GC is less pronounced. Nonetheless, PEC retains its strong performance and outperforms or matches all the basclines considered here on all datasets. This corroborates its robustness to experimental conditions.
C.2 PERFORMANCE OF REHEARSAL- METHODS WITHOUT LABELS TRICK
In the main text, we reported the results of rehearsal-fin methods with Labels trick added on top. For completeness, here we provide results without the Labels trick. Results for the one-class-per-task and multiple-classes-per-aaas are presented in Table and Table 17 respectively.
22

---

Published as a conference paper at ICLR 2024
Table 15:
Empirical evaluation of PEC against suite of CIL baselines, in experimental setups based on van de Ven et al. (2021). see Appendix C.I Final average accuracy (in %)) is reported, with mean and standard error from IO seeds. Methods separated into the following categories: baselines, regularization-batt -baeed, bias- con-coorect generative replay, generative classification, PEC. FE; pre- pre-trained feature extractor is used.
Method
MNIST (5/2)
Fashion-MNIST (5/2)
CIFAR-10 (5/2)
CIFAR-100 (10/10). FE
CORe50 (5/2). FE
Fine-tuning
19.92 (+ 0.02) 98.27 0.03)
19.73 (+ 0.25)
18,72 (+ 0.27)
7-98 0.10)
78,52 (+ 0.29) 71.50 0.20)
Joint
89.50 = 0.08)
82.61 (# 0.12)
51.42 0.13)
EWC
20.00 (.00)
19.75 (#: 0.24)
18.59 (1: 0.29)
8.34 0.08)
18.59 (1 0,34)
19.98 0.11)
0.42)
18.15 0.31)
8.54 0.08)
18.65 (+: 0.27)
CWR
30.18 (+: 2.59)
21.80 (= 2.25)
18.26 (+ 1.38)
22.16 (+: 0.69)
40.17 (#: 0.82)
CWR+
38.61 2.93)
19.80 2.77)
19.06 1.79)
9.48 0.32)
40.35 (1 0.98)
ARI
44.27 (.L. 2.50)
29.14 (L 2.17)
24.60 (1 1.08)
17.49 (LL 0.34)
46.77 (.L 0.93)
Labels Trick
33.05 (+: 1.32)
20.78 (# 1.90)
24 0.40)
24.02 (+ 0.39)
42.81 (+ 1.08)
DGR
91.30 (d: 0.53)
69.18 (#: 1.70)
17.73 (+: 1.58)
15.14 (+: 0.20)
56.67 (1 1,21)
SLDA
87.30 (+: 0.02)
81.50 (+ 0.01)
38.39 (+ 0.04)
44.48 (+ 0.00)
70.80 (+ 0,00) 70.74 (::: 0.08)
VAE-GC
96.29 (#: 0.04)
82.90 (= 0.06)
55.92 (#: 0.13)
49.46 (#: 0.08)
PEC
96.73 (.L. 0.04)
85.22 (1 0.07)
57.75 (.L. 0.09)
49.80 (.L 0.09)
70.81 (.L. 0.11)
Table 16: Performance of rehearsal-free methods without the Labels trick in the one-class perssss-eettak case. Final average accuracy (in %) is reported, with mean and standard error from 10 seeds. This table complements Table in the main text.
Method
MNIST (10/1)
Balanced SVHN (10/1)
CIFAR-10 (10/1)
CIFAR-100 (100/1)
minilmageNet (100/1)
EWC
10.09 (#: 0.00)
9.92 (1: 0.08)
10.55 (#: 0.44)
1.00 (= 0.00)
0.99 (1: 0.02)
SI
12.73 (#: 0.99)
10.00 (#: 0.00)
10.08 (#: 0.06)
1.10 (2 0.04)
0.97 (:: 0.07)
LWF
11.83 (1 0.58)
9.94 (1 0.03)
10.12 (1: 0.09)
0.93 (1: 0.02)
0.97 (1 0.04)
Table 17: Performance of rehearsal-fice free methods without the Labels trick the multiple-classes-prre per-task case. Final average accuracy (in %) reported, with mean and standard error from 10 seeds. This table complements Table inn the main text.
Method
MNIST (5/2)
Balanced SYHN
CIFAR-10 (5/2)
CIFAR-100 (10/10)
minilmageNet (20/5)
(5/2)
EWC
20.10 (+: 0.18)
12.99 (+ 1.43)
12.39 (+: 0.86)
6.81 (# 0.12)
1.88 (d: 0.20)
21.46 (1. 0.48)
19.13 (1 0.02)
18.75 (1: 0.02)
6.42 (1: 0.12)
2.94 (1 0.11)
LwF
21.49 (1: 1.86)
23.88 (1 0.62)
20.14 (1: 0.18)
7.75 0.13)
3.31 (1 0.03)
23

---

Published as a conference paper at ICLR 2024
Table 18: Performance with parameter hudget of M. for the one-class-per-task case. For discriminative methods, Narrow R8 architecture is used. Final average accuracy (in %) is reported, with mean and standard error from 10 seeds.
Method
Buffer
Balanced SVHN (10/1)
CIEAR-10 (10/1)
CIFAR-100 (100/1)
minilmageNet (100/1)
DER++ X-DER
77.07 (#: 1.01)
30.98 (# 2.45)
3.73 (# 0.12)
2.59 (=: 0.12)
71.30 (14777
36.46
12.17 0.39)
10.36 (11 0.31)
Labels trick (I.T)
9.87 (+ 0.06)
10.05
1.03 (+ 0.03)
1.03 (+ 0.05)
EWC+LT
10.04 0.07)
9.72
0,93 0.03)
(1
SI+LT LT
9.99 (+ 0.05)
9.52
0.99 0.06)
1.05 (+ 0.05)
9.92 (+ 0.13)
9.83
0.98
1.00 (=
EWC ST LwF
9,91 (1 0.07)
(1.04)
0,98 0.08)
1.01 0.06
10.00 (+ 0.00)
10.00 (.000)
1.00 0.03)
0.96 (+ 0.05)
9.97 (1 0.07)
9,91 (1 0.04)
1.02 (1 0.05)
0.99 (1 0.01)
PEC (ours)
60.17 (+ 0.21)
53.58 (+ 0.31)
19.67 (+ 0.36)
10.84 (+ 0.18)
Table 19: Performance with parameter budget of 1.1M, for the many-classss-per-task case. For discriminative methods, Narrow ResNet18 architecture is used. Final average accuracy (in %) is reported, with mean and standard error from 10 seeds.
Method
Buffer
Balanced SVHN CIFAR-10 (5/2)
CIFAR-100 (10/10)
minilmageNet (20/5)
(5/2)
DER++ X-DER
84.44 (.L. 0.36)
42.86 (1 0.70)
16.83 (1 0.39)
9.86 (.L 0.29)
71.30 (+ 3.70)
$2.16 1.08)
22,91 (+ 0.38)
16.48 (+ 0.33)
Labels trick (LT)
26.56 (#: 1.23)
22.87
0.67)
8.03 0.17)
3.99 (:: 0.06)
EWC+LT
25,28 (+ 1.06)
23.33
8.01 (+ 0.14)
3.38 (+ 0.11)
SI+LT
28.12 (=: 1.18)
38.08 (1
7.39 0.188)
(+
LT
41.37 (1 1.48)
10,92 (1 0.29)
(1
FWC SI LWF
17.51 (+ 0.87)
18,46 (+
2.94 0.09)
2.18 0.05)
(1.
5.31 0.09)
2.91 0.04)
26.14 (+ 0.18)
20.99 0.30)
5.89 (+ 0.03)
2.86 0.04)
PEC (ours)
60.17 (+ 0.21)
53.58 (# 0.31)
19.67 (# 0.36)
10.84 (+: 0.18)
C.3 PERFORMANCE EVALUATION WITH LOWER PARAMETER BUDGETS
In this section, we perform additional experiments, where we replace the variant of ResNet18 that we used in our main experiments with narrower version used by Lopez-Paz & Ranzato (2017):
Chaudhry et al. (2019b); Aljundi et al. 2019a): Caccia et al. (202T); Sooutit-Cormeraat et al. (003)) We call this variant Narrow ResNet18. We note that the only difference between the architecture
used in the main experiments and the Narrow ResNet18 is the width, with the former having width factor of 64 and the latter of 20, We also adjust the parameter count for PEC to match the 1.1M count of the Narrow ResNet18. 8. For this experiment, we performed limited hyperparameter searches: for each method, we take the hyperparameters that were used for the variant with the bigger (default) parameter budget and tune only learning rate. Results are reported in Table [8 for the mhe-class-per-tarl
case and in Table 19 for the many-classes-per-aae retains its strong performance in this regime and still outperforms all non-rehearsal baselines in all considered cases. PEC's performance looks favorably even compared to challenging rehearsal-basee approaches, DER++ and X-DER, as PEC outperforms them out of 8 considered cases.
case. We can see that PEC
C.4 PERFORMANCE WHEN VARYING NUMBER OF ENSEMBLE MEMBERS
Here we study how the performance changes if we allow for an ensemble with independent student- teacher pairs. For simplicity, we did not include this possibility in description of PEC and experiments in the main text. However, theoretical support from Scction 3.2 Results in Figure demonstrate increased performance when using more than one student-teacher pair. The gains are roughly comparable to those when the width of single networks increases, Figure 3
a natural variant consider, especially given the
24

---

Published as a conference paper at ICLR 2024
Performance on MNIST
Performance on CIFAR-10
62.0
accuracy 90
96
61.5
95-
Annll accuracy
61.0
94
60.5
93
59.5
2
5
59.0
Ensemble members
Ensemble members
Figure 5: CIL performance for varying number of ensemble members, Plotted are the means (solid lines) and standard errors (shaded areas) from 10 seeds.
2
4
5
6
7 8
9
true data
2
3
4
5
6
7
8
9
01
2
3
4
5
6
7
8
9
VAE
01
3
4
5
6
7
8
9
PEC
Figure 6: Original data from the MNIST dataset is compared to generations from VAE and PEC modules used in our experiments. Each column shows samples for different class.
C.5 Is PEC SECRETLY A GENERATIVE MODEL?
Here we examine if one can generate samples from datasets' class distributions given PEC modules. Since the score of PEC module is low for examples from its corresponding class, try to generate samples by starting from random input and then minimizing the PEC score for given class by gradient descent on inputs. Specifically, we utilize Adam with the learning rate of 0.00003, and perform 30K updates. Generated samples are presented in Figure a and compared against samples from the class- class-specific VAE modules of VAE-GC, well as original data. Interestingly, samples from PEC bear resemblance to respective digits. However, the quality of generations very poor.
C.6 RESULTS FOR METHODS WITH MEMORY BUFFER FOR VARYING BUFFER SIZE
In main text experiments, when comparing against methods that leverage memory buffer, focused on moderate buffer size of 500. Here we include comparisons for the case of small and big memory buffers, containing respectively 100 and 5000 examples, following regimes from Buzzega et al. (2020). For these experiments, we perform the same hyperparameter searches as for the main Text results, see Appendix B.3
Results for the case of one class per task are presented in Table 20 and for the case of multiple classes per task are presented in TTble As expected, the performance of these methods drops significantly we move to the smaller bulfer, increasing the advantage of PEC. In the the one-class-per-task setting and scores the best for CIFAR- 100 and
of big buffeer size,
PEC still competitive minilmageNet datasets, while in the multiple-classe--pre per-task setting its advantage mostly fades away. This is not surprising, as replay-based methods are increasingly similar to the joint training as the buffer size grows
25

---

Published as a conference paper at ICLR 2024
Table 20: Results for the varying buffer size for the methods that use memory buffer in case one class per task. Final average accuracy (in %) is reported, with mean and standard error from seeds. Results for PEC are included to case comparison.
Method
Buffer size
MNIST (10/1)
Balanced SVHN
CIEAR-10
CIFAR-100 (100/1)
minilmageNet (100/1)
(10/1)
(10/1)
100
68.67 (#: 1.04)
46.53 (#: 1.98)
28.39 (# 0.80)
(=107)
2.76 (#: 0.15)
A-GEM
100
63.90 (1: 1.21)
10.35 0.18)
(1.40
0.90 (# 0.04)
1.50 0.07)
DER
100
81.27 (+: 0.64)
$7.67 (+: 2.42)
25.33 (1: 0.76)
1.00 (1:0
1.01 (#: 0.04)
DER++
100
81.41 (1 0.39)
(1.11(11.89)
23.33 1.23)
(1
2.45 (LL
X-DER
100
80.18 (+: 0.61)
48.64 3.86)
30,93 ( 0.44)
(+
4.92 0.17)
CaRL
100
7- 0.54)
22.16 (+: 2.99)
28.99 (# 0.39)
5.06 (=116))
3.46 (# 0.16)
BiC
100
64.48 (1:: 1.59)
14.39
12.48 0.72)
3.12
1.16 1.16 0.05)
ER
500
84,42 0.266
(1
10.59 1.17)
12.47 (.:3)
5.72 0.24)
A-GEM
500
(4: 0.82)
9.93
(+.01)
(.00))
1. 0,07)
DER
500
91.65 (+
(+
40.03 1.52)
(+
0.99 (+
DER+
500
91.88 (= 0.19)
(+.66
35.61 ( 2.43)
(
1.40 0.09)
X-DER
500
82.97 (1: 0.14)
70.65 70.65
43.16 (1 0.46
15.55
8.22 0.35)
iCaRL
500
83.08 0.28)
57.18
37.80 (+
7.52
BiC
500
86.00 (# 0.37)
70.69 0.52)
35.86 (= 0.38)
6,44 (44
1.46 0.05)
ER
5000
92.34 (# 0.41)
82.06 0.68)
43.90 0.59)
20.76 ( 0.65)
(3:
A-GEM
5000
59.27 (# 0.96)
(+
11.32 0.599
0.97 0.006
1.03 0.04)
DER
5000
93.74 (1: 0.08)
85.47 (1:
51.41 (1
0.95
0.97 (1 0.04)
DER++
5000
95.01 (+: 0.07)
88.36 0.25)
49.51 2.04)
(
1.56 0.13)
X-DER
5000
86.23 (+ 0.09)
78.54 0.2)4)
52.32 (# 1.23)
22.33 (=880)
3.31 0.15)
iCaRL
5000
88.02 (1: 0.21)
81.97 (= 0.28)
61.13 (1 0.22)
8.64
14.17 0.52)
BiC
5000
93.11 (#: 0.17)
86.89 (# 0.25)
62.84 (# 0.40)
2.84 (= 0.06)
2.95 (# 0.14)
PEC (ours)
92.31 (1: 0.13)
68.70 (+: 0.16)
58.94 (1 0.09)
26.54 (1 0.11)
14.90 (1L: 0.08)
Table 21: Results for the varying buffer size the methods that use memory buffer case of multiple classes per task. Final average accuracy (in %) reported, with mean and standard error from 10 seeds. Results for PEC are included ease comparison.
Method
Buffer size
MNIST (5/2)
Balanced SVHN
CIFAR-10 (5/2)
CIFAR-100 (10/10)
minilmageNet (20/5)
(5/2)
100
60.89 (:: 1.42)
58.07 (#: 0.45)
38.75 (1: 0.63)
10.78 (# 0.34)
4.37 (d: 0.14)
A-GEM
100
57.45 (.L. 1.26)
20.40 (1L 0.85)
18.10 (1 0.15)
6.07 (1 0.05)
2.67 (.L 0.16)
DER
100
81.68 (1L 0.37)
70.40 (1: 1.25)
37.09 (1 0.83)
8.62 (1: 0.18)
3.95 (1. 0.08)
DER+
100
81.52 (+: 0.71)
66.64 (: 1.67)
40.31 40.31 (:
6.41 6. 0.23)
X-DER
100
84.19 0.94)
74.79 0.81)
$1.60 0.81)
19.44 0.224)
8.83 (+ 0.81)
iCaRL
100
74,53 (1: 0.52)
58.45 (1: 0.74)
48.05 (1 0.34)
10.87 (1: 0.14)
9.78 (1: 0.13)
BiC
100
68.10 1.44)
27.84 (1 0.88)
23.34 (1L. 1.07)
8,42 (1 0.23)
4.08 (.L 0.02)
ER
500
86.55 (+: 0.17)
80.00 (+ 0.544
$7.49 (+: 0.54)
(+
(4: 0.21)
A-GEM
500
50.82 (t: 1.36)
19.71 (1 0.49)
18.21 (1: 0.14)
6,43 (4
2.47 (1: 0.18)
DER
500
91.64 0.11)
78.22 0.94)
43.98 1.20)
7.69 (+2277)
3.86 (+ 0.04)
DER+
500
92.30 (1: 0.14)
86,50 0.43)
5U.61 (1 0.80)
19.88 (+225)
10.50 0.42)
X-DER
500
81.91 (11 0.53)
73.88 (1 1.59)
54.15 1.92)
84
14.65 (1 0.40)
CCRLL
500
80.41 (+: 0.45)
73.65 0.57)
$4.33
13.31
13.46 (+ 0.20)
BIC
500
88.08 (#: 0.81)
77.65 0.45)
46.74 (# 0.83)
14.12
6.90 0.19)
ER
5000
94,60 (+: 0.11)
87.62
63.55 0.28)
9.14 (+: 0.37)
A-GEM
5000
46.52 (+: 1.37)
18.33 0.52)
18.49 (1: 0.13)
6.66 0.05)
2.92 (1: 0.12)
DER
5000
95.02 (11 0.06)
84.08 (1L 0.84)
55.57 0.72)
7..2
3.85 (1 0.06)
DER+
5000
95.70 0.07)
0.15)
61.41 (+57))
24.07 (+440)
5.31 0.41)
X-DER
5000
87.78 (#: 0.47)
79.79 (+: 1.22)
59.80 (# 0.52)
34.58 (+.30
16.94 (#: 0.58)
iCaRL
5000
87.56 0.29)
85.03 (1 0.17)
63.85 0.23)
19.39 (1 0.18)
BIC
5000
94.55 (1: 0.14)
88.38 (1 0.19)
69.84 (1 0.22)
25.42
(4 0.21)
19.94 (1: 0.16)
PEC (ours)
92.31 (LL 0.13)
68.70 (L 0.16)
58.94 (L 0.09)
26.54 (L10.
14.90 (LL 0.08)
26

---

Published as a conference paper at ICLR 2024
Table 22: Performance comparison of PEC for various initialization schemes. Reported is the final average accuracy (in %), with mean and standard error from 10 seeds.
Method
MNIST
CIFAR-10
PEC, Kaiming (vanilla)
92,31 (# 0.13)
58,94 (#: 0.09)
PEC, Xavier
91.70 0.13)
58.02 (1: 0.14)
PEC, random uniform from
-0.001, 0.001]
88.52 (+ 0.22)
57.33 (+ 0.22)
PEC, random uniform from -0.01, 0.01]
91.73 0.09)
57.60 (# 0.20)
PEC, random uniform from -0.1,0.1
92.01 0.14)
57.46 (# 0.20)
C.7 PEC WITH DIFFERENT INITIALIZATION SCHEMES
We verify how much the performance PEC depends on the chosen initialization scheme. We study Kaiming (He et al. 2015) (our default choice), Xavier (Glorot & Bengio 2010), and random uniform with different ranges. For each choice of initialization scheme, we use to initialize the parameters of both the teacher and the student networks. The results in Table 22 indicate that the performance of PEC is relatively stable with respect to the way the networks are inttialized.
C.8 PEC WITH LINEAR ARCHITECTURES
Here we further explore architectural choices for PEC. We compare PEC with our vanilla architecture, which has a single hidden layer with a nonlinear activation funetion, to PEC with linear networks that contain no activation function. We consider two variants. For both variants, the described changes are applied to both the teacher and the student networks. In Removed nonlinearity, we take the vanilla PEC architecture and remove the nonlinear activation function, resulting in a linear network that retains some inductive biases of the original architecture (e.g. the use of convolutions). In Single linear, we just take single linear layer with the output size chosen to match the parameter count of the original architecture. Results in better than the linear variants. Interestingly, for Balanced SVHN Single linear performs better than vanilla. A reason for this might be the suboptimality of the used convolutional architecture, which we did not tune extensively.
indicate that typically vanilla PEC performs substantially
C.9 PEC VERSUS ENSEMBLE DISCRIMINATIVE CLASSIFIER
Here report additional experiments in which we test whether the strong performance of PEC could be explained by network ensembling (i.e., training a separate network per task), rather than that is due to the PEC classification rule. To test this, we compare PEC with various variants of an 'ensemble discriminative classifier baseline'. For this ensemble baseline, we train for each task a separate network equipped with task- -peeiffc softmax layer, and during inference we take the argmax of the predicted scores across the softmax layers of the different models. We consider three variants of this ensemble baseline. In Ensemble, full, we use for each task a separate copy of the original discriminative network (i.e., two-layer MLP for MNIST, ResNet18 for the other datasets). Note that this baseline uses times more parameters than PEC, with T the number of tasks. In Ensemble, small, we use for each task a version of the original discriminative network (i.c., two-layer MLP for MNIST, ResNet18 for the other datasets), but with the width of the networks reduced in order to match the parameter count of PEC and other baselines. Finally, Ensemble, PEC-architecture. we use for each task the same network architecture as for PEC, except with a softmax output layer instead of the d-dimensional linear output layer and with the width of the hidden layer adjusted in order to match the parameter count of PEC. This last variant is the most direct comparison, because it as close to PEC as possible, except that does not use the PEC classification rule.
To give these ensemble haselines the best possible chance, we run these three variants in the easier multiple-classe-peeee ter-tkkkk setting. (In preliminary experiments we confirmed that these ensemble baselines indeed perform substantially worse in the paagle--lass-pe-tass. case.) The results are reported in Table [2] We see that, on all five datasets, PEC substantially outperforms all three variants of the ensemble baseline. This provides strong evidence that the PEC classification rule is critical for the strong performance of PEC.
27

---

Published as a conference paper at ICLR 2024
Table 23: Performance comparison of vanilla PEC versus variants of PFC with linear networks. Reported is the final average accuracy (in %6), with mean and standard error from 10 seeds.
Method
MNIST
Balanced SVHN
CTFAR-10
CIFAR-100
miniImageNet
PEC
92.31 (1 0.13)
68.70 0.16)
58.94 (1 0.09)
26.54 (L 0.11)
14.90 0.08)
PEC, Removed nonlincarity
92.41 (+ 0.12)
61.85 (# 0.20)
46.19 (# 0.14)
19.41 (# 0.11)
10.55 (# 0.11) 7.30 (# 0.11)
PEC, Single linear
80,39 (# 0.24)
71.08 (= 0.18)
40.04 (# 0.14)
13.21 (= 0.10)
Table Comparison PEC with several versions of the ensemble discriminative classifier baseline, multiple-classed classes-per-task case. Reported is final average accuracy (in %). with mean standard error from 10 seeds.
Method
MNIST
Balanced SVHN
CIEAR-10
CHFAR-100
minilmageNeet
PEC
92.31 (# 0.13)
68.70 (+ 0.16)
58.94 (# 0.09)
26.54 (# 0.11)
14.90 (# 0.08)
Ensemble, full
55.47 1.38)
52.17 (#: 1.70)
41.15 (# 1.00)
(1 0.21)
8.83 0.19)
Ensemble, small
48.04 (+: 2.16)
47.78 (1: 0.98)
38.31 (1: 0.77)
17.58 (1: 0.22)
7.75 (1: 0.23)
Ensemble, FEC-arcbitecture
51.09 (#: 1.71)
47.70 0.28)
36.39 (1: 0.13)
24.52 (1.00.1.117)
12.82 0.18)
D DETAILED THEORETICAL SUPPORT SUPPORR FOR PEC
D.1 PROOFS THE PROPOSITIONS FROM MAIN TEXT
P (Propsiiion from Closek et al. 2020), rephrased), Let be a Gaussian Process and X dataset. Then the following inequality holds:
H (x ) 2 o (Hx)
(7)
Proof. We can directly Proposition from (Coseek al. 2020, Section 4.1, Appendix F.1) with from the original proposition statement being equal to H, and function from the original proposition statement being equal to (XX, f(Xx, x²) ((X,h(X) = gd. (x"),, for any H. Note that (1..) and o (a.) from the original statement correspond
HH (x*) and a (x=) respectively.
L
Lemma 1 (Lemma 4 from Ciosek et al. 2020). rephrased), Let h B-K R.M. be function with the domain restricted to (x 2 S Tj. U be a constant such that |[](x))) S for all x in the domain, and X be a data distribution with support in (x || 11. Furthermore, assume that K 2 3, and that our procedure for training neural networks g gh.X returns Lipschitz, functions with constant L.. Then, for dataset X == ..,,...,,N.. X, the following inequality holds with probubility 1 8:
fipschitefunctions
N
M MIl9 (ilgx((x"))) m(0")11 (x)))))111
llg (.) 1(x;))12
MN
(8)
+LU 1
log( log(1/6) N
Proof. Let us directly use Lemma 4 from (Ciosek et al., 2020) Appendix F.2,, with f from the
original formulation being h. and hx, from the original formulation being g n.X
Li
Proposition Let be a Gaussian Process, X be a data distribution, and X == {II,.... NNN X a finite dataset. Assume that our class of approximators approximated by gh,XX so that, for arbitrarily large training sets, the training error is smaller than O. Then, under technical assumptions, for any 8 > 0,
such that for any ~ H, can be
e > O with probability at least holds with probability at least δ that
H. (x*) Exaxx s
Set KN.
(9)
where EN goes to 0 as N co.
28

---

Published as a conference paper at ICLR 2024
Proof. The technical assumptions that are needed are those of Lemma 1 Let hı H be the sample used by sf X and use Lemma 1 for h1. On the right side of Equation we have the training error plus another factor that goes to when N grows, which we denote by K.- Thus, combining [Equation (inequality holding with probability
d) with the assumption on the training error (inequality get that both positive events occur with probability at least in which case we obtain
holding with probability 1 - 2),
1- ?
Bxtna 81
[ M.X (xrt)
Regia [llg (xx) -(*))111]]
N
1
log( log(1/6) 1/0)
(x:) Mxx))1 .O
Set KN.
N
which we wanted to prove.
D.2 ADDITIONAL PROPOSITION ABOUT GP POSTERIOR VARIANCE RULE
In Section 3.2
we provided an intuitive argument for the use of the GP posterior variance rule. Here we present additional proposition that shows convergence to 0 of scores for in-distribution data.
We assume the setting without the observation noise. In such case, the formula for a posterior variance of a Gaussian Process H == GP k) at point x" given dataset (X, y) is follows (Ras mussen & Williams 2006):
σ (Hx)(x)) k(æ", x-)- k(o", -)-
(10)
Proposition 3. Let H be Gaussian Process with continuous kernel function k, X be a dataset sampled according to X, and x² ~ Then the posterior variance a [Hx)(x) goes to 0 in the limit of infinite data.
(x(2x))) Proof. Since k is continuous, so is the function v1x) := k(x", ,x)) ) k(x 3(x",x)kt .x)((, x) k(z, x²) Therefore, for any 0, there exists 8 0 so that [|b((1)) v1(x)) )1 < whenever || x=11 < 6. We also see that
((r) +b(x")1 = [<(x,,,") k(=" )a(ar, vo) k: x:
0|1
which, using equation 10. is posterior variance with respect to the dataset { In the limit of infinite data, X will contain a point zo such that 1|z0 - x² < 8. In such a case, we will have
o (Hx)(==)
(z*) ())==) S a (H(m))) <c
Here, first, we decompose computation the posterior Hx into taking posteriors with respect to {zo} and X { and then we use the fact that the prior variance is not bigger than the posterior variance. The resulting inequality concludes the proof.
E RESOURCES
For rumning experiments, we used Slurm-based elusters with heterogenous hardware, including nodes with A100 GPUs, V100 GPUs, and CPUs only. We note that because of the tngle-pass-through setting and relatively small dataset sizes, our experiments were feasible to run in all of the mentioned hardware setups. To produce our main benchmarking results in Tables and 2 1300 runs needed, each taking on average 21.5 minutes on an 1000 GPU, which makes 466 hours total. During working on the project, we executed over 65K experimental runs.
29


Reviews:
[{"review_id": "gKgplldHVs", "reviewer": "Reviewer_TNjc", "summary": "The authors present a challenging variant of continual learning, known as Class-incremental learning (CIL). The goal in CIL is to learn to distinguish all classes that are introduced incrementally. Existing strategies often exhibit excessive forgetting and score imbalance for classes not seen together during training.\n\nThe authors introduce a novel approach, Prediction Error-based Classification (PEC). Unlike traditional paradigms, PEC calculates a class score by gauging the prediction error of a model trained to mirror the outputs of a static random neural network for that class. This method is likened to a classification rule based on Gaussian Process posterior variance.\n\nPEC offers several practical benefits, such as sample efficiency, ease of tuning, and effectiveness even when data are presented class by class. Empirical results demonstrate that PEC performs robustly in single-pass-through-data CIL, outperforming other rehearsal-free baselines and, in most cases, also rehearsal-based methods with a moderate replay buffer size across multiple benchmarks.\n\nSmall issues:\nIn 4.4: \"We hypothesize this is due to [...]\" - Please re-read the sentence.", "questions": "- In the intro it says: \"Nonetheless, these methods typically perform worse with class-incremental learning than in the easier task-incremental setting\" - How is task incremental easier than class incremental? Is it not simply a more general problem?\n- You say: \"[...] the teacher network’s middle layer is typically wider than the student’s one.\" Why?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- Very well written and easy to follow, despite the complexity. The authors aim to provide intuitive explanations, without sacrificing formal clarity. Thanks!\n- The method offers clear practical advantages of existing alternatives and good performance.\n- Theoretical connections are drawn to substantiate the motivation.\n- Empirical results are convincing.\n- Experimental setup is described in depth.", "weaknesses": "- I previously saw this on Arxiv. This is not a double-blind review. I don't see any guidelines of how to deal with this. Hence, I at least want to mention it here.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SBDoqIendQ", "reviewer": "Reviewer_McZJ", "summary": "The authors present PEC (Prediction Error-based Classification), a method for continual learning and class-incremental learning in which for each class, a student neural network learns to reproduce a randomly-initialized-and-then-frozen neural network only on examples from that class. Continual learning concerns supervised learning where data is not iid and may evolve. Task-based class incremental learning involves single-pass training in stages where only a few classes need to be discriminated between but detecting the union of all classes is required at test phase (in production).  The authors argue that the one-model-per-class approach of PEC prevents forgetting and interference between classes and can handle one-class-at-a-time-presented scenarios. They argue that using a neural net mapping is easier and simpler than creating a generative model for each class.   Theoretical connections between the method and gaussian processes are derived, given known results about the correspondence between very wide neural nets and gaussian processes. Mostly strong experimental results are presented in comparison with 11 competing approaches , although imbalanced training sets can give PEC some trouble", "questions": "Do you have experiments on 0-hidden-layer teachers?\n\nDo you have experiments on sensitivity to initialization? Can you emphasize the importance of sensible initialization in the main body of the paper? Not everyone will know to use an out-of-the-box reliably smart initialization like Kaiming which is designed to get the neural to be living in the \"right\" regions of the activation function.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is extremely well written and clear. The approach is original as far as I know, although I'm not an expert on continual learning. I've refereed for NeurIPS, ICML, ICLR and AAAI for the past several years, and this is one of the best written papers I've refereed.  The concept and motivation underpinning the approach are easy to understand and well presented. There is a nice dovetail between theory and experiment with the correspondence with gaussian processes, and it's particularly theoretically satisfying to see neural net width helping but depth not helping given the theoretical correspondence of GPs with wide neural nets. It's also a  nice counterweight to the prevailing research neural net trends of the past decade, in which the benefits of depth (the D in DNN), i.e. > 1 hidden layer, have been the main focus.\n\nThe experimental results look strong to me but also credible in their honesty, with PEC usually but not always winning.", "weaknesses": "My biggest concern with the paper is a lack of emphasis on or elucidation of a crucial aspect of the approach, namely, the details of the random parameter generation for the teacher neural network.  Algorithm 1 says 'initialize phi with random values' but doesn't say how. I had to dig into appendex B to figure out that default PyTorch initialization was used, i.e.,  Kaiming.   Those of us who have coded up neural nets completely from scratching using only low level programming in e.g. C++ or even Python with only numpy know well that the details of the initialization can make a big difference to the speed and/or success of training, certainly for sigmoidal activations and to some extent for ReLu/GeLu. If the neural net weights are too large relative the input magnitudes, you can get stuck in the saturated, near-constant regions of the activation function and the training error will barely move. If the weights are too small, you can get stuck for a long time in the linear region of the activation function and it can take a long time for training to implement any nonlinearity.  Now, it may well be that Kaiming initialization with GeLu or ReLU and then freezing will ~always result in a reasonable, useful teacher network which successfuly discriminates between classes, but this needs to be emphasized by the authors. I also wish they had done experiments to see how sensitive the results are to the initialization. \n\nI also think this sentence on page 3 is wrong/confusing:\n\n\"As the architectures for g and h, we use shallow neural networks with one hidden layer, either\nconvolutional or linear, followed by a final linear layer\"\n\nWhen I read that in some cases the hidden layer is linear and the final layer is linear, I found it strange...that would mean the whole neural net is a linear mapping and it would seem surprising to me that a linear mapping would characterize the class well enough. However, in Table 5 on page 13, I saw that in fact there is a GeLu layer, so there is some nonlinearity. This should be clarified on page 3.\n\nThat brings up another suggestion: although architecture choices are explored in section 4.5, it would be a nice ablation-ish experiment to see how 0-hidden layers (i.e. purely linear logistic regression) does as a target teacher network.  Is the nonlinearity of the neural net actually important? The GP theory correspondence would suggest yes, but it would be nice to either confirm that or learn from a result of a linear teacher actually being successful,\n\nAnother minor point: I don't think the title conveys the core of the approach very well.  Technically, \"prediction error\" is indeed involved, but prediction error usually refers to a prediction you actually care about intrinsically in the real world rather than predicting a random neural net mapping.  Something along the lines of 'random neural net class modeling for continual learning' might be a better title. That's just my two cents, though, and I'm not even sure you're allowed to change the title.\n\nI'm definitely open to boosting my score with a few tweaks to address concerns.\n\n*** Update after reading rebuttal ***\n\nI appreciate the additional experiments and clarifications regarding initialization and linear ablation studies. I note that random, small magnitude initialization does hurt performance (particularly MNIST) substantially, so I'm glad that initialization is now emphasized more. Accordingly, I have raised my score to an 8. However, I will point out to the Area Chair and any other meta-reviewers that my confidence is only a 3. Reviewer HWad seems to have more familiarity with continual learning and the concerns raised in that review make my slightly nervous. I wouldn't be prepared to fight vigorously for acceptance. Nonetheless, I did increase my score to 8.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "A5zY5vlADV", "reviewer": "Reviewer_pyfp", "summary": "The authors proposed the Prediction Error-based Classification (PEC) method in class-incremental learning, which mitigates the possible issue of data storage in the discriminative classification, and the issue of learning good generative models with limited data. To resolve these issues, the authors proposed to train simple class-wise generative (student) networks, which are used to replicate the results from the frozen random (teacher) network. The decision then is made by choosing the class with the smallest error between a student and the teacher network.", "questions": "* “If, on the other hand, some out-of-distribution input x is considered, then the error … is unlikely to be small” in Section 3.1: Even though seen $\\boldsymbol x$ can push $g_{\\theta^c}$ and $h_\\phi$ to be closer by learning $\\theta^c$, does this necessarily means that the difference $\\| g\\_{\\theta^c}(x)-h\\_\\phi(x) \\|$ for OOD $\\boldsymbol x$ is unlikely to be small? \n* Why does PEC have identical performances on different task splits as in Tables 1 and 2?", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "* The proposed PEC is a novel method as an efficient alternative to the generative models in class-incremental learning.\n* The extensive experiments support the good performance of the proposed method in single-pass-through-data class-incremental learning.", "weaknesses": "* Since the student networks in the proposed method are decoupled from each other, the method ignores the intrinsic interdependence among classes.\n\n* The \"efficiency\" claimed in the contribution is less empirically supported.  Furthermore, due to the individual design for student networks, is this a good plus when the number of classes increases?\n\n* The notations are bad. In algorithms, the number of classes is $N$ but later it is denoted by $C$ in equation (1). In proposition (2), I believe $N$ denotes the sample size but it was denoted as the number of classes in Algorithm 1.\n\n* The proposition (2) is problematic. \n  * The $\\epsilon$ should relate to $\\gamma$ and $N$, otherwise, it sounds your approximator can achieve any $\\epsilon$ under the current samples and probability tolerance. Additionally, why there is no $\\gamma$ in the proof of proposition (2)?\n  * As the authors pointed out $H$ is the Gaussian process which is approximated/replaced by the network $h$. Thus, there should be another term to quantify this approximation error, which is dependent on the width of the neural network. However, I cannot see anything implying this information in the theorems.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BKmUJ3220j", "reviewer": "Reviewer_HWad", "summary": "This paper introduces a method called Prediction Error-based classification (PEC) for Online Class-Incremental Learning. Unlike existing methods in the continual learning literature, relying upon discriminative and generative classification paradigms, it presents a novel classification approach for continual learning. The method involves fitting a shallow predictor network for each task class to match a single random prior network. At inference, the final classification is determined by selecting the class with the smallest error between all the fitted models up to the current task and the output representation of the random prior network. The authors emphasize that the classification rule they employ is theoretically justified as an approximation of a rule based on Gaussian Process posterior variance. This approach operates in an exemplar-free scenario, doesn't rely on a pre-trained network, and can handle the  problem of having a single class per task. The authors conducted empirical experiments, showing  that their approach enhances performance compared to exemplar-free baselines and exemplar-based methods relying on generative or discriminative classification paradigms. The authors highlight that their approach exhibits class-imbalancing issues, and they propose two methods to mitigate this problem: Buffer Balancing, which relies on an evolutionary optimization algorithm and an exemplar memory buffer, and Equal Budget, which equalizes the number of training iterations for different classes.", "questions": "My major concerns and questions are in the weakness section. However, there are other experiments that can further support the evaluation of the proposed approach and can help understand the performance of the  proposed method: \n\n* **Ablation on the proposed method - PEC vs Ensemble Discriminative Classifiers**: PEC takes inspiration from Random Network Distillation (Burda et al. 2018), originally  designed for detecting novel states in reinforcement learning, and from work of Ciosek et al. 2020. [q], which focused on  uncertainty estimation for out-of-distribution (OOD) detection. Notably, Ciosek et al. demonstrated that Random Priors outperformed Network Ensembling. PEC adapts the concept of Random Prior to the  incremental learning setting, using a Random Prior Network and training a single prediction network per class. This naturally leads to the question of how well their proposed method performs in comparison of  ensembling with multiple discriminative classifiers. Ensembling multiple discriminative classifiers involves training small networks  equipped with a Softmax Layer for each task, and then taking the argmax of the prediction across the models. This evaluation can be conducted with multiple classes per task, which is relatively easier than the single-class per task scenario but quite common in the online continual learning community. To perform these experiments requiring a network per task, an architecture with the structure similar to the one provided by Zenke et al. ([d] Appendix A) can be employed. \n\n\n* **Ablation on the proposed method - PEC vs Discriminative Classifier**: Although the reduced Resnet18 is used for comparison (as mentioned in the first point of weakness section), it is essential to note that training a deep network is inherently more challenging than training a shallow network, as performed by PEC. Additionally, when using a reduced ResNet18 architecture, a state-of-the-art online method should address the issue of Batch Normalization, which is currently a challenge in online class incremental learning [p]. PEC may have an advantage in this context since it employs shallow models for which batch normalization is not necessary.  To gain a deeper understanding of how well PEC performs compared to the discriminative methods, it would be valuable to evaluate some of the sota  methods (exemplar-free and exemplar-based) on SplitCifar100 or minImagenet datasets without using ResNet architecture. An option is to use the network provided  by Zenke et al. [d](Appendix A), not using batch normalization.This can help eliminate any bias stemming from architecture-specific issues in  incremental learning providing a better understanding of how well Prediction Error Based Classification performs in comparison to Discriminative classification.\n\n\n\n\n**Summary Of The Review**\n\nThe paper introduces an adaptation of the Random Prior Network for Online Continual Learning. The theoretical foundation of this adaptation is not entirely novel, as it relies on the theory presented by Chosek et al. [q]. However, the application of the theory to the PEC classification framework in the context of online continual learning is interesting. \n\nUnfortunately, I have significant concerns  pertaining the experimental evaluation. The paper lacks a comprehensive and properly aligned comparison with the current literature on online continual learning, as mentioned in Weakness Section Points 1, 2, and 3.  Additionally, the paper does not provide  comprehensive assessments of Memory Impact and FLOPS Comparison, as noted in Weakness Section Point 4. Finally to enhance the quality of the experimental evaluation and to understand how much the proposed method works well, it would be beneficial to include a more thorough ablation study of the proposed method, as suggested in Question Section Points 1 and 2. \n\nTaking into account the above considerations, I believe that the paper still needs substantial work to be carried out, before it can be considered for publication. I thus believe that it shall not be accepted at the current issue of ICLR.\n\n**References**\n\n[d] Zenke et al. (2017). Continual Learning Through Synaptic Intelligence. Proceedings of machine learning research. \n\n[p] Q. Pham, C. Liu, H. Steven, \"Continual Normalization: Rethinking Batch Normalization for Online Continual Learning,\" in International Conference on Learning Representations, 2022.\n\n[q] Ciosek, Kamil et al. “Conservative Uncertainty Estimation By Fitting Prior Networks.” International Conference on Learning Representations (2020).", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "*  The introduction is well-written and they clearly motivate why they propose prediction error-based classification.\n*  The novel usage of a Random Prior Network and multiple predictor network for online class incremental classification. \n*  The PEC's ability to work in exemplar-free, from scratch, one class-per-task settings is a valuable feature. These settings, especially the combination of these ones, are infrequently assessed by the literature.", "weaknesses": "The major weakness of this paper lies in the experimental evaluation where the authors assess the performance of the proposed method:\n\n\n* **SOTA Architecture Comparison**:  The  authors have designed the PEC predictor models to align  with the parameter count of reference literature networks.  Specifically for the evaluation of SOTA methods, the PEC authors employ simpler networks architecture for  SVHN, MNIST, while they employ the standard Resnet18  for Cifar10, Split-Cifar100 and the miniImagenet dataset. These latter tasks are considered more challenging in comparison to SVHN and MNIST.  My primary concern revolves around the use of the standard Resnet18 that the authors employ for evaluating SOTA methods. Training the standard Resnet18 can be difficult, especially when the number of training iterations is limited (i.e. in online setting), primarly due to the large number of parameters (11.2M as reported in the  table 6 of the Appendix B.1 of the paper). In response to this challenge, recent sota online approaches rely upon a **reduced** version of Resnet18, which consists approximately on 1.1M of parameters [g] [h] [i] [l] [m] [n]. This reduced network is commonly favored for online continual learning due to its computational efficiency and ease of optimization. Utilizing the reduced Resnet18 as a reference is essential to ensure a fair comparison with the existing literature, especially since PEC uses shallow models easier to train than Resnet18. It's worth noting that doing so would require the overall number of parameters in the PEC models to be reduced to 1.1 million to match the reference architecture.\n\n\n* **Discriminative Exemplar-Free Class Incremental (EFCIL) Comparison**: Comparing PEC, an online exemplar-free method, with other common exemplar-free approaches is crucial. In the main paper, only EWC [a] and Label Trick[b] are evaluated, with EWC designed for offline continual learning and not suitable for online settings. Other common weight regularization methods, such as MAS [c] and SI[d], are typically evaluated in the online continual learning literature, since they allows to compute online the weight importance. Moreover, LwF[e], regularizing output with knowledge distillation, is known to outperform weight regularization methods and is suitable for online settings.The absence of this comparison is noteworthy. The only other baseline provided by the authors in the main paper is \"Label Trick\"[b], involves training only the head of the current task. Nowadays, this is a common practice in exemplar-free approaches, since it has been shown to work better. All the exemplar-free implemented methods (included LwF) on FACIL (Masana et al. [f]) use this practice. All the exemplar-free method reported in the main paper must apply this trick. Moreover, in Appendix C.1.1, various EFCIL methods are presented, and it might be beneficial to move some of them to the main paper, implementing the Label Trick where applicable for a more comprehensive evaluation.\n\n\n \n* **Discriminative Exemplar-Based Class Incremental (EBCIL) Comparison**: PEC is an **Online** method. When comparing it with exemplar-based method, it is essential to align the compared methods with the current **Online** EBCIL literature. ER, A-GEM, DER++, ICARL, already reported in the main paper,  are common comparison in the online literature. On the other hand, BiC is an **offline**  method. This is because it requires additional epochs on a validation set to calibrate the classifier(thus violating online incremental constraints). X-DER, a recent work, is original designed in the multi-epoch setting (i.e. offline) but has been adapted in this paper to work in online setting. To ensure a comprehensive comparison with the current online literature, it's important to consider additional relevant approaches.  A straightforward  baseline to provide is \"ER + LwF\". Notably, ER-ACE [g] has recently delivered state-of-the-art results in online continual learning, so it is advisable to include this comparison in the main paper. Other approaches that should be evaluated include SCR [l] and RAR [m]. For a more extensive list of relevant comparison, you can refer to  Cormerais et al. [n].\n\n* **Memory Requirements & Inference FLOPS Computation**: Regarding the **Memory Requirements**, PEC requires a linear increase of parameters with the number of classes encountered. While the authors acknowledge this intrinsic challenge in their approach and have discussed it in the limitations section, a more detailed analysis is necessary. Specifically, it's essential to examine how many parameters are utilized by each predictor networks per dataset and by the teacher network, to understand how much the memory increases across the tasks. As for the **FLOPS comparison**, the authors have not provided an analysis of the number of Floating Point Operations (FLOPS) required by their model. Since online methods are intended to work in environments with hardware resource limitations, it is advisable to disclose the number of FLOPS for both backward and forward passes. For training, FLOPS evaluation can be carried out, as demonstrated by [o]. As regards the inference step, in Section 3.1 the authors say that only two forward passes are needed to obtain the classification scores – one for the a merged student, and one for the teacher network. While this is true, considering the online nature of their method and its intended use in resource-constrained settings, it is crucial to provide and compare the overall number of FLOPS required by the forward pass for both teacher and student networks. Therefore, it is necessary to determine how much the FLOPS of PEC method increases compared to current state-of-the-art methods \n\n\n* **Buffer Balancing and Equal Budget**: The authors assert that their approach is afflicted by class imbalancing.However, they do not provide any theoretical motivation for this claim; they hypothesize that the issue may be linked to the number of gradient updates, as mentioned in Section 4.4 of the main paper. In cases where no theoretical motivation is available, it is necessary to conduct a detailed analysis that elucidates the factors contributing to this behavior. To address the class imbalancing issue, they authors introduce two strategies in the experimental section: Buffer Balancing and Equal Budget Strategies. While the easier Equal Budget Strategy it is well-explained in the paper (and in supplementary Section B.2), the **Buffer balancing**, based on CMA-ES optimization is considerable more complex and difficult to grasp. Therefore, additional details on how this optimization algorithm is used for online-class incremental learning should be provided to enhance understanding. Finally, a natural question arises: How well do existing methods perform in the class imbalancing setting compared to PEC? \n\n**References**\n\n[a] James Kirkpatrick et al. \"Overcoming catastrophic forgetting in neural networks,\" in Proceedings of the National Academy of Sciences, 2017.\n\n[b] Chen Zeno  et al, \"Task Agnostic Continual Learning Using Online Variational Bayes,\" 2019.\n\n[c] T. Aljundi et al. , \"Memory Aware Synapses: Learning What (not) to Forget,\" in  ECCV 2018.\n\n[d] Zenke et al. (2017). Continual Learning Through Synaptic Intelligence. Proceedings of machine learning research. \n\n[e] Z. Li, D. Hoiem. \"Learning without Forgetting,\" in IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018.\n\n[f] Masana, M. et al. \"Class-Incremental Learning: Survey and Performance Evaluation on Image Classification,\" in IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.\n\n\n[g] Lucas Caccia  et al, \"New Insights on Reducing Abrupt Representation Change in Online Continual Learning,\" in International Conference on Learning Representations, 2022\n\n[h] Aljundi et al. , \"Online Continual Learning with Maximal Interfered Retrieval,\" in Advances in Neural Information Processing Systems, 2019.\n\n[i] Chaudhry, A., et al. \"Continual Learning with Tiny Episodic Memories\" 2019.\n\n[l] Zheda Mai et al. \"Supervised Contrastive Replay: Revisiting the Nearest Class Mean Classifier in Online Class-Incremental Continual Learning,\" in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW),  2021\n  \n[m] Kumari, L., et al, \"Retrospective Adversarial Replay for Continual Learning,\" in Advances in Neural Information Processing Systems, 2022 \n\n\n[n] Soutif-Cormerais  et al, \"A Comprehensive Empirical Evaluation on Online Continual Learning,\" in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2023,\n\n[o] Ghunaim, Y., et al, \"Real-Time Evaluation in Online Continual Learning: A New Hope,\" in Conference on Computer Vision and Pattern Recognition (CVPR), 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)