Continual Learning in Deep Networks: an Analysis of the Last Layer
Authors: Timothee LESORT
Thomas George
Irina Rish
Venue: ICLR 2022 Submitted (2022)
Keywords: Continual Learning
Linear Models

Abstract: We study how different output layers in a  deep neural network learn and forget in continual learning settings. The following three  factors  can affect catastrophic forgetting in the output layer: (1) weights modifications, (2) interference, and (3) projection drift. In this paper, our goal is to provide more insights into how changing the output layers may address (1) and (2).  Some potential solutions to those issues are proposed and evaluated here in  several continual learning scenarios. We show that the best-performing type of the output layer  depends on the data distribution drifts and/or the amount of data available. In particular, in some cases where a standard linear layer would fail, it turns out that changing  parameterization is sufficient in order to  achieve a significantly better performance, whithout introducing a continual-learning algorithm and instead using the standard SGD to train a model.  Our analysis and results  shed light on the dynamics of the output layer in continual learning scenarios,  and suggest a way of  selecting the best type of  output layer for a given scenario.

Under review as a conference paper at ICLR 2022
CONTINUAL LEARNING IN DEEP NETWORKS: AN ANALYSIS OF THE LAST LAYER
Anonymous authors Paper under double-bling review
ABSTRACT
We study how different output layers in a deep neural network learn and forget in continual learning settings. The following three factors can affect catastrophic forgetting in the output layer: (1) weights modifications.
(2) interference, and (3) projection drift. In this paper, our goal is to provide more insights into how changing the output layers may address (1) and (2). Some potential solutions those issues are proposed and evaluated here several continual learning scenarios. We show that the best-performing type of the output layer depends on the data distribution drifts and/or the amount of data available. In particular, some cases where a standard linear layer would fail, it turns out that changing parameterization is sufficient in order to achieve significantly better performance, whithout introducing SGD to train a model. Our analysis and results shed light on the dynamics of the output layer in continual learning scenarios, and suggest way of selecting the best type of output layer for given scenario.
continual-learminn algorithm and instead using the standard
INTRODUCTION
Continual deep learning algorithms usually rely end-to-end training of deep neural networks, making them difficult to analyze given the complexity (non-lined non-linee,r,ty, non-convexity) the training dynamics. In this work, we instead propose evaluate the ability of network to learn forget in simplified learning setting: We decompose our model as a feature extructor part consisting of all but the final layer, and classifier part which is given by the output layer. Using this decomposition, can isolate the role of the output layer parameterization in continual learning scenarios.
Recent works on continual learning (CL) point out some drawbacks of parameterizing the output layer as linear layer, especially in incremental settings (Wu et al. 2019 Zhao et al.) 22200) Hou et al. 2019). Among these problems is the unbalance of bias and/or norm the output layer's 's vectors. These works propose some solutions but do not study them independently from the feature extractor. Indeed, CL scenarios, the function learned by the feature extractor changes, and therefore the embedding space changes accordingly; we call this change the projection drift. This drift characterizes the change through time of the embedding on given observation.
Zhao
By isolating the output layer from the feature extractor, we can study it in controlled environment, i.., without projection drifts. Decoupling the feature extractor from a sub-network has shown to be helpful, e.g., in reinforcement learning (Lesort et al. 2018 Raffin et al., 2019 Stooke et al. 2020).
We now list our contributions:
We propose evaluation of large panel of output layer types in incremental, lifelong, and mixed continual scenarios. We show that depending on the type of scenario, the best output layer may vary.
We describe the different sources of performance decrease in continual learning for the output layer: forgetting, interference, and projection drifts.
We propose different solutions to address catastrophic forgetting in the output layer: simplified weight normalization layer, two masking strategies, and an alternative to Nearest Mean Classifier using median vectors.

---

Under review as a conference paper at ICLR 2022
2 RELATED WORKS
In most deep continual learning papers, the last layer of the deep neural network is implemented as a linear layer as typical classifier architectures in i.i.d. settings such as VGG (Simonyan & Zisserman 2014), ResNet (He et al.) (016), and GoogleNet (Szegedy et al. 2014). Recently, several CL approaches questioned this approach, showing that the norm or the bias can unbalanced for the last classes observed in class-incremental settings. For example, BIC (Wu et al. (019)) proposed to train two additional parameters using subset of the training dataset after training the rest of the model to correct for the unbalance of the bias. In the same spirit, (Zhao et al. 2020) proposed to compute a Weight Aligning value to apply to vector to balance the norm of past classes vectors and the new classes vectors of linear layer. (Zhao et al. 2020) also experiments with a layer, the weight normalization layer (Salimans & Kingmal (016)) that decouples the norm of the weight matrix from its direction in order to normalize the vector of linear output layers. (Hou et al., 2019; Caccia et al.) 2021 proposed to apply a cosine normalization of the layer to avoid the norm and bias unbalance.
Linear Discriminant Analysis (LDA) Classifiers in incremental settings have been used for dozens of years (Aliyari Ghassabeh ct al., 2015 Chatterjee & Roychowdhury 1997, Demir & Ozmehmet 2005). The streaming version SLDA from (Shaoning Pang et al. (2005) has recently been revisited in deep continual learning in (Hayes & Kanan 2020). SLDA combines the mean and the covariance of the training data features to classify new observations.
more straightforward approach, as implemented in iCaRL (Rebuffi et al. (017) is to average the features for each class and apply nearest neighbor classifier also called Nearest Mean Classifier (NMC).
In this work, we study the capability of those various types of top layers with a fixed pre-trained model in incremental and lifelong settings as defined in (Lesort et al., 202T]. Continual learning research field studied many strategies, e.g. rehearsal, generative replay, dynamic architectures or regularization. In this study, we do not aim at modifying the training process or proposing a new one. We study how different layer types learn in various continual scenarios without CL approach.
OUTPUT LAYER TYPES
We here review the output layer parameterizations studied in this work. We introduce the notations, and present the main characteristics of linear layers. Then, explain how standard layer parameter- ization struggles to learn continually, and we present different parameterizations to overcome the problems. Finally, present several other types of output layers not trained by gradient descent.
3.1 NOTATIONS
We will study functions fo(:) parameterized by a vector of parameters 0 representing the set of weight matrices and bias vectors of a deep network. In continual learning, the goal is to train fa(-) on a sequence of task [To, T .... I)- such that v(Ie, Vt) ~ Te (t E [0, 1)). fo (x) y.
We can decompose fe(-) into two parts, (1) a feature extractor dig (-) which encodes the observation I, into feature vector 2,, (2) output layer gut (-) which transforms the feature vector into non-normalized vector or (the logits). 0 and 6 are two complementary subsets of e.
3.2 LINEAR LAYER
A linear layer is parameterized by a weight matrix A and bias vector b, respectively of size N x and N, where h is the size of the latent vector (the activations of the penultimate layer) and N is the number of classes. For = a latent vector, the output layer computes the operation o = Az + b. We can formulate this operation for single class with (z, A,) + by = or, where (-) is the euclidean scalar product, A, is the ith row of the weight matrix viewed as a vector and by is the corresponding scalar bias. can be rewritten:
| cos(<(2,, + b, o,
Where 2(.-)) is the angle between two vectors and ||-||| denotes here the euclidean norm of vector.
Eq. [ highlights the components that need to be learned to make a correct prediction ÿ = argmax,(o:): The norm of A, its angle with the latent representation, and the bias. In particular,
2

---

Under review as a conference paper at ICLR 2022
if the training dynamics makes particular ||A,]] b, larger than others, then the network will be considerably biased into predicting the class 2. In incremental learning, the vectors A, and values b are learned sequentially, hence, the model will grow the ||AA and b, for current classes and reduce them for past classes to ease current task ( Figure I. This phenomenon may lead to forgetting.
3.3 LINEAR CLASSIFIESS IN CONTINUAL LEARNING AND REPARAMETERIZATION
In continual learning, assuming that the true function f(x) = y fixed, there are two main cases of data distribution drift (see e.g. Lesort et al. (20TT): either we get new data of known classes (domain drift - lifelong scenario or receive new data from new classes (virtual concept drift- - incremental scenario). For a linear output layer, forgetting is caused by different mechanisms in both scenarios.
In incremental learning, the vectors A, and b; are learned sequentially (one by one or set by set). When learning new A, (and b,), the training algorithm does not have access past classes, and hence cannot ensure that learning weight vectors for new classes does not interfere with past classes with eq. L As an illustration, suppose two classes alike in the latent space. In that case, their respective latent vectors might be similar, and interference might happen between the two classes (see fig. 2 fortiori if norm and bias are not balanced.
Figure I: Illustration of norm and bias unbalance at the end of a CIFARIO continual experiment. tasks with 2 classes each. (left) the norm of each output vector at the end of the last task, (middle left) bias at the end of the last task, and (middle right) the difference between the last task's 's bias and current bias. We can see a clear unbalance in norm and bias for the last active vectors (classes 8 and 9). The middle right figure shows us that the imbalance of bias is primarily due to the modification of bias from the previous task (classes 6 and 7). (right) Illustration of Forgetting: we plot the difference between the first 50 weights after task 44 and after task 45 in the scenario Core 10Mix. Task 45 is composed of one class different from 44's one. Those two figures illustrate the impact of the masking. Indeed it avoids the modification of weights of other classes.
One solution found in the bibliography to avoid interference is to counteract unbalance between the norms of vectors or bias. We can modify eq. T in several ways mitigate such unbalance:
Removing the bias (Linear no bias layer):
|| ||,1| cos( <(2), o:
(2)
Normalizing output vectors (WeighuNorm layer): - |=|| cos (((z; A:))
(3)
Measuring only the angle (CosLayer): + cos(<(2,A)) A:)) =
(4)
WeightNorm (eq. 3 is similar to the original WeightNNrrm layer (Salimans & Kingma (2016), here denoted by Original WeightNorm) experimented in Zhao al. (2020) continual learning context:
7 ||||||| cos( ((,, A.)) + b, 0,
However, in the original WeightNorm, the additional scaling parameter y and the bias b are learned during training. These parameters are akin to the parameters in BatchNorm layers (Toffe & Szegedy 2015), which have been shown to hurt the training of intermediate layers in continual learning (Lomonaco et al. 2020). Hence, our proposed WeightNorm layer (eq. B) avoids such interference by ensuring a unit norm for all vectors and removing bias and gamma parameters.
Forgetting might also be caused by weight modification: this happens when the optimizer modifies weight vectors from past classes that strategies (e.g. EWC (Kirkpatrick et al. 2017), KFRA (Ritter et al. 2018)) aim at avoiding such forgetting by penalizing modification of important parameters.
present in the current task. In the literature, regularization
We introduce a more radical strategy to avoid that the update on a specific class affects another past one. The strategy consists in masking some classes during the update step. We propose two types of
3

---

Under review as a conference paper at ICLR 2022
masking: in single masking, we only update weights for the output vector of the true target and in group masking, we mask all classes that are not in the mini-batch. With the following strategies, the update step cannot change A, and b; of past classes and avoid sub-sequential interference. In Figure I we illustrate the impact of single masking in a task with only one class from one experiment the paper. In incremental scenarios, all masking strategies can also be seen as regularization strategy that strictly forbids the modification of past weights in the last layer.
The masking strategy resembles the update step in multi-head architecture (van de Ven & Tolias 2019) for gradient descent since in this case, the gradient is computed only for a subset of selected classes. However, in our setup there is only one head, and the inference is achieved without the help of any external supervision, such as a task label.
EICE
(a) Angles between vectors
(b) Mean angles vectors-data.
(c) Interference risk
Figure 2: Illustration of interference for the linear layer: We plot (left) the angles (in degree) between the output vectors A,, (middle) the mean angle between the latent vectors of each class and the A; vectors and (right) ratio between the mean angle with wrong classes, and the mean angle of the target class: high value indicates a higher risk of interference. In this experiment, there is a high risk of interference between examples for class 4 and examples of class or 3, as pictured by dark red cell. See appendix F for a similar experiment for WeightNorm.
In lifelong learning. forgetting happens when new instances of previously learned class make the output layer forget features that were important for past instances or modify the direction of the output vectors. However, those problems are usually less prone catastrophic forgetting but may still have a significant impact in complex scenarios. One of the reasons that catastrophic forgetting is less important that all classes are available simultaneously, which makes it possible for the output vectors to be coordinated and avoid interference.
3.4 SIMILARITY-BANE BASEDD CLASSIFIERS (TRAINED WITHOUT GRADIENT)
To avoid the interference and weight modification, we can use another type of layers that rely on the similarity between training latent vectors and testing latent vectors.
One of the most popular classifiers that can easily be adapted into a continual or online learning setting the k-nearest neighbors (KNN) classifier. This classifier searches for the nearest exemplars of given observation in the training set and predicts the class based on the class of the k neighbors. This classifier's 's aaper-parameter are the number of neighbors and the distance used (here the usual euclidean norm).
lighter strategy is the nearest mean classifier (denoted MeanLayer) as proposed iCarL (Rebuffi et al. 2017), this strategy consists of saving the mean of the features of each class & while learning (called a prototype (4) and predict the class given by the nearest mean for test latent vectors.
In this paper, we propose to evaluate the performance of median classifier (denoted MedianLayer), where we replace the mean of the features with the median of the features. The median vector is computed by selecting the median value each dimension. The idea behind this approach is that the prototype might be more central in the feature distribution and maybe avoid interference with similar classes more effectively.
An alternative is the linear discriminant analysis classifier, which uses both the mean of the features and a covariance. The online version of this algorithm is streaming linear discriminant analysis
4

---

Under review as a conference paper at ICLR 2022
(SLDA) (Shaoning Pang al. 2005). We used the approach with online updates of the covariance matrix of (Daguutt & Hsu 2007) as in (Hayes Kanan 2020) (see appendix B)
Similarity- based classifiers very effective in the literature. However, they cannot be directly used to train the feature extractor since they are not differentiable. Hence, they need to store samples from past tasks as rehearsal strategies or be applied in setting without projection drift.
In this section, we introduced the different output layer types that we will use in experiments. We introduce three modifications: (1) simplified WeightNorm (referred to as WeightNorm), (2) two masking strategies, single masking and group masking, compatible with any reparameterization of the linear layer, and (3) a Nearest Median Classifier.
EXPERIMENTS
In this section, we will review how the different output layer types, presented in section B can solve various continual scenarios. We isolate the output layer from the remaining of the neural network by using frozen pre-trained model. This methodology consists of creating a scenario where no projection drift occurs. Indeed, projection drifts may only lead Hence our study can eliminate approaches that would, in all cases, fail under such conditions. Our findings can also be directly transferred settings where the projection drift is negligible.
deterioration of performance.
In a preliminary experiment, we benchmark all output layer types in an i.i.d. setting to fix some of their hyper-pare parameters for further continual training. The conclusion of these experiments are in section 4.1 but the full experiments are in appendix section[[aand in Table 3
In a second setup, we train the various output layers in continual learning scenarios. Finally, we experiment with vanilla continual approach close to rehearsal approach: we train using only subset of the training data, and compare layer parameterizations in this low data regime.
4.1 DATASETS AND PRE-TRAINED MODELS
We conduct our experiments on CIFARTO/C CTFARTO/CIFAR1O (Krizhvssky & Hinton 2009) and Core50 (Lomonaco & Maltoni 2017). Core50 is a dataset composed of 50 objects from 10 categories (e.g. glasses, phone, cup..., filmed 1l environments (8 for training and 3 for evaluation). We denote Corel0, the 10 classes version where we only predict the category, and Core50, the full 50 classes version. We use both to create various types of scenarios.
The preliminary experiments consist of testing the basic output layer (without masking) to select learning rate (among 0.1, 0.01 and 0.001 and architecture for pre-trained models (among ResNet, VGG16 and GoogleNet available on the torchvision library (Paszke al. 2019)). While choosing pre-trained models, we made sure that pre-training had not been done using the same dataset that we used in our continual learning experiments (e.g. we did not choose a model pre-trained on CIFARIO for our CIFARIO 10 experiments), which would otherwise be "cheating" as the pre-training would have been made using all available data, and not been restricted data sequentially made available as in continual learning.
Following the results (appendix Cp. we choose a learning rate of 0.01 for the original linear layer and original linear layer without bias (Linear_ne _no bbas) as well as for their masked counterpart and 0.1 for the other layers. Secondly, we found that for Core50 and Core experiments, the ResNet model performs best. We also eliminated the CIFAR 100 dataset from our incremental experiments because the pre-trained model we tested (pre-trained on CIFAR 10) did not permit to achieve a good enough accuracy. We kept CIFARIO, Core 10, and Core50 for subsequent experiments. We added CUB200 and a modified version of CIFAR 100 (referred to as CIFAR PEFARROOLifelongg for continual experiments. Moreover, we that 5 epochs per task are sufficient these datasets, and will keep this number of epochs for all experiments
4.2 CONTINUAL EXPERIMENTS SETTINGS
In these experiments, we evaluate the capacity of every output layer to learn continually with fixed feature extractor. We evaluate the CL performance virtual concept drifts (incremental scenario),
5

---

Under review as a conference paper at ICLR 2022
on domain drifts (lifelong scenario) (cf section 3.3
Incremental settings consist of sequence of tasks where all new tasks bring new unknown classes. Lifelong settings consists of a sequence of tasks where each new task brings new examples of known classes. We also evaluate the best layers in a mixed scenario with both kinds of drifts in the end. Our experiments use the single-head framework (Farquhar& & Gal 2018), as opposed to multi-head where the task id is used for inference to pre-select a subset of classes. We do not use task labels during training nor testing. The change in labels can signal a change of tasks. However, it does not trigger any continual mechanisms in our experiments, and all training are done using stochastic gradient descent with a momentum of 0.9.
Incremental Scenarios: CIFAR1O, Core50, CUB200: Incremental scenarios are characterized by a virtual concept drift between two tasks. These settings evaluate the capacity of learning incremental new classes and distinguishing them from the others. The core50 scenario is composed of 10 tasks; each of them is composed of 5 classes, the CIFAR 10 scenario is composed of 5 tasks with 2 classes each, and the CUB200 scenario is composed of 10 tasks of 20 classes each.
Lifelong settings are characterized by a domain drift between two tasks. The classes stay the same, but the instances change. These settings evaluate the capacity of improving at classifying with new data. The Core10Lifelong scenario is composed of 8 tasks with 10 classes in a given environment, each new task, we visit a new environment. CIFAR100L Data are labeled with the coarse labels of CIFAR1O0. However, data are shared between tasks using the original label to ensure domain drift between tasks (More detail in appendix in section (A).
Lifelong Scenarios: Core 10Lifelong, AIFARI00Lifelond
is composed of tasks, with 20 labels each that also stay the same.
Mixed Scenario: Core10Mix: In this scenario, a new task is triggered by either a virtual concept drift or a domain drift. In this case, the domain drift is characterized by the transition from one object to another in the same category. This scenario is composed of 50 tasks, where each task corresponds to new object (with category annotation) in all its training domain. So new task is always a new object, but it is either data from new class or from a known class visited through another object.
The results for those experiments are in section 5.1 over the whole test set of the scenario (with data of all tasks). This measure makes it possible to report the accuracy on a fixed test set and makes the visualization of progress in solving the complete scenario easier. For easy reproducibility, we put in appendix H how the scenarios are created with the continuum library (Douillard Lesort 202T).
We report at each epoch the average accuracy
4.3 SUBSET EXPERIMENTS SETTINGS
In many continual learning approaches, a subset of the training is saved, either for rehearsal purposes or for validation purposes (Buzzega et al., 2020 Douillard et al. 22000] Prabhu et al.] (020).. In this experiment, we evaluate the capacity of each layer be trained from scratch with such subsets. It could be more efficient to continually train the feature extractor in a continual end-to-end setting and learn a suitable output layer afterward from subset of data. This procedure is very fast, and it avoids all problems that projection drifts or any other drift can create. The only (not so simple) problem to solve is which examples store and how many. Nevertheless, the successes in few-shot learning (Lake et al.) 2011 Fei-Fei et al. 2006; Wang et al.) a (019)) or fast adaptation (Caccia et al.) 2020) show that when the feature space good enough, only a small set of data is enough to identify a class.
5 RESULTS
5.1 CONTINUAL EXPERIMENTS
We split the results of continual experiments into two groups of figures; the first group (Fig. 3 only compares the various parameterizations of linear layer presented in section 3 The second group (Fig. 4h) compares the best performing layers of the first group (CosLayer and WeightNorm) to layers that are not trained by gradient descent, namely: KNN, MeanLayer, MedianLayer, and SLDA. The results presented in fig. 3 and fig. lead us to the following conclusions:
Among the reparameterizaations of linear layers, Coslayer and WeightNorm are the overall best performing. Indeed, in incremental settings (figs. 3a 3d Be). they are always among the best performing methods. Both of them only rely on the angle between data and output vectors, which

---

Under review as a conference paper at ICLR 2022
Linerr
We chtNormm
bias Mesked
Orgnstiveghtlomm Mesked
GMasked
Lineernng, bies
MiginntMeighttoom Liner Masked
CosLayer, Masked
Linear GMasked Lnner bias _MMasked
WeightWorm, Norahi
Castayer
weightNorm Masked
Orfginatwwightoomm (aMaseed
02
03
62
(a) CIFARIO, 5 tasks
(b) Core Corc 10Lifelong 8 tasks
(c) Core50, 10 tasks
03
03
03
14
(d) CIFAR100Lifelong, 5 tasks
(c) CUB200, 10 tasks
(f) Core1OMix tasks
Figure 3: Experiments realized different task orders. We plot the test accuracy on the full test set for each epoch. We compare the different parameterizations of the linear layer. A vertical line represent a task transition. The notation Masked denotes layer trained with single masking while .GMasked denotes layer trained with group masking
reduces the risk of interference as described in section 3 that in WeightNorm, the predictions are weighted by the norm of the latent vectors (ef Coslayer eq. WeightNorm eq.
The only difference between these layers
3) which scales the gradients thus may change the learning dynamics.
Masking is efficient in incremental settings to avoid weight modifications. In incremental sce- narios (figs. Bal 3c 3eb the best performing methods are all masked. Except CUB200 (Fig. (e) where (only) the WeightNorm layer outperforms the masked layers. It shows that in incremental settings, avoiding weight modification with masking can significantly improve results. However, single masking" creates instability in lifelong scenarios (figs. 3d] (b). Masking may prevent good organization of output vectors together and make the training unstable.
Removing the bias the linear layer has effect. We did not observe any improvement by removing the bias from the linear layers in our experiments (Linear and Linear Leraar_n_____ee _o__ias almost coincide). This questions Wu et al. (2019)'s observations that correcting the bias can mitigate catastrophic forgetting. Nevertheless, we hypothesize that the effect of the bias in continual learning depends on the task hand.
Similarity- based layers perform well regardless of the data drift type (fig. 4 These layers are easy train since they need to see the data only once achieve their best performance. In fig. these layers look to perform very well in all settings in comparison with linear layers, especially SLDA. Nevertheless, we should not forget that they cannot be used to train an end-to-end neural network. They could, however, be associated with another layer to train the feature extractor as in iCARL Rebuffi et al., 2017).
In lifelong, we cannot avoid weight modification with masking. In lifelong scenarios, tasks use the same output vectors, i.e., the same weights; therefore, freezing weights forbids further learning. In Core10Lifee 10Lifelong setting (fig. 3b), weight modification does not look to be problem, and no decrease in performance indicates catastrophic forgetting, while in CIFAR 100Lifelong (fig. (3d] all layers seem to forget while learning new task. This phenomenon is due to the similarity of the different modes
We removed the layer masked by group in the lifelong experiment because it is useless when all classes available simultaneously.
7

---

Under review as a conference paper at ICLR 2022
Table Subset experiments: Mean Accuracy and standard deviation on runs with different seeds.
Outl Layer
Dataset \ Subset
100
200
500
T000
All
CosLayer
Core50
10.30#1.98
12.43+3.06
18.11
48
53.9320.90
WeighiNorm
Core50
32. 22.06.2..0 99
44.3211.92
61.9111.10
69. 1..877
77.0410.33
Neighttomm
Core50
62
45.6811.96
62.3911100
69.1740.65
76.8510.61
CosLayer- Massed
Core50
30.62+2.57
41.22+1.68
56.47+1.18
63.10+0.c.
63.4345.78
WeightNorm--N Masked
Core50
12.9512.0r
14.98
21.1311.2
23.42.12.00
31.18117.00
Origina/WeighhNorme Masked
Core50
12.27 1/27712.330 58
17.4212252
20.8214.94
22.21
22.4314.36
Lincar
Core50
32.80+2.67 67
44.88
61.80
69.03
76.29
Linear-no-bian
Core50
32.8012 6e
44.8911.04
61.8011 on
69.02...85
26.30
Linear-Masked
Core50
1..9
8.2012.68
14.02.5.84 88
18.4612..0
27.25
Linear-no- bias Maskeed
Core50
7.36=1.97
8.22=22.7
14.0113.89
18.45
25.81
KNNN
Core50
25.95
34.171:214
45.5012
52.1311:10
65.5020.14
SLDA
Core50
17.3011.25 20
30.2315.58
17.7111.47
60.14.1..3
10.65
McanLayer
Core50
28.81+2.68 68
40.71+1..00
56.52+1.20
63.20+0.79
71.51
Medianl Layer
Core50
26.83 20
36.03s .a5
53.07:21.15
60.7320.17
70.2220.00
of the data distribution, in Core10L 10Lifelogg only the background changes from one task to another. At the same time, in CIFAR 100Lifelong, the data are very different from one task to another, i.e., for the coarse label "aquatic mammals" the data go from beavers to dolphins to otters to seals to finally whales in separate tasks.
Linear layers do not perform similarly while exposed to different types of drift. tendency that emerges from these experiments is that all layers compatible with lifelong learning even if masking may create some instability, In incremental and mixed scenarios, the layers relying only on angles between data and output vectors and masking helps to avoid weight modification.
KNN
MeanLayer
CosLayer
CosLayer Masked
CosLayer_GMasked GMasked
SLDA
MedianLayer
WeightNorm
WeightNorm Masked
l GMasked
(a) CIFARIO split, 5 tasks
(b) Core CorelOLife,ong, 8 tasks
Core50, 10 tasks
Scodla
(d) CIFAR 100Lifelong, 10 tasks
CUB200, 10 tasks
(f) 10rxx 50 tasks
Figure 4: Comparison between layers trained by gradient descent and layers trained without gradients. Experiments realized on different task orders (cf Appendix G). We plot the full accuracy.
5.2 TRAINING WITH SUBSET
In these experiments, we randomly select sub-sample of the training dataset, and we train the output layers on it. Experiment's 's results are gathered Table (Core50) and Table 4 (other datasets). The goal is to measure how the layers previously experimented can learn with low amount of data. It gives us also a good continual baseline close to the GDump approach (Prabhu et al., 2020) where we samples in the memory, and train the end of the scenario only the memory's samples.
8

---

Under review as a conference paper at ICLR 2022
This approach performs particularly well in comparison many continual approaches. Here some conclusions of these experiments:
Masking does not work in an iid setting. In table I we note that most of the masked layers underperform in this setting, except the CosLayer, which performs better with a mask.
WeightNorm is still very competitive among linear layers. In most of the experiments, Weight- Norm is among the best reparameterization of the linear layer.
Similarity- based layers tend to slightly underperform linear layers. In most of the results, these layers do not perform as well as in the continual scenarios, even if they can still have good accuracy.
MedianLaye Layer does not show much interest in comparison with MyanLayer. MedianLayer works quite well in our experiment but shows no real advantages against MeanLayer; moreover, Mean- Layer is easier to compute online. In dataset with many outliers, MeanLayer could show some advantages against the MeanLayer. In our experiments, only training on CIFARIO with all data made MedianLayer better than MeanLayer (ef appendix Table
These experiments with iid training on a low data regime show that in such settings, reparameterizatioo of the linear layer are pell-performing conclusion of all the experiments, recommend using WeigthNorm (especially in lifelong) or CosLayer in its masked version in continual scenarios.
DISCUSSION
In this paper, we studied and proposed various approaches that can help avoid catastrophic forgetting in the output layer by addressing weights modification and interference, by restricting ourself a setup where there is no projection drift.
On the opposite, multi-head architectures use the task label at test time to train a head specific to the current task. It avoids interference and weight modification by training separate heads. Interestingly, in multi-heads settings, using regularization (Kirkpatrick et al. 2017 Zenke et al. 2017 Ritter et al.,
2018) or dynamic architecture (Rusu et al. 2016 Mallya et al. 2018 Rajasegaran et al., 2019), have been shown to perform well end-to-end tasks. Then, they are able to deal with projection drifts: the head task still works when applied to the feature extractor trained up until the last task.
Unfortunately, end-to-end approaches that work in multi-heads can not be directly connected with our findings to create a good end-to-end single head approach. Indeed, single-head models need discriminative inter-task representations that are not necessary for multi-heads settings. Therefore, the constraints on the feature extractor are not the same, and direct transfer is not possible.
Dealing with projection drift in single-head settings stays then one of the biggest challenge in continual learning. Replay might be a simple solution to simulate an iid training and avoid problems with projection drift. However,
if replay might be necessary in continual learning scenarios (Diethe et al. 2018 Lesort et al. 2019), replaying data may have consequences in the memory needs and compute needs. Therefore, it worth finding training procedures, architectures, or regularizations that do not necessarily replace replay but at least reduce its need.
7 CONCLUSION
In this paper, conduct an empirical evaluation of the various output layers of deep neural networks in CL scenarios. This evaluation gives us clear insights into how output layers learn continually or on low data regime. We also showed how data distribution drifts might affect the output layers differently, namely in lifelong and incremental settings. This perspective should incite researchers to study more the connection between data distribution drifts characteristics and catastrophic forgetting.
On one hand, classifiers such as KNN, MeanLayer and SLDA provide very strong baselines that are very effective with a frozen feature extractor. On the other hand, our results show that by a reparameterization of the classical linear layer, we can greatly improve performance, especially in incremental settings. This reparameterization helps to reduce the risks of interference and weight modifications. Some output layer types learn continually when there is projection drift with stochastic gradient training. These findings could be integrated end-to-end training to make continual training more efficient.

---

Under review as a conference paper at ICLR 2022
REFERENCES
Youness Aliyari Ghassabeh, Frank Rudzicz, and Hamid Abrishami Moghaddam. Fast incremental
Ida feature extraction. Pattern Recognition, 48(6):1999-200) 2012, 2015. ISSN 0031-3203, doi: http:///o..ogg//0000liimaatggggp scieene//rriceee///////111111110
htLps:ts:// www. www.sciennddi.ee......o. com/
Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: a strong, simple baseline. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems 33, pp. 15920-15930 15930. Curran Associates, Inc., 2020. URL htttps:/ nips psp..... cc/ 661171666666 (ew.666666600eeeeeeeee.
paper/ wtw.:/ilel/// /tt.://///0..0.00000000000011000
Lucas Caccia, Rahaf Aljundi, Tinne Tuytelaars, Joelle Pineau, and Eugene Belilovsky. Reducing representation drift in online continual learning. arXiv preprint arXiv:2104.05025, 2021.
Massimo Caccia, Pau Rodriguez, Oleksiy Ostapenko, Fabrice Normandin, Min Lin, Lucas Caccia, Issam Laradji, Irina Rish, Alexandre Lacoste, David Vazquez, and Laurent Charlin. Online fast adaptation and knowledge accumulation: a new approach to continual learning. NeurlPS, 2020. 05856
URL https::: /ttaxxiivv.o
C. Chatterjee and V.P. Roychowdhury. On self-organizing algorithms and networks for class- separability features. IEÉE Transactions Neural Networks, 8(3):663-678, 1997. doi: 10. 100/ 72.572105.
Sanjoy Dasgupta and Daniel Hsu. On-line estimation with the multivariate gaussian distribution. In International Conference on Computational Learning Theory, pp. 278-292. Springer, 2007.
G.K. Demir and K. Ozmehmet. Online local learning algorithms for linear discriminant analysis. Pattern Recognition Letters, 26(4):421-43) 2005, ISSN 0167-8655, doi: https://doi.org/10..uu 1016/ 1ppatrec.2204.08.0000. URL https: www. www.seciieeeiiiieeeeoc.cceeeeeee com cctm///ciec///ttccceeee p11/50167665504000 4001 825
ICAPR 2003.
Tom Diethe, Tom Borchert, Eno Thereska, Borja de Balle Pigem, and Neil Lawrence. Continual learning in practice. In NewrIPS Continual Learning Workshop, 2018. URL httttt/:s///xt
org/abs/1903 05202
Arthur Douillard and Timothée Lesort. Continuum: Simple management of complex continual 06253
learning scenarios, 2021.
Arthur Douillard, Matthieu Cord, Charles Ollion, Thomas Robert, and Eduardo Valle. Podnet: Pooled outputs distillation for stall-tasks incremental learning. In Proceedings of the IEEE European
Conference on Computer Vision (ECCV), 2020. htt.:httts:///e www.. www.vvanee/papee..eeee. lecv__ @eccv_2020paaaaeeeeeeee SeaapperssBcc/ppppes///22CC0
Sebastian Farquhar and Yarin Gal. Towards robust evaluations of continual learning, arXiv preprint arXiv:1805.09933, 2018 UK//ittpss///rx...or/r//ass /tt.b////.10.030333333 09733
Li Fei-Fei, Rob Fergus, and Pietro Perona. One-shot learning of object categories. IEEE transactions on pattern analysis and machine intelligence, 2844):594-61,, 2006.
Tyler L Hayes and Christopher Kanan. Lifelong machine learning with deep streaming linear discriminant analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220-221, 2020.
Kaiming He. Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition. pp. 770-778, 2016.
Saihui Hou. Xinyu Pan, Chen Change Loy, Zilei Wang. and Dahua Lin. Learning a unified clas- sifier incrementally via rebalancing. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
10

---

Under review as a conference paper at ICLR 2022
Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, pp. 448-444455 456.. PMLR. 2015.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinsker et al. Overcoming catastrophic forgetting in neural networks. Proc. of the national academy of sciences, 2017. URL
https:
www pnas. org/ cont crottent/pnas ccnteett/pnas/11//13//33333/33//
full.pdl
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical Report, 2009.
Brenden Lake, Ruslan Salakhutdinov, Jason Gross, and Joshua Tenenbaum. One shot learning of simple visual concepts. In Proceedings of the Annual Meeting of the Cognitive Science Society, volume 33, 2011.
Timothée Lesort, Natalia Rodrigodr,,,,z, -can-François Goudou, and David Filliat. State repre- sentation learning for control: An overview. Neural Networks, 2018. ISSN 0893-6080. doi: https//bbboogggl...oooo/eeeeeeee Ssieenee/arriceee///////83338881
URL httt:://www.scienceiireee. com/
Timothée Lesort, Andrei Stoian, and David Filliat. Regularization shortcomings for continual learning. 2019.
arXiv preprint
Timothée Lesort. Massimo Caccia, and Irina Rish. Understanding continual learning settings with data distribution drift analysis. arXiv preprint arXiv:2104.0163 2021.
Vincenzo Lomonaco and Davide Maltoni. Core50: a new dataset and benchmark for continuous object recognition. In Sergey Levine, Vincent Vanhoucke, and Ken Goldberg (eds.). Proceedings of the 1st Annual Conference on Robot Learning. volume 78 of Proceedings of Machine Learning Research, pp. 17-26. PMLR, 13-15 Nov 2017. URL http: httt:://proceedd.gss mlr .resss/ v78/lomonacolo html
Vincenzo Lomonaco, Davide Maltoni, and Lorenzo Pellegrini. Rehearsal-free continual learning over small non-iid batches. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 989-998. IEEE Computer Society, 2020.
Arun Mallya, Dillon Davis, and Svetlana Lazebnik. Piggyback: Adapting a single network to multiple tasks by learning mask weights. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 67-82. 2018.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luea Antiga, Alban Desmaison, Andreas Kopf. Edward Yang. Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high- perfforrfom deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. dAlché Buc. E. Fox, and R. Garnett (eds.). Advances in Neural Information Processing Systems 32, pp. 8034-8035. Curran Associates, Inc., 2019. URL http: www.inn.uussccoooo wctpppprcccpppp...ppppppp
ibrary.
pdf
Ameya Prabhu, Philip HS Torr, and Puneet Dokania. Gdumb: A simple approach that questions our progress in continual learning. In European Conference on Computer Vision, pp. 524-540. Springer, 2020.
Antonin Raffin, Ashley Hill, Kalifou René Traoré, Timothée Lesort, Natalia Diaz-Rodríguez, and David Filliat. Decoupling feature extraction from policy learning: assessing benefits of state representation learning in goal based robotics. Workshop on "Structure and Priors in Reinforcement Learning" (SPiRL) ICLR, 2019.
Jathushan Rajasegaran, Munawar Hayat, Salman H. Khan, Fahad Shahbaz Khan, and Ling Shao. Random path selection for incremental learning. CoRR, abs/1996.01 120, 2019. URL http? 7/ //arxiv.cm ora. 906 01120
11

---

Under review as a conference paper at ICLR 2022
Sylvestre-ANN Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001-2010, 2017. URL
abs/1 3bs/1611.07725
Hippolyt Ritter, Aleksandar Botev, and David Barber. Online structured laplace approximations for overcoming catastrophic forgetting. Advances Neural Information Processing Systems, pp. 3738-3748. 2018.
A. A. Rusu, N. C. Rabinowitz, G. Desjardins, H. Soyer, J. Kirkpatrick, K. Kavukcuoglu, Pascanu, and R. Hadsell. Progressive neural networks. ArXiv e-prints, jun 2016. URL https://axxxi
org/abs /a///b//// 1606. 04671
Tim Salimans and Diederik P Kingma. Weight normalization: A simple reparameterization to accelerate training of deep neural networks. arXiv preprint arXiv: 2016.
Shaoning Pang, S. Ozawa, and N. Kasabov. Incremental linear discriminant analysis for classification of data streams. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cyhernetics), 35 (5):905 914, 2005. doi: 10.1 00.1109/SMMCBB2005...8 847744.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 1556, 2014.
Adam Stooke, Kimin Lee, Pieter Abbeel, and Michael Laskin. Decoupling representation learning from reinforcement learning. arXiv preprint arXiv:2009.08//9 2020.
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions, 2014.
Gido M van de Ven and Andreas S Tolias. Three scenarios for continual learning, arXiv preprint arXiv:1904.07734 2019.
07734
Wei Wang, Vincent W. Zheng, Han Yu, and Chunyan Miao. A survey of zero-shot learning: Set- tings, methods, and applications. ACM Trans. Intell. Syst. Technol, 1002):3:11113:37 January 2019, ISSN 2157-6904. doi: 10.1 10/114//3993318 3293318
Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 374-382, 2019. /ttp:: http://awwwww..o...//////ooo 13260
Friedeman Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pp. 3987-3995 International Convention Centre, Sydney, Australia, 06-11 Aug 2017. PMLR. URL http
V/proceedings ml pressppressseeeee peress/v70. prress//70/zenkell. zenkel
7a html
Bowen Zhao, Xi Xiao, Guojun Gan, Bin Zhang, and Shu-Tao Xia. Maintaining discrimination and fairness in class incremental learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13208-13217, 2020.
12

---

Under review as a conference paper at ICLR 2022
APPENDIX
A SCENARIOS DESCRIPTION
A.1 CIFARIO 5 TASKS
classical incremental setting, with classes per task with original CIFARIO dataset.
A.1.1 CIFARIOOLIFELOO 5 TASKS
CIFAR100 100 dataset has two labelization systems: the classes labels and the coarse labels. There are 20 coarse labels gathering 5 classes each. create CIFAR 100Lifelo tasks, we share data into tasks such having ensure that the data is different for each task.
data for each coarse label in each task. On the other hand, we use class labels to
We have then 5 tasks with data for all coarse labels and from 20 different classes. A class does not appear in tasks, as an incremental. However, the model should predict the coarse label. We obtain scenario with 5 tasks with always the same 20 labels. However, the data change, and the model should learn continually to predict the coarse labels for data for all classes correctly. This scenario makes it possible to measure the resistance mode change (change of class) while continually learning the coarse label classification.
A.2 CORE50 10 TASKS
classical incremental setting, with classes per task with the 50 objects core50.
A.3
CORE CORE1OLINE OLIFELONG 8 TASKS
Core50 is composed of 50 objects, equitably shared into 10 classes filmed environments (8 environments for train and for tests.) In Core 1OLielon,,,,,, have tasks where explore each time one environment with all objects. Objects are annotated with their class labels. Hence, we have tasks of 10-way classification. The object stays the same from task another; only the environment changes. This scenario makes it possible to measure the resistance to background change while learning continually.
A.4 COREIOMIX 50 TASKS
In this scenario, each task contains the data corresponding to one object in all of its environments. We have hence a sequence of 50 tasks. However, objects are annotated with their classes labels. Hence all classes are revisited 5 times in the scenario (in random way). This scenario is a mixture of incremental and lifelong to measure the ability to solve scenario with several types of data distribution drifts.
B SLDA LAYER
The approach we use (proposed by Hayes Kanan (0220) stores one mean vector per class μ4 with counter E IR well as covariance matrix of the same size as A (common to all classes). The covariance matrix's 'ss update is done with:
Σ11 t,, A.
(6)
t+1
Where Ag is:
A U/ - -2
t
For prediction, SLDA uses a precision matrix A= [[1 - + cl|-1, where c = 10-4 is called the shrinkage parameter and [dd the identity matrix.
Then two vectors w and are computed wa Apk. b = = Apoo)
And finally: Ok (zz, wk) by
13

---

Under review as a conference paper at ICLR 2022
C PRELIMINARY EXPERIMENTS
C.I PRELIMINARY EXPERIMENTS SETTING
The preliminary experiments are designed to see how the different output layers work without continual learning constraints. This experiment is also conducted to select the learning rate and architecture for further experiments.
We experimented with CIFARIO and CIFARI00 datasets, and we use resnet20 pre-trained model from here 2 (BSD 3-Clause License). The main idea is to compare those results with our results. We take the frozen pre-trained model and replace the output layer with new one we train all data. (1) We experiment with training on CIFARIO O with a neural network pre-trained on CIFAR100 and the opposite. (2) We also trained on CIFARIO with the model pre-trained on CIFAR1O to see if we would recover the original accuracy, and we did the same for CIFARIO0. Doing continual learning on a dataset with model pre-trained on this dataset is not recommended since it means having access to the whole data from the beginning of training. We did not use (2) setting in the further experiments, and it was just used as a reference.
(3) We also experiment also with Core50 dataset (Lomonaco & Maltoni 2017). We used models
pre-trained on ImageNet available on the torchvision library (Paszke et al. 2019p. We used VGG16, GoogleNet and Resnet, which have a latent space of dimension 2048, 1024, and 512. We experimented with the Core50 dataset, Core50 setting with 50 classes corresponding to the object id, and Core10 setting with 10 classes corresponding to the object category.
In these preliminary experiments, we evaluate all layers without their masking variant to verify that the pre-trained models were suited to the continual learning scenarios.
C.2 PRELIMINARY EXPERIMENTS
The summary of the preliminary results is reported in Table 2 The full results can be found in Table 3 As described section C. we experiment with the simplest output layer for this preliminary experiment, i.e., linear layer, linear layer without bias (Linear_ND) NB), CosLayer, WeightNorm layer, Original WeightNorm (OWeightNorm), KNN, SLDA, and McanLayer.
The results of those preliminary experiments are that globally the learning rate of 0.1 is adapted to all settings. Secondly, we found that for Core50 and Corel0 experiments, the ResNet model was the best suited. Moreover, we see that 5 epochs per task are sufficient to learn a correct solution. Concerning CIFAR experiments, the reported top-1 accuracy on CIFAR 10 and CIFAR 100 of the original pre-traine are-trained models respectively 92. 92.60%, 69. 69.83%. The settings CIFARIO pre-traie pre-trained on CIFARI0 10 (same for CIFAR100) show us that the training procedure can recover almost the best performance. In CIFAR100 there is drop of 6% of accuracy, but it stays reasonable in 100-way classification task.
Even if not very surprising, one interesting result is that training on CIFAR 100 with a model pre- trained on CIFARIO does not work at all. It illustrates that using pre-trained model is not always a solution in machine learning or continual learning. They should be selected carefully, even if the data might look similar to CIFAR10 and CIFAR 100. On the other hand, the experiment on CIFAR10 with CIFAR100 models shows a significant decrease in the performance for all layers.
fa:
Table 2: Preliminary experiments to select learning rate and architectures with existing type of layers. Experiment realized on 8 seeds [0, 1, 3, 4, 5, 6, 7].
Daraset
Archlecture restes
Preinaning
Linear
CooLayer
WeightNorm 122
OWeighhNorm
KNN
SUDA
MeunLayer
CIFARIO
0.1
CIFARTO
22.05amm2
12.00-mm
21.70a
CIFARIO
0.1
restet
CIFAR100
05.37am
08
60:20
05500a
63.77am
CIFARICO
0.1
rosnet
CIFARIO
17.52a8.9
17.14
13.0
16:00m 17
Maa
14.45
CIFARI00
0.1
CIFAR1O0
63.974.
15
60
01.50amm:
Coma50
0.1
restet
ImageNet
70.881341
27:0
85188
05.50-014
15
71.5110.0
Core50
0.1
VRE
ImageNet
71.4610.30
01:10-0300 30
72 48
73 5..53a
56.86-00
06.07400)
Core50
0.1
pooglenee
ImageNet
70.1810.3
47.81
66 20.74
07.274
39.45.0
7000
00.851c.35
Care Offifoo
0.1
ImageNei
85 35
85.62461
78
80 30
75.71 m
8500
70001ara
Corc 10l
0.1
*05
ImageNet
85 80.52
85.344n33
79. 10.300mm is
85.00 65.90amm
65.664
75.
83.00an.
78.66
Corc10Liftionn
0.1
googlese
ImageNer
$5,0848:07
$5.0846.
78:21-033 2s
83.05
51.57400
77.57
76.9240.0
14

---

Under review as a conference paper at ICLR 2022
Table 3: Preliminary experiments to select learning rate and architectures with existing type of layers
Pistaiiig
Conlaper
WeighiNoem
OWegheNomn
255081
CHRARIO
02.39
92.0415/4
92,001011
CUFARIO
255061
12..0
02.47:444
17.001.0023
92.30...0
92.577m04
91.790033
05
755061
17.3210.5
CIFARIO0
40.20
CIFARI0
CIFARIO
6.7
2.40ama22
CHRARI00
CIRARIO
14.201663
26.44110
14.65
CIFAR100
CIFAR100
CIFARI00
54.5
CHFAR100
CIPAR100
60.00.0032
Corc3l
Core50
5,001009
Core50
62 12.0411772
82.651.072
2.210.55
10,00
60.421085
Core50
04.301034
Core50
ImggeNet
71 50
T1. 1:
61133
Core50
ImggeNet
72 7200
91
Con350
0.001
ImegeNe
08 7M
68 68-88am
3.47
Cone00
ImegeNa
50.90
09 98a
Conc30
70.
06.37167
Curco
0.01
pouglan
Imggae
68 18.99
66 Ma
8.7
61 77
66 60
Corcão
0.001
poogleet
Imggaee
40.02
2.50a
21.91
49. 10.76amas
Coreão
goodenet
ImazaNet
50 20.451633
70.60.c.1
00,85
Cie Ed iellng
01
85, 85.B5ar-a
75.7
86 Mite
Cuez 10.iduugg
0.01
ImggaNa
96. 18
65.264134
85. Miee
Cue OLiidung
0.001
InngaSa
82 62.
82.5 82.6damsa
18.38e2.0
76.12
68.11
Cure 1Liffung
ImngaNet
76.7141
85 00acaa
70.6
Lane Miildnng
0.1
InngeNes
H6. 16.3316.0
Cire LLLifoite
0.01
126
ImazeNet
85.8010.3M
5T, 27.87
72.441192
85 85.37
85.7410.5
Core LLLiferg
0.001
ImageNet
84 20.14
$4.30 $4,301031
24.59.3.17
82 1410.7
$4.110011
Come LLLeeeng
vi5
ImageNet
75.14
H3 10.
78 78.
Com (d ldiieeggg
0.1
pooglent
ImogeNet
85.0815.4
18.25
83 18...
84.9/an.2
Com seelng
0.01
peogleet
ImogeNet
8550775
$0.07200
02 07.3773300
82.11a
80 80.97an
Corld dedong
0.001
proglma
ImogrNet
70.31 m
14.85g700
75.90am-w
all 133n
(Coe100 ifdlnng
pvidla
InageNia
77.576n 99 86-39er I8
These preliminary experiments make us able to fix some hyper-parameters to study all layers in common setting. also offers us interesting insight into the drop of performance in simple experiments when using pre-traing pre-trained model instead of training end-to-end. The results of this experiment can used as a baseline continual experiments and subsets experiments.
15

---

Under review as a conference paper at ICLR 2022
D FORGETTING IN LIFELONG SCENARIOS
D.1 CIFARIOOLIFELOONG
Linear
Masked Linear_ne no bias Masked
OriginalWeightNomm Coslayer Masked
Weightorrm, Masked
Linear_me bias
WeightNorm
OriginalWeighhtormm Masked
1.0
0.8
test accuracy task 0.0
0.6
0,4
0.2
0.0
10
15
25
Epochs
(a) CIFARIO. tasks
Figure 5: Test accuracy on task 0 in CIFAR100Lifelong Scenario.
that the model is forgetting the first tasks and the masking have effect on it. Moreover, since the single masking makes learning less efficient with many classes, it can make the performance bad from the beginning.
In Fig.
D.2 CORE OLIFELONG
Unfortunately, the Core50 test set does not make it possible to evaluate the forgetting in each task of lifelong scenarios. Indeed, the environments the test set are different from the train environments. Then it is not possible to evaluate with the test set the evolution of accuracy past tasks. In the lifelong scenario, we can only evaluate the performance on 3 other unknown environments, similarly out of distribution evaluation.
16

---

Under review as a conference paper at ICLR 2022
E FULL SUBSETS RESULTS
17

---

Under review as a conference paper at ICLR 2022
Table 4: Subset experiments: Mean Accuracy and standard deviation on 8 runs with different seeds.
Outlayer
Subset
200
1000
Cosl ayer
CIFARIO
31.81
39.00
48.2313.51
54.09
66.5519 65599988
WeightNorm
CIFARIO
50.82#191
56.90+0.82
62.64:0.78
65.41=0.40
68.55 68.55+00.21
Original Noight/orm
CIFARIO
18.2242.19
52.33+1.68
55.91
61.66.0.74
69.21+0.30
Cosl Layer- Musked
CIFARIO
53.4511.63
58.54499.4
61.25100.7
62.51 2:0.0
67.3661:900
WeightNorm-Mase
CIFARIO
36.55+5.20
39.0442.25
41 11.23:2.115
42.44=1.62
33.89+5..5
Original BrggnallWightNorm-NNe
CIFARIO
22.79
22.08
21.07
22.4586.78
33.18
Linear
CIFARIO
49.1322.31
54.31
60. 60.16a9.72
64.0220.77
75.13:10
Linear-no-hias
CIFARIO
49.11:2.27
54.31
60.15:1075
64.04
75.13
Linear-Masken
CIFARTO
23.0228.05
25.78
27.45a.3.1
40.02
12.57-1:00
Lincar Masked
CIFARIO
25, 25.964s..0
27.74
39.20
KNN
CIFARIO
45.21+2.03
19.47
$4.19+11.1
57 57.00
60. 11.22620.22
SLDA
CIFARIO
49 19.23222.67
61.51 a:16.9
68. 455447133.3
70.47
66.90a...u
Myen
CIFARIO
67.34
66.58
68.65
69.58
63.27
MedianLayer
CIFARIO
60.65#18.27
15.60
63.59
68.84 18..61
70.44+12.78
CosLayer
CIFARTOOL feelngg
4284.88
36.6 15.10
47 4..00
53.30=1.87
16.42++0.25
WeightNorm
CIFARI
19.19+2.30
57.33+1.14
63.67
60.39=0.cc
70.52
Originu/WeightNomm
[IFARI00Lifelong Liffloog
47.68a22.2
55.1611.10
60.70a0.01
62.83:0
70.53
CosLayer- Masked
Mask-M
SEARIOOLLien
53.27#174
58 88
62.88
64.5620.55
65.74
WeightNorm-Maskke
23.17
26.00
a7.09
28. 00.6.0
30.78
Original WeightNorn-Muusee
CIFARI00Lifelong
16.1328.01
18.4919.51
16. 16.1687.50
18.12a6.
24.8015.10
Linear
CIFAR100) Lffelon
47.84222.1
35.8881.00
62.46:2077
65.5620. 50
70.36:10.10
Linear-no-bias
CIFARTOOLLifelam
47.24
55.70
62.35.20.70
65.38 as
70.28 29
Linear Masked
CIFARIOOLifelon
14.29
14.76
16.39+2.11
Lincar-no- sercarrno-biin Masked -ibka----eed
CIFARIO0Lifclon 10Liiflong
11.81
12.41$3.47
12.77+2.48
13.16=2..8
24.53
KNN
CIFARTOOL 10Liiflong
44.78a2.0
52.98
61.87.40.8
65. 20.93
66.71
SLDDA
CIFART0OL MIFAR100Lifeeong
37.0322.04
55,02:0.88
63.07.10.5
65.75
MeanLayer
CIFARI (00Lifelong
51.0 $1.03+2.17
58.01#1.05
+2.3220.45
64.13
65.55-+0.00
MedianLayer
CIFAR100Liieloog
48 23:1.74
55.30
60. 44...a
62, 62.76.0..77
64. 64.17.00000
CosLayer
Core50
10.30+1.9
12.4343.66
18.1144.17
26.18:4.16
53.93
WeightNorm
Core50
32.0622.01
44.32s1.0
61.9141.0
69.44:20.0
77.04
Weightion
Core50
19:.922.
45.68
1..39
69.1 7.000000
76.85-20.81
CosLayer- CosLayer-Maskod
Core50
30.62+2.67
41.22s1.68
56.47
63.10=0.62
FrrightNorm-Maskee
Core50
12.95.2207
14.98
21.13:1.22
23.42
31.18417.0
Original WeightNommmmm
Masked Masked
Core50
12.27 22.59
17.42+2.52
20.82
22.21 25550
22.433t-1330
Linear
Core50
32. 80.8000
44.88
61. 21.80.0.00
69.03
76.29a...
Lincar-no-bin
Core50
32.80:2.08
44.89
61.80:1.00
69.02
76.29 20..
Linear-Musked
Core50
7.37=1.00
8. 8.20=2.66
14.02
18. 18.46=2.00
27.25
Linear-no-bal -i-us---e-bias-MMskked
Core50
7.36
8.22=
14.04
18. 18.4822.55
25.81
KNN
Core50
25.95-15.00
34.17:2.14
45.50+2.05
52.1321.11
65.501:0.0
SLDA
Core50
17.30
30.23*s5.68
17.71+1.47
80.14:0..93
78.55+00.0
MeunLayer
Core50
28. 2:2.68
40.71
56.52+1.29
63. 20:2077
71.51 20000
Medianl ayer
Core50
36.033
53.07
60.73220.7
70.222.00000
CosLayer
Core OLLifelong
31.45210.00
43.25
52.17
65.40.5..m
78.76.1..0 10.00.30
Original WeighttlWiig
WeightNorm
Core101 Jifelong
68.5244.10
76.78s1.4
$1.38:0.89
83, $3.89920.00
06.29-80.2 2..20
Core loreCOiig
67.50+6.30
75.86=1.15
80.33:.0.5
82. 82.42:00.75
86.07 #0.51
CosLayer-Musked
Core 10Lifelogg
65.5844.18
72.2842.19
76.80.1.29
78.
79.56.1..
WeighttNom--aaee Masked
Core10l afelong
42.05:25.58
48.2662:.05
54.12:44 1/2.6.72
57.782
58.74 38.74
OriginalWeighte
-anke-
26.76a5.72
34.75
30.12a11.0
31.73
46.05 .96
Lincar
Core ifelong
68.47
76.33+
80.7920..9
83.0820.57
Linear-no-bias
CorelOLifelong
68.47
76.33
80.79:20.91
83.08 =0.00
86. 19..0..3
Linear-Musked
COLieoLig
23. 27.17.0000
24.20
29. 79.25::7.40
30.79
38 18.08
Linear Masked
afelong
24.0618
28.8247.3
30.76210.04
38.27:428
KNN
Core10Lifelen
55.05 45.05
64.80
72.20 22.20
76.51 1.65
SLDA
Core10Lifclene
63.00=a.0a
63. 63.9642.a5
35. 15..5..
73. 06:4620..22
88 06:20.01
Meanl ayer
afelong
4..06
11.3022.00
76.25
T8.33
MedianLayer
Core10Lifelean
4..46222
69 2..21
74.81
76.09
78.1610.
Cosl ayer
CUB200
6. 6.1997.77
5.41=1.00 on
6.77
8.74 .91
30.0020.75
WoightNorn
CUB200
11
15.78+0.88
25.30+0.9s
34.96=a0.05
54.41+0.50
Origina/WeightNorm
CUB200
11.1740.87
16.1640.00
26.10.0.93
35. 5..50....5
22.9020.07
Cuss ayer Masked
CUB200
11.4361.90
16.8529.1
26.7940.77
36.0330077
33.07 19.05
WeightNorm-He Masked
CUB200
7.51
9. 1.100
12.37
14.40
16.22 16.24
Original SriginalWWightNoamee Masked
CUB200
3.96=1.02
3.92=
4.39-2..4
4.96:
5.75
Linear
CUB200
10.88+0.82
15.38
24.27
33. 20.299...
28. 28.8110.75
Linear-no-bias
CUB200
10.73:0.82
15.07
24.23:0..0
33 33.49.0.00
28. 18.92.20.75
Linear Masked
2.07
2.50
2.5520
2.7720-42
Linear-no Maskedd Masked
CUB200
2.42=0.s5
2.15=
2.36+0.3
2.72=0.38
2.08
KNN
CUB200
10.33+1.22
14.31+0.06
20.41+1.0
25.54=:0.75
43. 13.79:a0.00
1.94
2.37
3.7320.44
15.93222.5
MeanLayer
CUB200
10.37
15.07+0.46
25.24
34.55=0.74
64.1440..00
MedianLayer
CUB200
10.77+1.02
11.92:..54
23.07:20.30
31.91=100
53. 53.28-s0.00
18

---

Under review as a conference paper at ICLR 2022
F VISUALIZATIONS
(a) Linear: Angles between the (b) Linear: Mean Angles between different vectors of the output the vectors and data class by class. layer
(c) Linear: Interference Risks
(d) WeightNorm: Angles between (e) WeightNorm: Mean Angles (f) WeightNorm: the different vectors of the output between the vectors and data class Risks by class.
Interference
layer
G
TASK ORDERS
from continuum. scenarics import ereate_subscenario import numpy as no
we suppose scenario, num_tasks and seed already set np. random. seed (seed)
task_order np.arange (num_tasks)
scenario create subscenario (scenario, task task_order) order)
(a) Code to change task order of the original scenario. We used seeds [0, 1,2,3, 4,5, 6, 7]. Note: for seed 0, we use original order without modification.
19

---

Under review as a conference paper at ICLR 2022
H SCENARIO REPRODUCIBILTY
from cont inuum. datasets import Core50 from cont inuum import ClassIncremental
cataset dataset Core50 ("./attsttttt
drwnload-True, train-train)
scenario Class incremental (dataset,
nb t..tasks=10,,
(a) Code for Core50 secnario
from inuum datascts import Core50 from cont inuum import ContinualScenario
dataset Core50 (*. /ddaaaattss"
scemario-"ddmains"
train-train)
scenaric ContinaaaSSeeeeriitttttaeeee
transformtttonssttaasststmt
(b) Code for Core10Lifelong scenario
from continuumddatseett... import Core50
from continum import ContinualScenario
dataset Core50 (" /datasets""
scenario="abjects")
clasilicatioa==*ateeeer
train=train)
scenario = Co (dataset,
bransforratiions-trre
(c) Code for Core10Mix scenario
from cont inuum, datasets import CUB200
from cont inuum import ClassIncremental
dataset CUB200 (" /ddaasets"
train-train)
sccnaric Class (dataset,
nb_tasks=10,
tranaffommtiions--traaafooomm
(d) Code for CUB200 scenario
from www.iinnuuaaaaasee.e import CIFAR100
from continuum import ContinuslScenario
dataset CIFAR100 (" /datasets",
"bbels_ttpe==categorr
wwwabbb=="iffflee lask_labels=11fellnnn
train-train)
scenario = ContimualScemario (waeaarrddddddddaaaaea
(dataset,
transforrattionsstraaeee
(e) Code for CIFAR100Lifelon scenario
Figure 8: Code to reproduce the scenarios used in the paper with continuum library.
20


Reviews:
[{"review_id": "qL4iDjvyFZE", "reviewer": "Reviewer_uN9P", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength: \n+ The summary of different design choices, including the weight normalization, \n+ Some illustrations are valuable for understanding the cause of forgetting, eg. the norm, bias, and their changes during the incremental process; and the interference plots. \n\nWeakness: \n-  The assumption on non-projection-drift is not very realistic. In continual learning problems, it is very often that the feature representations are shifted over time, along with new data streams.  However, under the paper's assumption, it requires a fixed feature extractor. \n-  Since the study is based on a non-projection-drift setting, it requires pre-trained models as the feature extraction as stated in the experiment part, which is not often practical in many settings. Although it does not use the pre-trained models trained on the target dataset, it is still unfair to compare with other works in the literature. \n- For Figure 2 (interference), what is the root cause for the interference? Why the mean angle of z and A_i (middle) is not as discriminative as the angles among A_i (left).\n- The paper proposed a modified version of the weight normalization layer, but the proposed modified layer operation does not show clear improvement but slight drop over the original designs in most cases (weight_norm vs original_weight_norm in Table 1). Similar to the proposed median layer vs original mean layer. \n-  The proposed masking approach is not stable and tends to produce bad performance.  \n- The figure plots of accuracy are a bit cluttered and hard to read. ", "summary_of_the_paper": "This paper presents a study on different last output layers, under the continual learning problem. More specifically, the study is conducted in a controlled environment without projection shift. The paper proposed several design choices of the last layer, including a modified weight normalization layer, masking operations, and median vector. Different scenarios of incremental/lifelong and mixed learning have been experimented with. ", "summary_of_the_review": "The study of the feature drift is important, otherwise, the study has to rely on the pre-trained feature model, which makes the setting impractical in many cases.  \nThe paper provides some summarization of multiple tracks to deal with the weights normalization and preserving in the last layer but the insights brought from the work are limited. The analysis is strait-forward on the observation of weights but there is not much deeper and richer analysis on the causes of such observation.  \nThe experiment results do not support most hypotheses made, but the cause of the failures remains unclear. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "9e7AMSc-yhM", "reviewer": "Reviewer_BTLN", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Clarity:\n\nOverall writing quality of the paper can be improved a lot. Especially the Sec5 Results which needs to highlight the current \ncontributions and can explained bit more in-detailed. The method proposed is well described and it would take some time to have reproduce the results.\n\nNovelty:\n\nThe methodological contributions do not seem to be novel and all the results are as expected w.r.t  the current\nline of research of doing continual learning keeping the feature extractor part of the network fixed.\n\nPros:\nVery well detailed experimental setup covering all the scenarios. Authors evaluated the proposed ideas on Incremental Scenarios(CIFAR10,Core50,CUB200), Lifelong Learning Scenarios(Core10LifeLong, CIFAR100LifeLong) and Mixed Scenario(Cire10Mix)\n\nCons:\n\nAs mentioned the Novelty is pretty much limited and results as as expected w.r.t current line of research.\n\n", "summary_of_the_paper": "The authors analyzed the effect of changes to the last output\nlayers of the neural network in the continual, incremental learning scenario\nand effects of it two scenarios 1. weight modification 2.Inference.\nAuthors also claim to propose simplified weight normalization layer, two masking strategies, and an alternative to NearestMean Classifier using median vectors to address catastrophic forgetting in the output layer in both continual, Incremental scenarios.\n\nDuring the experiment setup authors separated the network into two parts\n\n1. feature extractor part consisting of all but the final layer (kept fixed to avoid projection drift)\n2. classifier part which was given by the output layer.[Much of the analysis is concentrated on Part-B]", "summary_of_the_review": "Please refer before sections\n\nChanges:\nIn the Sec 5.2 (Median Layer...): Third line \"MeanLayer Could Show some more advantages against MeanLayer\"", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "7EuY5U6TJpT", "reviewer": "Reviewer_Kc24", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper isolates the output layer for study yet in continual learning all layers are updated. No concrete motivation is provided as to why the output layer should be studied in isolation. \n\nMoreover, some of the observations highlighted in the paper may not hold if the neural network is allowed to be updated and the neural network is studied holistically.\n\nFurthermore, most observation (e.g., _masking is efficient in incremental settings to avoid weight modifications_) are well known from prior work. Most observations are not contrasted with prior work.", "summary_of_the_paper": "This paper studies the output layer in a deep neural network in the continual learning setting. The paper studies multiple types of output layers under different continual learning configurations. Some suggestions are made based on observed empirical trends.", "summary_of_the_review": "The validity and novelty of the results and observations in the paper are somewhat dubious. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "4TgYp0D-29V", "reviewer": "Reviewer_e1ZF", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "1. I am not clear on why using neural networks became essential in this study. \n\nGiven that the feature extractor part of a given neural network is frozen, and the paper doesn't provide any insight on feature learning during continual learning, I'd recommend the authors to start from simple linear models, including logistic regression, and SVMs. In that case, the impact of the neural network architectures can be ruled out, and the conclusion could be clearer.\n\n2. As mentioned later in the paper, arguments made regarding the lengths and the biases of vectors in the weight matrix in the last layer don't seem consistent across various settings and datasets in continual learning.\n\nPersonally, I think the main issue is that the learnt feature spaces are drastically different within the selected models, and the confidence of the top linear layer in classification also varies, therefore, when only studying the top linear layers, behaviours may not be consistent. \n\nThe secondary issue is that the feature vectors are all in very high-dimensional space where, with high probability, vectors are equal-distant from each other, which causes trouble and issues when analysing their relations. That probably also explains why the proposed 'MedianLayer' performed similarly to the previously proposed 'MeanLayer'\n\n3. From the experiments and analyses in the paper, as also is mentioned in the discussion and conclusion section, we can see that classic prototype-based approaches still stand out, and the proposed methods don't add much more value to them, either performance or understanding in terms of their behaviours in continual learning. ", "summary_of_the_paper": "the paper analysed behaviours of the last linear layer of pre-trained convolutional neural networks in the settings of continual learning where either new instances are provided in new episodes or new classes are given in new episodes. Based on arguments and observations provided in the paper, the authors proposed to constrain the weight matrix in the last linear layer to only have unit-length vectors, so that the lengths and biases of these vectors won't introduce any semantic drifting in continual learning.", "summary_of_the_review": "I have major concerns about the clarity of messages conveyed in the paper, and also the insights that this paper could potentially provide on continual learning if the paper was accepted. Therefore, I do not recommend accepting this submission.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes an empirical study on the effect of various types of output layers of deep neural networks in different scenarios of continual learning. The authors draw several insights, such as ways of selecting the best output layer depending on type of scenario and a description of the different sources of performance drop (forgetting, interference, and projection drifts). The paper proposes different ways of mitigating catastrophic forgetting: a weight normalization layer, two masking strategies, and a variant of NMC using median vectors.

The paper presented a detailed experimental setup covering a large number of scenarios of continual learning: incremental, Lifelong Learning and Mixed Scenario. This was highlighted by Reviewer BTLN, and the AC agrees. 

The main point of criticism for the work is the lack of novelty and the low significance of the findings. These were highlighted by all four reviewers.

Perhaps the aspect limiting significance is the fact that the feature extractors are assumed to be fixed, which is unlike most interesting settings in continual learning. This was mentioned by reviewers BTLN, uN9P, e1ZF. It is unclear whether the findings provided in this work would generalize to that setting. On that note, Reviewer e1ZF points out that not adapting the feature extractor could be the source of some inconsistencies observed. Studying this further would improve the work.

Overall, all four reviewers recommend rejecting the paper. The AC agrees with this decision and encourages the authors to consider extending the analysis to situations where the feature extractor is not fixed.
Decision: Reject