On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning
Authors: Lorenzo Bonicelli
Matteo Boschini
Angelo Porrello
Concetto Spampinato
Simone Calderara
Venue: NeurIPS 2022 Accept (2022)
Keywords: continual learning
lifelong learning
incremental learning
rehearsal
lipschitz continuity

Abstract: Rehearsal approaches enjoy immense popularity with Continual Learning (CL) practitioners. These methods collect samples from previously encountered data distributions in a small memory buffer; subsequently, they repeatedly optimize on the latter to prevent catastrophic forgetting. This work draws attention to a hidden pitfall of this widespread practice: repeated optimization on a small pool of data inevitably leads to tight and unstable decision boundaries, which are a major hindrance to generalization. To address this issue, we propose Lipschitz-DrivEn Rehearsal (LiDER), a surrogate objective that induces smoothness in the backbone network by constraining its layer-wise Lipschitz constants w.r.t. replay examples. By means of extensive experiments, we show that applying LiDER delivers a stable performance gain to several state-of-the-art rehearsal CL methods across multiple datasets, both in the presence and absence of pre-training. Through additional ablative experiments, we highlight peculiar aspects of buffer overfitting in CL and better characterize the effect produced by LiDER. Code is available at https://github.com/aimagelab/LiDER.

On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning
Lorenzo Bonicelli'
Matteo Boschini'
Angelo Porrello'
Concetto Spampinato
Simone Calderara'
AlmageLab University of Modena and Reggio Emilia PPeRCeiVe Lab University of Catania
Abstract
Rehearsal approaches enjoy immense popularity with Continual Learning (CL) practitioners. These methods collect samples from previously encountered data distributions in a small memory buffer; subsequently, they repeatedly optimize the latter prevent catastrophic forgetting. This work draws attention to hidden pitfall of this widespread practice: repeated optimization on a small pool of data inevitably leads tight and unstable decision boundaries, which major hindrance to generalization. To address this issue, we propose Lipschitz- DrivEn Rehearsal (LiDER), a surrogate objective that induces smoothness in the backbone network by constraining its layer-wise Lipschitz constants w.r.t. replay examples. By means of extensive experiments, we show that applying LiDER delivers stable performance gain to several state-of-the-ae art rehearsal CL methods across multiple datasets, both in the presence and absence of pre- pre-training. Through additional ablative experiments, we highlight peculiar aspects of buffer overfitting in CL and better characterize the effect produced by LiDER. Code is available https///gttuubcommmmaaggglll//Dp
Introduction
The last few years have seen renewed interest in aiding Deep Neural Networks (DNNs) to acquire new knowledge and, at the same time, retain high performance on previously encountered data. In this regard, the mitigation of catastrophic forgetring (52) has driven the recent research towards novel incremental methods [42 68), often framed under the field of Continual Learning (CL). Among other
valid CL strategies, rehearsal approaches caught the attention of a large body of literature 62 21 [
thanks to their advantages. Simply, they maintain small fixed-size buffer containing fraction of examples from previous tasks; afterward, these examples are mixed together with the ones of the current task, hence provided continuously as training data. In this respect, different approaches
establish different regularization strategies top of the retained examples [63] 49], as well which
kind of information to store (e.g. model responses [15 [2], explanations
25 etc.).
In spite of their widespread application, these approaches fall into common pitfall: as the memory buffer holds only a small fraction of past examples, there is a high risk of overfitting on that memory (75), thus harming generalization. Several approaches mitigate such an issue through data- augmentation techniques, either by generating different versions of the same buffer datapoint [5]
or by combining different examples into a single one [14, [1] Other works R 4 16, 85], instead, select carefully the valuable samples that should be inserted into the buffer: they argue that random selection may pick non-informative and noisy instances, affecting the model generalization.
This work tackles the issue described above from different perspective, viewing catastrophic forgetting in the light of the progressive deterioration of decision boundaries between classes
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Rehearsed Examples
50
Adv. Perturbation Magnitude
20
0
15
Declsion Magnitue Stog
25
10
7u
-50
Roh-Reheersed Examples
50
-5
-10
-25
-20
70
-25
-50
40
Random Perturbation Magnitude
Figure 1: Diagrams derived from 87 describing how the magnitude of an input perturbation affects the model's prediction, measured the difference between the correct logit and the maximum one. The dashed lines delimit the green areas, where the correct response is preserved. The analysis is carried on the examples of the first task of Split CIFAR-I0 and spans across tasks progressing, from (left) to (right). In different rows, the decision surfaces around datapoints either contained in the memory buffer (first) or non-rehearsed (second). We refer to Sec. 5.3 for additional insights.
Indeed, while for the examples of the current task we expect the decision boundaries to be already smooth and robust against local perturbations, the same could not be said for past examples. In fact, the restrained access to only small portion of past tasks increases the epistemic uncertainty [38] of the model: as a consequence, we expect the decision surfaces tied to past classes to slowly erode everywhere, with the exception of certain input regions i.e., those close to the neighborhood of buffer datapoints (thanks to their repeated optimization). We refer to Fig for a visualization of such phenomenon, which shows the evolution of the decision surfaces around the points of the first task of Split 10, from the first (left) to the last one (right). We differentiate the target of the analysis non-rehearsed examples) in distinct rows:
(rehearsed examples the input region where the model outputs correct predictions against local input perturbations [87]) tightens around buffer datapoints (first row) and erodes for non-rehearsed examples (second row).
be seen, the green
Such an intuition motivates our research of novel mechanisms for guaranteeing the robustness of the decision boundaries. To this aim, resort to enforcing the Lipschitz continuity of the model w.r.t. its input: indeed, long-standing research trend 82 45
46
80
29
has pointed out that
such property favors generalization capabilities and robustness to adversarial attacks. In particular, constraining the Lipschitz constant a model intuitively. a bound on how much the model's response change in proportion to change in its input [5] has proven to strengthen the decision surface around a point [22 45 46 80 9I] preventing attacks a given magnitude from changing the output of the classifier.
While these works assessed Lipschitz regularization in the clas- sical scenario (i.e., single joint i.i.d. task), we advocate that is even more beneficial in continual learning, particularly for those approaches based on replay memories. As shown in the inset figure, without explicit regularization, the Lipschitz constant of a model increases for smaller memory buffers: in other terms, its corresponding function space becomes increasingly sensitive w.r.t. local input perturbations (as also highlighted by the lower accuracy attained in the presence of adversarial attacks). In light of the above considerations, uncertainty, which derives from subjecting the model to a low-data training regime.
.59
Adv. Acc.
.55
[M] 1k
4k
L
ascribe such tendency to the higher
To the best of our knowledge, our work is the first attempt to assess the effectiveness of Lipschitz- constrained DNNs in continual learning: particular, we have equipped several widely-known and tfte-offtheeare rehearsal approaches with our Lipschitz-guider optimization objective named Lipschitz-Dris Rehearsal (LiDER), showing that systematically leads to better results in several benchmarks.

---

2 Related works
Continual Learning. CL examines the capability of a deep model learn from a sequence of non-ii.d. classification tasks [23 588 while preventing the onset of catastrophic forgetting [52). To achieve this goal, models are trained according to specifically designed strategies, meant influence their evolution and maximize the retention of previously acquired knowledge.
Among them, regularization methods work by introducing additional constraints in the form of loss terms; they are designed to limit the amount of total change either in parameter space [4 89
or functional space 47 TO Differently, structural methods purposefully organize the allocation of model capacity to prevent interference and facilitate parameter sharing [ 51 66 37.. Lastly,
rehearsal methods store and reuse a subset of previously seen data-points prevent overfitting on current data and avert forgetting [16 49 4 33. While rehearsal strategies are by far the most frequently adopted thanks their effectiveness and flexibility (26] 4], it not uncommon to adopt solutions combining multiple approaches 13 IS
60 [8]. In this paper, we similarly propose a strategy that leverages an existing replay memory buffer to compute an additional regularization term, aimed at conditioning the learning dynamics and avoiding overfitting.
CL evaluations are often carried out in the so-called Task- Inseemncremnttll setting (TIL) 42
89
23
19
that is, the model is provided a task identifier at test time to avoid interference across predictions of distinct tasks. However, recent works put an increasing focus on the harder Jaass-Incrementall setting
(CIL) 32 15 81, which entails the production of a unified prediction encompassing all seen classes. W...t the latter, TIL has been criticized less challenging and realistic benchmark [26] 4] we therefore conduct our main experiments on state-of-the-ar CL models in the CIL setting
Lipschitz- based Regularization. Naturally trained DNNs typically suffer from their overparametriza- tion 190 571. leading to the tendency to overfit the training data by producing jagged decision bound- aries that closely fit the seen examples. On the contrary, model's 's reliability depends on its capability for generalization, which is linked to the appearance of smooth decision boundaries [82] g 86 29].
Starting from the first studies focusing on this simple dichotomy, the Lipschitz constant L of DNN
has been established as a commonplace measure of both smoothness and generalization 82 72 39
and still constitutes key ingredient for current evaluations of model capacity g 30
Most notably, Szegedy et al. 72] verify that constraining L reduces the model's vulnerability to adversarial perturbations. Many current approaches to Adversarial Learning similarly operate either by minimizing L at the global or local level [46] [3 45] or by devising models characterized by a small L by design 22 33]. In other areas, the smoothing effect of L-based regularization has been favorably applied to both GAN training [55) and neural fields [48].
Method
A CL problem usually involves learning a function f from stream of data, which we formalize succession of separate datasets T = (To, Ti..... TTr) : }. where 7, {(x,, ) and T. Tr Ø; the label set Y; for each Tt are non-overlapping, In this setting, the ideal objective consists in minimizing the overall loss over all tasks experienced, formally:
argmin E. E(x.))= [(((())) w]]
(1)
where L is an appropriate loss for solving the task at hand. In a continual scenario, only data from the current task is freely available; therefore, CL methods need to maintain knowledge from the past t tasks in order to solve the overall problem.
For the sake of simplicity, we consider feed forward neural network f(-) = (H K oo* o K-1 K-1 o (.), i.e., a sequence of a-ccivvated linear functions H²(h) = W h (biases are omit- ted). final projection head g(-) = softmax(-) is applied to produce per-class output probabilities. As stated in other works [29]. other common transformations that make up DNNs (e.g., convolutions, max-pool) can also be seen terms of matrix multiplications, thus making our approach applicable to more complex networks.
However, we remark that TIL can also be useful, as it reveals forgetting disentangled from other incremental learning effects. This motivates us to adopt TIL in some of our additional experiments.
3

---

Lipschitz continuity. A function f is said to be Lipschitz continuous if there exists a value L E R such that the following inequality holds:
fx) f S L||x yll2.
Vx,y E R".
(2)
If such value exists, the smallest L that satisfies the condition is usually referred the Lipschitz norm Ilf Therefore, for a single point x E IR", we obtain:
[I f(y)ll2
/
sup *øyEER"
X
y/12
Unfortunately, computing the Lipschitz constant of even the most simple multi-layer perceptron is NP-hard problem [77]. Therefore, several works relied on its estimation by computing reliable upper bounds. As stated in [86] 67], effective way to bound the Lipschitz constant of f f() is to compute the constants of each linear projection H and then aggregate them. In more detail:
W x W y
W max (W&): 11lll2
|| ||.
sup "
sup ÇAO:SER"
(4)
where o max (W&) k.) is the largest singular value of the weight matrix W& (also know as its spectral norm WR sN). To account for non-linear composite functions (e.g.. the residual building blocks of most convolutional architecture), we leverage the following inequality:
[l((((x)) g(=(y))ll2 ||9]|c]|=(x) =(y)112
|||||/||||/.|| yll2 + || gll S ||g||x.|=/|1r
where g(-) and z(-) are two Lipschitz continuous functions characterized by the constants 1 & and || In the case of ReLU-activated networks (but the following result can be extended other common non-linear functions), the forward pass through o' 1, 2, .....
be re-arranged as matrix multiplication by diagonal matrix D E &e x whose diagonal elements equal either zero or one. Therefore, their corresponding Lipschitz constant | 1. On top that, we can compute an upper bound on the Lipschitz constant of the entire network:
K
K II [
|1/||z < || 1|.. 110 11o ..... || 11.
II A=1
H
(5)
Computing the spectral norm of weights matrices. The computation of |Wallsn can be done [55] 29 naively through the Singular Value Decomposition (SVD), yielding, among the others, the largest singular value. Such approach has been applied in recent works (55] 29 however, for complex structures (e.g., convolutions or entire residual blocks) the SVD decomposition is inaccessible, Hence, we rely on the approximation introduced in (67 and compute the largest eigenvalue X; of the Transmitting Matrix TM* (which represents a good proxy of [
TM
[F*) [F*) (F*-1)) [(**)
(6)
where F R Bxds is the L2-normalized feature map produced by the 1-th layer from batch of B samples. Finally, our approach exploits the power iteration method [56] to compute the largest eigenvalue of TMK which backpropagation-triendde
3.1 Lipschitz-Driss Driven Rehearsal
In a continual setting, model is asked i) to be adaptable to incoming samples from the stream (plasticity), and ii) to accurate on past tasks (stability). We seek to ensure balance between these clashing objectives through the two following loss terms.
Controlling Lipschite-continuity To mitigate overfitting on buffer datapoints, we firstly impose that each layer behaves as c-Lipschitz continuous function, for a given real positive target constant cpt
K
Lo-Lip =
M x
ckl-
K
We refer the reader to (67 for additional justifications for this step.

---

During the computation of each Xf, we discard the activation maps incoming from the examples of the current task. Indeed, as we have access to the entire training set (and not subset as holds for old tasks), additional regularization is not needed: the decision boundaries tied to the current task are less prone to the risk of over-adapting to certain points, Regarding the target constants Ck. we could fix them as hyperparameters of our learning objective (as done in [48]) and exploit them as sort of budget assigned to each layer; however, we empirically observed that it is beneficial, instead, learning these targets by means of gradient descent (see Sec. 554). especially in CL scenario where there is no access to the full data distribution. Indeed, these can be interpreted as additional learnable parameters, which represent the appropriate level of strictness each layer should be subjected to.
To avoid trivial solutions, we also encourage the estimated upper bounds to be as much as possible close to zero:
1
Lo-Lip
M47 K
0
Intuitively, when Af + 0, the outputs of the corresponding k-thh layer have low sensitivity to changes in its input. In our intentions, this could relieve continuous rehearsal from eroding the decision surface in way that fits well only certain examples (i.e., those retained in the memory buffer).
Overall objective. The overall objective of LiDER combines the two introduced loss terms; formally:
LIDDE = a + /Lo-LLP--
This objective can be plugged in almost any rehearsal approach. For such a reason, we keep it general and avoid reporting the common loss terms asking for accurate predictions, as their form depends on the specific choices made by each approach. Finally, we further remark that the introduced loss terms require minimal additional computation. Moreover, they do not need additional samples to be retained, besides those that are already present in the memory buffer.
3.2 Relation with other regularization approaches
At first glance, the regularization of our approach can be understood mean to enforce flat minima for each of the tasks, as advocated by Mirzadeh et al. (54] and Yin et al. [84. We remark that they reason in parameter space and pursue flatness of the loss landscape w.r.t. weights: namely, they encourage the model to be robust when perturbations are applied to its weights. Differently, seek models that are robust w.r.t. changes in input space. The two lines may exploit the same mathematical tools such as the Hessian and Lipschitz continuity - but build upon orthogonal axes (weights input): we recommend to always assess which of the two is used as reference property.
In this respect, the bridge between the two objectives is worth- expth-ixploring and still open to debate [83] 81 (6). The authors of [8 reported that, theoretically, no correlation exists between the Hessian w.r.t. weights and the robustness of the model w.r.t. the input. Such a statement is corroborated by Fig. 1 of (87): although a flat minimum is reached in parameter space, non-smooth variations appear in input space. However, the authors of [83] empirically found that models with higher Hessian spectrum w.r.t. weights are also more prone to adversarial attacks. A similar thesis has been argued by the authors of [87), while the third result reported in (36) seems to refute it. Furthermore, Sec. 5.3 of our paper investigated the opposite link and revealed that CL models trained to be robust w.r.t.
input changes tend to attain flatter minima in parameter space.
4
Experiments
To assess our proposal, we perform a suite of experiments through the Mammoth framework 12 17 g
27 13
50). an open-source codebase introduced in [15) for testing CL algorithms. In particular, we show that our method can be casily applied to state-of-the-arn replay methods and enhance their performance in a wide variety of challenging settings and backbone architectures. Moreover, we show that our proposal remains rewarding and can improve the generalization capabilities of CL models even when pre-trained model employed. Such scenario important for twofold reason: i) as shown in [53, pre-training implicitly mitigates forgetting by widening the local minima found in function space, thus making the model more robust to input perturbations; additionally, ii) we accommodate for real-world scenarios where pre-training is usually involved as an initial step. Due to space constraints, we kindly refer the reader to the supplementary material for additional experimental details (e.g., optimizer, hyperparameters, etc.).
5

---

4.1 Benchmarks
We focus our evaluation on the CIL setting and rely on commonly used and challenging image classification tasks. In each experiment, classes from the main dataset are split into separate and disjoint sets, which are then used sequentially train the models herein described.
Split R-00. An initial evaluation is carried by splitting the 32 x 32 images from the 100 classes of CIFAR-100 100 43 into 10 tasks. In detail, run tests: i) using randomly initialized
ResNet18 [3 which constitutes a common scenario in literature [89 62 21 [6]; ti) adopting the
same backbone but with its parameters pre-trained on Tiny ImageNet 77]
Split minilmageNNet. This setting is designed to assess models on longer sequences of tasks. It splits the 100 classes of minilmageNett [76] subset of ImageNet, with images resized to 84 x 84 into 20 consecutive tasks. For this test, we opt for an EfficientNet-B2 [69] backbone with no pre-train.
Split CUB-200. A final, more challenging benchmark involves classifying large-scale 224 x 224 images from the CalD Birds-200-2011 77 dataset, organized in a stream of 10 20-fold classification tasks. This evaluation is designed to highlight the importance of protecting pre-trained weights from forgetting. Indeed, due to the limited size of the training set, competitive performance can only be achieved if each task can benefit from the initialization [20, 88). As backbone network, we opt for the commonly available ResNet50 architecture pre-trained on the ImageNet dataset (24).
We report results in terms of the classification accuracy averaged across all tasks (Final Avg. Acc., FAA). We refer the reader to supplementary materials for the Final Forgetting [19] (FF). For further reference, we include the results of a model jointly trained on all classes which represents an ideal non-CL upper bound- and of model trained sequentially without countermeasures to forgetting.
4.2 Comparison with rehearsal approaches
Benchmarked models. We conduct experiments on multiple state-of-the-ard iidearsal-baset models. If not specified, all methods use reservoir sampling 78 to update the memory buffer.
enhances standard ER by introducing a separate loss for datapoints coming either from the stream and the buffer.
Experience Replay with Asymmetric Cross-Entropy (ER-ACE) [7]:
Dark Experience Replay (DER++) [1]] the authors exploit self-distillation (28) to take advantage of the previously learned knowledge. In addition to storing the one-hot labels, they enforce consistent responses through time by minimizing the L2 norm between current and past logits.
eXtended Dark Experience Replay (X-DER) [1]: it advances DER++ by introducing several amendments. In particular: i) it updates the logits contained in the memory buffer by implanting novel secondary information made available for past classes; ii) it exploits supervised contrastive learning [40] to prepare the model for future tasks. Among the variants discussed in Sec. 5.2
of [2), use the lightweight version called X-DER w/ RPC [5 to prepare future heads.
Incremental Classifier and Representational Learning (iCaRL) 622 it learns a representation suitable for a gearrest-nighhte classification w.r.t. class prototypes stored in the buffer. Additionally, forgetting is prevented by distilling the responses of the model's snapshot at the previous task to both current and replay examples. The buffer is managed through the herding strategy.
Greedy Sampler and Dumb Learner (GDumb) [I]] it trains the model only on buffer datapoints at the task boundaries. By doing so, the authors meant to challenge the improvements made on CL.
Results of our evaluation can found Tab. I across the board, we find LiDER to be capable of improving the performance of all base methods in all evaluated scenarios, both in terms of FAA and FF metrics. Most notably, we find especially beneficial for methods that feature the most compelling results usually DER++ and iCaRL -, suggesting that their higher generalization capability can still benefit from increased smoothness in latent space. Differently, scenarios where a method fail to prevent forgetting i.e. GDumb with reduced buffer size usually feature lower degree of benefit from LiDER but still notable improvement. However, as it takes advantage both from increased buffers and pre-trained initialization, Notably, on Split CIFAR- 100, GDumb with a buffer size of 500 moves from a performance gain of +0.98% to +6.46% when increasing the buffer to 2000 samples and +2.99% with pre-train.
observe considerable performance gap with our proposal.
Finally, the average performance gain of LiDER (i.e., 2.32%, 2.08%, and 4.36% on Split CIFAR-100. Split minilmageNet, and Split CUB-200 respectively) seems to confirm our intuitions.

---

Table 1: For different rehearsal approaches, Final Average Accuracy (FAA) [1] several benchmarks w/wo LiDER regularization.
Benchmark
Split CIFAR-100
Split minilmageNet
Split CUB-200
Pre-training
x
Tiny ImageNer
X
ImageNer
Joint (UB)
73.29
75.20
53.55
78.54
Finetune
9.29
9.52
4.51
8.56
Buffer Size
500
2000
500
2000
2000
5000
400
1000
iCaRL [62
44.04
50.23
56.00
58.10
22.58
22.78
56.52
60.09
+ LiDER
47.02
51.21
57.24
60.97
23.22
23.95
57.12
60.37
DER++ [[S
37.13
52.08
43.65
58.05
23.44
30.43
49.30
61.42
+ LiDER
39.25
53.27
45.37
60.88
28.33
35.04
57.90
67.97
X-DER RPC 12
44.62
54.44
57.45
62.46
26.38
29.91
58.23
64.90
+ LiDER
45.22
54.71
57.76
62.78
29.15
32.56
60.00
65.98
GDumb [6
9.28
19.69
23.09
36.05
15.22
27.79
9.36
18.98
+ LiDER
10.22
26.15
26.09
41.98
15.24
29.49
9.67
19.51
ER-ACE 117
36.48
48.41
48.19
57.34
22.60
27.92
41.83
51.98
+ LiDER
38.43
50.32
48.97
57.39
24.13
30.00
50.89
60.92
Table 2: Comparison between different regularization strategies (FAA, (t]).
Benchmark
Split CIFAR-100
Split minilmageNee
Split CUB-200
Pre-training
x
ImageNet
Buffer Size
500
2000
2000
5000
400
1000
ER-ACE
36.48
48.41
22.60
27.92
41.83
51.98
sSGD
39.59
49.70
22.43
24.12
22.67
29.88
+ oEwC
35.06
45.59
24.32
29.46
48.34
59.74
oLAP
36.58
47.66
23.19
28.77
42.64
52.86
LiDER
38.43
50.32
24.13
30.00
50.89
60.92
DER++
37.13
52.08
23.44
30.43
49.30
61.42
sSGD
38.48
50.74
19.29
24.24
31.08
41.69
oEwC
35.22
51.53
24.53
31.91
51.86
62.54
oLAP
34.48
50.80
25.02
32.78
49.56
63.27
LiDER
39.25
53.27
28.33
35.04
57.90
67.97
4.3 Comparison with regularization approaches
To provide thorough evaluation, we investigate how the following existing regularization techniques (coupled with replay strategies such as ER-ACE and DER++) compare with our solution:
Stable SGD (sSGD) [44 which introduces some specific alterations to the model's training regime that bias the optimization towards flat minima of the loss landscape;
online EwC (oEWC) 655 and Online Laplace (oLAP) (64), applying Hessian-based parameter-
importance estimation to constrain the most significant model parameters for previous tasks;
As reported in Tab. 2 sSGD boosts ER-ACE and DER++ only on Split CIFAR- 100; whereas, its performance degrades severely both on the more complex Split minilmageNet and on Split CUB-200, where we suspect it fails effectively exploit the pre-trained network. By contrast, we find the application of oEwC and oLAP to be rewarding, especially in the presence of pre-training. In this respect, we recall that pre-training has known flattening effect on the loss landscape (53] in such setting, encouraging the model to stay close to its prior (as done by these methods) could be the key to explain the improvements they earry out. Concerning our approach, it proves almost always more effective than any of the other tested approaches.
7

---

Buffer Guessing Game
Table 3: FAA after poisoning the buffer for DER+ with and with- out LiDER.
1
8
8
.4
.4
.4
p
+ER+
+LiDER
.2
.2
.2
MM
500
M
1000
|MM| 2000
.0%
37.14
39.25
0
0
2
4
66
8
.2
.4
.6
.8
0
.2
.4
.6 8
.01%
36.13
38.08
.1%
31.35
35.53
ER-ACE (AUC: 66. 58, .5)
ER-ACE#LIDER (AUC: .66, 50,, .54)
.2%
28.74
30.78
Figure 2: ROC curves for the Buffer Guessing Game, showing the likelihood of given sample belonging in M.
Model analysis
5.1 On the memory buffer
Robustness to buffer poisoning. To evaluate the performance of ML model, practitioners usually resort synthetic benchmarks that simulate the progressive arrival of novel knowledge. While this methodology is useful to compare the effectiveness of new proposals a key element is usually neglected: the labels given to each sample may and in the real-world usually does be incorrect. Notably, this aspect is even more relevant in a CL scenario and especially when rehearsal strategy employed. As previously shown, samples included in the buffer are the ones that are most likely to be overfitted, which would induce severe loss of performance if the label is incorrect.
In the following, we assess the harmful effect of mislabeled examples included in the memory buffer: namely, we randomly assign with probability a new label to the samples as these are added in the buffer (i.e., label poisoning). To better suit a CL scenario, poisoned labels are chosen from those of the current task. Results reported in Tab. show the evaluation performed on top of DER++ for the Split CIFAR-100 benchmark. As one could expect, performance degrades as p increases; however, we also observe that LiDER retains higher level of accuracy against poisoning. This effect suggests that our proposal leads to looser decision boundary around the items stored the buffer.
Buffer Guessing game. We propose a novel experiment to further illustrate the tendency of rehearsal- based CL models to overfitting- As posited in Sec. [ substantially different regime affects replay and stream examples: the former plays much larger role in shaping the decision boundary w.r.t. the latter. To validate this intuition, we propose a simple buffer guessing game: given a rehearsal-baser model ff fully trained on CL benchmark and the dataset To used in its first encountered task, we aim to find M To (i.e. the subset of data-points that are included the model's buffer).
We approach the game by associating each x E with a score S computed in a neighborhood of x. Such a score quantifies the mean height of the decision surface, i.e., the difference between the probability of the right class and the highest one. As specified in [87. we model the neighborhood by leveraging random perturbations; moreover, we compute 85 w.r.t. to the TII. prediction function in order to avoid the influence of inter-task biases our results. Finally, to assess whether these scores can help separating bn-buffer examples from all the other, we evaluate the ROC curve obtained from these scores. Fig. reports the results for ER-ACE and ER-ACE+LiDER
Split CIFAR- 100 across different buffer sizes. We see that: i) ER-ACE makes easier to reconstruct the content of the buffer, as indicated by larger ROC-AUC scores w.r.t. ER-ACEELiDER; ii) in line with expectations, this effect increased when employing smaller memory buffers, as this leads to the repeated optimization of smaller pool of data.
5.2 Applying LiDER on current task's examples
Our approach devotes regularization specific for buffer datapoints; however, it could be investigated its benefits extend also to the current task. In this respect, we have performed several experiments switching the target of regularization: not the examples from the replay memory, but those belonging to the current task. Tab. 4 reports the results: while the original approach delivers consistent improvements, applying LiDER on incoming examples delivers inferior results.
8

---

Table 4: Comparison (FAA) between two possible targets of regularization: examples from the
current task (stream) vs buffer datapoints (standard LiDER).
Benchmark
Split CIFAR-100
Split minilmageNet
Split CUB-200
Pre-training
X
Tiny ImageNet
ImageNet
Buffer Size
500
2000
500
2000
2000
5000
400
1000
ER-ACE
36.48
48.41
48.19
57.34
22.60
27.92
41.83
51.98
LiDER (curr. task)
37.54
50.37
48.94
57.07
23.35
29.25
48.44
59.60
LiDER (buffer)
38.43
50.32
48.97
57.39
24.13
30.00
50.89
60.92
DER++
37.13
52.08
43.65
58.05
23.44
30.43
49.30
61.42
LiDER (curr. task)
34.78
49.76
44.48
59.39
24.84
31.05
56.96
66.63
LiDER (buffer)
39.25
53.27
45.37
60.88
28.33
35.04
57.90
67.97
Adv. Pert. Magnilude
ER-ACE
ER-ACE
Magn. S(a)
50
25
25
17
8
0
Decision 2
25
-17
-50
To
72
70
72
-25
40
Random Perturbation Magnitude
Figure 3: Effect of our proposal on the robustness of the decision boundary produced by ER-ACE across subsequent tasks. Higher values (in green) indicate areas of high confidence and span the possible directions where perturbations are not disruptive (best seen in color).
Such an evidence can be explained by reviewing the distinct data regimes the current and previous tasks are subject to. While learning the current task, indeed, the model can access many and many samples from its underlying distribution; therefore, the epistemic uncertainty [3 reduces and the learned decision boundaries are likely to be smoother. In this case, the Lipschitz regularization could represent an overkill, threatening to restrain the learning with no advantages. In our formulation, instead, only few examples are available for retaining the knowledge of past tasks: the risk of pro- gressive overfitting which we expressed through the progressive degradation of decision boundaries more severe: therefore, tailored countermeasures more likely be effective.
5.3 Generalization measures
Decision surface of LiDER. Fig. presents depicts the model's tolerance input perturbations in the form of decision surface plot [87] Such a visualization is constructed by focusing on a set of
perturbations Fp - x -- j.B computed around data-poine x (a is a random divergence direction and B corresponds to the direction induced by the first step of non-targeted FGSM attack
(44). The plot shows the respective values of the decision function S(x,), where S(x) ) & f(x), max; f (xr); (with f(-) indicating pre-softmax responses) and highlights decision boundary of the model (i.e. the locus of (S(p)((rr))) 0}). in correspondence of which the model accuracy fails.
In Fig. we adopt the same approach to compare the decision boundaries around rehearsed to examples for ER-ACE with and without LiDER. While both models start with similar robust decision landscape in To. later tasks reveal a clear shrinking behavior in ER-ACE. On the contrary, introducing L-based regularization lead minimal decision boundary deterioration in later tasks.
Loss Landscape of LiDER. Previous CL works proposed investigating the generalization capabilities of CL model by evaluating the flatness of its attained minima [1] 12 We likewise evaluate the
effect of LiDER both in terms its resilience to weight perturbations 57 39 and the eigenvalues of
the Hessian of the loss function 39 34 in Fig. We see that DER++ and ER-ACE combined with
LiDER see an improvement of both metrics. This expected, as the Lipschitz constant of DNN
has been commonly interpreted as generalization measure [82 72

---

Weight Perturbation [4]
Hessian Eigenvalues [L]
Table 5: Ablation: fixed Lips-
chitz targets learned targets.
15
DER
Tr((F) Tr(FF) [x10
1.5
10
ER-ACE
LLDER
Model
[M]
Fixed Igt Eq.
+LIDER
.5
A ER-ACE
+DER++ 500
36.42
39.25
10
0.1
0.2
0.32
0.42
0.53
0.63
070
888
0.90
DER
LIDER ER-ACE
ERACE
+UDER
pre +
500
2000
51.52
34.99
38.43
53.27
Sigma
ER-ACE
2000
46.70
48.97
Figure 4: (Left) Robustness models regularized with LiDER
DER++ 500
43.16
45.37
against weight perturbations (best seen in color). (Right) Flatness around the minima found during optimization, measured as the sum of the eigenvalues of the Hessian matrix.
ER-ACE 500
45.21
48.97
DER++ 2000
59.53
60.68
ER-ACE 2000
54.82
57.39
5.4 Optimization with fixed target
Instead of constraining the optimization of the Lipschitz targets of Eq. by an additional contribution, one might argue that similar effect could be obtained by fixing an initial value and force the model to reduce its capacity until it meets the desiderata [48]. In Tab. 5 we empirically show that such approach does not lead to satisfactory results in a CL setting, with our proposal consistently exceeding its performance on various settings and base methods.
We finally refer to the supplementary materials for additional studies e-g., the evaluation of the single- epoch scenario [49] the efficiency-accuraee trade-off, and a sensitivity analysis of hyperparameters.
Conclusions
We present LiDER, novel regularization strategy to compensate for the phenomenon of buffer overfitting in rehearsal-bace samples to be smoother by bounding the complexity of the model. We show that such approach can be readily applied to diverse state-of-the-ar models and remains competitive across various settings. Finally, we provide an extensive investigation assess the validity of theoretical assumptions.
Continual Learning: it enforces the decision boundary of replayed
Limitations & Societal Impact
Our approach relies coarse estimation of the Lipschitz constant and, in particular, on its upper bound (the exact computation is NP-hard problem). We highlight the tighter bounds have been proposed (35 333, at the expense of higher complexity. which could clash with the incremental scenario subject our work. Moreover, our approximation cannot be applied not-Lipschitz continuous layers (e.g., cross-attention) [1). In the following, we briefly recap the notions through which our approximation can be favorably refined, leaving for future works the investigation of the efficiency/accuracy trade-off inherent in these advanced estimations.
could help provide, for each activation function o', tighter upper bounds of its Lipschitz constant || - It is noted that our assumption ||o||l < refers to the global 1-Lipschitz continuity, which requires the inequality hold for all points from the input domain. Such a requirement is too strict and not representative of DNNs, as the input of a layer does not distribute uniformly but is often constrained subspace, whose shape depends on the activations of the previous layers. For this reason, recent works 35 33
refer to a different property i.e., the local Lipschitz continuity, which bounds output perturbation only for certain input regions. Notably, it has been proved to provide tighter upper bound (see Theorem 1 of (33) ) and hence more effective regularizing signal.
Concerning societal impact, we do feel that this work will have detrimental applications that might affect any public. We only remark that, as our approach is based on rehearsal techniques that store in-plain data, it cannot be used in those scenarios where privacy constraints are crucial.
Acknowledgement. This paper has been supported by: Italian Ministerial grant PRIN 2020 "LEGO.AI: LEarning the Geometry of knOwledge in Al systems", n. 2020TA3K9N; FF4EuroHPC: HPC Innovation for European SMEs, Project Call 1. FF4EuroHPC has received funding from the European High-Perte Biigh-Perfrrmacce Computing Joint Undertaking (JU) under grant agreement 951745.
10

---

References
[1] Davide Abati, Jakub Tomezak, Tijmen Blankevoort, Simone Calderara, Rita Cucchiara, and Babak Ehte- shami Bejnordi. Conditional channel gated networks for task-aware continual learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2020.
[2] Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) forget. In Proceedings of the European Conference on Computer Vision, 2018.
[3] Rahaf Aljundi, Eugene Belilovsky, Tinne Tuytelaars, Laurent Charlin, Massimo Caccia, Min Lin, and Lucas
Online continual learning with maximal interfered retrieval. Advances Neural Information Processing Systems, 2019.
[4] Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio. Gradient based sample selection for online continual learning. In Advances in Neural Information Processing Systems, 2019.
[5] Cem Anil, James Lucas, and Roger Grosse. Sorting out lipschitz function approximation. In International Conference on Machine Learning, 2019.
[6] Elahe Arani, Fahad Sarfraz, and Bahram Zonooz. Learning fast, learning slow: A general continual learning method based complementary learning system. In International Conference on Learning Representations Workshop. 2022.
[7] Jihwan Bang, Heesu Kim, YoungJoon Yoo, Jung-Woo Ha, and Jonghyun Choi. Rainbow memory: Continual learning with a memory of diverse samples. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2021.
[8] Peter L Bartlett, Dylan J Foster, and Matus J Telgarsky. Spectrally- normalized margin bounds for neural networks. In Advances in Neural Information Processing Systems, 2017.
[9] Giovanni Bellitto, Matteo Pennisi. Simone Palazzo, Lorenzo Bonicelli, Matteo Boschini, Simone Calderara. and Concetto Spampinato. Effects of auxiliary knowledge continual learning. In International Conference on Pattern Recognition, 2022.
[10] Ari S Benjamin. David Rolnick, and Konrad Kording. Measuring and regularizing networks in function space. In International Conference Learning Representations Workshop, 2019.
[11] David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel. Mixmatch: A holistic approach to semi-supervised learning. Advances in Neural Information Processing Systems, 2019.
[12] Mattco Boschini, Lorenzo Bonicelli, Pietro Buzzega, Angelo Porrello, and Simone Calderara. Class- incremental continual learning into the extended ver-verse. IEEE Transactions Pattern Analysis and Machine Intelligence, 2022.
[13] Matteo Boschini, Lorenzo Bonicelli, Angelo Porrello, Giovanni Bellitto, Matteo Pennisi, Simone Palazzo. Concetto Spampinato, and Simone Calderara. Transfer without forgetting. In Proceedings of the European Conference on Computer Vision, 2022.
[14] Matteo Boschini, Pietro Buzzega, Lorenzo Bonicelli, Angelo Porrello, and Simone Calderara. Continual semi-supervised learning through contrastive interpolation consistency. Pattern Recognition Letters, 2022.
[15] Pietro Buzzega, Matteo Boschini, Angelo Porrello. Davide Abati, and Simone Calderara. Dark Experience for General Continual Learning: a Strong, Simple Baseline. In Advonces in Neural Information Processing Systems, 2020.
[16] Pietro Buzzega, Matteo Boschini, Angelo Porrello, and Simone Calderara. Rethinking Experience Replay: Bag of Tricks for Continual Learning. In International Conference Pattern Recognition, 2020.
[17] Lucas Caccia, Rahaf Aljundi, Nader Asadi, Tinne Tuytelaars, Joelle Pineau, and Eugene Belilovsky, New Insights on Reducing Abrupt Representation Change in Online Continual Learning. In International Conference on Learning Representations Workshop, 2022.
[18] Hyuntak Cha, Jacho Lee, and Jinwoo Shin. Co21: Contrastive continual learning. In IEEE International Conference on Computer Vision, 2021.
[19] Arslan Chaudhry, Puncet K Dokania, Thalaiyasingam Ajanthan, and Philip HS Tor. Riemannian walk for incremental learning: Understanding forgetting and intransigence, In Proceedings of the European Conference on Computer Vision, 2018
11

---

[20] Arslan Chaudhury Mare' Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient Lifelong Learning with A-GEM. In International Conference on Learning Represeniations Workshop. 2019.
[21] Arslan Chaudhay. Marcus Rohrbach, Mohamed Elhoseiny. Thalaiyasingam Ajanthan, Puneet K Doka- nia, Philip HS Tomr, and Mare' Aurelio Ranzato. On tiny episodic memories in continual learning. In International Conference on Machine Learning Workshop. 2019.
[22] Moustapha Cisse, Piotr Bojanowski, Edouard Grave, Yann Dauphin, and Nicolas Usunier. Parseval networks: Improving robustness adversarial examples. In International Conference on Machine Learning, 2017.
[23] Matthias De Lange, Rahaf Aljundi, Mare Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Greg Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.
[24] Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Fei-Fei. Imagenet: large-scale hicrarchical image database. In Proceedings of the IEFE conference on Computer Vision and Pattern Recognition, 2009.
[25] Sayna Ebrahimi, Suzanne Petryk, Akash Gokul, William Gan, Joseph E Gonzalez, Marcus Rohrbach, and Trevor Darrell. Remembering for the right reasons: Explanations reduce catastrophic forgetting. Applied Al Letters, 2021.
[26] Sebastian Farquhar and Yarin Gal. Towards Robust Evaluations of Continual Learning. In International Conference on Machine Learning Workshop, 2018.
[27] Enrico Fini, Victor G Turrisi da Costa, Xavier Hameda-Pinedds Elisa Ricci, Karteek Alahari, and Julien Mairal Self supervise models are continual learners. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. pages 9621-9630, 2022.
[28] Tommaso Furlanello, Zachary C Lipton, Michael Tschannen, Laurent Itti, and Anima Anandkumar. Born again neural networks. International Conference on Machine Learning, 2018.
[29] Henry Gouk, Eibe Frank, Bernhard Pfahringer. and Michael J Cree. Regularisation of neural networks by enforcing lipschitz continuity. Machine Learning, 2021.
[30] Florian Graf, Sebastian Zeng. Mare Niethammer, and Roland Kwitt. On measuring excess capacity in neural networks. arXiv preprint arXiv:2202.0800 2022.
[31] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2016.
[32] Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning unified classifier incemennally via rebalancing. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2019.
[33] Yujia Huang, Huan Zhang, Yuanyuan Shi, J Zico Kolter, and Anima Anandkumar, Training certifiably robust neural networks with efficient local lipschitz bounds. In Advances in Neural Information Processing Systems, 2021.
[34] Stanislaw Jastrzebski, Zachary Kenton, Devansh Arpit, Nicolas Ballas, Asja Fischer, Yoshua Bengio, and Amos Storkey. Three factors influencing minima in sgd. In International Conference on Artificial Neural Networks, 2018
[35] Matt Jordan and Alexandros G Dimakis. Exactly computing the local lipschitz constant of relu networks. Advances in Neural Information Processing Systems, 2020.
[36] Sandesh Kamath, Amit Despande, and KV Subrahmanyam. On adversarial robustness of small vs large batch training. In International Conference on Machine Learning Workshop. 2019.
[37] Zixuan Ke, Bing Liu, and Xingchang Huang. Continual learning of a mixed sequence of similar and dissimilar tasks. In Advances in Neural Information Processing Systems, 2020.
[38] Alex Kendall and Yarin Gal. What uncertainties do we need in bayesian deep learning for computer vision? Advances in Neurai Information Processing Systems, 2017.
[39] Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang. On large- batch training for deep learning: Generalization gap and sharp minima. In International Conference on Learning Representations Workshop, 2017.
12

---

[40] Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot.
Ce Liu, and Dilip Krishnan. Supervised Contrastive Learning. In Advances in Neural Information Processing Systems, 2020.
[41] Hyunjik Kim, George Papamakarios, and Andriy Mnih. The lipschitz constant of self-attention. In International Conference on Machine Learning, 2021.
[42] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Saabska-Barrinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 2017.
[43] Alex Krizhevsky al. Learning multiple layers of features from tiny images. Technical report, Citescer, 2009.
[44] Alexey Kurakin, Ian Goodfellow, Samy Bengio, et al. Adversarial examples in the physical world. In International Conference on Learning Representations Workshop. 2016.
[45] Sungyoon Lee, Jaewook Lee, and Saerom Park. Lipschitz-cernn ceattitz-certifiable training with a tight outer bound. In Advances in Neural Information Processing Systems, 2020.
[46] Klas Leino, Zifan Wang, and Matt Fredrikson. Globally-robust neural networks. In International Conference on Machine Learning, 2021.
[47] Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017.
[48] Hsueh-Ti Derek Liu, Francis Williams, Alee Jacobson, Sanja Fidler, and Or I Litany. Learning smooth neural functions via lipschitz regularization. arXiv preprint arXi: 2022.
[49] David Lopez-Paz and Marc' Aurelio Ranzato. Gradient episodic memory for continual learning. In Advances in Neural Information Processing Systems, 2017.
[50] Divyam Madaan, Jachong Yoon, Yuanchun Li, Yunxin Liu, and Sung Ju Hwang. Representational continuity for unsupervised continual lcarning. In International Conference on Learning Representations Workshop, 2022.
[51] Arun Mallya and Svetlana Lazehnik. Packnet: Adding multiple tasks to a single network by iterative pruning. In Proceedings of the IEEE conference on Computer Vision and Puttern Recognition, 2018.
[52] Michael McCloskey and Neal Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. Psychology of learning and motivation, 1989.
[53] Sanket Vaibhav Mchta, Darshan Patil, Sarath Chandar, and Emma Strubell. empirical investigation of the role pre-training in lifelong learning. In International Conference on Machine Learning, 2021.
[54] Seyed Iman Mirzadeh, Mehrdad Farajtabar, Razvan Pascanu, and Hassan Ghasemzadeh. Understanding the Role of Training Regimes Continual Learning. Advances in Neural Information Processing Systems, 2020.
[55] Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. In International Conference on Learning Representations Workshop. 2018.
[56] Yuji Nakatsukasa and Nicholas J Higham. Stable and efficient spectral divide and conquer algorithms for the symmetric cigenvalue decomposition and the svd. SIAM Journal on Scientific Computing, 2013.
[57] Behnam Neyshabur, Srinadh Bhojanapalli, David Mcallester, and Nati Srebro. Exploring Generalization in Deep Learning. In Advances in Neural Information Processing Systems, 2017.
[58] German Parisi, Ronald Kemker, Jose Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: A review. Neural Networks, 2019.
[59] Federico Pemicii, Mattco Bruni, Claudio Baccchi, and Alberto Del Bimbo. Regular polytope networks. IEEE Transactions on Neural Networks and Learning Systems, 2021.
[60] Quang Pham, Chenghao Liu, and Steven Hoi. Dualnet: Continual learning, fast and slow. In Advances in Neural Information Processing Systems, 2021.
[61] Ameya Prabhu, Philip HS Torr, and Puneet K Dokania. GDumb: A simple approach that questions our progress in continual learning. In Proceedings the European Conference on Computer Vision, 2020.
13

---

[62] Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lamperi. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition,, 2017.
[63] Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. Learning Learn without Forgetting by Maximizing Transfer and Minimizing Interference. In Interna- tional Conference on Learning Representations Workshop, 2019.
[64] Hippolyt Ritter, Aleksandar Botev, and David Barber. Online structured laplace approximations for overcoming catastrophic forgetting. Advances in Neural Information Processing Systems, 2018.
[65] Jonathan Schwarz, Wojeiech Czarnecki, Jelena Luketina, Agnieszka Trabska-Barwinske Yee Whye Tch, Razvan Pascanu, and Raia Hadsell. Progress compress: A scalable framework for continual learning. In International Conference on Machine Learning, 2018.
[66] Joan Serra, Didac Suris, Marius Miton, and Alexandros Karatzoglou. Overcoming Catastrophic Forgetting with Hard Attention to the Task. International Conference on Machine Learning, 2018.
[67] Yuzhang Shang. Bin Duan. Ziliang Zong, Liqiang Nie, and Yan Yan. Lipschitz continuity guided knowledge distillation. In IEEF International Conference Computer Vision 2021.
[68] Konstantin Shmelkov, Cordelia Schmid, and Karteek Alahari. Incremental learning of object detectors without catastrophic forgetting. In IEEE International Conference on Computer Vision, 2017.
[69] Yi-Fan Song. Zhang Zhang, Caifeng Shan, and Liang Wang. Constructing stronger and faster baselines for skeleton-based based action recognition. IEEE Transactions Pattern Analysis and Machine Intelligence, 2022.
[70] Stanford. Tiny ImageNet Challenge (CS231n), 2015. https:// /anp::/ Seaaggll.com// c/
[71] David Stutz, Matthias Hein, and Bernt Schiele. Relating adversarially robust generalization to flat minima. In IEEE International Conference on Computer Vision, 2021.
[72] Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing propertics of neural networks. International Conference on Learning Representations Workshop, 2014.
[73] Yusuke Tsuzuku, Issei Sato, and Masashi Sugiyama. mipschitz-margin training: Scalable certification of perturbation invariance for deep neural networks. In Advances in Neural Information Processing Systems, 2018.
[74] Gido M van de Ven and Andreas S Tolias. Three continual learning scenarios. In Neural Information Processing Systems Workshops, 2018
[75] Verwimp. Mauhiis De Lange, and Tinne Tuytelaars. Rehearsal revealed: The limits and merits of
revisiting samples in continual learning. In JEEE International Conference on Computer Vision, 2021.
[76] Oriol Vinyals, Charles Blundell, Timothy Lillicrap. Daan Wierstra, et al. Matching networks for one shot learning. In Advances in Neural Information Processing Systems, 2016.
[77] Aladin Virmaux and Kevin Scaman. Lipschitz regularity of deep neural networks: analysis and efficient estimation. In Advances in Neural Information Processing Systems, 2018.
[78] Jeffrey S Vitter. Random sampling with reservoir. ACM Transactions Mathematical Software, 1985.
[79] C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. The Caltech-UCSS Birds-200-2011 Dataset. Technical report, California Institute of Technology, 2011. 1.
[80] Lily Weng, Huan Zhang, Hongge Chen, Zhao Song, Cho-Jui Hsieh, Luca Daniel, Duane Boning, and Inderjit Dhillon. Towards fast computation of certified robustness for relu networks. In International Conference on Machine Learning. 2018
[81] Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Yc, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning, In Proceedings of the IEFE conference Computer Vision and Pattern Recognition, 2019.
[82] Huan Xu and Shie Mannor. Robustness and generalization. Machine Learning, 2012.
[83] Zhewei Yao, Amir Gholami, Qi Lei, Kurt Keutzer, and Michael W Mahoney. Hessian- based analysis of large batch training and robustness to adversaries. Advances in Neural Information Processing Systems, 2018.
14

---

[84] Dong Yin, Mehrdad Farajtabar, Ang Li, Nir Levine. and Alex Mott. Optimization and generalization of regularization-based continual learning: loss approximation viewpoint. arXiv preprini arXiv:2006.1097 2020.
[85] Jaehong Yoon, Divyam Madaan, Funho Yang, and Sung. Ju Hwang. Online coreset selection for rehearsal- based continual learning. In International Conference on Learning Representations Workshop, 2022.
[86] Yuichi Yoshida and Takeru Miyalo. Spectral norm regularization for improving the generalizability of deep learning. arXiv preprint arXiv: 10941, 2017.
[87] Fuxun Yu, Zhuwei Qin, Chenchen Liu, I. Liang 7hao, Yanzhi Wang, and Xiang Chen. Interpreting and evaluating neural network robustness, In International Joint Conference on Artificial Intelligence, 2019.
[88] Lu Yu, Bartlomicj Twardowski, Xialci Liu, Luis Herranz, Kai Wang, Yongmei Cheng, Shangling Jui, and Joost van de Weijer. Semantic drift compensation for class-incremental learning. In Proceedings of the IEEE conference Computer Vision and Pattern Recognition, 2020.
[89] Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In International Conference on Machine Learning, 2017.
[90] Chiyuan Zhang. Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. In International Conference on Learning Representations Workshop, 2017.
[91] Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent Ghaoui, and Michael Jordan. Theo- retically principled trade-off between robustness and accuracy. In International Conference on Machine Learning, 2019.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contribu- tions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [Yes]
(d) Have you read the cthics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material URLY? [Yes] The code for reproducing experiments can be found the supplemental material and will made publicly available publication.
(b) Did you specify the training details (e.g.. data splits, hyperparameters, how they were chosen)? [Yes] We cover these aspects in the main paper (Sec. and. more in depth, in the supplemental materials.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)? [Yes] Due to space constraints, variance values for experiments are included in the supplemental material.
(d) Did you include the total amount of compute and the type of resources used (c.g.. type of GPUs, internal cluster, or cloud provider)? [No] Experiments were conducted on an on on-premise desktop machine, equipped with a RTX 2080 consumer GPU. However, we did not keep a detailed record of total computation time.
4. If you are using existing assets (e.g., code, data, models) curating/relcasing new assets...
(a) If your work uses existing assets, did you cite the crcators? [Yes] In our work we use original code, with the only exceptions for the evaluation datascts whose original papers are eited.
(b) Did you mention the license of the assets? [N/A] Our work uses original code and freely available datasets.
(c) Did you include any assets either in the supplemental material URL? [Yes]
15

---

(d) Did you discuss whether and how consent was obtained from people whose data you're re us- ing/curating? [N/A]
(c) Did you discuss whether the data you using/curating contains personally identifiahle informa- tion offensive content? [N/A] We datasets commonly established and used in literature, for which, the best our knowledge, the issues mentioned above have never been found to affect them.
5. If you used crowdsoureing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
16


Reviews:
[{"review_id": "_9q6hARU2RN", "reviewer": "Reviewer_QKp5", "summary": "The authors focus on the Catastrophic Forgetting due to overfitting to the small dataset of memorised samples.  \nThey show the degradation of the decision boundary due to overfitting to a subset of data on the first tasks.  \nThey also show that the Lipchitz constant increases the smaller the memory buffer, which is tightly related to the decision boundary's deterioration.  \nThey introduce a regularisation scheme to minimise the Lipchitz constant over the samples in the memory.", "questions": "- In 5.4, could you please explain the selection process of the fixed target ?\n- Could you clarify which performance is reported in Table 4 ?\n\nTiny recommendations for clarity (don't impact the score) :\n- In the appendix, presenting the hyper-parameters in a table format may make them more readable :)  \n- I think it would be more clear to add the standard errors in the same table as the means. Maybe split the tables into two tables : FAA and FF.", "limitations": "The authors have adequately addressed the limitations and potential negative impact of the work. ", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Originality\n### 1- Are the tasks or methods new? \n\nThe method LiDER is new. However, the idea to constraint the Lipschitzness of the neural network and the way to do so is not new. \n\n### 2- Is the work a novel combination of well-known techniques? \n\nYes, the work is an application of techniques to minimise the Lipschitzness of neural networks to Continual Learning.\n\n### 3- Is it clear how this work differs from previous contributions? \n\nYes, the main way this work differs from previous contributions are. :\n- First application of the Lipschitzness minimisation to Continual Learning\n- The design of the optimisation objective on the Lipschitz constant for continual learning.\n\n### 4- Is related work adequately cited ?\nTo my knowledge, the related work is adequately cited\n\n## Quality\n### 1- Is the submission technically sound ? \n\nYes, the submission is technically sound.\n\n### 2- Are claims well supported (e.g., by theoretical analysis or experimental results) ?\n\nMost claims were supported by experiments as evidence.\n\n### 3- Are the methods used appropriate ?\n\nThe idea of focusing on the Lipschitzness seems relevant, given the improvements experimental results. \nAlso, several works have pointed to the link between the flatness of the loss and Catastrophic Forgetting. \nIntuitively, a network with a smaller Lipschitzness constant would converge to a more stable optima.  \n\n### 4- Is this a complete piece of work or work in progress ?\n\nThis is a complete piece of work.\n\n### 5- Are the authors careful and honest about evaluating both the strengths and weaknesses of their work ?\n\nThe authors discussed the limitations of their work in terms of : \n- computational requirements.\n- the non applicability of the method to attention based architectures.\n\n## Clarity:\n### 1- Is the submission clearly written?\n- The paper is well written and easy to follow.\n- The experimental setup is presented in detail\n- The background is clearly presented and helped me ramp-up on the Lipschitzness of neural networks.\n\n### 2- Is it well organised? (If not, please make constructive suggestions for improving its clarity.)\n\nOverall the paper is well organised and was easy to follow.\n\n### 3- Does it adequately inform the reader? (Note that a superbly written paper provides enough information for an expert reader to reproduce its results.)\n\nThe hyperparameters to reproduce the experiments are provided in the appendix. Also, the code was provided in supplementary materials.\n\n## Significance:\n### 1- Are the results important? \n\nStrengths : \n- The method is easy to implement and applicable as an addition to a large number of algorithms. \n- The method achieves improvements on the order of 5 to 8% gains in FAA for split-ImageNet and Split CUB-200, for  DER++, and up to 10% in AF.\n\nWeaknesses :\n- The gains LiDER bring are most significant, mostly in split miniImageNet and Split CUB-200 and for a subset of the algorithms.\n\n### 2- Are others (researchers or practitioners) likely to use the ideas or build on them?\n- Given that the method is easy to implement, practitioners are likely to benchmark the idea against other methods for applications.\n- However, it's unclear to which extend the computational overhead of the power iteration method makes LiDER applicable to large scale real world applications.\n\n### 3- Does the submission address a difficult task in a better way than previous work? \n\nThe method is designed to be plugged on top of other methods. In the experiments, this leads to an improvement or the same performance in most cases.\n\n### 4- Does it advance the state of the art in a demonstrable way? \n- The method achieves improvements on the order of 5 to 8% gains in FAA for split-ImageNet and Split CUB-200, for  DER++, and up to 10% in AF.\n- The gains LiDER bring are most significant, mostly in split miniImageNet and Split CUB-200 and for a subset of the algorithms.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Rlt9j9ugum", "reviewer": "Reviewer_tK25", "summary": "The authors propose to constrain lipschitz constant  for the Continual Learning settings in order to control problems related to reusing the same replay samples. They show that this leads to increased performance across a variety of rehearsal based methods", "questions": "- The method introduces additional hyperparameters alpha beta, how robust are these?\n- What would happen if the incoming data, not the buffer data was used to compute  L_c-Lip? ", "limitations": "Yes", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "*Strengths*\n\n- The method performs well across a variety of datasets and methods. \n\n\n\n*Weakness*\n- the motivation can be hard to follow at times, particularly the 3rd to last paragraph of the introduction is  vague and overly informal. Figure 1 should be more clearly explained as it takes sometime to understand what it is trying to show\n- The authors do not consider the online setting where rehearsal methods are commonly applied", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Gqf4j05RQPk", "reviewer": "Reviewer_oByL", "summary": "The paper proposes inclusion of a smoothness regulariser in rehearsal training for alleviating the problem of catastrophic forgetting.  Authors present empirical evidence showing that adding the proposed regularise to existing rehearsal techniques reduces forgetting. ", "questions": "Why is the regularisation objective in equation 9 is so complicated?  In on term you try to make eigenvalue equal to c, in the other term you try to minimise c.  Why not just create an objective that minimises the eigenvalue?  Surely, if eigenvalue is less than c, that’s good...and so the first term is not always doing the right thing…and the combination of two is just a very complicated way of minimisation of the eigenvalue.", "limitations": "The limitation is that based on the lack of evaluation of the propose regularisation method against existing ones, it’s hard to tell if it is that good on its own.  As an investigation of the effectiveness of combining rehersal and regularisation, it’s not exhaustive enough, because it only takes into consideration a new regularistion method.  I think, as it is, the work is a good start to a possibly interesting investigation, that needs much more done in terms of critical evaluation of various combinations of rehearsal and other regularisation methods.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "This paper tackles two aspects of catastrophic forgetting: a new regularisation objective and the idea of combining regularisation (that alleviates catastrophic-forgetting) with rehearsal methods.  So I’ll tackle these two aspect separately.\n\nAs a new regulariser, I think the propose idea is not that fresh.  It seems like it’s just another facet of existing regularisation techniques that attempt to find flat minima for each of the tasks (Mirzadeh.etal 2020, Yin.etal 2021).   In fact, both those works already propose minimisation of the eigenvalue of the hessian matrix of the model - the only thing new here would be doing that on a per layer basis on weights…which may make the proposed method computational simpler.  Also, whereas Mirzadeh.etal2020 and Yin.etal2021 make  assumptions about Lipschitz smoothness of the learning model, in this work the objective is to maximise that smoothness.  I wonder if this work (just like Fisher-matrix based Elastic Weight Consolidation) is a special case of the Hessian-based regularisation proposed in Mirzadeh.etal2020 and Yin.etal2021?  If so, is the proposed estimator of smoothness better than the one introduced in Ritter.etal.2018?  Hard to tell at the moment, since the authors here don’t compare their regulariser against existing ones.\n\nThe second aspect, of combining rehearsal methods with regularisation methods for the purposes of reducing catastrophic forgetting is kind of interesting - perhaps a bit obvious, but nevertheless something that needs to be explored and evaluated.  However, in this case, I think there needs to be an empirical evaluation using other regularisers in addition to the proposed one.  Is the combination of rehearsal and the proposed regulariser better than a combination of rehearsal and the smart choices of training that flatten the minima proposed in Mirzadeh.etal2020?  Or is it better than combination of rehearsal than the regulariser of Ritter.etal.2018?  Is it slightly worse, but much faster?  And since these existing regularisers rely on Taylor series approximation of the Hessian of the loss function at minima of given tasks, could the fact that in rehersal we have access to some data from previous tasks be useful to improve that estimation?  Yin.etal2021 show the exact relationship of the error in the estimate of the Hessian to forgetting.\n\n\n\nMirzadeh.etal2020 - “Understanding hte role of training regimes in continual learning”\nYin.etal2021 - “Optimization and Generalization of Regularization-Based Continual Learning: a Loss Approximation Viewpoint”\nRitter.etal.2018 - “Online structured Laplace approximations for overcoming catastrophic forgetting”\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "09BBmYVfavP", "reviewer": "Reviewer_PLZ4", "summary": "In this work authors propose LIDER - an extension to the existing continual learning methods with buffer replay, through additional regularisation based on Lipschitz constant that reduces overfitting to examples stored in the buffer, by improving the generalisation of the classifier. In experimental studies LIDER improves the performance of recent state-of-the-art methods.", "questions": "- In this submission a second - additional regularisation term is introduced that forces Lipschitz constants of individual layers to be as close to 0 as possible. As stated in the work, employment of this regularisation should lead to lower sensitivity to changes in the input. I miss the direct link, why this should prevent continually trained model from overfitting to limited examples in the buffer?\n- Can authors provide any explanation why Lipschitz constant is higher in a continual-learning setup with buffered examples? I intuitively understand why is it so, but how does it relate theoretically?\n\n- Not really a question, but an idea. It would be interesting for the future work to see if proposed solution might be used as a regularisation for generative rehearsal approaches in order to reduce overfitting to part of the data distribution that is better covered by a generative model.\n", "limitations": "Yes", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- Interesting initial insights and convincing motivation of the work\n- Well prepared experimentation section wit setups that provide better understanding of what is the main impact of the proposed solution. Good performance evaluation with well selected baseline methods that tackle the problem of buffer based continual learning in different ways\n- Experiments indicate that solution improves recent state-of-the-art approaches, for some setups by a large margin\n\nWeaknesses:\n- Comparison to related works is very limited, a few crucial state-of-the-art methods are missing in both related works and experiments. Most importantly similar work - \"Gradient-based Editing of Memory Examples for Online Task-free Continual Learning\" where buffer examples are adjusted in order to similarly to this method improve the performance of several baseline buffer based continual learning methods.\n- In Eq. 5  an observation is introduced that ReLU can be understood as matrix multiplication by a diagonal matrix with either zeros or ones. On top of this an upper bound of Lipschitz constant for ReLU activation is proposed that can be interpreted as switching ReLU to identity function. This is a very significant simplification that basically reduces calculation of proposed Lipschitz based regularisation for all complex architectures as if they were linear models.\n- For many experiments (iCARL - splitCUB, GDumb (smaller buffer), ER-ACE(pre-trained CIFAR100), Hessian Eigenvalues),  gain observed with proposed solution is marginal. Without confidence interval it is hard to evaluate if LIDER improves results over the random factor. \n\n\nSmall details:\n- [116] also known\n- [247] Why Unexpectedly? I guess it is rather expected that performance decreases when probability of switching labels in buffer poisoning increases.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept