Retrospective Adversarial Replay for Continual Learning
Abstract
Continual learning is an emerging research challenge in machine learning that addresses the problem where models quickly ﬁt the most recently trained-on data but suffer from catastrophic forgetting of previous data due to distribution shifts — it does this by maintaining a small historical replay buffer in replay-based methods.
To avoid these problems, this paper proposes a method, “Retrospective Adversarial
Replay (RAR)”, that synthesizes adversarial samples near the forgetting boundary.
RAR perturbs a buffered sample towards its nearest neighbor drawn from the current task in a latent representation space.
By replaying such samples, we are able to reﬁne the boundary between previous and current tasks, hence combating forgetting and reducing bias towards the current task.
To mitigate the severity of a small replay buffer, we develop a novel MixUp-based strategy to increase replay variation by replaying mixed augmentations.
Combined with RAR, this achieves a holistic framework that helps to alleviate catastrophic forgetting.
We show that this excels on broadly-used benchmarks and outperforms other continual learning baselines especially when only a small buffer is available.
We conduct a thorough ablation study over each key component as well as a hyperparameter sensitivity analysis to demonstrate the effectiveness and robustness of RAR.
Introduction
Traditional supervised machine learning methods often rely on the assumption that the data is drawn i i d. from a stationary probability distribution.
This assumption does not hold in many practical scenarios where the learner must continuously learn online and adapt to new tasks without revisiting previous tasks (and those tasks’ data).
This has motivated research in Continual Learning (also referred to as Lifelong Learning and Incremental Learning), where a machine learning model learns from a stream of data coming from a succession of different tasks [29, 45, 66].
The primary challenge in continual learning (CL) is to alleviate the “catastrophic forgetting” of the previously learnt tasks after learning new tasks [18, 39, 49].
This is mainly caused by a shift in the distribution of inputs and labels over time.
For example, as a model is updated using new-task gradients, the hidden-layer representations, encoding information about previous tasks, become biased towards these new tasks.
This leads to a model confusing the former with current tasks, thus producing incorrect predictions on older tasks.
A widely studied strategy to alleviate forgetting is experience replay (ER) [10, 49, 50], which repeatedly trains the model on buffered replay data from previous tasks while learning the current task.
However, the buffer in practice is often quite small [63], e.g., in autonomous driving, where the incremental data is large-scale and essentially never-ending.
A model can thus over-ﬁt to such small buffered data.
Although forgetting is mitigated, the goal of retaining broad and accurate knowledge about previous tasks is not achieved, and the problem remains severe.
An ideal strategy to address the above challenge is to focus on the replay of “marginal samples” easier to forget and confuse with the current task’s data, e.g., those near the boundary between the previous tasks’ data and the current task’s data.
However, selecting such samples [2, 55] from a small buffer only brings limited improvement because the buffer (often formed using reservoir sampling [62] and thus is uniform) does not well cover many such marginal samples.
Hence, we surmise that the core challenge is how to generate a small and sufﬁcient set of samples, ones that are easier to forget and hence be confused with current task’s data — given the limited buffer size, it is also important for these samples to be diverse and thus efﬁcient.
In this paper, we propose a novel targeted adversarial synthesis based replay method, Retrospective
Adversarial Replay (RAR), that can generate more informative replay instances with richer variations optimized to capture the forgetting frontier of continual learning, from a limited buffer.
Speciﬁcally, given an incoming batch of the current task’s data, RAR ﬁrst identiﬁes the most-likely-to-be-forgotten buffer samples (Stage
I in Fig. 1) and pairs them with their nearest neighbors within the current task’s data (Stage II in Fig. 1).
For each pair, we then apply a bounded and targeted adversarial perturbation to the buffer sample, which moves the buffer sample’s hidden-layer representation towards that of the paired current task’s sample
(Stage III in Fig. 1).
This three-step procedure, as illustrated in Fig. 1, generates ﬁne-grained adversarial augmentations of the buffered data that capture “forgetting” at the local boundary to the current task’s data.
We then effectively reduce the risk of forgetting by replaying these perturbed samples.
To improve the diversity and variation of the perturbed samples (thus increasing the learnable information from a limited-size buffer), we investigate the role of
MixUp [58, 67, 68], a data augmentation technique applied together with RAR — we ﬁnd that it brings substantial improvements when there are strict buffer size constraints.
For Stage I replay data selections, we can utilize any existing methods [2, 3, 10, 55, 65] resulting in an effective and data-efﬁcient replay solution in CL, hence making RAR generic and complementary to existing memory-retrieval & update methods.
Extensive experiments on RAR and comparisons to SoTA baselines from different CL categories demonstrate the advantages of RAR in mitigating catastrophic forgetting and improving the overall accuracy.
Moreover, we conduct an ablation study showing that the key components such as replay samples selection strategy, sample pairing & adversarial perturbation, MixUp, etc, each bring appreciable improvements.
Additional sensitivity analysis of the key hyperparameters implies that RAR is stable and consistently achieves these improvements.
Connection to human false memories & their correction :
In human psychology, "false memories" [36, 42] refer to a wide variety of human memory errors ranging from the minor misremembering of small details (e.g., distorted recollection) all the way to unmitigated fabrication.
The passage of time after experiencing an event increases the likelihood of these phenomena.
They are referred to as [42] a “natural by-product of a distributed memory system (DMS)”.
Rather than a simple search & recollect, DMS [12] helps us in recombining and reconstructing memories via the integration of episodes from the past and present, but it is not always error-free.
It is also true that when high-conﬁdence false memories are corrected, people attend more to the provided feedback [17].
Despite a tendency for belief perseverance [4], the information that breaks through this perseverance is greatly attended to.
The motivation behind RAR is in line with this theory.
In RAR, we ﬁrst combine/pair the replay buffer samples from previously seen tasks with the current data.
Then using the pairing, we ﬁnd existing high-conﬁdence false memories in the form of targeted adversarial perturbations that move the previous data closer to the current data in the model’s latent space.
Finally, we train the model to produce the correct target for the perturbed input during the replay stage.
On the contrary, existing ER-based CL methods lack this adversarial interpolation between past and present data but mainly rely on simple retrieve and replay of past data.
Related Work
Existing approaches in CL can be broadly characterized into two groups based on the model’s architecture.
Methods dealing with dynamic architecture maintain separate parameters for each task [34, 53, 64].
These methods require the knowledge of task descriptors, are expensive to train and often do not scale well when dealing with a long sequence of online tasks.
In this work, we focus on the task-free setting of CL [61].
Fixed architecture approaches use regularization techniques to selectively regularize parameters (either weights or outputs) important for old tasks [1, 8, 27, 32, 35, 52, 66], and require task information in general.
This category also includes memory-based approaches for constrained parameter updates on new samples to minimize interference with previous tasks [9, 37] and for replaying them along with the new task’s data [2, 3, 5, 6, 10, 13, 38, 41, 43, 44, 47, 48, 59, 55, 65].
Other ER methods [2, 51, 56, 60] train generative models on previous tasks which can be challenging owing to the online (single-pass) learning setting.
CL using Adversarial Information : ACL [16], belonging to the dynamic architecture category, uses adversarial learning to train a shared feature encoder and a discriminator using a GAN’s minimax objective [19].
ASER [55] proposes using the kNN-speciﬁc Shapley value [24, 54] to select replay samples that are representative of the buffer and adversarially close to the decision boundaries of incoming classes.
GMED [25] uses gradient updates to edit buffered samples individually based on their loss increase after a look-ahead update performed only on the incoming batch.
Compared to these methods, RAR is a ﬁxed architecture task-free CL framework that leverages the local pairwise relationships between the replay and current task’s samples to locally perturb each replay sample in a targeted adversarial manner.
Our targeted perturbation generates samples close to the forgetting frontier that are hence likely confusing for the model but that are also representative of previous data — this results in a good set to further train on.
To elucidate the differences in detail, we also provide a more extensive survey of the relevant prior literature in Appendix B.
Retrospective Adversarial Replay
Formally, the continual learning problem consists of a stream of data distributions ( D 1 , D 2 , . . .
, D T ) that corresponds to T tasks (in this work, the boundaries between tasks are not known).
We get data points uniformly sampled from the data distribution of the current task.
Our target is to train a machine learning model f ( · ; ✓ ) parameterized by ✓ that minimizes the losses as we learn each task in the given sequential order without increasing the loss on previously learned samples.
In other words, the objective is
[Equation]
Here, ` denotes the loss function and i is an index over the set of tasks.
To retain the knowledge of previous tasks and mitigate catastrophic forgetting, similar to [2, 55, 65], we utilize a limited memory buffer to store samples representing previously seen data.
Let M denote the memory/buffer, which is essentially a set of data points and |M|  m .
Similar to ER (Alg. 2), as we train on task i , we get an incoming batch ( X i , Y i )
⇠ D i , and we combine it with a subset of memory samples ( X 0 M , Y 0 M ) ⇢ M to compute the overall loss.
We can then update the memory M by replacing some samples with the ones encountered in the current training step while keeping the size of the buffer unchanged.
In general, ER-based methods typically deal with two problems: (1) Memory Update for populating the buffer and (2) Memory Retrieval for selecting samples from the buffer for replay.
In this work, we focus on synthesizing adversarial memory samples which provide the most conﬂicting information on the look-ahead probing step.
In other words, the model trained only on the incoming task (one-step look-ahead) misclassiﬁes the synthesized samples (anchored around previously seen classes stored in the memory) as the incoming classes.
Thus, by learning on such adversarially synthesized memory samples in retrospect, we prevent any conﬂicts in the probing step.
For problem (1), we use reservoir sampling to update the memory (Appendix C), which can give us uniform samples from the streaming data points and is widely adopted in many continual learning methods [2, 10, 55].
For problem (2), one can utilize any existing memory retrieval methods [2, 3, 10, 55, 65], thus making
RAR complimentary and easily integratable with many existing CL methods.
3.1 Retrospective Adversarial Perturbation
Existing methods addressing the memory-retrieval problem optimize the model parameters using ` ER in Eq.
2 where ( X 0 M , Y 0 M ) are selected from the buffer M based on their proposed selection strategies.
For the ease of notation, we use y ( x ) to denote the ground-truth label of sample x .
[Equation]
ER [10] uses naive random sampling to select X 0 M , GSS [3] encourages gradient diversity while sampling X 0 M , ASER [55] uses kNN Shapley value to select samples which are representative of
M and close to the samples in X i in the latent space, and MIR [2] selects replay samples which are maximally interfered/forgotten after the look-ahead probing step.
In the look-ahead probing step, the model is updated only on the incoming task samples such that ✓ 0 = ✓  ⌘ r ✓ ` ( X i , y ( X i ); ✓ ) .
Although the methods mentioned above cover crucial aspects (diversity, representativeness, similarity to incoming batch) for selecting X 0 M , they do not capture the local pairwise interactions between the memory and current-task samples.
Motivated by this, Retrospective Adversarial Replay Loss ( ` RAR ) ﬁnds a cardinality-constrained pairing ( E ) between memory and incoming task samples such that the loss on memory samples that are perturbed towards the paired current-task samples is maximized:
[Equation]
Here, X M denotes the entire input samples present in M , and g ✓ 0 l ( x M , x i ) described mathematically in Eq.
4 can be seen as a targeted perturbation of x M that is visually similar to x M but close to the latent embedding of target x
i 2 X i in the l -th layer latent space modeled by ✓ 0 .
Since the min-objective in Eq.
4 aims to move the perturbed replay sample close to the current-task sample, y ( x M ) 6 =
y ( x i ) for a particular pairing to be considered and to ﬁlter out any trivial pairings while optimizing Eq.
[Equation]
where
[Equation]
Since x M and x i have different labels, such perturbed samples that we get from Eq.
4 are confusing to the continual learner in terms of distinguishability from the current-task sample x
i , while still representing previous tasks based on their visual similarity to x M 2 M .
Eq.
3 essentially helps in the synthesis of targeted adversarial samples that are close to incoming samples
X i in the latent space and hence the learner will miss making the correct prediction y
( X M ) despite the fact that the adversarial samples are anchored around the original memory samples X M .
Training on such samples can help the model to learn about the boundaries between previous and current tasks and also retain the knowledge of previous tasks [7, 11, 23].
Note that the perturbations are performed based on the look-ahead parameters ✓ 0 .
In other words, assuming we only learn from the current-task samples
X i and update the parameters to ✓ 0 , the generated adversarial samples give us many training samples that show conﬂicting behaviors on ✓ 0 and get easily forgotten by the model.
In retrospect, we can potentially prevent such forgetting when we are still at ✓ by using those targeted adversarial samples as training signals, i.e., we minimize the RAR loss (Eq. 3) to update the model ✓ .
Optimizing Eq.
3 over all feasible pairs of X M and X
i is computationally challenging as we effectively need to compute the adversarial perturbations on all such pairs to select the most confusing subset incurring the maximum RAR loss.
Moreover, each adversarial sample synthesis (Eq. 4) requires multiple gradient steps as we need to run adversarial attack algorithms such as the iterative-FGSM [15, 20] covered in detail in Appendix H.
Thus, to eliminate this bottleneck, we propose two pruning strategies covered in sections 3.2 and 3.3 to reduce the candidate anchor-target pairs in X M ⇥
X i for adversarial perturbations and approximately optimize Eq.
3.2 Selecting Replay Data from Buffer
To reduce the computational overhead, we only consider a subset of the memory buffer X 0 M ⇢ M instead of considering all memory input samples X M .
Memory-retrieval methods such as ER [10],
GSS [3], ASER [55], and MIR [2] can be used to select a smaller replay set X 0 M .
This again reiterates that RAR is generic and complementary to existing memory-retrieval methods.
3.3 Pairing between Buffer and Current-task Data
To further bring down the computational costs of adversarial perturbations, we perform another pruning over the possible pairs between X 0 M ⇥ X i based on samples’ hidden representations.
I.e., we ﬁrst compute the distance d ✓ 0 l ( x 0 M , x i ) , which is essentially the target value for the adversarial perturbation (Eq. 4) for every pair ( x 0 M , x i ) 2 X 0 M ⇥ X i with different labels y
( x 0 M ) 6 = y ( x i ) and then screen out the pairs with larger distance values, maintaining only the smaller-distance ones, in accordance with the min-objective of Eq.
To utilize the information from the selected memory samples, we pair each x 0 M 2 X 0 M with its nearest neighbor with a different class/label in the incoming batch
X i of current task.
Nearest-neighbor-based matched pairs already represent the most confusing samples as the samples in a pair belong to different classes.
Thus, it is easier for the optimization procedure approximating Eq.
4 to further reduce the distances between pairs coming from different classes (shown in Fig. 2b), aiding in the synthesis of even more confusing adversarial samples.
Computational overhead for optimizing Eq.
3 : First, we select memory samples for replay X 0 M (Sec. 3.2) which has the same costs as the backbone memory-retrieval method.
Next, we compute the distances between latent features of X 0 M and
incoming batch X i with computational complexity of
O ( | X 0 M | ⇤ | X i | ) .
This is minimal given the online nature of CL where the training batch size is often set to 10 .
After performing the nearest neighbor matching, we optimize Eq.
4 via iterative-FGSM [15] which introduces some additional computational costs, but we keep it marginal by using only two steps of iterative-FGSM.
We provide further details in Appendix E and H.
Combining our RAR loss with the loss on selected memory samples and incoming batch samples, we have our overall objective (similar to Eq. 2) deﬁned as follows, where  is a hyperparameter to control the trade-offs between the losses on original and adversarially synthesized memory samples.
`
[Equation]
1: Input: Tasks ( D 1 , D 2 , . . .
D T ), Model ✓ , Learning Rate ⌘ , Memory M , Memory Size m , Replay Budget k , Trade-off Parameter  , Perturbation Strength ✏ & number of gradient steps n 2: Output: Model ✓ 3: for task i {+
+}
[ T ] do 4: for ( X i , Y i ) ⇠ D i do 5: ✓ SGD( ` ( X i , Y i ; ✓ ) , ✓ , ⌘ ) { // Look-ahead update of model parameters} 6:
Select ( X 0 M , Y 0 M ) from M s.t.
| X 0 M |
= k { // using existing memory-retrieval methods} 7:
Let S : = ; 8: for x 0 M 2 X {+
+}
M do 9: x 0 i 2 argmin x i 2 X
i ,y ( x 0 M ) 6 = y ( x i ) d ✓ 0 l ( x 0 M , x i ) 10:
S S [ { ( x 0 M , x 0 i ) } 11: end for 12: ` RAR ( S, ✓ , ✓ 0 )
P ( x 0 M ,x i ) 2 S ` ( g ✓ 0 l ( x 0 M , x i ) , y ( x 0 M ); ✓ ) { // perform the targeted adversarial perturbations (Appendix H) and deﬁne the corresponding RAR loss} 13: ` all ` ( X i , Y i ; ✓ ) +  ` RAR ( S, ✓ , ✓ 0 ) {++ (1   ) ` ( X 0 M , Y 
+}
M ; ✓ ) 14: ✓ SGD( ` all , ✓ , ⌘ ) 15:
M ReservoirUpdate( X i , Y i , M , m ) 16: end for 17: end for
Description of RAR framework :
In Alg. 1, we describe our retrospective adversarial replay algorithm for continual learning.
The algorithm essentially consists of three steps: (1) in Lines 5-6, we ﬁrst perform the look-ahead parameter probing using the incoming samples of the current task to get ✓ 0 , and select the subset X 0 M using existing memory retrieval methods, such as ER, MIR,
ASER, etc. (2) in Lines 7-11, we prune down the pairings between X 0 M and X i based on distances between hidden representations computed using Eq. 5, and (3) in Line 12, we deﬁne the RAR loss by performing adversarial perturbations on the selected pairs S .
Note that since we use the selected pairings S of size k as an approximation of the true maximum-valued pairs, we can just sum over entries in S instead of picking the maximum subset as done in Eq.
In Line 15, we update the memory buffer M based on incoming samples ( X i , Y i ) using reservoir sampling [62] and keep the memory limit m .
3.4 MixUp based Retrospective Adversarial Augmentation
We also propose another method mix-RAR utilizing the mixup technique [67] to bring in more diversity to the pool of replay examples selected from the buffer.
After creating virtual examples (˜ x M , ˜ y M ) using x i , x j 2 X 0 M and  2 [0 , 1] such that ˜ x M =  x i + (1   ) x j and ˜ y M =  y i + (1   )
y j , we follow the original RAR algorithm to achieve pairing and subsequent targeted adversarial perturbation as shown in Alg 5 in Appendix A.
The proposed MixUp strategy is different from [21, 40] as we apply mixup among the replay samples and then generate RAR perturbed samples anchored around them.
This is consistent with RAR’s objective.
Given a current-task’s sample from class (c), training on an RAR perturbation applied to a mixup between two buffered samples from different past classes (a,b) can potentially help in capturing the forgetting frontier between three pairs of classes (a,c), (b,c) and (a,b), thus minimizing forgetting on both past classes.
Experiments
Datasets : We evaluate RAR on four supervised image classiﬁcation benchmarks for task-free CL.
Split-MNIST [31] has ﬁve disjoint tasks, each having two classes.
(2) Split-CIFAR10 [28] consists 5 disjoint tasks with each task having two classes and 10k training examples.
(3) Split-CIFAR deals with 20 disjoint tasks with each task having 5 classes.
(4) Split-miniImageNet [14] consistsdisjoint tasks with each task having 5 classes and 2.5k training examples.
We cover further statistics in Appendix D. All these datasets deal with the class-incremental scenario [61].
Our implementation code is available at https://github.com/lillykumari8/RAR-CL.
We average results over 15 runs for all except for Split-MNIST where we average over 20 runs.
[Table]
Evaluation Metrics :
Following [8, 37], we use two standard metrics used in CL to evaluate RAR’s performance.
Final average accuracy ( A T ) assesses the overall performance of the model whereas
Final average forgetting ( F T ) measures how much the model has forgotten each task once the online learning has completed.
Lower forgetting is better.
Below, a i,j denotes the accuracy evaluated on the test set of task j after the model has experienced all tasks up to i .
[Equation]
Settings and Hyperparameters : We follow the same setup as [2] for deciding model architectures for all four datasets.
For Split-MNIST, we use an MLP classiﬁer with two hidden layers, each with
We average results over 15 runs for all except for Split-MNIST where we average over 20 runs
[Table]
[Figure] [Table] 400 units with ReLU activation, followed by a linear classiﬁer layer with 10 units.
For Split CIFAR10, Split CIFAR-100, and Split mini-ImageNet, we use a reduced ResNet-18 classiﬁer [22].
The replay budget k is the same as the mini-batch size (ﬁxed to 10) irrespective of the buffer size m .
For hyperparameters tuning on each dataset, we hold-out 5% of the training samples for each task and use it as a validation set.
We provide additional details about the implementation settings in Appendix F.
4.1 Main Results
[Figure]
buffered replay samples and its nearest neighbor in the current tasks’ batch evolves across different stages of RAR
We report the average accuracy and forgetting in Table 1 and 2 respectively for experiments on all four datasets under various buffer size constraints.
We use three different memory-retrieval methods, namely ER [10], MIR [2], and ASER [55] and compare RAR and mix-RAR with their respective baselines which do not use the RAR loss (Eq. 3).
For fair comparison, we compare mixRAR with baseline retrieval methods using mixup between replay samples.
We also compare against standard baseline methods: (a) Fine-tuning , which trains the model continuously on the stream of tasks without employing any strategy to avoid forgetting, (b)
Averaged Gradient
Episodic Memory (AGEM)
[9] (c) iid-online , which trains a model on the iid-data sampled from entire dataset D =
[ Ti =1 D i for a single epoch, (d) iid-ofﬂine , which trains a model similar to the iid-online setting but for multiple epochs (three epochs considered in the reported results).
We compare against another memory sample perturbation method called GMED [25] in Table 3 to avoid repetition.
We provide the pseudo-code for ER (Alg. 2), ER-RAR (Alg. 3), MIR-RAR (Alg. 4), and mix-RAR (Alg. 5) in Appendix A to describe how our proposed framework can be uniﬁed with existing CL methods.
Also, we leave out the results against GSS [3] as both MIR [2] and ASER [55] outperform it in terms of accuracy and forgetting.
As shown in Table 1, both RAR and mix-RAR using different memory-retrieval methods consistently outperform their respective non-RAR baselines (without and with mixup) across all four datasets.
From Table 2, we observe that the improvements obtained by integrating RAR with respective replay selection methods {ER, MIR & ASER} (both without and with mixup) are signiﬁcant.
In case of Split-CIFAR10, MIR-mix tends to perform slightly better than
MIR-mix-RAR in terms of forgetting, but based on accuracy improvement that MIR-mix-RAR brings, the relative difference is negligible since forgetting is dependent on the maximum accuracy that a continual learner achieves across different tasks.
In the case of Split-miniImageNet, ER-mix-RAR method achieves the best average accuracy of 28 .
8% , outperforming other MIR & ASER based methods, and has similar observation on the
Split-CIFAR100 dataset.
We hypothesize that for large buffer sizes such as 10 k ( 100 examples per class), the ER-based selection aids in sampling a diverse set of classes from the memory buffer compared to MIR & ASER, thus improving the overall accuracy.
However, since MIR & ASER consider samples suffering interference from the new task’s data, MIR-mix-RAR & ASER-mix-RAR help minimize the forgetting on the old tasks.
The improved performance of ER-RAR, MIR-RAR, and ASER-RAR compared to ER, RAR, and
ASER respectively demonstrates that RAR is able to alleviate catastrophic forgetting on old tasks via generating diverse and confusing samples close to the forgetting frontier.
To ensure fair comparison in terms of computational costs, in Table 5 in Appendix
I
[Table]
we also compare RAR to GMED [25],
ER-3-iters
[Table]
and MIR-3-iters.
3-iters means that the model is trained for multiple iterations on the incoming batch and buffer samples as a new batch comes in.
[Table]
[Table]
[Figure]
Case study: Visualizing the Three Stages of RAR For the Split-MNIST dataset, we visually analyze the three stages of the RAR framework for a buffer size of 500 using MIR as the memory-retrieval method.
Fig.
2a shows the samples selected for replay using the MIR score (Eq.
8) and their paired samples from the current task’s batch based on distances computed using Eq.
Fig.
2b shows how the distance between a replaying sample and current task’s data changes over the three stages: (1) for each sample selected from the buffer, the blue bar represents its average distance to all samples in the current-task batch; (2) after pairing each buffered replay sample with its nearest neighbor as described in Section 3.3, its distance to the paired sample (orange bar) becomes smaller than the blue bar; (3) after applying adversarial perturbations to each buffered sample as described in Section 3.1, the distance is further reduced.
The decrease of distance indicates that the pairing and adversarial perturbation in
RAR are effective in generating samples closer to the forgetting frontier than the original buffer data selected for replay, meaning the RAR perturbations are similar to X M but very close to the paired samples in X i (current task’s batch) in the l-th layer latent representation space of model ✓ .
In Fig.
3, we show four examples of RAR perturbed samples generated via Eq.
4 from four buffer samples and their paired samples from the current task.
For most of the buffer samples, a single gradient step of ✓ replaying them and their RAR perturbed counterparts sufﬁces to correct their wrong predictions (e.g., the current task’s classes), hence effectively alleviating the model’s forgetting on previous tasks.
4.2 Advantage of MixUp in Small-Buffer Cases
On each dataset, as reported in Table 2, all mix-RAR methods perform better compared to RAR methods in terms of reducing forgetting on previous tasks with slight improvement in average accuracy.
The proposed use of MixUp among the replay samples before RAR is able to increase the variation of RAR perturbed samples, which is essential to CL with a small buffer.
To verify its effectiveness, we conduct experiments that store only 20 examples per class.
Figure 4 shows that ER-mix-RAR & MIR-mixRAR consistently outperform other ER & MIR methods respectively.
We can achieve even more signiﬁcant improvements in forgetting, particularly for more challenging tasks such as Split-CIFAR100.
4.3 Ablation Study to Verify the Effectiveness of RAR
In Table 3, we compare RAR methods to two baseline methods: (a) Random which add a perturbation of the same tolerance as the RAR method to the replay samples before the experience replay stage, (b) GMED [25] which performs smart gradient updates in the input space by treating each replay sample individually (no local pairing considered between buffer and current task’s data).
As can be seen in Table 3, ER-RAR and MIR-RAR consistently outperform other methods conﬁrming RAR’s superior ability to tackle catastrophic forgetting via generating targeted perturbations inﬂuenced by the local interference between different task’s data.
4.4 Sensitivity Analysis of Hyperparameters
In Fig.
5, we plot the sensitivity of RAR’s performance with respect to the number of gradient steps n used to generate the retrospective adversarial samples via optimizing Eq.
4 in the input space.
For Split-CIFAR10 and Split-miniImageNet (where we report results using two gradient steps), we observe that for a larger number of steps {5, 10}, the performance stays comparable.
This suggests that nearest-neighbor-based pairing using distance computations (covered in Sec 3.3) helps achieve anchor-target pairs which are already optimized and hence do not require numerous gradient updates to get classiﬁed as the paired target class from the current task.
[Figure] [Figure]
Next, we study the sensitivity of RAR corresponding to  which denotes the trade-off coefﬁcient associated with ` RAR in the ﬁnal loss objective Eq. 6 — we use only one coefﬁcient to reduce hyperparameter search costs.
Fig.
6 shows that RAR (using MIR) outperforms vanilla MIR in terms of both accuracy (solid blue line is always above the dashed blue line) and forgetting (solid orange line always stays below the dashed orange line) for various non-zero values of  across the studied datasets.
Conclusion
In this paper, we study how to perturb limited buffer replay data to mitigate catastrophic forgetting in continual learning.
We propose “Retrospective Adversarial Replay (RAR)”, which focuses on previous tasks’ data near the forgetting boundary to the current task in the model’s representation space.
RAR ﬁrst allocates the most confusing pairs of a buffered sample and a current task’s sample and then applies a bounded targeted adversarial perturbation to the former, which further moves it towards the latter in the model’s representation space and creates a perturbed sample close to the boundary.
Moreover, we study the role of MixUp in increasing the variation of replay augmentations, which significantly improves CL in the small buffer regime.
When integrated with existing replay methods, RAR consistently outperforms previous CL methods across benchmark datasets under different settings
Acknowledgements
This work was supported in part by the CONIX Research Center, one of six centers in JUMP, a
Semiconductor Research Corporation (SRC) program sponsored by DARPA.
We would also like to thank the Melodi lab students: Arnav Das and Gantavya Bhatt for their useful discussions.
[-In all experiments, we fix this subset size (also referred to as the replay budget) as 
-]