Advancing Prompt-Based Methods for Replay-Independent General Continual Learning
Authors: Zhiqi KANG
Liyuan Wang
Xingxing Zhang
Karteek Alahari
Venue: ICLR 2025 Poster (2025)
Keywords: Continual Learning
General Continual Learning
Catastrophic Forgetting
Prompt Tuning
Sharpness-Aware Minimization
Primary area: transfer learning, meta learning, and lifelong learning
TL;DR: We propose an innovative prompt-based approach with forgetting-aware initial session adaptation and non-parametric logit mask to facilitate general continual learning.

Abstract: General continual learning (GCL) is a broad concept to describe real-world continual learning (CL) problems, which are often characterized by online data streams without distinct transitions between tasks, i.e., blurry task boundaries. Such requirements result in poor initial performance, limited generalizability, and severe catastrophic forgetting, heavily impacting the effectiveness of mainstream GCL models trained from scratch. While the use of a frozen pretrained backbone with appropriate prompt tuning can partially address these challenges, such prompt-based methods remain suboptimal for CL of remaining tunable parameters on the fly. In this regard, we propose an innovative approach named MISA (Mask and Initial Session Adaption) to advance prompt-based methods in GCL. It includes a forgetting-aware initial session adaption that employs pretraining data to initialize prompt parameters and improve generalizability, as well as a non-parametric logit mask of the output layers to mitigate catastrophic forgetting. Empirical results demonstrate substantial performance gains of our approach compared to recent competitors, especially without a replay buffer (e.g., up to 18.39, 22.06, and 11.96% points performance lead on CIFAR-100, Tiny-ImageNet, and ImageNet-R, respectively). Moreover, our approach features the plug-in nature for prompt-based methods, independence of replay, ease of implementation, and avoidance of CL-relevant hyperparameters, serving as a strong baseline for GCL research. Our source code is publicly available at https://github.com/kangzhiq/MISA

Published as a conference paper at ICLR 2025
ADVANCING PROMPT-BASEL METHODS FOR REPLAY- INDEPENDENT GENERAL CONTINUAL LEARNING
Zhiqi Kang Inria*
Liyuan Wangi Tsinghua University
Xingxing Zhang Tsinghua University
Karteek Alahari! Inria*
ABSTRACT
General continual learning (GCL) is broad concept describe real-world contin- ual learning (CL) problems, which are often characterized by online data streams without distinct transitions between tasks, i.e., blurry task boundaries. Such re- quirements result in poor initial performance, limited generalizability, and severe catastrophic forgetting, heavily impacting the effectiveness of mainstream GCL models trained from scratch. While the use of frozen pretrained backbone with appropriate prompt tuning can partially address these challenges, such prompt- based methods remain suboptimal for CL of remaining tunable parameters the fly. In this regard, we propose an innovative approach named MISA (Mask and Initial Session Adaption) to advance prompt-based methods in GCL. It includes a forgetting- aware initial session adaption that employs pretraining data to initial- ize prompt parameters and improve generalizability, logit mask of the output layers to mitigate catastrophic forgetting, Empirical re- sults demonstrate substantial performance gains of our approach compared to re- cent competitors, especially without replay buffer (e.g., up to 18.39, 22.06, and 11.96 points performance lead on -IFAR-100, R, respectively). Moreover, our approach features the plug-in nature for prompt- based methods, independence of replay, ease of implementation, and avoidance of CL-relevant hyperparameters, serving as strong bascline for GCL research. Our source code is publicly available at
well non- con-parametric
Ima-eee,,Net, and ImageNet-
com/Kangzhiig/MIAA
INTRODUCTION
Continual learning (CL) (Wang et al., 2024b De Lange et al. 2021 focuses on the lifelong acqui- sition of knowledge in response real-world changes. Although conventional CL research often relies on offline training of each task with distinct transitions between tasks (i.e., clear task bound- aries), online scenarios with blurry task boundaries tend to more practical yet challenging. Such "online" scenarios demand the model to swiftly adapt to new information, which is essential for real-time applications. Blurry task boundaries further reflect realistic data distributions without dis- tinct transitions between tasks, i.e., old classes disappear gradually
time and might re-appear when new classes emerge (see Fig. [ Left). The above considerations, collectively referred to as general continual learning (GCL) (De Lange al.] (202T), have received increasing attention in cent years, However, most of the earlier works (Aljundi et al.) 2019 Buzzega et al.) [020 Koh al. 2021 Bang al., (02T)) address GCL by learning a model from scratch. This strategy often leads to challenges such as poor initial performance, limited generalizability. and severe catastrophic forgetting (McCloskey & Cohen 1989) such a complex setting.
One promising direction that emerged in GCL literature (Moon et al. 2023) is to employ prompt- based methods (Wang et al., 2022d c 2024a) on the basis of pretrained models (PTMs), which involves keeping the PTMMs frozen and introducing a few prompt parameters for representation learn- ing. Such methods outperform previous train-from from-scratch ones by a significant margin. The frozen backbone pretrained on a scrge-scale dataset provides a solid initialization with strong generalizabil- ity and is resistant to forgetting, However, the particular challenges of GCL persist for the remaining
1 CCoresponding authors: L. Wang and K. Alahari. "Uniy. Grenoble Alpes, CNRS, Grenoble INP, IJK, 38000 Grenoble, France.

---

Published as a conference paper at ICLR 2025
Anytime Interence Accuvery
Bluny Dispoind
80
Sension accuricy
-50
Ovg
30
Ast
20
10
Ours
DuelPrompt
LnF
Ours
DuelPrompt
LeF
Sessin
Time
L2P
Figure 1: Problem setup and motivation. Left: illustration of the GCL data stream. Mid: average prediction accuracy at different timesteps in GCL. Right: session accuracy, where evaluate the retention of knowledge acquired at session after each session. All methods are tested without a replay buffer.
tunable components, i.e., the prompt parameters and the output layers, making the direct application of prompt-based methods to GCL suboptimal.
Studies have demonstrated that prompt parameters and output layers are particularly vulnerable in the challenging setting of GCL. First, prompt-tuning is known to suffer from limited capacity in vision tasks (Lester et al. 2021 Vu et al., 202T Chen al., 202T). Training these parameters on the fly in online scenarios of GCL can result in poor iniiial performance and limited generalizability. As shown in Fig. [ (Mid), state-of-the-art methods have moderate initial performance and drop dra- matically after session transitions (timestep 4). Moreover, it is well studied in the CL literature (Wu et al., 2019 Ahn al., 202T) that the output layers suffer from forgetting class-incremental learn- ing, challenge that exacerbated in GCL due the naturally imbalanced data stream and the interference between new and old knowledge caused by blurry task boundaries. While recent GCL works (Moon et al. 2023) have introduced specialized learning strategies for the output layers, they still fail to outperform conventional prompt-based methods. Specifically, the session 1 accuracy in Fig. (Right) shows the capacity retain acquired knowledge from the first session in subsequent sessions, where all existing methods suffer to retain this knowledge. Thus, there is a clear need for learning-efficient and forgetting-less strategies address the particular challenges of GCL.
Our method, MISA (Mask and Initial Session Adaption), is designed with two objectives: a forgetting-aware initial session adaption (ISA) for better learning efficiency and generalizability of the prompt parameters, and non-parametric logit masking for less catastrophic forgetting of the output layers. The initial session refers to a warm-up of the model parameters prior to any GCL sessions. Specifically, we reuse the pretraining data of the PTMs to warm up the prompt parameters. As naïve pretraining tends to enhance mainly the initialization rather than the generalizability. we devise novel forgetting-awan minimization technique in ISA to improve the generalizability of prompts to distribution shift. Unlike conventional CL methods that address forgetting after it oc- curs, our approach proactively prevents future forgetting and maintains the pretrained knowledge for downstream GCL. As for reducing forgetting the output layers, we propose a non-param parammeettiic strategy that is based ple strategy, which is not effective for the wain-from-scratee paradigm, can benefit from the stable representation space retained by the frozen backbone and effectively rectify the output layers.
class appearance mask the output logits. We show that such sim-
In summary, our MISA is specifically designed for tunable parameters of prompt-based methods in GCL. We emphasize the plug-in nature of MISA for different methods and scenarios, it is replay-independent, hyperparameter-fiee and effective in GCL. Accordingly, our approach brings substantial performance gains compared to recent strong basclines. For instance, there is up to 18.39, 22.06, and 11.96 % points performance lead on CIFAR-100 100, Imay-lmageNet, and ImageNet- datasets, respectively. Our contributions can summarized as follows:
We propose innovative approach that addresses the poor initialization, limited generaliz- ability, and severe forgetting of GCL, which features the ease of implementation. avoidance of CL-relevant hyperparameters, and plug-in nature for prompt-bascd methods.
Our proposed forgetting aware initial session adaptation effectively improves the initial- ization of prompt parameters and their generalizability to distribution shift, and our non- parametric logit masking rectifies the output layers alleviate catastrophic forgetting.
2

---

Published as a conference paper at ICLR 2025
Across multiple challenging benchmarks, approach outperforms variety of existing methods by a wide margin, setting a new state the art for GCL research.
2 RELATED WORK
Continual Learning (CL). The default setting of CL involves offline training of each task with explicit task boundaries. Representative methods for this setting can be divided into three groups (Wang et al., 2024b). The first one is replay replay-based methods, which store generate few old
training samples for subsequent reuse (Buzzega et al. 2020; Wang et al., 2022a Kang et al. 2023 Ostapenko al. 2019). The second is parameter isolation methods (Serra
2018 Jung al. 2020), which allocates task-specific parameters to prevent overwriting. The third is regularization- based methods, which introduce regularization in loss functions to mitigate forgetting (Wu et al. 2019 Cha et al. 202T, Douillard et al. (020)). There have been attempts to regularize the sharpness
of the loss surface to alleviate forgetting (Mirzadeh et af., 2020; Yang et al. [023] Mehta et al.)
2023 but these are limited to wraining-frome from-scratch paradigm or shallow networks
Prompt-based methods (Smith al.) 2023) Wang et al.) 2022d 2024a) provide another avenue for CL. In particular, they are state of the art for conventional CL settings, wherein the PTMs are frozen and a few prompt parameters are introduced for representation learning. They can be cat- egorized into task-specifie prompts (Razdaibiedina et al.) 2023a, Wang al. 2022b). shared-task prompts (Wang et al. [022d]; Smith et al.) (202)) and their variants (Wang et al. [002cc] Hong et al.) 2024). Although these methods perform well in conventional CL scenarios with adequate pretrain- ing, they exhibit many problems in GCL, such as poor initial performance, limited generalizability. and severe catastrophic forgetting. Moreover, many advanced prompt-based methods are not adapted to GCL by design. For instance, CODA-Prompt (Smith et al. 2023) and NSP2 (Lu et al. 2024) as- sume orthogonality between tasks, which cannot be satisfied the case of blurry task boundaries. RanPAC (McDel et al. (224) and HiDe-Prompt (Wang et al. 2024a) lack the design for online updates class statistics in GCL.
General Continual Learning (GCL) is broad concept to describe a variety of practical challenges in CL (De Lange et al., 2021). ). In addition to the primary focus on "online learning" and "blurry task boundaries" considered in our approach, other requirements of GCL like "constant memory" and "no test-time oracle" also play a crucial role in enabling the model to better adapt to real-world applications. These additional aspects will be briefly discussed in this paper well to demon- strate that our approach is suitable for GCL. Various settings have been proposed as realizations of GCL (Aljundi et al.) 2019 Koh et al. 2021 Moon et al. 2023). Si-Blurry (Moon al. 2023 is one of the recent approaches for GCL to evaluate the two primary facets, which assumes that the distributions of training samples belonging to each class
randomly sampled in each task. Al- though there have been numerous attempts (Aljundi et al. 2019 [Buzzega et al.] 2020: |Bag et al.) 202I to address GCL, they often focus on training from scratch and thus require replaying a few old training samples, achieving inferior performance with potential privacy issues. Gummadi et al. (022) dealt with blurry task boundaries with novelty detection yet is designed for offline scenarios. The most recent and only prompt-based GCL method (Moon et al.) 2023) demonstrated improved performance when using a pretrained backbone compared to training from scratch. However, remains suboptimal in addressing the GCL. challenges, yielding only marginal improvements over conventional prompt-based CL. methods.
3 PRELIMINARIES
In this section, we review the problem setup of conventional CL and GCL, as well as the advanced prompt-based methods that address the former.
Problem Setup. Let be the set of available classes whose cardinality denoted as N = |C]. Suppose that there are T learning sessions or tasks. In conventional CL, we create a uniform partition {C.)=.... of C, where C₁ represents the classes assigned to task t. The number of classes in each task is identical, i... Vi,j. # j, |C,l |Cyl. In addition, at each task, the model can extensively iterate on the data with epoch E N. contrast, GCL relaxes the assumptions of clear boundaries and offline learning. Due to such blurry task boundaries, we opt for the notion of learning sessions,
3

---

Published as a conference paper at ICLR 2025
Embrsiing
Transfrme Becibons
Outout Layer
Enieed
Frease
Cancotonate
PAM
Appeared
samples
Disjent
Bluryy
Traradome Bacidorne
Loc
Data
Lar go scale dastanet
(4) General continual leaming
(b) Forgeding ovare Initial session adaptation
(c] Non-garamctrie mask
An overview of our MISA with a frozen pretrained backbone GCL. (a) Data GCL consists of disjoint and blurry classes. (b) Initial session adaption is conducted prior to any CL ses- sions. Once finished, only the warmed-up prompt parameters are reused for CL., (c) Non-parametric logit mask which retains logits of available classes in batch or session.
Figure 2:
which represent different time steps in one-pass data stream of GCL., Specifically, classes overlap between learning sessions, such that:
(1) where denotes probability. Moreover, due to the one-pass constraint on the data, the model can only perform one epoch over the data, such that K 1. Consequently, GCL. emphasizes realistic data distributions and the swift adaptation the model.
Vi j. P(C, * 0) > 0,
The latest realization of GCL is Si-Blurry (Moon et al. 2023). Here, the available classes C are ran- domly divided into non-overlapping CD and T for disjoint classes and blurry classes, respectively. The disjoint class ratio is defined titions }t=1... and {C 1=1... T. unlike the uniform distribution in conventional CL. Although all data corresponding to C is divided into disjoint sessions, there is ratio of n training samples from C being randomly shuffled across sessions, with n as the blurry sample ratio. The stochastic nature of Si-Blurry controlled by and As realization of GCL, the shuffling of blurry classes satisfies Eq. [ Moreover, Si-Blurry assumes an online scenario that naturally fits in the one-pass requirement of GCL. We also show in Appendix A.3 that Si-Blurry is realization of generalized class incremental learning (Mi et al. 2020).
|CPI//C]. C and C are divided into non-uniform par-
Prompt-Based Methods. We denote pretrained vision transformer (ViT) (Dosovitskiy et al., 2020) as f = fe fr fe, where fo is the input embedding network, f, is stack of self-attention layers, and fe the output layer(s). For an input image x and one-hot label vector y. let fa(x) = x. E RL be the embedding features with L the token length and D the embedding dimension. Prompts are learnable parameters Pe E R4,xD prepended to the embedding features as σ, = [pei v. with Lp the prompt length. The extended features are forwarded to the network such that Ja Sr (x. with ý E NR the prediction vector. Existing prompt-based methods (Wang) cct al., 2022dd c Smith et al., 2023) for conventional CL usually train pool of prompts and cach time select the most relevant ones through query-key matching mechanism. Consequently, the poorly trained prompts can not only impair representation learning but also disrupt the matching mechanism, leading to degraded performance. Therefore, enhancing the knowledge encoded in prompts is crucial for improving these methods.
4
METHOD
We now present the two main components of our MISA, as shown in Fig. 2 (1) forgetting-aware ini- tial session adaption (ISA), which warms up prompt parameters with forgetting-awann minimization (FAM) and prompt augmentation; and (2) non- non-parametric logit masking that rectifies the output of the model to avoid the interference of old and new knowledge. These two components provide ben- efits such as ease of implementation, avoidance of CL-relevant hyperparameters, and their plug-in nature for other prompt-based methods.
4

---

Published as a conference paper at ICLR 2025
4.1 INITIAL SESSION ADAPTION
We start by defining the initial session. In conventional (offline) CL. it has been overlooked that using the pretrained models for prompt-based methods implies an initial learning session of the backbone prior to any downstream CL sessions. To fully exploit the benefits of pretraining data, we propose to incorporate prompt parameters in the initial session of GCL. In particular, we leverage the pretraining dataset Dinit to prepare the prompt parameters for swift adaption. We warm up the prompt parameters in a supervised manner for ISA, following the pretraining paradigm of the backbone. Accordingly, the supervised learning objective is defined as:
min Pe-fe (z. (en
Σ
(ee (offffoo((eee
fr((pei (e)e:i)))..)))
where Lee is the cross-entropy loss. We omit the regularization term in Eq. 2 for simplicity. Both Pe and Sc are randomly initialized. The learned Se will be discarded after this step, and only the learned Pe will be reused afterwards. The class prototypes learned from sisee in fe are irrelevant to downstream tasks, as class identities are expected to change, making them ineffective for down- stream use. Note that using the same pretraining dataset at the backbone a design choice to ensure a fair comparison with existing methods, as the adaptation of Pr. does not require any additional data, rather than a strict requirement of the ISA dataset. Additional experiments when pretraining data is partially completely unavailable can be found in Appendix A.4.9
There are several reasons for which ISA should naturally be effective. parameters is equivalent to the pretraining operation in an offline setting, which has already been shown effective in various settings. Second, ISA can also be seen as a particular initialization of prompts, which proved to be essential for the effectiveness of visual prompt tuning (Tsai et al, 2024 Shen et al., 2024). Lastly, from a domain adaptation perspective, the warmed-up prompts effectively reduce the domain gap to downstream GCL compared to randomly initialized prompts. It should be noted that ISA is designed to improve the poor learning capacity of visual prompts in GCL. Such limitation is less significant in conventional (offline) CL, sufficient training with adequate differentiation of task-specific knowledge allows the prompts to eventually converge to near-optimal stage. This highlights that our design is tailored to the particular challenges of GCL.
First, ISA for the prompt
4.2 FORGETTING- MINIMIZATION
While naively initializing the prompt parameters using Eq. 2 on pretraining data can address is- sues due to poor initialization, this straightforward strategy lacks considerations for generalizability. Inspired by sharpness- aware minimization (SAM) that encourages the model to converge flat minima, we propose forgetting aware minimization (FAM) in ISA, which seeks forgetting-awe aware flat minima and improves generalizability to distribution shift of prompt parameters.
Vanilla SAM. SAM (Foret et al., 2020) aims to reduce the sharpness of the loss surface by a minimax game: minimizing the maximal loss changes from small perturbation on the model's learnable parameters 0. The optimization objective is defined as:
min max Lusin ( + e), |el|ممم
where p the size of the neighborhood and Luain represents a general learning objective. The correlation between the flatness of the loss surface and less forgetting in CL has been revealed by MMrzadehh et al. (2020); Shi et al. (2021): Mehta et al. (2023), which motivates the use of SAM in CL (Yang et al. (023). However, previous efforts mainly focus training- maaning-from-sseee wwwm-frrracchh paradigm of shallow ResNets, rather than a pretraining paradigm of large transformer backbones.
be calculated by using first- order Taylor expansion of Luain (0 + c) around B. This simplifies the original optimization problem into a linear-constrained one, which can be approximated as (Foret al. 2020
Perturbation in Vanilla SAM.
The optimal perturbation
V (8)
arg max Euain (0 + c) R | Lurinn (0)
(4)
Wee leave self-supervised warm-up a potential future direction.
5

---

Published as a conference paper at ICLR 2025
Although this strategy has shown effectiveness in various situations, applying SAM directly to CL neglects the inherent conflict caused by distribution shift - a common challenge in CL. In particular, when the distribution of new task diverges significantly from that of the previous task(s), the model is likely to be pulled from the initial flat loss surface to a sharp and unconstrained one. This phenomenon is widely recognized as a cause of forgetting, but here, it also contributes a loss of generalizability of the initialized prompt parameters. Therefore, the perturbation in vanilla SAM is not a reliable estimation of future forgetting in CL, significantly limiting its effectiveness. To address this issue, our approach is to simulate such distribution shifts during ISA and estimate perturbations from them, thereby enhancing the robustness of the initialized parameters against forgetting and the loss of generalizability.
Forgetting-Aware Minimization. Our design improves generalizability by using gradients com- puted for external tasks perturbations. We define the data from the target task in-distribution data Dia and the data from the external task as out-of-distributin the training loss under parameter perturbations, we target the perturbations that are consistent with the external task gradient, rather than those that lead to the maximum changes of the training loss. Consequently, this gradient represents the maximum forgetting for Dia data caused by Dood- Such perturbation pulls the model out of the loss space of Dia and encourages the model to flatten as well the loss surface for out-of-distribution data. This is a realistic simulation of the parameter update for downstream CL tasks after ISA. The overall optimization objective is defined as:
data Dood- While also minimizing
min #
Luain (0 + 6; Dia):
(5)
V Cuain (0; D Dood) 8 = -=-PA P V Irrain Vo (0; D Dood) 1|2
subject to
(6)
Similar to SAM, p is a constant controlling the neighborhood radius. Like SAM, our design requires only one additional backpropagation step. In practice, avoid any additional data for ISA, simply split the ISA dataset D ima without overlap into a large subset of Dia and a small subset of Dood
In summary, our FAM aims to not only find a flat loss surface, but also mitigate the loss of gen- eralizability and performance for parameters initialized from ISA by proactively preventing future forgetting. Unlike conventional CL methods that address forgetting after it occurs, we take pre- ventative approach with the help of pretraining. While vanilla SAM offers some benefits in this regard, its perturbation is randomly chosen from the high-dimensional loss space unrelated to for- getting, and is ineffective in handling distribution shifts in CL. Accordingly, we use ISA data to create pseudo-downstream tasks, optimizing on more targeted and informative direction of per- turbation to obtain a more forget-aware flat loss surface. Notably, Eq. and Eq. 6 @exhibit a similar form to MAMI. (Finn et al. (017) (i.e., representative meta-learning framework), suggesting that our FAM promotes generalizability of grgettingg-snnsitve directions in a data-driven manner.
Prompt Augmentation. In preliminary experiments, observe that vanilla SAM faces dif- ficulties in optimizing in the limited and constrained space of the prompt parameters (see Ap- pendix A.4.3). Therefore, we propose to use a prompt augmentation technique in ISA directly increase the complexity of the learnable parameter space. Specifically, we have p'_ = P fMLP(P() with fMLP a shallow multi-layer perception (MLP) (Razdaibiedina et al., 2023b ab. With the ad- ditional parameters and non- non-linearity introduced by the MLP, the learning capacity is improved to enable the flatness- aware minimization for prompt-based methods. At the end of ISA, fMLP will be discarded and we store the augmented p', for later purposes, as shown in Fig. (b).
4.3 NON-PARAMETRIC LOGIT MASKING
Logit Masking, Although the blurry task boundaries of GCL result in a natural replay that partially alleviates forgetting, they make it more difficult for the model to handle the balance between learning and forgetting, especially in the output layers. Specifically, GCL not only leads to an imbalance in class distribution but also enforces the model overfit the re-appeared classes CB and thus
"We leave the choice of external datasets as a future direction since the focus of this work is to enhance prompt parameters without demanding any external data.

---

Published as a conference paper at ICLR 2025
overwrite the knowledge of other previous tasks, especially the disjoint classes CD, This is typical cause of forgetting when the model is trained with a supervised cross-entropy loss Lee. Output logit masking is known to be effective for this in conventional CL, as it isolates the logits of non-repeated classes to maintain their activation at test time. We have:
min
Σ
Coe ④ v.y).
(7)
(2.D)ED
where D is the dataset available and @ is an element-wise multiplication, m E R is a masking vector that masks out some logits and keeps the rest. One of the latest works GCL, MVP (Moon et al. 2023), trains learnable mask to improve information retention. We empirically find that such a learnable mask not desirable, as the method still suffers from strong forgetting to retain the knowledge of previous classes (see Fig. I Right).
Non-Parametric Logit Masking. After amalyzing the purpose of the logit mask and the sub- optimality of existing learnable masks, we propose a simple logit mask that is parameter-fiee free and can effectively reduce the information interference between classes. The first masking strategy is a session-level mask, where the mask is renewed at the end of each session and keeps the logits of available classes in the current session. However, it requires the notion of sessions, which is not always meaningful in online setting. To this end, further propose batch-level mask that operates in the same way but at the batch level. We thus have:
1, yf
y; = for (z, y) E EDD
mi - 10. otherwise,
(8)
where y, is the i-th entry of the one-hot label vector, and D is the set of available data. More precisely, when D refers to the data from a mini-batch, m is a batch-level mask, as shown in Fig. (c). Instead, when 2 refers to the data available for one session, m is a session-level mask. We then multiply the mask with the prediction vector defined in Eq. [
Despite its simplicity, non- con-maraetrric masking brings a substantial improvement in reducing forget- ting, which allows the model to better adapt to GCL. The effectiveness of this simple logit masking arises from the disentanglement of representation learning and learning the classifier. With a frozen PTM ensuring stable representation space, the mask effectively mitigates information interference in the output layers. In contrast, training- training-from-screee froo--srmattcch approaches update all parameters simultane- ously, reducing the efficacy of the logit mask. We include further analysis and comparison with the existing logit masking strategy Caccia et al. (2021) in Appendix A.4.10)
5
EXPERIMENTS
5.1 EXPERIMENTAL DETAILS
Datasets. We consider three representative datasets: CIFAR-100, Tmny- and ImageNet-R with 60k, 100k, 30k training samples and 100, 200, 200 classes, respectively. We follow the setting in Si-Blurry (Moon et al. 2023) in our GCL experiments. Specifically, the disjoint class ratio m is set to 50% and the blurry sample ratio n is set to 10%. We perform 5-session GCL on these three datasets. An extended study of one new class at each session can be found in Appendix A.4.8 Unless specified, the results are obtained on 5 runs with independent seeds. By default, our ISA uses the ImageNet- 1k dataset (Deng et al. 2009) of 1000-class large-scale images.
Baselines. We consider variety of CL methods from different categories, including replay-basee based methods: ER (Rolnick et al., (2019), DER++ (Buzzega et al.) 2020). ER-ACE (Caccia et al. 202I). Rainbow Memory (RM) (Bang et al. 2021). and CLIB (Koh et al. 2021); regularization based methods: LwF (Li & Hoiem 2018). EWC (Kirkpatrick et al. 20177: and prompt-based methods: L2P (Wang al.] (022d) and Dual Promptompt (Wang et al. (022c).. We also compare with the best- performing state-of-the-arr prompt- based GCL method to date: MVP (Moon et al. 2023). Continual fine-tuning and linear probing are considered as the lower-bound methods. All the methods share the same frozen pretrained vit base paass-ppate patthh 6-224 model, except EWC++ and fine-tuning that conducted full parameter tuning on the same pretrained model. Following the model architecture of MVP for a fair comparison, unless specified, we opt for DualPrompt as the baseline method and then add our components. We note that our proposed method not restricted to specific model designs.
7

---

Published as a conference paper at ICLR 2025
Table 1: Average accuracy (%) with standard deviation of different methods, tested with 5-task CIFAR-100, Tmaeeem and ImageNet-R.
CIFAR-100
Tiny-ImageNee
ImageNNt-RR
Buffer
Method
AAuc 7
A
Aaur 7
Aree
Axur
Araei
Finctoning Lincar Probe
39.7
10.42+4.52
15. 40.74
10.42+4.92
7.51
2.29 2.29.0..5
19.6926.00
42. 13
21.9726.43
29. 39.2441.22
16.87
EWC
49.51
52.83
51.70
31.0413.12
31.58
20.72+11 1 11
Lx=
55.51
36.53+11.00
49.00
27.47+7.69
31.61+1.5a
20.62+3.c0
L2P
57.08 43
41.63++12.75
62.09 1.99
35.05+6.75
29.60+1.4a
19.55+44.re
DualPrompt
67-07
56. 36.8213.49
66.09
48.7213341
40. 40.11
29. 29.24:1
MVP
10-+...4.
62.59+2.38
18.99
52.78+22.08
40.60+1.21
31.96+3.00
Ours
80.55=2.11
1.09
80.44;0.00
71.84.1..54
50 00.89911000
20..37
ER
66.07 +4.77
60.68+1.18
59.46 1.51
40.60+2.71
40.31+1.sa
28.85+
DER++
66. 66.92:4 1.
65. 6..63
61.00 1.19
16.03
10. 32 32:00.0
31.53 1.
69.3613.01
10.02
64.52.0.m 7e
56.8210.07
41.0611.33
36.59110 52
500
RM
40.86
23.94
31.96 20.00.00
7.43
18.31
4.14..0.18
60.08 +2.20
07.167
1.66
48.97+1148
37. 17.1821.52
0.0.8
MVP
76.06
79.32+1.28
76.52
65.19+0.0s
49.07
44.17:11.72
Ours
82.37 -1.64
82.27 40.73
79.08
69.9 69.91 +0..2
54.72 +
47 47.48 47.67
ER
69.86
71.81
66.75
55.07
45. 45.74
38. 38.13g 13
DER+
60.42+33.0
65.68+0.72
+0.58
56.81 +-0.65
+2.79+1.31
36.06+ 04
ER-ACE
70.59 +3.09
74.75+0.19
66 06.86.+0.84
68.40+0..8
43. 13.02+1..51
40.49++0.2 0.
2000
RM
23.00
20.51
17 1.
44.5510.37
27. 27.88
24.25-10.0
CLIB
71.53
72.09
65.47
56,87
42.6941.30
35.4310
MVP
78.65
84.42+0.44
80.67 +0.76
74.3420.32
52.47+1.46
50.54+
1.77
85.32+0.26
82.91+0.47
+0.33
57.67
53.62 +3.68
Table 2: Ablation studies of the proposed forgetting-awe aware initial session adaption (ISA-FAM) and non-parametric logit mask (Logit Mask) in MISA.
Component
CIFAR-100
ImageNet-R Axuc
Method
ISA-FAM
Logit Mask|
Aauc f
FiL 4
FLea
Baseline
67.07
33.1222.44
40.11a1.27
43.27
68.97
30.01
41.09g1.74
41.51
Ours
71.81=2 s9
11.71=1.00
15.39+1.71
20. 20.8115.44
80.55
10.35g1.12
50.89_1.s
19.91
Implementation Details. For existing methods, we reuse the implementation of Moon et al. (2023). For MISA, we follow the same training configuration to ensure fair comparison, e.g., using an Adam optimizer with learning rate 0.005 and batch size 32. In ISA, timizer with learning rate 0.0001 and batch size 128 to train the prompts in an offline manner for three epochs. For FAM, we split ImageNet-1k into 900 classes for Dea and 100 classes for Dood without overlap. We randomly sample 10 classes as Dood to simulate a small-scale downstream task and resample new ones when iterate over this subset. More aggressive augmentation is applied on Dood data as we applied double auto-augmentation (Cubuk et al. (019). For a fair comparison with existing logit masking strategies, apply our batch-level mask in MISA by default it does not require a session identifier. More details can be found in Appendix A.I
an Adam op-
Evaluation Metrics. We follow the evaluation protocol of Moon et al. (203)) using Avuc and ALast as the main evaluation metrics. AAUC (Koh et al. 2021) measures the performance of anytime inference, with no task labels provided to satisfy the "no lest-time oracle" requirement in GCL. ALast is equivalentt to the final average accuracy conventional CL. As we identified forgetting essential issue in existing methods, include FLast the average forgetting at the end of training. More details on these metrics can be found in Appendix A.2
5.2 BENCHMARK RESULTS
We compare our MISA to existing methods in Tab Our approach consistently outperforms all the other methods by a significant margin, especially when the model is trained without rehearsal, i.e., buffer size 0. This an extremely challenging case previous GCL methods often rely reply buffer to alleviate forgetting. Specifically, MISA outperforms the previous of MVP by 18.39, 22.06, and 11.96 % points in ALa CIFAR-100 ImageeeeeNet, and ImageNet-R datasets, respectively. While Araa focuses on the overall performance at the end the training, AAUe better exhibits the effectiveness in adaptation to the online data stream. Our approach outperforms MVP
8

---

Published as a conference paper at ICLR 2025
Table 3: Comparison of different masking strategies with different batch sizes. All masking strate- gies are applied to the same baseline method.
Buffer
Buffer 500
Mask
Class 35.41
37.1511 39.4013.72
42.68
48.91
45.32
Learnable
36.00=1.54
39. 18
37.71+1449
14.40=1..7
11.83=1.86
43.49+2.11
Batch-Level
1.50
45.59+1.71
48.41+2.14
34.1440.59
49.00+1.46
47.87+2.60
Session Level 44.1910.1
48.71
48.1912.3
14.5810.77
48.9912.00
50.2611.10
Table 4: AAUC scores CIFARI00 and ImageNet-R with prompts obtained by different strategies.
Method
Logit Mask Naive ISA |SAM| FAM
CIFAR-100
ImageNet-R
Bascline
74.84 99
45.59
77.21+2.60
47.71+1.31
Ours
79.13=2.38
49.99
80.55
11.09
by margin of 12.45, 11.49, and 10.29 % points in AAUC the three datasets, respectively. In particular, MISA beats all conventional CL methods in this measure, which was not achieved by MVP. Furthermore, our approach significantly reduces the variance of the performance with different stochastic classes and data compositions, which confirms its robustness to different realistic learning scenarios.
The integration of the replay buffer can further improve our approach and ensure its standout perfor- mance over existing methods in the same configuration. We also note that MVP benefits more from the replay buffer than MISA. We believe it is because MVP suffers from strong forgetting due to their ineffective logit mask. The existence of replay buffer relieves MVP from this issue. In con- trast, MISA better handles the interference of new and old knowledge through our non-parametric logit masking and performs consistently well with or without replay buffer. Therefore, the replay- independent nature makes MISA more suitable choice in practice.
5.3 FURTHER ANALYSIS
Ablation Study. We present the ablation study on CIFAR-100 and ImageNet-R for our proposed components in Tab. 2 The baseline method is overwhelmed by severe forgetting, which prevents the model from accumulating knowledge for good average accuracy, even with forgetting aware ISA (ISA-FAM). Instead, with the integration of our non-parametric logit masking, forgetting is largely reduced and the performance improves. Finally, without the interference of forgetting, ISA- FAM becomes much more effective. These experiments justify the importance of swift adaption and knowledge retention, as well their complementary roles in GCL.
Logit Masking vs Batch Size. We examine the effectiveness of different logit masking strategies for different batch sizes in Tab. 3 Our session-level masking performs especially well when no This corre-
replay buffer is available. Unsurprisingly batch-level mask fails when batch size sponds to an extreme case of online learning where the model receives one sample at time, which out of the scope for our paper. In addition, the batch-level mask can quickly recover the perfor- mance with replay buffer. Notably, both our paramermettiic masks outperform the learnable mask most cases.
Effectiveness of FAM in ISA. We perform a specific ablation study for our forgetting- aware ISA training strategy, by comparing the impact of prompt parameters with different initialization strate- gies in Tab. 4 All the methods are equipped with our pormmmraet logit mask to overcome the negative impact of strong forgetting. The baseline method utilizes uniform initialization for prompt parameters. Initializing prompts from a naive ISA. i.e., pretraining with Eq. 2 provides better ini- tialization but lacks considerations for generalizability. Thus, direct integration of SAM can improve performance, Our FAM further boosts the model by enhancing the robustness to distribution shift to maintain the generalizability in downstream GCL, as highlighted by the AaUC score. Note that both SAM and FAM are equipped with prompt augmentation to enable batness-awan minimization, whose ablation study can be found in Appendix A.4.4

---

Published as a conference paper at ICLR 2025
Table Validation of transferability of our proposed components on existing prompt- based methods on ImageNet-R. For ISA, both methods reuse the same prompts as our approach.
Baseline
ISA-FAM & Logit Mask
Aauc
ALax
29.42=11466
20.46
L2P
34.11
27.86.1.11
10. 10.80=1121
31.96+5.00
MVP
43.01 +1120
33.52233.38
Table 6: Performance of pre- preitrained checkpoint not overlapping with CIFAR- or Tiny- ImteeNt.
CIFAR-100
Imay-Ne
ImageNet-R
NCH
Method
AxuC
ALas
AauC
ALaa
AAuc
Aus
AxcC
ALa
EWC
11.526.8 e
29.4+3.s
10.9
23.3
31.8413
19.9
50.02124
32.6.9.7
DunlPrompt
47.242.8
42.744.6
48.2+1.1
31.2+2.7
33.741.0
26.740.8
60.6+1177
49.447.8
MVP
48.815.1
35.516.1
46.512.0
26.814.5
35.411.4
24.224.4
60.4
43.5g116
Ours
55.7
53.823.8
56.5+2.0
47.4:1.7
41.2211
35.920.9
71.243.3
61.847.4
Transferability of MISA. Our approach is designed to be general and plug-in for prompt-based CL methods. To demonstrate this transferability, we evaluate our approach with two other meth- ods: L2P and MVP. The former is representative prompt-based method for conventional CL, and the latter is the previous state-of-the-art prompt-based method for GCL. As shown in Tab. 5 both methods benefit from our design. Note that these methods share the same initialized prompts parameters as our approach, without performing additional ISA. The full results can be found in Appendix A.4.1
Knowledge Overlap Analysis. This analysis aims to reduce the potential knowledge overlap between pretraining data and downstream GCL, Specifically, we consider a particular pretrained checkpoint (Kim et al., 2022). which removes 389 classes from ImageNet-Ik that are similar to CIFAR- 100 or Tiny-Imane ImageNe. We repeat our experiments on this subset ImageNet- Ik to exclude information leaks to either the backbone or prompt parameters from ISA. Moreover, included completely duttof-distribution dataset NCH (Kather et al.) (2018) from the medical image domain to showcase the transferability of our learned prompts. As shown in Tab. 6 our approach still achieves substantial improvements over all baselines (see details in Appendix A.4.2)
Memory and Computation Analysis. We analyze in 7 the complexity of representative methods. Specifically, we report the average execution time for each method to learn from one batch on 1 RTX A5000 GPU, Our approach brings a significant per- formance gain without introducing additional param- eters or increased execution time. Combined with the eolly-iineepedeent nature, our MISA has further sat- isfied the requirements of GCL in terms of constant memory.
Table 7: Comparison of resource overheads with the same machine and configuration.
Method
Param.
Time
Accuracy
DualPrompt
637k
1.31s
67.07
MVP
639k
2.75s
68.10
Ours
637k
1.32s
80.55
CL-Relevant Hyperpaarameter. We argue that the hyperparameters of MVP selected with vali- dation data from static view is contradictory to the stochastic nature of data distribution. In con- trast, our approach does not require CL-relevant hyperparameters, which is an important advantage for facilitating the use of our components in other methods or scenarios such as GCL.
6 CONCLUSION
In this paper, we advance prompt-based methods in the challenging GCL setting with our novel approach MISA, which consists of forgetting-awan initial session adaption to improve learning efficiency and generalizability of the prompt parameters, and nonamaramt logit mask to al- leviate forgetting of the output layers. Our extensive experiments show the effectiveness of each component and the significant performance gain compared to existing methods, setting new state of the art for GCL. Moreover, our approach is compatible with other prompt-based CL methods, which seamlessly benefit from components. Finally, approach overcomes the need for CL-relevant hyperparameters and is independent of replay data, enabling robustness and flexibility for CL. applications.
10

---

Published as a conference paper at ICLR 2025
REPRODUCIBILITY STATEMENT
To ensure the reproducibility of our work, we have provided comprehensive details about the ex- perimental setup Section [ including datasets used, baseline methods, implementation details and evaluation metrics. More details about the implementation can be found in Appendix [A.1] All code, models, and configuration files required to replicate our results are made publicly available in repository.
REFERENCES
Abien Fred Agarap. Deep learning using rectified arXiv: 2018.
linear units (relu).
arXiv preprint
Hongjoon Ahn, Jihwan Kwak, Subin Lim, Hyeonsu Bang, Hyojun Kim, and Taesup Moon. SS-II.: Separated softmax for incremental learning. In ICCV, pp. 844-853, 2021.
Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio. Gradient based sample selection for online continual learning. NeurIPS, 32, 2019.
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey Hinton. Layer normalization. arXiv preprint arXiv: 1607.06400 2016.
Jihwan Bang. Heesu Kim, Youngloon Yoo, Jung- Woo Ha, and Jonghyun Choi. Rainhow memory: Continual learning with memory of diverse samples. In CVPR, pp. 8218-8227, 2021.
Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: a strong, simple baseline. NeurIPS, 2020.
Lucas Caccia, Rahaf Aljundi, Nader Asadi, Tinne Tuytelaars, Joelle Pineau, and Eugene Belilovsky. New insights on reducing abrupt representation change in online continual learning. arXiv preprint 104.05025 2021.
Hyuntak Cha, Jaeho Lee, and Jinwoo Shin. Co21: Contrastive continual learning. In CVPR, pp- 9516-9525, 2021.
Xiangning Chen, Cho-Jui Hsich, and Boqing Gong. When vision transformers outperform resnets without pre-training strong data augmentations. arXiv preprint arXiv:2106.011 2021.
Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc Le. Autoaugment: Learning augmentation strategies from data. In CVPR, pp. 113-123, 2019.
Matthias De Lange, Rahaf Aljundi, Mare Masana, Sarah Parisot, Xu Jia, Aleš Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting classification tasks. PAMI, 44(7):3366-3385, 2021.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, pp. 248-255 IEEE, 2009.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x 16 words: Transformers image recognition at scale. arXiv preprint arXiv:2010. 11999, 2020.
Arthur Douillard, Matthieu Cord, Charles Ollion, Thomas Robert, and Eduardo Valle. Podnet: Pooled outputs distillation for small-tasks incremental learning. In ECCV, 2020.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learming for fast adaptation of deep networks. In ICML, pp. 1126-1111 1135. PMLR. 2017.
Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness- awaree aware minimiza- tion for efficiently improving generalization. arXiv preprint 12X1v22010.01412, 2020.
11

---

Published as a conference paper at ICLR 2025
Meghna Gummadi, David Kent, Jorge A Mendez, and Eric Eaton. Shels: for novelty detection and continual learning without class boundaries. In Conference on Lifelong Learning Agents. pp. 1065-1085. PMLR, 2022.
Exclusive feature sets
Chenxing Hong, Yan Jin, Zhiqi Kang, Yizhou Chen, Mengke Li, Yang Lu, and Hanzi Wang. Dynamically anchored prompting for task-imbalanced continual learning. arXiv preprini 2.Xiv:2404.14721, 2024.
Sangwon Jung, Hongjoon Ahn, Sungmin Cha, and Taesup Moon. Continual learning with node- importance based adaptive group sparse regularization. NeurlPS, 33:3647-3658, 2020.
Zhiqi Kang, Enrico Fini, Moin Nabi, Elisa Ricci, and Karteek Alahari. soft nearest-neighhor framework for continual semi-supervised learning. In ICCV, pp. 11868-11777, 2023.
Jakob Nikolas Kather, Niels Halama, and Alexander Marx. 100,000 histological images of hu- man colorectal cancer and healthy tissue, May 2018. URL https: http:////oooooogg//// 50211
cerodo..114455.
Gyuhak Kim, Bing Liu, and Zixuan Ke. A multi-head model for continual learning via out-of- distribution replay. In Conference on Lifelong Learning Agents, pp. 548-563, PMLR, 2022.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A et al. Overcom-
Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka ing catastrophic forgetting neural networks. PNAS, 2017.
Hyunseo Koh, Dahyun Kim, Jung-Woo Woo Ha, and Jonghyun Choi. Online continual learning on class incremental blurry task configuration with anytime inference. arXiv preprint arXiv:2110.10031, 2021.
Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2 104.004.00. 104.08691, 2021.
Zhizhong Li and Derek Hoiem.
Learning without forgetting. PAMI, 2018.
Dongze Lian, Daquan Zhou, Jiashi Feng, and Xinchao Wang. Scaling & shifting your features: A new baseline for efficient model tuning. Advances in Neural Information Processing Systems, 35: 109-123, 2022.
Yue Lu, Shizhou Zhang, De Cheng, Yinghui Xing, Nannan Wang, Peng Wang, and Yanning Zhang. Visual prompt tuning in null space for continual learning. arXiv preprint arXh::2406.05653 246680666866, 2024.
Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. Psychology of Learning and Motivation, 24:109-165, 1989.
Mark D McDonnell, Dong Gong, Amin Parvaneh, Ehsan Abbasnejad, and Anton van den Hengel. Ranpac: Random projections and pre-trained models for continual learning. NeurlPS, 36, 2024.
Sanket Vaibhav Mehta, Darshan Patil, Sarath Chandar, and Emma Strubell. An empirical investiga- tion of the role of pre-training in lifelong learning. JMLR, 24(214):1-50, 2023.
Fei Mi, Lingjing Kong, Tao Lin, Kaicheng Yu, and Boi Faltings. Generalized class incremental learning. CVPR Workshops, pp. 240-241, 2020.
Seyed Iman Mirzadeh, Mehrdad Farajtabar, Razvan Pascanu, and Hassan Ghasemzadeh. Under- standing the role of training regimes continual learning. NeurIPS, 33:7308-7320 2020.
Yen-Y Moon, Keon-Hee Park, Jung Uk Kim, and Gyeong-Moon Park. Online class incremental learning on stochastic blurry task boundary via mask and visual prompt tuning. In ICCV, pp. 11731-11741, 2023.
Oleksiy Ostapenko, Mihai Puscas, Tassilo Klein, Patrick Jahnichen, and Moin Nabi. Learning to remember: A synaptic plasticity driven framework for continual learning. In CVPR, 2019.
12

---

Published as a conference paper at ICLR 2025
Anastasia Razdaibiedina, Yuning Mao, Rui Hou, Madian Khabsa, Mike Lewis, and Amjad Almahairi. Progressive prompts: Continual learning for language models. arXiv preprint arXiv:2301.12314, 2023a.
Anastasia Razdaibiedina, Yuning Mao, Rui Hou, Madian Khabsa, Mike Lewis, Jimmy Ba, and Amjad Almahairi. Residual prompt tuning: Improving prompt tuning with residual reparameter- ization. arXiv preprint arXiv:2305.03937, 20236.
David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Experience replay for continual learning. NeurIPS, 32, 2019.
Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention the task. In ICML, 2018
Sheng Shen, Shijia Yang. Tianjun Zhang. Bohan Zhai, Joseph E Gonzalez, Kurt Keutzer, and Trevor Darrell. Multitask vision-language prompt tuning. In WACV, pp. 5656-5667, 2024.
Guangyuan Shi, Jiaxin Chen, Wenlong Zhang, Li-Ming Zhan, and Xiao-Ming Wu. Overcoming catastrophic forgetting in incremental few-shot learning by finding flat minima. NeurIPS, 34: 6747-6761, 2021.
James Seale Smith, Leonid Karlinsky, Vyshnavi Gutta, Paola Cascante- Bonilla, Donghyun Kim, Assaf Arbelle, Rameswar Panda, Rogerio Feris, and Zsolt Kira. Coda-prompt. Continual de- composed attention-based prompting for rehearsal-fre continual learning. In CVPR, pp. 11909- 11919, 2023.
Bart Thomee, David A Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li-Jia Li. Yfee100m: The new data multimedia research. Communications of the ACM, 59(2):64 73, 2016.
Yun- Yun Tsai, Chengzhi Mao, and Junfeng Yang. Convolutional visual prompt for robust visual perception. Advances in Neural Information Processing Systems, 36, 2024.
Tu Vu, Brian Lester, Noah Constant, Rami Al-Rfou, and Daniel Cer. Spot: Better frozen model adaptation through soft prompt transfer. arXiv preprint arXiv:2110.07904, 2021.
Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-uesd birds-200-2011 dataset. Technical Report CNS-TR-201 1-001, California Institute of Technology, 2011.
Fu- Yun Wang, Da-Wei Zhou, Han-Jia Ye, and De-Chuan Zhan. Foster: Feature boosting and com- pression for flass-incrementa learning. In ECCV, pp. 398-414. Springer, 2022a.
Liyuan Wang, Jingyi Xie, Xingxing Zhang, Mingyi Huang, Hang Su, and Jun Zhu. Hierarchical de- composition of prompt-based continual learning: Rethinking obscured sub-optimality. NeurIPS, 36, 2024a.
Liyuan Wang. Xingxing Zhang, Hang Su, and Jun Zhu. A comprehensive survey of continual learning: Theory, method and application. PAMI, 2024b.
Yabin Wang, Zhiwu Huang, and Xiaopeng Hong. S-prompts learning with tre-trained transformers:
An occam's 's razor for domain incremental learning, NeurIPS, 35:5682-5695, 2022b.
Zifeng Wang, Zizhao Zhang, Sayna Ebrahimi, Ruoxi Sun, Han Zhang, Chen-Yu Lee, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, et al. Dualprompt: Complementary prompting for rehearsal-free continual learning. In ECCV, pp. 631-648 Springer, 2022c.
Zifeng Wang, Zizhao Zhang, Chen- Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vin- cent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. In CVPR, pp. 149, 2022d.
Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning. In CVPR, 2019.
Enneng Yang, Li Shen, Zhenyi Wang, Shiwei Liu, Guibing Guo, and Xingwei Wang. Data aug- mented flatness-aware gradient projection for continual learning. In ICCV, pp. 5630-5639, 2023.
13

---

Published as a conference paper at ICLR 2025
A APPENDIX
A.1 IMPLEMENTATION DETAILS
We follow the implementation of Moon al. (2023) for a fair comparison with existing methods. Specifically, all methods use pretrained ViT checkpoint VIT-B/16 (Dosovitskiy et al.) 2020). We freeze the backbone, learning only the prompt parameters and the classification head. As such, the total number of parameters is around 86M, whereas the trainable one is around 0.6M. We equip our approach with the same memory buffer as DualPrompt in the implementation of Moon et al. (2023) with size 500 and 2000, if applicable.
For forgetting- aware optimization, we use the first 900 classes of ImageNet- Ik as Did for initializing the parameters and the last 100 classes for Dood to calculate perturbation. During training, randomly sample classes as actual Dood. Since this subset is smaller than Did. whenever we go through the entire small subset, until the end of training. To apply a more aggressive data augmentation on Dood data, we applied two times Aut Auggmen to augment the image, while for Dia data, only applied it once.
will resample the other 10 classes. We will repeat this process
For prompt augmentation, we use a two-layer MIP with projection dimensions (768, 96) and (96, 768), which corresponds empirically found it more effective than up-sampling. We add LayerNorm (Ba et al., 2016) and ReLU (Agarap 2018) activation in between these layers. Once the ISA is finished, this MLP discarded.
down-sampling rate of 1/8 for the feature dimension D. We
A.2 EVALUATION METRICS
We start from conventional evaluation metries in CL. Let Rs, be the evaluation score of the model after session i with respect to the data in the session j. We then have an evaluation matrix I such that the column R3 shows the history of evaluation after each session with respect to the data in the session j. The final accuracy ALasi can be defined as:
A Last
R
(9)
T
and the forgetting can be defined as:
1 T Σ (max(R3) RT,;),
Fiast
(10)
where Arast is the higher the better and Fiast is the lower the better.
AxiC is the anytime inference metric proposed by Koh al. (2021) to evaluate the GCL perfor- mance. Here, the evaluation is not performed at the end of each session but whenever the model observes number of samples. Let the total number of evaluations performed be L and / be the time-step for evaluation. We have:
L
AAUC Σ R
(11)
1=1
where Rt shows the evaluation score of the model at the time-step / with respect to the data it has observed until time-step (.
A.3 SI-BLURRY AS A REALIZATION GENERALIZED CLASS INCREMENTAL LEARNING
Following the definition of generalized class incremental learning (GCIL) in Mi et al. (020)), we reformulate the Si-Blurry scenario in this GCIL framework. Let C be the set available classes whose cardinality is N |C|. Let (CP, CB3 be partition of C where CD represents disjoint classes and 8 represents blurred classes. m is the disjoint class ratio such that m = [C I/ Let X be the set of available samples, we thus have: X {z; Y/ E CD) and X {z, : y/ CB where y; is the class label of the sample T;. Let T be the total number of learning sessions.
(CP) J==1.. and ( (CB) I=1... are then partition of CD and C to allocate classes for each session,
14

---

Published as a conference paper at ICLR 2025
Table 8: Validation of transferability of our proposed components on existing prompt-based meth- ods. We perform experiments on 5-task ImageNet-R, averaged over 5 runs.
Component
ImiggeNei-R
Baseline
Init-Adape
Logit-Mask
Axcc
ALai
29. 42 1.44
20.46 20.4644.11
31.72+1.90
21.09=a.00
L2P
31 +4..9
25.88
2..35
27.8641.11
40.60
31.96
41.12
32.05
MVP
40 98. 40.86
32.3 +3.32
1:28
33.5243.36
respectively. In Si-Blurry, the number of classes for each session is not uniform across tasks as the number is randomly sampled. For disjoint data, we have X = fa; E CPP. ). Let {x _²3 be partition of x with blurry sample ratio n 1/1x/ BI. x is normal set and 5 is the blurred set. X will be divided into sessions as X {2) 2; E X ;) E CF) In a blurred boundary setting, XB is non-empty and will be shuffled and randomly distributed to different sessions as { The final set of data available to the session t then X, = x U X U UX Accordingly, the classes available for the session is C, = C. U CP U CB where Vr, E x E ČB. With this formulation, we revisit the properties of GCIL defined in Mi al. (2020) to show that Si-Blurry can be seen as a realization of GCIL in an online learning paradigm.
Property 1:
The number of classes could differ in each session. We have:
+. P(CC) + (C,)) > 0,
(12)
where P(-) is the probability.
Property 2:
Classes could appear in different sessions. We have:
Vi + j, P(C, * 0) > 0,
(13)
Property 3:
The number of samples for each class could be different in one session. For session
f, we have:
Vi, j, H P( * (x 0,
(14)
where [X]] is the number of samples for class in session t.
As GCIL in Mi et al.) (2020) has no explicit constraint on the number of iterations allowed for learning, Si-Blurry features the more complex and realistic setting with the one-pass requirement.
In summary, Si-Blurry can be seen as realization of GCIL of Mi et al. (2020) with an online learning paradigm.
A.4 ADDITIONAL EXPERIMENTS
A.4.1 TRANSFERABILITY
We include the full table for the validation of the transferability of our proposed components on existing prompt-base methods for the effectiveness of each component, as shown in Tab, g
A.4.2 KNOWLEDGE OVERLAP ANALYSIS
To ensure that there is no overlapping of data with ImageNet, Kim et al. (2022) manually removed 389 classes from the original 1000 classes in ImageNet that similar/identical the classes in CIFAR-10, -EFAR-10., or Iny-ImageNet. They pre-trained the network with the remaining subset of 611 classes of ImageNet and released the checkpoint for public research purposes.
While the list of the 389 classes is not publicly available, we reached out to the authors for the list and performed additional ISA to obtain prompt parameters that initialized without knowledge
15

---

Published as a conference paper at ICLR 2025
Table 9: Validation of ISA for different types prompts. All methods are equipped with our logit mask to exclude the impact of strong forgetting.
Prompt Type
ImageNNet-R
Baseline
G-prompt
E-prompt
Axuc f
ALax f
47.84
40.82 20.73
DualPrompt
18.10+1.20
12. 12.66:2..44
50.89-100
43.92 43.92+0.35
Table 10: CL experiments on ImageNet-R with prompts obtained by different strategies. We use our logit masking for all to exclude the impact of strong forgetting.
Method
Logit-Mask
SAM Augmentation
AATC
Baseline
45 4..39+1177
47 66..6.1.... 1.24
Ours
47.7111:31
49.99
overlap. These additional experiments ensure that the improvements of our proposed components not from leak of information from the pretraining data.
A.4.3 SAM WITH PROMPT-TUNING
Although SAM has been shown be effective in improving the generalizability of CL models, works most of the time with full-parameter tuning. The effectiveness with parameter-efficien tun- ing methods, i.e., a frozen pretrained backbone and few learnable parameters, remains unclear. We first conducted toy example by applying SAM with a pretrained ViT backbone and prompt parameters a two-task conventional CL setting. The evaluation on the first task shows the conver- gence of the model, whereas the evaluation the second task is a direct verification of the model's generalizability.
As shown in Fig. directly integrating the SAM optimizer into prompt-basee tuning cannot work well, or even hinders the model's convergence. We speculate is because the simplicity and scarcity of the prompts make them incapable of modifying the loss surface of the pretrained model. More- over, the imposed flatness constraint prevents the loss from converging to the original sharp minima. Based on this analysis, further conduct experiments by de-freezing more parameters the train- ing. Specifically, 0.3M refers to tuning the prompts and the output layer fe, while 7M and 14M introduce additional parameters from the last one or two self-attention blocks of fr. respectively. We observe that when more parameters are available for training, the SAM begins to show its power. This experiment confirms that SAM has difficulty when applied large frozen pretrained model and few learnable prompt parameters.
93
0.3M
92
0.3M +SAM
91
SAM
Accuracy
14M+SAM
89
88
87
86
85
84
T1
T2
Tasks
Figure 3: Toy example of implementing SAM with prompt tuning. We performed 2-task CL with randomly sampled 50 classes from ImageNet-1K. The first task is optimized with SAM and the second task uses a standard Adam optimizer. 0.3M, 7M, and 14M represent the number learnable parameters, and additional parameters are from the last layers of the pretrained backbone fr. SAM works better with more parameters becoming learnable.
16

---

Published as a conference paper at ICLR 2025
Table 11: Validation of the effectiveness of ISA with respect to the training iterations.
ImageNet-R
Baseline
Epoch
Axuc
Atuo
48. 18.1441.20
42.4010 57
49. 49.79 41.111
43.3340..a
DrolPrompe
49. +9.99
43.48-0.43
19.87
13.22
Table 12: Validation of the effectiveness of downstream task adaptation.
ImageNet-R
Baseline
Adaptation
Axuc
A
49.99 19.99+1.21 11.1
43.48+0.45
DualPrompt
50.16;1.2
43.56=0.42
A.4.4 ABLATION STUDY OF PROMPT AUGMENTATION
We show in Tab. that direct integration of SAM shows only marginal performance gain, as SAM struggles to work with prompt-tuning. In the end, with our prompt augmentation technique, we observe additional boost to the performance.
A.4.5 IMPACT OF PROMPTS (G-PROMPT AND E-PROMPT)
As our baseline method (Wang et al. 2022c is composed of two types of prompts: G-prompt, which for capturing general information, and E E-prompt, which for capturing task-specific information. We conduct an ablation study in Tab. 9 on these two types of prompts to validate the effectiveness of each type of prompt.
A.4.6 IMPACT OF TRAINING EPOCHS
Our ISA involves joint offline training the large-scale dataset. The training only lasts for epochs. We show Tab. [1 the prompt parameters the best generalizable warm-up at the end of epoch Further training brings significant improvements.
A.4.7 IMPACT OF DOWNSTREAM CL ADAPTATION
Our approach directly uses the prompts obtained from ISA to downstream tasks. However, in the cases where there is significant domain gap between the ISA dataset and the downstream CL. dataset, it is preferable to apply dedicated domain adaptation module to compensate for this mis- match. We found that a simple shift-and-scale method (Lian et al. 2022) can help the model bet- ter adapt the downstream tasks without hindering the knowledge in the ISA prompts, shown in Tab. [2 We leave this adaption as an interesting future direction since it was not the main focus of this work.
A.4.8 CASE STUDY OF ONE CLASS TIME GCL.
We conducted a case study of an extreme case where each new task comprises only one class. Note that the tasks are not strictly disjoint, as the data stream can still contain data from previous tasks due to GCL's blurry task boundary requirements. Typically, any class-incremental scenario can be decomposed to one-class increments, without assuming the number classes in each new task. This challenging setting demands the model to swiftly adapt and learn new knowledge about new single class without forgetting previous knowledge that is underrepresented in the current data stream. The results presented in Tab. 13 Specifically, conducted 100-task CIFAR-100 100 and 200-task ImageNet-R experiments with 5 random seeds. We note that our method still outperforms existing methods by substantial margin.
17

---

Published as a conference paper at ICLR 2025
Table 13: Performance of different methods in GCL with one incremental class at time.
ImageNet-R
CIFAR-100
Method
AAUC
ALast
AAUC
A
EWC
1.1
3.1
68.1
64.5
DualPrompt
25.615.6
31.211..6
80.410.4
79.1
MVP
1.1+1.4
5.244.4
78.7
76.2+0.2
Ours
31.1.0.66
42.710.7
85.3
83.710.3
A.4.9 EXPERIMENTS OF PARTIALLY OR COMPLETELY UNAVAILABLE PRETRAINING DATASET.
Although our ISA experiment was based on the same pretrained dataset as the backbone, this is not strict requirement. The corresponding pretraining data might be partially or completely unavail- able regarding large pretrained vision, vision-language, and language models. In these cases, the ISA should be able to learn robust and transferable general knowledge for the prompt parameters to deal with the potential knowledge mismatch with the backbone. We conduct experiments with two cases: (1) the pretrained data is partially available, (2) the pretrained data is completely unavail- able. For the former, we use subset (ImageNet- 100, denoted at IN-100) of the pretraining dataset (ImageNet- lk, denoted as IN-1k), and for the latter, we use ImageNet- Ik as the ISA dataset whereas the backbone (CLIP-VIT ViT was trained ct al. 2016). Additionally, we highlight that general and diverse ISA datasets are beneficial for the prompt parameters capture transferable knowledge. To showcase this, also conducted ISA on a fine-grained dataset, i.e., CUB200 (Wah et al.] (011). where we observed clear performance drop. The evaluation was performed with 5-task GCL in ImageNet-R with 5 random seeds. The results are in Tab. [4]
completely unrelated dataset, i... YFCCI00M (Thomee)
A.4.10 ANALYSIS OF OUR LOGIT MASKING STRATEGY.
In this section, we compare our non parameetttiic logic masking with that of ER-ACE [Caccia et al. 2021 Although they notations of ER-ACE in this analysis.
similar, they differ in their cases and objectives. We follow the
First of all, the mask of ER-ACE is replay-dependent whereas ours is replay-ineepenin And are primarily interested in the case where we do not use replay buffer.
The second difference lies in the objective of the mask. Although both masks deal with the existence of current-class data Xeur and old-class data in one mini-batch, the nature of the data is different. In ER-ACE, old-class data xb is sampled from replay buffer, whereas in our case, old-class data X is the new data of old classes from the blurry task boundaries. Let C be the class in data batch X, and Cseen denote all the classes the model has seen. Thus Csf Ceur S Czeen. For ER-ACE, they applied 1c for Your and 1 for xM with 1c a binary vector masking out classes not in C. The idea was to prevent the gradient update of Your from interfering with representations of previously seen classes. This is rather conservative strategy as the idea to prevent old-class representation from being changed. Instead, since our Xold are new data of old classes, unlike x²r, we can opt for a more proactive strategy to encourage representation update of Cour and Cooa at the same time. Thus our mask to Xeur U Xold is 1cau.UCare" Similarly, exclude other classes that have not been seen in the current mini-batch to avoid interference with their representations. The overall optimization objective of ER-ACE is
Lx0
LCE(X yeurlCouui)
whereas ours is
Lee Lee U Xeur yold U U U Ccur):
where Lee (X, Y/C) the cross-entropy loss with 1c the binary mask on the data Y.
18

---

Published as a conference paper at ICLR 2025
Table 14: Experiments of partially completely unavailable pretraining data for ISA.
PTMINNIIHHSSANN11000) PIMYFCCCIIM)::SAIINNII PIMIN-IA-SSAACUBBB000
Method
Auuc
ALas
AxuC
A
AAUC
ALast
EWC
31.541.0
20.7
30.6+1.3
22.2+3.8
31.541.0
20.7+1.1
DualPrompt
40.141.2
29.2
37.940.9
30.3
40.1
29. 29.244.6
MVP
40.641.2
31.943.0
34.0+1.3
25.0+4.4
31.943.0 40.6
Ours
49.5+11.3
43.0
49.3=1.2
42.3
43.4+1.9
36.6
19


Reviews:
[{"review_id": "hPdQWsWNEl", "reviewer": "Reviewer_n9s5", "summary": "In this paper, the authors proposed a new prompt based method for general continual learning. Here they utilize forgetting-aware initial session adaption that employs pretraining data to initialize prompt parameters to improve generalizability. Additionally, they propose to use a simple non-parametric logit mask at the output layers to mitigate catastrophic forgetting. With comprehensive experiments and analysis they show the effectiveness of the method in GCL setup.", "questions": "See weakness.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The idea of the paper is clearly presented with comprehensive experiments, ablations, and analysis. \n2. Though the paper uses concepts from other previous works the combination of the ideas and their effective implementation in the GCL setup is interesting and useful for the continual learning community.", "weaknesses": "1. The paper aims to address the challenging general continual learning (GCL) setup. However, in this setup, the classes can appear one by one and the model may need to learn one class at a time. No discussion on how to handle such a situation is presented in the paper. \n2. Though the authors presented experiment with blurry task/class boundaries, in true GCL setup the labels of the classes might not be known ahead of time. In such cases, the model needs to detect novel classes and then learn that new classes (see [1]). How the current algorithm would handle such scenarios is not discussed in the paper. \n3. The algorithm assumes the availability of the pertaining data. This is a strict constraint. In most practical cases this would not be available. For instance for large opensourced pre-trained vision, vision-language, and language models, the corresponding pretraining data is not publicly released. In the absence of any pertaining data or with very limited accessibility of pertaining data how this algorithm will perform? \n4. The paper failed to cite and/or compare with many related works. For example: [2,3,4].  \n\n\n[1] M. Gummadi, D. Kent, J. Mendez, and E. Eaton, “SHELS: Exclusive\nFeature Sets for Novelty Detection and Continual Learning Without Class\nBoundaries,” PMLR Conference on Lifelong Learning Agents, pp. 1065-\n1085, 2022\n[2] Coda-prompt: Continual decomposed attention-based prompting for rehearsal-free continual learning. In CVPR, pp. 11909–\n11919, 2023.\n[3] RanPAC: Random Projections and Pre-trained Models for Continual Learning. (https://arxiv.org/abs/2307.02251)\n[4] Visual Prompt Tuning in Null Space for Continual Learning (https://arxiv.org/abs/2406.05658)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3UyeKYkYIG", "reviewer": "Reviewer_iGuV", "summary": "This paper introduces MISA, a novel method that enhances prompt-based approaches in Generalized Continual Learning (GCL) by incorporating a forgetting-aware initial session adaptation and a non-parametric logit mask. Experiments demonstrate the proposed method's effectiveness and its ability to improve performance on three representative datasets: CIFAR-100, Tiny-ImageNet, and ImageNet-R.", "questions": "Please address the concerns mentioned above in weaknesses.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 1, "strengths": "1. The paper is well-structured and easy to understand.\n2. The authors aimed to propose an effective methodology for a scenario that is closer to real-world conditions, compared to the common CL scenarios assumed by existing prompt-based CL methods.", "weaknesses": "1. The methodology appears to lack substantial novelty. The approach of using SAM for effective initialization and leveraging previously learned knowledge with masking for robustness against forgetting has been proposed and utilized numerous times in CL.\n2. In the context of GCL, demonstrating the performance of a pre-trained model on natural images only with similar or closely related benchmarks weakens the paper's claims. It would be important to show that the SAM + masking with a pre-trained model approach remains strong when tested on benchmarks composed of datasets from significantly different domains, such as completely different views, 3D data, or medical imagery.\n3. The scenario used in the paper is based on an already proposed (limited) GCL scenario that assumes blurry and task-free settings, which may not fully capture the (real) general scenario. I believe including experiments that reflect more realistic situations, such as not knowing the number of classes in the beginning or noisy datasets, would enhance the paper's contribution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AqLjmLMKqC", "reviewer": "Reviewer_y3su", "summary": "The paper presents MISA, an innovative approach for tackling General Continual Learning (GCL), arguably the most challenging form of Continual Learning. They do so by starting from a pre-trained model, refined with prompts, and introducing two key components:\n\nISA-FAM, which is applied at the start of each learning session (essentially, at the beginning of each GCL task). This component ensures proper prompt initialization. Here, \"proper\" means achieving both robust generalization capabilities and mitigating future forgetting. To enhance the learning effectiveness of prompts in this phase, the authors introduce a “prompt augmentation” technique, where prompts are trained using an approach involving an MLP that is later thrown away at the end of this phase..\n\nNon-parametric Logit Masking: similar to prior CL methods, the authors propose a session mask that preserves the final activations of classes not present in the current learning session. Since GCL involves blurred task boundaries and often lacks task identifiers, they use a unique batch-level mask designed to function universally across scenarios.\n\nThe experimental validation utilizes widely recognized CL datasets, with a particularly rich ablation analysis that provides in-depth insights into the method’s effectiveness.", "questions": "1) In the ISA-FAM phase, the classification heads are trained with the prompts but later discarded (replaced by the previously trained heads) when the real learning session begins. Why is that? This way, I suppose prompts would \"communicate incorrectly\" with the classification heads, as the latter are replaced;\n2) is there any reason why DualPrompt was selected as the prompting method? Did you consider more performing methods like Coda-Prompt[1] or HiDe-Prompt[2]?\n3) you did cite [3] but not [4] for General Continual Learning, although the latter was published earlier. Is there any particular reason for this? Did you prefer the former over the latter in your GCL formulation?\n\n[1] Smith, James Seale, et al. \"Coda-prompt: Continual decomposed attention-based prompting for rehearsal-free continual learning.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023. \n\n[2] Wang, Liyuan, et al. \"Hierarchical decomposition of prompt-based continual learning: Rethinking obscured sub-optimality.\" Advances in Neural Information Processing Systems 36 (2024).\n\n[3] Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: a strong, simple baseline. 2020.\n\n[4] Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. PAMI, 44(7):3366–3385, 2021.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1) The method introduced by the authors meets their claim and proves to be the SOTA in GCL;\n2) the ablation section is really rich and thorough, probing deeply every aspect of the method;\n3) while the components used by the authors are not exactly novel per se, their combination and the proposed modifications prove to be interesting and effective;\n4) by cleverly leveraging the pre-trained data, the authors eliminated the need for a replay buffer. At the same time, they showed that the model can benefit from storing past exemplars in case it is allowed;\n5) the key components of MISA are plug and play, as shown by one of the ablations (Table 5), meaning that they could work out of the box if applied on other methods;\n6) The code is available in the supplementary material, which greatly helps reproducibility.", "weaknesses": "1) While it is true that a buffer is not mandatory for MISA, it still depends on the pre-train data for the ISA-FAM phase. This still poses a limitation, as the pre-train data may not be available;\n2) the experimental section lacks the case of a completely out-of-distribution dataset.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "62S1g14CJa", "reviewer": "Reviewer_1W4v", "summary": "The paper proposes a prompt-based method for General Continual Learning (GCL), an incremental setting where task boundaries between consecutive tasks are not rigid, allowing certain classes to reappear in later tasks. GCL represents a challenging yet realistic scenario, as the absence of clear boundaries limits the applicability of many algorithms and techniques, particularly those relying on model expansion. The authors leverage two key strategies: (1) refining the pre-trained model’s initialization to improve resilience to distribution shifts, thereby preserving performance; and (2) using masking techniques during cross-entropy loss computation, selectively based on the classes present in each batch. Experimental results indicate strong, promising performance relative to existing solutions, complemented by extensive ablation studies.", "questions": "See section above.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The writing is brilliant. It is a pleasure to review papers like this one.\n- The idea behind FAM is noteworthy and represents the main contribution of this work.\n- The experiments are extensive and cover all that is needed to understand the proposed approach.", "weaknesses": "I have only a few minor questions and suggestions.\n\n1. While reading Sec. 4.1, I was initially puzzled by the optimization problem outlined in Eq. 2. Specifically, I questioned the advantage of fine-tuning prompts using the same data employed during pre-training. Since the gradients should be near zero around the pre-training weights, this raised some concerns about the utility of prompts. The rationale becomes clearer a few lines later; however, to improve readability, I suggest that the authors provide some context earlier in Sec. 4.1.\n\n2. While reading Sec. 4.3, I began to wonder if FAS could also be effectively applied to standard class-incremental learning. The idea does not seem strictly constrained to the GCL setting.\n\n3. In reviewing Eqs. 5 and 6, I noticed similarities with Meta-Learning, particularly Meta Agnostic Meta Learning (MAML). These equations suggest an optimization setup where some data is used to train the model (inner loop) and other data is used for “differentiable” evaluation (outer loop). I believe that discussing this connection in the main paper would add value, but I’d like to hear the authors' perspective on this.\n\n4. The prompt augmentation is the only aspect that doesn’t fully convince me. Even with an MLP layer, the resulting complexity seems comparable to that of a straightforward learnable prompt. Perhaps the MLP affects the training trajectory, introducing a smoothing effect across iterations. In my experience, concatenation-based prompting strategies tend to be less effective, though this may not be due to reduced parameters or complexity. Instead, I suspect it depends on how these prompts are incorporated into the pre-trained model. I recommend that the authors test their approach with [a] as a fine-tuning strategy (i.e., using addition instead of concatenation).\n\n[a] Liu, H., Tam, D., Muqeeth, M., Mohta, J., Huang, T., Bansal, M., & Raffel, C. A. (2022). Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. Advances in Neural Information Processing Systems, 35, 1950-1965.\n\n5. The masking strategy used in this work is identical to that of ER-ACE (ICLR 2022). I think the paper should be more transparent about this similarity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)