OVOR: OnePrompt with Virtual Outlier Regularization for Rehearsal-Free Class-Incremental Learning
Authors: Wei-Cheng Huang
Chun-Fu Chen
Hsiang Hsu
Venue: ICLR 2024 poster (2024)
Keywords: Rehearsal-free continual learning
Class-incremental learning
Parameter-efficient fine-tuning
Outlier regularization
Primary area: transfer learning, meta learning, and lifelong learning

Abstract: Recent works have shown that by using large pre-trained models along with learnable prompts, rehearsal-free methods
for class-incremental learning (CIL) settings can achieve superior performance to prominent rehearsal-based ones.
Rehearsal-free CIL methods struggle with distinguishing classes from different tasks, as those are not trained together.
In this work we propose a regularization method based on virtual outliers to tighten decision boundaries of the classifier,
such that confusion of classes among different tasks is mitigated.
Recent prompt-based methods often require a pool of task-specific prompts, in order to prevent overwriting knowledge
of previous tasks with that of the new task, leading to extra computation in querying and composing an
appropriate prompt from the pool.
This additional cost can be eliminated, without sacrificing accuracy, as we reveal in the paper.
We illustrate that a simplified prompt-based method can achieve results comparable to
previous state-of-the-art (SOTA) methods equipped with a prompt pool, using much less learnable parameters and lower inference cost.
Our regularization method has demonstrated its compatibility with different prompt-based methods, boosting
those previous SOTA rehearsal-free CIL methods' accuracy on the ImageNet-R and CIFAR-100 benchmarks. Our source code is available at https://github.com/jpmorganchase/ovor.

Published as a conference paper at ICLR 2024
OVOR: ONEPROMPT WITH VIRTUAL OUTLIER REGULARIZATION FOR REHEARSAL-FREE CLASS- INCREMENTAL LEARNING
Wei-Cheng Huang, Chun-Fu (Richard) Chen, Hsiang Hsu
Global Technology Applied Research, JPMorgan Chase, USA {wei-cheng huang, richard. cf chen, hsiang hau @ jpmchase com
ABSTRACT
Recent works have shown that by using large pre-trained models along with learn- able prompts, rehearsal-free methods for class-incremental learning (CIL) settings can achieve superior performance to prominent rehearsal-based ones. Rehearsal- free CIL methods struggle with distinguishing classes from different tasks, those are not trained together. In this work we propose a regularization method based virtual outliers tighten decision boundaries of the classifier, such that confusion of classes among different tasks is mitigated. Recent prompt- based methods often require pool of task-specific prompts, in order to prevent overwrit- ing knowledge of previous tasks with that of the new task, leading to extra com- putation in querying and composing an appropriate prompt from the pool. This additional cost can be eliminated, without sacrificing accuracy, as we reveal in the paper. We illustrate that a simplified prompt-based method can achieve results comparable to previous state-of-the-art (SOTA) methods equipped with a prompt pool, using much less Icarnable parameters and lower inference cost. Our regu- larization method has demonstrated its compatibility with different prompt-based based methods, boosting those previous SOTA rehearsal-free CIL methods' accuracy the ImageNet-R and CIFAR- 100 benchmarks. Our source code is available at com/ wpw.rrgancaaee//or..
INTRODUCTION
Continual learning, with its aim to train model across sequence of tasks while mitigating catas- trophic forgetting as initially identified by McCloskey Cohen (1989), addresses the challenge of severe performance degradation on previously-seen data when exposed to new samples. Within the continual learning paradigm, class-incremental learning (CIL) involves presenting the model with sequential tasks, each containing samples from different classes, with the ultimate goal of building universal model spanning all seen classes across tasks. Continual learning algorithms can be broadly categorized into two main streams: rehearsal-based and rehearsal-free methods. Rehearsal-basee al- gorithms store subsets of data from previous tasks for model fine-tuning, but this approach may always be practical due storage limitations and privacy concerns. In contrast, rehearsal-free algorithms manipulate model weights to encourage the recall of previously-seen samples through regularization techniques. Recent advancements in prompt-based rehearsal-free methods for CIL, including works by Wang et al. (2022c). Smith et al. (2023a), and Wang et al. (2022b), have show- cased that by leveraging parameter-efi efficiient fine-tuning (PEFT) techniques introduced by Lester et al. (2021), Hu et al. (2022), and Li & Liang (2021), these methods can achieve comparable, and sometimes even superior, results when compared their rehearsal-bacee counterparts.
Despite the notable achievements of prompt- based rehearsal-free methods, they face two significant challenges. First, these methods require the model to make output predictions among all available class labels without the explicit task information, leading inter-task confusion when evaluating on previously seen samples from multiple tasks. This confusion arises because neural networks tend to assign high output scores to outlier inputs (Nguyen et al., 2015), resulting in the model misclas- sifying classes from different tasks. From the model's perspective, all data from different tasks can be seen outliers since they have never been presented together before. Second, prompt-based

---

Published as a conference paper at ICLR 2024
approaches often employ a pool of learnable prompts that are inserted into the pre- pre-trained model. Additionally, subsets of this prompt pool are allocated to different tasks, following the parameter isolation concept (Lange et al., 2022), to prevent knowledge interference between tasks. However, this setup necessitates a mechanism for selecting or combining specific subsets of the prompt pool to generate the appropriate prompt. The methods discussed earlier typically use the input's encoding by the pre-trained model as the query, incurring additional computational costs, Moreover, there is uncertainty regarding whether the correct task can be reliably identified from the query, particularly in the context of class-incremental learning (CIL), where task-to-clas assignments are random and lack semantic meaning.
In this paper, we aim to address these two challenges at once. We propose regularization method with virtualy-synthesizzed outliers to reduce the inter-task fusion under the rehearsal-free setting, leading to better performance; while revisiting the necessities of using complicated mechanism to compose prompt from a prompt pool, and propose a simple prompt-based CIL. that leads to better efficiency in both computations and parameters.
Our contributions can be summarized as follow:
1. We propose an outlier regularization technique that tightens the decision boundaries of the classes among different tasks, and serves as an additional regularization term that can be applied to various prompt-based CII. methods to promote the performance thereof.
2. We demonstrate that for rehcarsal free CIL, prompt-based method using one single prompt, named OnePrompt, can obtain comparable results with previous state-of-the-arr (SOTA) prompt-based methods using much more prompts, leading to a far faster inference speed and less number of trainable parameters.
3. We propose OVOR, combining OnePrompt and the Virtual Outlier Regularization tech- nique, with extensive evaluations on ImageNet-R and CIFAR- 100. Empirical evidence shows that our method can reach performance on par with or superior to existing SOTA prompt-based methods in terms of computations, parameters and accuracy.
RELATED WORK
Continual Learning. For addressing catastrophic forgetting continual learning, existing ap- proaches are often categorized (Lange et al., 2022; Mai et al., 2022) into regularization-based, replay-based and parameter isolation. Regularization-ban based methods usually apply regularization during training that encourages similarity between the new and old models, whether in model weights (Kirkpatrick et al., 2016; Zenke et al., 2017: Aljundi et al., 2018) or in model outputs (Li & Hoiem, 2018 8; Hou et al., 2018; Cha et al., 2020). As our proposed regularization method does not require an old model that the current model needs to compare with, we avoid the additional cost in storage and computation that entails.
Replay-based methods, also known as rehearsal-based methods, have replay buffer that stores data from previous tasks, which can be used with current task data train the model (Rebuffi al., 2017: Chaudhry et al., 2019; Buzzega et al., 2020; Prabhu et al., 2020; Cha et al., 2021; Chen et al., 2023). Compared to rchearsal-fred methods, the shortcomings of replay include the need for a large buffer to reach reasonable accuracy. and the privacy and availability issues in storing past data. One variant of replay, sometimes called pseudo-replay learns generative model from past data, and samples data from it in lieu of the replay buffer (Ostapenko et al., 2019; van de Ven et al., 2020). Although they ameliorate the storage problem, the privacy concern persists because of the possibility that the generative model can reveal past data.
Parameter isolation methods dedicate different subsets of model parameters to different tasks (Rusu et al., 2016; Serrà et al., 2018; Ebrahimi et al., 2020; Wang al., 2020). Either the size of the model grows as more tasks come in, or less parameters will reserved for each task. This line of approach can be natural solutions for situations when the correct task is known beforchand, otherwise way to distinguish the tasks will be needed.
Rehearsal- Class-Incremental Learning. Inspired by works in parameter-effictee fine- tuning (PEFT) with large pre-trained models (Lester et al., 2021; Hu et al., 2022; Li & Liang, 2021),
2

---

Published as a conference paper at ICLR 2024
prompt-based methods (Wang et al., 2022c;b; Smith et al., 2023a) combine PEFT with parameter isolation for rehearsal CIL. fixed pre-trained ViT encoder is used along with a pool of small learnable prompts, where subsets of the pool are reserved for individual tasks, in the fashion of pa- rameter isolation. These prompt-based methods have shown to achieve SOTA performance, as the good generalizability provided by the pre-trained model mitigate the need for rehearsal. One recent work called S-Prompt (Wang et al., 2022a) also generally follows the aforementioned prompt-based methods, but is designed for the
scenario, instead of CIL.
On top of prompt-based methods, there are also recent works (Zhou et al., 2023; Ma et al., 2023) exploring the usage of prototype classifier. Their classifier heads consists of prototypes for each class, and the class whose prototype is closest to the encoding vector of a given input will be the prediction. They leverage fixed pre-trained models similar to prompt-based methods, and can also achieve comparable performance. Two proposed methods under this direction that are contempora- neous to our work are SLCA (Zhang et al., 2023), which stores one Gaussian distribution for each task instead of class encoding mean, and RanPAC (McDonnell et al., 2023), which employs random projections and a Gram matrix for the prototype classifier.
flass-Incrementa Learning with Task Prediction. As discussed before, one main bottleneck for parameter isolation methods to solve CIL is to identify the correct task for the input. There are works that classify tasks explicitly (von Oswald et al., 2020; Rajasegaran et al., 2020; Abati et al., 2020: Henning al., 2021), some training separate network, while others use entropy for the prediction. Kim et al. (2022) further establishes the connection between task prediction and outlier detection, showing that training outlier detectors can help class-incremental learning. However, as has separate detectors for each task, it requires replay buffer to calibrate them. For these methods that try to predict the task, they either require rehearsal, or have inferior performance.
PREREQUISITES
3.1 CLASS-INCREMENTAL LEARNING
In the CIL setting of classification, dataset D (x,, M)) where æ; E RT and y, D. is split into T' tasks, each with disjoint sets of classes, and usually each task contains the same number of classes (C2). Thus, the training and test set for each task are denoted as D Dirain Dr Di t E (1. T) For a classification model, it consists of a feature encoder, fo and a classifier head, h which are parameterized by 0 and Φ, respectively. Then for a given input æ, the feature encoder projects it into the feature space Id to produce the feature vector 2, then the head produces the prediction scores ý corresponding to all classes given x, and the class with the highest score becomes the prediction:
2 = fa(x) E Rd, ÿ= (=)) E ((CxTT)
(1)
We use superscripts & to indicate the model trained after task t, such that the randomly initialized model before training is represented by f4o). and (for, for) are the model after updating (for-1,ø-- on the data of task f. Let A denote the learning algorithm (e.g., stochastic gradient descent) that updates the model, such that for each task, given the model trained after task t 1, the training data Dr Di and training loss L (Cxx) x x - RR, the CII. can be formulated as:
(0', Φ0) A(( Drc train: L),
where Di Dirmin for rehearsal-free methods, i.e., only new data alone is available during the training; on the other hand, for rehearsal-based methods, Dr Dirain can be any subset of the union of all encountered data till now: Dj Di C U Dirnin. We use OlΦ and folhs interchangably to denote the feature encoder and classifier head, respectively.
3.2 PROMPT-BASED CLASS-INCREMENTAL LEARNING
In this section we outline recent prompt-based CIL methods, in particular L2P (Wang et al., 2022c), DualPrompt (Wang al., 2022b), and CODA-P (Smith et al., 2023a).
3

---

Published as a conference paper at ICLR 2024
Given the fixed pre-trained ViT encoder feo, classifier head hg, and the prompt pool contain- ing pairs of keys and prompts P = {(km: Pm)
E {1..... M)}, where km Pm e R. Let F be an algorithm that composes the appropriate prompt á E R2 given query function q and prompt pool P, such that = F(g(z), P), e.g., in DualPrompt, q(.)
(-) and F : p = P: arg ma (1,", .M ((=)"<; After the p is composed, it will be inserted into the en- coder foe along with the input and we denote it as for, p(x)= R (See Section 4.2 in Wang et al. (2022b) for the different types of prompt insertion.).
For prompt-based models, only the prompts P and the parameters of head Φ are updated, while the weights of encoder part 0 stays unchanged throughout the process. Similar to θ and Φ, we use p to denote the updated prompt pool after the t-th task. Then, use Fpi as shorthand of F(), such that ý = Fp (q())). Additionally, in practice the head can be decomposed based on the number of tasks, and each output score for classes that belongs a particular task. We differentiate them by underscripts, such that Ø = {Ø1.... ... Φr). Following Wang et al. (2022b), each Φ; will only be trained on task t, in order avoid recency bias. Thus, the prediction of the classifier head can be expressed as follow:
ý halfor [hap (fov (2)); hhp (foo,p()))] where ha. RC:
(3)
Now in the context of prompt-base rehearsal-free CIL, Equation 2 becomes:
(0), Pt) =Al(Joo, - Fpi- D L).
(4)
In the above learning process only P and of are updated, where 0 and the remaining parts of Φ are not, hence they are not present in the output of A. To conclude, aforementioned prompt-based methods follow the process below:
8' = F()o
ÿ -= hhooJoon (x)),
(5)
4 METHOD
For rehearsal-free CIL., one common problem is that the classifer head could easily confuse a data sample from one task with other tasks since no data in different tasks trained together dif- ferentiate them. We called this phenomenon "inter-task confusion". Fig. (top row) describes an example that even though each classifier trained in each task could achieve high accuracy per task; however, lacking data between tasks results in confidence scores from the classifiers of each task that cannot be directly compared. Even so, during the inference, the model usually picks the class with max score its prediction. To this end, we proposed model-agnostic regularization method on the classifier head to reduce the inter-task confusion.
4.1 VIRTUAL OUTLIER REGULARIZATION
In rehearsal- based CIL, since there are buffers to save data from other tasks, we could use them to train the model with the current data to avoid for inter-task confusion; however, it is impossible for rehearsal-free CIL. Therefore, we propose generating virtual outliers (c.g., NPOS (Tao et al., 2023)) to further regularize the decision boundary formed by the classifier head. Figure I (bottom row) illustrates that, with outliers, the decision boundary for each class becomes more compact such that each classifier head only generates higher scores for the samples whose distribution is close to the training data; therefore, the overall performance of CIL. are improved due to the lower inter-task confusion.
Thus, for each task, we generate a set of virtual outliers, u ~ Duttier e E Rdd, where Da G(for æ (Piran)) )), where G is outlier generator, and use with the feature vectors of current training data foo.pe p (Dir to regularize the classifier head. We adopt two hinge loss terms used in Liu et al. (2020) for custrf-disttibution detection as regularization to enforce the classifier head (()) to produce higher confidence of the samples in current task via Tuurent while lower the
4

---

Published as a conference paper at ICLR 2024
Class
Class
Decision Boundary
Boundary
Without
1.0
Score cluss for 0.65
Score class for 0.70
0.5
With Outlier
0.0
Scare of class Tor 0.75
Score of class 3 for 0.35
(a)
(b)
(c)
Figure I: An example of inter-task confusion under rehearsal-free CIL (2-task CIL, each task with classes); class I and 2 are in task 1 while class 3 and 4 are in task 2. Each row shows the illustration without and with the proposed regularization. (a) feature vectors for data from all classes, and outliers marked by crosses in the bottom
(b) and (c) are the decision boundary for class 1 and 3, and the associated scores (We omit class 2 and 4 for simplicity). Given a sample æ belonging to class 1, without regularization, during the inference, is classified class the score of class higher than class since there are no other task data to further confine the decision boundary to avoid inter-task confusion; while, with regularization, the decision boundary become more compact such that the classifier only generates the high scores for those samples that are in the distribution of training data; thus, this time, it gives x higher score for class and correctly classifies it.
confidence of the virtual outliers through TOutlier
Loutlier =Rren [Cs(max(O, Eccurente (x, () TCurrent))
+ EuwDooatee [max(0, Toutller Eoutlier (u, t))]],
where Ecurrent (xx, 1) - log E
Eoutlier (v, = - log
(6)
and ÿ Nip (Soo (2r)) C. "0" = (v) E RCC,
where Ls(-) is the Huber loss (Huber, 1964). Note that square-error (MSE) loss to provide more stable training for better convergence. Note that the outliers only used to regularize the classifier head has instead of training separate outlier detectors in existing outlier synthesis works (Du et al., 2022: Tao et al., 2023). In CIL setting, all tasks are sepa- rate, thereby requiring detector per task; thus, it will be difficult design an appropriate way choose the correct task with these detectors, since all of them are trained locally to each individual task. By not having separate detectors, we eliminate the need for introducing and searching for good set of respective hyperparameters.
the Huber loss instead of the mean-
In this work, adopt NPOS (Tao et al., 2023, Section 3) as virtual outlier synthesis and briefly mention the steps here-for for more details, please refer to Appendix D. It first selects the boundary samples as those among input samples with the largest distances with respect the rest of the input samples. Then, the candidates of virtual outliers are generated by adding randomly sampled noises from Gaussian distribution NO, I2),, to the selected boundary samples. After that, for each candidate its distance to the input samples are calculated, and only those with the largest distances are chosen as outliers.
The proposed virtual outlier regularization (VOR) is agnostic to any prompt-based CIL, e.g., CODA- (Smith et al., 2023a), DualPrompt (Wang et al., 2022b) and L2P (Wang et al., 2022c). next section, we further explore efficient prompt-based CIL in terms of FLOPs and parameters, named OnePrompt, which only use single prompt through the whole CIL, process rather than using a complicated mechanism to compose prompt for different tasks.
5

---

Published as a conference paper at ICLR 2024
Table Accuracy difference between the first and final evaluation of test data each task (I.e. right after each task and task 10) for OnePrompt on 10-task ImageNet-R, given the correct task ID. There almost no drop, implying little significant representation shift after the prompts are updated.
Task
Task 2
Task 3
Task 4
Task 5
Task 6
Task 7
Task 8
Task
+2.05
+1.64
+1.34
+0.40
+0.61
+0.24
+0.23
-0.04
+0.36
4.2 ONEPROMPT AS PROMPTING METHOD
Despite prompt-based CIL achieved superior performance, the common design is to compose differ- ent prompt P for different tasks to learn different feature representation because the feature charac- teristics could vary task by task (Smith al., 2023a; Wang al., 2022b;c). Moreover, those methods usually require a query function over the input to compose the prompt since the task index is not given during the index. This bring two drawbacks: first, the query function is usually as complex as the pre-trained model; second, as the classes are randomly shuffled during the training, there is no exact semantic meaning for each task, unlike domain incremental learning whose each task is related to one specific domain. Thus, we first revisit the prompt- based method with a single prompt through the CII,, which means = {pi). í = Pt, and no need for F since always outputs the same prompt. We name this method as OnePrompt since only one prompt is used.
Surprisingly, we found that without composing prompts for different tasks, OnePrompt already out- performs L2P and DualPrompt on ImageNet-R with 10-task setting by 4.0% and 1.8% on average accuracy, respectively. Note that, OnePrompt provides ~ 2x inference speedup because there is no need to compose the prompt from the prompt pool; moreover, it also contains fewer additional parameters to train.
Even though that OnePrompt has delivered good performance. further investigate whether or not it has the common problem of representation drift when the same set of prompt is continuously updated through all tasks. To validate that, evaluate OnePrompt the correct task index given, so that the factor of inter-task confusion can be eliminated. Table shows performance difference after first and last evaluation, for test data of each task. If there is significant drift in the representation, the corresponding accuracy should be dropping. As one see, there is almost no degradation, implying little representation drift for OnePrompt. We conducted more analyses to show the representation drift is marginal in Appendix E. We conjecture that since the learnable parameters with OnePrompt only a small fraction (~0.3%) the learned feature representation won't 't drift too much along with the tasks (Smith et al., 2023b).
10-task ImageNet-R, with
4.3 PROPOSED ONEPROMPT WITH VIRTUAL OUTLIER REGULARIZATIO
We proposed OnePrompt with VOR (OVOR) as our prompt-based CIL such that it balances the performance came from OnePrompt and VOR and the efficiency delivered by OnePrompt. Figure illustrates the overview of training procedure of our method. First, for each task t, the model is
Input
Feature Vectors
Classifier
Images
Head ltg.
LeB
Insert Promps [+
Update
(a)
Input Images
6'
Peature Vectors
Outleer Generator C
Outliers
Feature
h&
LCE LLouatio
Insert
Outliers
p
Update
(b)
(c)
Figure 2: Overview of the training process of proposed OVOR. For each task and the associated training data, three steps are performed to update the model: (a) first, we train the prompt Ø and the head his with cross-entropy loss, similar to previous methods, where the next steps are the novel parts in OVOR. After model trained as in (a), the prompt is also frozen as in (b), and all feature vectors of inputs of the current task computed. These feature vectors are used to generate virtual outlier samples, and in (c) the two sets vectors are used to update the classifier head hgg with both the cross-entropy loss and the proposed regularization loss.

---

Published as a conference paper at ICLR 2024
Algorithm OVOR: OnePrompt with Virtual Outler Regularization
Input: Pre-trained ViT føe, Initialized prompt p", classifier head hav, Training set Derain Learning algorithm A, Loss function LCE, Loutlier, Outlier generator G. Scalar À
for each task t E {1..... do do (øi, - A((or ,-,,, A(,for,p-,,,ii, (0-,,,0;-1)) D Piraius LCE) D Outlier +- {g(fom .5 (x)) E Dirainl
Fig.2 (a)
Fig.2 (b)
Φ + A((for p'. hwt)) (D D Duttier); LCE + XLoutlier)
Fig.2 (c)
end for
trained with standard cross-entropy loss learn good feature representation, obtaining updated p
and has: second, generate virtual outliers Outlier from the features for (x), x E Prrain' third, use the virtual outliers along with the training data to update the head hdg with the combination of cross-entropy loss and the outlier regularization loss. Algorithm also details the training steps of our proposed OVOR.
5 EXPERIMENTAL RESULTS
Datasets. For evaluation, we use four widely- datasets, ImageNet-R (Hendrycks et al., 2021a) and CIFAR-100 (Krizhevsky et al., 2009), ImageNet-A (Hendrycks et al., 2021b), CUB-200 (Wah al., 2011). Among them, CIFAR-100 has 100 classes in total, and other three have 200 classes. Following Wang et al. (2022b), we split the datasets randomly into a number of non-overlapping sets, one for each task, each with the same number of classes. Similar Smith al. (2023a), we report numbers for all methods on 5, 10, 20 tasks CIL setting on ImageNet-R, and 10-task CIL setting on CIFAR-100. We test on 10-task setting on CUB-200 and ImageNet-A, following Zhou et al. (2023), also using their train/test split for ImageNet-A. For all experiments, we run 5 times with different random seeds, for the metrics to be described below. We report the mean of 5 runs in Table 2-3, and include the version with standard deviation Table A.5-A.6 in the appendix.
Metries. We use two metrics to measure the performance the methods, including average accuracy (Ar), average forgetting (Fr). The forgetting metric quantifies the drop in accuracy for task over time, then averaged over all tasks. For average accuracy and forgetting, we follow the definition in Wang et al. (2022b):
1 1
R=1
A,
S
F. P=
max (Sp.i-St.) - St,i) (E,
where St, is the evaluation accuracy on Diest for the model after training on task i. Note that average accuracy is more important than average forgetting as the Ar already encompasses both plasticity and forgetting, with Fr only providing additional context, advocated by Smith et al. (2023a). Moreover, we found each paper use different way to compute the average forgetting, so we only report the average forgetting for the methods we reproduced
Training Details. For experimental configurations of CIL training, we follow Smith et al. (2023a) much possible, using the Adam (Kingma & Ba, 2015) optimizer, and batch size of 128. The model is trained 50 epochs per task on ImageNet-R and 20 for other three datasets. The last 20% of epochs is used for outlir-regularizedd training when applicable. Similar to Smith et al. (2023a), we adopt the vision transformer-base (ViT-B) checkpoint from Steiner et al. (2022) that pre-trained on ImageNet-21K then fine-tuned on ImageNet- IK (Russakovsky et al., 2015), and a cosine-decayiin leamingg schedule with the initial learning rate of le 33 is used.
For the regularization loss, we set 0.1, same as Liu al. (2020). We use TCurrent -24.0 and TOutlier = -- 3.0 for the two threshold hyperparameters in the loss term, and 8 = 1.0 as in the Huber loss Lg, which a component of the regularization loss. During the generation of virtual outliers, C samples are selected as boundary samples from the inputs, and it outputs 3 C. outliers,
Despite that Smith et al. (2023a) and Smith al. (2023b) also reported average forgetting, they were calculating it in different way than what they reference to.
7

---

Published as a conference paper at ICLR 2024
Table 2: Results on 10-task ImageNet-R, CIFAR-100, CUB-200, ImageNet-A setting, The number in the parenthesis denotes the result with VOR.
ImageNet
CIFAR
CUB
ImageNet-A
Method
Ar
Fr
Ar
Fr
Ay
Fr
Ar
Fy.
1.2P
(71.14)
7:27 (7.10)
(83.87)
(8.71)
70.62 (76.13)
(9.38)
8.86
DualPrompt
71.47 (73.27)
6.02 (6.38)
83,28 (84.85)
5.75 (7.50)
71.85 (78
(8.23)
44,85 (45.45)
10,80 (8.82)
CODA.P
74.87 (76.82)
6.73 (8.05)
(86.53)
6.39 (9.17)
72.74
10.37 (8.73)
50.20 (51.04)11.4)
(7.78)
OnePrompt
(75.61)
(5.77)
85.60 (86.68)
(5.25)
72.19 (78.12)
10.18 (8.73)
47.36 (48.49)
9.75 (7.83)
Table 3: Results ImageNet-R (INR), for 5, 10 and 20 tasks, and CIFAR-100, CUB-200,
ImageNet-A for 10 tasks.
Dataset Method
INR-5
INR-10
INR-20
CIFAR-100
CUB-200
ImageNet-A
Ar
Fr
Ar
Fr
Ar
Fr
Ar
Fr
Ar
Fr
AT
Fr
L.2
76.06
A-Prompt
72.82
87.87
SimpleCIL
61.28
61,28
61.28
76.21
85.16
49.44
ADAM
74.27
72.87
70.47
85.75
85.67
53,98
L,PP
71.00
6.43
69.31
7.27
65.68
8.30
82.43
6.18
70.62
10.90
40.16
8.86
DualPrompt
73.11
5.66
71.47
6.02
67.91
6.51
83.28
5.75
71.85
10.31
44.85
10.80
CODA-P
76.29
5.92
74.87
6.73
71.70
6.62
86.41
6.39
72.74
10.37
50.20
8.91
OnePrompt
74.65
4.60
73.34
5.36
70.42
5.57
85.60
4.79
72.19
10.18
47.36
9.75
OnePrompt- Deep
75.45
5,27
73.92
4.88
71.35
5.46
85.82
5.55
71.56
10.44
46.53
10.48
OVOR
76.79
4.88
75.61
5.77
73.13
6.06
86.68
5.25
78.12
8.73
48.49
7.83
OVOR-Deep 76.82
7.58
76.11
7.16
73.85
6.80
85.99
6.42
78.29
8.40
47.19
7.78
where C is the number of classes per task, and use K-NN to approximate the distance. We use 1.0, 10, 160, 100 for a, a, 3, and K, respectively. These are the configuration for ImageNet-R, R, for detailed hyperparameters settings for all datasets, please refer to Table C.17 and Table C.18 in the appendix.
For OnePrompt, adopt prefix tuning to insert prompt, and the prompt is inserted into the first five layers of the ViT model, the prompt length is (5, 5, 20, 20, 20) respectively for the five layers, according to Wang al. (2022b). Additionally, we test with inserting all layers with same length of 5, same as VPT-Deep in Jia et al. (2022), and we denote it as OnePrompt-Deep. Similarly, OVOR- Deep being OnePrompt-Deep with VOR. We reproduced the results of the compared methods of L2P (Wang et al., 2022c), DualPrompt (Wang et al., 2022b) and CODA-P (Smith et al., 2023a). Please refer Appendix C for our implementation details.
5.1 EFFECTIVENESS OF VIRTUAL OUTLIER REGULARIZATION
We experiment with four different prompt-based methods: L2P (Wang et al., 2022c), Dual- Prompt (Wang et al., 2022b), CODA-P (Smith et al., 2023a), and OnePrompt, on the aforementioned datasets under the CIL setting. First, in Table 2, demonstrate the ability of VOR to improve per- formance for different prompt-based methods. shows that across all four datasets, all four methods improve their accuracy when they are accompanied by VOR.
One can notice that whether trained with VOR or not, the four methods are ranked the same in performance. Besides that OnePrompt consistently outperforms DualPrompt and L2P, it is also no- table that DualPrompt beats L.P. As DualPrompt uses both task-specific and task-agnostic prompts, and L2P has no equivalent shared prompts. this trend further validates our reasoning behind the designing of OnePrompt, as described in Section 4.2.
5.2 COMPARISON TO SOTA REHRARSAL- CIL.
For more extensive evaluation results, we present the experimental results on all settings of ImageNet-R, CIFAR-100, CUB-200 and ImageNet-A. For comparison, also include the results from several recent prototype-classific-
methods: SimpleCIL (Zhou et al., 2023), ADAM (Zhou et al., 2023), and A-Prompt (Ma et al., 2023). Additionally, for L2 (Smith et al., 2023b), which just fine-tunes all attention layers in the pre-trained ViT with L2 regularization. For these works, simply quote their numbers from the papers.
Table 3 shows results on all datasets. Among results for ImageNet-R, one can observe similar trend as in Table 2, where among the four prsmpt-based methods, CODA-P always performs the best, fol-
8

---

Published as a conference paper at ICLR 2024
lowed by OnePrompt, DualPrompt, and then L2P. Nonetheless, VOR can improve OnePrompt such that it can outperform CODA-P. OnePrompt-Deep performs better than OnePrompt, and this remains the case when VOR is added, with OVOR-Deep being the best all three settings. Prototype- classifier-hased A-Prompt and ADAM usually beat I.2P and DualPrompt, while underperforming CODA-P and OVOR. Generally, number of tasks increases, accuracy decreases, forgetting in- creases, and the improvement from VOR grows, For the 10-task setting, OVOR-Deep can beat L2, which fine-tunes the entire pre-trained model, updating far more parameters than prompt-based methods (OnePrompt updates less 0.3% of total parameters).
As for results on CIFAR- 100, CUB-200, and ImageNet-A. On these datasets, the prototype-base methods beat prompt-base based methods, with the largest margin on CUB-200. Apart from that, One- Prompt generally performs better than OnePrompt-Deeg on these datasets. Among the prompt-based methods, they exhibit similar ordering to that on ImageNet-R, and that VOR still always improves the accuracy for them.
The trend of the four prompt-based methods that present in Table 2, indeed corroborates the design of OnePrompt. For CODA-P which beats OnePrompt, as it uses an attention layer to produce weights for combining all prompts in the pool, it resembles adding an additional transformer self-attention layer, explaining its edge over OnePrompt. Nonetheless, the edge of CODA-P does come with cost: As indicated in Table 4, CODA-P has much more learnable parameters than other prompt-base based methods. From Table one can also see that OnePrompt only spends half inference cost as others, with only fraction of number of learnable parameters.
Table 4: Number of learnable parameters and FLOPs (and the ratio to OVOR) for the ImageNet- 10-task setting.
Learnable Parameters (M)
Method
FLOPs (G)
L2P
35.18 (1.99x)
0.64 (2.44x)
DualPrompt CODA-P
35.19 (1.99x)
1.11 (4.26x)
35.17 (1.99x)
3,99 (13,28x)
OVOR
17.60 (1.00x)
0.26 (1.00x)
Beside the main results in the paper, we perform ablation on following hyperparameters to validate our design choices: regularization weight X, epochs used by regularized training, Huber or MSE loss in regularization loss, thresholds TCurreut, TOutlier in the two hinge losses, and NPOS hyperpa- rameters (a, a, B. and K), well where the prompts are inserted into the model. Their results are detailed in Appendix B, Tables B.7-B.16 16, respectively.
6 CONCLUSION
We introduce novel way to regularize classifiers with synthetic outliers, for rehearsal-free class- incremental learning. Our method is shown to be compatible with different prompt- based methods, improving their performance across benchmarks. Nevertheless, currently OVOR only applicable to scenarios where the input spaces of tasks are non-overlapping, that there is no obvious way to apply it to domain-incremennal learning or blurred boundary continual learning (Buzzega et al., 2020). The outlier generation process also requires storing and searching among feature vectors, which may bring scalibility issues when the size of dataset grows. We are aware of the recent line of work which uses prototypes in lieu of the classifier head, and it would be interesting future direction of our work to try different ways combining the two techniques. For instance, leveraging virtual outliers in caleulating the prototypes, instead of just using class means. Finally, as OVOR is agnostic to the modality of the input, it would also be worthwhile to explore its performance on textual, tabular, time-series data.
Disclaimer This paper was prepared for informational purposes by the Global Technology Ap- plied Research center of JPMorgam Chase Co. This paper is not product of the Research De- partment of JPMorgan Chase & Co. or its affiliates. Neither JPMorgan Chase & Co. nor any of its affiliates makes any explicit or implied representation or warranty and none of them accept any lia- bility in connection with this paper, including, without limitation, with respect to the completeness, accuracy, or reliability of the information contained herein and the potential legal, compliance, tax, accounting effects thereof. This document is not intended as investment research investment advice, or as a recommendation, offer, or solicitation for the purchase or sale of any security, finan- cial instrument, financial product or service, to be used in any way for evaluating the merits of participating in any transaction.

---

Published as a conference paper at ICLR 2024
Ethics statement. We are not aware of potential ethic concerns regarding the proposed OVOR algorithm.
Reproducibility statement. Our efforts facilitate reproducibility are summarized below:
We describe the datasets we used in Section 5.
We document the details and hyperparameters for reproducing prompt-based baselines in Section 5 and Section C.
Our method is fully described in Section 4, with pseudo-code provided in Algorithm 1, and hyperparameters in Section 5 and Section C.
10

---

Published as a conference paper at ICLR 2024
REFERENCES
Davide Abati, Jakub Tomezak, Tijmen Blankevoort, Simone Calderara, Rita Cucchiara, and Babak Ehteshami Bejnordi. Conditional channel gated networks for task-aware continual learning. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pp. 3930-3939. Computer Vision Foundation IEEE, 2020. doi: 10.1109/CVPR422000222220 2tw.:2000000000000339999 URL http:: http:////uttttttt//////eeeeeee STTEe__vVR_2200//22
/ttpttppppp///pepcsssssssss/ssss thecvf. com/
/ttaa/baatiooiiiiinaaaaaaaaaaeea wtt.ioooan_hhaaeeeeeeeee Gate__aeeeeedddeee Soo__Tas-AAan Swak--wwaeeem Cont nua htwprreooootttaaaaaaannnnnnnnnoo
html.
Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In Proceedings of the European Confer- ence on Computer Vision (ECCV), September 2018.
Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calder- ara. Dark experience for general continual learning: strong. simple baseline. In Hugo Larochelle, Mare' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan- Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Con- ference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020. URL httpt::tp///oo.... ppocced neur. neurips ps. cttccc//////a httpp/aappp/200/// b7 2774ea2233909 9 607661 www.ff6cc4444c0 Aastte-bbbttttt...... html.
Hyuntak Cha, Jacho Lee, and Jinwoo Shin. Co²1: Contrastive continual learning. In 2021 IEEE/CVF International Conference Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10- 17, 2021, pp. 9496-9505, IEEE, 2021. doi: 10. D00110/9CCC 1009388. URL https: //doi. www.oogg...... 11 1109/ICCV48 9922 2021 00938.
Sungmin Cha, Hsiang Hsu, Taeback Hwang. Flavio P Calmon, and Taesup Moon. CPR: classifier- projection regularization for continual learning. In Proceedings of the International Conference on Learning Representations, 2020.
Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet Kumar Dokania, Philip H. S. Torr, and Mare' Aurelio Ranzato. Continual learning with tiny episodic memories. CoRR, abs/1992.. 2019. URL http::/arxiv.ot/ wwtp:///.rr/aa///////oo/ooosss/s wttpg//////./.../../0g0
Haoran Chen, Zuxuan Wu, Xintong Han, Menglin Jia, and Yu-Gang Jiang. Promptfusion: De- coupling stability and plasticity for continual learning. CoRR, abs/2303.07223, 2023. doi: 00.48550/arXiv.2333.7777 077233.
/ooi. org/10. 1t1//1..85555 48550/ 2303. 07223.
Xuefeng Du Zhaoning Wang, Mu Cai, and Sharon Li. Towards unknown-aware learning with virtual outlier synthesis. In International Conference on Learning Representations, 2022. URL
https: / wwwpreveeeeeeeeeeeeeeee/eeeeeeee net/ het/forummideeeeeeeee Torum
Sayna Ebrahimi, Franziska Meier, Roberto Calandra, Trevor Darrell, and Mareus Rohrbach. Ad- versarial continual learning. In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (eds.), Computer Vision ECCV 2020 16th European Conference, Glasgow, UK, Au- gust 23-28, 2020, Proceedings, Part XI, volume 12356 of Lecture Notes in Computer Sci- ence, pp. 386 402.402. Springer, 2020. doi: 10. 0000077978-3030-880000 23. URL https: //doi. 10. 10t.//111.1109988 0300 00--0000558865 __23.
Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song, Jacob Steinhardt, and Justin Gilmer. The many faces of robustness: A critical analysis of out-of-distribution generalization. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 8320-8329, 2021a. doi: 10.1109/CCV4492222001.000 2..002..
Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Nat- ural adversarial examples. In IEEE Conference Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021, pp. 15262-15271. Computer Vision Foundation / IEEE, 2021b. doi: 10.1109CVPR4644.22211..085 nttps://openaccess thecvf. com/ wwt.//ooo///eeeeeeee http///ntetttttceeeeeeeee http://///E2221//////eeeee Natural AaarralAdversariia al Fxamples_cVPP
URL
htwp:///c/..e/eeecccook...... es CVVP_ paper html.

---

Published as a conference paper at ICLR 2024
Christian Henning, Maria R. Cervera, Francesco D' D'Angelo, Johannes Oswald, Regina Traber, Benjamin Ehret, Seijin Kobayashi, Benjamin F. Grewe, and João Sacramento. Posterior meta-replay for continual learning. In Mare' Aurelio Ranzato, Alina Beygelz- imer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), Advances in Neural Information Processing Systems 34: Annual Conference
Neural Informa-
tion Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, pp. 14135- 14149, 2021. URL https::/prroeeednngg weweeiinnnnnn.u.ss...eeee..e.iee cC/ wwtpc//pppeeeeee... (ppppe//c2//aaeeeeeeepppppeeaeee Absttratt.bbbrttt...... html.
Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Lifelong learning via pro- gressive distillation and retrospection. In Proceedings of the European Conference on Computer Vision (ECCV), September 2018.
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. The Tenth Inter- national Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. forum? www.ormmdddnneeeeeeeeeeo wew.iddeeeeeeeeeeeeeeee OpenReview.net, 2022. URL https: : / /twpn/evieneeeeee///e//epe net/ wwtttfrumm/eeeeeemmmo
Peter J. Huber. Robust estimation of location parameter. Annals of Mathematical Statistics, 35: 492-518, 1964.
Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge J. Belongie, Bharath Hariha- ran, and Ser-Nam Lim. Visual prompt tuning. In Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner (eds.), Computer Vision ECCV 2022 17th
European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XXXIII, vol- ume 13693 of Lecture Notes in Computer Science, pp. 709-727. Springer, 2022. doi: 10. 1001 978-3-031- 1-8-7--9922744.444 41. URL https:/ /doii www.do...oooi.oooooooe /10 1007/00000000 41.
031 1111-19880 19827-92---
Gyuhak Kim, Changnan Xiao, Tatsuya Konishi, Zixuan Ke, and Bing Liu, A theo- retical study solving continual learning. In S. Koyejo, S. Mohamed, A. Agar- wal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Pro- cessing Systems, volume 35, pp. 5065-5079. Curran Associates, Inc., 2022. URL neurips wwweeeeiiiinnnnnnniieeeeeeeeeeee 20f4 00F4 Papereee-PPPP-eeeeeeeeeeeeeeeeee Coneeeeeeoooeee
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings. 2015. URL http: / //arxivv.org abs / 1112. 6980.
James Kirkpatrick, Razvan Pascanu, Neil C. Rabinowitz, Joel Veness, Guillaume Desjardins, An- Demis
drei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Overcoming catastrophic for- getting in neural networks. Proceedings of the National Academy of Sciences, 114:3521 3526, 2016.
Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network representations revisited. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9- 15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pp. 3519-3529. PMLR, 2019. URL httt:://proceedingga mir /tt.ceeecinngnm...ppsss/////// kornblith1e 9a. html.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny im- 42099, 2009.
Matthias De Lange, Rahaf Aljundi, Mare Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Gregory G. Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. IEEE Trans. Pattern Anal. Mach. Intell., 44(7):336-3338 2022. doi: 1099/0MLT 2021.3057446. URL https: /ow. .oreiiorgg1000 / 1109 / TPAMT. 2021.3057446,
12

---

Published as a conference paper at ICLR 2024
Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. In Stari-Frraccin Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 I November, 2021, pp. 3045- 243.
3059. Association for Computational Linguistics, 2021. doi: 10. wttp3///1/2211eemlpmm-m URL https:////// 1tt../.........//11111111/
Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. In Chengqing Zong. Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 1ith International Joint Conference on Natural Language Processing, ACI/IJCNLP 2021, (Volume 1: Long Pa- pers), Virtual Event, August 1-6, 2021, pp. 4582-4597. Association for Computational Linguis- tics, 2021. doi: 10. 12.11886/1/22221 LacI-longg.353 URL https: //tp::///oooiie/ http:///t..rrgg....// 2021. acl- long. 353.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(12):9935-2477 2018. doi: 10. 10.110/TPAM2201:2233366
Weitang Liu, Xiaoyun Wang, John D. Owens, and Yixuan Li. Energy-based out-of- distribution detection. Hugo Larochelle, Mare' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hlsuan-Tier Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020. URL https: httpshttt///sooeeeeeee httt::////ccceceeeeeeeeee new.eeeeeedinnnnnnn............ neurips pccpppper/222//aat// Abstract Absttaaaaatttttttttttttt.... html.
Zhiheng Ma. Xiaopeng Hong, Beinan Liu, Yabin Wang, Pinyue Guo, and Huiyun Li. Remind of the past: Incremental learning with analogical prompts. CoRR, abs/2303. 2023. doi: 10.48550/arXiv.2303. 13898. URL 1tthstps: /ooi org/10. 18550/arXis 2303 13898.
Zheda Mai, Ruiwen Li, Jihwan Jeong, David Quispe, Hyunwoo Kim, and Scott Sanner. On- line continual learning image classification: An empirical survey. Neurocomputing, 469: 28-51, 2022. doi: 10. 10.1016/j wet.m...eeccom.221.1..... n.uccmmm 2021. 10. 021.
URL htttt:::/dcc..rr///1//
Michael McCloskey and Neal J. Cohen, Catastrophic interference in connectionist networks: The sequential learning problem. Psychology of Learning and Motivation, 24:109-16: 1989.
Mark McDonnell, Dong Gong, Amin Parvaneh, Ehsan Abbasnejad, and Anton van den Hengel. RanPAC: Random projections and pre-trained models for continual learning. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL httpt::tt//peeeeeee /opprrevie.rree...e net fort//orum?i id- aec58UfBZA.
Anh Mai Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, June 7-12, 2015, pp. 427-436. IEEE Computer Society, 2015. doi: 20.109/CVPR.2015.299666 URL https: doii http://oooooooo///////// /rc 1109/CVPR, 2015. 7298640.
Oleksiy Ostapenko, Mihai Marian Puscas, Tassilo Klein, Patrick Jähnichen, and Moin Nabi. Learning to remember: A synaptic plasticity driven framework for continual learning. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pp. 11321-11329, Computer Vision Foundation IEEE, 2019. doi: content
the.cc Lww.nanarnnnnne to Remember A A__Snaapti______ Plasticity_Driven http:y/pe.....eeeaaaaeeeo.ooooo. ot ftwp:re_ooooooooooaaaaaaaaaaaaa CVPR_ 201 __paperrp html.
10.1109/CVVR.2019..111 CVPR 11Fe__2099//mmm0staaaee aatpaaaeennoLLeeaanneee
URL Beraanniingttt_RRemeeeeeeeee
Ameya Prabhu, Philip S. Torr, and Puneet K. Dokania. Gdumb: A simple approach that questions our progress in continual learning. In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan- Michael Frahm (eds.). Computer Vision - ECCV 2020 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part II, volume 12347 of Lecture Notes in Computer Science, pp. 524-540. Springer, 2020. doi: 10.10077978-3 01188---000-80----3-33 31. URL https:
//doi. org 10. 1007 / 978- 00000030---00 5003-58536-5220
13

---

Published as a conference paper at ICLR 2024
Jathushan Rajasegaran, Salman H. Khan, Munawar Hayat, Fahad Shahbaz Khan, and Mubarak Shah. itaml: An incremental task-agnostic meta-learning approach. In 2020 IEEE/CVF Conference Computer Vision and Pattern Recognition, CVPR 2020, Sear- tle, WA, USA, June 13-19, 2020, pp. 13585-13594. Computer Vision Foundation IEEE, 2020. doi: 1.1109/CVPR4220020200000
URL https : /pwp//cccseccseseeeeeeeecc/sssse thecvf. iAMIL An In_Incermentall htmap....mml..................mm
2020 com/ httpe_cVEER_2//hmmmm/e/ea /html
Task T Agnostic_ Met Learning hewpaeaaeaaaapappccccccccocee
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H. Lampert. icarl: Incremental classifier and representation learning. In 2017 IEEE Conference on Computer Vision and Puttern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, pp. 5533-5542. IEEE Computer Society, 2017. doi: 00.1109/CVP.22017..88 URL https: doi org/10.
#109/CVPR.200
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhi- heng Huang, Andrej Karpathy, Aditya Khosla, Michael S. Bernstein, Alexander C. Berg, and Li Fei-Fei. Imagenet large scale visual recognition challenge. Int. J. Comput. Vis., 115(3): doi: 10.10077/1123--055-001666 s 111163-015-085 6-y.
211-252, 2015.
URL httpp:://cc..o.oooooooooo
Andrei A. Rusu, Neil C. Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. CoRR, abs/1600.04661, 2016. URL http: / /arxiv..... org/ httg/bs///////// abaa abs6 04671.
Joan Serrà, Didac Suris, Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention the task. In Jennifer G. Dy and Andreas Krause (eds.), Proceed- ings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmåssan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Re- search, pp.
PMLR, 2018. URL http: htap:://proceee http:/poocceeeeieeeeeeee mlr. www.s/.eessee....... /VC// serral8 8a. html.
James Seale Smith, Leonid Karlinsky, Vyshnavi Gutta, Paola Cascante-Bonilla, Donghyun Kim, Assaf Arbelle, Rameswar Panda, Rogerio Feris, and Zsolt Kira. Coda-prompt: Continual de- composed ceeention-baset prompting for rehearsal-free continual learning. In Proceedings the IEEÉ/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11909-11919, June 2023a.
James Seale Smith, Junjiao Tian, Shaunak Halbe, Yen-Chang Hsu, and Zsolt Kira. A closer look at rehearsal-fres continual learning * In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 2410-2420, 2023b. doi: 00.11099. 2023.00239.
Andreas Steiner, Alexander Kolesnikov, Xiaohua Zhai, Ross Wightman, Jakob Uszkoreit, and Lu- cas Beyer. How train your vit? data, augmentation, and regularization vision transform- ers. Trans. Mach. Learn. Res., 2022, 2022. URL https: httphhttt:///peeeee http::/://///eeeeee..... hwtp:wwwwwwweeeeeeeeeeee.em wtt.o//wwweeeeeettme//e foormmiii 4nPswr1KeP.
Leitian Tao, Xuefeng Du, Jerry Zhu, and Yixuan Li. Non-parametric outlier synthesis. In The Eleventh International Conference on Learning Representations, 2023. URL https:
// //oopneevvie.... net
Gido M. van de Ven, Hava T. Siegelmann, and Andreas Savas Tolias. Brain-inspired replay for continual learning with artificial neural networks. Nature Communications, 11, 2020.
Johannes von Oswald, Christian Henning, João Sacramento, and Benjamin F. Grewe. Contin- ual learning with hypernetworks. In 8th International Conference Learning Representa- tions, ICLR 2020, Addis Ababa, Ethiopia, April 26-30. 2020. OpenReview.net, 2020. URL https: openreview net
C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. The caltech-uesd birds-200- 200-2011 dataset. Technical Report CNS-TR-2011-0 1-001, Califomia Institute of Technology, 2011.
14

---

Published as a conference paper at ICLR 2024
Yabin Wang, Zhiwu Huang, and Xiaopeng Hong. S-prompts learning with pre-trained transformers:
An occam's razor for domain incremental learning. In NeurlPS,
2022a. URL http::/ 25886d7a7cf4en 7588cc777off3333447722220ooo
nips. co/paper fwwess/appe/eee//e/ee httpperr/22/hhhhh////
-a-.0---bbraatt--oofeeeeeeee html.
Zifeng Wang, Tong Jian, Kaushik R. Chowdhury, Yanzhi Wang, Jennifer G. Dy, and Stratis Ioannidis. hearn-prune-shaee for lifelong learning. Claudia Plant. Haixun Wang. Alfredo Cuzzocrea, Carlo Zaniolo, and Xindong Wu (eds.), 20th IEEE International Conference on Data Mining, ICDM 2020, Sorrento, Italy, November 17-20, 2020, pp. 641-650. IEEE, 2020. doi: /BP////CMM0000.200000000000 URL https: /oo. .oril.orggggoo 1100/ 2020. 00073.
ICDM501 000111099//0000111 108.
Zifeng Wang. Zizhao Zhang, Sayna Ebrahimi, Ruoxi Sun, Han Zhang, Chen- Yu Lee, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Dualprompt: Complementary prompting for rehearsal-free continual learning. In Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner (eds.), Computer Vision ECCV 2022, pp- 631-648, Cham, 2022b. Springer Nature Switzerland.
Zifeng Wang, Zizhao Zhang, Chen-Yo Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 139-149, 2022c. doi: 11099/CYPR2228..2222000000
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In Proceedings of the 34th International Conference Machine Learning Volume 70, ICML'17, pp. 3987-3995. JMLR.org. 2017.
Gengwei Zhang, Liyuan Wang, Guoliang Kang. Ling Chen, and Yunchao Wei. Slea: Slow learner with classifier alignment for continual learning on pre-trained model. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 19148-19158, October 2023.
Da- Wei Zhou, Han-Jia Ye, De-Chuan Zhan, and Ziwei Liu. Revisiting class-incremental learning with pre-trained models: Generalizability and adaptivity are all you need. CoRR, abs/2303.07333 2023. doi: 10.48550aaxXw.22333373333 URL https: / org/10.....00 4777/11.4850000 48550/ 2303.07335
arxiv.
15

---

Published as a conference paper at ICLR 2024
Table A.5: Results on ImageNet-R, for 5, 10 and 20 tasks.
# of Tasks
5
10
20
Method
Ar
Fr
Ar
FT
Ar
Fr
L2
76.06.10.65
A-Prompt
72.82:1:0.00
SimpleCIL
61.28
61.28
61.28
ADAM
74.27
72.87
70.47
L2P
71.00=0.229
6.43-10.3
10.44
7.27=0.60
65,6810.23
8.30
L2P-VOR
72.74=10.31
6.4410060
71.1410.32
7.10=0.73
67.57:10.40
8.52-10.77
DualPrompt
73.11:0.221
5.66100.2
71.4710.48
6.02=0.45
67.9110.32
6.51+0032
DualPrompt-VOrl VOR
74.92:10.22
5.4110.53
73.2710024
6.38-0061
70.0810.55
6,46-10.3
CODA-P
76.29-10.45
5.9210066
74.8710.28
6.7310.36
71.7011.29
6.62
CODA-P-VOR
77.72:10.26
8.0010.56
76.82.10.29
8.04910.50
74.00.10.42
7.17:00.8
OnePrompt
74.65+00.48
4.60=0.31
73.34+0077
5.36=0.53
70.42+0.08
0.57
OnePrompt-
75.45+0.11
5.27=0.34
73.92+0.53
4.88=0.47
71.35+00.54
5.46=00225
OVOR
76.79=0.31
4.88=0.69
75.61
5.77=0.53
73.13+0.35
6.06
EVOR-Deep
76.82:20.67
7.58=1.10
76.11
7.16=0.34
73.85+0.29
6.80=0.65
Table A.6: Results CIFAR-100, CUB200, ImageNet-A
Dataset Method
CIFAR-100
CUB200
ImageNet-A
Ar
FT
Ar
FT
Ar
Fr
A-Prompt
87.87+0.24
SimpleCII.
76.21
85.16
49.44
ADAM
85.75
85.67
$3.98
L2P
82.43+0.31
6.18+1.61
70.62:#0.85
10.90=2.73
40.16=2.007
8.86+2.94
L2P-VOR
83.87=0.61
8.71#1.19
7..33+:1227
9.38+3.00
41.27=4.07
8.79+2.23
DualPrompt
83.28+0.60
5.75+1.08
11.85+1.14
10.31=2.34
44.85+2.39
10.80+1.73
DualPrompt- VOR
84.85
7.50++1.26
7..44411.24
8.23+11.3
45.45:14.52
8.82414.09
CODA-P
86.41+0.43
6.39+11.27
72.7440.78
10.37+-2.12
50.20-H2.92
8.91+1.81
CODA-P-VOR
86.53110.48
9.17
78.72:1.199
8.73
51.04::.115
7.78
OnePrompt
85.60++0.41
4.79+1.25
72.19++0.8
10.18+22.30
47.36+2279
9.75++1.80
OnePrompt-
85.82+0.20
5.55+1.26
71.56+00.3
10.44+2.13
46.53++2.07
1..4
OVOR
86.68+00.26
5.25+1138
78.12+1.14
8.73+1.87
48.49+4.30
3.83+
OVOR-Deep
85.9910.889
6.42.12.03
78.2911.200
8.40.11.42
47.1914.229
3.78
APPENDIX
A COMPLETE EXperiMENTAL RESULTS WITH STANDARD DEVIATIONS
B RESULTS FOR ABLATION STUDY
Here in this section we present the ablation results, for the options introduced by our regularization method. All experiments are conducted with OVOR on 10-task ImageNet-R setting, and we vary the hyperparameters around their default configuration detailed in Section
Tables B.7-B.1 shows results for regularization weight X, number of epochs for regularized train- ing, the choice H function in the definition in the regularization loss, and thresholds TCurrent> TOutlier, correspondingly. The following Tables B. 1.11.B.15 shows those for NPOS hyperparameters a, a, 3, and K. In these tables, the values in bold are the ones used by the default configuration. Most of them are rather insensitive fluctuations, where the ones that can cause larger drops in ac- curacy are number of epochs and H. However, note that even those with the lowest accuracy are still outperforming the one without the regularization. The number of epochs used by regularized train- ing can affect the performance more significantly. it is too large, the prompt will not have enough time be fully-tuned, causing worse feature representations; if it is too small, the classifier head
16

---

Published as a conference paper at ICLR 2024
Algorithm 2 NPOS: Non-Parametric Outlier Synthesis
Input: Encoder fe Dataset D, Distance measure AK. Hyperparameters a, 3. C.N
D font & UZED fa.p(a)
D Doundary topkuEDoom (a C, A D
......,,, N. n4 (O, ol)} nosse + {n: i iE {1, ..,,..,
Dcandidase t u n u E D boundary: 78 E noise D Doutlier + t (B C, A (v, Pieat))
return Doutlier
will be under-regulated. For function H. the difference between Huber loss and MSE loss is how they respond to values far away from the target. Due to how the outlier generation algorithm NPOS works, it is possible for it to output samples that actually lie within the training samples. When such samples are used as outliers, they generate energy scores very far from the outlier threshold Toutler-- In this case, MSE will generate much larger loss value than Huber loss, it squares the distance in its calculation, being more sensitive to such errors.
B. ABLATION ON POSITION OF INSERTING PROMPT
As mentioned in Section 5, by default OnePrompt inserts length-5 prompts in first two layers, and length-20 prompts in subsequent three layers. Here report the results varying the insertion posi- tions, in Table B.16. The first row is the configuration for OnePrompt. As can be seen in the table, this configuration is not the best, but we it to match the configuration of DualPrompt, in order to make direct comparison. The second and third rows insert same-length prompts to all layers, which same as VPT-Deep, achieve best accuracy among all. We choose the second one, with length 5, for OnePrompt-Deep, because it uses less prompt parameters than OnePrompt.
C IMPLEMENTATION DETAILS
We build our implementation upon the official code repository of CODA-P (Smith et al., 2023a) ?, which also implements two other prompt-based methods: I.2P (Wang et al., 2022c) and DualPrompt (Wang et al., 2022b). We make the several modifications on it, implementing OnePrompt and VOR, and added code for better reproducibility when building dataloaders in parallel.
For all prompt-based methods, their pre-trained VIT-B checkpoints are provided by the timm li- brary 3, which in turn use the one trained by Steiner al. (2022) 4, Please see Table C.17 for complete configuration.
D ALGORITHM FOR GENERATING VIRTUAL OUTLIERS
Algorithm shows the NPOS algorithm used for generating virtual outliers, with being number of classes per task, and N sufficiently large integer (we used 600 as in (Tao et al., 2023)). NPOS defines distance measure A (æ, D) as the distance between the point æ and its K-nearest-neighhor in D, while topkæep(K,) works similarly to arg max but instead returns top K elements.
Prompt
hhttsss//ttorrap.ggoogeeppp com/vitt.t http:///oooooooooa//////////////
0. 1.000-aag_mediae wwt.:mediummm-e 0.1-d0.1.... 0. o.0-sd 0.0- - 20k 0.
01-res 11-res_224.npz
Table B.7: Ablation result on weight for regularization loss.
X
0.05
0.1
0.5
1.0
AT
75.23
75.61
75.61
75.58
17

---

Published as a conference paper at ICLR 2024
Table B.8: Ablation result on number of epochs using cegollarzze-ttrainnng
Last (%) of total epochs
10%
20 %
30%
AT
74.42
75.61
73.99
Table B.9: Ablation result on function H. in regularization loss calculation.
Huber
MSE
Ar 75.61
73.75
E ADDITIONAL ANALYSES ON REPRESENTATION DRIFT
In order to prove that OnePrompt does not suffer from representation drift even when its prompt is updated throughout all tasks, we calculate the drift in its prompt and the representation vectors of same inputs. Table E.19 shows the cosine similarity of the prompt parameters concatenated between consecutive tasks, e.g. prompt parameters before and after training on task 2 data have cosine similarity of 99.26%. According to it, change in prompt parameters of OnePrompt is marginal, and can explain why OnePrompt has minor representation drift. This conclusion is further reinforced by the results in Table E.20, where measure the similarity in representation of test samples, in similar way of Table 1. In addition to cosine similarity, we include the CKA similarity metrics (Kornblith et al., 2019) well.
Table 1010: Ablation result on energy score threshold TCurrent for training data samples.
TCurent
-21.0
-24.0
-27.0
AT
75.61
75.61
75.57
Table B.11: Ablation result energy score threshold TOutlier for outliers.
TOutlier
0.0
-3.0
-6.0
AT
75.38
75.61
75.70
Table 12: Ablation result on covariance parameter or in generating outliers.
0.1
0.5
1.0
2.0
AT
75.57
75.57
75.61
75.59
18

---

Published as a conference paper at ICLR 2024
Table B.13: Ablation result on a, which is used to calculate number of selected boundary samples.
7.5
10
12.5
15
Ar 75.60
75.58
75.61
74.58
74.46
Table 14: Ablation result on 8, which is used to calculate number of generated outliers.
3
100
120
140
160
180
200
AT
75.48
75.50
75.58
75.61
75.58
74.60
Table B.15: Ablation result on K in KNN distance, used in outlier generation.
K
50
100
150
AT
75.38
75.61
75.38
Table 1.16: Ablation result on prompt position, with OnePrompt on 10-task ImageNet-R. The left- most column is the start and end layer for length-5 prompts, and the middle column for length-20.
Length-5
Length-20
Accuracy
(1,2)
(3,5)
73.34
(1,12)
73.92
(1,12)
74.69
(3,4)
(3,5)
73.46
(5,6)
(3,5)
73.41
(6,7)
(3,5)
73.89
(1,2)
(2,4)
72.89
(1,2)
(4,6)
73.55
(1,2)
(5,7)
73.93
Table C.17: Detailed configuration for implementation.
Random seeds
0, 2, 23,224,22) 224)
Input image size
Input normalization
mean (0, 0,) std 1,) 50 on ImageNet-R, 20 CIFAR-100
Epochs
Batch size
128
Optimizer
Adam, 81 0.9, 32 0.999 1e
Learning rate
Schedule
Cosine decay
Number epochs for VOR
Last 20% of total epochs 8 = 1.0
Huber loss
Image Preprocessing
ImtgeNet-R (train)
RandomResizeedrrop2244 RandomHHorizontallii((....)) Normalize()
ImageNet-R (test)
Resize(256), CenterCrop(224), Normalize() RandommesszeedCrop2222 RandomtHonizonntllll1((.... Normalize() Resize(224), Normalize()
Other (train)
Other (test)
L2P
Prompt length: (5, 5, 5,5), Prompt pool size: 30
DualPrompt CODA-P
Prompt length: (5, 5, 20, 20, 20), Prompl pool size: one per task
Prompt length: (4,4, 4, 4, 4), Prompt pool size: 100
OncPrompt OnePrompt-Deep Prompt insertion method
Prompt length: (5, 5, 5, 20, 20), Prompt pool size:
Prompt length: (5, 5, 5, 5), Prompt pool size:
Seeix-Tuning (Li & Liang, 2021)
19

---

Published as a conference paper at ICLR 2024
Table C.18: VOR Hyperparameters for each dataset.
Dataset
TCurrent
TOutlier
K
ImageNet-R
0.1
-24.0
-3,0
1.0
10
160
100
CIFAR-I00
0.1
-24.0
-3.0
1.0
10
160
100
CUB-200
0.1
-24.0
-3.0
0.1
60
100
ImageNet-A
0.1
-15.0
-3.0
1.0
4
60
100
Table E.19: Drift in Prompt Parameters. For OnePrompt on 10-task ImageNet-R, we calculate the cosine similarity between the prompt before and after training of each task.
Task
Task
Task 4
Task 5
Task
Task
Task
Task 9
Task 10
99.26%
99.60%
99.53%
99.78%
99.81%
99.87%
99.82%
99.88%
99.89%
Table E.20: Drift in feature representations. For OnePrompt 10-task ImageNet-R. calculate the cosine and CKA similarity between the representations of test samples of a task before and after training that task.
Metric
Task
Task
Task
Task 5
Task 6
Task 7
Task 8
Task 9
Task 10
Cosine
88.28%
88.69 6.6
88.21%
89.30%
89.18%
89 89.02%
88.79 78.79%
88. 28.78%
89.00%
CKA (lincar)
91.63%
92. 90.
90.59%
93. 13.100
93.39%
92.72%
92.34%
92.68%
92.53%
CKA (kernel)
92.12%
93.22%
91.13%
93.65%
93.92%
93.36%
92.93%
93.25%
93.25%
20


Reviews:
[{"review_id": "kM8bN0ygDp", "reviewer": "Reviewer_2hZK", "summary": "This paper considers rehearsal-free class-incremental learning (CIL), and proposes the \"OnePrompt with Virtual Outlier Regularization\" technique (OVOR) that performs equally or better than existing prompt-based CIL methods in terms of accuracy and computation. The proposed method consists of two elements, 1) the \"OnePrompt\" - a single prompt based mechanism that gives comparable results to mechanisms with multiple-prompts, 2) an outlier regularization technique that reduces the inter-task fusion by tightening the decision boundaries of the classes among different tasks.", "questions": "- Is there any mathematical explanation/proof as to why the outlier method works well in tightening the decision boundaries, based on the Gaussian mechanism employed?\n- Is there any intuition as to how why the single prompt method is comparable to the multiple prompt cases in terms of the performance?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The proposed method for rehearsal-free CIL outperforms (or performs equally) existing methods in terms of accuracy, computation and storage.\n- The pipeline is clearly explained", "weaknesses": "- The functionality of the virtual outliers is not very clearly explained.\n- The paper lacks intuition on how the single prompt method works equally or better than the multiple prompt methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VGw4IVqz44", "reviewer": "Reviewer_Dx4f", "summary": "There are two primary motivations in this paper. The first one is to show that the need to have a pool of prompts (something that most prompt-based approaches in CL do) is not necessary. For this, the authors propose OnePrompt, which is a simple baseline that uses only one prompt to learn all tasks instead of having task-specific prompts. The second motivation is that rehearsal-free approaches struggle to differentiate classes from different tasks since they are never optimized simultaneously. For this, the authors propose regularizing the decision boundary using  Virtual Outliers. The authors mentioned that these outliers help to define the decision boundary of each class better. The results show promising results in two commonly used benchmarks. Surprisingly, OnePrompt does not present high forgetting.", "questions": "- It is not clear which setting OnePrompt works. Does it work in a Class incremental or task incremental setting (with a share classifier)?\n    - This confusion appears especially after seeing Table 1. Why is this table important? How does the same experiment behave in CIL?\n- Is there an intuition on how and why OnePrompt works so well? \n    - How much does the prompt change in each task? Since there is no forgetting, it shouldn't change much or no?\n    - How does the representation of previous tasks change after training new tasks?\n- Have you run experiments when training only the classifier? Together with using the approach mentioned in Eq 3.\n- Is there an explanation for why the VOR increases forgetting? As shown in Table 2.\n- In Section 5, you mentioned that for OnePrompt, you insert the prompts into the first 5 layers. Did you do an ablation of this hyperparameter? \n- How costly is the VOR approach?\n- Do you have the results for OnePrompt for Table 4? In the last 2 paragraphs, you mentioned a comparison between OnePrompt and other methods, however, I don't see it in the tables. Are you referring to OVOR?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 3, "strengths": "- The idea of having a OnePrompt method is interesting. And it surprised me how well it worked.\n    - I especially like the idea of reducing the computational cost by not requiring to find the closest key-pair values.\n- The results showed that the proposed regularization method helps improve the accuracy. The motivation is clear.\n- The paper shows promising results in 2 standard benchmarks. However, only state-of-the-art results in one.\n- The background section in a paper is very helpful since it presents the notation and everything it needs to know about the contributions", "weaknesses": "- I understand and share the motivation that rehearsal-free approaches struggle to distinguish classes from different tasks. However, it would be good to add a more realistic example, values or theory that support this statement.\n    - Or at least something that shows what you think is happening.\n- In multiple parts of the paper, it is mentioned that prompt-based approaches require a pool of task-specific prompts. Although I understand the idea behind it, this is only partially true, as the authors say in section 3.2. They explain that the selection depends on a function; ideally, one would expect this feature to be able to be separated by task, but it is not a requirement of the methods. \n- The structure of the paper needs to be clarified. For example, why present Virtual Outlier Regularization first rather than OnePrompt?\n    - At least for me, OnePrompt is like a simple baseline. It's a good one, as it shows in the results, but only a base method to be compared.\n    - I believe that adding Virtual Outlier to other methods is the main contribution. However, how to obtain the virtual outliers is not explained in the paper. I understand that citing is a proper way of explaining it. However, as it has been such an essential part of the contribution, it is better to have a small explanation.\n- In Table 3, it should be good to include OnePrompt for comparison. Also, add some vertical lines to separate # of tasks clearly.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Hf1ZpoUMjl", "reviewer": "Reviewer_DjrG", "summary": "This paper describes a method for class incremental learning (CIL) that makes use of frozen pre-trained ViT transformer networks. The essence of the method is to learn prompts, while keeping the pre-trained network frozen, and hence ensure it is unable to forget during CIL. Past work in this area, namely L2P, DualPrompt and CODA-P, is therefore highly relevant.  The paper identifies several weakness with those past methods for prompting ViT networks for CIL, and proposes improvements to the general methodology. Firstly, it proposes the use of \"VOR\", i.e. Virtual Outlier Regularization, in which the learned output head is trained further following freezing of newly learned prompts, on a combination of feature vectors, and synthesised outliers. The idea here is to try to tighten the decision boundaries between classes. Secondly, it proposes OnePrompt, a simplification to past prompting mechanisms, namely removing the use of different prompt sets for different tasks. When VOR and OnePrompt are combined (as OVOR), the paper shows better results than L2P and DualPrompt, with comparable or better results than CODA-P.  The number of parameters and FLOPs is reduced in OVOR compared to those methods.", "questions": "This is repeating the content of my response under \"Weaknesses\": my questions are these: \n\n1. What is the performance of VOR when combined with L2P, DualPrompt and CodaPrompt, and how does that compare with OVOR? \n2. What is the performance of OVOR, and ablations (VOR alone and OnePrompt alone) on at least 2 more datasets?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- Originality: the paper isolates weaknesses in comparison methods and proposes a solution, namely the VOR method, which is the first use of such a method in the prompting of ViT models for CIL. Moreover, the proposed simplification to past prompting methods for CIL is novel, as is the combination of VOR and OnePrompt\n- Quality: the paper sets out the context and problem setting well, clearly explains the rationale for the methods, and tests the method on datasets that are of clear interest in the CIL community.\n- Clarity: the writing is clear and the structure easy to follow; this is why I have very few questions.\n- Significance: Given prompting methods using pre-trained ViTs are very new in CIL, this is a potentially important paper. I think the community is currently unclear on whether prompting methods can be significantly improved on, and there is a need for studies such as this in which simplifications, parameter efficiency and additional regularization are used to enhance prompting methods.", "weaknesses": "- 1. The main weakness of the paper is that important comparison results in the use of pretrained ViT networks are omitted. The significance of the contribution in this paper would be much easier to appraise if these are added by the authors.\n    - 1.1 First, there are two omitted relevant papers that use alternative methods to prompting. These both produce higher final average accuracies using pre-trained ViT models than any reported here.  The first of these is the SLCA method [1]. The second is the RanPAC method [3] that builds on the work of the paper by Zhou et al [2], which is already cited in the paper.\n        - In the current paper, OVOR on ImageNet-R achieves 75.6% for T=10, and CIFAR-100 achieves 86.7% for T=10.\n        -  In ref [1], ImageNet-R achieves 77% for T=10 and CIFAR-100 achieves 91.53% for T=10. \n        - In ref [3], ImageNet-R achieves 78.1% for T=10 and for CIFAR100 achieves 92.2% for T=10 ( T=5 and T=20 results can also be seen in Table A4 of [3]).\n    - 1.2 Second, a clear distinction between refs [1,2,3] and the current paper is that [1,2,3] report results on many more datasets, whereas here only two are used: split CIFAR-100 and split ImageNet-R. \n        - The current paper's contribution and standing relative to refs [1,2,3] would be much clearer if results for OVOR are reported on at least 4 datasets. E.g. [1] also reports results on split CUB and split Cars, while [2] and [3] each report on 7 datasets for CIL in total. There is no good reason to not demonstrate OVOR's performance on at least 4, and preferable 6 datasets such as those in [1,2,3].  This is particularly important given that table 2 shows that OnePrompt alone produces lower final average accuracy than CODA-P, and only the combination of OnePrompt with VOR does better than CODA-P.\n    - Note: I am aware that paper [3] (and possibly [1]) might be deemed \"contemporaneous\" according to the FAQ on this point (which is unclear re arXiv). Therefore, my recommendations to compare to those are aimed at improving the value of the paper should it be published. However, my recommendation to add more datasets is independent of any date comparisons, especially given Zhou et al's other datasets could have been used to add more dataset comparisons, to help the reader understand whether there truly is a trend for the method to outperform coda-prompt, DualPrompt etc.\n\n- 2. The paper does not thoroughly explore  ablation results with regards to the combination of OnePrompt or other prompting methods with VOR.  While ablations for OnePrompt are shown for ImageNet-R in Table 2, there is no data for CIFAR-100.  My above comments re more datasets extends to this point - for the reader to understand the contributions of OnePrompt and VOR both individually and combined, we really need to see data for ablations on more than just a single dataset. Moreover, the authors state \"The proposed virtual outlier regularization (VOR) is agnostic to any prompt-based CIL.\" This therefore means it is of high interest to know how well VOR combines with CODA-P, given that CODA-P without VOR outperforms OnePrompt without VOR.\n\n- 3. On page 3, in the second paragraph, the authors refer to the prototype methods of Zhou et al and Ma et al and state \"However, it is unclear whether they can be considered rehearsal-free, as they claim to be.\"\n    - This is a misleading statement that should be deleted. Rehearsal methods require storage of samples from past tasks for inclusion in training in newer tasks.  In my view it's very well understood in the CL community that this means retaining a copy of the full original datasample, e.g. an input image, and using such samples individually alongside new samples to update the parameters of a model. This is really quite different to class prototypes as used by Zhou et al (and also, for example, Ref [3]). In Zhou et al, class prototypes are firstly certainly not the original samples, as they are formed from the output feature representations (e.g. a length 512 vector instead of an RGB image). More importantly, prototypes are an average over all the length 512 vectors for a class, so the result cannot in any way be thought of in the same way as \"rehearsal\", since there is no way to recover individual samples, and in this sense are more like the weights in a neural network than actual samples. Finally, and most important of all for this point, in Zhou et al  class prototypes from past tasks are not used to update any parameters in any subsequent task, which is in total contrast to the entire point of  rehearsal memory. In summary, I think its false to suggest that Zhou et al and related methods are not \"rehearsal free\".\n    - The authors then make a statement about privacy concerns - this is a totally independent point to the distinction between class prototypes and rehearsal memory, but I am skeptical about how much privacy concern there can be about a class-mean vector. I'm willing to stand corrected, but I cannot see any chance of original samples being extracted from a mean created from many orders of magnitude of samples to form a feature vector....\n    - The inclusion of these statements has the appearance of being an attempt to justify why a prompt based CIL method should be preferred over a prototype method like Zhou et al. But I don't think it is necessary to try to justify this. Regardless of whether prototype methods may or may not achieve better results than prompting methods, the current paper's advantages over past prompting methods are still potentially important, because (i) there may be benefits from combining prompting and prototype methods, which to my knowledge has not yet been done; and (ii) it could happen that future improvements on ViT transformer networks may benefit more from prompting than they do from prototype methods. \n\n- 5. Minor point: the paper's intro assumes the reader understands well that the approaches in the paper apply only to pre-trained ViT networks. For the larger Continual Learning community, this is still unusual as most methods for CL focus on training CNNs such as ResNets, and on mitigating forgetting when the benefits of a pre-trained model are not available. The paper would benefit from making these points clear early on.\n\n- Summary: I have chosen a rating of 5 at this time. In order to reconsider my rating, the authors will need to (i) provide the reader with a clearer view on how OVOR compares to other methods on more datasets  (and, ideally, comparison with refs [1,3] as well), and (ii) show thorough ablations compared with the best competing prompting method, CODA-P, i.e. CODA-P with and without VOR, and OnePrompt with and without VOR, all on at least 4 datasets.\n\n- Refs:\n    - [1] SLCA: https://arxiv.org/abs/2303.05118 (submitted to arXiv 9 March 2023, now published in ICCV 2023).\n    - [2] https://arxiv.org/abs/2303.07338 (cited in the paper as Zhou et al. (2023), submitted to arXiv 13 March 2023)\n    - [3] RANPAC: https://arxiv.org/abs/2307.02251 (submitted to arXiv 5 July 2023, now accepted in NeurIPS 2023).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)