S-Prompts Learning with Pre-trained Transformers: An Occam’s Razor for Domain Incremental Learning
Authors: Yabin Wang
Zhiwu Huang
Xiaopeng Hong
Venue: NeurIPS 2022 Accept (2022)
Keywords: Prompts Learning
Pre-trained Transformers
Occam's Razor
Domain Incremental Learning

Abstract: State-of-the-art deep neural networks are still struggling to address the catastrophic forgetting problem in continual learning. In this paper, we propose one simple paradigm (named as S-Prompting) and two concrete approaches to highly reduce the forgetting degree in one of the most typical continual learning scenarios, i.e., domain increment learning (DIL). The key idea of the paradigm is to learn prompts independently across domains with pre-trained transformers, avoiding the use of exemplars that commonly appear in conventional methods. This results in a win-win game where the prompting can achieve the best for each domain. The independent prompting across domains only requests one single cross-entropy loss for training and one simple K-NN operation as a domain identifier for inference. The learning paradigm derives an image prompt learning approach and a novel language-image prompt learning approach. Owning an excellent scalability (0.03% parameter increase per domain), the best of our approaches achieves a remarkable relative improvement (an average of about 30%) over the best of the state-of-the-art exemplar-free methods for three standard DIL tasks, and even surpasses the best of them relatively by about 6% in average when they use exemplars. Source code is available at https://github.com/iamwangyabin/S-Prompts.

S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learning
Yabin 2 Huang"! Xiaopeng Hong 3.4,1 Xi' an Jiaotong University, P. R. China Singapore Management University, Singapore 3 Harbin Institute of Technology, P. R. China
*Pengcheng Laboratory, R. China
lamaangyabinOstu xtt.:/aaabbnstux..u...u.. .du..edu.e. cn, zzhiwu. huang@gmail com, hongxiaopong@ieee.co
Abstract
State-of-the-art deep neural networks are still struggling to address the catastrophic forgetting problem in continual learning. In this paper, we propose one simple paradigm (named as S-Prompting) and two concrete approaches to highly reduce the forgetting degree in one of the most typical continual learning scenarios, i.c., domain increment learning (DIL). The key idea of the paradigm is to learn prompts independently across domains with pre- pre-trained transformers, avoiding the use of exemplars that commonly appear in conventional methods. This results in win-win game where the prompting can achieve the best for each domain. The independent prompting across domains only requests single cross-entropy loss for training and one simple K-NN operation as a domain identifier for inference. The learning paradigm derives image prompt learning approach and novel language-image prompt learning approach. Owning an excellent scalability (0.03% parameter increase per domain), the best of our approaches achieves remarkable relative improvement (an average of about 30%) over the best of the state-of-thee of-th--rrt exemplar- free methods for three standard DIL tasks, and even surpasses the best of them relatively by about 6% in average when they use exemplars. Source code is available at hitpp:: github........ com/ /ttpa/aaaaaaabn//--rarraoa
Introduction
State-of-ther of-the-rrt deep neural networks are capable of performing impressively wide variety of individual machine learning tasks. However, learning multiple tasks in sequence, generally called continual learning, remains substantial challenge for deep learning. When trained new task, standard neural networks typically forget most the knowledge related to previously learned tasks. This is well-known phenomenon named catastrophic forgetting.
There are three common continual learning scenarios. fask-incremental learning (TIL) is provided with task indexes for inference and thus using task-specific neural networks could overcome forgetting. Class- inss-incremental learning (CIL) increases classes in sequence with task indexes being unknown for inference. Nevertheless, the classes generally from the same domain, which more less decreases the challenge. this paper, we focus on exploring where classes keep the same but the involved domains commonly vary lot in sequence, with task indexes being not provided for inference. The larger domain gaps result in more severe forgetting.
learning (DIL),
'This work is done when Yabin Wang visits Singapore Management University I Corresponding Author *Corresponding Author
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Non-Prooapting
DyTox
Progosed S-Prompes
Classifies
Pretrained Fixed
Methods
Estractor
Extractor
Extractor
Extractor
Prompes
image Tokens
0000
0000
Previous traind & Now Fixed
Incremental Tuned/ Leamable
Streum
Session
Session
Sesson
Sesiion
Previous Methods
Propesed S- S-Prompls
Insoace ficnuree of clscs 11- Senciun vn
Space
Insaace floturr Seacir of Domain centers of
Eneremental Upáate (Dependent)
Incrementaal Insert (Independent)
Figure 1: Comparison of the proposed S S-Prompts paradigm against non- -rommpttn methods, and prompting methods (L2P [55], DyTox [14). Exiting methods generally leam sequential tasks/sesions/tomains dependently, producing single feature space where the classes from different domains are less scparable (more forgetting or worse transferring). In contrast, the proposed S- S-Prompts learn the tasks independently. This paradigm leads to one subspace for every domain, making the classes more separable (less/no forgetting and better transferring).
To alleviate catastrophic forgetting, numerous methods have been proposed for continual learning. One of the most effective methods is to leverage a buffer of exemplars from old tasks perform either rehearsal or distillation when tuning the whole network to learn new tasks (see Non- Non-Prompting Methods in T. Both the rehearsal and the distillation typically decrease the forgetting degree. However, it is often more desired that the exemplars of old tasks are not stored for better data security and privacy. Moreover, storing a large amount of exemplars might run out available memories. Therefore, our focus is shifted to the challenging exemplar-free DIL in this paper.
One of the most promising solutions the exemplar-free DIL problem is to learn set of prompt over transformers that achieve the state-of of-te-of-theearts in wide range of areas. In this solution, the Nomain-speeiff knowledge is stored by a prompt pool, and hence rehearsal buffer is no longer mandatory to mitigate forgetting. For instance, two recent prompting methods [14] 55 aim at learning tass/domain-specifee prompts dependently across domains in an incremental update fashion. In [14, the prompts are added task by task, and the newly added prompts share the same task- attention learning with the old prompts. This requests a joint tuning on all the prompts and the entire transformer using a distillation loss to keep a balance between the new prompting and the old ones. In (55), a pool of prompts is initialized at the right beginning, and a set of the prompts from the pool are selected for each instance. The instance-level prompts generally share task-level knowledge among any similar instances, by jointly tuning the selected prompts and the whole classifier, with the pre-trained transformer being fixed. While using different prompting approaches, [14] (55) both follow a commonly-respectee principle that the prompting should keep sharing across tasks/domains, However, the sharing-drive dependent prompt tuning paradigm well as many of the non- non-prompting paradigms generally result in a fug-Or fug-of-war (or a zero-sum game), in which one side's ' gain is equivalent to the other's loss, studied in [47 30 In other words, these paradigms generally accumulate new knowledge in the same feature space, which is very likely to mix up the subspaces of old/new knowledge resulting in less separable classes (more forgetting worse transferring) (Fig[I).
In this paper, we explore rule-breaking idea to instead play a win-win game, i.e., learning the prompts independently across domains so that the prompting can achieve the best for each domain. The excellent generalization capability of the transformers and the strong transfer learning ability of the recently appeared prompting mechanisms allow for the realization of this idea. In particular, we introduce a new learning paradigm that learns the prompts independently domain by domain, and incrementally inserts the learned prompts into pool. As shown in Fig T compared to the other prompting and non non-promp methods, the new paradigm suggests merely tuning the current domain-
'For simplicity and consistency, we use 'prompt' represent 'prompt token" for context/domain knowledge encoding, we utilize 'token' to denote those normal tokens on images and classes, throughout the paper. A concurrent work [12 learns class prompts dependently transformers for the CIL problem.

---

related prompts and classifiers with all the rest unrelated network components being fixed. The frozen pre- pre-trained transformer is capable of extracting general features, and the independent prompting reaches the best fit on each domain. The independent prompting is able to generate a feature space where each subspace is spanned by one single domain, and all the subspaces are less overlapped (see Feature Space in Fig D This could lead to no forgetting across domains. The learning process only requests the most naive cross-entropy loss for supervision. The inference is also simple and efficient. It simply uses K-NN to search for the nearest domain centers generated by K-Means on the training data to the features of the given test samples, followed by prepending the learned domain-associant prompts with the image tokens to the transformer for the final classification. As S Nomain--elated prompts will be finally learned independently, where S is the total number of domains, we name the proposed paradigm S- Prompts or S-P S-Prompting for simplicity throughout the paper. We hope the proposed S- S-Prompting becomes an Occam's 's Razor (i.e., a simple and elegant principle) for DII..
The S- S-Prompting paradigm can be applied to any pre-trained transformers. In this paper, we exploit two approaches of S-Prompts learning. One based on vision transformer (ViT) [1], and the other is based on Contrastive Language-Image Pre- Training transformer (CLIP) 45 Particularly, we suggest the CLIP-based approach that exploits a new language- image prompting scheme. The key idea is to prompt the pair of language-image transformers synchronously with a pair of learnable language-imag image prompts at the two ends. This highly enhances CLIP's transfer learning ability on a variety of new domains. As result, this approach owns a great ability to reach favorable feature space where the subspaces of different domains are pushed far away from each other and the classes are separated clearly, leading to highly impressive DIL performances. To the best of our knowledge, this is the first time to introduce the paired language- image prompting scheme in the literature.
The contributions of this paper can be summarized as follows:
We introduce a simple and effective S-Prompts learning paradigm with pre-trained transformers for the DIL problem. This paradigm breaks the
principle in continual learning. We exploit an Image S-Prompts (S-iPrompts) learning approach and novel Language-Image Image S- S-Prompts (S-liPrompts) learning approach, based on two different pre-trained transformers.
The best S-Prompts surpasses the best of the state-of- sitte-of-the-aae the-art exemplar-free methods significantly (an average of 30% relative improvement) for three standard DIL benchmark datasets, and even outperforms them relatively by an average of 6% when they use exemplars. S-Prompts can scale to large number of domains with a tiny parameter increase, e.g., 0.03% per domain in S-liPrompts.
2 Related Work
Continual Learning. Numerous methods have been exploited address catastrophic forgetting. For instance, memory-based methods alleviate forgetting by either replaying a small set of examples from past tasks saved in a memory 48
46 39
21 25
replaying synthesized data from previous
tasks using generative models 550
13 Distillation- based methods (e.g., [55] 46
20,
[3 [56] 58
[2]
37 apply the technique of knowledge distillation [99 to mitigate catastrophic forgetting. In
addition, there exist other kinds of methods like gradient-based methods (e.g., [47 [47660 60
16 49 51
54
regularization-basse ones (e.g. 22
61
and expansion-based ones (e.g. (77
23
Recently, some of work like 40
28
26 33
55 17 64] adapts the CIL methods to the DIL problems
such continual deepfake detection, where each domain distinguishes between deepfakes and reals. Following them, one of our approaches uses the common classifiers from the CIL methods. While each domain share the same classes, the incremental classifiers simply treat them as difference classes. The finer-grained task could make networks more powerful 44
Prompt Learning. The general idea of prompting is to learn function to modify the input texts or images, such that the language or image model obtains additional information about the task (55) There have been emerging bunch of prompting methods, such as to name a fe Nevertheless, these methods merely learn either image-end prompts or language-end prompts (at most using conditions from image-end), while ours explicitly learns joint language-image prompting. Recently, L2P (55) and Dy [14] exploit dependent prompting methods for continual learning, which leads to tug-of warr issue, discussed in Sec I Our suggested S- S-Prompting paradigm instead learns task/domain associated prompts independently for a win-win gain.
33
34 24
62
63 3 22 55 14]
FFor a comprehensive survey the prompting methods, please refer to 36
3

---

Imsage Tokena
[CLASS
Too Tokers
Lestraeeiieed brage Porppe
Darmin Cester
Learahee Teat Troepee
Frewnnt withees Trorepes
Prodiction Prebubility
Image Tmasforme:
Training image of sission s
Text Transfomerr
Bax
Bust
Cat
Van
Text Features
K-Means
Insert
[CLASS]
Image Prompt Pool
Feature Space
Language Prompt Pool
Figure 2: Illustration of the proposed language-image S-Prompts (S- -iiiirpPtts) learning approach. 8, 8 1, 8 denote the scssion/domain indexes. For the inference, we simply adopt the following stops: 1) feeding the tokens of a given test image into the image transformer to get the image feature, 2) using K-NN to search for the nearest centroid obtained by K-Means on training data to the feature of the given test image. 3) feeding the image token and the nearest centroid (domain center) related image prompt into the image transformer, with the centroid related language prompt and class tokens being fed into the text transformer, 4) computing prediction probabilities for the final classification. Note that both K-Means and K-NN are performed on the image feature space of the fixed pe-trrined image transformer without using any prompts.
language- -mmage
3 Proposed Approach
3.1 Problem Formulation and S-Prompts Learning
We focus on the problem of exemplar- frre domain incremental learning (DIL). In common and practical DIL scenario, the model is expected learn knowledge of various domains in sequence to finally become a universal expert for all the domains. Formally, at domain the model accesses to N. the incoming data ID, {af , uf J ;=1 where xf is the i-th input image from domain and yf is the corresponding label, and the total number of samples at domain s is Ng. In the entire incremental learning stream, S domains are presented sequentially, and each domain has data D3 from different or even highly heterogeneous domains (Fig D The set of class categories the model observed is C, and in DIL all categories are identified from the beginning and will not increase along the incremental learning stream. The exemplar-free DIL requests zero exemplar from the learned sessions during the training and the inference for better data security, privacy and cheaper memory consumption.
To address the exemplar-free DIL problem, we propose an S-Prompts learning paradigm that is simple and effective. The key idea is to learn prompts domain by domain independently with pre-trained transformers. The pre-trained transformers are fixed to extract highly expressive low-level features, and the prompts are tuned to encode the high-level domain knowledge. In this case, each domain knowledge will be encoded by one prompt or one set of prompts. This not only avoids the use of exemplars but also highly reduces the catastrophic forgetting. Accordingly, our focus is on learning the prompts of S domains or so-called S-Prompts. As shown in Fi the suggested S-Prompting merely tunes the current domain related prompts and classifier with all the rest neural network components, including the transformer, the current session unrelated prompts and classifiers, being fixed. In this case, each set of domain-specific prompts are learned independently from the other domains related prompts. This requests a domain identifier for inference. In particular, we apply K-Means store the centriods for each domain during training, and K-NN to search for the nearest centroid of the given test image feature to identify its corresponding domain during inference. Note that both K-Means and K-NN are performed on the feature space of the fixed pre-trained transformer without using any prompts. Finally, we prepend the domain associated prompts and the image tokens to the transformer for image classification. The inference steps are detailed in the caption of Fig 2 Since DIL's domain shifts are generally explicit and the state-of of-the-art transformers are commonly ly good at image classification, the suggested domain identifier performs well in standard DIL tasks. Due to the strong generalization capability of transformers, the suggested S- S-Prompting still performs well for those cases with incorrect domain identification or those on unseen domains. Thanks to this mechanism, the suggested S- S-Prompting can achieve the best transferring for each domain, as well as the least forgetting. Moreover, the proposed S-Prompting only requests the most naive cross-entropy loss for the guidance classification, which further enhances the paradigm's simplicity.

---

In this paper, we mainly study two concrete S-Prompting approaches: 1) Image S-Prompts (S- iPrompts) learning based on the pre-trained vision transformer (ViT) [3, and 2) Language-Imas Prompts (S-liPrompts) based on the pre-trained Contrastive Language-Imar Pre-Training transformer (CLIP) [4).. In essential, the main differences between these two approaches are two-folds: 1) prompt design, and 2) classifier design. We detail them below for these two approaches.
3.2 Image S-Prompts (S-iPrompts) Learning Approach
Prompt Design. In the approach of S-iPrompts, for one domain we suggest using independent set of continuous learnable parameters (i.e., one image prompt) P: E R RL.xD; as a part of inputs to the pre-trained VīT, where L; E IR and D; E IR indicate the prompt's length and embedding dimension respectively. As shown in Fig. 2 the whole embedding of any single image from domain s follows the format of pre-trained class tokens of ViT. The extended image embedding is used as an input of the transformer blocks as normal ViT does. When trained
Fing: P Letsl where Timg denotes the image tokens and Fels represents the
new domain 1, a new independent set of images prompts P is added. Hence, learning all the domains sequentially results in a pool of domain-wise prompt pool. The image prompt pool can be defined as P {Pi. Pi..... } where P E LR a single set of prompts belonging to domain s, and the total number of domains.
Classifier Design. The classifier of S-iPrompts is a normal fully connected (FC) layer, which is made of linear projection parameterized by [W,, b.). where W. E RRCD, by E RC, Di, C indicate the embedding dimension and the total class number respectively, and s is the domain index. Formally, the classifier output calculated by
1 o + ba),
where f(x) represents the output image feature of the ViT encoder f. and is the softmax activation operation, a22) Σt eth for
1.2...., (C indicates the number of classes). Every learning session owns such one classifier, and thus we use a classifier pool ple ([W,, bi), [22 b2), ..., [Ws, s; bs]] } to store all these classifiers that are independent to each other. During the inference, once the domain center is selected, the associated prompts and the corresponding classifier will be chosen for the prediction on each given test image.
3.3 Language-Image S-Prompts (S-liPrompts) Learning Approach
The S-liPrompts approach aims at exploiting an effective prompting scheme on CL.PP 45 so as to transfer the pre-trained CLIP to variety of domains. As illustrated in Fig the CLIP model consists of two transformers for image and text encoding respectively. The CLIP model was trained on an enormous amount of image-text pairs in contrastive learning manner, which aligns the pair of language-image transformers very tightly. Hence, it is essential to prompt the pair of CLIP transformers synchronously for effective transfer learning. To this end, we follow the paradigm of S-Prompts to exploit a joint language-image prompting scheme. The scheme learns a pair an image- end prompt and a language-ent prompt domain by domain, with the pre-trained CLIP always being fixed. On one hand, fixing CLIP enables the prompting model to inherit the extraordinary capability of CLIP in terms of zero-shot learning and few-shot learning [62]. On the other hand, the joint prompting can make the image-end transformer has an excellent fit into any upcoming domains that might be highly different from the pre-training data. This is mainly because synchronously prompting the language-end transformer could get more semantic information and more nomain-aware (Fig 4
Prompt Design. The design of image-end prompts for S-liPrompts is the same as that of S-iPrompts For the design of language-end prompts, we suggest using a learnable context prompt to replace the artificially designed prompts used by the original CLIP. This design can better adapt pre-trained vision- language models to downstream tasks, especially for few-shot learning. In particular, for the learning domain
M learnable vectors where L,, Dj are the prompt's length and embedding dimension respectively. We regard it as text prompts corresponding to image prompts used in VIT. For the j-th class, the whole input t, of the text encoder is of the format t; {P', ). where ej is the word embeddings of the j-th class names (CLASS) and has the same dimension of D,. The text prompts are also associated with one single domain, and they are independent to those from the other domains. After learning all the sequential domains, pool of text prompts is formed as p {Pl, P/ PL where S is the domain number.
the context prompt P! {U1, U2 ... UMs) I <
5

---

The learnable context prompt can better adopt the pre-trained CLIP to various domains, which is an essential property for incremental learning.
Classifier Design. The text encoder g of CLIP receives the input i, defined above, and produces vectorized representation g(tz), where j = ...,..., C, and C denotes the class number. The contrastive language-image pre-training of CLIP uses a contrastive loss to learn joint embedding space for vision and language models. One of its objectives is maximize the cosine similarity of each image with its matched text and minimize the cosine similarities with all other unmatched texts. The other objective is computed in a similar fashion for each text. Let f(x be the image feature generated by the image encoder f. and {g(t;)} is a set of weight vectors produced by the text encoder g(),, each g(t,) representing the text feature of j-class. The prediction probability is computed as
exp((f(x),
p(usla) Ei=1 exp((f(#)
where (.,-) is the cosine similarity, and (ar, v)
z-y We conduct the softmax activation operation
| over the result of to obtain the final logits for image classification.
Like S-iPrompts, S-liPrompts also grows the pool of domain domain-based classifiers (Fig Compared with the FC-based classifiers in S-iPrompts, there are two main advantages using the CLIPP-bsed classifier in S-liPrompts. Firstly, the language model provides more domain information using the domain-speciffe text prompts for DIL, because it generally leads to more domain-separable features (see Fj. Secondly, the incremental process, only one domain context needs to be stored for each session, while the class names are shared, which highly reduces the memory consumption.
Experiments
Datasets. We perform experiments on three standard DIL benchmark datasets. They are CDDB [33 CORe50 (38), and DomainNet [43). CDDB (33] is a dataset for continual deepfake detection, which a challenging DIL task. It designs Easy, Long, and Hard tracks. We select the most challenging track (i.e., the Hard track). The track requests learning on 5 sequential deepfake detection domains (about 27,000 images), which are GauGAN, BigGAN, WildDeepfake, WhichFaceReal, and SAN respectively. CORe50 (38 is a widely used dataset for continual object recognition. It has 50 categories from distinct domains. The continual learning setting uses domains (120,000 images) for incremental training and the data from the rest (i.e., unseen) domains as test set. DomainNet 43 a dataset for domain adaptation and DII., which has 345 categories and roughly 600,000 images. Images from DomainNet is split into 6 domains. The DIL setup on DomainNet is the same that of CasSLe [1]. We follow [33] to report the average forward detection accuracy and the average forgetting degree EDDB-Hard For CORe50 and DomainNet, we follow (55] and [17] to present the average forward classification accuracy.
Methods. We compare the proposed S- S-Prompts methods (S (S-iPrompts, S S-liPrompts) against the state-
of-the-art CIL/DIL methods. They include non-prompiin methods EWC (29). LwF [35] (8]
ER
GDumb 44
BiC (56), DER++ [4] and Co2L [6]. as well as the prompting methods DyTox [4
and 1.2P
55
and self-supervised learning method CaSSLe [17]. Among them, since EWC
29
LwF
(55) CaSSLe [17] can work for examplar- free DII., we regard them as our real competitors. For DyTox and I.2P, we use their official implementations by tuning their parameters for better performances. To compare fairly,
35
1.2P
use the same ViT model (i.e.. VIT-B/16 [3] for most of the real competitors as well as our Seroomtts/S-iiProppp The proposed S-liPrompts' 's text-end encoder is the same as that of CLIP [45] For more implementations details and studies, please refer to Appendix.
Overheads. For each domain, our proposed S-Prompting methods additionally use set of prompts as well as a set of centroids from K-Means. For S-liPrompts, the overhead in memory is merely of +0.03% per domain. Compared to the other prompting methods DyTox and 1.2P, S-liPrompts enjoys much less memory overhead of the growing pool of domain-based classifiers due to its CLIP based classifier design (i.e., extremely cheap prompts and shared class names) when the class number
7 'mmon the competitors, only Dy Tox 14 uses more advanced VIT model(Con 15) since it fails to get promising results (e.g.. g.. the result is even inferior than that of the random guess CDDB) using ViT-B/Ie [13].

---

Table 1: Results on CDDB-Hard for deepfake DIL. Bold: best exemplar-fres DIL results, Underline: second best exemplar-free DIL results. Upper-bound: supervised finetuning on the i.id. data of all tasks, which is usually regarded as the upper bound performance a method can achieve [5] denotes results copied from 66
Method
Buffer size
Average Ace (1)
Forgetting (1)
LRCHL
76,39*
-4.39*
iCaRL
100/class
79.76*
-8.73*
LUCIR
20
82.53*
-5.34*
LRCIL 42
74.01
-8.62*
iCaRL 40
73.98
-14.50*
LUCIR 20
50/class
80.77
-7.85*
DyTox 14
86.21
-1.55
EWC 29
50.59
-42.62
LwF 35
60.94
-13.53
DyTox 14
51.27
-45.85
0/class
L,P [55
61.28
-9,23
S-iPrompts (ours)
74.51
-1.30
S-liPrompts (ours)
88.65
-0.69
Upper-bound -rorppts
85.50
Upper-bound (S- (S-lirrompts)
91.91
Accrrayy:
100
90
80
70
BigGAN
WidDeeptlik
WhichFaesReal
EWC-VIT
DyTox/0
S-LiPrompt
S-iPrompts
Upper- hound---ee-i
Upper Epper-Sound-lnn plnrppp
Figure 3: Accuracy of the competing methods and the proposed S-Prompts CDDB-Hard.
is large (e.g., 345 classes in DomainNet). Accordingly. the proposed methods own an excellent scalability, On the other hand, S-liPrompts takes 11.84 millisecond (ms) in average classify an image on a single RTX-3090, while the pre-trained CLIP model takes an average of 9.37ms. This shows that the additional domain identification merely takes around 2.5ms, which affordable for the real-world scenarios. The appendix presents more details about the overheads.
Results. The results in Table T Table 2 and Table demonstrate that the proposed S-iPrompts and S-liPrompts significantly outperform the other exemplar- free methods including the two recent prompting methods I.2P and Dy'Tx. We compute that the proposed S-liPrompts obtains considerable relative improvement (an average of roughly 30%) over the best of these methods in terms of forward classification accuracy. From Table I and Fig 3 could also find the proposed S- S-Prompting methods' forgetting degrees are much less than those of the others. On the other hand, compared those methods using exemplars of old tasks, our S-liPrompting still enjoys the clear superiority (an average of about 6% relative improvement) over the best of them. Besides, it is fairly surprising that our S-liPrompts even surpasses its upper-bous bound (joint training version) on DomainNet that contains highly heterogeneous domains. This is very likely that the joint learning cannot handle the large domain shifts well, and our independent prompting avoids this issue. Another interesting phenomena is that our S- S-Prompts still enjoy the extraordinary generalization capability (both S-iPrompts and S-liPrompts clearly surpass others) on the unseen domains of CORe50 where all the tested domains do not appear in the incremental training. The inferior performance of 1.2P on DomianNet might be due to the less transfer learning capability of its dependent instance-level prompting (prompt sharing) on heterogeneous domains. While DyTox reaches closer to the proposed S-liPrompts on DomainNet, it requests large memory overhead (i.e., 30x345=17,250 where 50, 345 are the exemplar number per/class and the class number respectively) for the storage of exemplarg
Ablations. The studies on the cases fixing either FC length/weights or Prompt length/weights in Table justifies the necessity of using the proposed growing classifier pool and independent
DDy Tox 14 often fails without using exemplars, it requests distillation exemplars for better prompting.
7

---

Table 2: Results on CORe50 for object DIL. Bold: best exemplar-free DIL results, Underline: second best exemplar-free DIL results. Upper-bound: supervised finetuning on the i..d. data of all tasks, which is usually regarded as the upper bound performance a mcthod can achieve [55]. indicates results copied from [55.
Method
Buffer size
Average Ace (1)
ER g
80.10
GDumb
74.92
BiC 56
19.28
DER++ 4
50/class
79.70 00
Co 6
79.75 0:084
DyTox [4
79.21 10.10
L2P 55
81.07=00.13*
EWC 22
74.82=00.0*
LwF [35
75.45=00100
1,2P [55]
O/class
78.33
S-iPrompts (ours)
13.13
S- S-liPrompts (ours)
89.06
Upper-bound (S-iPrompts)
84.01 +0.53
Upper-bound (S-liPrompts)
13.1
Table 3: Results (task- agnostic test results) DomainNet for object DIL. Bold: best exemplar-free DIL results, Underline: second best exemplar- DIL results. T: the total buffer size is 50x345==7,250, where 345 is the total number of DomainNet classes. It is large memory overhead. Upper-bound: supervised finetuning on the i.i.d. data of all tasks, which is usually regarded as the upper bound performance a method can achieve [55] indicates results copied from 17
Method
Buffer size
Averuge Ace (1)
DyTox 14
50/class!
62.94
EWC
29
47.62
LwF
49.19
SimCLR [OS-Ca
44.2
BYOL [I8-CaaSLLe
17
49.7
Barlow Twins [59] CaSSLe
17
48.9*
Supervised Contrastive 27
CaSSLe
17
O/class
50.9
L2P (55)
40.15
S-iPrompts (ours)
50.62
S-liPrompts (ours)
67.78
Upper-bound (--iPrompt)) Upper-bound (S liPrompts)
63.22
64.08
image/language prompting for S-iPromps/S-liProone are randomly selected for the inference. It is surprising that the accuracy (80.12%) does not drop a lot and is still better than those of the state-of-the-ar methods. In fact, the domain identification accuracies are not high using either ViT-based or CLIP-based features, as shown in Table 55 Both of these two results show that the proposed S- P-romptings are not fundamentally influenced by the domain identification accuracy. As shown in Fig 4 the intrinsic reason might be that the proposed S S-iPrompts and S-liPrompts actually result in a clearer domain separation compared to the pre-trained ViT and CLIP, and also they inherit the excellent generalization capabilities of the used transformers. This is justified by the result of S-liPrompts (K = 5 & Zero-shot) that is merely trained on the first domain and tested on all the domains. Through the study on different K values K-Means, we find that the proposed S-liPrompts works very stably for different K values. Hence, we consistently use K = for all the experiments on CDDB- Hardd CORc50 and DomainNet.
We also study the case that the domains
Apart from the number of centers K for K-Means (studied in the Table there are three other key hyper-parameters for the proposed S-Prompts, including the length of an image prompt L,, the length of a language prompt L, (only for S-liPrompts), and the value of K in K-NN selection algorithm. We perform further ablations on the proposed S-liPrompts by keeping one of these parammerrreeeers varied and fixing the rest hyper- seper-pprameters on CDDB-HoL Hard. 5 (Top Left) illustrates the accuracy curve of using different image prompt lengths. We can see that too small L; has negative impact, which means that the prompts have not much ability to transfer the pre-trained model to downstream tasks. However, too large I, can not help the model to get better performance. 5ig55 (Top Right) illustrates the accuracy curve of using different language prompt length. Compared with the study
8

---

Table 4: Results ablating components of proposed two S-Prompis (i.e.. S-IPrompts, S-liPrompts) methods
on CDDB-Har for exemplar-fre deepfake DIL. Different K values for K-Means.
Method
Average Acc (t)
Forgetting (1)
S S-iPrompts (fixed FC length)
52.07
28.89
S-iPrompts (fxed FC weights)
72.13
-0.86
S-iPrompts (K' = & 7ero-shot)
62.08
S S-iPrompts (final, K = 5)
74.51
-1.30
S-liPrompts (fixed Language Image Prompt length)
64.90
-12.7
S-liPrompts (fixed Language Prompt length)
52.80
-29.13
S-liPrompts (fixed Language Prompt weights)
72.29
-0,99
S S-liPrompts (K 5 & Zero-shot)
66.57
S-liPrompts (K( 5 & Domain Random Selection)
80.12
-1.04
S S-liPrompts (K 1)
88.72
-1.09
S S-liPrompts (K 3)
88.55
-0.77
S-liPrompts (K
88.40
-0.82
S-liPrompts (K -9)
88.32
-0.88
S S-liPrompts (final, K =
88.65
-0.69
GauGANioul
Big0ANical
SANtee
SANDSO
Presobed
Prppodd SiProepe
Proposed Phoroees
L-SNE visualization on resulting feature spaces of pre-trained VIT. proposed S-iPrompts, pre- trained CLIP and proposed S S-liPrompts on CDDB-Hard.
Figure 4:
Table 5: Domain identification accuracy (lask-wise) when K = 5 for K-Means and K 1 for K-NN on CDDB-HHard
GauGAN
BigGAN
WildDeepfake
WhichFaceReal
SAN
Average Ace (f)
ViT
0.78
0.98
0.98
0.56
0.82
CLIP
0.74
0.68
0.97
0.97
0.66
0.80
A
A
RX 00
85.65
86.
$4.40 42
83.65
10
15
20 Lengh
12
Logh
Accrercy
Armmmay
65.05
90.10
$7.50
$6.22
85.16
80.00 00
53.20
$4.00
Neighter Number
Number
Figure 5: Domain incremental learning accuracies the proposed S-liPrompts (Top Left) using different image prompt lengths with the language prompt length being fixed as 16, (Top Right) using different language prompt lengths with the image prompt length being fixed as 10, (Bottom Left) using different K values for K-NN based domain identification, and (Bottom Right) domain identification accuracies (task-agnostic) of the cases with different values for K-NN, the CDDB-Hard dataset.
on image prompts, language prompts also favor a sufficient length (the longest one is 16 in our case). Fig (Bottom Left) studies the influence of using different K values in K-NN the domain incremental learning accuracy. From the results, we can find that 1-NN works clearly better than the other cases. The main reason is that the other K-NN cases result in clear performance drop for domain identification (Fig (Bottom Right)). Therefore, we suggest using 1-NN that is the simplest and the most effective for the proposed S-Prompts.
TIL vs. DIL. Table shows the results of the proposed S-liPrompts the scenarios of task- learning (DIL). TIL offers domain indexes for
incremental learning (TIL.) and the inference, while DII. does not provide the information. We report task-wir wise average accuracy (AA) that computes the average on all the task-based accuracies, and also present task-agnostic

---

Table 6: Accuracies (%) of the proposed S-liPrompts the setups task inckeeeareetlll learning (TIL) and domain-incremental learning (DIL) on CDDB-Hard. TIL provides domain indexes for the inference phase, while DIL does not offer domain indexes for the test. AA: average accuracy. Task-wise AA: average on all the task- accuracies. Task-agnostic AA: average all the test images without considering the task indexes.
GauGAN BigGAN WildDeepfake WhichFaceRcal
SAN
Task-wise AA Task-agnostic AA
TIL DIL
99.85
99.50
82.21
96.30
85.56
92.72
92.51
99.30
96.75
82.06
96.25
68.89
88.65
91.54
Table 7: Accuracies (%) of the application of the trained S-liPrompts on S1-S5 to out-of-domains OOD1- 0OD3 that are 3 unseen domains used in CDDB. S1-S5: GauGAN, BigGAN, WildDeepfake, RhichFaceReal. SAN, which are used to train S-liPrompts on CDDB-Hard. OOD1-O0D3: FaceForensic+*, Glow, StarGAN, which are not used to trained on S-liPrompts. AA: average accuracy. Task-wise AA: average on all the task-
based accuracies. Task-agnostic AA; average all the test images without considering the task indexes. 1
CDDB Hard runner-up method that uses 50 exemplars per class, while utilizes ZERO exemplar.
SI
S2
S3
S4
S5
OODI
00D2
00D3
Task-wise AA Task-agnostic AA
ST (ours)
99.85
87.75
49.15
50.50
50.00
50.92
63.52
71.10
65.35
68,43
S2 (ours)
99.90
98.88
51.24
52.50
50.00
74.68
73.42
80.06
72.59
76.56
S3 (ours)
99,90
98.75
82.16
$2.25
51.11
73.29
66.83
78.15
75.31
77.89
S4 (ours)
99.85
98.50
82.06
96.25
52.22
75.60
64.58
92.60
82.71
82.78
85 (ours)
99.30
96.75
82.06
96.25
68.89
75.69
64.44
92.46
84.48
82.63
S5 (DyTox
[44)
97,90
96.12
83.23
94,50
62.22
81.42
54.83
55.77
78.25
69.02
AA that computes the average on all the test images without considering the task indexes. Therefore, the task-agnostic AA is actually more in line with the real-world settings. The task aass-agnostic AAs show that S-liPrompts almost closes the gap between DIL and TIL.
Generalization. We test the generalization capability of the proposed S-liPrompts on unseen domains i.e., out-of-domain data (OOD). In detail, we apply the pre-trained S-liPrompts on domains to OOD data, which are FaceForensic*+, Glow, StarGAN used CDDB 33 Table [ reports the results of the OOD experiment. The accuracies on the three OODs increase clearly as S-liPrompts is trained on more seen data (SI-S5). We report the accuracies in both task-wise and task-agnostic cases, The results show that our S-liPrompts has better generalization ability to these OODs than DyTox 14, which is the runner-up on CDDB-Hard.
5
5 Discussion and Conclusion
Continual learning studies the problem of learning from data stream from changing domains/tasks. The general principle of conventional methods is to adapt to new domains/tasks by accumulating previously acquired knowledge, while protecting previous learning from being crased. Beyond this commonly-respecter principle, this paper proposes a rule-breaking paradigm, which is simple but effective, for the domain incremental learning problem. In particular, the proposed paradigm performs incremental learning without using any expert knowledge from previously learned domains except for general knowledge from pre-trained transformers. Instead, it merely learns the expert knowledge independently domain by domain, through simply prompting the state-of-the-art transformers. The independent knowledge learning mechanism avoids the traditional tug-of-warr among different learn- ing sessions and instead plays a win-win game to achieve the best for each learning session. The learned expert knowledge for each domain is finally gathered in a pool for general inference on varying domains. Our comprehensive empirical study shows that the proposed learning paradigm enjoys groundbreaking gain (about 30% relative improvement) over the state-of-the-arr art competitors as well as a great scalability to a large number of incrementally appearing domains.
There remain limitations on the suggested paradigm. For example, we do not obtain appealing results when applying directly to the incremetalt learning (CIL) tasks. This is because these tasks are more sensitive to the incorrect domain/task identification. Nevertheless, the proposed paradigm still has a high potential to inspire a wide range of research areas. For instance, the suggested prompting can be applied directly to any transfer learning problems, as it essentially learns to prompt the transformers to any upcoming new domains. In addition, it provides an inspiration to zero-shot and few-shot learning, since the proposed S- S-liPrompts shows superior generalization capability on unseen domains i.c., out-of-domain data (OOD).
Acknowledgments. This work is funded by the Singapore Ministry of Education (MOE) Academic Research Fund (AcRF) Tier 1 grant IMSS2CC002) This work is also supported by the National Natural Science Foundation of China (62076195) and the Fundamental Research Funds for the Central Universities (AUGA5710011522).
10

---

References
[1] Afra Feyza Akyünek, Ekin Akyürek, Derry Wijaya, and Jacob Andreas. Subspace regularizers for few-shot class incremental learning. In ICLR, 2022.
[2] Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In ECCV, 2018.
[3] Hyojin Bahng, Ali Jahanian, Swami Sankaranarayanan, and Phillip Isola. Visual prompting: Modifying pixel space to adapt pre-trained models. arXiv preprint 2rXiv:2203.17274 2022.
[4] Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: strong, simple baseline. In NeurlPS, 2020.
[5] Francisco M Castro, Manuel J Marin-Jiméned Nicolás Guil, Cordelia Schmid, and Karteek Alahari. End-to-end incremental learning. ECCV. 2018.
[6] Hyuntak Cha, Jacho Lee, and Jinwoo Shin. Co21: Contrastive continual learning. In ICCV, 2021.
[7] Arslan Chaudhry, Mare' Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with a-gem. arXiv preprint arXiv:1812.00422 2018.
[8] Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny. Thalaiyasingam Ajanthan, Puneet Dokania, Philip HS Torr, and Mare Aurelio Ranzato. On tiny episodic memories in continual learning. arXiv preprint arXiv: /902. 10486, 2019.
[9] Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny. Thalaiyasingam Ajanthan, Puneet Kumar Dokania, Philip HS Torr, and Marc' Aurelio Ranzato. Continual learning with tiny episodic memories. arXiv preprint arXiv: /902. 10486, 2019.
[10] Ting Chen, Simon Komblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations, In ICML, 2020.
[11] Ali Cheraghian, Shafin Rahman, Pengfei Fang, Soumava Kumar Roy, Lars Petersson, and Mehrtash Harandi. Semantic-aware knowledge distillation for few-shot class-incremental learning. In CVPR, 2021.
[12] Jieren Deng, Jianhua Hu, Haojian Zhang, and Yunkuan Wang. Incremental prototype prompt-tuning with pre-trained representation for class incremental learning. arXiv preprint arXiv:2204.03410, 2022.
[13] Alexey Dasovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenhorn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthis Minderer, Georg Heigold, Sylvain Gelly, Jakoh Uszkoreit, and Neil Houlsby. An image is worth 16x 16 words: Transformers for image recognition at scale. In ICLR, 2021.
[14] Arthur Douillard, Alexandre Ramé, Guillaume Couairon. and Matthieu Cord. Dytox: Transformers for continual learning with dynamic token expansion. In CVPR, 2022.
[15] Stéphane d' Ascoli, Hugo Touvron, Matthew L Leavitt, Ari S Morcos, Giulio Biroli, and Levent Sagun. Convit: Improving vision transformers with soft convolutional inductive biases. ICML, 2021.
[16] Mehrdad Farajtabar, Navid Azizan, Alex Mott, and Ang Li. Orthogonal gradient descent for continual learning. In AISTATS, 2020.
[17] Enrico Fini, Victor G Turrisi da Costa, Xavier Alameda- Pineda, Elisa Ricci, Karteek Alahari, and Julien Mairal. Self-supervised models are continual learners. In CVPR, 2022.
[18] Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya. Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to supf-supe learning. NeurIPS, 2020.
[19] Geoffrey Hinton, Oriol Vinyals, Jeff Dean, et al. Distilling the knowledge a neural network. arXiv preprint 10X.vv1503..0531,, 2015.
[20] Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning a unified classifier incrementally via rebalancing. In CVPR, 2019.
[21] Wenpeng Hu, Zhou Lin, Bing Liu, Chongyang Tao, Zhengwei Tao, Jinwen Ma. Dongyan Zhao, and Rui Yan. Overcoming catastrophic forgetting for continual learning via model adaptation. In ICLR, 2018.
11

---

[22] Tony Huang. Jack Chu, and Fangyun Wei. Unsupervised prompt learning for vision-language models. arXiv preprint FrXiv:2204006643 2022.
[23] Ching- Yi Hung, Cheng- Hao Tu, Cheng En Wu, Chien- Hung Chen, Yi-Ming Chan, and Chu-Song Song Chen. Compacting, picking and growing for unforgetting continual learning. In NeurlPS, 2019.
[24] Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Scr-Nam Lim. Visual prompt tuning. arXiv preprint arXiv:2203. 12/19, 2022.
[25] Ronald Kemker and Christopher Kanan. Fearnet: Brain-inspired model for incremental leaming. In ICLR, 2018.
[26] Sohail Ahmed Khan and Hang Dai. Vidco transformer for deepfake detection with incremental learning. In ACMMM, 2021.
[27] Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. In NewrlPS, 2020.
[28] Minha Kim, Shahroz Tarig, and Simon S Woo. Cored: Generalizing fake media detection with continual representation using distillation. ACMMM, 2021.
[29] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska- Barwinska, et al. Overcoming catastrophic forgetting in neural networks. In Proceedings of the national academy of sciences, 2017.
[30] Jeremias Knoblauch, Hisham Husain, and Tom Diethe. Optimal continual learning has perfect memory and is np-hard. In ICML, 2020.
[31] Bang-Woo Lee, Jin-Hwa Kim, Jachyun Jun, Jung-Woo Ha, and Byoung- Tak Zhang. Overcoming catas- trophic forgetting by incremental moment matching. In NeurlPS, 2017.
[32] Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. arXiv preprint (...v::214400660 2021.
[33] Chuqiao Li, Zhiwu Huang, Danda Pani Paudel, Yabin Wang. Mohamad Shahbazi. Xiaopeng Hong, and Luc Van Gool. A continual deepfake detection benchmark: Dataset, methods, and essentials. arXiv preprint arXiv:2205.055 054677....55..... 2022.
[34] Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint 2101:22001.000 2021.
[35] Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions pattern analysis and machine intelligence, 40K 22:2935-299, 2017.
[36] Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing, arXiv preprint arXiv:2107.
2021.
[37] Yaoyao Liu, Yuting Su, An-An Liu, Bernt Schiele, and Qianru Sun. Mnemonics training: Multi-class incremental learning without forgetting. In CVPR, 2020.
[38] Vincenzo Lomonaco and Davide Maltoni. Core50: new datasct and benchmark for continuous object recognition. In Conference Robot Learning, 2017.
[39] David I Lopez-Paz and Marc' Aurelio Ranzato. Gradient episodic memory for continual learning. In NeurlPS, 2017.
[40] Francesco Marra, Cristiano Saltori, Giulia Boato, and Luisa Verdoliva. Incremental learning for the detection and classification of gan-generated images. WIFS, 2019.
[41] Sudhanshu Mittal, Silvio Galesso, and Thomas Brox. Essentials for class incremental learning. In CVPR, 2021.
[42] Lorenzo Pellegrini, Gabriele Graffieti, Vincenzo Lomonaco, and Davide Maltoni. Latent replay for real-time continual learning. In IROS, 2020.
[43] Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. In 2019.
12

---

[44] Ameya Prabhu, Philip HS Torr. and Puneet K Dokania. Gdumb: A simple approach that questions our progress continual learning. In ECCV, 2020.
[45] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Míshkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In /CML, 2021.
[46] Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In CVPR, 2017.
[47] Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. Learning to learn without forgetting by maximizing transfer and minimizing interference. In ICLR, 2018.
[48] Anthony Robins. Catastrophic forgetting, rehearsal and pseudorchearsal. Connection Science, 7(2):123-
146, 1995.
[49] Gobinda Saha, Isha Garg, and Kaushik Roy. Gradient projection memory for continual learning. In ICLR, 2020.
[50] Hanul Shin, Jung Kwon Lee, Jachong Kim, and Jiwon Kim. Continual learning with deep generative replay. arXiv preprint arXiv:7005.08690 2017.
[51] Shixiang Tang, Dapeng Chen, Jinguo Zhu, Shijie Yu. and Wanli Ouyang. Layerwise optimization by gradient decomposition for continual learning. In CVPR, 2021.
[52] Xiaoyu Tao, Xinyuan Chang, Xiaopeng Hong, Xing Wei, and Yihong Gong. Topology-preserren class- incremental learning. ECCV, 2020.
[53] Gido M van de Ven, Zhe Li, and Andreas S Tolias. Class-incremental learning with generative classifiers. In CVPR, 2021.
[54] Shipeng Wang. Xiaorong Li, Jian Sun, and Zonghen Xu. Training networks in null space of feature covariance for continual learning. In CVPR, 2021.
[55] Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. In CVPR, 2022.
[56] Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Yc, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning. In CVPR, 2019.
[57] Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically 2017.
expandable networks. arXiv preprint
[58] Lu Yu, Bartlomicj Twardowski, Xialci Liu, Luis Herranz, Kai Wang, Yongmei Cheng, Shangling Jui, and Joost van de Weijer. Semantic drift compensation for class iisseinncemattlll learning. In CVPR, 2020.
[59] Jure Zhontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny. Barlow twins: Self-supervised learning via redundancy reduction. In JCML, 2021.
[60] Guanxiong Zeng, Yang Chen, Bo Cui, and Shan Yu. Continual learning of context-dependent processing in neural networks. Nature Machine Intelligence, 1(8)364 372, 2019.
[61] Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In ICML, 2017.
[62] Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning prompt for vision-language models. arXiv preprint arXiv:2109.01) 2021.
[63] Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision-language models. In CVPR, 2022.
[64] Kaiyang Zhou, Jingkang Yang. Chen Change Loy. and Ziwei Liu. General incremental learning with domain-aware categorical representations. In CVPR, 2022.
13

---

Checklist
1. For all authors...
(a) Do the main claims made the abstract and introduction accurately reflect the paper's contribu- tions and scope? [Yes]
(b) Did you describe the limitations your work? [Yes] Section
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]
(c) Did you report bars (e.g.. with respect the random seed after running experiments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g.. type of GPUs, internal cluster, or cloud provider)? [Yes]
4. If you using existing assets (e.g., code, data, models) curating/releasing new assets...
(a) your work uses existing assets, you cite the creators? [Yes]
(h) Did you mention the license of the assets? [N/A]
(c) Did you include any
either in the supplemental material URL? [No]
(d) Did you discuss whether and how consent obtained from people whose data you're 're
ing/curating? [N/A]
(c) Did you discuss whether the data you using/curating contains personally identifiable informa- tion offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A]
(b) you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "sPqkMChnQS8", "reviewer": "Reviewer_WPXn", "summary": "This paper proposes to address DIL by (1) learning a pool of prompts and classifier heads (one for each domain) for ViT and CLIP (2) inferring domain label based on K-means and KNN. ", "questions": "Can the authors clarify or disentangle the contributions from (1) CLIP as a backbone and (2) learning independent prompts for each domain?", "limitations": "See weaknesses and questions above. ", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n1. This paper is written clearly and generally easy to follow.\n2. The proposed idea is simple.\n\nWeaknesses\n1. I’m not sure whether the performance improvement over baselines (especially L2P and DyTox) are due to the proposed method or the usage of CLIP. The comparisons based on the same backbone, ViT (corresponding to S-iPrompts), don’t show a clear improvement over the two prompt-based baselines. And the significant improvement comes from the CLIP-based models. \n2. Related to the previous point, the good performance of random domain selection (Table 4) seems to imply that much of the performance is due to a simple prompt + backbone transformer.\n3. Minors: there’re typos and grammatical errors and they should be corrected. e.g., cheep prompts -> cheap prompts?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "mT0j2wB1cJ9", "reviewer": "Reviewer_GPZj", "summary": "The submission focus on the catastrophic forgetting problem in domain-incremental learning, in which a model needs to learn from different domains sequentially. To achieve the goal in an exemplar-free way, the authors design an S-Prompting mechanism. A new trainable prompt is introduced every time a new domain appears. The trained prompt is then added to a prompt repository. At inference time, when new data comes, the prompt of the corresponding domain is retrieved and prepend to the input of ViT or CLIP. Extensive experiments are conducted and the best S-Prompts surpasses not only exemplar-free methods but also some exemplar-based method.", "questions": "1. I could that see there is an analysis in Section 4 and supplementary materials about the overheads of the proposed methods compared to its backbone, but there is no information about the memory overheads of the baselines. I would suggest listing the parameter scale of the baseline methods as well.\n\n2. What do you mean by ``dynamic classifers'' in line 246 and line 272? The term never appears in the approach section.", "limitations": "The limitations are adequately discussed in the submission.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. The proposed approach is clear and elegant. It uses prompt to store the domain-related feature while keeping the backbone model unchanged though it is not the first time that prompt method is used for continual learning. Besides, no additional space is required to store the exemplar.\n\n2. The experiment is extensive and persuasive. Previous methods are properly considered. The performance is impressive, especially when compared with exemplar-based methods.\n\n3. The writing is clear and easy to follow.\n\nWeaknesses:\n\nNo serious weaknesses found.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "5a8MvLkaIJw", "reviewer": "Reviewer_Lxav", "summary": "This paper proposes to tackle the problem of domain incremental learning (in which its setting is distinct from the problem of class-incremental learning. In domain incremental learning, different domain data arrive in a sequence, one domain per incremental phase, while these domains share the same group of object/image classes), in which the domain indexes are not given during the inference (therefore dissimilar to the setting of task-incremental learning). As other incremental learning scenarios, the catastrophic forgetting is one of the biggest challenges in domain incremental learning, where the domain gap is main reason behind to cause the forgetting (i.e. while learning to classify new domain data, the model's ability on classifying old domain data could deteriorate). In order to tackle such catastrophic getting issue, instead of additionally maintaining portion of old domain data as what replay-based approaches do (under the consideration for better data security and privacy), the proposed method learns a set of prompts over (pretrained and fixed) transformers where the domain-specific knowledge is stored by a prompt pool, in which during the inference the classification is done by firstly identifying the domain ID (via K-NN to search for the nearest domain centroids obtained by K-means on training data to the test image feature) then feeding the corresponding domain-specific prompt and the image tokens to the transformer to perform the classification via the corresponding domain-specific classifier. With the fact that the proposed method is able to not only achieve image prompt learning (via ViT) but also a brand-new language-image prompt learning (via CLIP), the extensive experiments are conducted to show the superior performance of the proposed method against various baselines (including exemplar-free and replay-based incremental-learning methods, self-supervised learning method, and the recently published prompting methods) on several datasets. ", "questions": "Overall I pretty enjoy reading the whole paper and like the novel idea of learning domain-specific prompts in a domain-independent way (which strike a much better balance between alleviating the forgetting and learning to recognize new domain data) as well as its applicability on pretrained and fixed image/language-image transformers. The questions that I would encourage the authors to address in the rebuttal, for further strengthening the paper, are the ones listed in cons, including the potential contradiction between the domain-specific prompts and the robustness to erroneous domain identification, and the insignificant separation between different domains. ", "limitations": "there is no potential negative societal impact", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros:\n+ The proposed method is built upon pretrained (image or even language-image) transformers (without any fine-tuning needed) with advancing the existing prompting methods of continual learning on a novel idea: although existing methods also aim to learn the domain-specific prompts, their learning is dependent across domains thus potentially leading to less separation for the classes from different domains, while the proposed method particularly drives the prompt learning independently across domains to achieve the best for each domain. Experimental results as well as ablation study will verify the contribution of such novel idea. \n+ The proposed prompt learning (with only little overhead needed to store the domain-specific prompt) surpasses not only other prompting methods but even the replay-based incremental learning approaches (where the exemplars of old domain data are explicitly stored in the replay buffer and used during the learning for new domain data to alleviate the catastrophic forgetting) and the expansion-based incremental learning approaches (e.g. DER++, where different feature extractors are adopted per domain) , which again demonstrates the contribution of the novel idea behind the proposed method.\n\nCons:\n- Though the authors do provide additional experiments to demonstrate that the random assigning domain ID or having errors on domain identification (i.e. to firstly recognize which domain that the test data belongs to) would not have huge impact on the final classification performance, such argument seems to be contradictive to the idea of having domain-specific prompts and learning them in a domain-independent way. In other words, if using the wrong domain-specific prompt can still provide the slightly worse but comparable performance, then perhaps it could be unnecessary to perform domain identification. Instead, directly having the ensemble over the results produced by using each of the domain-specific would potentially contribute the best performance? The authors are highly encouraged to provide more discussion on this potential contradiction and experiment the variant of ensemble in the rebuttal.\n- Though the t-SNE visualizations show that the proposed method is able to produce more separation between the classes from different domains, but actually the degree of separation is not that significant (if we consider the high-level idea illustration provided in Figure.1, where the domain identification ideally would produce different subspaces across domains). The authors are highly encouraged to provide further analysis (e.g. are we able to define a metric to measure the degree of separation thus having more objective comparison between different methods) and discussion on such aspect. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "1vElwhZa_de", "reviewer": "Reviewer_CnBo", "summary": "This paper proposed S-Prompting and two concrete approaches to highly reduce the forgetting degree in continual learning scenarios, i.e., domain incremental learning (DIL).\n\nThe main idea is to learn independent prompts across domains with pre-trained transformers. \n\nThe independent prompt can achieve the best for each domain.\n\nThe learning method derives an image prompt learning approach and a brand-new language-image prompt learning approach. \n\nThe methods outperformed all three standard DIL tasks.\n", "questions": "In Tables 1 and 2, for fair comparisons with others, the architecture should be the same as the base-ViT model. \nIn the appendix, as the authors state, the number of parameters -ConViT(78M) and base-ViT(86M) are almost similar. However, I think if the structures are one another, the representations also differ.\nI would recommend the authors compare the performances under the same architecture such as the base-ViT(86M).\n", "limitations": "The authors discuss the limitation of the S-prompts properly in the script.\n", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "(+) Compared with prior works: L2P and DyTox, the proposed S-Prompts learn the tasks independently.\n\n(+) The S-prompts are technically sound and seem to be well supported by experimental results on the three standard Domain-IL benchmark datasets, and outperformed others. \n\n(-) In Tables 1 and 2, for fair comparisons with others, the S-prompts compared with others under a little bit different architecture.\n\n(-) S-prompts (S-liPrompts) seems strongly dependent on CLIP (a pre-trained Image-text mapping function), which decreases the main novelty.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept