Few-Shot Adversarial Prompt Learning on Vision-Language Models

Authors: yiwei  zhou
xiaobo  xia
zhiwei  lin
bo  han
tongliang  liu
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-03 17:24:01

Abstract: The vulnerability of deep neural networks to imperceptible adversarial perturbations has attracted widespread attention. Inspired by the success of vision-language foundation models, previous efforts achieved zero-shot adversarial robustness by aligning adversarial visual features with text supervision. However, in practice, they are still unsatisfactory due to several issues, including heavy adaptation cost, suboptimal text supervision, and uncontrolled natural generalization capacity. In this paper, to address these issues, we propose a few-shot adversarial prompt framework where adapting input sequences with limited data makes significant adversarial robustness improvement. Specifically, we achieve this by providing adversarially correlated text supervision that is end-to-end learned from adversarial examples. We also propose a novel training objective that enhances the consistency of multi-modal features while encourages differentiated uni-modal features between natural and adversarial examples. The proposed framework gives access to learn adversarial text supervision, which provides superior cross-modal adversarial alignment and matches state-of-the-art zero-shot adversarial robustness with only 1\% training data. Code is available at: https://github.com/lionel-w2/FAP.

Main paper: 
 introduction: 
 The seminal works [1, 2] reveal that adversarial examples [2] , consisting of malicious perturbations imperceptible to humans, can easily mislead state-of-the-art deep neural networks (DNNs) [3] [4] [5] [6] into making incorrect predictions. This vulnerability limits the application of DNNs in safety-critical areas, such as medicine [7] , healthcare [8] , and autonomous driving [9] .Human cognition is immune to the distribution variations induced by adversarial attacks, reflecting a fundamental difference between human and machine cognitive understanding. Humans primarily problem for an image x and its corresponding label y ∈ {1, . . . , K}. To perform zero-shot evaluation, x is first divided into M patches and converted into the patch embeddings e(x). A class token c cls is then appended to the patch sequence as e(x) = {c cls , e 1 (x), . . . , e M (x)}. Afterward, the image encoder I processes this embedded patch sequence with ViT [29] blocks to produce the latent image feature representation z (I) = I(e(x); θ I ). For the text branch, we prepare hand-craft prompts t i ∈ t = {t 1 , . . . , t K } by appending the class name to a word template, such as 'a photo of a {class}'. Subsequently, t i is tokenized and embedded as w(t i ) = {w 1 (t i ), . . . , w N (t i ), i}, where i corresponds the i-th class. The text encoder T then encodes these work embeddings into the latent text feature representation z (ti) = T (w(t i ); θ T ). For zero-shot classification, the probability of the image x in the i-th class isp(y = i | x) =exp cos z (I) , z (ti) /τ K j=1 exp cos z (I) , z (tj ) /τ ,where cos (•, •) denotes the cosine similarity score and τ is the temperature parameter.CLIP-based prompt learning. Instead of adopting a hand-crafted prompt, prompt learning attempts to train lightweight learnable prompts P t with a few examples from downstream data. To be concrete, P t is inserted into word embeddings as w(t i , P t ) = {P t , w 1 (t i ), . . . , w N (t i ), i}. Then, the text feature representation is z (ti,Pt) = T (w(t i , P t ); θ T ). To preserve the alignment characteristics of the joint image-text feature space for zero-shot capabilities, CLIP-based prompt learning optimizes the prompt tokens by narrowing the gap in the distribution between text-image logits and the ground-truth label using cross-entropy:P * t =Pt E (x,y) L CE cos(z (I) , z (ti,Pt) ), y ,where cos(z (I) , z (ti,Pt) ) corresponds the text-image logits. We suggest readers check Zhou et al. [28] for more details about CLIP-based prompt learning.Adversarial visual prompt. Adversarial prompt learning optimizes prompt tokens through adversarial training, enhancing model robustness in a relatively small adaptation cost without altering the pre-trained model. Mao et al. [11] achieves this by adjusting the visual prompt of adversarial images in joint text-image feature space. Notably, owing to the application of text-image contrastive loss during the generation of adversarial examples, the adapted model reveals zero-shot adversarial robustness on downstream tasks. Formally, let (X , d ∞ ) be the input feature space X with the infinity distance metric, where d ∞ (x, x ′ ) = ∥x -x ′ ∥ ∞ . Adversarial data x falls in to close ball B ϵ (x) of radius ϵ centered at x ∈ X . That is, B ϵ (x) = {x ′ ∈ X | d ∞ (x, x ′ ) ≤ ϵ}. The learnable image prompt P v is inserted to the visual patch embedding of x, as e(x, P v ) = {c cls , P v , e 1 (x), . . . , e M (x)}. Then, adversarial data x is generated by maximizing the text-image contrastive loss asx = arg max x∈Bϵ(x)L CE cos(z (I,Pv) , t), y ,where z(I,Pv) = I(e(x, P v ); θ I ). The learnable prompt token P v is optimized given the adversarial example x, hand-craft prompts t, and ground-truth label y, by minimizing the adversarial text-image contrastive loss:P * v =Pv E (x,y) L CE cos(z (I,Pv) , t), y .Here, L CE cos(z (I,Pv) , t), y is defined as a text-image contrastive adversarial training (TeCoA) loss by Mao et al. [11] that highlights adversarial text-image alignment.Drawbacks of previous methods. Despite the promising zero-shot adversarial robustness achieved through adversarial visual prompts, certain inherent characteristics impede its widespread application.(1) The zero-shot adversarial robustness in downstream tasks originates from the alignment of image and text embedding on a large-scale generic dataset like the entire ImageNet during prompt tuning. This necessitates an extensive amount of training data and employs prompts of considerable size (token-level prompts with a size of 200), which not only causes significant prompt-related overhead but also precludes the benefits of lightweight adaptation on the top of the pre-trained models that prompt tuning typically offers. Note that only prompt tokens as well as the deep projections from image to text are tuned while the rest of the model is frozen. Our method promotes a consistent cross-modal similarity distribution between natural and adversarial examples, while encouraging differences in uni-modal representations. The adversarial-aware text supervision learned in this manner can better align adversarial features and establish robust decision boundaries with a limited number of examples. The natural and adversarial forward processes of the image encoder share parameters.(2) Due to the distinct visual representation distribution between clean and adversarial examples, static hand-crafted prompts lack adversary-related hints, thereby only providing content-related information without effectively supervising the adversarial components contained in the images. However, manually adjusting hand-crafted prompts to inject additional adversarial hints is also challenging, as the imperceptibility of adversarial perturbations limits their feature description, and the intensity and distribution of these perturbations are variable throughout the training process.(3) The current learning objective directly trains to provide prompts with adversarial examples, yet it overlooks the model capacity for natural generalization in downstream tasks. This presents a potential risk of failure, especially in the context of few-shot prompt tuning where the pre-trained model shows inadequate natural generalization on a sampled few-shot dataset. 
method: 
 Overview. To address the limitations of previous methods, we propose FAP, a few-shot adversarial prompt learning framework. Our framework uses lightweight learnable prompts on the top of the pre-trained CLIP in a few-shot manner, as the case in natural prompt tuning [28] . In more detail, we introduce learnable prompt tokens for adversarial examples, which allows the model to provide more appropriate text supervision that helps balance natural and adversarial generalization. Based on CLIP's dual-encoder architecture, we further provide a novel training objective that guides the discrimination of natural and adversarial embeddings in uni-modal feature space. This promotes uni-modal divergence to incorporate an adversarial-aware mechanism, facilitating the learning of adversarial text supervision. The overview of the proposed framework is provided in Figure 1 . Below, we discuss the FAP framework step by step.When adapting the CLIP model, a slight change in wording could have a huge impact on performance [28] . With the existence of adversarial examples, the situation has become worse. The distribution differences between natural and adversarial examples necessitate the design of special-ized text supervision specifically for adversarial samples. Therefore, we introduce text prompt tokens that are end-to-end learned through adversarial examples.Formally, our adversarial prompt learning is implemented on a few-shot subset S, created by sampling m examples from each of the K classes in the original dataset. Learnable prompts consist of both visual and text branches, denoted as P = {P v , P t }. The visual prompt token P v is incorporated into the image embedding, as observed in an adversarial visual prompt, while text prompt token P t is inserted into word embedding, as is the case in natural prompt learning. To preserve mutual synergy between visual and text branchs, P t is obtained from P v through linear projection h, which can be denoted as P t = h (P v ). The proposed framework can be categorized as a cross-modal prompt [30] with minimal modification for adversarial robustness tasks. We offer a comprehensive analysis of the prompt design in Section 4.3.For adapting the CLIP model to adversarial robustness tasks, the existing method [11] proposes the TeCoA loss (Eq.( 4)). This method minimizes the discrepancy between the distribution of adversarial text-image similarity and one-hot ground-truth labels. While this strategy effectively aligns text representations during adversarial adaptation, it potentially compromises the model's generalization ability in specific recognition tasks under few-shot conditions.The method's effectiveness depends on the similarity between the downstream task's distribution and the pre-trained representations. When the downstream task closely aligns with the pre-trained representation, the CLIP model shows preferable natural generalization, and adding learnable prompts for robustness adaptation is advantageous. However, a significant mismatch between the downstream distribution and pre-trained representations challenges the CLIP model's natural generalization capabilities. In such cases, expecting prompt tokens to learn both natural and robust generalization from a few adversarial examples is overly ambitious.Balancing natural and adversarial generalization. Inspired by the success of TRADES [31] in standard adversarial training, we propose a surrogate adversarial text-image contrastive loss that decouples the adversarial text-image contrastive loss into natural and adversarial terms. By encoding image and text embeddings with their respective transformer encoder and calculating similarity across modality, we have the natural and adversarial text-image logits: cos(z (I,Pv) , z (t,Pt) ) and cos(z (I,Pv) , z (t,Pt) ), where z (t,Pt) = {z (t1,Pt) ), . . . , z (t K ,Pt) )}. The learning objective can be stated as: Pv) , z (t,Pt) ), y + λL KL cos(z (I,Pv) , z (t,Pt) ), cos(z (I,Pv) , z (t,Pt) ) , (5) where L KL denotes the Kullback-Leibler (KL) divergence and λ is a weight parameter. In Eq. ( 5), the first term encourages minimizing the natural error between the natural text-image similarity and label.L = L CE cos(z (I,The second term minimizes the boundary error by narrowing the distribution gap between natural and adversarial text-image similarity to ensure cross-modal adversarial consistency. Note that a balanced two-term objective is crucial for downstream generalization, as this design alleviates the potential failure in robustness caused by discrepancies in natural generalization. We provide more analysis on the natural generalization gap in Appendix D.2.To fully leverage the structural advantages of CLIP, we go beyond enforcing consistency constraints on cross-modal text-image features and tailor adversarial robustness enhancements for uni-modal features. Specifically, we introduce an adversarial-aware mechanism for visual features, guiding the distinction between natural and adversarial examples. To the best of our knowledge, this is the first initiative to foster differentiated representations in adversarial regularization.Given the distinct distributions of natural and adversarial examples, we argue that driving consistent outputs for natural and adversarial examples in visual models constitutes a compromise, trading off generalization for robustness. In contrast, within CLIP, we achieve robustness by maintaining adversarial consistency in the text-image joint space with the adversarial term in Eq. ( 5), while preserving the distributional differences of features in the uni-modal visual space to minimize the impact on generalization performance. Here, we append an extra constraint on the adversarial term with cosine similarity:EQUATIONwhere the constant 1 maintains the non-negativity of L cos . We introduce the adversarial-aware mechanism by adjusting prompt tokens to minimize similarity, thereby distinctly differentiating between natural and adversarial visual features. During the training process, the text branch learns to provide proper text supervision for different visual features, ensuring that the outputs in the text-image joint space are consistent for natural and adversarial embeddings, which have significant distributional differences in the visual space.Objective for outer minimization. The overall training objective can be obtained by introducing uni-modal adversarial aware mechanism L cos to Eq. ( 5) as:EQUATIONObjective for inner maximization. The goal of inner maximization is to generate the adversarial example x. Here, we leverage the adversarial term in Eq. ( 5) as this surrogate loss and find the adversarial example x as follows:x = arg max x∈Bϵ(x)L KL cos(z (I,Pv) , z (t,Pt) ), cos(z (I,Pv) , z (t,Pt) ) .Note that strong attacks can help robustness. Here, the general PGD attack formulation with the CE loss like Eq. ( 3) is also applicable. With the learning objective outlined in Eq. ( 7), we adapt learnable prompt P = {P v , P t } tokens on the few-shot dataset S as:EQUATIONOur learning objective highlights the differentiated processing of features under different modalities, in which we introduce an additional adversarial-aware mechanism with uni-modal image features. We discuss the intuition behind the design concept. We visualize the uni-modal embedding to demonstrate the impact of the adversarial-aware mechanism on the model's feature learning. In Figure 2a , we find that certain adversarial embeddings closely resemble natural examples. This suggests that the consistency of cross-modal features between natural and adversarial examples arises from the model's tendency to minimize loss by generating minimal adversarial perturbations. These exceedingly small perturbations do not effectively promote robust learning. In contrast, the adversarial-aware mechanism clearly separates the natural and adversarial embeddings in Figure 2b , preventing the minimal perturbation shortcut and guiding the model to recognize the differences between natural and adversarial image embeddings.For better understanding, we discuss different training objective designs and their results in Section 4.3 and describe our adversarial prompt learning and adversarial prompt testing pipeline in Appendix A. Additionally, we demonstrate the significant robustness gains our learning objective brings to other prompt designs through a case study. More details can be checked in Appendix D.4. 
setups: 
 Baselines. To demonstrate the expertise of the proposed method, we employ the adversarial version of multiple commonly used prompt learning designs for comparison. We categorize our baselines into two groups: (1) Methods using hand-crafted text supervision, such as zero-shot CLIP [12] and AdvVP [11] . (2) Methods utilizing learnable text prompts, including AdvVLP and AdvMaPLe [30] . Note that we primarily focus on learnable prompts extending the AdvVP framework. Details on pure text prompt effects in adversarial settings (AdvTP) [28] are discussed in Appendix D.11. Additional information about these methods and static prompt templates for each dataset are provided in Appendices C.1 and C.2, respectively. Datasets. To evaluate the proposed method, we align with previous works [28, 33] and utilize 11 diverse image recognition datasets that span multiple vision tasks. Specifically, the datasets include two generic object datasets: ImageNet-1K [20] and Caltech101 [32] ; a texture recognition dataset: DTD [34] ; five fine-grained object recognition datasets: FGVCAircraft [35] , OxfordPets [36] , Flowers102 [37] , Food101 [38] , and StanfordCars [39] ; a scene recognition dataset: SUN397 [40] ; an action recognition dataset: UCF101 [41] ; and a satellite image classification dataset: EuroSAT [42] .Implementation details. We conduct experiments on the ViT-B/32 CLIP architecture and report the average results over three random seeds. All models are trained for 5 epochs in cross-dataset evaluation and 10 epochs for other benchmark settings by using an SGD optimizer with a momentum of 0.9. The initial learning rate is set at 0.0035. We apply a cosine learning rate scheduler and a warm-up strategy during the first epoch. For adversarial prompt learning, we use token prompts of size 2 in both the vision and text branches across the first 9 transformer blocks. Attacks are generated under ℓ ∞ threat model through a 2-step PGD attack, with a perturbation boundary ϵ = 1/255 and a step size α = 1/255, following the methodologies outlined in [11] . The adversarial robustness is evaluated using a 100-step PGD attack.Note that due to the limited space of the main paper, we provide comprehensive evaluations, including cross-dataset evaluation (Appendix D.1), the comparison with AdvMaPLe (Appendix D.3), alternative CLIP architectures (Appendix D.5), different attack strengths (Appendix D.6), various choices of adversarial robustness evaluation methods (Appendix D.7), and different training-time attack generation (Appendix D.8).Adversarial few-shot learning. In this scenario, we evaluate the model's ability to develop robust representations with a severely limited amount of downstream data. Specifically, we tune the model using {1, 2, 4, 8, 16} shots from each class. As shown in Figure 3 , the static text prompt of baseline method struggles to align with adversarial input images under a few-shot setting. Even with an increased number of training samples, the model's performance fails to improve, indicating difficulties in adversarial learning. AdvVLP and AdvMaPLe, through end-to-end learning of adversarial text prompt tokens from adversarial examples, have acquired the capability to adjust prompts from limited samples to gain adversarial robustness. By further training with our proposed objective, our method achieves superior average natural and adversarial accuracy across 11 datasets.Adversarial base-to-new generalization. We present a more challenging adversarial base-to-new generalization setting, where datasets are bifurcated into base and new subclasses. Here, models are trained with a 16-shot dataset from the base classes and are subsequently evaluated on both base and new classes. In this setting, as the number of categories in datasets is generally much smaller than the number of examples per class, models need to learn intrinsic features within each dataset and robust representations from limited examples to effectively generalize large amounts of test data.From Table 1 , we observe that our method not only surpasses all its counterparts in robust metrics, but also reveals superior natural generalization due to the joint consideration of natural and robust features in our training objective. Additionally, our method also reveals much better stability (lower standard deviation). That is, even sampled few-shot subset has a natural generalization gap, our learning objective still works well and prevents potential failure. Matching Benchmark Zero-Shot Results Adapted with ImageNet-1K. In addition to comparing with the baseline AdvVP under few-shot settings, we also benchmark against zero-shot results, where robustness is evaluated through cross-dataset evaluations. Initially adapted on ImageNet-1K, our method does not require adaptation across the entire dataset nor extensive prompt designs like the AdvVP [11] , which uses embedding-level token prompts of size 200 and pixel-level pad prompters of size 40. As shown in Table 2 , our method aligns with benchmark performance using just 1.25% of ImageNet-1K examples, significantly accelerating the training process by over 97%. Moreover, enhancements from 16-shot to 32-shot training and deepening prompt layers from 9 to 12 allow our method to exceed previous adversarial prompt tuning results. From Table 3 , we can conclude that as λ increases, the proportion of the adversarial component in the total loss increases, and the natural accuracy declines continuously. Meanwhile, adversarial robustness gradually improves, reflecting the trade-off between natural and adversarial generalization. However, when λ becomes too large (λ > 2.5), continuing to increase the proportion of the adversarial component does not lead to further improvements in robustness.Prompt depth and prompt length. We provide architectural ablation results for prompt design concerning different prompt depth and length settings. In Table 4 , we can observe that increasing both prompt depth and prompt length introduces more learnable parameters, thereby resulting in improved performance. Furthermore, we can also conclude that the performance gain obtained by increasing prompt depth is higher than that achieved by increasing prompt length, and the improvement in robustness metric is larger than in natural accuracy. Ablation for training objective design. In Section 3.4, we present our proposed novel training objective tailored for adversarial prompt learning. Our loss follows a two-term design, comprising a natural term and an adversarial term. The adversarial term further considers both the consistency and diversity of natural and adversarial features. In practice, we use KL divergence to constrain cross-modal consistency and encourage uni-modal diversity with cosine similarity. In Table 5 , we present other possible designs for the loss function and conduct an ablation study under the adversarial base-to-new setting. Our method provides the best robustness across all these loss function settings.Instability analysis for deep prompt interaction. We report an instability of generalization performance caused by the improper deep prompt interaction, revealing that the standard cross-modal prompt interaction design, from text to image prompt token, is not plug-and-play under the setting of adversarial robustness. When natural and adversarial terms are present in a certain moderate ratio in the learning objective, the performance of the model may experience a significant decline.From Figure 4 , we find that the instability intensity caused by the text-to-image design varies across different datasets, and the values of λ leading to this instability are also different. For instance, on some generic datasets, the performance degradation it usually brings is not significant (Figure 4c ). However, on some fine-grained datasets, the significant performance degradation caused by this instability is unacceptable (Figure 4b ). To understand this, we plot the loss curve during the training process under both stable and unstable settings. As revealed in Figure 5 , in unstable cases, we observe that the robust loss drops to zero early in training and remains nearly unchanged at this low level during the mid-phase, while the overall loss does not decrease as expected. This suggests the text prompt falls into a trivial local solution during optimization, equating natural and adversarial logits. This nullifies the adversarial term but overlooks natural generalization, causing consistently high natural loss. This issue typically occurs when the natural and robust terms are balanced in a moderate ratio in the training objective.We propose a minimal refinement to prevent instability: switching the deep prompt interaction to an image-to-text scenario. Here, the text prompt is derived from the image prompt projection, limiting its adaptability. This prevents the adversarial loss from reaching zero, thus avoiding the issue. 
conclusion: 
 In this paper, we focus on adversarial prompt tuning on vision-language models, a domain with significant potential for zero-shot downstream adversarial robustness. We precisely reveal the issues of previous methods that perform adversarial visual prompts with static text supervision. Our method distinguishes itself by introducing learnable adversarial text supervision combined with a new training objective, facilitating effective learning in a few-shot setting. The proposed method enjoys excellent algorithmic properties and matches state-of-the-art performance, notably with reduced computational demand. We believe that this work can provide some insights to the community and stimulate further research in this area.  
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes]Justification: We include means and standard deviations from multiple repeated experiments for experimental results in Section 4.2 and Appendix D.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.• The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: We discuss the boarder impact of our paper in Appendix E.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: Our paper does not include generative models and typically uses open-source datasets for training and evaluation.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: The creator of assets [28] used in our paper states the license in their repository (MIT License).Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 13. New Assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [NA] Justification: Although we will submit the code in the supplementary materials, we will continue to improve the codebase and make it publicly available after the paper is officially accepted. Currently, we have not released any new assets. Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: Adversarial robustness. Adversarial attacks fool models by overlaying carefully designed imperceptible perturbations on input data [1, 2, 43] . In response to the susceptibility of models to such attacks, adversarial training [2, 44, 31, [45] [46] [47] [48] [49] has emerged as one of the most effective empirical defense methods to enhance model robustness. It incorporates adversarial data into the training process and ensures the model's predictive distribution for adversarial images closely aligns with the ground truth label. Moreover, recent advancements have seen the incorporation of contrastive learning into adversarial training [50] [51] [52] , which enables models to learn robust feature representations through instance discrimination tasks. As a result, models can align predictions for natural and adversarial image pairs in a self-supervised manner [53] [54] [55] [56] [57] [58] [59] . Additionally, there's a growing interest in aligning predictions for adversarial image-text pairs in a text-supervised context [11, 60] , offering new avenues for zero-shot adversarial evaluation. Nevertheless, current research utilizes CLIP text encoding to produce static text supervision, which, although effective for clean images, may not adequately cater to the nuances of adversarial examples.Adversarial few/zero-shot classification. Adversarial training possesses a significantly larger sample complexity of robust generalization [61] , making it challenging to learn robust representations from sparse data. Existing works in adversarial few-shot classification fall into two categories: meta-learning based [62, 63, 21] , which optimize an adversarial meta-learner using both clean and adversarial examples, and non-meta-learning based [22, 64] , employing strategies like auxiliary corrective classifiers [22, 64] or reweighted mechanisms [22] for learning robust embeddings. Additionally, Yucel et al. [65] initiated the investigation of adversarial robustness in a zero-shot learning setting, where no downstream statistics are available during training. Inspired by the successes of Vision Language Models (VLMs), recent studies [11, 66] have unanimously chosen to incorporate semantic information from text supervision to bridge the generalization gap.Vision-language models (VLMs). Foundational VLMs [12-19, 67, 68] integrate interactions derived from image and text encodings for multi-modal pre-training. Depending on their specific objectives, VLMs can be trained through image-text contrastive learning [12, 13, 15-17, 69, 19] , image-text matching [17, 19] , and text generation [17] [18] [19] . Utilizing large-scale image-text datasets (e.g., 400M pairs for CLIP [12] , 1B for ALIGN [13] ) and end-to-end pre-training, these models acquire semantic relations between text and image features, thus exhibiting a profound understanding of open-vocabulary concepts. Consequently, VLMs have emerged as state-of-the-art solutions for various visual and vision-language tasks [70] [71] [72] [73] [74] [75] [76] . Nevertheless, some recent researches [77, 78] reveal that VLMs are also highly susceptible to adversarial perturbations.Prompt learning for VLMs. Prompt learning, initially introduced in the NLP community [23] [24] [25] , involves adapting pre-trained models by adding a small number of new learnable parameters in the input data for downstream tasks, without altering the pre-trained weights. This method stands out among other lightweight adaptation approaches due to its exceptional adaptability and flexibility [79] . It has garnered increasing attention for adapting vision [26, 27, 80] and visionlanguage models [28, 33, 81, 82, 30, 83, 84] . Specifically, in VLMs, CoOp [28] pioneers prompt engineering for adapting CLIP models by modeling learnable context vectors to replace hand-crafted text prompts. CoCoOp [33] further enhances the generalization ability of CoOp by introducing conditional prompts specific to each visual input instance. MaPLe [30] integrates vision and language prompts with inner synergy for cross-modality prompt learning. Two recent works, ProGrad [84] and PromptSRC [83] , concurrently advance the generalization of prompt learning by employing regulating constraints from zero-shot CLIP predictions to prevent the forgetting of general knowledge.All experiments are conducted in an environment running PyTorch 1.10.1 and CUDA 11.3 on Python 3.8. Experiments of adversarial prompt tuning on the ImageNet-1K dataset are carried out on a single NVIDIA RTX A40 GPU, while experiments on the other 10 datasets are performed on a single NVIDIA RTX 4090 GPU.Adversarial visual prompt. We implement the adversarial visual prompt following all architectural and parameter settings in [11] for a fair comparison. In detail, we follow their code implementation to use a token-level prompt with size 5 and an image padding prompt for 30 pixels around the image. An SGD optimizer and a consine learning rate scheduler are used to train 10 epochs with an initial learning rate of 40.Adversarial text prompt. We adopt a CoOp architecture [28] as our text prompt baseline and adapt learnable context vectors with adversarial examples. We typically follow [28] to use 16 context tokens with an additional class token appended at the end of the context vector. An SGD optimizer and a consine learning rate scheduler are used to train 200 epochs with an initial learning rate of 0.002, which aligns with the training settings in CoOp.Adversarial multi-modal prompt. Adversarial multi-modal prompt in this work follows all the design choices as MaPLe [30] , but are adapted with an adversarial text-image contrastive loss. To sum up, it contains a token-level learnable token with size 2 in both text and visual branches in the first 9 transformer layers, and the deep prompts are coupled through a text-to-image projection. The above prompt tokens as well as the deep projections are optimized for 10 epochs with SGD optimizer and cosine learning rate scheduler from an initial learning rate of 0.0035.Adversarial vision language prompt. Adversarial vision language prompts possess the same vision and language prompt design as adversarial multi-modal prompts, but vision and language prompts are independently adapted without interaction. All learnable prompts are adapted for 10 epochs with SGD optimizer and cosine learning rate scheduler from an initial learning rate of 0.0035.Overall methodological explanations. We summarize the prompt design and loss function of both baselines and our methods in Table 6 . Note that the prompt design for baselines follows the original settings in their corresponding paper, while we replace their loss function with the TeCoA loss for adversarial training and evaluation. This is consistent with the methods used in Mao et al. [11] . We report the hand-crafted prompt templates used in Zero-shot CLIP, AdvVP, and our method for initialization on 11 image recognition datasets in Table 7 . For the cross-dataset evaluation, models are adapted on the ImageNet-1K dataset using 16 shots and then assessed for their zero-shot adversarial robustness across 10 distinct datasets, without further downstream tuning. As shown in Table 8 , our method outperforms its counterparts in 8/11 datasets and baseline in all 11 datasets. Moreover, it reveals that robust adaptation takes the cost of natural accuracy, as models obtained using various robust adaptation methods exhibit a decline in zero-shot natural accuracy performance on downstream datasets, compared to the original CLIP model.We identify a failure risk in few-shot adversarial prompt learning using TeCoA loss [11] , where insufficient natural generalization on the sampled dataset impedes robustness learning. Figure 6a displays the loss variation during training under this setup. Under the same experimental setup using the TeCoA loss, different trials exhibit completely different trends: the curve for the failure case shows that the loss quickly ceases to decline and becomes stable shortly after training begins, whereas the loss in the normal case continues to decrease as the training progresses.We presume that this failure stems from a lack of natural generalization ability. To confirm this, we first conduct natural tuning on the problematic few-shot dataset and then apply adversarial prompt learning. This restores the model's robust fine-tuning performance, as evident in Figure 6b , where natural and robust accuracies improve significantly after natural example adaptation. Besides, we validate the learning process on the same few-shot dataset with a dual-form loss in the training objective that considers both natural and adversarial terms (red lines in Figure 6a ). It is revealed that this two-term loss effectively acts as a surrogate for the aforementioned two-stage method, avoiding potential failures caused by the natural generalization barrier in end-to-end training.By examining the structural vulnerabilities (Figure 4 ) and the inadequate natural generalization (Figure 6 ) inherent in AdvMaPLe's learning objectives, we have proposed straightforward yet effective improvements. We generally have two improvements from AdvMaPLe:• Imp.1: Regarding the prompt design, we optimize the projection direction under the adversarial prompt learning situation for superior stability.• Imp.2: Regarding the learning objective, we not only consider the natural generalization gap that may cause the adversarial prompt to fail in the few-shot setting, but also make full use of the CLIP structure to design a differentiated robust learning strategy between different modalities.For a clear picture of the empirical boost, we demonstrate the incremental changes concerning AdvMaPLe. We separately report the performance changes resulting from modifications to prompt direction alone, the learning objective alone, and the combination of both in Table 9 .We find from Table 9 that adopting our provided learning objective alone can enhance model performance. However, the performance change brought about by modifying the projection direction alone on the basis of AdvMaPLe is subtle, as the model's performance on most downstream datasets is not saturated at this point. On the other hand, further modifications to the projection direction based on row 3 can lead to additional improvements in model performance due to the repair of instabilities on certain datasets. We further illustrate the adversarial robustness enhancement brought by using our proposed training objective for prompt learning through an intuitive case study. Here, we adapt AdvVLP with both TeCoA loss and our L final . In Figure 7 , our loss improves zero-shot adversarial robustness across ten out of eleven datasets.Additionally, our training objective results in evident performance gain under few-shot base-to-new generalization, as revealed in Table 10 . That is, we not only achieve better base natural accuracy (+11.11%), base PGD-100 accuracy (+6.67%), new natural accuracy (+2.03%), new PGD-100 accuracy (+0.87%), but also maintains superior stability across different trails.We provide the adversarial cross-dataset transfer results on another CLIP ViT backbone, ViT-B/16, that is adapted to the proposed method. With the same architectural design, ViT-B/16 divides the input image into smaller patches to better capture and learn image details. This makes ViT-B/16 generally have superior performance over ViT-B/32 in natural image recognition due to its finer granularity, but it also incurs higher computational costs due to longer input sequences. However, when considering tasks involving adversarial robustness, more complex models do not necessarily yield better performance [85] . We report the results on ViT-B/16 in Table 11 . We find that ViT-B/16 does not bring about improved robustness performance, which is due to adversarial prompt learning focusing more on feature alignment and understanding between different modalities rather than As can be seen in Figure 8 , a larger perturbation bound brings a stronger attack, thus decreasing the zero-shot robust performance. As a lightweight adaptation method, prompt tuning for superior zero-shot adversarial robustness to large attack strength requires more training data. We consider more powerful Auto-Attack [86] to evaluate our adapted model. Now that adversarial prompt tuning does not rely on the obfuscated gradient, we use two APGD variants, APGD-CE and APGD-DLR, in Auto-Attack to evaluate our models. In Table 12 , we can conclude that Auto-Attack provides a stronger attack and causes varying degrees of performance degradation in each model. Our model still exhibits better robustness to Auto-Attack compared with AdvVP, AdvVLP, and AdvMaPLe. Moreover, by adapting AdvVLP with our learning objective in Appendix D.4, we achieve further performance gain under all three different perturbation bound settings. Note that Auto-Attack uses a fractional attack generator which explores that fraction space by automatically adjusting step size α, it serves as a more effective and powerful attacker for zero-shot adversarial robustness evaluation. We adopt Eq. ( 8) to carry out adversarial attacks during the training process. We did not take L cos into account in Eq. ( 8). Including L cos in the generation of adversarial samples would make the gradient information focus on the differences between natural and adversarial examples, thereby generating stronger adversarial perturbations with greater differences from the natural examples. However, since we have incorporated this term in our adversarial defense, the model will gradually provide stronger attacks during iterative learning to ensure differences in image features between natural and adversarial samples, making it somewhat redundant in function.We validate this with the experimental results in Table 13 . We can observe that the results of these two methods for generating adversarial attacks are quite similar, indicating that adding L cos in the attack is indeed redundant. Therefore, for the sake of simplicity, we did not include L cos for training-time attack generation. For adversarial few-shot prompt learning, we plot curves showing how the average natural and robust accuracy change with varying shot numbers in Figure 3 . Here, we present the mean and standard deviation of natural (Table 14 ) and robust (Table 15 ) accuracy for all experimental settings, datasets, and shot numbers, based on our multiple trials. For our proposed method, when given a smaller number of training samples, both the standard deviation of natural accuracy and robust accuracy are relatively high, indicating that the performance of learning robust representations at this stage depends on the quality of the examples. As the shot number increases, our method exhibits a significant reduction in the standard deviation for both natural and robust accuracy, demonstrating its ability to acquire adversarial robustness stability.For adversarial base-to-new generalization results in Section 4.2, we further provide the detailed results on each dataset. In Table 16 , our method demonstrates preferable learning performance on the majority of datasets. Specifically, in recognition datasets for fine-grained tasks that significantly differ from generic knowledge (DTD, Flowers102, OxfordPets, FGVCAircraft, etc.), our training objective effectively avoids potential failures caused by natural generalization barriers in robustness learning, thus yielding more stable results across multiple trials.We design most of the baseline settings on the top of the adversarial vision prompt framework. As a result, most of them belong to a cross-modal prompt family, with learnable prompt tokens not only exist in both vision and text input sequences. However, for completeness, we also consider the design of prompts in a uni-modal context, namely adversarial vision prompts (AdvVP) and adversarial text prompts (AdvTP). In Figure 9 , we find that, as the number of available examples increases, both vision and text prompts fail to acquire more robustness correlated hints for promoting adversarial robustness. However, although it seems difficult for AdvTP to learn proper adversarial text supervision, AdvTP is capable of maintaining preferable natural performance even when only adversarial examples are visible. We believe this can be attributed to the text prompt's ability to capture semantic information.This research aims to contribute positively to the machine learning field by enhancing model robustness against adversarial attacks. While we believe our work is unlikely to have direct negative societal impacts, we acknowledge the importance of considering potential misuse scenarios, such as in the context of security applications. The broader implication of our study is that it enables neural models to maintain adversarial robustness with minimal adaptations, making it particularly suitable for real-time applications in mobile and embodied systems. Such advancements could lead to more secure and reliable applications in various real-world scenarios, including mobile device security. During the reviewing process, the source code is supplied anonymously as part of the supplementary materials. Additionally, upon the acceptance of the paper, this code will be publicly released.This paper introduces a framework that leverages the architecture of cross-modal prompts to enhance model robustness. This is achieved by adjusting the prompts to learn adversarial-correlated text supervision. However, prompt learning is merely a parameter-efficient strategy for model adaptation, and other parameter-based adaptation methods, such as full-finetuning, are not considered in this work. Furthermore, while our method has empirically shown that a comprehensive consideration of the connections and distinctions between natural and adversarial examples can better learn adversarial text supervision, a systematic theoretical analysis and proof remain elusive. We regard addressing these limitations as our future direction. Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in the appendix, or as supplemental material.With additional pixel-level pad prompt. 


Decision: This paper proposes a few-shot adversarial prompt framework aimed at improving the adversarial robustness of vision-language models (VLMs). Overall, all reviewers enjoyed reading this paper and found the proposed method is novel and the provided empirical results are strong. But meanwhile, several concerns are raised, including some technique aspects need to be further clarified, and additional ablations are needed. 

The authors' rebuttal successfully addresses most of these concerns, and all reviewers unanimously agree to accept this paper.

Reviews: review: paper_summary: paper_summary: This paper addresses adversarial robustness for image classification with VLMs (e.g. CLIP model). To this end, the authors proposed the FAP framework to adapt VLM models by learning prompt tokens in a few-shot manner with adversarial examples. The loss function is proposed to promote accuracy on clean images, to align the text-image embedding interaction across clean and adversarial images and to align the image embedding similarity across clean and adversarial images. Experiments are carried out on image classification tasks, with the few-shot setup on multiple datasets, and the proposed method achieves good results on both base and novel datasets.
strength_weakness: strength_weakness: * The framework architecture is not easy to understand without reading the MaPLe paper. For example, is $P_v$ in line 150 shared shared or separate across different encoder layers? And the modification of $h$'s direction is not clear without reading the appendix. The presentation should be improved and more self-contained.
* From Table 1, it seems that most of the gains are coming from the architecture of MaPLe. For the proposed loss in equation 7, the components of them are not carefully ablated.

Nitpicking:
Line 108: $\theta_T$ -> $\theta_I$
questions: questions: In Eq 3 and 4, $\bf{t}$ seems to be a list of strings/text prompts (following line 83), how is $cos$ computed on strings? Or are we missing some $z$ here?
In Appendix A Algorithm1, are $P_v$ and $P_t$ separate or one is dependent on another? It's not very clear in that codeblock.
limitations: limitations: Limitations are adequately addressed in the appendix.
paper_summary: paper_summary: Adversarial prompt learning on vision-language models has traditionally focused on aligning text with corresponding images to ensure coherence and contextual accuracy. This paper extends this approach by making the image features of natural and adversarial examples distinct while still aligning them with the relevant text descriptions. This novel contribution enhances the model's robustness against adversarial attacks. The authors introduce a new framework that leverages adversarial text supervision to improve cross-modal adversarial alignment. This framework allows for the learning of adversarial prompts, significantly boosting the model’s ability to handle adversarial examples. Remarkably, the proposed method achieves state-of-the-art zero-shot adversarial robustness while utilizing only 1% of the training data, demonstrating both efficiency and effectiveness in enhancing model performance.
strength_weakness: strength_weakness: Depth of Analysis: The paper introduces a novel training objective that is both interesting and unexpected. However, there is a need for a deeper analysis and explanation of why this proposed method is effective. Understanding the underlying mechanisms and reasons for its success would provide valuable insights and strengthen the paper's contributions.
questions: questions: When would this few-shot setup particular useful? If we already have LAION and other big dataset?
limitations: limitations: No.
paper_summary: paper_summary: This paper introduces a novel few-shot adversarial prompt framework for enhancing the adversarial robustness of vision-language models. The authors propose a method that achieves state-of-the-art zero-shot adversarial robustness using only 1% of training data, addressing limitations of existing approaches such as heavy adaptation costs and suboptimal text supervision. The key contribution is a new training objective that improves multi-modal feature consistency while differentiating uni-modal features between natural and adversarial examples.
The experiments rely on one multimodal model, but executes experiments on 12 datasets.
While the approach shows promise, potential limitations include questions about generalizability across different model architectures.
Overall, this paper presents an innovative approach to improving adversarial robustness in vision-language models with promising results.
strength_weakness: strength_weakness: - Generalizability: It's unclear how well this method generalizes across different vision-language model architectures. The authors should provide more information on the range of models tested.
- Comparative analysis: While the abstract mentions matching state-of-the-art performance, a more detailed comparison with existing methods would strengthen the paper.

Writing:
 - Line 74: a cross reference would ease reading.
questions: questions: - Have you thought of how your method performs across different types of adversarial attacks?
- What are the limitations of your approach, and are there specific scenarios where it might not perform as well?
limitations: limitations: The authors have addressed some limitation in the appendix, directing towards more theoretical analysis of the method. 


In my opinion, long-term robustness could be addressed as well: The paper could have addressed (at least in the future work) how the proposed method performs against evolving adversarial attacks such as adaptive attacks [1].

[1] https://proceedings.mlr.press/v162/croce22a.html
paper_summary: paper_summary: This paper proposes a few-shot adversarial training methodology for vision-language models to ensure robustness in downstream tasks of pretrained vision-language models. Specifically, instead of using traditional adversarial training methods, it adapts the TRADES loss, a prominent adversarial training approach, for vision-language models. Additionally, it leverages the structural characteristics of vision-language models by adaptively adjusting the weight of the regularization loss, enabling the model to effectively learn robustness in few-shot learning scenarios. As a result, the proposed methodology allows vision-language models to easily acquire robustness through few-shot learning, demonstrating high performance.
strength_weakness: strength_weakness: - The motivation does not seem sufficient. Among the drawbacks of previous methods raised by the authors, it is not clearly conveyed how (1) and (2) are addressed and resolved later in the paper. Instead, it might have been better to emphasize in section 3.5 that existing methods or simply applying TRADES can lead to small perturbations that do not promote robust learning, which can be particularly disadvantageous in the constrained context of few-shot learning. For (3), if the issue is that the existing clean accuracy is compromised by training only with adversarial examples during the few-shot learning process, a baseline experiment analyzing the performance when both clean and adversarial examples are used together might have been more insightful.
- There is a lack of analysis regarding the proposed Uni-Modal Adversarial-Aware Mechanism. It is unclear why this method is implemented as a multiplicative factor to the KL divergence loss rather than adding a separate loss. The reasons behind the results shown in Figure 2 using this learning method are not clearly explained. While I understand that it is advantageous for learning if clean and adversarial examples are distinctly separated in the embedding space, I am concerned that if clean and adversarial examples are separated in this manner in the final trained model, it might actually reduce robustness.
- The analysis of the experiments is insufficient. Although the proposed methodology showed an increase in the overall average performance on the tested datasets, some datasets exhibited similar or even decreased performance compared to existing methods. A deeper consideration and explanation of these observations are necessary.
questions: questions: Please provide answers to the mentioned points. Additionally,
- Were the adversarial examples in Figure 2 generated using PGD or were they generated by maximizing the KL divergence loss?
- Are there any comparative experimental results based on full fine-tuning?
limitations: limitations: Mentioned above

Comments: title: Thank you for the reply
comment: Dear Reviewer KV22，

Thank you for your time and effort in reviewing our paper. We are grateful for your feedback and pleased to hear your positive remarks!

Best regards,

Authors of #1695
thanks for addressing my questions and helping to improve the presentation of the paper. I updated the rating to Borderline accept given the authors' response.
Dear reviewer M4S3:

Thanks for your feedback and claiming that we have clarified your questions and the results on the C&W attack are impressive. We will take your advice and reflect it in the final version of this paper. 

Warm regards,

Author of #1695
Thanks for your updates. You have clarified my questions. Ad 4) Please, be careful when using the word “adaptive”, because it does not mean that the APGD attack is not adaptive at test-time. Your results on the C&W attack are impressive.
Dear Reviewer KV22,

Thanks for your thorough review of our paper. We have made every effort to address the concerns you raised. As the deadline for discussions between reviewers and authors is approaching, we would like to confirm whether there are any explanations or descriptions in our response that remain unclear. We are prepared to provide further clarification if needed.

Warm regards,

Authors of #1695
Dear Reviewer Rdwb,

Thank you for taking the time to review our paper. We appreciate your effort and are delighted to receive positive feedback from your comments！

Best regards, 

Authors of #1695
Thank you for the response. Most of my concerns and questions have been resolved. It seems that the drawbacks of previous works that served as motivation were broadly addressed by the methodology, making it difficult to distinguish them clearly. I appreciate the confirmation experiment on the uni-modal loss. While I wonder why TRADES has only now been applied to this task, it seems likely that this paper is the first to successfully adapt it to this task. This appears to be a commendable contribution. Accordingly, I raised my score.
Thank you for your reply. Your comments really help improve this work.
My concerns are answered.