ADAPT to Robustify Prompt Tuning Vision Transformers
Authors: Masih Eskandar
Tooba Imtiaz
Zifeng Wang
Jennifer Dy
Venue: Accepted by TMLR ()

Abstract: The performance of deep models, including Vision Transformers, is known to be vulnerable
to adversarial attacks. Many existing defenses against these attacks, such as adversarial
training, rely on full-model fine-tuning to induce robustness in the models. These defenses
require storing a copy of the entire model, that can have billions of parameters, for each task.
At the same time, parameter-efficient prompt tuning is used to adapt large transformer-
based models to downstream tasks without the need to save large copies. In this paper,
we examine parameter-efficient prompt tuning of Vision Transformers for downstream tasks
under the lens of robustness. We show that previous adversarial defense methods, when
applied to the prompt tuning paradigm, suffer from gradient obfuscation and are vulnerable
to adaptive attacks. We introduce ADAPT, a novel framework for performing adaptive
adversarial training in the prompt tuning paradigm. Our method achieves competitive
robust accuracy of ∼ 40% w.r.t. SOTA robustness methods using full-model fine-tuning, by
tuning only ∼ 1% of the number of parameters

Published in Transactions on Machine Learning Research (01/2025)
ADAPT to Robustify Prompt Tuning Vision Transformers
Masih Eskandar
ekamddr..mChhorreusseee....
Department of Electrical & Computer Engineering
Northeastern University
Tooba Imtiaz
imtiaz.t@norie wwwoorteeeettetr.ccc.. edu
Department Electrical & Computer Engineering Northeastern University
Zifeng Wang
zifenguonng@ece.ee.cco
Department of Electricnl & Computer Engineering Northeastern University
Jennifer Dy
jdy@ece. edu
Department of Electrical & Computer Engineering Northeastern University
Reviewed on OpenReview: https: // opt.r/viee....ew...mmmmm net/ farum?? id= bZaXgheUSD
Abstract
The performance of deep models, including Vision Transformers, known to be vulnerable to adversarial attacks. Many existing defenses against these attacks, such as adversarial training, rely on full-model fine-tuning induce robustness in the models. These defenses require storing copy of the entire model, that can have billions of parameters, for each task. At the same time, parameter-officient prompt tuning is used to adapt large transformer- based models to downstream tasks without the need to save large copies. In this paper, we examine parameler-efficenn prompt luning of Vision Transformers for ream tasks under the lens of robustness. We show that previous adversarial defense methods, when applicd to the prompt tuning paradigm, suffer from gradient obfuscation and are vulnerable to adaptive allacks. We introduce ADAPT, a novel framework for performing adaptive adversarial training in the prompt tuning paradigm. Our method achieves competitive robust accuracy of ~ 40% w.r.t. SOTA robustness methods using full-model fine-tuning, by tuning only ~ 1% of the number of parameters.
1 Introduction
Despite their success, deep neural networks, including the popular ViT architecture, have been shown to be vulnerable to adversarial attacks (Szegedy et al. 2014 Mahmood et al. 2021). These attacks are modifica- tions to input images, which are generally imperceptible to humans but can drastically change the prediction of neural network. Many countermeasures have been introduced to induce robustness against such attacks, one common approach being adversarial training (Madry et al.] 2018), where the input is augmented with said perturbations during training. While originally introduced for CNNs, adversarial training has been extended to ViTs in the full-model fine-tuning paradigm (Mo et al. 2022). However, adversarial training and similar mothods are computationally expensive and difficult. to perform for large datasets as it requires performing multi-step evaluation of adversarial examples. Hence ViTs are generally pre-trained without such considerations.
Transformer-based language models (Brown et al. 2020) which can consist of billions of parameters require pretraining on massive amounts of data. To fine-tune such models for different tasks, one would have separate copy of the weights per task, making it highly inefficient in terms of parameters. However,

---

Published in Transactions on Machine Learning Research (01/2025)
it is shown that these models can be adapted to downstream tasks with fewer parameters, for example using prompt design (Brown et al. 2020) prompt tuning (PT) (Lester et al. 2021). Tn prompt tuning, optimizable tokens are prepended to the model input in order to leverage the pretraining knowledge and adapt to the downstream task without changing the weights of the model. This allows for a lightweight
adaplation of these large models. ViTs can benefit. from the samne technique (Jia et al. 2022 especially as their paramoters continue to increase (Dehghani et al., 2023
It is therefore important to explore the utility of prompt tuning for the adversarial robustness of ViTs. In this paper, we show that existing adversarial defense methods, when applied to the prompt tuning paradigm, suffer from gradient obfuscation (Athalye et al. 2018). This means the gradients of the model are not useful for obtaining adversarial perturbations, and training using existing methods will lead to a false sense of security. We demonstrate this phenomenon empirically using single-step attacks, and then design an adaptive attack expose the vulnerability of existing methods. Finally, we propose a framework for Daptive Adversarial Prompt Tuning (ADAPT) to overcome this vulnerability. We empirically demonstrate the superior robustness of our method in multiple settings.
To the best of our knowledge, are the first to study this issue extensively, A summary of our contributions is as follows:
We investigate the adversarial robustness of the prompt tuning paradigm for ViTs. We show that the existing methods, when applied to the prompt tuning scenario, suffer from gradient obfuscation (Athalye et al. 2018).
We design an adaptive attack accordingly and show that existing methods are significantly vulnerable to our attack, resulting in a drop in their robust accuracy to approximately 1% (See fig, D
We propose a novel loss function that emphasizes conditioning on the prompts craft adversarial examples during training. We quantitatively demonstrate the superiority of our method (ADAPT), in achieving robustness compared to existing methods.
Our implementation is available on Githut H
2 Related Works
Vision Transformers. Inspired by the groundbreaking success of Transformers for Natural Language Pro- cessing (NIP) tasks (Vaswani et al., 2017 Kenton & Toutanoval 2019 OpenAT 2023 Vision Transformers (ViTs) (Dosovitskiy et al. 2020) are an adaptation of the architecture for computer vision tasks. Similar to the 'tokens' fed to NLP transformers, here the input images are split into fixed-size patches, which are then embedded and fed to the transformer layers of the VIT. ViTs are powerful architectures and achieve
state-of-the-art performance on several downstream vision tasks (Tuvvonn et al. 2021 Caron et al. 2021
He et al. 2022).
Prompting. This technique was introduced in the NLP literature to eliminate the need to fine-tune large pro-trained language models for downstream tasks (Liu et al., 2023 Initially, Prompting was used finc- tune the behavior of language models by pre-pending a set of fixed prompt templates to the input text. Intuitively, prompting reformulates the downstream task that it closely resembles the task that frozen model has been pre-trained to solve. For example, Gao et al. (022)) use prompting for few-shot learning of language models. Related to this work, Raman et al. (0223) investigate the effects of adversarial robustness of prompting language models. However, these works are fundamentally specific to the NLP literature, as they require pre-determined prompts, and investigate the problem in the text input domain which is discrete. The extension of fixed prompt templates to the vision space, where inputs are continuous is non-trivial. To that effect, prompt tuning was introduced [esteer et al., 2021) which uses continuous tunable vectors in the embedding space as prompts. Prompt-tuning, i.e., training the prompts for downstream tasks, has been extended to ViTs and is highly effective for domain generalization (Zheng et al. 2022 continual lcarning Wann et al. 2022,, and several other vision lasks.
2

---

Published in Transactions on Machine Learning Research (01/2025)
10-
có (traditiona)) ed, (adaptive)
ed (traditional)
eb, (adaptive)
8
8
Cross Entropy Loss
Cross Entropy Loss
6
6-
2-
2-
0.0
0.5
1,0
1.5
2.0
0.0
0.5
1.0
1.5
2.0
(a) Adversarial Training
Prompt Tuning
(b) ADAPTCE + Prompt Tuning
Figure 1: Existing methods do not exhibit robustness to adaptive attacks. Comparison of the cross entropy loss of random batch along two adversarial perturbation directions. Left (a) depicts the loss values of prompt trained with traditional adversarial training and right (b) shows the loss values of prompt trained with ADAPTсE- Adversarial Training + Prompt Tuning does not exhibit a significant increase in the loss under the traditional PGD direction, but shows significant vulnerability to adaptive attacks. contrast, the proposed method ADAPTCE, exhibits robustness to both perturbation directions.
Adversarial Robustness. Adversarial Training (AT) was first introduced for CNNs and was thoroughly investigated (Szegedy et al. 2014 Goodfellow et al.) 2015 Madry et al. 2018) with several variations (Buck- man et al. 2018 Zhang et al. 2019 2020). ViT's have also been shown to be vulnerable to adversarial attacks [Mahmood et al., 2021p. However, AT and its variants are computationally expensive due to requir- ing multiple forward and backward passes to obtain adversarial examples to train with. This makes them difficult use for pre-training ViI'. Therefore, adversarial defense methods are typically only adopted during fine-tuning for downstream tasks (Mo et al. 2022).
Robust Prompts. Prompt tuning has been employed for enhancing the robustness of NLP transformers (Yang et al. 2022], which deal with discrete inputs. Extending the same robustness to the continuous input space for ViTs is non-trivial and has not been explored in the literature.
There are other works which explore robustness of visual prompt tuning (Chen et al. 2023a However, they are tailored the transformer paradigm. Moreover, their number of tuned parameters grows with the number of classes in the dataset. Visual prompting adversarial reprogramming was proposed before the
advent of prompt tuning for adapting CNNs to downstream tasks Elsayed et al. 2019 As such, are the first to investigate robust prompting for vision transformers.
3
Notations and Preliminaries
3.1 Vision Transformer
The Vision Transformer (Dosovitskiy et, al. 2020) has proven to be quite effective at performing various computer vision tasks. For our purposes, we consider the classification task. transformer fo.v as a classifier consists of a feature extractor and a linear classifier vi operating on the extracted features. That is,
Φ x Rdd v Rd DM (..0()) softmar (V(
(1)
(2)
where is the input space, the label space, | the number of classes, and is the hidden dimension of the transformer (which stays the same throughout its layers). As the transformer architecture operates
3

---

Published in Transactions on Machine Learning Research (01/2025)
imputs that are sequences of sigh-dimensional vectors, we consider the input space of the transformer to be the space of all d-dimensional (the hidden dimension of the transformer) sequences of arbitrary length A:
x== = U Ra
VkEN
As such, to transfer images from the data distribution support, iee. R4xwxe, to the transformer input space X, the image is divided into n non-overlapping patches and then transformed via linear transformation or through convolutional neural network (CNN). Formally, we have
T wxxxxx + Rd
(4)
as the embedding function. For the sake of brevity, except in sections where the embedding function being discussed directly, we omit T' from our notation and assume that inputs are being passed through it.
3.2 Prompt Tuning
With the advent of transformers and large language models with billions of parameters pre-trained on extremely large amounts of data, it is desirable to tune these models for downstream tasks at minimal cost. While full-model tine-tuning/ is effective option, it requires training and storing an entirely new set of model weights for each downstream task, which becomes inefficient in terms of memory as both the number of tasks and the number of model parameters grow. A simple yet effective solution to this is prompt tuning [Lester et al. 2021), which proceeds by simply appending tunable prompt tokens to the transformer imput space. Depending on the model size and number of tokens, it can yield comparable performance to full-model fino-tuning. This allows us to adapt to downstream tasks without having to store copics of extremely large models, and instead only storing parameter-effiiien prompls.
Formally, pre-trained Transformer f4-. .v- is obtained by training large amounts of data, í.e.,
Φ V arg min (xy-r[((Je.((t).))
For supervised learning, this can be solved using Empiriaal Risk Minimisation (ERM), where P is a data distribution used for pre-training, e.g. ImageNet, and C is the loss function used for training. The goal is then to adapt to a downstream data distribution D given 4*.
Full-model fine-tuning (which we will refer to Fine-Tuning (FT)) seeks to find some a & to minimize the expected loss Ez,y~D L(f&, & (x), v) by using &' a starting point for ④. Prompt tuning, instead, seeks to solve the following optimization problem
0p. v
argiinn Exy-D L(f(Op. z),
(6)
0 W
Prompt tuning can be performed in multiple ways. In the case of simple Prompt Tuning (PT), a set of prompts is prepended to the input layer of the transformer. prompted VIT ((0p.z) is defined as follows
S(0p) x) = So (10p,()) 0, E Rd
(7)
(8)
where [.,.] is the concatenation operator, d is the hidden dimension of the transformer, and is the prompt length (recall that the input space of the transformer is arbitrary length sequences). Another way to perform prompt tuning is Prefix tuning LLi & Liang, 2021) PT-v2 tuning (PT2) (Liu et al. 2022). where similar to PT, the backbone weights are kept unchanged. However, set of prompt tokens are instcad directly prepended to the input of every layer of the transformer, by only prepending to the key and value matrices of the self-attention mechanism (while keeping the same number of tokens per layer). That is, x(),,)) in the case of PT2 is as follows
[(0p,)) = (1))
(9)
0p E 12 xxxxm
(10)
4

---

Published in Transactions on Machine Learning Research (01/2025)
where [0p: ( is shorthand for prepending tokens to the imput of every layer of the frozen feature extractor Φ*, I is the number of transformer layers and m is the prompt length, i.e. the number of tokens tuned per layer.
3.3 Adversarial Robustness
Given an arbitrary classifier f: X + 3/ input E X' with the true label y V, an adversary seeks to find some imperceptible perturbation with the same dimensionality as such that x + 8 E X and f(z + # To achieve such imperceptibility, 5 is typically constrained w.r.t. an lg norm, the most popular choices being
= and oo.
An untargetted adversarial attack for classifier f bounded in ly norm by can be defined the following way
x' = x + orgmax L(f(z + 6)
(11)
The adversarial example defined above is an optimization problem that is typically solved using methods such as Projected Gradient Descent (PGD) (Madry al., 2018). As doep learning models are highly susceptible to such norm-bounded perturbations (Szegedy et al. 2014), is crucial to defend against these attacks.
However, it is important to note that PGD assumes full knowledge of the model parameters and inference algorithm, i.e. white-boz setting, for the adversary. Adversarial robustness can also be investigated through the lens of adversarics with limited knowledge, namely the black-bol (Ilyas et al.] 2018), and grey-boz (Xu et al. 2021 settings. In this work, we focus on the white-box scenario.
One widely used defense method against adversarial attacks is Adversarial Training (addr et al. 2018 If So is a classifier parameterized by 0, AT tries to solve the following minimax optimization problem:
mén E 0
max [(Se(1 + +0, [1
(12)
Practically, this objective trains the model adversarially perturbed inputs with untargeted attacks.
3.3.1 Obfuscated Gradients.
An adversarial defense method suffers from gradient masking or obfuscated gradients (Athalye et al. 2018 when does not have "useful gradients" for generating adversarial examples. Typically, there are three main causes for gradient obfuscation: exploding and vanishing gradients, stochastie gradients, and shattered gradients. We show that traditional adversarial defense methods in the prompt tuning paradigm suffer from shattered gradients. This means that the gradient of the model w.r.t. the input is incorrect and does not properly maximize the classification loss. To validate this claim, we use the observation of (Athalye et al. 2018) that in the presence of obfuscated gradients, single-step atlacks are typically more effective than their multi-step counterparts.
4 Robust Prompt Tuning
In this section we introduce ADAPT, a novel framework for robustifying the parameter efficient prompt tuning of ViTs. We design an adaptive attack tailored to the prompt tuning paradigm by conditioning on the prompts during each gradient step. Using our adaptive attack formulation, we propose a novel loss function for adaptive robust training of prompts while fine tuning for downstream tasks.
4.1 Adaptive Adversarial Attack
In the prompt tuning scenario, Vaf(x) does not necessarily equal V..f(0,, Therefore, the existence of the prompt may lead to a form of gradient obfuscation (Athalye et al.) 2018), giving us a false sense of security (See fig. Tap. In the experiment section, we demonstrate this effect. by designing adaptive attack
5

---

Published in Transactions on Machine Learning Research (01/2025)
and empirically show that while previous adversarial training methods may be robust against traditional attacks, they are vulnerable to our adaptive attack which reduces their robust accuracy to nearly zero.
In the prompt tuning paradigm, we reformulate the optimization objective for finding an adversarial example for classifier f as follows
= + argmaz LoE(f(0p; 8p).
(13)
16,1le
We can solve this optimization problem using Projected Gradient Descent (Madry et al. 2018) similar to eq. 11 Here, the key difference lies in conditioning on the prompt during cach gradient step.
4.2 Prompt Adversarial Training
To perform adversarially robust prompt tuning, conditioning on the prompt is crucial for both generating the adversarial examples, as well as training the prompt itself.
As such, propose a loss function for ADaptive Adversarial Prompt Tuning (ADAPT)
CDAAPT = Lp(f(Op: 2). y) + Ma
(14)
where Lp is the standard prompt tuning loss for any downstream task, and Lods is the adversarial loss function which we will define below. Intuitively, the first term performs parameter-effiiient fine-tuning by leveraging the information present the frozen backbone, while the second term promotes robustness.
In this paper, we propose different choices for Ladvi
Cross Entropy (CE). We can choose the cross-entropy loss for the prediction of the adaptive adversarial example:
Ladu = LCF( +4).
(15)
which results in loss function that promotes correct classification of samples perturbed with the adaptive attack.
KL Divergence. We can choose Ladv to be the KL-divergence between the prediction distributions of the unperturbed and perturbed samples:
Ladv =x) ((0,,
(16)
With this formulation, the first term in eq. 14] promotes the correct prediction of unperturbed samples, and the adversarial term eq. (16 matches the prediction distributions of unperturbed and perturbed samples.
4.3 Additional Design Choices
Prompting the feature extractor. Prompt tuning can be performed in multitude of ways by changing the definition of f(pp. z). In experiments, observe that for adversarial robustness, using PT2 (other- wise known as Prefix Tuning) significantly outperforms using only PT. This behavior is consistently observed when using the same training scheme and total number of tuned parameters across the two methods. We hypothesize that this performance gap exists since the feature extractor d- the frozen backbone is not adversarially robust. Since PT simply prepends additional tokens only the input of the frozen feature extractor, this results the same function 4' on different set of inputs. However, PT2 prepends prompt tokens directly in each layer of the feature extractor, resulting in modified feature extractor $

---

Published in Transactions on Machine Learning Research (01/2025)
Tuning the embedding layer. Traditionally, the prompt tokens are introduced after the embedding function T and do not influence the mapping of the image patches to the embedding space. That is, the patch embedding remains unchanged by prompt tuning, which can cause vulnerability. Since the patch embedding does not have a significantly large number of parameters compared to the frozen backbone, we opt to lune the patch embedding parameters T along with the prompt lokens 0g and the linear classifier head V.
Algorithm shows pseudocode of our training algorithm. We provide code supplementary material, and we will release an open-source version of our code upon acceptance.
Algorithm ADAPT training step
Require: data pair æ, y, frozen classifier f. tunable prompts Bp, attack step size a, maximum perturbation magnitude 6, number of attack steps 8, learning rate ?) D U is the uniform noise distribution
r/ 4 + (--,e)))
for i = + - do d' x +a (o, z2)))) y + --rroeect p; 6, +
> eq. 13
b Project oy onto the c ball
p
end for
L < LADAPT
eq. 14)
0, 0g + Vo,
Optimizer gradient step
5
Experiments
We evaluate methods benchmark datasets with foeus increasing adversarial robustness while keeping the number of tuned paramcters relatively low. When using prompt tuning, we demonstrate the gradient obfuscation occurring in previous methods, and show that our proposed method overcomes this issue. Finally, we perform ablation studies on the different components of our method. To the best of our knowlodge, we are the first to explore the robustness of the prompt tuning paradigm for ViTs.
Ii is imperative to note that the advantage of the prompt luning paradigm the adaplation of a single frozen backbone to different tasks by using a small number of additional parameters per task. For example, if model was required for a hundred different downstream tasks, traditionally we would have to save and load hundred fully fine-tuned copies of the large backbone with hundreds of millions of parameters for each task. Instead, prompt tuning requires training only fraction of the original parameters per task to adapt a single frozen pre-trained model for each downstream task. With that said, prompt tuning does not benefit the time required for training or inference for the downstream tasks as the prompts are used in conjunction with the entire frozen backbone, This is true for any method that adopts the prompt tuning paradigm and not just our scenario. In any case, we report training time details in the appendix.
5.1 Experimental Setting
Datasets. We perform experiments on (OTAR--O et al. ab, CIFAR100 Krizhevsky et al. b). and Imagenette (Howard). Note that these datasets are widely employed benchmarks for evaluating adver- sarial defenses since training and evaluation on ImageNet using multi-step gradient-based attacks requires significant computational time and resources.
Adversarial Robustness Methods. We compare ADAPT to the following state-of-the-art methods es- tablished in the literature:
Adversarial Training Madry et al. 2018:: As described in eq. 12 Adversarial Training solves a minimax optimization objective by augmenting the inputs during training using traditional adver- sarial attacks.

---

Published in Transactions on Machine Learning Research (01/2025)
Table 1: Test set accuracy of previous adversarial robustness methods combined with prompt tuning against unperturbed imputs, traditional adversarial attacks, PGD10 (multi-step) and FGSM (single step) as well as our adaptive attack. All methods perform worse against FGSM than PGD10, clear sign of gradient obfuscation. Additionally, all methods show very little robustness the adaptive attack (Adaptive PGD10). indicates that the attack in question is non-adaptive while" indicates an adaptive atlack eq. (13
Params
Method
Clean
PGDt
FCSM
PGD*
PT PT2 + Emb PT PT2 + Emb
93.21
87.62
78.92
1.15
AT
94.84
90.3
82.49
3.04
TRADES TRADES
89.79
84.87
79.65
1.01
92.64
89.65
85.93
3.17
TRADES (Zhang et al. 2019: TRADES identifies trade-off between robustness and accuracy. designs a defense method accordingly, which performs training with a modified loss function, performing natural training and at the same time reducing the prediction distance of unperturbed samples with that of samples perturbed to maximize the same distance.
MART builds on top of TRADES and emphasizes on misclassified examples during training to improve robustness.
MART (Wang et al. 2019;
NFGSM (de Jorge Aranda el al. 2022) NFGSM proposes to use a high-maguilude noise perturbation as an initialization point for generating adversarial attacks during single-step adversarial training avoid overfitting
Architectures and Tuned Parameters. We perform most of our experiments on the ViT-B model. For prompt tuning (PT), we keep the number of tuned parameters within ~ 1% of the total model parameters. We provide additional results on the ViT-S and ViT-I model configurations.
As discussod in design choices, due superior performance, we perform most experiments with PT2 and tune the patch embedding as well as the lincar classifier (all of which we take into account into the number of tuned parameters). We also perform ablation studies on PT while freezing the patch embedding layer.
Evaluation Metrics. We evaluate each method with regards to accuracy on the test set. We evaluate on umperturbed samples, also known as clean accuracy, as well as perturbed samples. In one experiment, as mentioned, we use traditional adversarial attacks as well as adaptive ones to validate our claims of gradient obfuscation. In all other sections, excluding the black-box attacks section, our attacks are white-bon and adaptive ensure a fair worst-casc evaluation. We use adaptive PGD10 eq. (13), adaptive CW (Carlini & Wagner 2017), which is obtained by replacing the loss in eq. 13 with the CW loss, well as AutoAttack (AA) (Croce & Hein 2020), ensemble of diverse and parameter free white-bax and black-box attacks. Finally, evaluate against two state of the art blackbox attacks: RayS (Chen & Gu 2020 and AdaEA CChen et al. 2023b).
5.2
False Sense of Security
In this section, we evaluate the robustness of existing methods, combined with prompt tuning, against traditional and adaptive attacks. The results presented in table [ and are consistent with what we discussed in adaptive attacks section. The results show that using traditional adversarial examples and adversarial training can lead to gradient obfuscation (Athalye et al.) 2018), and false sense of security. Indeed, existing adversarial defense methods initially exhibit high performance when used with prompt tuning. However, as per (Athalye et al.) 2018), a sign gradient obfuscation is that single-step gradient- basod attacks are more effective than their multi-stop counterparts. As expected, previous methods are more susceptible single-step FGSM attacks than multi-step PGD attacks and show significant vulnerability our adaptive attack.
8

---

Published in Transactions on Machine Learning Research (01/2025)
5.3 Adaptive Adversarial Training
We experiment with prompt tuning using previous SOTA adversarial delense methods and compare them to ADAPT. We present the results in table 2 The results show that while previous methods show little to no robustncss against adaptive adversarial attacks, ADAPT achieves competitive accuracy on unperturbed samples while showing considerable robustness to adaptive adversarial attacks. We provide additional results for different model sizes, ViT-T, and ViT-S, in the appendix, the results are consistent with that for VYT-B.
We also evaluate our methods on Imagenette (Howard), which is a subset of ImageNet with 10 classes, and CIFAR100 (Krizhevsky et al. b). which consists of 100 classes of 32 32 images. Results are presented in table and table respectively, and show consistency with the CIFAR10 experiments.
Note that for fair comparison, we perform PT2 with the same number of tokens and tune the patch embedding and the linear classifier head for all methods, resulting exactly the same number of tuned parameters. We include the results of full model fine-tuning (FT) as an upper bound on performance, which requires tuning all the ViT parameters. We conduct experimentation with PT and the effects of training the patch embedding in the ablation study (lable 7 The results confirm the eflectiveness of our design choices. Furthermore, investigate the impact of the number of tuned parameters in the following sections.
Test set accuracy on the CIFARIO dataset for different methods. The upper section corresponds to methods using prompt tuning, and the lower section corresponds to FT upper bound on performance. Existing methods show little to no robustness to adaptive attacks. In the prompt tuning scenario, our method shows substantial improvement in robustness compared to previous methods. indicates that the attack in question is adaptive (eq. 13
Table 2:
Method
Params.
Clean
PGD10"
CW"
AA
# params
AT
PT2 Emb
94.84
3.04
0.67
1.7
820K
TRADES
PT2+ Emb
92.64
3.17
1.85
4.2
820K
MART
PT2+ Emb
52.28
12.1
9.62
15.25
820K
NFGSM
PT2+Fmm
77.95
0.1
0.6
2.03
820K
ADAPTCR
PT2+ Emb
79.05
38,27
35.94
19.90
820K
ADAPTK&
PT2++Em E-bb
68.43
40.21
35.9
18.05
820K
AT
FT 82.42
53.41
46.67
34.77
86M
TRADES
FT
84.02
54.19
47.45
36.48
86M
Table 3: Test set accuracy on Imagenette
Method
Params
Clean
PGDI0
CW
# params
TRADES
PT2 Emb
89.32
9.04
6.82
7.08
820K
MART
PT2+Emb
44.2
17.35
14.03
19.36
820K
NEGSM
PT2 Emb
71.82
2.08
2.08
5.68
820K
ADAPTCE
PT2 Emb
59.95
31.34
28.25
820K
ADAPT
PT2 Emb
63.16
37.61
34.01
820K
TRADES
FT
82.4
53.4
43.6
16.33
86M
Table 4: Test set accuracy on CIFAR100
Method
Params
Clean
PGD10
CW
AA
# params
TRADES
Pmb
78.84
5.54
3.52
3.4
820K
MART
PT2+Emb
29.29
6.59
4.99
8.5
820K
NFGSM
Pmb+Emb
57.91
0.6
0.9
1.67
820K
ADAPTсE
PT2 | Emb
60.2
22.15
19.59
7.92
820K
ADAPT
P124 Emb
50.63
21.77
17.56
5.94
820K
TRADES
FT
65.5
32.89
28.93
9.99
86M
9

---

Published in Transactions on Machine Learning Research (01/2025)
5.4 Evaluation against Blackbox Attacks
In this section, evaluate the robustness of ADAPT and existing methods against black-box attacks in the prompt-tuning paradigm. We evaluate against two SOTA black-box attacks: one query-based attack, RayS (Chen & Gu, 2020) and one transfer-based ensemble attack, AdaEA (Chen et al.) 2023b). Table 5 shows that ADAPT is robust to both the query-based and consfer-based attacks, while AT and TRADES (with prompt tuning) only show robustness the transfer-based attack and are not robust to the query-based attack. We hypothesize that the rcason is AT and TRADES (with prompt tuning) are trained on attacks generaled using the gradient of the naturally trained frozen VIT. The transfer-based atlack also relies on the gradients of naturally pre-trained models to construct its attack, while the query-based attack eliminates the need to use gradients. This aligns with our discussion of gradient obfuscation. Furthermore, against existing defenses, both black-box attacks are significantly more successful than white-box non-adaptive attacks (see table 2 which is another sign of gradient obfuscation in existing defenses when using prompt tuning.
Table 5: Test-set accuracy against black-box attacks, same training settings as Table 2.
Method
Params
RayS
AdaFA
AT TRADES
P2T++Emb
5.06
66.55
P2T+Emb
11.71
66.27
ADAPTсE
P2T Emb
43.05
58.21
ADAPTKΣ
P2T Emb
40.09
52.06
TRADES
FT
56.78
61.73
5.5 Beyond the Number of Parameters
We notice in our experiments that for adversarial training under the same conditions, PT2 significantly outperforms PT. We hypothesized that this is due to the fact that PT is simply augmenting the input to the frozen feature extractor Φ+ of the transformer which is vulnerable to adversarial examples, while PT2 inserts tokens in the intermediate layers of the feature extractor, modifying it directly. We perform an experiment to validate our intuition and confirm that the superior performance of PT2 over PT, not simply due an increased number of parameters (as with a fixed number of tokens per layer, PT2 requires more tuned parameters).
We consider set of experiments with fixed number of tuned tokens, and we gradually change the scenario from PT to PT2 in discrete steps as described below. We tune a fixed number of M tokens, prompting the first h layers of the transformer with m tokens each, with varying numbers of is and m while keeping m x M.
For example, we consider a pool of 96 tokens. In the first experiment, we perform training and testing while only prompting the first layer with all 96 tokens (which is equivalent to PT). In the following experiments, we perform training and testing while prepending to more and more layers, c.g. the first two layers with 48 tokens, and so on, until prepending to all 12 layers each with 8 tokens (which is equivalent to PT2). Tn all of the above experiments, the same number of tuned parameters is used (96 tokens, plus the classifier head).
The results are presented in table G and show a substantial increase in performance as we prompt more and more layers despite using the same number of tokens. This further validates our insight about needing modify the feature extractor and shows that the performance improvement not simply due to an increase in the number of tuned paramcters.
6 Conclusion
We formulated and extensively analyzed the adversarial robustness ViIs while performing parameter- efficient prompt tuning. We found that the application of existing adversarial defense methods to the prompt tuning paradigm suffers from gradient obfuscation. This leads a false sense of security and vulnerability to adaptive attacks, which we demonstrated through experiments with single-step and adaptive attacks. To
10

---

Published in Transactions on Machine Learning Research (01/2025)
Table 6: Test set accuracy and the number of tuned parameters of different combinations. m x ko means we prompt the first k layers directly with m tokens each. Each setting adopted during both training and testing. We measure against unperturbed samples and the adaptive PGD10 attack.
Combination
Clean
PGD10
# params
96x1
27.33
18.34
81K
48x2
32.74
21.07
81K
24x4
40.03
24.83
81K
16x6
40.78
25.49
81K
12x8
41.14
25.68
81K
8x12
40.24
25.17
81K
Table 7: Ablation study on different prompt tuning scenarios on the CIFAR10 dataset. Performance is measured by test set accuracy againt unperturbed samples and samples perturbed by adaptive PGD10.
Method
Params
Clean
PGD10
# params
ADAPTcE
PT
44.72
15.24
19K
ADAPTAL
PT
39.77
9.54
19K
ADAPTCE
PT2 PT2
67.
26.75
230K
ADAPTKL
58.4
29.25
230K
ADAPTcE
PT+Emb PT+Emb
67.8
29.44
600K
ADAPTAL
56.48
30.31
600K
ADAPTCE
PT2 Emb
79.05
38.27
820K
ADAPTKL
PT2+Emb
67.24
39.23
820K
address these problems, proposed novel loss function for adaptive adversarial training of prompt tuning methods. We evaluated our method on various datascts and model sizes, and performed an ablation study on the components of our framework. We empirically demonstrated the superior adversarial robustness of our method while performing parameter-efficient prompt tuning.
7 Broader Impact Statement
Adversarial perturbations can be used fool machine learning models make decisions based on the ad- versary's intent which be malicious. For example, in facial recognition for security systems, an adversary may try to pose someone else to breach the system. In self-driving cars, adversary can change the classification of traffic signs or objects to alter the course of the car and cause deadly crashes. As such, it is imperative to explore and improve the robustness of machine learning models. In this work, we showcase the significant. vulnerability of previous adversarial defense methods to adaptive adversarial attacks under the prompt tuning paradigm. To this end, we provide a framework for adaptive adversarial training of prompts combat vulnerability to adaptive attacks. This makes models using the prompt tuning paradigm more robust to adversaria manipulation.
8 Acknowledgments
This project was supported by NIH grants RO1CA240771 and U24CA264369 from NCI. We would like to thank the TMLR editors and reviewers as well as our colleagues and friends for helping improve this work.
References
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense of sccurity: Circunventing defenses to adversarial examples. In International conference on machine learning, pp. 274-283. PMLRR 2018.
11

---

Published in Transactions on Machine Learning Research (01/2025)
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakanian, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877 1901, 2020.
Jacob Buckmaa, Aurko Roy, Colin Raffel, and lan Goodfellow. Thermometer encoding: One hot way to resist adversarial examples. In International conference on learning representations, 2018.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp), pp. 39-57. Iece, 2017.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the International Conference Computer Vision (ICCV), 2021.
Aochuan Chen, Peter Lorenz, Yuguang Yao, Pin-Yu Chen, and Sijia Liu. Visual prompting for adversar- ial robustness. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5. IEEE, 2023a.
Bin Chen, Jiali Yin, Shukai Chen, Bohao Chen, and Ximeng Liu. An adaptive model ensemble adversarial attack for boosting adversarial transferability, In Proceedings of the LEEE/CVE International Conference on Computer Vision, pp. 4489-4498, 2023b.
Jinghui Chen and Quanquan Gu. Rays: ray searching method for hard-label adversarial attack. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1739 1747, 2020.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-fre free attacks. In International conference on machine learning, pp. 2206-220622 2216. PMLR, 2020.
Pau de Jorge Aranda, Adel Bibi, Riccardo Volpi, Amartya Sanyal, Philip Torr, Grégory Rogez, and Puneet Dokania. Make some noise: Reliable and efficient single-step adversarial training. Advances in Neural Information Processing Systems, 35:12881 12893, 2022.
Mostafa Dehghani, Josip Djolonga, Basil Mustafa, Piotr Padlewski, Jonathan Heek, Justin Gilmer, An- dreas Peter Steiner, Mathilde Caron, Robert Geirhos, Ibrahim Alabdulmohsin, et al. Scaling vision transformers to 22 billion parameters. In International Conference on Machine Learning, pp. 7480-7512. PMLR, 2023.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Un- terthiner, Mostafa. Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXic:2010. 11929, 2020.
Gamaleldin F. Elsayed, Ian Goodfellow, and Jascha Sohl-Dickstein. Adversarial reprogramming of neural networks. In International Conference on Learning Representations, 2019. URL https::/openrevver. htt.f/ormmiddSyee SS00tmmm0mmmmmmm
Tianyu Gao, Adam Fisch, and Danqi Chen. Making pre-trained language models better few-shot learners. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference Natural Language Processing (Volume 1: Long Papers), pp. 3816-3830, Online, August 2021. Association for Computational Linguistics. doi: 10. 10.18653/w1/2022 lon- 295. URL https::/accanthhllll org/
1021.acl-lona 295/
lan J. Goodfellow, Jonathon Shlens, and Christian Szegedy, Explaining and harnessing adversarial examples. In Yoshua Bengio and Yann LeCun (ods.), 3rd International Conference on Learning Representations, ICLR 2015, Son Diego CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL http: atexxi..rrg/bbs/14400 http///bb/111..5//// 6572
12

---

Published in Transactions on Machine Learning Research (01/2025)
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CV conference compuler vision and pattern recognition, pp. 16000-10000000 16009,, 2022.
Jeremy Howard. Imagewang. URL http:////ggtaab..com/tattaaa////
Andrew Ilyas, Logan Engstrom, Anish Athalye, and Jessy Lin. Black-box adversarial attacks with limited queries and information. In International conference on machine learning, pp. 2137-2146. PMLR, 2018.
Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Visual prompt tuning. In European Conference on Computer Vision, pp. 709 727. Springer, 2022.
Jacob Devlin Ming- Wei Chang Kenton and Lee Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACI. pp. 4171-4186, 2019.
Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Cifar-10 (canadian institute for advanced research). a.
URL http:://ww.... wtt.:c.......ooo sarronto.edu/-krri//iiii. html
Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Cifar-100 (canadian institute for advanced research). b.
URL http::/wwx. wwwp wtwp:wccs..rooooooo boorontoo.ed/-kkiizze
html
Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for paraaeeter-fffiient prompt tuning, In Maric-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (cds.), Proceedings of the 2021 Conference on Empirical Melhods in Natural Language Processing, pp. 3045 3059. ACL, November 2021. doi: 10. n0.18633/v1 1t//222222mmml. wwn.p-main.222 URL httpi/ttcccaaatttololooooo
axgg/22211
mappmmain.... 243
Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th Annual Meeting of the Associ- ation for Computational Linguistics and the 11th International Joint Conference on Natural Language Pro- cessing (Volume 1: Long Papers), pp. 4582 4597, Online, August 2021. Association for Computational Lin- guistics. doi: 10.18653/vvl 2212222..ccl-ooo/
URL
actp:///w....o//...a.11a-a---- acw-long...33303 353/
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre-train, prompt, and predict: systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9):1 35, 2023.
Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Tam, Zhengxiao Du, Zhilin Yang, and Jie Tang. P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Compu- tational Linguistics (Volume 2: Short Popers). pp. 61-68, Dublin, Treland, May 2022. Association for Computational Linguistics. doi: 10. 156633/11/002..acshhtttt.. URL https://aadlaatoooooo.or////////
acl acl-short.8/ 8
Alcksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018. URL htttps://openreviii het.
Kaleel Mahmood, Rigel Mahmood, and Marten Van Dijk. On the robustness of vision transformers to adversarial examples. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7838-7847, 2021.
Yichuan Mo, Dongxian Wu, Yifei Wang, Yiwen Guo, and Yisen Wang. When adversarial training meets vision transformers: Recipes from training to architecture. Advances in Newral Information Processing Systems, 35:18599- 18611, 2022.
OpenAL. Gpt-4 technical report, 2023.
Mrigank Raman, Pratyush Maini, J Zico Kolter, Zachary Chase Lipton, and Danish Pruthi Model-tuming via prompts makes NLP models adversarially robust. In The 2023 Conference on Empirical Methods in Natural Longuage Processing, 2023. URL http:///gggggeeeeeeaeee///aaaaaa
13

---

Published in Transactions on Machine Learning Research (01/2025)
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, lan J. Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In Yoshua. Bengio and Yann LeCun (eds.), 2nd International Conference on Learning Representations, ICLR 2014, Banff AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014. URL http://arxiv, org/ abs 1312. 6199
Hugo Touvron, Matthicu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Herve Jegon. Training data-efficient, image transformers; distillation through attention. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 10347-10357. PMLR, 18-24 Jul 2021. URL attps://proceedlin html
mlr. grres//139/touvrorn
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial ro- bustness requires revisiting misclassified examples. In International conference on learning represenlations, 2019.
Zifeng Wang, Zizhao Zhang, Chen- Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. In Proceedings of the IEEE/CV, /CFF Conference Computer Vision and Pattern Recognition, pp. 139--19, 2022.
Ying Xu, Xu Zhong, Antonio Jimeno Yepes, and Jey Han Lau. Grey-box adversarial attack and defence for sentiment classification. In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Turs Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (eds.), Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, pp. 4078 77884407, Online, June 2021. Association for Computational Linguistics. doi: 10.18653/v v 1222 wtw./22..a..lmmmi...... maicc-mmin.331. URL (ttps://aclantte
org. 2021 naacl-maiii 321/
Yuting Yang, Pei Huang, Juan Cao, Jintao Li, Yun Lin, Jin Song Dong, Feifei Ma, and Jian Zhang, A prompting-basee based approach for adversarial example generation and robustness enhancement. arXiv e- prints, pp. arxiv 20-02203-42 2022.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theo- retically principled trade-off between robustness and accuracy. In International conference machine learning, pp. 7472-7482. PMLR, 2019.
Jingfeng Zhang, Xilie Xu, Bo Han, Gang Niu, Lizhen Cui, Masashi Sugiyama, and Mohan Kankanhalli. Atlacks which do not kill training make adversarial learning stronger. In International conference on machine learning, pp. 11278 11287. PMLR, 2020.
Zangwei Zheng, Xiangyu Yue, Kai Wang, and Yang You. Prompt vision transformer for domain generaliza- tion. arXiv e-prints, pp. arXiv-2208, 2022.
A Different Model Sizes
We provide results for the experiment setting of table 1, for different VIT configurations, namely VIT-S in table M and VIT-I. in table g The results consistent with that of table 1.
B Training Time Analysis
Prompt tuning secks to adapt large models to downstream tasks with minimal paramcters. This allows us to load and store only one set of weights for a large backbone, and that set can be adapted by storing and loading a low number of parameters for each downst task.
14

---

Published in Transactions on Machine Learning Research (01/2025)
Table 8: CIFAR1O test set accuracy for the ViT/Smal model
Method
Params.
Clean
PGD
CW
params
TRADES
PT2--mbb
86.06
3.98
3.81
740K
ADAPT+CE
PT2
Emb
73.7
34.33
31.52
T40K
ADAPT KL
PT2
Emb
63.37
36.1
31.74
740K
TRADES
FT
81.1
51.71
45.16
57M
Table 9: CIFAR10 test set accuracy for the ViT /aarge model.
Method
Params.
Clean
PGD
CW
# params
TRADES
PT2+Emb
94.12
3.68
2.36
1.4M
ADAPT+CE
PT2+Emb
80.79
39.49
36.82
1.4M
ADAPT4KI
PT2+Emb
69.98
41.16
36.82
1.4M
TRADES
FT
83.88
54.67
48.12
307M
Subsequently, it important to note that the advantage of prompt luning is a reduction the number of tuned parameters and not reduction in training time, as the forward and backward passes still flow through the entire model. This results in slight increase in computation cost, adding prompt tokens increases the number of tokens that go through the transformer. This is the case for any application of PT and PT2 and is not specific to our scenario. However, in our experiments, the prompts, we can train for fewer epochs with a cyclic learning rate to achieve our best-case performance. The same was not observed in the fine-tuning scenario and more training epochs were required to achieve the best performance in terms of validation accuracy.
empirically observe that while tuning
With all that said, we provide training time details for the scenarios in table 10 for those interested. We report the training time for each method in minutes. Each method was trained using an NVIDIA Tesla V100 SXM2.
Table 10: Total training time as measured in hours (rounded to the closest integer) for the methods presented in Tab. 2 the main paper. Each method in the prompt tuning scenario was trained for 20 epochs with a cyclic learning rate while each method in the fine tuning scenario was trained for 40 epochs with an anncaling learning rate.
Method AT
Params.
Total Training Time (hours)
PT2+Emul
15
TRADES
Pmb
16
ADAPT-CE (Ours)
PT2+Fmb
18
ADAPT-KL (Ours)
PT2+Emb
18
AT
FT
11
TRADES
FT
13
15


Reviews:
[{"review_id": "SW40nA28Jz", "reviewer": "Reviewer_W2Wn", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The problem investigated by the paper is interesting.\n2. The paper is easy to follow, and the method is easy to understand.\n3. The proposed method can achieve promising results under various adversarial attacks.\n\nWeakness:\n1. The experiment is insufficient. The baselines adopted by the paper are not state-of-the-art.\n2. The proposed method seems not novel enough. It is more like a simple extension of traditional adversarial training.\n3. Although the model's robustness has improved, its clean accuracy has significantly declined.\n4. Some details of the experiments are missing.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper investigated the robustness of the prompt tuning paradigm in ViTs and found that traditional adversarial training can't efficiently provide the robustness for prompt-tuned ViTs under the adaptive attack. To tackle this issue, the paper proposed a new framework called ADAPT to improve the robustness of prompt-tuned ViTs under adaptive attacks.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1.  Authors should incorporate more baselines for comparison. Since the paper is focused on robustness of ViT, some relevant works [1,2] that focus on improving the robustness of VIT should be added.\n\n2. In the related works section, the authors should cite additional papers to better reflect recent advancements in the topic addressed by the paper.\n\n3. The results from the auto-attack are not provided in Table 3 and Table 4. Authors should incorporate these results for fair comparisons.\n\n3.  The value of the hyperparameter $\\lambda$ is not clearly specified. Since adversarial training is highly sensitive to $\\lambda$, an improper setting could lead to a significant drop in clean accuracy or even corrupt the training process. Notably, the proposed method exhibits a substantial decline in clean accuracy. Therefore, I wonder if the authors have selected an appropriate value for $\\lambda$.\n\n4. From Table 1 - 4, the performance of the plain model without any adversarial training should also be provided.\n\n\n\n[1] Li, Yanxi, and Chang Xu. \"Trade-off between robustness and accuracy of vision transformers.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.\n\n[2] Jain, Samyak, and Tanima Dutta. \"Towards Understanding and Improving Adversarial Robustness of Vision Transformers.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "kNXUyiFuGl", "reviewer": "Reviewer_3NS8", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n1. The paper is easy to follow. \n2. Despite some editorial issues, the overall flow of the writing is coherent and maintains the reader's engagement.\n3. The observation made by the authors about having obfuscated grads are interesting. \n\n\n## Weaknesses\n1. There are 3 important papers in the literature that are missing as baseline methods:\n\n    **(a)** [1]: The authors introduced an adversarial robustness method through Model-tuning Via Prompts (MVP), which shares conceptual similarities with ADAPT. It is essential for the authors to explicitly articulate the differences between ADAPT and [1], if any, as this could directly impact the perceived novelty and contribution of their work. Additionally, [1] should be included as part of the benchmarking methods to provide a fair comparison. \n\n   **(b)** [2]: This work explores fine-tuning the entire model using prompts in a few-shot setting, which is relevant to the prompt tuning paradigm. Including it as a baseline would strengthen the evaluation.\n\n    **(c)** [3]: This paper proposes fine-tuning additional tunable parameters via prompts on top of a frozen model using the full training set, offering another closely related approach that should be considered for benchmarking.\n\nThe similarity between ADAPT and [1] raises questions about the originality of the proposed method. A thorough and careful response from the authors is necessary to clarify how ADAPT differs and what unique contributions it brings to the field. Addressing this issue comprehensively is critical to ensure the work's contribution is well-justified.\n\n2. In the experiments section, the authors state, \"We compare ADAPT to the following state-of-the-art methods established in the literature:\" but proceed to benchmark their method against AT and TRADE. These methods, while foundational, are outdated and no longer representative of the current state-of-the-art. The authors should include comparisons with more recent and relevant methods to provide a fair and robust evaluation of ADAPT. Without these, the claims of state-of-the-art performance are less convincing and may not reflect the advancements in the field.\n\n3. The results against AutoAttack are conspicuously missing from Tables 1, 3, and 4. AutoAttack is widely regarded as the strongest and most reliable adversarial attack in the current literature. Typically, the robustness of models is assessed against the strongest possible adversary, as it represents the worst-case scenario that any defense method should withstand. By omitting AutoAttack, the evaluation may appear incomplete or less rigorous. The authors should include results against AutoAttack to provide a comprehensive and realistic assessment of ADAPT's robustness, ensuring the method's effectiveness against state-of-the-art adversarial threats.\n\n\n### Minor issues: \n\n“from the same techniqueJia et al. (2022)” in the intro, space is missing\n\n“Datasets. We perform experiments on CIFAR-10Krizhevsky et al. (a)” space is missing\n\nAdversarial Training (AT) is abbreviated more than one time\n\n____\n**References:**\n\n[1] Raman, M., Maini, P., Kolter, J. Z., Lipton, Z. C., & Pruthi, D. Model-tuning Via Prompts Makes NLP Models Adversarially Robust. In The 2023 Conference on Empirical Methods in Natural Language Processing.\n\n[2] Gao, T., Fisch, A., & Chen, D. (2021, August). Making Pre-trained Language Models Better Few-shot Learners. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) (pp. 3816-3830).\n\n[3] Li, X. L., & Liang, P. (2021, August). Prefix-Tuning: Optimizing Continuous Prompts for Generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) (pp. 4582-4597).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors reveal that existing adversarial defense methods, when applied to the prompt tuning paradigm, are prone to gradient obfuscation. Through empirical analysis, they demonstrate this phenomenon using single-step attacks and subsequently design an adaptive attack to expose the vulnerabilities in these methods. Finally, they introduce a novel framework ADAPT, providing a more robust approach to adversarial defense in the prompt tuning setting.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please see the weaknesses I mentioned above.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "8aJS0FGUK8", "reviewer": "Reviewer_BfPs", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strength\n\n1. This paper focuses on an interesting topic, the robustness of the prompt tuning. It is totally different with the adversarial attack on the neureal network model.\n2. It demonstrates a parameter-efficient solution by tuning only ~1% of the model parameters while achieving robustness comparable to full-model fine-tuning.\n3. Provides a comprehensive comparison with state-of-the-art (SOTA) methods, including Adversarial Training (AT) and TRADES, highlighting improvements in robust accuracy.\n\n\n## Weakness\n1. The comparison of Adaptive Adversarial Attack is inappropriate. Since the entire model has already been optimized using prompt tuning, the \"traditional attack\" presented in Fig. 1 is not valid. After optimization with prompt tuning, the traditional attack should essentially be considered as an adaptive adversarial attack. To demonstrate the effectiveness of Adaptive Adversarial Attack, it should be compared with attacks performed on models optimized after prompt tuning. For instance, the comparison could explore whether there are inconsistencies between multi-step and single-step Adaptive Adversarial Attacks (PGD* and FGSM* in Table 1), or involve attacks based on scores.\n2. It is interesting to explore the tuning time. Although the paper mentions that prompt tuning requires significantly fewer parameters compared to fully fine-tuning, the specific training time is even more intriguing because it directly relates to the time required for backpropagation in prompt tuning.\n3. In Table 2, 3, 4, it is evident that under the clean setting, ADAPT significantly reduces performance, whereas fine-tuning does not. This is an intriguing phenomenon that warrants further exploration to understand the underlying reasons.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper investigates adversarial robustness in Vision Transformers (ViTs) using prompt tuning—a parameter-efficient alternative to full-model fine-tuning. The authors identify gradient obfuscation in prior defenses when applied in the prompt tuning paradigm, exposing vulnerabilities to adaptive attacks. To address these, they propose ADAPT, an adaptive adversarial training framework leveraging custom loss functions for better robust accuracy. Their approach reportedly achieves robustness comparable to full-model fine-tuning while tuning only a fraction of the parameters.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Thanks for your response. Your answers have addressed my concerns.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "aef67kY8p5", "reviewer": "Reviewer_DZfo", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n+ Authors propose a well-motivated problem of adaptive adversarial defence for Vision Transformers  \n+ The paper is clear and easy to follow with several experiments on large-scale benchmarks \n\nWeakness:\n- There is no theoretical motivation for the proposed attack and defence for vision transformers, specifically for the proposed KL-divergence component of the ADAPT loss, when we are already minimizing the Cross-Entropy of perturbed examples with the empirical distribution of unperturbed samples.\n- Section 4.1 does not clarify how ADAPT circumvents gradient obfuscation when doing Adversarial Training with the perturbed samples from the model with tunable prompt.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors study the robustness of parameter efficient prompt tuned vision transformers. The authors study existing adversarial defences to identify that these existing defences mainly rely on gradient obfuscation and are vulnerable to adaptive attacks. Thus giving a false sense of security. They also perform an evaluation of the existing defences with the black box and transfer attacks. The authors also propose the ADAPT framework for performing adaptive adversarial training to robustify prompt tuning-based vision transformers.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Authors should highlight how their proposed method (ADAPT_CE) is distinguishable from performing adversarial training with the prompt tunable model and how their method circumvents gradient obfuscation by clarifying section 4.1 further.\n2. ADAPT should also be further evaluated for more transfer attacks, with examples generated with PGD on other defences, e.g., TRADES.\n3. Authors should also justify their motivation for using KL divergence in addition to cross-entropy for their defence.", "broader_impact_concerns": "I foresee no broader impact concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The recommendation is based on the reviewers' comments, the action editor's evaluation, and the authors’ response. 

This paper studies a new robust prompting training approach to improve the adversarial robustness of vision transformers in a parameter-efficient manner. All reviewers find the studied setting novel and the results provide new insights. The authors’ rebuttal has successfully addressed the major concerns of reviewers. Therefore, I recommend acceptance of this submission. I also expect the authors to include the new results and suggested changes during the rebuttal phase to the final version.
Decision: Accept (poster)