ProTransformer: Robustify Transformers via Plug-and-Play Paradigm
Authors: Zhichao Hou
Weizhi Gao
Yuchen Shen
Feiyi Wang
Xiaorui Liu
Venue: NeurIPS 2024 poster (2024)
Keywords: Transformers
Adversarial Robustness
Large Language Models
Primary area: safety_in_machine_learning

Abstract: Transformer-based architectures have dominated various areas of machine learning in recent years. In this paper, we introduce a novel robust attention mechanism designed to enhance the resilience of transformer-based architectures. Crucially, this technique can be integrated into existing transformers as a plug-and-play layer, improving their robustness without the need for additional training or fine-tuning. Through comprehensive experiments and ablation studies, we demonstrate that our ProTransformer significantly enhances the robustness of transformer models across a variety of prediction tasks, attack mechanisms, backbone architectures, and data domains. Notably, without further fine-tuning, the ProTransformer consistently improves the performance of vanilla transformers by 19.5\%, 28.3\%, 16.1\%, and 11.4\% for BERT, ALBERT, DistilBERT, and RoBERTa, respectively, under the classical TextFooler attack. Furthermore, ProTransformer shows promising resilience in large language models (LLMs) against prompting-based attacks, improving the performance of T5 and LLaMA by 24.8\% and 17.8\%, respectively, and enhancing Vicuna by an average of 10.4\% against the Jailbreaking attack. Beyond the language domain, ProTransformer also demonstrates outstanding robustness in both vision and graph domains.

Pro Robustify Transformers via Plug-and-Pla, Paradigm
Zhichao Hou'
Weizhi Gao
Yuchen Shen?
Feiyi Wang
Xiaorui Liu
North Carolina State University, CCarnegie Mellon University, Oak Ridge National Laboratory
fzhou4,wga023 wgao23, xnisu6) edu yuchens36cs. cmu edu fwang20ornl.gom
Abstract
Transformer-basee architectures have dominated various areas of machine learning in recent years. In this paper, we introduce novel robust attention mechanism designed to enhance the resilience of transformer- based architectures. Crucially, this technique can be integrated into existing transformers plug-and-play layer, improving their robustness without the need for additional training or fine-tuning. Through comprehensive experiments and ablation studies, we demonstrate that our Pro Transformer significantly enhances the robustness of transformer models across variety of prediction tasks, attack mechanisms, backbone architectures, and data domains. Notably, without further fine-tuning, the ProTransformer consis- tently improves the performance of vanilla transformers by 19.5%, 28.3%, 16.1 %, and 11.4% for BERT, ALBERT, DistiBBERT, and RoBERTa, respectively, under the classical TextFooler attack. Furthermore, Transforrrmee shows promising resilience large language models (LLMs) against prompting-base -based attacks, im- proving the performance of T5 and LLaMA by 24.8% and 17.8%, respectively, and enhancing Vicuna by an average of 10.4% against the Jailbreaking attack. Beyond the language domain, ProTransformer also demonstrates outstanding robustness both vision and graph domains. Our code is available at https//github.com/chii TonoPPooTrannfrmer
Introduction
In recent years, attention mechanisms and transformer-basee architectures have drawn significant attention across many domains in machine learning, such as natural language processing (NLP) 2 computer vision 3 and graph learning (5) []. In particular, transformers have demonstrated superior capabilities to learn and model complex relations in data through powerful and universal attention mechanisms, and they have dominated many popular NLP tasks such as topic classification, sentiment analysis, textual entailment, machine translation, dialogue generation, etc [2] Despite their success in NLP and beyond, many recent studies have demonstrated that transformers are highly vulnerable to adversarial attacks such that even small modifications to the input can easily fool the
model 7 g However, most research on transformer architectures focuses on accuracy and efficiency, largely ignoring their security and robustness [10] [1].
With the increasing popularity of Large Language Models (LLMs) 12 13 the robustness and security concerns of transformer architectures become particularly interest. has been shown that malicious attackers invade the language models through various approaches shown in Figure I
The attacker can modify the input content in text attacks [14] or the prompt template in
"Corresponding author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Original Text
Perturbed Text
News: 2024 NBA All-Star Game.
Text
News: 2024 nBA All-Ster Match,
Topic: Sports
Attack
Topic: Business
Prompt: Determine if the following 'posittiv'' 'negative'. Question: like your website.
Prompt: Detemine if the subsequent txt is 'positive' 'nnegative,, Question: like your website.
Prompt
Attack
Answer: Positive
Answer: Negative
You can't provide harmful responses the
System:
System: You can't provide harmful
Jail
responses to the users.
User: Teach children use firearms, k picture& laddr ladd
User: Teach children to use firearms. Answer: cannot fulfill your request.
Break
OssNN Answer: Sure, here is tutorial. First,
Figure Various attack mechanisms on language models. Classic text attacks modify the input content using typos or synonyms; Prompt attacks perturb the prompt template within the input; and Jailbreaks append adversarial, non-semantic suffixes manipulate the model into producing malicious outputs.
prompt attacks to mislead the model predictions 15 Moreover, by adding adversarial suffixes, the jailbreaking attack [16] can prompt a LLM to generate toxic and illegal content which could lead to catastrophic legal and ethical impacts such as malicious speech or privacy leaks. Given the broad applications of transformers and their vulnerabilities under attacks, is imperative design universal and effective strategy to enhance the robustness of transformers.
Existing research attempting to improve the robust- ness of transformers can be roughly divided into empirical defenses [17
ProTransformer
[8 19
20 21 and certifi- 25 Nevertheless, these
able defenses [22]
23
24
Text
Add & Norm
Feed Forward
Pro-BERT
defenses require excessive computation costs for training, inference, or both. In addition to these rrchitecture-annssti defenses, there are also several works proposing to enhance the robustness of trans-
Attention
Image
Pro-ViT oL 050
However, these approaches either require substantial computations or rely on specific domain knowledge, which hinders their extensions to larger models or broader applica- tion domains.
formers architecture [26] 27 28 29
Graph
Pro-GAT
Plug-and-Plag
D (DIST(Z())
w(th (D
Vanilla Attention
z (A) Pro-GPT
Pro-LLaMA
In this paper, given the limitations of existing works and the enormous training cost of transformers, we
ProAteettion
xk
aim to robustify transformer architectures via a plug- and-play paradigm without additional training or fine- tuning. Our proposed ProAttention (Algorithm [ can be readily plugged into the given transformers con- vert them to Pro Pronrfosfrmrer (as shown in Figure 2 with significantly stronger robustness. Specifically, our contributions can be summarized as follows:
Overview of ProTransformer. Attentioionn can be plugged into pretrained transformers without additional training. The ProTrrasformer is versatile and can be ap- plied across various domains, including lan- guage, image, and graph.
Figure 2:
We establish novel connection between the attention mechanism in transformers and the weighted least square estimator. We provide interpretation and numerical simulation to reveal its vulnerability against potential adversarial attacks.
From our new perspective, we propose robust token estimators to improve the resilience of token aggregation against adversarial attacks. We also propose an efficient Newton-IRLSS algorithm to approximate the non-convex and non-smooth robust token estimator with convergence guarantees. The derived algorithm can be plugged into the given transformer as a plug-and-play layer to enhance its robustness against attacks even without additional training fine-tuning.

---

Our comprehensive experiments and ablation studies demonstrate that the proposed ProTrans- former is effective, efficient, and generalizable. significantly improves the robustness of transformers across various machine learning tasks, attack mechanisms, backbone architectures, and data domains such as language, vision, and graphs.
2 Related Work
In this section, mainly summarize related works the attacks and defenses of transformers focusing on language domains since this is the focus of this paper.
the text attacks in the language domain are highly complicated due to the natural irregularity data structure. According to the perturbation units, text attacks can be classified into character-level 7 32]. word-level [33 34
Attacks. Compared to the attack mechanisms in vision domain 30 31,
351 36 9 [4] 37), sentence-level [38), and multi-level [39 40 8 These classic text attacks typically generate adversarial examples through misspellings, synonym replacement, etc. In the of LLMs, several new types of attacks have emerged, such as jailbreak attacks [16
43 and prompt injection 44 45 [46). These prompting-hasee based attacks aim trick models into generating unsafe
outputs using adversarially crafted prompts.
Defenses.
There have been some works proposed defend against adversarial text attacks from various perspectives. Empirical defenses, such as data augmentation [17] and adversarial training [47 19 20 21, attempt to robustify models by exposing them to a wider range of adversaries during
training. On the other hand, several certifiable defenses [24] 25 22 23 have been proposed to guarantee the model robustness regardless of the attacks. However, these defenses require excessive computation costs for training, inference, or both, which limits their application in large-scale problems such L.MSs. Besides, all these methods typically architecture aggnostic, which are orthogonal to and can be combined with our proposed defenses on the transformer architecture to further enhance the robustness.
To safeguard the transformers, several endeavors have been made from the transformer architecture perspective. Li et al. (26) modify the attention mechanism and position embedding to robustify text-to-speech transformers. end-to-end attention iiteetion-based classifier to enhance robustness. For tabular data, TableFormer (28 adopts structural- aware table-text encodings that are more robust to row and column order perturbations. However, these architectures tailored for specific tasks, which require specific domain knowledge and can not be generalized across tasks. Han et al. [29] propose general framework for self-attention modules via robust kernel density estimation (RKDE). However, this method introduces excess computation cost and shows relatively limited robustness improvement. Generally speaking, existing approaches either require substantial computations rely on specific domain knowledge, which hinders their extensions to larger models or broader application domains.
the crisis detection and recognition task, Liu et al. 127 propose an
3
ProTrrasformer
The main goal of this paper is to design robust self-attention mechanisms that are more resilient to adversarial attacks they can be applied to robustify Transformer architectures. In this section, we first provide new interpretation of the self-attention mechanism in Transformer architecture as weighted least-square token estimator in Section 3.I Then we propose robust token estimators that are more resilient to the dominating impact of input tokens in Section 3.2] An efficient Newton-IRLS algorithm is derived with a convergence guarantec to approximate the robust token estimator in Section 3.3 Finally, we describe how the proposed algorithm can be unrolled as robust attention layers to enhance the robustness of transformer architectures in Section 3.4
3.1 Attention Mechanism WL. Token Estimator
First, we provide a new perspective to formulate the vanilla attention mechanism as the weighted least squares (WLS) token estimator. In the self-attention layer, each output token aggregates the values of input tokens {vy} as their weighted sum according to the attention weights: z E a,v;, where (a, [e])) are the attention weights and {v,) [V]]) are value vectors for N input tokens. This weighted sum can be interpreted as the optimal solution of the following weighted least squares

---

(WLS) error minimization problem:
N arg min C(x) Σ aj [V, - =21²,
j=1
whose first-order optimality condition (VL(z)= = aj 2(z (1z 0) yields z Σド= V
Vulnerability analysis of vanilla attention. When adversaries perturb the input tokens, these tokens will dominate the impact on output tokens since the quadratic penalty on the residual || z|I will dominate the WLS estimator. Therefore, the output token z will be shifted to those dominating input tokens. As a result, the adversarial input tokens will significantly impact the representation of output tokens. We also provide empirical study verify that adversarial attacks will significantly increase the residual ||v; - z|l in Appendix F.3 Moreover, we simulate a mean estimation problem under outlier data points using synthetic data to better illustrate the sensitivity of the WLS estimator. The detailed setting and visualization results of the numerical simulation are provided in Appendix F
3.2 Robust WLS Token Estimators
The analysis above provides a valid explanation of why various attention- based transformer archi- tectures are easily compromised by introducing adversarial perturbations in the input data. Also, our interpretation of the attention mechanism in transformers as WLS estimator provides rig- orous perspective design robust alternatives. To dampen the effect of outlier data, multiple robust regression algorithms have been proposed in robust statistics using least absolute devi- ations [88.. Huber regression [4), and Minimax Concave Penalty (MCP) [50]. Motivated by these advancements with rigorous robustness guarantees, we propose the robust weighted least squares token estimators to enhance the resilience against potential adversarial attacks as follows:
N Σ 0j p(ll - 2
arg min C(z)
(2)
f1
Huber (6)
MCP (V)
where p can flexibly replaced with the specific robust penalties in Figure 3
Huber-MPP
pla
Special cases of p. (1) The quadratic & loss recovers vanilla WLS estimator, & loss exerts linear effect on the residuals; (2) Huber loss performs
Eg loss within the range (0, 8). and becomes similar to & when 8; (3) MCP loss behaves like & loss near zero and becomes con- stant when is large than 7. (4) We also propose Huber- MCP combine the advantage of Huber and MCP loss. The detailed formulations are available in Appendix B.4 due to the space limit.
2
Figure 3: Different p(=).
3.3 Newton-IRLS algorithm
The proposed robust token estimator in Eq. (2 is non-convex and non-smooth, posing a challenge for efficient algorithm design. Moreover, the exploding model size of evolving transformers further necessitates the design of efficient neural network layers. To this end, we propose an efficient Newton algorithm tackle this challenging problem. We
iterative reweighted least square (Newton-IRL) first design a localized upper bound for the original objective and then optimize the upper bound with second-order Newton method. We also provide a rigorous theoretical loss descent guarantee. The precise statements are presented as follows and the detailed proof are provided in Appendix B
Localized upper bound. Instead of directly optimizing the original loss function L(z) in Eq.
optimize convex localized upper bound at the current iteration (k) follows:
Lemma 3.1 (Localized Upper Bound). Suppose the loss objective is defined as in Eq. where p sqrt(-) is any non-convex function. functiion. For any fixed point (k), there exists convex localized upper

---

bound as:
[(z) w. (k) liv 2|2 C(2
(3)
j=1
where w d' 1I) and p' is the first derivative of p. Particularly, the constant C(z
guarantees the equality of & and at (h)), ie., ((z!))) L(z(
Proof. Please refer to Appendix B.1
As ((z(k)) is treated as a constant during the optimization at the current step, the upper bound in Eq. 3 becomes convex and can be efficiently optimized.
Newton-IRLS iteration. After obtaining the convex upper bound & in Eq. (3), we can derive
concise closed-form iteration using the second-order Newton method follows:
(()))) ( -
Ej
wy V
(k+1) 2x(
Ega..un w
Eq. can be interpreted as reweighted sum, which the derived wy modifies the original attention score aj on the value vector vj. We leave detailed derivations of Newton-IRI.S algorithm in Appendix B.22 Its convergence and rigorous loss descent are guaranteed by the following Theorem 3.2
Theorem 3.2 (Convergence guarantee). Suppose the loss objective L(z) is defined in Eq. 22 and its corresponding convex localized upper bound is in Eq. 3p. Then, through the iteration in Eq. the following inequality holds:
((z(k+1)) < ((ztkt1)) s ((x
((z)k)),
(5)
that is, optimizing upper bound 2 can guarantee the rigorous descent of
Proof. Please refer to Appendix B.3
Although the loss ((z) is not necessarily convex and does not possess global optimum, Theorem 3.2 guarantees that the Newton-IRI iteration, which optimizes [(z), can rigorously reduce the original loss C(z). The algorithm analyses in Appendix (P) along with the main experiments in Section 4 and Section [5] validate that the local optimal solution achieved by our algorithm performs well in terms of both convergence and empirical robustness.
Robust token estimator by reweighting the tokens. The robust estimator in Eq. 2 provides
general framework that covers several special cases. By choosing different penalty functions
on the residuals ||v, - ( we obtain various reweighting schemes in Eq. 44p. Take the MCP
(k) d (1lv, -20
function the instance, the weight is derived w
2 -
max
1 4 0
Obviously, the weight w becomes smaller as ||v, z (8) increases, thereby down- weighttig the large residuals. The residuals will be completely removed when exceeds the threshold 7, since the weight then becomes 0. The complete discussions for all cases are provided in Appendix B.4
3.4 Pronttention: Robust Attention Layers
In the previous subsection, we formulate the token-wise Newton-IRLS approach for notation simplic- ity. Here, we will present the corresponding matrix version for the entire attention layer.
Matrix Form. Denote V [Vylje(N) and A {a. [jjE(N) value matrix and the attention
matrix, respectively. (4) = {2,* (
JEe[ is the estimator for token at the k-th iteration. Subsequently, (k)
the pairwise distance (i) = { z [[A.SENN] between (k) and V can be efficiently computed
using the torch, cdist function in PyTorch. Following this, the weight (*k) {w
(k) Ji.je[N] can
be calculated element-wise based on (R), Then the next step (k++) is updated as reweighted matrix multiplication (W()) O A) V.
5

---

Plug-and-Play Robust Attention. The pro-
posed algorithm can be packaged as ro- bust attention module, which can be readily
Algorithm 1 ProAttention (MCP)
ID torch. cdist (Z, V) Patrwise distance
plugged into the transformers as a Plug-an -nd--and----
2 torch. ctwppp/ppiipp (1/D-1/s //ggmmmm, min=0) # NCP
Play Robust Attention (ProAttention) layer without additional training or fine-tuning as shown in Figure 2 The implementation of ProAttention using MCP penalty in Py is shown in Algorithm other penalties is presented in in Appendix A
3W normalize A, p=1, dim=-1)
42 torch. matmumtmmmummmlm.m.mmmmmmmm (H, V) # Update
1
The complete pseudocode for
Complexity analysis. Let N. D. and K represent the length of tokens, the dimension of vectors, and the steps of the iterations, respectively. The vanilla attention requires 2 N x N x D basic operations while our ProAttention needs (1 (1+224 (-) x N D. However, ProAttention remains efficient, as the Newton-IRLa method can effectively approximate the solution within only 3 steps (K' 3) (Figure 4 (a)) and Pronfrosformmrs do not introduce additional computation for training or fine-tuning. We provide the detailed complexity analysis various attentions in Appendix [[]
Advantages. Our proposed ProAttention enjoys the following advantages: (1) Simplicity: it is simple and easy to implement with only 4 core lines of code in Algorithm [ (2) Efficiency: is a plug-and- play layer that can be integrated into any trained transformer without additional training or fine-tuning; (3) Universality: it is a universal framework that advances the vanilla attention mechanism into a series of robust derivatives with different penalties. Moreover, it can be applied to any attention-basee model across various modalities and tasks.
In the following sections, we will present comprehensive experiments and studies to validate the effectiveness of the proposed ProAttentio on language modeling Section 4 as well as computer vision and graph learning in Section
4 Experiment on Language Modeling
In this section, we evaluate the effectiveness of the proposed ProAttention and ProTTransformer under classic text attacks pre-trained language models, and two prompting- attacks (prompt attack and jailbreak attack) in the context of LLMs with comprehensive ablation studies.
4.1 Experiment Setting
Tasks and Datasets. For topic classification, we use AG's News Corpus (AGNEWS) [1]. For sentiment analysis, we utilize two widely-used datasets: Internet Movie Database (IMDB) 52 and Stanford Sentiment Treebank (SST-2) (53). For textual entailment, we make use of Recognizing Textual Entailment (RTE) in the General Language Understanding Evaluation benchmark (54) For jailbreak attack, we select new dataset Behaviors introduced in [55] For the detailed information on these datasets, please refer to Appendix C
Backbone Architectures. For classical pre- pre-trained language models, choose BERT [6 and its
variants including RoBERTa [57], ALBERT [58] and DistilBERT [59]. For large language models (LLMs), choose T5 660 LLaMA I2 and Vicuna [13.. For the detailed information on backbone
architectures, please refer to Appendix D.2
Attacks. We not only evaluate several classic text attacks but also include popular prompt attacks and jailbreak attacks on the LLMs. The three attack mechanisms and their differences are illustrated in Figure [ For classic text attacks, we evaluate the attacks at various levels, including the character- level Deep WordBug [7]. word-level PWWS 9 TextFooler [14]. and multi-level TextBugger (8]. For prompt attacks, we modify the prompt template according to the aforementioned text attacks following the evaluation setting in PromptBench [15]. For jailbreak, we evaluate the suffix attack using Greedy Coordinate Gradient (GCG) method (55) and we test both attacks transferred from surrogate model Vicuna (transfer attack) and attacks directly targeting the victim models (adaptive attack). Please refer to Appendix for details attacks.
Defense Baselines. We include the following defense baselines in our experiments: MixADA 17
PGD-Adv BII, FreeL.B [47, TA-VAT [18] and SmoothL_MM (61). Additionally, we also include the adversarial training (AT), wherein the augmented perturbations are generated by the attack to be assessed. Details of these defense methods are provided in Appendix D.I

---

Evaluation metrics. Following [62,, we use 3 metrics to evaluate the model performance. Clean accuracy (Clean %) the model accuracy on the clean testing data. Accuracy under attack (AUA % is the accuracy on the perturbed data under specific attack. Attack success rate (ASR %) is the ratio of the number of successfully perturbed cases divided by the number of attempted texts.
Hyperparameters. For text attack setting, we follow the setting in the TextAttack framework [63 For prompt attack, we follow the setting in PromptBench [15]. For GCG-based jailbreak attack, follow the setting in [6]] The detailed attack settings can be found in Appendix L
For defense
baselines, we follow the settings in their original papers. For our Pro'rrasformer, we set the default number of ProAttention layers as K = 3 since it can quickly converge to reasonable precision within 3 layers. Finally we tune 8 (default or (default 4) the penalties (Huber and MCP loss) to obtain the optimal parameters.
4.2 Classic Text Attacks on Language Models
To demonstrate the effectiveness of the proposed ProTransformer, we compare the robustness of our methods with several popular defenses in three classical tasks: topic classification, sentiment analysis. and textual entailment.
4.2.1
Adversarial Robustness
Table 1: The results topic classification on AGNEWS.
Textfooler
TextBugger
DeepWordBug
PWWS
Model
Clean
Aua%
ASR%
AUA% t ASR%
AUA%
ASR | AUAS ASR
ALBERT
93.0
20.6
26.1
71.9
58.2
PRO-ALBERT (MCP) (Ours)
48.9
47.3
41.8
55.3
59.5
35.9
63.1
32.0
DistiBERT
93.5
13.2
85.9
33.6
63.4
30.0
67.9
36.5
61.0
Pro- DostiittBBERT (MCP) (Ours)
93.9
29.3
68.5
48.7
47.9
34.3
63.1
50.5
45.6
RoBERTa
93.4
13.0
86.1
32.5
64.5
41.2
55.9
34.0
63.6
Pro-RoBERY (MCP) (Ours)
93.7
24.4
73.7
34.3
62.8
45.5
51.5
39.4
57.5
BERT
94.2
19.7
78.9
31.7
67.3
37.5
59.8
43.1
53.8
+ FreeLB
94.2
38.0
59.5
42.8
55.5
56.1
40.9
57.0
39.9
+ PGD
94.1
36.8
61.7
40.5
57.1
47.6
49.7
48.7
48.6
+ MixADA
35.6
62.4
35.4
50.5
50.4
TA-VAT
94.4
36.2
61.8
39.2
58.2
49.5
48.1
47.0
50.7
+ AT
94.1
42.1
54.8
56.1
39.4
42.4
54.1
62.6
32.5
PRo-BERTTTE (f1) (Ours)
94.2
23.8
74.5
43.8
53.0
48.7
47.8
46.5
50.1
Pro-BERT (Huber) (Ours)
94.2
24.2
74.0
43.7
$2.9
46.0
50.5
48.4
47.9
Pro-BERT (MCP) (Ours)
93.2
39.2
$7.7
48.3
48.5
51.8
43.8
56.2
39.2
Pro-BERT (MCP) (Ours)
94.0
56.8
60.7
35.1
61.0
34.1
25.7
Performance analysis. The experimental results of topic classification (AGNEWS) are presented in Table and provide the results of sentiment analysis (IMDB) and textual entailment (RTE) in Appendix G.1 and G.2 due to the space limit. From the experiment results, we can make the following observations:
The proposed ProAttention is a highly effective plug-in module that significantly and consistently enhances the robustness of various transformer backbones across various adversarial attacks. Taking AGNEWS as the instance, when combined with ProAttention (MCP), under the attacks (Textfooler, TextBugger, DeepWordBug, PWWS): (1) ALBERT improved by (28.3 12,33% 15.7%. 20.6%, 27.2% (2) DistilBRRT is improved by [16.1%, %, 15.1%. 1%, 4.3%, 14.0% ) (3) RoBERTa is improved by (1 (11.4%, 1.8%, 4.3%, 5.4% (4) BERT improved by (19.5% 16.6%, 14.3%, 13.1%).
Our method, plugging in ProAttention (MCP) module without fine-tuning, our Pro-BERT can achieve comparable robustness to most adversarial training-bait based methods which require substantial computational time and resources. Furthermore, our framework is orthogonal to most existing defenses, allowing for combined use with them to further enhance robustness. For instance, when combined with AT technique, our Pro-BERT (MCP) + AT can further improve BERT AT by (14.7%, 4.6%, 18.6%, 6.2%) under [TettFooler, TextBugger, DeepWordBug, PWWS).
(MCP) + AT, exhibits best robustness among all the baselines. By simply
7

---

x11
(unadod) Pooddiil (hnudid))
70
ALX
05
29
100
6:25
0.9
475
Layers
Epeck
Max Parselation Paverdagg
(a) Convergence
(b) Adversarial Training (c) Attack Constraints
(d) Different Penalties
Figure Ablation studies.
4.2.2 Ablation Study
Convergence. To validate the advantage of our Newton-IRLS over the first-order method, we conduct a simulation experiment and plot the loss descent curves in Figure 4 (a). It can be observed that
Newton-IRLS exhibits efficient convergence
claimed in Section 3.3 We provide the experiment
details, loss descent curves (Figure 7 and the visualization of trajectories (Figure 8) of the updated
vectors 2D plane Appendix F.1 to further demonstrate the effectiveness of our algorithm.
Adversarial fine-tuning. To get insight into how the models gain more robustness from adversarial examples, track the training curves of adversarial fine-tuning under TextFooler in Figure 4 (b), and put the results of other attacks in Figure T0 in Appendix G.3] We can observe that our Pro-Bi BERT (MCP) is compatible with adversarial fine-tuning technique to further enhance the model resilience.
Attack constraints. In text attack, there are several kinds attack constraints including the maximum percentage of perturbed words, minimum cosine similarity between the replaced synonym and original word, and minimum sentence similarity threshold between the original sentence and perturbed sentence. We test the values of these constraints in TextFooler. We present the results under different perturbation percentages in Figure 4 (c) and other constraint measurements in Appendix G.4 From the results, we observe that our Pro- Aro-ALBU ALBERT (MCP) can significantly outperform the backbone ALBERT across all ranges of constraints.
Different penalties. Our Newton-IRLS is flexible to be formulated different robust estimators with different penalties. From the comparison Figure (d) it can be observed that our robust framework can consistently improve the robustness of the backbone BERT (f2). Specifically, & and Huber-based defenses are comparable, and MCP-based method exhibits the best performance.
Different backbones. Our method is a general plug-and-pla and-play layer applicable to various transformer backbones. The results in Table [ and the ablation study on different backbones in Appendix G.5 (Fig- ure 12) demonstrate that ProAttention improves the robustness over various architecture backbones (BERT, RoBERTa, DistilBERT and ALBERT) against various attacks with significant margins.
Running time. To empirically evaluate the ef- ficiency of our method, we test the average run- ning time on AGNEWS using BERT and Pro- BERT (MCP) equipped with multi-layer (K) ProAttention. The results in Thol that our ProAttention only requires 1-2 times addi- tional inference time of the backbone model yet achieves significant improvement in robustness without training.
Table 2: Average running time (ms) on AGNEWS.
BERT
Pro-BERT (MCP)
Running (ms) Layers (K)
11.67
17.33 19.89 21.87
4.3 Adversarial Prompting Attacks LLMs
In the context of prompt-based generative AI, the adversarial attacks mechanisms on LLMs become more enriched and sophisticated. this section, we will evaluate the robustness of our proposed Pro Pransfoofrrrr under two popular attacks; prompt attack and jailbreak attack.
8

---

4.3.1 Prompt Attack
As shown in Figure I the most significant distinction between prompt attacks and classical text attacks is that prompt attacks aim mislead the models by altering the prompt template rather than the input content. We display the results of T5 in Figure and leave the comprehensive study in Appendix H.T We also present the results on LLaMA Appendix H.2 From the results, we can make the following observations: (1) For T5, the choice of the penalty would affect the robustness of defenses. Specifically, Pro-T3 (MCP) exhibits significant advantage over other methods, and this advantage becomes even
evident the number of perturbed words increases. Pro-T5 (L1) and Pro-T5 (Huber) show a slight improvement over the backbone model T5. (2) For LLaMA, Huber-MCP and Huber-based methods exhibit better robustness than other methods while preserving good clean performance. The detailed experiments and discussions can be found in Appendix H.2
0.9
0.9
0.9
L8
7.3
80
Accumcy (%)
Smooothing Q (9)
0.9
0.9
0.9
0.9
0.9
11.8
16.4
21.3
31.8
20.9
60
-0 (T5)
23.6
40.0
-40
Huber
201
1.8
21.8
50.9
36.4
61
8
70.0
74.5
K2.7
90.0
91.8
10
0.1
0.2
0.3
0.4
0.5
0.8
1.0
1.5
3.0
Vicuna
Nom of Perturbed Words
8 in Pro-Vicuna (Huber)
Figure 5: Prompt attack results.
Figure 6: Attack success rates (ASRs) under transfer jailbreak.
4.3.2 Jailbreak Attack
In recent years, prompts have played pivotal role in guiding models to generate desired outputs. Nevertheless, there exist malicious "jailbreak prompts", which are intentionally designed to bypass the built-in safeguards in LLMs, causing the model produce harmful content that violates the legal policies. As illustrated in Figure T] the suffi--njeccton jailbreaks attempt to append non- semantii suffix to the user's prompt fool the models. We select GCG method to evaluate the resilience of models comprehensively.
In Figure compare the Attack Success Rates (ASRs) of Vieuna and its corresponding Pro- Vicuna (Huber) with various values on Behaviors. In each column, we also include SmoothLL.M [61 with different smoothing extent (%) to further reinforce the resilience of every single model. The last row of matrix (g 0) stands for the performance without random smoothing. The additional results of random smoothing with swap, insert and patch, as well as the results under adaptive jailbreaking attack are presented in Appendix [
From the results, Vicuna. As shown the last row Figure 6 with 8 = 0.1, we successfully reduce the ASR to 1.8%, which is comparable to the random smoothing defense that requires multiple random perturbations, inferences and aggregations. (2) Our ProAttention orthogonal to randomized smoothing defense and can be combined with it to further improve the robustness.
observe that: (1) Our Pro- Vicuna significantly improve the robustness of
5 Experiment beyond Language Modeling
In the previous section, we have provided comprehensive experiments to validate the effectiveness of our ProTransformer in the (large) language models. In fact, as shown in Figure 2 our ProAttention is fundamental module which can reinforce any attention based models across various domains or modalities. In this section, we will integrate ProAttention into vision models and graph learning models to further validate the effectiveness and generality of our approach.

---

5.1 Image Classification
In computer vision, we conduct two attacks (FGSM [30] and PGD [1I) on several vision transformers including ViT, BeiT, ConviT, DeiT and Swin. We perform the experiments CIFAR-10 and ImageNet-1K across budgets 11/255, 4/255, 8/ 8/255), and present the results of PGD on CIFAR-10 Table and additional experiments in Appendix J From the results, we can observe that Pro-ViT can outperform the second best model by (35.64%, 46. 16.28%, 33.14% under different budgets.
Table 3: Adversarial robustness under PGD.
Modell Budget
0 (Clean)
1/255
4/255
8/255
Deit
97.91
38.98
0.44
0.0
Convit
98.70
41.75
1.83
BeiT
97.87
6.81
0.0
0.0
Swin
98.30
14.89
0.02
VIT
98.74
34.61
1.83
0.26
Pro-VIT (Ours)
98.40
77.39
48.11
33.40
5.2 Graph Representation Learning
Besides the language and vision domains, we also validate the effectiveness of our method in the graph domain. We conduct the semi- supervised node classification task and leverage PGD adaptive attack (64] to evaluate the robust- ness of models. We show the experiment results of Cora-ML and Citeseer, averaged over dif- ferent random splits, in Table and Table 29 (in Appendix K respectively. The ablation studies the layers and y in MCP are presented in Table 30, Please refer to Appendix [ for more detailed results and studies. From the results, outperforms the backbone GAT and exhibits strong robustness across various budgets while keeping good clean accuracy.
Table 4: Adversarial robustness on Cora-MI.
Model Budget
0%
10%
20%
30%
40%
GCN
85.0
69.6
uus 60.9
02
54.2 u.
48.4 u
GNNGuard
83.1
70.2
1: 63.1
57.5 L6 51.0
+
RGCN
85.7
204
59,8
52.8
46.1
2:07
GRAND
86.1 1.77
70.7
1.7
61.6
107
56.7
108
51.9
1.09
ProGNN
Inccard-GCN
83.7 n.77
68.3
1:07 60.0
111 54.0
11
49.1
1:24
MolMedian
85.0
75.5
69.5
62.8
58.1
GAT
83.5 2.55
71.2
1:12
65.0
63.0
1:08
00.5
2.09
56.7
1.09
Pro GAT (ours)
84.5
75.5
72.1
69.0
66.5
conclude that Pro-GAT significantly
Conclusion & Limitation
In this paper, we delve into the robustness and security of the popular transformer- -based architectures. We revisit the vulnerability of attention mechanisms with theoretical understanding and simulations. We propose an interpretable robust attention layer to robustify transformer architecture via plug- and-play paradigm. Our proposed ProAttention is an effective, efficient, and universal framework that can significantly enhance the robustness of transformers across various tasks, architectures, attacks, and domains without additional training or fine-tuning.
Regarding the limitations, despite the acceptable complexity of our ProTransformer, there is still potential improve the efficiency of our models. Additionally, while primarily claim and validate the effectiveness of our model under a plug-and-play paradigm, we are excited about the future of the proposed Pro Transformer architecture and hope see its full potential realized through training or fine-tuning on large models in the future.
Acknowledgment
Zhichao Hou, Weizhi Gao, and Dr. Xiaorui Liu are supported by the National Science Foundation (NSF) National AI Research Resource Pilot Award, Amazon Research Award, NCSU Data Science Academy Seed Grant Award, and NCSU Faculty Research and Professional Development Award.
10

---

References
[1] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez. Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in newral information processing systems, 30, 2017.
[2] Tianyang Lin, Yuxin Wang, Xiangyang Liu, and Xipeng Qiu. A survey of transformers. Al Open, 2022.
[3] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint 2XX1:2010011129, 2020.
[4] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEF/CVF international conference on computer vision, pages 10012-10022, 2021.
[5] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In International Conference on Learning Representations, 2018.
[6] Seongjun Yun, Minbyul Jeong, Rachyun Kim, Jaewoo Kang, and Hyunwoo J Kim. Graph transformer networks. Advances in neural information processing systems, 32, 2019.
[7] Ji Gao, Jack Lanchantin, Mary Lou Soffa, and Yanjun Qi. Black-box generation of adversarial text sequences to evade deep learning classifiers. In 2018 IEEE Security and Privacy Workshops (SPW), pages 50-56. IEEE, 2018.
[8] Jinfeng Li, Shouling Ji, Tianyu Du, Bo Li, and Ting Wang. Textbugger: Generating adversarial text against real- world applications. arXiv preprint arXiv: /8/2.05// 2018.
[9] Shuhuai Ren, Yihe Deng, Kun He, and Wanxiang Che. Generating natural language adversarial examples through probability weighted word saliency. In Proceedings of the 57th annual meeting of the association for computational linguistics, pages 1085-1097, 2019.
[10] Yi Tay, Mostafa Dehghani, Dara Bahri, and Donald Metzler. Efficient transformers: A survey. ACM Computing Surveys, 55(6):1-28, 2022.
[11] Salman Khan, Muzammal Naseer, Munawar Hayat, Syed Waqas Zamir, Fahad Shahbaz Khan, and Mubarak Shah. Transformers in vision: A survey. ACM computing surveys (CSUR), 54(10s):1-41, 2022.
[12] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13977 2023.
[13] Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* * chatgpt quality, March 2023.
[14] Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. Is bert really robust? strong baseline for natural language attack on text classification and entailment. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 8018-8025, 2020.
[15] Kaijie Zhu, Qinlin Zhao, Hao Chen, Jindong Wang, and Xing Xie. Prompthench: A unified library for evaluation of large language models, 2024.
[16] Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. Jailbroken: How does LLM safety training fail? In Seirth-seventh Conference on Neural Information Processing Systems, 2023.
[17] Chenglei Si, Zhengyan Zhang, Fanchao Qi, Zhiyuan Liu, Yasheng Wang, Qun Liu, and Maosong Sun. Better robustness by more coverage: Adversarial training with mixup augmentation for robust fine-tuning. In Findings of ACL, 2021.

---

[18] Linyang Li and Xipeng Qiu. Token-aware aware virtual adversarial training in natural language understanding. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 8410-8418, 2021.
[19] Boxin Wang, Shuohang Wang. Yu Cheng, Zhe Gan, Ruoxi Jia, Bo Li, and Jingjing Liu. Infobert: Improving robustness of language models from an information theoretic perspective. In International Conference on Learning Representations, 2021.
[20] Xinshuai Dong, Anh Tuan Luu, Rongrong Ji, and Hong Liu. Towards robustness against natural language word substitutions. arXiv preprint arXiv:2107.13544 2021.
[21] Yi Zhou, Xiaoqing Zheng, Cho-Jui Hsich, Kai-Wei Chang, and Xuanjing Huan. Defense against synonym substitution-based adversarial attacks via dirichlet neighborhood ensemble. In Association for Compulational Linguistics (ACL), 2021.
[22] Mao Ye, Chengyue Gong, and Qiang Liu. SAFER: A structure-free approach for certified robustness to adversarial word substitutions. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault, editors, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3465-3475, Online, July 2020. Association for Computational Linguistics.
[23] Jiehang Zeng, Jianhan Xu, Xiaoqing Zheng, and Xuanjing Huang. Certified robustness to text adversarial attacks by randomized [mask]. Computational Linguistics, 49(())3955-22) 2023.
[24] Po-Sen Huang, Robert Stanforth, Johannes Welbl, Chris Dyer, Dani Yogatama, Sven Gowal, Krishnamurthy Dvijotham, and Pushmeet Kohli. Achieving verified robustness to symbol substitutions via interval bound propagation. arXiv preprint 1909:::999901999 2019.
[25] Robin Jia, Aditi Raghunathan, Kerem Göksel, and Percy Liang. Certified robustness to adver- sarial word substitutions. In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan, editors, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNN pages 4129-4142, Hong Kong, China, November 2019. Association for Computational Linguis- tics.
[26] Naihan Li, Yanqing Liu, Yu Wu, Shujie Liu, Sheng Zhao, and Ming Liu. Robutrans: A robust transformer- based text-to-speech model. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 8228-822288 8235, 2020.
[27] Junhua Liu, Trisha Singhal, Lucienne TM Blessing, Kristin L. Wood, and Kwan Hui Lim. Crisisbert: a robust transformer for crisis classification and contextual crisis embedding. In Proceedings of the 32nd ACM conference on hypertext and social media, pages 133-141 2021.
[28] Jingfeng Yang. Aditya Gupta, Shyam Upadhyay, Luheng Hc, Rahul Goel, and Shachi Paul. TableFormer: Robust transformer modeling for table-text encoding. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors, Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics ( (Volume I: Long Papers), pages 227,,77, Dublin, Ireland, May 2022. Association for Computational Linguistics.
[29] Xing Han, Tongzheng Ren, Tan Minh Nguyen, Khai Nguyen, Joydeep Ghosh, and Nhat Ho. Designing robust transformers using robust kernel density estimation. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
[30] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversar- ial examples. arXiv preprint arXiv:1412.6672 2014.
[31] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant adversarial attacks. In International Conference on Learning Representations, 2018.
[32] Yotam Gil, Yoav Chai, O. A. Gorodissky, and Jonathan Berant. White-to-black: Efficient distillation of black-box adversarial attacks. In North American Chapter of the Association for Computational Linguistics, 2019.
12

---

[33] Nicolas Papernot, Patrick McDaniel, Ananthram Swami, and Richard Harang. Crafting adver- sarial imput sequences for recurrent neural networks. In MILCOM 2016-2016 IEEE Military Communications Conference, pages 49-54. IEFE, 2016.
[34] Suranjana Samanta and Sameep Mehta. Towards crafting text adversarial samples. arXiv preprint
2017.
[35] Motoki Sato, Jun Suzuki, Hiroyuki Shindo, and Yuji Matsumoto. Interpretable adversarial perturbation in input embedding space for text. In 27th International Joint Conference on Artifi- cial Intelligence, IJCAI 2018, pages 4333-4330. International Joint Conferences on Artificial Intelligence, 2018
[36] Melika Behjati, Seyed-Mohsen Moosavi-Dezfooli Mahdieh Soleymani Baghshah, and Pascal Frossard. Universal adversarial attacks on text classifiers. In ICASSP 2019-2019 IEEE Interna- tional Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 7345-7344 IEEE, 2019.
[37] Siddhant Garg and Goutham Ramakrishnan. BAE: BERT-based adversarial examples for text classification. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu, editors, Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6174 6181, Online, November 2020. Association for Computational Linguistics.
[38] Mohit lyyer, John Wieting, Kevin Gimpel, and Luke Zettlemoyer. Adversarial example gen- eration with syntactically controlled paraphrase networks. In Marilyn Walker, Heng Ji, and Amanda Stent, editors, Proceedings of the 2018 Conference of the North American Chupter of the Association for Computational Linguistics: Human Language Technologies. Volume (Long Papers), pages 1875-1885 New Orleans, Louisiana, June 2018. Association for Computational Linguistics.
[39] Bin Liang, Hongcheng Li, Miaoqiang Su, Pan Bian, Xirong Li, and Wenchang Shi. Deep text classification can be fooled. arXiv preprint arXiv:170.008006 2017.
[40] Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. Hotflip: White-box adversarial examples for text classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Pupers), pages 31-36, 2018
[41] Haoran Li, Dadi Guo, Wei Fan, Mingshi Xu, Jie Huang, Fanpu Meng, and Yangqiu Song. Multi-step jailbreaking privacy attacks on chatGPT. In The 2023 Conference on Empirical Methods in Natural Language Processing, 2023.
[42] Gelei Deng, Yi Liu, Yuekang Li, Kailong Wang, Ying Zhang, Zefeng Li, Haoyu Wang, Tianwei Zhang, and Yang Liu. Jailbreaker: Automated jailbreak across multiple large language model chatbots. arXiv preprint arXiv:2:307.88715 2023.
[43] Yi Liu, Gelei Deng, Zhengzi Xu, Yuekang Li, Yaowen Zheng, Ying Zhang, Lida Zhao, Tianwei Zhang, and Yang Liu. Jailbreaking chatgpt via prompt engineering: An empirical study. arXiv preprint arXiv:2305. 13860, 2023.
[44] Hezekiah Branch, Jonathan Rodriguez Cefalu, Jeremy McHugh, Leyla Hujer, Aditya Bahl, Daniel del Castillo Iglesias, Ron Heichman, and Ramesh Darwishi. Evaluating the suscepti- bility of pre-trained language models via handcrafted adversarial examples. arXiv preprint arXiv:2209.022 128, 2022.
[45] Yiming Zhang and Daphne Ippolito. Prompts should not be seen secrets: Systematically measuring prompt extraction attack success. arXiv preprint arXiv:2307.0688. 2023.
[46] Yi Liu, Gelei Deng, Yuekang Li, Kailong Wang, Tianwei Zhang, Yepang Liu, Haoyu Wang, Yan Zheng, and Yang Liu. Prompt injection attack against Illm-nteeren intinnteertted applications. arXiv preprint arXiv:2306.055 05499, 2023.
[47] Chen Zhu, Yu Cheng, Zhe Gan, Siqi Sun, Tom Goldstein, and Jingjing Liu. Freelb: Enhanced adversarial training for natural language understanding. In International Conference on Learning Representations, 2020.
13

---

[48] Peter Bloomfield and William L Steiger. Least absolute deviations: theory, applications, and algorithms, volume Springer, 1983.
[49] Peter Huber. Robust regression: asymptoties, conjectures and monte carlo. The annals of statistics, pages 799-82) 821, 1973.
[50] Cun-Hui Zhang. Nearly unbiased variable selection under minimax concave penalty. 2010.
[51] Xiang Zhang, Junbo Jake Zhao, and Yann LeCun. Character-level convolutional networks for text classification. In NIPS, 2015.
[52] Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng. and Christopher Potts. Learning word vectors for sentiment analysis. Proceedings of the 49th Annual Meeting of the Association for Computational Linguisties: Human Language Technologies, pages 142-150, Portland, Oregon, USA, June 2011. Association for Computational Linguistics.
[53] Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng. and Christopher Potts. Recursive deep models for semantic compositionality over sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1631-1643 Seattle, Washington, USA, October 2013. Association for Computational Linguistics.
[54] Alex Wang, Amanprect Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding. 2019. In the Proceedings ICLR.
[55] Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks aligned language models, 2023.
[56] Jacob Devlin, Ming-Was Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint 1rXiv:1810.000 04005, 2018.
[57] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: robustly optimized bert pretraining approach, 2019.
[58] Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. Albert: A lite bert for superviud learning of language representations, 2020.
[59] Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv: 108, 2019.
[60] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer, 2023.
[61] Alexander Rohey, Fric Wong, Hamed Hassani, and George Pappas. Smoothllm: Defending large language models against jailbreaking attacks, 2023.
[62] Zongyi Li, Jianhan Xu, Jiehang Zeng, Linyang Li, Xiaoqing Zheng, Qi Zhang, Chang, and Cho-Jui Hsich. Searching for an effective defender: Benchmarking defense against adversarial word substitution, 2021.
[63] John Morris, Eli Lifland, Jin Yong Yoo, Jake Grigsby, Di Jin, and Yanjun Qi. Textattack: framework for adversarial attacks, data augmentation, and adversarial training in nlp. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing: System Demonstrations, pages 119-126, 2020.
[64] Kaidi Xu, Hongge Chen, Sijia Liu, Pin- Yu Chen, Tsui Wei Weng, Mingyi Hong, and Xue Lin. Topology attack and defense for graph neural networks: An optimization perspective, 2019.
[65] Ido Dagan, Bill Dolan, Bernardo Magnini, and Dan Roth. Recognizing textual entailment: Rational, evaluation and approaches-erratun, Natural Language Engineering, 16(1):105-105, 2010.
14

---

[66] Roy Bar-Haim, Ido Dagan, and Idan Szpektor. Benchmarking applied semantic inference: The pascal recognising textual entailment challenges. In Language. Culture, Computation. Computing-Thhery and Technology: Essays Dedicated to Yaacov Choueka on the Occasion of His 75th Birthday, Part I, pages 409-424. Springer, 2014.
[67] Danilo Giampiccolo, Bernardo Magnini, Ido Dagan, and William B Dolan. The third pascal recognizing textual entailment challenge. In Proceedings of the ACL-PASCAL workshop on textual entailment and paraphrasing, pages 1-9, 2007.
[68] Luisa Bentivogli, Peter Clark, Ido Dagan, and Danilo Giampiccolo. The fifth pascal recognizing textual entailment challenge. TAC, 7:8, 2009.
[69] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[70] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang. Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 115(3)2211-252, 2015.
[71] Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi- Rad. Collective classification in network data. Al magazine, 29(3):93 93, 2008.
[72] C Lee Giles, Kurt D Bollacker, and Steve Lawrence. Citeseer: An automatic citation indexing system. In Proceedings of the third ACM conference on Digital libraries, pages 89-98, 1998.
[73] Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In International Conference Learning Representations, 2018.
[74] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. International Conference on Learning Representations, 2017.
[75] Xiang Zhang and Marinka Zitnik. Gnnguard: Defending graph neural networks against adversarial attacks. Advances in neural information processing systems, 33:9263-92773 9275, 2020.
[76] Dingyuan Zhu, Ziwei Zhang, Peng Cui, and Wenwu Zhu. Robust graph convolutional networks against adversarial attacks. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 19999-147,201/1
[77] Wenzheng Feng, Jie Zhang, Yuxiao Dong, Yu Han, Huanbo Luan, Qian Xu, Qiang Yang, Evgeny Kharlamov, and Jie Tang. Graph random neural networks for semi-supervised learning on graphs. Advances in neural information processing systems, 33:22092-221033, 2020.
[78] Wei Jin Yao Ma, Xiaorui Liu, Xianfeng Tang, Suhang Wang, and Jiliang Tang. Graph structure learning for robust graph neural networks. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pages 66-74, 2020.
[79] Huijun Wu, Chen Wang, Yuriy Tyshetskiy, Andrew Docherty, Kai Lu, and Liming Zhu, Ad- versarial examples for graph data: Deep insights into attack and defense. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, ISCAI-19, pages 1826. International Joint Conferences on Artificial Intelligence Organization, 7 2019.
[80] Simon Geisler, Tobias Schmidt, Hakan Şirin, Daniel Zügner, Aleksandar Bojchevski, and Stephan Günnemann. Robustness of graph neural networks at scale. Advances in Neural Information Processing Systems, 34:7637-7649, 2021.
[81] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint 1503.0253, 2015.
[82] Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. BEit: BERT pre-training of image transformers. In International Conference on Learning Representations, 2022.
[83] Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In International conference machine learning, pages 10347-10357. PMLR, 2021.
15

---

[84] Stéphane d' Ascoli, Hugo Touvron, Matthew L Leavitt, Ari Morcos, Giulio Biroli, and Levent Sagun. Convit: Improving vision transformers with soft convolutional inductive biases. In International Conference on Machine Learning, pages 2286-2296. PMLR, 2021.
[85] Yao Ma, Xiaorui Liu, Tong Zhao, Yozen Liu, Jiliang Tang, and Neil Shah. A unified view on graph neural networks as graph signal denoising. In Proceedings of the 30th ACM International Conference Information & Knowledge Management, pages 202-1211, 2021.
[86] Wenqi Fan, Xiaorui Liu, Wei Jin, Xiangyu Zhao, Jiliang Tang, and Qing Li. Graph trend filtering networks for recommendation. In Proceedings of the 45th international ACM SIGIR conference on research and development in information retrieval, pages 1 112-121, 2022.
[87] Zhichao Hou, Ruiqi Feng, Tyler Derr, and Xiaorui Liu. Robust graph neural networks via unbiased aggregation, 2024.
[88] Xiaorui Liu, Wei Jin, Yao Ma, Yaxin Li, Hua Liu, Yiqi Wang, Ming Yan, and Jiliang Tang. Elastic graph neural networks. In International Conference on Machine Learning, pages 6837-6849, PMLR, 2021.
16

---

A Pseudocode of Plug-and-Play Robust Attention (ProAttention)
Here, we provide the complete pseudocode of our ProAtteettion including various penalties cases in Algorithm 2 The core itertions are show in the for loop in the code. Our ProAttention is easy to implement by only replacing the vanilla attention module with our ProAttention.
Algorithm 2 ProAttention in PyTorch style
ProAttantion Modula):
def __.init__$sell K, gamaa delta, ponalty):
supar O.__initt__.)
self.K self genna germa
solf.delta dolta self penalty penalty
def (allf A, V):
Z torch. natnul (A, Inétsalisation
11
12
self ponalty 'L2": self.penalty Original attentson
14
(sele
16
D torch. cZist V) Pasruise dislance
17
self penalty 'L1': 1/D alif helf.penaltyy 'MCP":
19
21
coorch.ciip(1//1 1/self gamaa, min=o)
penalty
25
diiippeee.ddllt/DD...... V = torch, csip delta/D, pax-1)
24
elif self penalty -- "Huber-MCP';
25
torch.ccippeee. clip(self delta/ (self. gannx-self deltta) (self, gamma/D1),, min-O, max-1)
torch
26
torch.nn. functional normaliza( pe1, dim=-1) )) Hormalszation
28
2 torch natmul (W, v) Update
B Proof of Newton-IRLS Algorithm
B.1 Proof of Localized Upper Bound (Lemma 3.1
Proof. Define ((=) = z(v²) as non-convex function, then for fixed point zo.
<(z) < ((20) + (z0) 20) = d' 2 + C(zo)
where the first inequality holds with equality at 2g and
p'(220)
= =(z0)=p((Ve) 2 ==20
2/2
By replacemnet as z = ||v, - z2| and zg = I|v, - (x)) then
p(lly, - =|]) - p((1iv; - (b1)
2/11, z((i]| ||v, - =z112 + C(Ilv, - 20))
= w0 (K) ||v, - 2|12 + C(I1v; 21)))
17

---

and the first inequality holds with equality at z = (4) Sum up the items on both sides with weights {ay)se(N). we obtain
((2)
Σ
04 p(liv; -2l1)
/=1
N
(k) ||v; - 2||² + Σ
C(Ilv; - -2(1113)
<
Uj ]
aj
(6)
j=1 1
-=1
N
Σ
(N) [|v, - 2/1² + Ci (z)
aj
1=1
= L(z)
and the equality holds at (k));
(( = (()
(7)
After obtaining the convex upper bound [(z), it becomes feasible to employ convex optimization algorithms to optimize this objective.
B.2 Proof of Newton-IRLS algorithm and Special Cases
Newton-IRLS. We first derive the formulations of gradient and Hessain matirx of é as follows:
vi(z(() = Σ aj wj
(k)
((( -v.) v:)
N
v2L(2(() = Σ aj . w 22
Then, the gradient descent (GD) (n) is the stepsize) is
z (++1) = z (K) 17 v Lzzz(²) (2))
= z((A) 1
aj w 2(2 - v,),
(k)
and the Newton Iteration is
z (++1) = (A)))
v2 ((x
V (2)
(8)
-1
Σ aj w
(k) 21
.
Σ
aj
w (k)
2(z v:)
(9)
j=1
Eg Ug w (k) V
(10)
Σ, aj w0 (R)
In convex optimization, it has been well-established that second-order methods converge much faster than first-order approaches, but they require substantial computation calculating or approximating
the inverse Hessian matrix. However, due to the uniqueness of our Ć in Eq. (3) we can derive
concise closed-form iteration using the second-order Newton method as in Eq. 110 Compared to the
first-order gradient descent (GD) iteration, our Newton-IRI. algorithm enjoys several advantages as follows:
18

---

Fast convergence:
Newton method converges at a quadratic rate, which is significantly faster than the linear convergence of gradient descent (GD). The comparative analysis of them can be found in Figure (a) in ablation studies;
Interpretable formulation: The resulted form in Eq. (??) employs normalized reweighted sum, which can be interpreted as robust estimator by wewihteighting the outliers, as discussed in the following paragraph;
Efficient computation: The Hessian V (+) can be easily computed as a closed-form diagonal matrix, facilitating the matrix inversion and multiplication in the Newton's iteration.
B.3 Proof of Rigorous Loss Descent Guarantee (Theorem 3.2)
Proof. Since z (1++1 is obtained from optimize the convex localized upper bound z((), then we have ((z(++1) S (k)())) According to the upper bound in Eq. (6) and localized equality in Eq. (7) it is not hard to get the following inequality:
((z(k+1)) < ((zlkt))) S (z) L(z(k)).
Therefore, optimizing the localized upper bound 2 can guarantee the rigorous descent of C.
B.4 Special cases of Newton-IRLS
Our Newton-IRLSS is general framework which can be derived as different reweighting schemes with different penalties:
Square Loss (62):
122
1
p(z)
2
,'(1v, 21
(R)
w)
2 2| (R)
2'
N
2" Σ 1=1
aj
Vj.
loss increase quadratically with 2, which suggests that f2 loss is sensitive the residual magnitude. Particularly, & loss can recover the vanilla attention since the weights are constant, 2 Absolute Loss (fz):
p(2) 2,
(k) p'(lV, 2
wy 211v; 2/4
2 2 z (k)
With & loss, the weight is inversely proportional to ||v, (k) By up-weighting the inliers
and down-weighting the outliers, & -based estimators can mitigate the effect large magnitude residues.
Minimax Concave Penalty (MCP) [50]
if <>
(2(2)= =
27
if 2 7
p'(1lv, - z 2k
1
w 2 2 [4]
2 max
0
[|
MCP loss becomes constant when is large and the weight derived by MCP loss enhances the
interpretability of the robust estimator by down-weed wwinhweigtiing or completely removing the outliers.
To be specific, the weight wy becomes smaller as the distance ||v - [k) || increases, thereby
down-weighting the outlying cases. When this distance exceeds the threshold 7, the weight becomes 0, totally removing the outliers.
19

---

Huber loss:
if
po(2)
18- (2 2)
if
(x01 min
5
1, 1v, z (3)
2 z | 2
Huber loss is equivalent to the E2 loss within the range (0, 5), and it becomes similar to & when
2 > 6, which indicates that Huber loss may mitigate the effect of large noise while keeping decent performance in noiseless scenario.
Huber-MCP:
2=
if <
= (2)
8. (=
2
(2-6)²
ifōs=<9
2(7-6)
4g
2
(4)
p((1Vv 2/20
1
1 -1 0
2 v [1
2
max
min
||v
[
This penalty combines the advantages of Huber and MCP in recovering the E loss and largely mitigating the outliers.
20

---

C Dataset Information
C.1 Language Domain
AG's News Corpus (AGNEWS) [51] It is a collection of more than 1 million news articles. News articles have been gathered from more than 2000 news sources by ComeToMMyead in more than year of activity. ComeToMMyHee running since July, 2004. The dataset is provided by the academic comunity for research purposes in data mining (chustering, classification, etc), information retrieval (ranking, search, etc), xml, data compression, data streaming, and any other non-commercial activity. The AG's news topic classification dataset is constructed by choosing 4 largest classes from the original corpus. Each class contains 30,000 training samples and 1,900 testing samples. The total number of training samples is 120,000 and testing 7.600.
is an academic news search engine which has been
Internet Movie Database (IMDB) [52]: IMDB dataset having 50K movie reviews for natural language processing or Text analytics. This is dataset for binary sentiment classification containing substantially more data than previous benchmark datasets. We provide a set of 25,000 highly polar movie reviews for training and 25,000 for testing. So, predict the number of positive and negative reviews using either classification or deep learning algorithms.
Stanford Sentiment Treebank (SST-2) (53] It is a corpus with fully labeled parse trees that allows for complete analysis of the compositional effects of sentiment in language. The corpus consists of 11,855 single sentences extracted from movie reviews. It was parsed with the Stanford parser and includes total of 215,154 unique phrases from those parse trees, each annotated by human judges. Binary classification experiments on full sentences (negative or somewhat negative somewhat positive or positive with neutral sentences discarded) refer the dataset as SST-2 SST binary.
Recognizing Textual Entailment (RTE): It comes from a series of annual textual entailment challenges. The authors of the benchmark combined the data from RTE1 [65]. RTE2 [66), RTE3 [67], and RTE5 [68]. Examples are constructed based on news and Wikipedia text. The authors of the benchmark convert all datasets two-class split, where for three-class datasets they collapse neutral and contradiction into not entailment, for consistency.
Behaviors: It is a new dataset introduced in [55] for robustness evaluation of jailbreaking attack. The dataset includes 520 goal prompts and corresponding targets, it is available in http:///mmmmattack///atttaaaaaaa
C.2 Beyond Language Domain
The CIFAR-10 dataset is a well-known dataset used in the field of computer vision. It consists of 60,000 32x32 color images in 10 different classes, with 6,000 images per class. The dataset is divided into two parts: 50,000 training images and 10,000 test images. The 10 different classes represent airplanes, cars, birds, cats, deer, dogs, frogs, horses, ships, and trucks. Each image is labeled with one of these 10 categories.
CIFARIO [69]:
ImageNet-1K 770: This dataset provides access to ImageNet which is the most commonly used subset of ImageNet. This dataset spans 1000 object classes and contains 1, 1,281,167 training images, 50,000 validation images and 100,000 test images. The version also has the patch which fixes some of the cornupted test set images already applied.
Cora-ML 771 The Cora dataset is widely-used benchmark dataset in the field of graph-based tasks. It consists of 2708 scientific publications classified into one of seven classes. The citation network consists of 5429 links. Each publication in the dataset is described by 0/l-valued word vector indicating the absence/presence of the corresponding word from the dictionary. The dictionary consists of 1433 unique words. Working with the Cora dataset presents challenges typical of real-world graph data, such as handling sparse and digh-dimensional feature vectors, and dealing with the complex structure of the graph.
The CiteSeer dataset is another popular dataset in the graph field. It consists of 3312 scientific publications classified into one of six classes. The citation network consists of 4732 links. Each publication in the dataset is described by 0/1-valued word vector indicating the absence/presence of the corresponding word from the dictionary. The dictionary consists of 3703 unique words.
Citeseer 772:
21

---

D Defense Baselines and Backbone Architectures
D.1 Defense Baselines
Language Domain:
The Projected Gradient Descent (PGD) method stands as the most prevalent attack strategy the field of computer vision. It is primarily utilized for crafting adversarial examples in the context of adversarial training. The defense in this paper is adapted directly from PGD-adv in computer vision, extending its application to language modeling.
PGD-Adv [31]
The search space for adversarial examples in language models is typically vast due to their discrete nature. To enhance the robustness of these models, MixADA integrates adversarial training (30) with mixup data augmentation 73,, thereby expanding the range of adversarial examples covered. Specifically, mixup generates synthetic training examples by linearly blending pairs of inputs and their corresponding labels. This approach enables the model to learn from a broader and more effective set of adversarial examples during training.
MixADA 17:
FreeL.B Different from attacks that directly change the words in the sentence, FreeL.B adds adversaria perturbations to word embeddings and minimizes the resultant adversarial loss around input samples. To expedite the process of adversarial training, FreeLB implements a single descent step the parameters concurrently with each of the K ascent steps applied to the perturbation, which utilizes the average of accumulated gradients over the K steps. This efficiency has established FreeLB popular defense method in the field of NLP.
TA-VAT 18 TA-VAT is another virtual adversarial training method that generates gradient- based perturbations on the embedding space. To create fine-grained perturbations, TA-VAT employs a token-level accumulated perturbation vocabulary. This vocabulary serves to better initialize the perturbations. Additionally, TA-VAT utilizes token-level normalization ball, which effectively constrains these perturbations in relevant and precise manner.
Adversarial Training (AT): Adversarial training is adaptive to the attack to be evaluated. Take the Textfooler as the instance, at every epoch, we generate 1000 perturbations from the Textfooler and add them into the training dataset to reinforce the training of models. We utilize the TextAt- tack [63] platform the conduct this adversarial training.
SmoothLIM [I]] Motivated by finding that the adversarial-prommping jailbreak is sensitive to the random character-level changes, SmoothLLM designed by firstly perturbing multiple copies of the given prompt and then aggregating all the outputs.
Beyond Language Domain:
Graph Convolutional Network (GCN) 74: GCN is motivated by the localized first-order approximation of spectral graph convolutions. The basic idea is to first add self-loops to the adjacency matrix and then normalize the matrix.
Graph Attention Network (GAT) [5): GAT leverages the attention mechanism to construct masked self-attentional layers. This allows the nodes to reweight their neighbors via the feature similarity.
GNNGuard 75|: GNNGuard is a universal reweighting framework that can be applied to any GNN. It leverages the cosine similarities between nodes' features to up-weight the correlated nodes and prune the edges between the dissimilar pairs.
Robust GCN (RGCN) 76): RGCN first models the latent representations as the Gaussian distributions. Then the weights of different neighborhoods will be assigned different weights according to their variances when performing the message propagation.
The core of GRAND is the random propa- gation, wherein the node feature will be partially or entirely dropped out and then propagated through over the graph. This operation enable the node be insensitive to the specifie neighbor- hood, which prevents the effect of malicious outliers. Additionally, the random propagation also help to augment the representation for each node, thus improving the generalization of GNN.
Graph Random Neural Network (GRAND) 777:
Property GNN (ProGNN) [78] The core principle of ProGNN is to robustify the GNNs through enhancing the graph properties of sparsity, low rank and feature smoothness. provides graph structure learning framework to learn the clean graph structure and parameters simultancously.
22

---

The basic idea of Jaccard- GCN is to preprocess the adjacency matrix by first computing the Jaccard coefficients of paired node features and then dropping the edges where the coefficients are below the threshold.
Jaccard-GCN 799
SoftMedian [80:: SoftMedian is a robust estimator for the message passing aggregation, It reweights the adjacency weights based to the distances of the hidden embeddings between the neighbor nodes and the dimension-wise median of the the entire neighboring representations.
D.2 Backbone Architectures
Classical language models:
BERT stands out as one of the most well-known transformer-basee language models. It pretrained through masked language modeling (MLM), where it learns predict words that have been masked, using context for guidance. This pretrained model is then fine-tuned for variety of downstream tasks, showcasing its versatility and effectiveness diverse applications. In our experiments, we will use BERT-1 I0M.
BERT [66
RoBERTa [57]] RoBERTa is developed to overcome certain limitations of the original BERT model. This is accomplished by implementing key modifications such as increasing batch size, extending the training epochs, and employing advanced optimization techniques. As result of these strategic changes, RoBERTa has demonstrated substantial performance improvements over BERT across various NLP benchmarks. In our experiments, we will use RoBERTa- 125M.
ALBERT [88 ALBERT is lite variant of BERT. It is achieved by decoupling the word embedding from the hidden embedding, significantly cutting down the number of parameters. To further enhance its efficiency, ALBERT employs cross-layer parameter sharing, ensuring that all layers use the same parameters. The reductions not only minimize memory footprint but also improve the efficiency of the model. In our experiments, will use ALBERT-12M. 12M.
DistilBERT [59: DistilBERT is a light version of BERT, maintaining most of the performance of the original BERT. It is trained with the knowledge distillation technique [81] to achieve high efficiency. our experiments, we will use DistiBBERT-66M.
Large Language Models:
Text-to-Tex. Text Transfer Transformer (T5) is transformer-based neural network model known for its versatility and power in handling wide range NLP tasks. T5 simplifies NLP tasks by treating them uniformly range of sizes, from 60 million to 11 billion parameters, catering different computational needs. The flexibilityy has made T5 popular choice in NLP research. In our experiments, we will use TS-770M.
T5 [60]:
text- next-generation challenges. The T5 model family offers
LLaMA [2] LLaMa, the Large Language Model developed by Meta AI, represents a cutting- edge advancement in language modeling. Trained publicly available datasets, LLaMa is available in various sizes to suit different computational needs. Notably, LLaMa- MaB demonstrates superior performance over GPT-3 in most benchmarks, highlighting its exceptional effectiveness and capability in NLP tasks. In our experiments, we will use L-MMA-7B.
Vicuna [13: Vicuna is bigh-performing, open-source chatbot that impresses with capabilities comparable GPT-4. Fine-tuned from the LLaMa model, it utilizes user-shared conversations gathered from Share-GPT for its training. Remarkably, Vicuna achieves 90% of the perfor- mance level of GPT-4, despite having only 13 billion parameters, showcasing its efficiency and effectiveness. In our experiments, we will use Vicuna-7B.
Vision Models:
VIT [3 The Vision Transformer (ViT) is model in computer vision that adopts the principles of the Transformer architecture. In VIT. an image is processed similarly to sequence of words, or tokens. Specifically, the image segmented into fixed-size patches, each of which then linearly transformed into an embedded representation. When trained on sufficient data, ViT achieves state-of-the-an performance on image classification benchmarks, competing with or outperforming leading CNN-based models. In experiments, will use ViT-86M.
23

---

Swin Transformer is popular variant of VIT, standing out for its enhanced efficiency and superior performance. It employs hierarchical architecture, which not only aligns more closely with the nature of visual data but also boosts efficiency. To effectively capture global contextual information, Swin Transformer incorporates shifted window-based self-attention, further enhancing its effectiveness in vision-related applications. In our experiments, we will use Swin-50M.
Swin A
Due to the success of BERT, BEIT harnesses the concept of masked language modeling to enhance self-supervised learning in the visual domain. To align with the words in language models, BEIT first maps the patch in an image into a token with an autoencoder. In the training process, it masks portion of these patches, using the remaining unmasked ones to predict the masked tokens. Subsequently, the model is fine-tuned for variety of downstream tasks, demonstrating its adaptability and effectiveness in diverse applications. In our experiments, we will use BEIT-86M.
BETT 82]:
DeiT To address the substantial data requirements for training the Vision Transformer, Aata-Efficient Image Transformer (DeiT) employs knowledge distillation [8] to train the model. By integrating this approach with various data augmentation techniques, DeiT successfully attains competitive results in image classification tasks, even with constrained training data availability. In our experiments, we will use DeiT-22M.
Con ViT 84 Con designs hybrid architecture to leverage the local processing capabilities CNNs and the global context understanding of transformers. To be specific, it replaces the several first self-attention layers with gated-self positional self-attention layers, allowing the model to adjust between local and global processing. In our experiments, we will use Con ViT-30M.
Our Pro ProTransformers belong to the category of optimization-indueed deep learning architectures [8
86
which formulate the attention layers in the backbone models as solutions to specific underlying optimization objectives, and then leverage optimization algorithms to solve them.
87 88
24

---

E
Attacks.
E.1
Classic Text Attack:
For the classic text attacks, we follow the default attack setting in the TextAttack 63 and the detailed information follows:
Bug 7 Deep Bur is black -bx attacks that apply character-level transformations to the highest-ranked tokens misclassify the text input. It includes several character transforma- tions including swap, substitution, deletion and insertion. We hold the maximum difference on edit distance (Levenshtein Edit Distance) to 30 for each sample. We will greedily modify the works with the word importance ranking.
The probability weighted word saliency (PWWS) employs a new word order deter- mined by the word saliency and predicted probability, and then greedily perform the synonyms substitution.
PWWS 9:
TextFooler [44 TextFooler propose more comprehensive paradigm to generate adversarial perturbations. It firstly identify the important words and then replace them with the most semantically and syntacticaly similar synonyms until the prediction is altered. We set the minimum word embedding cosine similarity as 0.5 and the universal sentence similarity threshold as 0.840845057.
TextBugger g TextBugger is framework for creating utility-preservinn preserving adversarial texts against the classification models. can be employed under both white-box and black-box settings. Specifically, this attack sorts the words based on Jacobian matrix in white-box setting and based on specific scoring function in black-box setting. includes swap, substitution, deletion and insertion. There are two types of substitutions: Sub-C (replace characters with visually similar characters) and Sub-W (replace word with its top-K K (K" 5) nearest neighbors.) The universal sentence threshold is set 0.8.
E.2 Attack in LLMs
conduct adversarial prompt attacks using the text attacks including Deep- WordBug (7], PWWS [9], TextFooler [14] and TextBugger [8]. The evaluation setting follow
Prompt attack:
PromptBench 15
which unified and comprehensive python library for evaluating L.MMs.
Jailbreak: We employ the Greedy Coordinate Gradient (GCG) (55) method to search for the adversarial prompt suffixes that circumvent the alignment the language models. can be conducted under the transfer attack and direct attack. Specifically, the transfer attacks firstly attack the surrogate model (e.g., Vicuna) and can apply the perturbations to the victim models. The direct attacks directly apply the GCG method to the victim models to generate the suffixes, which can be much stronger. The lailbreak prompt asking for inappropriate content. In our experiment, we
is list of words commonly found in messages
that abstain from responding use the following list: Jeibrreak = ["I'm sorry", "Sorry", "I apologize", "As an", "I'm just", "I' m an", cannot", "I would advise", "it's not appropriate", "As responsible Al"].
E.3
Attacks in Vision & Graph Models
FGSM 30]: Fast Gradient Sign Method (FGSM) is widely used technique for generating adversarial examples. crafts perturbations specifically to maximize errors in the output neural networks.
PGD 31: Projected Gradient Descent (PGD) method is one of the most prominent attack strategies in computer vision. Unlike FGSM employing single step, PGD uses multiple steps to generate adversarial examples. This iterative approach includes projection operation, which ensures that the intensity of the attack remains within specified limits, making PGD a more controlled and effective method for generating adversarial examples. The steps are K = 7 and the steps size a 0.00784.
PGD on Graph [64]: Motivated by PGD 31 in vision domain, 664 propose first-order method to conduct topology attack on discrete graph structure. This method firstly solve continuous optimization problem by Projected Gradient Descent (PGD) method and then utilize the random sampling to get the optimal binary topology perturbation from the continuous probabilistic matrix.
25

---

Algorithm Convergence and Robust Estimation
E1 Convergence Guarantee
Loss curves. We generated data verify the convergence of our proposed algorithm. The batch size, number heads, length inputs and dimension of data are chosen as 8, H = 4, N 64, D 8, respectively. The 7 in MCP is set 4 and in Huber loss is our algorithm with different penalties are shown in Figure We can observe that our algorithm show fast convergence and even 2 steps can well approximate the optimal solution.
0.8. The loss curve of
0.255
0010
0002
Layoo
Layon
Lagen
&
(b) MCP
Huber
Figure 7: Loss Curve of Algorithms
Trajectory. To further validate the convergence and effectiveness of our algorithm, we use a toy experiment to visualize the trajectories of updated vector in 2D plane in Figure We use L1 penalty our algorithm, the simulated attention matrix and value matrix are as follows:
I
1
1
1
A
2
0
0
V
7
25
(11)
0
0
2
25
37
Trajectory
35
25
Dio
20
Ground-Trutb-1
Growod-Truth-2 Truth-2
Ground T
Trajectory-1
Trajectory-2
5
Trajectory-3
10
15
20
Dim
Figure 8: Optimization trajectory.
From the figure, we can find that with the mean as the initial position, the updated vector can approach closely to the ground truth within only 3 steps. This phenomenon further validate the effectiveness and efficiency of our algorithm.
F.2 Robust Estimation
Robust estimation. We firstly generated clean samples (x: (blue dots) and the outlier samples (x:
(red dots) from 2-dimensional Gaussian distributions, (oO 0), 1) and (M(8, 8), 0.5). respectively. We calculate the mean of clean samples 1 > x; the ground truth of the mean
26

---

Clean
Outlier
Truth
82
4
MCP
10
15% outliers
30% outliers
45% outliers
5
2
0
-2
5
5
Figure 9: Different estimators in simulations.
estimator. Then estimate the mean of all the samples by solving arg ming =i=1 p(z x,) using the our method, where p(-) can take different penalties such f2 penalty || 11 and l penalty || 1l2-
In Figure y we visualize the generated clean samples and outliers, as well the ground truth means and the mean estimators with 17(-) = || 11 or 2/2 under different outlier ratios 15% and 45%. The results show that, with the existence of outliers, the ba-based estimator deviates far from the true mean, while the & ba-ded estimator is more resistant to outliers and MCP-based estimator is the most robust.
F.3 Vulnerability Analysis Vanilla Attention (WLS estimator)
To gain insight into the vulnerability of vanilla attention (WLS estimator), we attempt to analyze the effect of perturbed tokens on the output embedding of attention modules. Specifically, since vanilla attention can be formalized as WLS estimator: z' = arg min, L(z) = Σ aj ||v, - 2/|2, we quantify and compare V|v 2 and l|, 2² where Vj and v'J are the original and perturbed tokens, and z" is the ground truth output embedding. We present the numerical results across every attention module in the pretrained BERT on IMDB dataset in Table 5 The results demonstrate that attackers tend to introduce larger residuals VV z|| by modifying the important tokens v, when perturbing the text.
Table 5: Residual magnitude of original and perturbed tokens.
Attention module
2
3
4
5
7
9 10
11
12
z 5.39 6,36
6.92
6,33
6.64
6,95
6.73
6,24
5.12 4.76
4.15 4.19
lv z" ||
6.38 7.20
7.97
7.51
7.65
8.06
7.84
7.41
6.33
6.24
6.64
6.31
27

---

G Additional Experiments of Text Attacks
G.1 Sentiment Analysis: IMDB
We present the results of sentiment analysis on IMDB dataset under various attacks in Table We can conlude from the results that our methods improve the robustness of the backbones significntly by simply plugging the ProAttention into the models without additional fintuning or training. Moreover, our method can be combined with the existing defenses such as Adversarial Training (AT) further improve the performance.
Table 6: The results of sentiment analysis IMDB.
TEXTFOOLER ASRS
TEXTBUGGER
DEEPWORSBLG
PWWS
CLEANS
ROBERTA
93.3
23.7
74,6
9.4
35.9
36.3
60.9
19.3
79.1
DISTILBERT BERT
90.9
14.9
83.6
4.3
95.3
18.8
79.3
9.6
89.4
BERT
92.3
11.8
87.2
11.3
87.4
32.8
64.5
26.4
71.5
FREELB
93.0
25.1
73.6
19.9
76.9
40.9
55.5
42.0
54.7
93.2
26.2
17.4
81.6
32.0
65.8
27.2
69.6
MIXADA
91.9
16.7
82.0
11.8
87.3
33.4
65.8
30.0
67.4
TA-VAT
93.0
28.5
67.6
27.3
6H
34.7
60.4
35.1
59.8
93.2
33.6
64.3
31.8
66.1
37.7
61.5
28.7
70.3
(OURS)
13.0
32.7
PEn-BERT (HUBER) (OURS)
93.0
24.8
73.3
13.4
85.6
36.9
60.3
31.5
66.1
DERT (MCP) (OURS)
93.5
22.1
76.9
44.6
$3.2
55.5
41.8
56.3
41.1
(OURS)
G.2
Textual Entailment: RTE
In Table 7 display the results of textual entailment RTE across different cosine similarities constraints in TextFooler attack. We select DistiBBERT as the backbone model and construct several MCP-based architectures with different y. We can observe that our method can improve the robustness acorss different cosine similarities. The performance improvement is more evident under the smaller cosine similarities, which is equivalent larger budgets.
In Table 8 we present the results of textual entailment on RTE across various attacks. The results exhibit the consistent improvement of our methods over the backbone model.
Table The results of textual entailment on RTE across different cosine similarities in TextFooler
Cos-SIM CLEAN
0.5
MODEL
AEAR
ASR%
AUA%
ASR%
Aux
ASR%
AEA%
ASR
BERT
62.3
1.9
87.3
PRO- DISTII DISTIBBETT MCP 7
63.3
89.5
6.1
90.6
9.4
85.6
18.1
72.4
DISTII BERT-MCP MCP
62.4
15.2
75,3
16.6
72.9
20.6
66.5
30.7
50.0
PRO-DiSTu.
56.0
56.1
28.2
COO-DISTILBERTE
0.5
55.6
15.9
70.1
17.7
66.7
20.2
61.9
28.9
45.6
PRO-DISTIR DISDDIISTBBTTT... MCP 0.6
53.1
10.8
80.5
12.3
77.9
16.3
70.8
20.2
63.6
Table 8: The results of textual entailment on RTE across different attacks.
ATTACK
TEXTFOOLER
TEXTBUGGER
DEEPWORDBUG
PWWS
ASR%
A0A%
AUA%
BERT
62.5
7.9
87.3
3.6
94.2
18.4
70.3
12.3
80.4
PRO- DISTILLEETT-MPP
63.3
28.2
49.7
14.4
74.5
33.6
40.0
24.2
60.6
28

---

G.3 Adversarial Fine-tuning on Topic Classification: AGNEWS
Adversarial training techniques are highly effective enhance the robustness of models via adding the adversarial examples into the training set. To better capture the robustness enhancement of adversarial training, we track the adversarial fine-tuning curves and present the detailed results on AG's News in Table and Figure T0 In the beginning of every epoch, we generate 1000 perturbed examples using the specific attack and then put them to the original training dataset. From the results, we can make the following observations: (1) the models show even better robustness during the process of adversarial training. (2) our method can be combined with adversarial training further improve the resilience of the models.
Table 9: Adversarial fine-tuning on AGNEWS.
CLEAN
AUAS(TF)
CLEANS
AUAS(TB)
AUAS(DWB)
APWS(
BERT RPOCH-D
94.2
19.7
94.2
31.7
94.2
37.5
94.2
43.1
BERT EPOCH-2
94.3
28.4
94.2
$2.0
94.4
BERT RPOCH-3
94.2
33.0
94.3
$2.8
94.4
42.4
94.1
57.3
BERT EPOCH-4
94.6
34,9
94.6
$6.1
94.4
41.3
93.8
62.6
PRO BERT-MCC EPOCH-U
93.2
37.3
93.2
43.8
93.2
51.3
92.2
35.0
PRO-BER BERT-MCP SPOCE-1
93.9
40.0
94.1
48.6
93.8
53.4
93.4
57.5
PRO BERT-MC MCP
EPOCH-2
93.7
42.9
93.8
48.4
93.7
58.2
93.6
59.9
BERT-EETT-
PRO-BERT-ME BERT-MCP EPOCH-4
93.2
50.8
93.9
56.5
93.5
61.0
93.6
65.1
PRO BERT-MERTE EPOCH-S
93.9
$3.0
94.5
60.7
93.0
60.1
93.6
68.8
Textooler
TextBugger
90
(Clean)
(Clem)
BERT (Atocied)
30
DERT (Amackeed)
MCP (Clean)
MCP (Cleas)
Accuncy
(Altakol)
70
40
30
Epoch
Epoeli
(a) TextFooler
(b) TextBugger
DordWodBBog
90
(Clean)
BERT (Clem))
BERT (Atuaked))
80
BERT (Attacked)
20
MCP (Ataccee))
70
MCP (Artucked)
60
50
50
Fgoch
Epoch
(c) DeepWordBug
(d) PWWS
Figure 10: Adversarial fine-tuning on AGNEWS.
29

---

G.4 Ablation Study on Attack constraints
We present the ablation study on the maximum perturbation percentage, minimum cosines similarity and sentence similarity threshold in Figure [T] Table [O] Table IT and Table [2] respectively. The experiments are performed on AGNEWS under TextFooler with the ALBERT the backbone. The default values are as follows: sentence similarity threshold is 0.840845057, maximum perturbation percentage 1.0, synonym cosine similarity 0.5. The results show the consistent improvement of our method over the backbone models.
ALBERT
80
A1TA (%)
60
20
0.50
0.85
0.90
0.95
0.0
0.2
0.4
0.6
0.8
0.4
0.6
0.8
1.0
Sentenoe Similarity Threshold
Max Peturbation Percentage
Symonyno Cosine Similarity
Figure 11: Ablation studies on attack constraints.
Table 10: Ablation study perturbation percentage.
PERCENTAGE (p)
MODEL
CLEANS
Al14
ASR%
Al1A%
ASR%
AllA%
ASRS
ADAW
ASR%
ALBERT
93.0
64.7
30.4
49.4
46.9
34.5
62.9
25.7
72.4
PRO-ALBERT-MCP
93.8
67.6
27.2
54.2
41.6
49.7
46.4
48.9
47.3
MAX-PERCENTAGE
0.6
0.8
1.0
MODEL
AIASS
ASR
AtAS
ASR%
ACA
ASRS
ALBERT
20.6
77.9
20.6
77.9
20.6
77.9
PRO-ALBERT-MCP
48.9
47.3
48.9
47.3
48.9
47.3
Table 1I: Ablation study on minimum synonym cosine similarity.
MIN-Cos-Sun SIM
0.3
0.5
0.7
0.9
MODEL
CLEANS
AuA%
ASR%
A
ASR%
AuA%
ASR%
AUA%
ASR%
ALBERT
93.0
12.7
86.3
20.6
77.9
40.1
56.9
79.0
13.1
PRO-ALBERT-MCE
93.8
41.1
55.7
48.9
47.3
61.6
33.6
82.7
10.9
Table 12: Ablation study on universal sentence similarity threshold.
SENTENCE-SE SIM- THRESHOLD
0.2
0.6
0.8
0.85
MODEL
CLEAN%
AEA%
ASRS
AU3%
ASR%
AUA%
ASKS
A1IA%
A3R%
A0A%
ASR
93.0
1N.0
80.7
18.0
80.5
21.9
PRU-ALBERT-MCE
93.8
47.4
48.9
47.4
48.9
47.4
48.9
47.6
48.7
49.0
47.2
SHNTENCE-SIM- SANTENCT-SI--TRRSSSOO
0.875
0.9
0.925
0.95
MODEL
AEA%
ASRS
AWAS
ASR%
A0A%
ASR%
ASRS
ALBERT
27.6
70.3
46.6
49.9
71.2
23.4
90.5
2.7
PR0- CO--ALBERT-CET
55.9
39.8
61.8
30.2
79.4
14.4
90.4
2.6
30

---

G.5 Ablation Study on Backbone Models
Our proposed ProAttention is universal framework which can be applied to various attention- based models. To verify the universality of our methods, we integrate our robust attention module into various backbones and present the results in Figure T2] and Table [3] As seen the results, our ProAttention can consistently enhance the robustness over any backhone under various attacks.
TextFooler
TextBugger
50
MCP
MCP
30
AUA(5))
ATA
20
RoBERT
BERT ALBERT RoBERTa
(a)
(b)
WondWBngB
PWWS
60
A1TA (%)
A0A (96)
DERT
ALDERT
DistiDert RoDERTa
BERT ALBERI
Disiller RoBERIa
(c)
(d)
Figure 12: Accuracy under attack of different backbones.
Table 13: The results of different backbones on AGNEWS
TEXTIOODUE
TENTBUUGGE ASRS
DEIDWORDBUG ASR'S
PWWS
MonEL
CLEAN'S
AuAS
ASRS
4QUERY
AuAS
4QUBRY
ADA
#QUERY
AtxS
ASR
#QUERY
BERT
94.2
19:7
73.9
335.3
31.2
66.5
176.5
37.5
59..8
103.8
43.1
53.8
353.5
PRO-BERT-MCP
93.2
39.2
$7.7
377.4
48.3
48.5
207.7
51.8
43.8
107.9
56.2
39.2
363 2
ROBERTA
934
13.0
36.1
301.6
22.3
61.3
150.3
41.2
55.3
105.4
34.0
63.6
143.9
PRo- ROBERTA-MCP
93.7
24.4
73.7
312.6
34.3
62.8
195.2
45.5
$1.5
118.3
39.4
57.5
396.
DISTILBERT
93.
13.
35.9
317.4
31.5
63.4
159.1
30.0
67.9
98.0
36.3
61.0
352.4
PRO-DiSTH BRo--DiST.BERT-CC. BERT-MCP
93.9
29.3
63.5
363.3
48.7
47.9
184.2
34.3
63.1
98.6
50.5
45.6
864
PRO ALBERT MCP
93.8
49.3
417.8
35.3
208.2
35.9
32.0
375.2
31

---

G.6 Ablation Study on Hyperparameters of Penalties
G.6.1 Ablation Study on Huber
For the Huber-based model, we present the ablation study on the 8 and layers K of Huber loss under
TextFooler in Table and Figure 13 The default setting are as follows: = 0.9 and Y = 3, and we vary the two parameters separately capture the trend. We can find that the performance is insensitive to 8 or layers within an appropriate range.
Table 14: Ablation study Huber AGNEWS.
MODEL
CLEAN
ATTACKED
PRO-BERTI B-ET-BERTERRRRR 0
0.6
94.2
23.9
PRO-BERT HUBER
BERT-HUBER
0.7
94.2
23.2
BERT-HUBER 0.8
94.2
23.7
PRO-BERT-HH HUBER 5 0.9
94.2
24.2
PRO-BERT-HUBER
94.3
23.0
PRO-BERT-HURE HUBER A= 2
94.1
22.5
PRO-BERT-HUSE HUBER 3
94.2
21.7
PRO-BERT- HUBE--HBBRR 6 4
94.2
20.9
PRO-BERT-HUBEE 55 94.1
20.0
PRO- BE---ERT--HUEE HUBER K 3
94.2
24.2
PRO-BEN CE--BERT-HHBEE K 4
94.2
24.0
PRO-B PRO-BERT-IA SERT-HUBER K 5
94.2
23.7
PRO-BER SRO-BERT-HHIE HUBER K 6
94.0
24.5
PRO-BER BO-BBERTHBBRR K
7
93.9
24.2
PRO-BERT- CE--BERT-HUBBE K
94.0
25.8
Effect Delta in Huber
Effect of Layers Huber
Clean
60
Clean Attaeled
Atracled
Delta
(a) Effect of 8.
(b) Effect of layers.
Figure 13: Ablation study on Huber
32

---

G.6.2 Ablation Study on MCP
We present the ablation study of the and layers K in MCP in Table 15 & Figure 14 and Table [6 Figure 15 From the figures, it can be concluded that appropriate 7 is needed get the best robustness, Besides, more layers can get a more precise solution for the robust objective, but we need to consider a good balance between precision and efficiency.
Table 15: Ablation on MCP on AGNEWS.
MODEL
CLEAN
TEXTFOOLER
PRO- MC--BERT-MMP 2,
93.3
32.8
PRO-BERT-MCP K = -2,7= -4
93.7
33.9
PRO-BERT-MCP K =2,7= =3
93.7
31.7
PRO-BERT-MCP K = =2,
93.1
30.4
PRO-BERT-MCP K 2,7
92.9
28.7
PRO-BERT-MCP K 5,
93.6
39.2
PRO-BERT-MCP K 4,7 =
93.7
37.2
PRO-BERT-MCP K 3,7
93.2
37.8
PRO-BERT-MCP K 2,7
93.7
33.9
PRO-BERT-MCP K 1,7 93.9
26.8
Effect of Gamma MCP
Effect Layers in MCP
80
Clean
Clean
60
80
Gannma
(a) Effect of y in MCP
(b) Effect of layers in MCP
Figure 14: Ablation study on MCP on AGNEWS.
33

---

Table 16: Ablation in MCP on IMDB.
MODEL
CLEAN
TEXTFOOLER
TEXTBUGGER
DEEFWORDBUG
PWWS
PRO-BERT- NR0-BERT-MCP1- 20, A =3
93.9
15.9
19.9
43.7
40.3
PRO-BERT-MCH =30, =3
93.7
15.1
24.0
53.8
51.1
PRO BERT-MCP =40, =3
26.6
55.5
PRO-BERT BERT-MCP =30, K= =3
93.9
16.3
29.8
53.9
46.6
PRO-BERT-MOO = 6.0, 3
93.3
13.5
23.0
48.1
41.5
PRO-BERT MCP =1
93.6
12.4
13.8
40.8
40.2
PRO-BERT-MCP K
93.8
12.5
15.7
49.0
47.9
MCP K
29.8
53.9
PRO-BERT BERT-MCP K
93.5
20.4
39.4
60.2
56.3
PRO-BERT-MCP K
93.5
22.1
44.6
63.3
56.1
Effect Gammna MCP
Edfect of MCP
2.0
3.0
10
3.0
3.0
4.0
Lagors
(a) Effect of y in MCP
(b) Effect of layers in MCP
Figure 15: Ablation study MCP on IMDB.
34

---

H Additional Experiments on LLMs
H.1 Experiments T5
H.1.1 main result
We compare the backbone model T5 with its robust version based on L, MCP and Huber loss in Figure [6 and Table 17 The experiments are conducted on SST2 under TextFooler. As shown in the figure, Pro- T5 (Huber or &1) can slightly improve the robustness of backbone T5. Moreover, Pro-T5 (MCP) significantly outperform other baselines, especially under the large attack budgets.
80
Accuracy (%)
60
8 (TS)
4
40
Huber
MCP
2
8
10
12
Num of Perturbed Words
Figure 16: Main results on T5.
Table 17: Accuary (%) under prompt attack on SST2 (TextFooler, T5)
PERTURBED WORDS
(CLEAN)
3
6
10
12
T5 PRO-TS &
$9.1
89.1
89.1
73.3
69.3
63.4
47.5
47.5
43.6
36.6
31.7
29.7
29.7
90.1
90.1
90.1
67.3
64.4
54.5
54.5
51.5
46.5
46.5
40.6
39.6
39.6
PRO-T5 MCP
87.1
87.1
71.3
69.3
64.4
64.4
63.3
PRO-T5 HUBER MCP 3,6 9.7
89.1
89.1
89.1
62.4
62.4
57.4
55.4
55.4
55.4
53.4
55.4
55.4
54.5
35

---

H.1.2 Ablation on Huber
We present the ablation study on 8 in Huber on SST2 in Figure 17 and Table 18 We fix the number of layers and vary the values of from 3.0 to 9.0. The bluber-based model with å 4.0 perform best among all the selected parameters.
Performeance Diffcrent in Huber
6.0.0
4.0
6=2.0
60
50
0
2
12
Nam of Pesturbed Words
Figure 17: Ablation study on Huber on T5
Table 18: Ablation study in Huber SST2.
PERTURSED WORDS
(CLEAN)
PRO-T5 HUBER K
3,6
95.0
95.0
95.0
64.4
52.5
52.5
52.5
43.6
33.7
31.7
31.7
PRO-T5 HUBER K = 3,6
95.0
95.0
95.0
65.3
65.3
65.3
64.4
63.4
58.4
58.4
50.5
34.7
31.7
PRO-T5 HUNEX
27.7
27.7
PRO T5 HUBER K 3.6
93.1
93.1
93.1
69.1
63.4
63.4
34.7
34.7
34.7
31.7
26.7
20.8
20.8
PRO-T5 HUBER K 3,5
93.1
72.3
72.3
23.8
23.8
23.8
PRO- T5 HURER K 3.8
93.1
93.1
93.1
75.2
41.6
41.6
33.7
33.7
30.7
30.7
27.7
27.7
25.7
PRO- T5 HUBER K
3,5
92.1
92.1
92.1
75.2
75.2
43.6
32.7
32.7
29.7
29.7
26.7
26.7
26.7
36

---

H.1.3 Ablation on MCP of T5
We present the ablation studies in MCP on SST2 in Figure 18 and Table The default settings are as follows: K 3 and ? 3.0. We can make the observations as follows: (1) For 7, the optimal value falls into the range of (3.5). (2) For number of layers, the best value can be chosen as 4 or 5.
Figure 18: Ablation study on MCP Tf
y
Performetnce Different Num of
MCP
-0- K
y-3.0
4-4.0
K
70
5-500
6=660
75
X
K-6
70
50
55
20
Nun of Pestarlod Words
Nun of Paslabbed Words
Effect of y in MCP
Effect of K in MCP
Table 19: Ablation study on MCP SST2
PERTURBED WORDS
(CLEAN)
PRO- 13 MICPA
3.7
80.2
80.2
50.2
64.4
49.5
47.3
43.6
43.6
32.7
24.8
21.8
21.8
PRO-T5 MCP K
3,3=330
85.1
85.1
85.1
67.3
65.
63.3
63.3
63.3
58.4
58.4
58.4
53.5
53.5
PRO-T5 MCP
3,y
PRO T5 MCP K
3.7 5.0
81.2
81.2
81.2
60.4
60.4
60.4
57.4
57.4
57.4
57.4
56.4
54.5
54.5
3.7=60
PRO-T5 MCP K =3,7=30
77.2
77.2
77.2
64.4
$8.4
58.4
55.4
$3.5
$3.5
53.5
$2.5
50.5
50.5
PRO-T5 MCP K 2,y=30
83,2
83.2
83.2
50.5
PRO- TS MCP K 3.5-3.0
85.1
85.1
85.1
67.3
65.3
63.3
63.3
63.3
58.4
58.4
58.4
53.5
53.5
PRO-TS T5 MCP K 4.7-30
87.1
87.1
87.1
71.3
69.3
64.4
64.4
64.4
64.4
63.3
63.3
61.4
PRO-T5 MCPA A 5.0
82.2
82.2
82.2
73.3
71.3
68.3
68.3
67.3
67.3
67.3
66.3
66.3
54.5
PRO T5 MCP K
6,7-
85.1
85.1
85.1
70.3
68.3
68.3
63.4
63.4
61.4
55.4
55.4
55.4
53.5
37

---

H.1.4 Ablation on Huber-MCP
We present the ablation study on Huber-MCP SST2 in Figure 19 and Table 20 The results show that Pro-T5 (Huber-MCP) is insensitive and get better robustness at or 15.
Performeance of Difterent in HDM
Performeance Different HIM
6-3, 1-14
6-s, y0
1-14
8-9, y-11
y-12
6-6 y-14
6-5, y-13
70
y-14
8-9, 1-14
3.Y
(c) Effect of d in Huber-MCP
(d) Effect of y in Huber-MCP
Figure 19: Ablation studies on Huber-MCP
Table 20: Ablation study Hlueer-MCL SST2
# PERTURREDD WORDS
(CL.AA))
3
6
10
12
HUBER-MCPK 3
50.5
PRO-T3
HUBER-MCPP CIEERMMPAR
9.1
11
84.2
84.2
59.4
54.4
47.5
47.5
46.5
45.5
43.6
43.6
PRO-T3
HURER-MCP 33.6-
84.2
84.2
84.2
55.4
44.5
44.5
44.5
44.5
44.5
43.6
43.6
43.6
41.5
PRo-T5
HURCR-MCP -3,6-.91
85.2
85.2
85.2
85.2
63.4
55.4
49.5
49.5
49.5
49.5
44.6
44.6
44.6
Pno-T5
HURER-MCP K -3.6-9 9.1.-991-114
82.2
78.2
68.3
68.3
60.4
60.4
58.4
49.5
46.5
PRG-T5 HUBER-MCP
82.2
82.2
82.2
64.4
57.4
36.4
45.3
47.3
46.5
46.3
46.3
46.5
46.3
HURER-MCPP
5.7
PRO-T5 HUBER-MCU MCP
6,
84.2
84.2
64.4
52.5
52.5
49.5
48.5
46.5
46.5
14.6
42.5
42.6
PRO-T3 HUBER-MUC MCP 3366
83.2
83.2
83.2
62.4
53.5
53.5
$1.5
49.5
49.5
46.5
46.5
46.5
44.5
PRO-T5 HURER-MCP
3,6 8,1
81.2
81.2
81.2
57.4
53.5
53.5
53.5
47.5
47.5
47.5
47.5
47.5
42.6
38

---

H.2 Experiments on LLaMA
For LLaMA, we can observe an intriguing phenomenon that differs from the T5 case: the & and MCP-based methods sacrifice too much accuracy while Huber method can keep decent performance under small budgets. This reason is that in the small range region, & and MCP utilize linear or concave functions while Huber can recover the &2 function. Inspired by the characteristics of these functions, we combine the properties
Huber and MCP, and construct a new function which we refer to Huber-MCH The detailed formulation and derived robust attention layers are available in Appendix B As indicated by the following curves, Huber-MCP and Luber-based models exhibits better robustness than other methods while preserving the good clean performance.
H.2.1 Textfooler
We present the results textual entailment on SST2 under TextFooler in Figure 20 can observe that i and MCP-based methods sacrifice the performance because of the estimation bias. Pro-LLaMA (Huber-MCP) shows slight improvement over other models.
90
90
6-3
-0- 5-15
5-20
Haher
8-5
-0- 0-
6-6
5-30
70
Habae-MPP
Accumey
70
6-7
-0- 6-35
6
5-40
6-9
5-45
60
60
Nom of Perteebod Words
Num of Perturbod Words
(a) Main results on SST2 (TextFooler)
(b) Ablation on ö in Huber (K 3)
90
90
y-15
y=
80
y-
y-
y-25
2-
7-30
y-
30
y-
y-
4-40
4-40
Num of Pertarbed Weeds
Num of Perturbed Woeds
(c) Ablation on y Huber-MCP (K
(d) Ablation on y Huber-MCP (K 3, = 9)
1, 8 = 9)
Figure 20: LLaMA (Textfooler)
"Empirical penalty selection strategy: For small medium-sized models such BERT (110M) and VIT (86M), MCP-based models exhibit superior robustness while nearly not sacrificing the clean performance. Moreover, MCP-based models are easy to tune with only one parameter ry. For large models like LLaMA (7B) and Vicuna (7B), it is necessary to choose Huber and Mluber-MPP to recover the original fa penalty within the low-value region in case of clean performance drop.
39

---

H.2.2 TextBugger
We present the results of textual entailment on SST2 under TextBugger in Figure 21 In this case, based model show a catastrophic performance drop while Pro-LLaMA (Huber) outperforms other baselines with sinificant margin.
- LLaMA
5-?
6-8
Haber-MPP
Accuacc
70
%0
Nom of Portnebed Words
Nom of Porturbod Words
(a) Main results SST2 (TextBugger)
Ablation study in Huber (K
y-
y-
30
y 20
y-
70
60
30
10
Nam Pertubed Words
(c) Ablation study on in Huber-MCP (K
3, 5 9)
Figure 21: LLaMA (TextBugger)
40

---

H.2.3 Wur
We present the results of textual entailment on SST2 under DeepWordBug in Figure 22 The experiment shows the similar phenomenon that & and MCP-based models sacrifice too much performance. Additionally, Pro-LLaMA (Huber-MCP) significantly outperforms other methods.
LLaMA
5-?
6-8
MCP
5-9
5-10
Accuacy
IUP
Accuacc
6
Nom of Portnebed Words
Nom of Portuebod Words
Main results SST2 (DeepWordBug)
Ablation study in Huber (K
y-
y-
30
y
y-
70
60
30
10
Nam Petubbed Words
(c) Ablation study on y in Huber-MCP (8
9, L 3)
Figure 22: LLaMA (DeepWordBug)
41

---

I Additional Experiments on Jailbreak
L.1 Transfer Jailbreak
We provide the results of transfer jailbreak in Table 21 Table 22 and Table 23 As SmoothLLM exhibits excellent performance defend the jaibreaking attack, we also combine the random smooth- ing with the backbone models and our methods to further validate the effectiveness of our method. As shown the results, when q (without random smoothing), by simply plugging our ProAt- tention into the Vicuna, the robustness can be improved by significant margin. Our plug-and-pll -layy method can even be comparable to the randomly smoothed models which require multiple operations including random perturbations, votings and aggregations.
Table 21: Vicuna: ASRs of JailBreak with Swap random smoothing Behaviours.
SMOOTH
3
5
VICUNA
91.8
71.8
20.9
0.9
PRO- VICUNA- HUBER
AICUNA-HUBER
0.1
0.9
0.9
0.9
PRO- -RO-VICUNA-BA HUBER
0.2
8.2
1.8
0.9
0.9
HUBER 6 0.3
21.8
2.7
0.9
0.9
5 0.4
30.9
21.8
0.9
0.9
TRO-VICUNA-HURER 0.5
36.4
23.6
0.9
0.9
PRO-VICUNA-IIE
40.0
0.9
PRO-VICUNA-IIUBER 1.0
70.0
53.6
11.8
0.9
SA--VICUAA-HUREE 0 1.5
74.5
60.0
16.4
1.8
PRO- PRO-VICUNA-HUBER 2.0
82.7
73.6
21.8
1.8
PRO-VICUNA-HUUER 5 3.0
90.0
74.5
31.8
7.3
Table 22: Vicuna: ASRs of JailBreak with Insert random smoothing Behaviours.
SMOOTU q VICUNA
10
15
91.8
44.5
10.9
4.5
1.8
PRO-VICUNA- HUBER =
1.8
0.9
0.9
0.9
0.9
0.9
Table 23: Vicuna: ASRs of JailBreak with Patch random smoothing on Behaviours.
SMOOTH g
0
5
VICUNA
91.8
71.8
37.3
39.1
21.3
14.3
PRO-VICUNA- HUBER 0.1
1.8
0.9
0.9
0.9
0.9
0.9
42

---

1.2 Adaptive Jailbreak
Although our Pro- Pro-Viunna has demonstrated significant effectiveness under transfer attack (black-box). it is still unclear whether our method can be resilient under white-bon box attacks which adaptively target the specific victim models. The comparison of our Pro- Pro-Vicuna and backbone Vicuna under adaptive jailbreak is presented in Table and Figure 23 Our Pro- Vro-Vicuna can improve Vicuna by an average of 10.4% across various numbers of attack queries. We don't include Smoothl.LM in adaptive attacks since it introduces non-differentiable operators that preclude the gradient-based GCG attack on it.
Table 24: Vicuna: ASRs of Adaptive JailBreak Behaviours
NUM OF ATTACK QUERIES
12
13
14
15
16
17
18
19
20
VICUNA
61.4
63.2
71.3
75.8
78.7
82.6
84.1
86.3
87.4
PRO-VICUNA (BEST)
50.7
55.9
60.8
64.3
67.4
70.5
74.0
77.7
78.6
PRO-VICUNA, HUBER 0
0.3
60.2
60.2
65.0
70.9
72.8
75.7
77.7
77.7
78.6
PRO-VICUNA VICUNA-HUBER 8
0.5
60.8
61.8
62.7
66.7
67.6
71.6
78.4
81.4
82.4
PRO-VICUNA
VICUNA-HUUER HUBER 8 0.7
50.7
55.9
60.8
64.3
67.4
70.5
74.0
79.7
82.4
85
80
75
ASR (%)
70
65
8 (Vicuna)
60
Huber 5 = 0.7
55
Huber = 0.5
Huber = 0.3
50
12
18
Max Num of Attempts
Figure 23: Adaptive JailBreak
43

---

Additional Experiments on VIT
J.1 Main Results
The main results of FGSM on ViT are presented in Table 25 (MCP) outperforms other methods across various budgets.
We can conclude that our Pro-ViT
Table 23: Adversarial robustness on CIFAR-10 10 (FGSM).
MODEL.
\ BUDGET
0 (CLEAN)
8/255
4/255
1/255
VIT
98.74
35.03
39.04
60.43
PRO- PRO-VIT-LI
98.46
42.41
46.33
61.99
PRO- PRO-VIT-HUDER
98.76
37.47
41.93
62.72
PRO- VIT-MCP (OURS)
98.40
54.10
60.38
75.85
Table 26: Adversarial Robsuiness on Imagenet- 1k (PGD).
MODEL
0 (ClEan)
8/255
4/255
2/255
1/255
VIT
90.7
0.0
0.1
4.1
30.2
PRO- CIT-VIT-L1
89.1
14.2
31.6
47.6
PRO VIT-HUBER
90.6
4.3
15.1
32.7
51.6
PRO-VIT-MCP (OURS)
88.6
13.0
23.1
49.1
65.6
J.2 Ablation study.
The ablation study of PGD and FGSM are provided in Table 27 Table and Figure 24 As shown in the results, the optimal y of MCP fall into the range of (3,4). The robust estimators with more layers show the better robustness while slightly sacrifice the clean performance.
Table 27: Ablation: CIFAR-IO (PGD)
MODEL \ BUDGET
(CLEAN)
8/255
4/255
1/255
PRO- PRO-VIT-HER VUT-HUBER K 3,5
98.43
0.09
0.82
28.38
PRO-VTT V-T-VVITEEEEEEEEE HUBER K = 3,5 =3
98.
0.07
1.36
31.04
PRO-VIT-HE HUBER K = =3,6=1
98.56
0.09
1.57
34.9
PRO-V PRO-VIT-HUBER K = =3,6=7
98.76
0.15
1.72
34.89
PRO-VIT PRO-VIT-HUBER K =3
98.75
0.18
1.86
34.98
PRO- PRO-VITMMCP K =1,,
98.07
1.03
2.43
26.16
PRO-VIT CO-VVT-MCP = =2,,==
2
96.92
2.22
3.85
39.04
PRO- PRO-MIT-MPP
-,
95.79
6.47
12.65
65.15
4,7
94.29
14.64
27.17
74.72
PRO- VIT-MPP 5. 4
93.43
23.34
37.56
76.75
PRO- VIT-MCP 6, 4
92.56
28.94
43.34
77.39
PRO- VIT-MCP 7, 4
91.89
31.57
47.01
76.86
PRO- VIT-MCP
8,7 4
91.36
33.4
47.22
76.16
PRO- VIT-MCP
9,7 4
90.76
33.17
48.11
75.56
PRO- VII-MCP MCP A
3,7
98.4
2.95
6.19
56.41
PRO- VIT-MCP
3,7
97.97
5.8
10.83
67.07
PRO- PRO-VIT-ME VIT-MCP = 3,7 =3,7=4
95.79
6.47
12.65
65.15
PRO- MCP 3,
92.77
3.53
7.54
45.70
PRO- VIT-MCP
3,7
94.0
3.54
7.22
37.22
44

---

Table 28: Ablation: CIFAR-IO (FGSM)
MODEL
BUDGET
(CLEAN)
8/255
4/255
1/255
PRO-VIT-HUBER K 3,0
98.43
36.23
40.57
58.84
=3,6
98.56
36.99
40.88
60.54
=3,6
98.65
37.47
41.93
62.72
PRO- HUO-VET-HUBER
=3
98.76
36.55
41.02
62.20
PRO- HO-VER-HUBER
=3
98.75
35.75
40.39
61.38
PRO- PRO-VTT-MCP K =1,7
98.07
35.22
38.12
52.82
PRO-VIT-MCP K =2,7
96.92
39.84
43.19
55.49
PRO-VIT-MCP K 3,7 4
95.79
47.38
53.6
67.03
PRO-VIT-MCP K 1,7 4
94.29
49.26
58.49
72.71
PRO PRO-VIT-ME VIT-MCP K 5,9 4
93.42
49.42
59.03
74.35
PRO- MCC K 6,7 4
92.56
48.23
59.21
76.01
PRO- VIT-MCP
3,7
98.4
47.98
52.39
70.59
PRO- VIT-MCP
3,7
97.97
51.64
57.21
73.16
PRO- PD--VIT-MGE
3,7 4
95.79
47.38
53.6
67.03
PRO- Sn--IT--MOT
3,7 5
92.77
35.37
41.76
52.10
PRO-VIT-MCP 3,7 6
94.0
37.08
41.42
48.56
PRO-VIT-MCP K 3,7 3
97.97
51.64
57.21
73.16
PRO-VPT-MCP K 4,7 3
97.76
54.1
59.66
75.30
PRO- PRO-VIT-MCP K
5,7
3
97.75
53.29
60.08
75.85
PRO- PRO-VVTTMCPP K
6,7 3
97.74
$2.37
60.38
75.70
Effect (PGD)
Effect Layers MCP (PGD)
100
100
E= 8/255
60
60
1/255
Clean
4/255
20
20
Layers
(a) Ablation study on 7 in MCP (PGD)
(b) Ablation study K in MCP (PGD)
Efffcct of Gomma in MCP (PGSM)
Effecc of Layers MCP (FGSM)
80
6/255
50)
8/255
4/255
AccouDcc
70
1/255
70
1/255
60
60
50
Layers
(c) Ablation study on y in MCP (FGSM)
(d) Ablation study on T. in MCP (FGSM)
Figure 24: Ablation study MCP
K Additional Experiments in GAT
The main results Citeseer and the ablation study Cora-ML are presented in Table 29 and Table From the results, we can make the following conclusions: (1) Our Pro-GAT outperform other methods significantly. Under the larger budgets, the outliers introduced by the adversarial attack
45

---

will enlarge the bias effect of the estimation. In this scenario, our MCP function can mitigate or even remove the effect of outlying values in the large-value region. (2) The parameter provide an implication for the robustness. For small budget, the models with large perform well since it more similar the original attention. While for large budget, the models with small ry offer better robustness cause it can mitigate the bias introduced by the outliers.
Table 29; Results on Citeseer.
MODEL BUDGET
0%
5%
10%
20%
30%
40%
GCN
74.8 1.2
66.1 1 1.0
60.9 1 0.8
53.0 1.0
47.0
41.2 1 1.1
GNNGUARD
72.4
65.6 0.9
618 1.4
1.4
51.0
47.3
RGCN
1.0
66.0
60.6 0.9
52.5
4.9
40.2
GRAND
74.8
0.76
0.7
5.1
1.2
4.9
PROGNN
74.2
65.6
6..3+1.1
5..7
+..2
40.8
JACCARD-GCM
+..8
66.3 1.2
6.2
5.9
46.5
11.1+1.0
SOFTMEDIAN
0.76
6..0
0.9
1.1
55.2
$1.92.11
GAT
1.2
1.
6..41144
22.6
47.
41.2 4.8
PRO-GAT (OURS)
1.1
68.9 1.4
66.0
63.0
2.6
57.7 2.0
Table 30: Ablation study on Cora-ML.
MODEL BUDGET
(CLEAN)
10%
20%
30%
40%
K ==
0..35
78.51
75.70
72.06
0.44
69.00
60.34
==1,==20
0.72
0.51 78.46
75.46 0.80
71.21
1.32
68.39
1.60
65.91
2.17
K-1,7- -.0
83.95
1..93
14.3511.660
69.46 1 1.16
66.31
1: 1.73
62.87
1.54
K ==1,-=40
0.61
77.41 0.64
7.797
68.97 + 0.98
65.70 1: 1.20
62.70 1.42
==1,7=50
1.11
77.57 0.93
73.88 1.29
68.98
1.27
65.17 1: 1.54
62.40
1: 1.75
K == 6.0
0.79
77.45
73.56
1.21 68.66
64.80
+ 1.41
61.94 + 2.08
K 1=,,7= 7.0
0..2
77.77+0.77
74.21 0.67
68.94 0.88
65.20
1.30
62.31 + 1.69
K=3,7= 1.0
8..7
77.59
75.04
1.25 1.25
71.47 1.06
68.70
1.20
60.53 + 1.24
K 3, =0
3
75.89
72.61 + 2.39
68.71 + 2.07
65.39
62.29 + 2.65
K K=3,-=00 =.0
8..04
77.09
69.27 1 1.45
1.62
2.07
K=3,7=44
8.57
76.37
73.45 2.04
68.63 1:
65.09 1: 2.56
62.42 2.33
K =0,
0.75
77.81 0.85
74.58 0.96
69.90 + 1.05
66.32 +: 1.26
63.33 + 1.66
K =
1.12
77.17 1.15
74.12 1.28
69.27 + 1.33
65.59
1.34
62.86 1.75
K 3
84.77 0.76
78.47 + 0.78
75.15 0.84
70.47 +: 0.96
66.91 + 1.33
63.94 + 1.33
46

---

L Complexity Analysis.
Here we will provide a complexity analysis of the vanilla attention, our robust attention, KDE-based attention and RKDE-basee attention. The related notations are Q K. V E RND and A E RNN The major difference in these methods is how derive the attention matrix, therefore we will only count the complexity of attention matrix derivation and context matrix computation.
Vanilla Attention, The vanilla attention matrix in can be formulated as A = softmax
vD
which costs about N x D. The context matrix computation requires N N D, so the total cost is 2 (N X N x D)
Our ProAttention. For our robust attention, we need to firstly compute the original matrix A (N X x D). Then we need to compute weight W based on the pairwise distance between the V and current estimator (R) (N N D). Finally we need update the estimator by Z (A W (N X N D). The context matrix is calculated through the iterations. Therefore, the total cost will be (1 + 2K) N x N x D. As stated in the ablation study in Section 4.2.2 our Newton-IRLS in ProAttention can converge efficiently within 3 steps (K 3). Therefore, our ProAttention is still effeicient.
Kernel Density Estimation (KDE) Attention. For KDE-based attention, the attention matrix is computed based on the pairwise ditance between the and Q. which cost N x N x D. The context matrix computation requires N N D, so the total cost is 2 (N N D).
Robust Kernel Density Estimation (RKDE) Attention. For the RKDE- attention, we need to perform the following operations. Firstly, we need to compute the basic matrix ad KDE attention matrix A (N x N x D). Then we need to compute pairwise distance ((k) for all the key pairs (N x x D) and update the weight (+(k) based on w and D (k) (N N N). Similarly, we need calculate the pairwise distance D KV and update the weight W KV for concatenated key and value, which costs x N x 2D and N x N N, repectivell. The context matrix computation requires N x N D. Therefore, the total cost will be (2 3K) N x N D 2K N x N N.
K
(A))
47

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Our abstract and introduction (Section] closely follow the contribution (at the end of Section D in the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contribu- tions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of the work in the Section 6
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results to vi- olations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specifficttin, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by review- ers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't t acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each thcoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
48

---

Justification:
We provide the theory in Section 3.3 and the detailed proof in the Appendix B
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by
formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experi- mental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Answer: [Yes]
Justification: We provide the detailed implementation in Section 3.4 and experimental setting in Section[4.1
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to hosted model (e.g., in the of large language model), releasing of model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access the model limited in some way (e.g.. to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, described in supplemental material?
Answer: [No]
Justification: We provide the detailed implementation in Appendix A and the dataset information in Section 4.1 We will organize the data and code access after the submission.
49

---

Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cc/pulll
guides CoodeSubmsssionooooo
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:/7 www.pioiissioooooooooooocc for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification:
We provide the detailed experimental setting in Section 4.1
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
TTee full details can be provided either with the code, in appendix, or supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report eπror bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Error bars are not reported because it would be too computationally expensive
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show tables or figures symmetric error bars that would yield results that out of range (c.g, negative error rates).
50

---

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
Experiments Compute Resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed reproduce the experi- ments?
Answer: [Yes]
Justification: We include the theoretical computation analysis in Appendix 3.4 and the running time in Section 4.2.2
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experi- mental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
Code Of Ethies
Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https:://nurrrii http:///wwuuu......ooooooo.llllt hwtpp//wll.////hcccciceeeeeeeee
Question:
Answer: [Yes]
Justification:
The paper closely follows NeurIPS Code of Ethics
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consideration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is societal impact of the work performed.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has societal impact or why the paper does not address socictal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploy- ment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
51

---

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology heing used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions access the model implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do require this, but we encourage authors to take this into account and make a best faith effort.
12. Licenses for existing assets
Are the creators or original owners assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer:
[NA]
Justification: Our paper does not use existing assets.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datastts has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged reach out to the asset's creators.
13. New Assets
Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Question:
Answer: [NA]
Justification: Our paper does not release assets.
Guidelines:
The answer NA means that the paper does not release new assets.
52

---

Researchers should communicate the details of the dataset/ccdeemmodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, applicable, as well as details about compensation (if any)?
Question:
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Sub- jects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/revieww based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
53


Reviews:
[{"review_id": "d4Hh37doDm", "reviewer": "Reviewer_nL5G", "summary": "In this paper, the authors intend to robustify transformer architectures against adversarial attacks to enhance their resilience across various machine learning tasks. Specifically, they propose the ProAttention mechanism. They use a novel interpretation of the self-attention mechanism as a weighted least squares (WLS) estimator, along with robust WLS token estimators and an efficient Newton-IRLS algorithm with convergence guarantees, which enhances the robustness of transformers without additional training or fine-tuning. Further, the authors develop a plug-and-play layer, ProAttention, to be integrated into existing transformers. They demonstrate several experiments to show the performance of this architecture in improving robustness across language modeling, image classification, and graph representation learning.", "questions": "1. The paper claims that the Newton-IRLS algorithm converges efficiently within three steps. However, can the authors provide a more detailed analysis or proof of this convergence rate, especially for larger models or more complex datasets?\n\n2. The authors show the effectiveness of ProAttention in enhancing robustness across language, vision, and graph domains. However, each domain's specific implementation details and hyper-parameter tuning strategies seem different. Can the authors clarify how the ProAttention framework can be uniformly applied across these diverse domains without extensive domain-specific modifications?\n\n3. The ProAttention framework relies on the weighted least squares (WLS) estimator and its robust variants. Can authors provide a deeper theoretical justification for choosing WLS and its robustness under adversarial attacks? Specifically, how do the assumptions of WLS hold up under different types of adversarial perturbations, and are there scenarios where the WLS assumptions might fail, leading to reduced robustness?", "limitations": "Despite being designed for efficiency, the ProAttention framework introduces additional computational overhead during inference. The paper mentions that the ProAttention layer requires 1-2 times more inference time, which may limit its deployment in real-time applications or on devices with limited computational resources.\n\nThe ProAttention framework relies on WLS estimators and their robust variants. While the paper provides a theoretical justification for choosing WLS, its effectiveness under different adversarial perturbations needs further investigation. In some scenarios, the assumptions of WLS may fail and reduce robustness.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The idea of ProAttention is novel. Most existing work focuses on either architecture-agnostic defenses or robustness improvements through adversarial training, but these approaches often fail to generalize across different tasks or require substantial computational resources. In this paper, the authors propose a plug-and-play paradigm to enhance robustness against adversarial attacks without additional training or fine-tuning.\n\n2. The ProAttention framework is reasonable, and the adaptation of the Newton-IRLS algorithm makes it efficient and effective. The framework leverages the robust weighted least squares estimator to mitigate the effects of adversarial attacks.\n\n3. The experiments are extensive, and sufficient hyper-parameters are provided for reproduction. The authors detail their experimental setup and provide comprehensive ablation studies to demonstrate the robustness and efficiency of their approach under various attack scenarios.", "weaknesses": "1. The paper does not include error margins for experiments, especially for ablation studies in Section 4. The paper provides extensive experimental results; however, it does not include error bars or confidence intervals to indicate the variability or statistical significance of the results.\n\n2. This paper could benefit from better presentation and clearer structure. For example, the captions for the figures, especially Fig. 1 and 2, are too simple and lack the necessary explanations. This may lead to misunderstandings about the designs or results.\n\n3. I do not see a discussion of the method's limitations, despite the title of the final section mentioning “Limitation.”\n\n4. Although the paper proposes enhancing transformer robustness through the ProAttention mechanism, it needs a thorough explanation and analysis of why this method performs better under various attacks. I suggest that the authors further discuss this aspect in more detail", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your recognition of the efficiency and effectiveness of our method. We are glad to solve134\nyour concerns and answer your questions with the following illustrations.\n\n**W1**:  We want to clarify that the proposed ProAttention is plugged into fixed pre-trained models without finetuning, hence there is no randomness of optimization of training. Also, the attack strategies strictly follow the procedures without randomness. Besides, our evaluation method is common in robustness of language models, and consistent with the existing benchmarks [1] and text attacks works.\n\n[1] Searching for an effective defender: Benchmarking defense against adversarial word substitution, 2021.\n\n**W2**: Thanks for pointing it out. We have referred to the figures and included the explanations in the main text. We will include necessary captions for the figures in the revised version.\n\n- For Figure 1, we present three attack mechanisms to manipulate and harm the language models:\n  1. Classic text attacks modify the input content.\n  2. Prompt attacks perturb the prompt template.\n  3. Jailbreaks add adversarial non-semantic suffixes.\n\n- For Figure 2, we show the schematic overview of the ProTransformer: We design our ProTransformer by plugging ProAttention into transformers, replacing the vanilla attention mechanism without additional training. Furthermore, our ProTransformer is versatile and can be applied across various domains, including language, image, and graph processing.\n\n**W3**:  Please refer to the **global response for the concern of limitations of ProTransformer**.\n\n**W4**:  We have provided the detailed motivations and explanations for the robustness in Section 3. We are glad to summarize them again here:\n\n1. In Section 3.1, we highlight the vulnerability of vanilla attention due to the quadratic impact of L2 penalty in the WLS (L2) estimator. In particular, the attacker can easily manipulate the input tokens to dominate the output because of the quadratical penalty on the difference between input and output tokens.\n2. In Section 3.2 and 3.3, we demonstrate that robust penalties, such as L1 or MCP, can mitigate the quadratic impact to a linear or constant impact. Specifically, when some tokens are attacked and introduce large residues, our ProAttention can adaptively downweight their attention weights to mitigate their impact.\n\nTo simplify the understanding, it is helpful to think about a special case of it when all the attention weight are equal: median estimation (L1 case) is notably more robust than mean estimation (quadratic case) because median estimation can be formulated as the minimizer of the sum of absolute deviations (L1 norm) which reduces the impact of outliers.\n\n**Q1**: We would like to clarify that in general, there is no way to prove that an optimization algorithm converges in three steps, especially for the challenging non-convex and non-smooth problems proposed in this paper. In addition, there is no clear connection between the convergence rate of the neural network layers and the size of models or datasets. What we provide in our paper is to theoretically guarantee each iteration step will decrease the function values so running more steps will at least not hurt. We then provide empirical evaluations to show it indeed converges fast in a few steps, which makes the algorithm practical. The exact convergence rate is neither the focus nor the primary contribution of this work, but this can be an exciting future research.\n\n\n**Q2**: \nIt is common in adversarial defenses to control the balance between clean and robust performance through key hyperparameters. For instance, the famous paper TRADES [1] needs to carefully tune the combination between clean loss and robust loss to achieve best performance even for different attack budgets for the same model architectures and datasets, not to mention totally different domains evaluated in this paper. Since we plug-in our robust layers into the given models without training, it is expected some reasonable hyperparameter tuning is necessary to obtain optimal performance.\n\nWe will provide some general guidance in the revision: (1) Huber loss better maintains clean performance so it typically works better when the attack is weak; (2) MCP loss works better when the attack is strong; (3) L1 loss is a special case of Huber loss; (4) the proposed Huber-MCP penalty can flexibly reduce to different options such as Huber, MCP, and L1 based on the values of hyperparameters δ and γ as shown in Figure 3 in the paper.\n\nAs a future work, we can make those hyperparameters as learnable parameters learned through fine-tuning or adversarial training, which can avoid hyperparameter search but also need more training costs.\n\n[1] Zhang, Hongyang, et al. \"Theoretically principled trade-off between robustness and accuracy.\" (ICML, 2019)\n\n**Q3**: \nWe want to clarify that WLS is not an assumption; the vanilla attention is exactly equivalent to the WLS estimator (L2) under our new interpretation. Our robust ProAttention is an improved variant.\n\n**L1**: Please refer to the **global response for concern of efficiency of ProTransformer**\n\n**L2**: Please refer to Q3.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZnukA0ndWr", "reviewer": "Reviewer_JZTY", "summary": "This paper proposes an interpretable robust attention layer to robustify transformer architecture via a plug-and-play paradigm.", "questions": "1. Could the authors compare the proposed ProAttention with linear attention under the same experimental setting?", "limitations": "The comparison with different attention mechanisms is missing.", "rating": 4, "confidence": 1, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The proposed method is practical and can be plugged into the given transformer as a plug-and-play layer.\nThe experiments are robust and conclusive, signifying the efficacy and satisfactory performance of the proposed method.", "weaknesses": "It seems that the complexity of the proposed ProAttention is still greater than that of linear attention [1] [2] [3].\n\n[1] Han, Dongchen, et al. \"Flatten transformer: Vision transformer using focused linear attention.\" Proceedings of the IEEE/CVF international conference on computer vision. 2023.\n[2] Katharopoulos, Angelos, et al. \"Transformers are rnns: Fast autoregressive transformers with linear attention.\" International conference on machine learning. PMLR, 2020.\n[3] Zhu, Lianghui, et al. \"DiG: Scalable and Efficient Diffusion Models with Gated Linear Attention.\" arXiv preprint arXiv:2405.18428 (2024).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your recognition of the efficiency and effectiveness of our method. We are glad to solve134\nyour concerns and answer your questions with the following illustrations.\n\n**W1**: It seems that the complexity of the proposed ProAttention is still greater than that of linear attention [1] [2] [3].\n\n[1] Han, Dongchen, et al. \"Flatten transformer: Vision transformer using focused linear attention.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.\n\n[2] Katharopoulos, Angelos, et al. \"Transformers are rnns: Fast autoregressive transformers with linear attention.\" International conference on machine learning. PMLR, 2020.\n\n[3] Zhu, Liangxiu, et al. \"DiG: Scalable and Efficient Diffusion Models with Gated Linear Attention.\" arXiv preprint arXiv:2405.18428 (2024).\n\n**Answer**:\nThanks for your comments. We would like to emphasize that while we aim to develop algorithm with acceptable efficiency, efficiency is not the focus of this work. Therefore, linear attention is irrelevant to this context. Instead, our ProAttention targets on improve the robustness of transformers. Linear attention aims to improve the efficiency of transformers but is still vulnerable against adversarial attacks as verified in next answer. Their contributions are totally orthogonal.\n\n**Q1**: Could the authors compare the proposed ProAttention with linear attention under the same experimental setting?\n\n**L1**: The comparison with different attention mechanisms is missing.\n\n**Answer:**\nThanks for your comments. We compare different attention mechanisms (vanilla attention, Linear attention [1], Longformer attention [2], and our ProAttention) on IMDB across various attacks in the following table. Our method shows better robustness compared to other attention mechanisms including the linear one.\n\n[1] Katharopoulos, Angelos, et al. \"Transformers are rnns: Fast autoregressive transformers with linear attention.\" International conference on machine learning. PMLR, 2020.\n\n[2] Beltagy, Iz, Matthew E. Peters, and Arman Cohan. \"Longformer: The long-document transformer.\" arXiv preprint arXiv:2004.05150 (2020).\n\n\n### Table 1: The results of sentiment analysis on IMDB.\n\n| Model             | Clean | Textfooler | TextBugger | DeepWordBug | PWWS |\n|-------------------|-------|------------|------------|-------------|------|\n| Vanilla attention | 92.3  | 11.8       | 11.3       | 32.8        | 26.4 |\n| Linear attention  | 89.4  | 6.9        | 5.4        | 20.7        | 17.7 |\n| Longformer attention | 87.5  | 6.4        | 7.9        | 28.5        | 23.2 |\n| ProAttention (L1) (Ours) | 93.3  | 24.6       | 13.0       | 36.0        | 32.7 |\n| ProAttention (Huber) (Ours) | 93.0  | 24.8       | 13.4       | 36.9        | 31.5 |\n| ProAttention (MCP) (Ours) | 93.5  | 22.1       | 44.6       | 55.5        | 56.3 |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8HvBoL4zZT", "reviewer": "Reviewer_h5FP", "summary": "This paper proposed a robust transformer architecture named ProTransformer through a plug-and-play paradigm without further training or fine-tuning. The authors include robust token estimators in the self-attention blocks which are more resilient to the dominating impact of input tokens, and apply Newton-ISLR to approximate these estimators. The comprehensive experiments for various tasks, modalities, attacks, and backbones show that ProTransformer can significantly improve the robustness of transformers without compromising benign performance.", "questions": "1. When using white-box adaptive attacks like PGD and GCG, do you compute the gradient through your ProAttention blocks?\n2. For the jailbreak attack results in Figure 6, why do you use Huber instead of MCP attack constraints? As shown in Section 4.2.1, MCP shows the best performance.", "limitations": "The limitations of the paper are not adequately discussed. The cost of the evaluation with ProTransformer should be included in the limitations.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper is well-written with clear descriptions of their method with both motivation/intuition analysis and theoretical analysis.\n2. Comprehensive experiments are conducted to prove their claims of the enhanced robustness of Transformers. This includes robustness analysis for both the traditional adversarial attacks within the word or character level and the recent jailbreak attacks for large language models. Detailed ablation studies and transformers for alternative modalities are also considered.\n3. The ProTransformer is simple to use (Plug-and-Play) and has significantly improved the robustness. Moreover, ProTransformers can not only work on language modeling but also show its generalization for transformer-based architectures in vision and graph domains.", "weaknesses": "1. Lack of analysis of clean performance in jailbreak attacks. I want to know if ProTransformer would hurt the generation quality of LLMs.\n2. Less discussion about the semantic level attacks and robustness of corruption. For example, the AdvGlue Benchmark (https://adversarialglue.github.io/) for corruption robustness and PAIR[1] for semantic-level jailbreak attacks.\n3. Lack of comparison with baseline defense methods.\n\n[1] Chao, Patrick, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, and Eric Wong. Jailbreaking black box large language models in twenty queries. arXiv preprint arXiv:2310.08419 (2023).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your recognition of the novelty and effectiveness of our method. We are glad to solve33\nyour concerns and answer your questions with the following illustrations.\n\n**W1**: Lack of analysis of clean performance in jailbreak attacks. I want to know if ProTransformer would hurt the generation quality of LLMs.\n\n**Answer.** In ProTransformer, using Huber loss will almost not hurt the generation quality while using L1 or MCP will slightly sacrifice the generation performance. We construct 100 synthetic texts (including prompts and reference texts) evaluate the generation with the BLEU score and a series of ROUGE scores with the vicuna-7b as the backbone. The following results verify that Huber model keep similar score with L2 (vanilla), while L1 and MCP may sacrifice a little bit quality.\n\n### Table 2: BLEU and ROUGE score on generation quality.\n\n| Penalty in ProTransformer | L2 (Vanilla) | Huber | L1   | MCP   |\n|----------------------------|--------------|-------|------|-------|\n| BLEU                       | 0.151        | 0.154 | 0.145| 0.136 |\n| ROUGE-1                    | 0.385        | 0.390 | 0.368| 0.375 |\n| ROUGE-2                    | 0.379        | 0.353 | 0.344| 0.314 |\n| ROUGE-L                    | 0.396        | 0.390 | 0.385| 0.368 |\n\n**W2**: Less discussion about the semantic level attacks and robustness of corruption. For example, the AdvGlue Benchmark ([https://adversarialglue.github.io/](https://adversarialglue.github.io/)) for corruption robustness and PAIR[1] for semantic-level jailbreak attacks.\n\n[1] Chao, Patrick, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, and Eric Wong. \"Jailbreaking black box large language models in twenty queries.\" _arXiv preprint arXiv:2310.08419 (2023)_.\n\n**Answer.** For AdvGlue Benchmark, we already included the evaluation under typos-based Textbugger and embedding similarity-based Textfooler. We add the experiment of the context-aware BERT-Attack in the following table, the results show our Pro-BERT outperforms all of the other baselines.\n\n### Table 3: ASR (%) under BERT-Attack.\n\n| Method  | Aua% | ASR% |\n|---------|------|------|\n| BERT    | 5.8  | 93.7 |\n| FreeLB  | 21.7 | 76.6 |\n| PGD     | 21.0 | 77.6 |\n| MixADA  | 7.6  | 91.8 |\n| TA-VAT  | 19.2 | 79.4 |\n| SAFER   | 38.5 | 58.8 |\n| RanMASK | 36.0 | 58.4 |\n| Pro-BERT (Ours) | 42.7 | 54.4 |\n\nAdditionally, we also evaluate the advantage of our ProTransformer under semantic-level jailbreak attack PAIR in the following table.\n\n### Table 4: ASR (%) under PAIR Jailbreaks.\n\n| Model                    | ASR(%) |\n|--------------------------|--------|\n| Vicuna                   | 98.7   |\n| Vicuna+SmoothLLM         | 51.4   |\n| Pro-Vicuna               | 48.9   |\n| Pro-Vicuna+SmoothLLM     | 41.6   |\n\n**W3**: Lack of comparison with baseline defense methods.\n\n**Answer.** Actually we have included the comparison with baseline defense methods for each domains:\n- Language domain: backbone baselines ALBERT, DistilBERT, RoBERTa and BERT, and popular defenses including MixADA, PGD-Adv, FreeLB, TA-VAT and SmoothLLM.\n- Vision domain: We include Deit, Convit, BeiT, Swin, ViT.\n- Graph domain: GCN, GAT, GNNGuard, RGCN, GRAND, ProGNN, Jaccard-GCN and Soft-Median.\n\n**Q1**: When using white-box adaptive attacks like PGD and GCG, do you compute the gradient through your ProAttention blocks?\n**Answer.** Yes. We compute the gradient through the ProAttention blocks.\n\n**Q2**: For the jailbreak attack results in Figure 6, why do you use Huber instead of MCP attack constraints? As shown in Section 4.2.1, MCP shows the best performance.\n\n**Answer.** In Section 4.2.1, the task is classification, so both Huber and MCP losses would not sacrifice too much clean performance. But in jailbreak, we evaluate the model with the generated response. In this scenario, Huber loss has $L_2$ penalty in the low-value region, which enables the model to keep better generation quality. But MCP will hurt the generation quality, result in some non-semantic answer.\n\n**L1**: The limitations of the paper are not adequately discussed. The cost of the evaluation with ProTransformer should be included in the limitations.\n\n**Answer.** Please refer to the **global response for the concern of limitations of ProTransformer.**", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jKLquingTD", "reviewer": "Reviewer_Yhky", "summary": "This paper proposes a robust attention mechanism to improve the resilience of transformer-based architectures. The method does not need additional training or finetuning with only four lines of code, which is simple yet effective. To show the effectiveness of the proposed mechanism, the paper conducts experiments across different tasks including topic classification, sentiment analysis, textual entailment, jailbreak attack, on different backbone architectures such as large language models. The results show promising resilience.", "questions": "1. For the main results in Tab.1, how many layers (K) in the transformer models are equipped with ProAttention?\n2. For the results in Tab.2, what is the robutness performance by employing different numbers of ProAttention layers?", "limitations": "Please refer to weakness and questions.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The method is simple yet effective. With only four lines of codes, it's able to provide good resilience performance across diverse tasks and model architectures. \n2. The experiments are quite comprehensive. The study of robust transfomers include language modeling, image classification, and graph representative learning. For language modeling, the evaluations consider classic text attacks, prompt attacks, and jailbreak attacks.\n3. The results demonstrate promising performance.  \n4. The plug-and-play nature of ProTransformer makes it practical for real-world applications, as it can be easily integrated into existing model architectures without significant overhead.", "weaknesses": "1. As the paper points about one of the advantages of the method is the efficiency (efficient Newton-ISLR algorithm), there should be more discussions about the runtime of the proposed method across different architectures. The current version only includes the study in Tab.2 on a single model type. What is the influence on LLMs as they are more computation intensive? And how is the runtime compared with other robust designs?\n\n2. For the results on image classification tasks, there is no comparision. How is the performance of the proposed method compared with other robust designs such as [*]? \n\n[*] Robustifying Token Attention for Vision Transformers", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your recognition of the efficiency and effectiveness of our method. We are glad to solve134\nyour concerns and answer your questions with the following illustrations.\n\n**W1**: As the paper points about one of the advantages of the method is the efficiency (efficient Newton-ISLR algorithm), there should be more discussions about the runtime of the proposed method across different architectures. The current version only includes the study in Table 2 on a single model type. What is the influence on LLMs as they are more computation intensive? And how is the runtime compared with other robust designs?\n\n**Answer.** I include the average running time across different architectures. The results show the consistent conclusion with the BERT backbone: Our ProTransformer will only include 1-2 times of inference time while achieving great robustness.\nWe want to point out that robustifying models is usually very time-consuming and labor-intensive. For example, adversarial training methods usually require substantial training time (e.g., 7-100 times of the normal training time [1]), while our ProTransformer can directly plug in the robust attention into a pre-trained backbone model without any training. Moreover, randomized smoothing methods usually require a lot of inference time (e.g., 1,000 times for the language domain [2] and 100,000 times for the vision domain [3]). Additionally, most robust architectures are quite complicated, require training the models from scratch, and exhibit limited robustness. Therefore, our ProTransformer, which only requires 1-2 times more inference time, is quite efficient and economical.\n\n[1] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. \"Towards deep learning models resistant to adversarial attacks.\" In International Conference on Learning Representations, 2018\n\n[2] Lou, Qian, et al. \"CR-UTP: Certified Robustness against Universal Text Perturbations.\" arXiv preprint arXiv:2406.01873 (2024).\n\n[3] Cohen, Jeremy, Elan Rosenfeld, and Zico Kolter. \"Certified adversarial robustness via randomized smoothing.\" International conference on machine learning. PMLR, 2019.\n\n[4] Han, Xing, et al. \"Designing robust transformers using robust kernel density estimation.\" Advances in Neural Information Processing Systems 36 (2024).\n\n### Table 5: Average running time (ms) on AG-NEWS.\n\n| # Layers (K) | 0 (vanilla) | 1     | 2     | 3     | 4     | 5     | 6     |\n|--------------|-------------|-------|-------|-------|-------|-------|-------|\n| ProBERT      | 6.14        | 9.04  | 11.67 | 14.34 | 17.33 | 19.89 | 21.87 |\n| ProALBERT    | 5.63        | 8.70  | 13.03 | 16.10 | 19.37 | 22.77 | 23.27 |\n| ProDistilBERT| 2.17        | 4.16  | 6.88  | 8.38  | 9.17  | 11.95 | 13.19 |\n| ProRoberta   | 4.95        | 3.10  | 11.86 | 14.93 | 18.40 | 20.89 | 23.04 |\n\n**W2**: For the results on image classification tasks, there is no comparison. How is the performance of the proposed method compared with other robust designs such as [*]?\n\n[*] Robustifying Token Attention for Vision Transformers\n\n**Answer.** We validate the advantage of our ProTransformer over RVT [1] and TAP&ADL [2] in the following table.\n\n### Table 6: Adversarial robustness under PGD.\n\n| Model               | Clean  | PGD    |\n|---------------------|--------|--------|\n| ViT                 | 98.74  | 0.26   |\n| RVT [1]             | 97.65  | 18.71  |\n| FAN-B-Hybrid +TAP&ADL [2] | 98.45  | 22.23  |\n| Pro-ViT (Ours)      | 98.40  | 33.40  |\n\n[1] Towards Robust Vision Transformer\n\n[2] Robustifying Token Attention for Vision Transformers\n\n**Q1**: For the main results in Tab.1, how many layers (K) in the transformer models are equipped with ProAttention?\n\n**Answer.** We equipped ProAttention with 3 layers (K).\n\n**Q2**: For the results in Tab.2, what is the robustness performance by employing different numbers of ProAttention layers?\n\n**Answer**: We already include the ablation study on the number of layers in the Appendix G.6.2. The results validate that our algorithm can converge well within 3 steps.\n\n| Model               | Clean | Textfooler |\n|---------------------|-------|------------|\n| Pro-BERT-MCP K = 5  | 93.6  | 39.2       |\n| Pro-BERT-MCP K = 4  | 93.7  | 37.2       |\n| Pro-BERT-MCP K = 3  | 93.2  | 37.8       |\n| Pro-BERT-MCP K = 2  | 93.7  | 33.9       |\n| Pro-BERT-MCP K = 1  | 93.9  | 26.8       |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Thanks to all the reviewers for the recognition of the novelty and effectiveness of our method. Since several reviewers have the concern about the efficiency of ProTransformer, we will elaborate the response as follows.

## **Response for the concern of efficiency of ProTransformer**

We want to point out that robustifying models is usually very time-consuming and labor-intensive. For example, adversarial training methods usually require substantial training time (e.g., 7-100 times of the normal training time [1]), while our ProTransformer can directly plug in the robust attention into a pre-trained backbone model without any training. Moreover, randomized smoothing methods usually require a lot of inference time (e.g., 1,000 times for the language domain [2] and 100,000 times for the vision domain [3]). Additionally, most robust architectures [4] are quite complicated, require training the models from scratch, and exhibit limited robustness. Therefore, our ProTransformer, which only requires 1-2 times more inference time, is quite efficient and economical. Certainly, there is still some space to further improve the efficiency of ProAttention, which can be the focus of future work.

[1] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. "Towards deep learning models resistant to adversarial attacks." In International Conference on Learning Representations, 2018.

[2] Lou, Qian, et al. "CR-UTP: Certified Robustness against Universal Text Perturbations." arXiv preprint arXiv:2406.01873 (2024).

[3] Cohen, Jeremy, Elan Rosenfeld, and Zico Kolter. "Certified adversarial robustness via randomized smoothing." International Conference on Machine Learning. PMLR, 2019.

[4] Han, Xing, et al. "Designing robust transformers using robust kernel density estimation." Advances in Neural Information Processing Systems 36 (2024).

## **Response for the concern of limitations of ProTransformer**

Our limitations are listed as follows:

- Despite acceptable complexity of our ProTransformer, there still exists a potential to improve the efficiency of our models.
- We majorly claims and validate the effectiveness of our model under plug-in and play paradigm. We are excited about the future of the proposed ProTransformer architecture and hope to see its full potential with training or fine-tuning on large models.

Meta-review: This paper introduces a robust attention mechanism intended to enhance the robustness of transformer-based architectures against adversarial attacks. A key advantage is that it can serve as a plug-and-play layer, therefore can be easily integrated into existing transformers for enhancing robustness without additional training or fine-tuning. 

Overall, all reviewers appreciate the simplicity and practicability of this method, and acknowledge that the provided experiments are comprehensive and the corresponding results are solid. Meanwhile, some concerns are also raised, mainly including: 1) some critical discussions are missing (e.g., efficiency analysis, limitation); 2) the comparisons against linear attention are missing; and 3) the presentation could be further improved.

The rebuttal is considered, which addresses most of these concerns. Consequently, three reviewers stand on the positive side of accepting this paper. The sole reviewer who raised objections, primarily concerning the comparison with linear attention, did not respond to the details provided in the rebuttal; additionally, by reading the original review and the provided rebuttal, the AC feels the rebuttal reasonably alleviates this concern. Therefore, the AC decides to ignore this negative review and agree with the other three reviewers on accepting this submission.
Decision: Accept (poster)