When Adversarial Training Meets Vision Transformers: Recipes from Training to Architecture
Abstract
Vision Transformers (ViTs) have recently achieved competitive performance in broad vision tasks.
Unfortunately, on popular threat models, naturally trained
ViTs are shown to provide no more adversarial robustness than convolutional neural networks (CNNs).
Adversarial training is still required for ViTs to defend against such adversarial attacks.
In this paper, we provide the first and comprehensive study on the adversarial training recipe of ViTs via extensive evaluation of various training techniques across benchmark datasets.
We find that pre-training and SGD optimizer are necessary for ViTs’ adversarial training.
Further considering ViT as a new type of model architecture, we investigate its adversarial robustness from the perspective of its unique architectural components .
We find, when randomly masking gradients from some attention blocks or masking perturbations on some patches during adversarial training, the adversarial robustness of ViTs can be remarkably improved, which may potentially open up a line of work to explore the architectural information inside the newly designed models like ViTs.
Our code is available at https://github.com/mo666666/
When-Adversarial-Training-Meets-Vision-Transformers .
Introduction
Recent years have witnessed the overwhelming advances of Vision Transformers (ViTs) [1, 2].
Different from widely deployed Convolutional Neural Networks (CNNs) [3, 4] that adopt a series of local convolutional operations on input images, ViTs adopt self-attention mechanisms [5] on a sequence of image patches.
Based on large-scale pre-training, ViTs have achieved competitive and even better performance compared to CNNs in several fields such as semantic segmentation [6], object detection [7] and image generation [8].
Despite their great success on a growing number of vision tasks, ViTs fail in providing more adversarial robustness than CNNs on popular threat models [9, 10].
To defend against such adversarial examples crafted by adding human-imperceptible perturbations to images [11, 12], ViTs still demand adversarial training (adversarial training) [13, 14, 15, 16], an approach that incorporates adversarial examples into training and obtains notable empirical robustness.
Notably, the implementation details of adversarial training make a big difference on CNNs backbones [17], which motivates us to comprehensively evaluate various training techniques for adversarial training of ViTs and thus provide the first implementation benchmark for the training recipes of
ViTs under adversarial training .
Since adversarial training is quite different from natural training, different insights are discovered, for example, pre-training is useful for adversarial robustness and
SGD behaves better than AdamW for ViTs’ adversarial training.
Based on our empirical findings, we provide a practical recipe for adversarial training of ViTs to help researchers in their future work.
Although ViTs can achieve reasonable adversarial robustness via the above training recipes of adversarial training, it is unexplored whether or not we are able to further improve its robustness through the specific architectural information from ViTs, after all ViTs are a kind of model totally different from CNNs.
Interestingly, when we randomly mask gradients from some attention blocks or mask perturbations on some patches during adversarial training, the robustness of ViTs can be further improved.
We term these two simple methods as Attention Random Dropping (ARD) and
Perturbation Random Masking (PRM) respectively, and conduct extensive experiments to verify their effectiveness.
Our main contributions are summarized as follows:
• We comprehensively evaluate training techniques for ViTs under the setting of adversarial training on popular threat models, and discover many interesting insights that are different from the natural training scenario.
• Based on the unique architectural information from ViTs, we propose Attention Random Dropping (ARD) and Perturbation Random Masking (PRM) as a warming-up strategy to improve the adversarial robustness of ViTs.
In particular, ARD randomly masks gradients from some attention blocks while PRM randomly mask perturbations on some patches.
• Our work not only provides the first implementation benchmark (bags of tricks) for adversarially trained ViTs, but also reminds researchers of the potential of architectural information inside the new brand of models like ViTs.
Related Work 2.1 Training Strategies for ViTs
Due to the lack of inductive bias [1], ViTs require more training techniques to reach their performance potential.
Dosovitskiy et al [1] first pretrained ViTs on the large-scale datasets (ImageNet-21k or
JFT-300M [18]) and achieved competitive performance on downstream tasks.
Touvron et al [19] applied strong data augmentations such as Randaugment [20] and Mixup [21] to train ViTs from scratch.
They also find AdamW [22] performs better than SGD [23] for training ViTs.
Other studies [1, 24, 25] adopted gradient clipping to stabilize and accelerate the convergence of ViTs.
Steiner et al [26] conducted a comprehensive study on data augmentation and model regularization of ViTs to improve its natural accuracy.
For the robustness of ViTs, the training strategies, especially under adversarial training, are not fully investigated yet.
That is exactly what we are doing in this paper.
2.2 Adversarial Robustness of ViTs
Given a clean example x with its class label y and a model f θ ( · ) with its parameters θ , the goal of an adversary is to find an adversarial perturbation δ that fools the network into making an incorrect prediction for the perturbed image ( i.e. , f θ ( x + δ ) ̸ = y ), while the perturbation norm does not exceed ϵ ( i.e. , ∥ δ ∥ ∞ ≤ ϵ ),
[Equation]
1) where L ( · , · ) is usually the cross entropy loss.
Previous studies [12, 27, 13] proposed several methods to generate adversarial examples, and Project Gradient Descent (PGD) [13] becomes the most popular one:
[Equation]
To defend against adversarial attacks, adversarial training (adversarial training) and its variants [13, 28, 14, 15, 16, 29] become the most promising and effective method [30, 31, 32], which directly incorporates adversarial examples into training:
[Equation]
Despite extensive understandings of adversarial robustness on CNNs, only a few studies focus on the robustness of ViTs, which can be classified into the following categories: 1) naturally trained (non-adversarially trained)
ViTs: In [33, 9, 34, 10], they revealed that naturally trained ViTs are more robust than CNNs against extremely small perturbations ( e.g. , under threat model of ϵ = 0 .on
ImageNet, which is much smaller than the popular threat model of ϵ = 4 / 255 ≈ 0 . 0157 ); 2) adversarially trained ViTs: In [35, 9], they demonstrated that adversarially trained ViTs fail in providing more adversarial robustness on popular threat models compared to CNNs.
However, these previous works all lack a comprehensive study on the adversarial training process of
ViTs, which may result in an incomplete and biased impression on the robustness of adversarially trained ViTs.
Therefore, under the setting of adversarial training of ViTs, we for the first time comprehensively evaluate bags of training tricks across benchmark datasets, and provide a whole adversarial training recipe for ViTs to obtain remarkable robustness.
Strategies for Adversarial Training of ViTs
At the first glance, we may think there are already many works studying the training strategies of
ViTs.
The BIG difference from them lies on the training paradigm: previous research focuses more on the natural training of ViTs while here we focus on ViTs’ adversarial training.
Since adversarial training is quite different from natural training, different insights might be discovered.
In this section, we comprehensively evaluate several training techniques in terms of data (pre-training and data augmentation) and training (optimizer, learning rate schedule, and gradient clipping) across various architectures (vanilla ViT and Swin) and datasets (CIFAR-10 and Imagenette), so as to provide the first training recipe for adversarial training of ViTs.
Datasets.
Since adversarial training is time-consuming, small datasets remain popular for adversarial robustness [32].
Here, we use datasets of CIFAR-10 [36] and Imagenette [37] (a subset ofclasses from ImageNet-1K).
Note that the latest version of Imagenette (imagenette-v2 2 ) reshuffles the sampled subset of ImageNet-1K and then splits the training and validation set.
This leads to a overlapping issue between the pre-training set (ImageNet-1K) and the reshuffled validation set.
To avoid the leakage of validation data during pre-training, we select the previous version of Imagenette (imagenette-v1 3 ).
Architectures.
We use the vanilla ViT (ViT-B) [1] and Swin-B [2] as baseline models.
ViT first successfully introduced Transformers from natural language processing to computer vision and achieved competitive recognition performance compared to CNNs.
With the help of multi-stage hierarchical architecture, Swin became a general-purpose backbone and achieved SOTA performance on several downstream tasks.
By default, all models are pretrained on ImageNet-1K 4 .
To accommodate the smaller image size of CIFAR-10, we downsample the patch embedding kernel from 16 × 16 to 4 × 4 .
Threat Models.
We apply the commonly-used ℓ ∞ attack ( ∥ δ ∥ ∞ ≤ ϵ ) and set ϵ = 8 / 255 for CIFAR10 and Imagenette.
When evaluating the adversarial robustness, we apply 20-step PGD (PGD-20) [13] with the step size 2 / 255 and Auto-Attack (AA) [31] that is the strongest attack to verify the empirical robustness via an ensemble of diverse attacks ( i.e. , two variants of PGD attack, FAB attack [38], and Square Attack [39]).
Training Settings.
All models (unless otherwise specified) are pre-trained on ImageNet-1K and are adversarially trained for 40 epochs using SGD with weight decay 1e-4, and an initial learning rate 0.
that is divided by 10 at the 36-th and 38-th epoch.
Simple data augmentations such as random crop with padding and random horizontal flip are applied.
During adversarial training, we use PGDwith step size 2 / 255 to craft adversarial examples.
3.1 The Usefulness of (Natural) Pre-training
The success of ViTs heavily relies on natural pre-training on large datasets [1].
Meanwhile, natural pre-training is commonly believed to be useless for improving the adversarial robustness of naturally trained ViTs [9].
A question is naturally raised here: under adversarial training of ViTs , whether natural pre-training is useful or not.
[Figure] [Figure]
We first plot the learning curve of ViT-B with and without pre-training.
Specifically, one model is updated from an initialization from natural pre-training on ImageNet-1K, while the other is from random initialization.
In Figure 1(a), we find that ViT without pre-training achieves low robustness because under-fitting on the training data hinders the performance (the training robustness is only ∼ 30%).
Meanwhile, ViT with pre-training fits training data better and improves robustness by a notable margin (+20%).
This suggests that the current random initialization for weight parameters of ViTs is not suitable for adversarial training, even on small datasets like CIFAR-10, while natural pre-training on large datasets is useful for successful adversarial training of ViTs.
This conclusion seems different from the previous research [9].
This is because, although both study adversarial robustness, we focus on the effects of natural pre-training for adversarially trained ViTs, while they focus more on the robustness for naturally trained models.
Further, we explore if a larger dataset can bring more robustness.
Before adversarial training on
CIFAR-10, we adopt 3 pre-training strategies respectively: 1) natural pre-training on ImageNet21K (an extremely large dataset with 14M images from 21K classes), 2) natural pre-training on
ImageNet-1K (a large dataset with more than 1.2M images from 1K classes), and 3) no pre-training ( i.e. , adversarial training from scratch).
The final robustness under AA is shown in Figure 1(b).
Compared to adversarial training from scratch, pre-training on both ImageNet-1K and ImageNet-21K can improve the robustness of ViTs by a notable margin ( ∼ 16% on average), while ImageNet-21K fails in providing higher robustness (it even slightly hurts the performance) compared to ImageNet-1K.
The similar trends can also be observed on Imagenette.
In conclusion, natural pre-training is necessary for ViTs to fit the training data during adversarial training, while a larger dataset is unable to provide better robustness.
3.2 The Necessity of Gradient Clipping
[Table]
[Figure]
In natural training, some Transformers implementations ( e.g. , ViTs and Swin) apply gradient clipping, while the others ( e.g. , DeiT) do not.
Here, we explore whether it is necessary to use gradient clipping during adversarial training.
First, we find that, without gradient clipping, it is difficult to adversarially train ViT or DeiT on small datasets.
For example, the final robustness of ViT-B and DeiT-S is only 10% on both CIFAR-10 and Imagenette in Table 1.
To study the difficulty, we visualize the norm of gradient during adversarial training in Figure 2, and find the gradient explodes at the very beginning of training.
Then we adopt gradient clipping [40] again to adversarial training of ViTs.
Specifically, we ensure the maximum of gradient vector is 1 under the l 2 -norm.
In Table 1, with the help of gradient clipping, ViT-B (DeiT-S) achieves the robustness of 48.33% (47.26%) and 64.20% (60.20%) on
CIFAR-10 and Imagenette.
Although Swin-B can be adversarially trained directly, gradient clipping still improves the performance by a notable margin.
In comparison, gradient clipping has almost no effect on the robustness of CNNs, as shown in Appendix B.
In summary, for ViTs, although gradient clipping is optional in natural training, it seems necessary for its adversarial training.
3.3 The Effect of More Training Epochs
The natural training of ViTs usually requires more epochs for better natural accuracy ( e.g. , 300 epochs for Swin [2] on ImageNet-1K) compared to CNNs ( e.g. , 90 epochs for ResNet [4] on ImageNet-1K).
Adversarial training of ViTs is then supposed to require longer training epochs for better adversarial robustness [9].
Here, we further investigate it by adversarially training ViTs for 80 epochs (the default setting is 40 epochs) with the learning rate decay at the 75-th and 78-th epoch.
Figure 3(a) illustrates that neither ViT-B nor Swin-B benefits from the longer training, which potentially hurts the final robustness instead.
[Figure] [Figure]
[Figure] [Figure]
[Figure] [Figure]
3.4 The Effect of Advanced Data Augmentations
In adversarial training of CNNs, a majority of advanced data augmentations like Mixup [21] and
Randaugment [20] are proven to be unsuccessful (without considering model weight averaging)
[41, 42], which is contrary to natural training.
Here, we explore if these advanced data augmentations can improve robustness for adversarial training of ViTs.
Specifically, we incorporate CutMix [43],
Mixup [21], and Randaugment [20] to adversarial training of ViTs respectively.
In Figure 3(b), on
CIFAR-10, ViT-B obtains higher robustness with CutMix or Mixup or both, while behaving worse with Randaugment.
We also find similar phenomena of Swin-B on Imagenette.
This is because
Randaugment is too difficult for adversarial training of ViTs.
In conclusion, a suitable combination of data augmentations can directly improve the adversarial robustness of ViTs, which is different from
CNNs (needing model weight averaging).
In the following experiments, we always adopt CutMix and Mixup to adversarial training of ViTs by default.
[Table]
AdamW+cyclic 78.
AdamW+piecewise 80.
SGD+cyclic 83.
SGD+piecewise 83.
Imagenette
AdamW+cyclic 92.
AdamW+piecewise 87.
SGD+cyclic 93.
SGD+piecewise 93.
3.5 The Effect of Optimizer and Learning Rate Scheduler
Recalling that SGD achieves the best adversarial robustness in CNNs [13, 41].
By contrast, the default optimizer of ViTs (natural training) is AdamW [22].
Here, we investigate what is the appropriate optimizer for adversarial training of ViTs in order to obtain better robustness.
We adversarially train models based on every combination of the optimizer 5 (AdamW and SGD) and the learning rate schedule (cyclic and piecewise).
The results are summarized in Table 2.
If fixing the learning rate scheduler, whatever cyclic or piecewise, SGD almost all outperforms AdamW in terms of adversarial robustness.
When fixing the optimizer, we find that cyclic learning rate scheduler is suitable for AdamW while the piecewise learning rate scheduler is a better choice for SGD.
Furthermore, SGD+piecewise achieves better robustness compared to AdamW+cyclic in all datasets and models, i.e. , on CIFAR-10, SGD + piecewise helps ViT-B improve the natural accuracy by 4.49% and the adversarial robustness by 2.90% over AdamW + cyclic.
In summary, the combination of
SGD and piecewise learning rate scheduler maybe a good choice for adversarial training of ViTs.
3.6 Insights on the Adversarial Training of ViTs
With the above comprehensive evaluation of different training techniques, we can provide a practical adversarial training recipe for ViTs.
First, pre-training and gradient clipping are almost necessary.
Second, it would be better if incorporating advanced data augmentations ( e.g. , Cutmix and Mixup) and adopting SGD optimizer with piecewise learning rate scheduler.
Third, there seems no need to train longer epochs for the robustness improvement.
[Table]
To show the potential of ViTs’ adversarial training, we adversarially train ViTs following the above recipe and CNNs following the default setting, ensuring both of them own similar numbers of parameters.
In Table 3(a), on CIFAR-10, similar to previous findings [9], ViTs indeed do not advance the adversarial robustness compared to CNNs, while we think that we should not be overly pessimistic about this, after all CIFAR-10 is low-resolution dataset, while ViTs are usually trained on high-resolution datasets.
We might need a better adaption method for this input size difference.
On the positive side, on high-resolution datasets like ImageNette (Table 3(a)) and ImageNet-1K (Table 3(b)), the robustness results are totally different.
On Imagenette, transformers of different sizes (Swin-Ti for small size, and Swin-S for medium size) outperform their competitors (ResNet50 or
WideResNet-50-2 respectively) with a similar even smaller number of parameters by a notable margin ( > 10% ).
On ImageNet-1K (a more difficult dataset for adversarial tasks), we achieve robustness of 38.61% 6 , which is higher than the current state-of-the-art result of 38.14% on RobustBench .
This indicates the great potential for ViTs in terms of adversarial training.
An Architectural Perspective for Adversarially Robust ViTs
In contrast to CNNs, ViTs are a completely different architecture.
For example, it divides images into sequences of patches before feeding into the model, and it directly integrates global information across the entire images even in the lowest layers using self-attention [1].
This fundamental difference naturally raises a question: whether or not we are able to further improve its robustness based on such specific architectural information.
In this section, we explore the impact of these two most obvious architectural changes (multi-head attention and patch-based image splitting) inside ViTs on the adversarial robustness.
Multi-Head Attention
Norm
MLP
Norm
EmbeddedPatches
Norm
MLP
Gradient
F o r w a r d P r op a g a ti on
B ac k w a r d P r op a g a ti on × 𝐿 × 𝐿
[Figure]
+ =
[Figure]
4.1 Attention Random Droppping (ARD)
In ViTs, embedded patches are repeatedly processed by encoder blocks, any of which consist of a multi-head attention and a MLP layer, as shown in Figure 4(a).
For the i -th block, its input z i − 1 is first processed by a multi-head attention A ( i ) ( · ) , and intermediate result z ′ i
=
A ( i ) ( z i − 1 )
+ z i − 1 is further processed by the remaining parts f ( i ) ( · ) (including a MLP layer and a skip connection).
For a
ViT with L blocks, its output of the last ( i.e. , L -th) block is
[Equation]
According to the chain rule in calculus, the gradient of a loss function L with respect to the input of the last block z L − 1 can then be decomposed as
[Equation]
[Equation]
Here, we propose the Attention Random Dropping (ARD), which randomly drops the gradient flowing the multi-head attention
A ( L ) ( · ) ,
[Equation]
where we set u L to 0 with the probability of p and to 1 otherwise.
We continue to use the chain rule to obtain the gradient of ARD with respect to the input image x ,
[Equation]
where z 0 is the input embedded patches and all random variables
u i are i.i.d .
Figure 4(a) also illustrates the backward propagation using ARD.
When ARD meets adversarial training, we apply ARD as a warming-up strategy.
After the warmingup periods, the training becomes vanilla adversarial training.
In particular, the parameter p in ARD decreases linearly from 1 to 0 during the first n w epochs, i.e. , p = 1 − n/n w where n is the current epoch ( n ≤ n w ).
Here, p is a dynamic parameter along with the training process.
At the beginning, relatively weak adversarial samples (large p ) are used to warm up while as the training continues, the probability p should be gradually smaller to 0 to ensure the strength of the generated adversarial samples during the whole adversarial training process.
4.2 Perturbation Random Masking (PRM)
ViTs always split input images into non-overlapping patches, and then process them further.
Considering this patch-based image splitting, we introduce the Perturbation Random Masking (PRM).
Specifically, the perturbation on a patch will be masked with probability k during adversarial example generation:
[Equation]
where δ is the adversarial perturbation, and M is the mask to remove the perturbations on some patches.
The second equation is exactly the update rule of PGD attack.
Similar to ARD, we incorporate PRM as a warming-up strategy.
Considering an image that is split into J patches, the perturbations on ⌊ J · k ⌋ patches will be masked for each iteration during adversarial example generation within adversarial training, and we linearly decrease k from 1 to 0 at the first n w epochs, i.e., k = 1 − n/n w where n is the current epoch ( n ≤ n w ).
Here, same to p , k is also a dynamic parameter along with the training process.
4.3 Overall Algorithm
Obviously, ARD and PRM are two individual methods that have impact on two different aspects, i.e., ARD influences the backward propagation while PRM influences the update of adversarial perturbations.
Thus, we can easily combine them as a mixed warming-up strategy:
[Equation]
M
[Equation]
The details of ARD and PRM based adversarial training for ViTs are summarized in Appendix C.
4.4 Evaluation on Benchmark Datasets
In this section, we evaluate the effectiveness of ARD and PRM for the adversarial robustness of ViTs on benchmark datasets.
Experimental Settings.
For models, we apply 4 kinds of ViT (vanilla ViT [1], DeiT [19], ConViT [44], and Swin [2]) and 3 kinds of scales (base, small, and tiny) for each respectively.
For robustness evaluation, we also adopt 20 steps CW [27] ( ℓ ∞ version of CW loss optimized by PGD-20) andsteps PGD ∞ [13] in addition to PGD-20 and AA.
We evaluate the performance on: 1) the vanilla setting (the basic setting determined in Section 3), 2) the setting with ARD, 3) the setting with PRM, and 4) the setting with ARD and PRM.
To reveal the potential of our proposed methods, we select the hyperparameter n w via a grid search over { 5 , 10 , 15 , 20 } for each combination of methods ( + ARD, + PRM, or + both) and architectures (ViT-S, ViT-B, DeiT-Ti, et al. ).
Experimental Results.
In Table 4, after comparing the same kind of ViTs with different scales, we observe that increasing the size of ViTs can improve both the accuracy on clean examples and the robustness against adversarial attacks.
Interestingly, the better architecture in natural training on the high-resolution dataset ( e.g. , Swin on ImageNet-1K) fails to lead to better robustness in adversarial training on the low-resolution dataset.
We conjecture the inductive bias (multi-scale objects and the high resolution of pixels on high-resolution images) inside Swin is no longer valid on low-resolution images, which results in unsatisfactory performance.
Moreover, diving into each ViT model with different architectural approaches, we find that our proposed ARD or PRM has already boosted the robustness and accuracy, and the combination of both methods can improve the performance further.
For example, compared to the vanilla adversarial training which achieves the robustness of 44.88% with Swin-S on CIFAR-10, we increase the robustness to 46.04% ( + 1.16%) using ARD or 46.01% ( + 1.13%) using PRM, while we further improve the performance to 46.17% ( + 1.29%) using their combination.
Similar results are also observed on large datasets ( i.e. ImageNet-1k) in Appendix D.
4.5 Combination with Other Defense Methods
Although the above discussions are mainly concentrated on standard adversarial training, we further demonstrate that our methods: ARD and PRM, can be easily combined with other stronger defense methods.
We adopt TRADES [14] and MART [15] here because of their outstanding performance evaluated in [32].
The settings are the same as in Section 4.4 except for two points: 1) we do not apply
[Table]
CutMix and Mixup for data augmentations to keep in line with their original papers, and 2) we always set n w as 10 for simplicity.
The experiments are conducted on DeiT-Ti, ConViT-Ti
[Table]
and Swin-Ti.
As shown in Table
[Table]
the robustness and accuracy of TRADES and MART are improved on both datasets.
For example, for DeiT-Ti on CIFAR-10, AA robustness increases by 0.77% on TRADES and 1.63% on MART.
It demonstrates that ARD and PRM can further improve the performance of existing defense methods
[Table]
4.6 Hyperparameter Analysis
For simplicity of our proposed warming-up strategy, we use the same hyperparameter n w as the number of warming-up epochs for both ARD and PRM.
Taking CIFAR-10 as an example, we firstly analyze this shared hyperparameter n w ∈ { 0 , 5 , 10 , 15 , 20 , 25 } on four ViT variants (ViT,
DeiT, ConViT, and Swin), and the settings are the same as Section 4.4.
AA is used to evaluate the robustness.
In Figure 5(a), the robustness is significantly improved as long as adopting our proposed warming-up strategy into adversarial training, i.e. , n w > 0 .
We obtain the highest robustness at the best n w ( e.g. , n w = 15 for Swin-S).
Besides, the improvements are not sensitive to the hyperparameter, which allows us to tune n w easily.
To further exploit the proposed method, we perform experiments on Swin-S using two individual hyperparameters n aw and n pw for ARD and PRM respectively.
When fixing one ( e.g., n aw ) as 15 (the
[Table]
+MART 75.
68.60 +Ours 77.
10 15 20 25 n w 45.
R o b u s t n e ss ( % )
ViT-SDeiT-SConViT-
[Figure]
Swin-S  10 15 20 25 n w
A
[Figure]
u
[Figure]
ViT
[Figure]
S
25 n aw or n pw 44.
[Figure] [Figure]
R o b u
[Figure]
n
[Figure]
PRM (
[Figure] [Figure]
[Figure]
=15)     25 n aw or n pw
[Figure]
[Figure] [Figure]
best one in the last experiment), we change another hyperparameter ( e.g. , n pw ) as { 0 , 5 , 10 , 15 , 20 , 25 } .
In Figure 5 (b), for robustness, these two situations show different trends, i.e. , a longer warm-up with
ARD (the orange line) increases the final robustness and a longer warm-up with blue
) decrease the final robustness.
We can find a good balance when n aw = n pw = 15 .
The former has improved the performance enough, and the latter has not decreased too much.
As a result, we set n aw = n pw = n w by default.
Conclusion
In this paper, we conducted comprehensive experiments to investigate various training techniques to bring the first implementation benchmark for adversarial training of ViTs.
We find that, under adversarial training, pre-training and gradient clipping are necessary while SGD is preferred over
AdamW. Besides, taking the architectural information into consideration, we incorporated two simple but effective methods, ARD and PRM, into adversarial training to improve the adversarial robustness further.
Extensive experiments demonstrated the effectiveness of our proposed methods.
We hope our work not only provides the first implementation benchmark for adversarial training of ViTs, but also reminds researchers of the potential of architectural information contained within newly designed models like ViTs.
Acknowledgment
Yisen Wang is partially supported by the NSF China (No. 62006153), Project 2020BD006 supported by PKU-Baidu Fund, and Open Research Projects of Zhejiang Lab (No. 2022RC0AB05). A Broader Impact
As brand-new models, the vulnerability of ViTs to adversarial samples motivates us to upgrade their security.
Therefore, in this paper, we improve the adversarial robustness of ViTs in terms of strategies and architectures.
Our approaches may contribute to a safer use of ViTs in the real world.
However, the results also show that we are still far from fully robust ViTs.
Meanwhile, our additional sampling process may bring negative impacts on the environmental protection ( e.g. the emission of carbon dioxide).
B Gradient Clipping on CNNs
[Table]
We have shown the necessity of gradient clipping (GC) for ViTs in Section 3.2.
Here, we explore whether GC is beneficial to CNNs.
We pretrain CNNs on ImageNet-1K. On CIFAR-10, we substitute the first convolutional layer (kernel size 7 , stride 2 , padding 3 ) with a convolutional layer without down-sampling (kernel size 3 , stride 1 , padding 1 ) similar to [16, 17, 15].
Finally, we adversarially train CNNs using the same settings as Section 3.2.
The results in Table 6 demonstrate that gradient clipping has almost no effect on the robustness of CNNs.
This indicates the necessity of gradient clipping is specific for ViTs.
C Adversarial Training Algorithm after Combining ARD and PRM
N , warming-up epoch n w , number of splitted patches J sampled variable for i-th attention block u i , number of batches R do 5:
6: k p 7: p
. , m do 9: δ
10: for j = 1 to do 11:
Randomly choose ⌊ Jk ⌋ patches from all patches 12:
Generate mask M 13:
Update
Equation14: end for 15:
←
16: end for 17:
18: end for 19: end for
D Evaluation on ImageNet
[Table]
In this section, we evaluate the proposed method on ImageNet-1K, the most commonly used largescale dataset.
We apply the most popular threat model on ImageNet-1K, i.e. , setting the perturbation budget ϵ
= 4 / 255 .
During adversarial training, ViTs are adversarially trained for only 10 epochs to save experimental time, and the learning rate is divided by 10 at the 6-th and 8-th epoch.
We use
PGD-5 with the step size 2 / 255 to craft adversarial examples on the fly during training.
In Table 7, our method improves both natural accuracy and robustness by notable margins.
For example, we improve the natural accuracy of ViT-B by 6.76%, and robustness (evaluated by AutoAttack) by 5.81%.
Similarly, on Swin-B, we achieve robustness of 38.61%, which surpasses the best results on ImageNet-1K [32], which indicates that we benchmark a new state-of-the-art robustness on ImageNet-1K.