Enhance the Visual Representation via Discrete Adversarial Training
Abstract
Adversarial Training (AT), which is commonly accepted as one of the most effective approaches defending against adversarial examples, can largely harm the standard performance, thus has limited usefulness on industrial-scale production and applications.
Surprisingly, this phenomenon is totally opposite in Natural
Language Processing (NLP) task, where AT can even beneﬁt for generalization.
We notice the merit of AT in NLP tasks could derive from the discrete and symbolic input space.
For borrowing the advantage from NLP-style AT, we propose
Discrete Adversarial Training (DAT).
DAT leverages VQGAN to reform the image data to discrete text-like inputs, i.e. visual words.
Then it minimizes the maximal risk on such discrete images with symbolic adversarial perturbations.
We further give an explanation from the perspective of distribution to demonstrate the effectiveness of DAT.
As a plug-and-play technique for enhancing the visual representation, DAT achieves signiﬁcant improvement on multiple tasks including image classiﬁcation, object detection and self-supervised learning.
Especially, the model pre-trained with Masked Auto-Encoding (MAE) and ﬁne-tuned by our DAT without extra data can get 31.40 mCE on ImageNet-C and 32.77% top-1 accuracy on Stylized-ImageNet, building the new state-of-the-art. The code will be available at https://github.com/alibaba/easyrobust .
Introduction
Nowadays, Deep Neural Networks (DNNs) has achieved excellent performance surpassing humans in most computer vision tasks.
Although remarkable progress has been made, the success of DNNs is actually a false sense when i.i.d hypothesis is not satisﬁed in wild.
Researchers have shown that deep models fail in most circumstances including adversarial perturbations [1], common corruptions [2], colors or textures changing [3, 4], etc.
There is still a long way to make DNNs closer to the robust human perception.
A possible way towards robust machine perception can be Adversarial Training (AT) [5], which automatically ﬁnds failure input cases of DNNs and augment online with these cases for ﬁxing "bugs".
With online augmentation of adversarial examples, AT greatly enhances the adversarial robustness, and helps for learning perceptually-aligned representations [6] with good interpretability [7, 8] and transferability [9].
However, AT is double-edged, which meanwhile degrades the standard performance caused by problematic regularization [10].
Such problematic regularization makes the decision boundaries over-smoothed and enlarges indecisive regions.
Surprisingly, previous works [11, 12] observe a strange phenomenon that AT behaves conversely in Natural Language Processing (NLP) tasks.
By automatically ﬁnding adversarial textual inputs,
AT will not hurt the accuracy and even beneﬁt for both generalization and robustness of language models.
This phenomenon motivates us considering whether the merit of NLP-style AT can be transferred to vision tasks.
We notice such merit could derive from the unique data organizing form of language models.
To be speciﬁc, an adversarial image perturbed in continuous pixel space actually differs with the truly "hard" examples appeared in real world.
Contrarily, text space is discrete and symbolic, where adversarial text is practically existing when a typo is made by humans.
Learning on such adversarial text will obviously improve the generalization on other more texts with confusing typos.
Therefore, we borrow the symbolic nature of languages, and apply it on CV tasks by discretizing continuous images into a more meaningful symbolic space.
Afterwards, AT is conducted for minimizing the maximal risk on such text-like inputs with symbolic adversarial perturbations.
In this paper, we propose Discrete Adversarial Training (DAT), a new type of adversarial training which aims to improve both robustness and generalization of vision models.
DAT leverages VQGAN [13] to learn a vocabulary of visual words, also known as image codebook.
For a continuous image input, each encoded patch embedding is replaced with its closest visual word in the codebook, and represented as a corresponding index.
Then the image is transformed to a sequence of symbolic indices similar with language input.
For generating adversarial examples based on such symbolic sequence, direct use of optimization methods in NLP like combinatorial optimization [14, 15, 16] or synonym substitutions [14, 16] can be challenging.
The reason lies in: 1) the large search space of images and 2) the non-existence of synonym in visual codebook.
To make it more efﬁcient, DAT adopts a gradient-based method which assumes the backward adversarial gradient goes straightthrough the complex discretization process, thus gradients on discretized image can be copied to original input.
Then it use one-step search along the direction of estimated gradient such that the discrete representation will altered adversarially during the discretization process, resulting a discrete adversarial examples.
Finally the discrete adversarial examples is fed into models for training.
Different from AT which always adds l p bound on augmented adversarial examples, DAT affects the discretization process to produce diverse adversarial inputs beyond l p bound for training.
We show in ablation experiment that DAT not only enhances the robustness on l p bounded attacks, but also is partly beneﬁcial in defense of unrestricted semantic attacks [17, 18].
The overall pipeline of discrete adversarial training is shown in Figure 1.
We further give an analysis to explain the effectiveness of our DAT from the perspective of distribution.
By comparing the distributional difference of training examples in AT and DAT with clean images, we ﬁnd the discrete adversarial examples in our DAT are much closer to the clean distribution.
Such ability of generating "in-distribution" adversarial examples makes DAT can improve the visual representation learning on multiple vision models and tasks, with no sacriﬁcing of clean accuracy.
Our contributions are summarized below:
• To the best of our knowledge, we appear to be the ﬁrst to transfer the merit of NLPstyle adversarial training to vision models, for improving robustness and generalization simultaneously.
• We propose Discrete Adversarial Training (DAT), where images are presented as discrete visual words, and the model is training on example which has the adversarially altered discrete visual representation. • DAT achieves signiﬁcant improvement on multiple tasks including image classiﬁcation, object detection and self-supervised learning.
Especially, it establishes a new state-of-the-art for robust image classiﬁcation.
By combining MAE [19] pre-training and DAT ﬁne-tuning, our ViT-Huge [20] achieves 31.40 mCE on ImageNet-C [2] and 32.77% top-1 accuracy on Stylized-ImageNet [3].
Related Work
Adversarial Training Adversarial Training (AT) [5] is ﬁrst proposed to improve robustness by training models with adversarial examples.
As one of the most effective defense, existing works [21, 22] have suggested a trade-off between adversarial and clean accuracy in AT.
Despite great efforts [23, 24, 25] have been made for mitigating this trade-off, the bad generalization of
AT still cannot be fully remedied till now.
In opposite way, some other works [26, 27] use AT to improve the clean accuracy rather than adversarial robustness.
The most close work to ours is the AdvProp [26], which splits batch norms to prevent the mixed statistics of clean and adversarial examples, thus learns better adversarial feature for generalization.
Pyramid AT [28] makes AT speciﬁc to ViTs by crafting pyramid perturbation with Dropout enabling, yielding imporved performance.
However, these methods are only applicable under speciﬁc models or tasks.
VILLA [29] is also a representation enhancement technique using AT.
But contrary to us, it applies vision-style AT only for vision-and-language representation learning.
AGAT [30] is another kind of AT beyond pixel space, which perturbs images along attributes, however it has strict requirement of attributes annotation.
Adversarial Augmentation
By borrowing the idea of AT, some previous works [31, 32, 33] propose to search augmentations adversarially for improving the hardness of training examples.
Adversarial AutoAugment [31] uses augmentation policy network to produce hard augmentation policies on a pre-deﬁned policy space.
AugMax [32] mixes multiple randomly sampled augmentation operators like AugMix [34], by using adversarially learned mixing factors.
AdA [33] optimizes the parameters of image-to-image models to generate adversarially corrupted augmented images.
MaxUp [35] uses the worst augmented data of each data point in a set of random perturbations or transforms for training.
However, these methods create adversarial inputs indirectly and rely on pre-deﬁned augmentations or translation models.
They cannot perturb the images locally.
Instead, our DAT directly modiﬁes the image with no need of pre-deﬁned transforms, and can craft local perturbations on images, which is more elaborate.
Discrete Visual Representation Learning Early technique [36, 37] of Bag-Of-Visual-Words (BOVW) model has shown great power of discrete representation in visual understanding.
VQVAE [38] uses DNNs to learn neural discrete representations, also known as visual codebook, by generative modeling the image distribution.
Recently, the idea of discrete representations learning has been widely emerged in many vision tasks.
In most Masked Image Modeling (MIM) methods [39, 40], visual codebook is needed for BERT-like self-supervised pretraining.
For image classiﬁcation, discrete representations strengthen the robustness by preserving the global structure of an object and ignoring local details [41].
For image synthesis, adversarial and perceptual objective can be added to VQ-VAE for learning perceptually-aligned visual codebook [13].
With growing power of generative models, now a well trained VQGAN can produce vivid images with 0.58 FID [42].
In this work, we use VQGAN for the discretization process in our DAT.
As a powerful generative model, VQGAN can greatly reduce the information loss in reconstruction process and ensure the high-quality of the generated discrete adversarial examples.
Method 3.1 Traditional Adversarial Training
We take the typical image classiﬁcation task as an example to show the formulation of Adversarial
Training (AT).
Consider the training image and label set D
= { x i , y i } ni =1 , and a classiﬁer F with learnable parameters θ , the classiﬁcation objective is always a cross-entropy loss L ( x, y, θ ) .
Adversarial Training (AT) ﬁnds the optimal θ by solving a minimax optimization problem:
[Equation]
where the inner optimization ﬁnds the perturbations δ on per-pixel values for maximizing the loss, and the outer minimization update θ to improve the worst-case performance of the network w.r.t.
the perturbation.
(cid:107) · (cid:107) p constraints the p -norm of δ to a small value (cid:15) .
A problem is that AT ﬁnds the failure case i.e. x + δ in continuous pixel space.
However, human does not create or recognize images from complex pixel values, but from discrete semantic concepts.
Although adversarial examples can successfully fool the models, they are still different from the real "hard" examples appeared in practice.
3.2 Discrete Adversarial Training 3.2.1 Image Discretization by Visual Codebook
For discrete adversarial training, it is desirable ﬁrst to learn an expressive visual codebook and represent the training image set in discrete space.
We utilize VQGAN [13] for image discretization.
More precisely, consider a continuous image x ∈ R H
× W × 3 , VQGAN learns an encoder Enc φ ( · ) , decoder Dec ψ ( · ) and quantization q Z ( · ) . Enc φ is a convolutional model which maps x to intermediate latent Find the worst 𝜹 by
𝐦𝐚𝐱 𝜹 𝓛
( 𝓠 𝒙 + 𝜹 , 𝒚 , 𝜽 )
Update model parameters 𝜽 by 𝐦𝐢𝐧 𝜽
𝓛 ( 𝓠 𝒙 + 𝜹 , 𝒚 , 𝜽 )
Enc 2 0 10 1 3
Dec
Codebook 𝓩
N … … 0 1 22 3 10 1 3
Visual Tasks
Self-Supervised Learning 00.20.40.60.81 cat tiger lion dog
Classification
Object Detection ∇ !"
ℒ 𝛿
ForwardDataFlow
Backward Gradient Flow Straight-Through Gradient ⨁
Straight-Through Gradient Flow 𝑥 + 𝛿 𝑥 𝑥
#$% =
𝒬 ( 𝑥 + 𝛿 ) * 𝑥 = 𝒬 ( 𝑥 ) vectors v =
Enc φ ( x ) ∈ R ( h × w ) × d , where h, w is the height, width of the intermediate feature map and d is the latent dimension.
Subsequently, q Z ( · ) learn a codebook Z
= { z k | z k ∈ R d } Kk =1 , such that each latent vector v ij ∈ R d can be quantized onto its closest codebook entry z k as:
[Equation]
where i, j present each location in feature map.
Then the decoder Dec ψ ( · ) outputs the reconstruction image ˆ x from the quantized vectors v q by:
[Equation]
VQGAN is trained by minimizing the reconstructed difference between ˆ x and x .
More details can be referred to Appendix A. So far, given a continuous image x , we can get its corresponding discrete reconstruction ˆ x .
For simplicity, we use Q to stand for the above image discretization process, and then we have ˆ x = Q ( x ) .
3.2.2 Discrete Adversarial Training
Based on the deﬁnition in Sec 3.2.1, we can generate discrete adversarial examples in inner maximization step of AT.
By slightly modifying the Eq 1, the objective of DAT is formulated as:
[Equation]
where Q transforms the continuous pixel space to discrete input space.
We delete the constraint term since there is no need to bound the per-pixel values of δ .
Suppose that Q is an ideal discretizer with no information loss in discretization process.
The problem lies how to ﬁnd the worst δ for maximizing the classiﬁcation loss.
Similar with traditional AT, we can use gradient-based methods to approximate δ by:
[Equation]
where α determines the magnitude of the perturbations along the gradient direction.
We set α = 0 .by
default in DAT.
To expand ∇ x L ( Q ( x ) , y, θ ) by chain rule, we have four partial derivative terms as follows:
[Equation]
Through analysing the feasibility of each term, we ﬁnd only ∂v q ∂v is hard to solve as the nondifferentiable nature of Eq 2.
Fortunately, as proposed in previous work, a straight-through gradient estimator [43, 44] can be used by copying the gradients from v q to v .
By replacing ∂ ˆ x ∂v q · ∂v q ∂v with ∂ ˆ x∂v , we can simplify the Eq 6 to ∇ x L ( Q ( x ) , y, θ ) =
∂
L ∂ ˆ x · ∂ ˆ x∂v · ∂v∂x , which has derivative everywhere.
Although the solution seems workable theoretically, the huge cost makes it impractical on large-scale vision tasks.
The bottleneck mainly lies on that ∂ ˆ x∂v and ∂v∂x require the adversarial gradients backward through Enc and Dec .
Actually, a generator capable of producing high-quality images always has a large amount of parameters.
Compared with original adversarial training which only needs F for gradient calculation, it requires more than tripled GPU memory and computation cost.
To solve this problem, we propose an efﬁcient alternative solution.
Since ˆ x (cid:39) x is empirically observed for an ideal discretizer Q , we can also use a straight-through estimator between ˆ x and x , which is given by
[Equation]
Finally, we can solve the worst δ by ∇ ˆ x L
(ˆ x, y, θ ) .
By this way, the computation cost of DAT has been largely reduced.
Compared with original adversarial training, it only has extra computation cost on VQGAN forward, which is relatively controllable.
For clarity, let us restate the pipeline of our DAT.
For each training image x , DAT ﬁrst use VQGAN to get discrete reconstruction ˆ x .
By feeding ˆ x to classiﬁer F , a worst-case perturbation δ can be estimated by computing the gradient of ˆ x towards maximizing the classiﬁcation loss.
The perturbed image thus can be created by adding δ on original x .
Finally, x + δ is discretized by VQGAN again and acts as the adversarial input, on which F is trained by minimizing the classiﬁcaiton loss.
The details of our DAT is summarized in Algorithm 1. [Figure]
F r e q u e n c y
[Figure]
F r e q u e n c y Explaining the Effectiveness of DAT from the Perspective of Distribution We give an empirical analysis to explain why DAT can improve the robustness and generalization without sacriﬁcing clean accuracy.
Previous work [26] has pointed out that the underlying distributions of adversarial examples are different from clean images.
Training on both clean and adversarial images will force the Batch Normalization (BN) [45] to estimate an inaccurate mixture statistics of feature distribution, and thus impact the standard performance.
We study this effect by sampling 1000 mini-batches in ImageNet validation set, and generate corresponding adversarial images for AT and our DAT.
For each batch we calculate the mean and variance statistics of last BN of ResNet50.
Then the Pearson correlation coefﬁcient (PCC) between the statistics on clean and adversarial examples is computed for showing the distributional difference.
Figureshows the frequency histogram of the distributional difference on 1000 mini-batches.
For training samples of DAT, the peak of the histogram is at 0.95, which is greater than AT.
It suggests that DAT generates discrete adversarial examples much closer with the clean distributions.
Therefore, training [Table]
[Table]
Experiments
To demonstrate the versatility of our method, we experiment Discrete Adversarial Training (DAT) on multiple tasks including image classiﬁcation, object detection and self-supervised learning.
4.1 Image Classiﬁcation
Implementation We implement DAT on two representative architectures: ResNet50 [46] and
ViTs [20].
For ResNet50, we ﬁrst experiment DAT with vanilla training recipes using "robustness" library 1 .
Then we combine DAT with other orthogonal robust training techniques: DeepAugment [47] and AugMix [34].
Only cross entropy loss is used for generating discrete adversarial examples.
The
JSD loss in AugMix is optimized merely on clean samples.
For ViTs, we adopt ViT-B/16 as baseline models, which is trained by the recipes in AugReg [48].
Besides, we use DAT to conduct supervised ﬁnetuning on downstream ImageNet classiﬁcation task based on a self-supervised ViTHuge pretrained by MAE [19].
By default, we refer ViT to ViT-B/16 in all tables and ﬁgures.
Benchmarks The trained model is evaluated in three aspects: 1) in-distribution performance on
ImageNet-Validation set; 2) adversarial robustness on white-box FGSM [50] and transfer-based black-box attack dataset DamageNet [51]; 3) out-of-distribution robustness on ImageNet(IN)-A,
IN-C, IN-V2, IN-R, IN-Sketch and Stylized IN.
Each of them represents a type of out-of-distribution scenario where the classiﬁer is prone to make mistakes.
IN-A [52] places the ImageNet objects in hard contexts; IN-C [53] applies a series of noise, blur, digital and weather corruptions; IN-R [47] collects online images with artiﬁcial creation, e.g., cartoons, graphics, video game renditions, etc;
IN-Sketch [54] contains images described by sketches; Stylized IN [3] destroys the texture but maintains the shape feature by conducting style transfer on ImageNet images.
Except for IN-C which is measured by mCE, we report the top-1 accuracy on all above datasets.
Results We report all results in Table 1.
For fair comparison, we add DAT on base methods without modifying the original training hyper-parameters, such that improvement is entirely attributed to the
DAT.
For ResNet50, DAT achieves signiﬁcant improvement on both clean accuracy, adversarial and [Table] out-of-distribution robustness.
The improvement seems greater when combining DAT with DeepAugment and AugMix.
For ViTs, we ﬁnd DAT is compatible with other complex augmentations such as MixUp [55], CutMix [56] or RandAugment [57] used in AugReg, yielding greater improvement.
Compared with plain ViT, AugReg-ViT and ViT with discrete representation called DrViT [41], DAT with AugReg achieves better performance.
The best result is from ViT-Huge pretrained by MAE [19] and ﬁnetuned by DAT, which suggests DAT is also effective in downstream ﬁne-tuning tasks.
Additionally, we also compare DAT with other robust training strategies in Table 2.
AugReg-ViT is adopted as the baseline model.
Most strategies, e.g., AdvProp [26] and Debiased [49] are proposed for ResNet with auxiliary BatchNorm.
We show these methods cannot work properly on ViTs with only LayerNorm.
Compared with Pyramid AT [28], our DAT has lower clean accuracy but yields stronger robustness.
More results of strategies comparison on ResNet50 refer to Appendix B.1.
4.2 Self-Supervised Learning
Implementation We experiment DAT on three self-supervised methods: SimCLR [58], SimSiam [59] and recently proposed MoCov3 [60].
The discrete adversarial training is only conducted during pre-training stage, and the learned representation is evaluated on downstream tasks by standard pipeline [61].
We craft adversarial examples based on RoCL [62], which attacks the pre-training objective by maximizing the contrastive loss.
For preventing the cost explosion, we pre-trainepochs for SimCLR, and 100 epochs for both SimSiam and MoCov3.
The batch size used for
SimCLR, SimSiam, MoCov3 is set as 1024, 512, 2048 respectively.
Benchmarks For self-supervised learning, we adopt ImageNet-1K for both training and indistribution testing.
Beyond that, in order to give more comprehensive assessment, we build three downstream tasks to metric the transferability of the learned visual representation.
Speciﬁcally, the linear evaluation reports the top-1 accuracy on ﬁve classiﬁcation datasets: Flowers, CIFAR10,
Caltech101, Cars and DTD.
For downstream evaluation of object detection, we present mAP, AP50,
AP75 on Pascal VOC2007 [63].
ADE20K [64] is used for semantic segmentation task, and both the mean intersection over union (mIoU) and accuracy are reported.
[Table] Results As shown in Table 3, DAT can enhance the learned representation on ImageNet and get 0.97%, 0.58% and 0.25% improvement on MoCov3, SimCLR, SimSiam respectively.
For excluding that DAT is not just over-ﬁtting on ImageNet, we transfer the representations on downstream recognition, object detection, semantic segmentation tasks.
The results suggest that DAT enhances the self-supervised representations with better transferability.
In particular, MoCov3 with DAT achieves signiﬁcant improvement in all downstream tasks.
This conclusion echoed with previous works [9], which ﬁnd adversarially robust models often perform better on transfer learning [Table] Models Datasets f
K d
FID ImageNet FGSM A
C ↓
R
Stylized
VQGAN [42]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
4.3 Object Detection
Implementation We implement DAT on two popular detectors: EfﬁcientDet [65] and YOLOv3 [66].
Object detection generally has two sub-tasks: classiﬁcation and localization.
DetAdvProp [67] proposes to select more vulnerable sub-task to generate adversarial images, and use auxiliary BN for training.
Instead, DAT regards the two sub-tasks as a whole, and attacks the overall detection loss to produce adversarial images.
Besides, DAT does not modify BN in detectors.
As the memory and computation cost of VQGAN increased quadratic with the input resolution, DAT become unaffordable when input size is large.
Therefore, we only experiment DAT on lightweight version of EfﬁcientDet and YOLOv3 with input size smaller than 512.
Benchmarks We train the detectors using the COCO 2017 object detection dataset [68] and evaluate them on COCO’s validation set.
COCO-C [69] is used for testing the robustness to natural corruptions.
We report mAP on COCO and COCO-C as the clean and robust performance respectively.
rPC [69] is used to to measure relative performance degradation under corruption.
Results We compare the detectors with DAT, Det-AdvProp [67] and vanilla training in Table 4.
Although Det-AdvProp has been shown effective on EfﬁcientDet with size larger than D0, we ﬁnd on smaller detectors, the promotion of Det-AdvProp is subtle.
On EfﬁcientDet-Lite0 and YOLOv3-320, it even gets worse on COCO mAP than vanilla training.
While our DAT achieves better result on both clean and corrupted input.
EfﬁcientDet-Lite0 with DAT get 27.3 and 17.89 mAP on COCO and COCO-C, resulting in 65.53% Relative rPC.
By comparison, vanilla YOLOv3 models have lower Relative rPC, showing it is more vulnerable than EfﬁcientDet at same clean mAP.
After equipped with DAT, YOLOv3-320 can get 20.55 mAP on COCO-C, leading to 7.75% improvement on rPC.
4.4 Ablations
Impact of the Codebook in DAT A general understanding is that the codebook with larger K should have stronger representation power.
As suggested in Table 5, when the K is reduced from 16384 to 256, the FID of generated images increases 0.35.
We show DAT on larger codebook size can achieve better generalization.
However, the clean accuracy seems no improvement with increasing of
K .
We think the reason is that the reconstruction quality has already met the demand of DAT.
And the improvement on clean accuracy is saturated when K further increases.
Different Types of Discretizer Q
Our work is based on the hypothesis of an ideal Q with no loss in discretization process.
However, such perfect Q is not existing in practice.
So we study if and how the ability of Q affects the performance of our DAT in Table 5.
The results show DAT performs better by using discretizer with higher FID.
We ﬁnd clean accuracy declines 1.3% on DALL-E, showing it is important for choosing a powerful discretizer.
Types of pre-training datasets have little impact.
We ﬁnd the discretizer pre-trained on OpenImages [70] and transferred to discretize ImageNet images, can also achieve comparable results.
The Performance on Different Magnitude α
We present the DAT results on ResNet50 and ViT with different magnitude α in Figure 3.
α = 0 means the model is only trained on images augmented by VQGAN with no adversarial training process.
We show DAT with α = 0 makes the models have high clean accuracy but the lowest robustness.
With the increase of α , there is still a robustness and accuracy trade-off.
DAT with α = 0 .
4 on ResNet50 achieves the best adversarial robustness but generalization is getting worse.
By contrast, we surprisingly ﬁnd ViT has the lower sensibility on α .
Even if ViT is trained on DAT with α = 0 . 4 , there is still no great drop of clean and OOD accuracy.
We suspect that ViT is more suitable for training on discrete examples, and the strong modeling ability makes ViT greater than ResNet50.
��� ��� ��� ��� ��� ��������� ���� ���� ���� ���� ���� � � � � � � � �� � � � � � �������� ��� (a) Clean Accuracy ��� ��� ��� ��� ��� ��������� ���� ���� ���� ���� ���� � � � � � � � � � � � � � � � � � � � � �� ��������
[Figure]
Results on Stronger Attacks In the main experiment, only one-step FGSM is used for examining the robustness under white-box adversarial attacks.
To give more comprehensive evaluation, we additionally test our DAT under stronger AutoAttack [73], and two unrestricted attacks named AdvDrop [17] and PerC-Adversarial [18].
The result is shown in Figure 4.
DAT brings the improvement of robustness under all three attackers.
ViT trained with DAT achieved extremely high robust accuracy on PerC-Adversarial, which suggests DAT also effects well in defense against unrestricted adversarial attacks.
More details can be referred to Appendix C. ��� ��� ��� ��� ��� ���� ��� ���� ���� ���� ���� � � � � � � � � �� � � � � � �������� ������������ ��� ������� (a) AutoAttack [73] ���� ���� ���� ���� ���� ������������������ ��� ���� ���� ���� ���� � � � � � � � � �� � � � � �
[Figure]
4: The adversarial robustness test under other stronger attackers
4.5 In-depth Analysis
Discrete Perturbations vs. Pixel-Space Perturbations
We compare the proposed Discrete Adversarial Examples (DAEs) with traditional Pixel-space Adversarial Examples (PAEs) in Figure 5.
DAEs have following three superior properties: 1) DAEs are more realistic.
By calculating the number of colors [17], we ﬁnd PAEs add more invalid colors, resulting in a noisy image.
While DAEs have minor changes on the color numbers, its modiﬁcation is harder to be perceived by humans.
To give quantitative results, we additionally report the FID score of DAEs.
The FID score of our DAEs is 14.65, which is lower than 65.18 of AEs.
The results are also consistent with our visual presentation result in Figure 5; 2) DAEs have less high frequency component compared with AEs in frequency analysis.
It may lead the DAEs more close to natural image distributions; 3) Discrete perturbations are more structural.
From the perturbation visualization in third row of Figure 5, we ﬁnd discrete perturbations have more structured information about objects, shown it attends to more important locations.
While pixel-wise perturbations are noisy and disordered.
Visualized Attention.
We visualize the attention of ViT trained by DAT in Figure 6(b).
For the object with unusual renditions in ImageNet-R, ViT cannot attend to semantically relevant image regions.
While our DAT can locate the attention to the central object more related for the classiﬁcation.
This phenomenon is also reﬂected by the statistical average of the attention in Figure 6(c).
By randomly sampling 1000 images in ImageNet-R and averaging the attention maps, we ﬁnd the attentions of our DAT are more global.
Compared with ViT which puts much attentions on the corners of the image found by prior work [20], our DAT additionally attends the central regions in where the classiﬁed object is often located.
80 c o l o r s 111 c
o l o r s 222 c
o l o r s 187 c
o l o r s 184 c
o l o r s 241 c o l o r s
Discrete AEs (Ours) Original Image
Pixel-Space AEs
Discrete AEs (Ours) Original Image
Pixel-Space AEs
N u m b e r o f c
o l o r s
F r e qu e n c y S
p e c t r u m
P e r t u r b a t i o n V
i s u a li z
a t i o n
[Figure]
Shape-bias Analysis.
We conduct an analysis based on shape-bias, which represents the fraction of correct decisions based on object shape.
The result is shown in Figure 6(a).
The averaged scores on 16 categories is denoted by the colored vertical line.
We compare decisions with Humans, ResNet w/ and w/o DAT, ViT w/ and w/o DAT.
Human decisions are highly based on shape, which achieve the best average fraction of 0.96 to correctly recognize an image by shape.
By comparison, ViT and
ResNet50 still have large gap with humans on the ability of learning shape features.
In this work, we ﬁnd our DAT can help for improving the fraction of shape-based decisions of models.
It suggests
DAT regularizes the models to learn texture independent shape features, and behave more like a human.
Limitations and Conclusions
In this paper, we ﬁnd transferring NLP-style adversarial training to vision models can enhance the learned visual representation effectively.
We propose Discrete Adversarial Training (DAT), where images are presented as discrete visual words by VQGAN, and the model is training on examples which have the adversarially altered discrete visual representation.
DAT needs not to modify the model architecture and works for both CNNs and ViTs across multiple tasks.
DAT reports the state-ofthe-art robustness on ImageNet-C and Stylized-ImageNet, exhibiting strong generalization.
However,
DAT still costs increased training time, this limitation also holds for any adversarial training.
The strict assumption of an ideal discretizer is another potential limitation, despite DAT has used powerful
VQGAN model to approach this assumption.
The effect of DAT is empirically studied without deeper theoretical explanation. All the above limitations are remained as the future optimization direction.
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] (c) Did you discuss any potential negative societal impacts of your work?
[No] There are no negative societal impacts needed to discuss.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[N/A] (b) Did you include complete proofs of all theoretical results?
[N/A] 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[No] The experiment is expensive and need long time, so we couldn’t run multiple times (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] See in Appendix 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] (b) Did you mention the license of the assets?
[Yes] (c) Did you include any new assets either in the supplemental material or as a URL?
[No] We do not release new assets (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
[N/A] 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]
The Appendix is organized as follows.
Appendix A extends the discussion on training of VQGAN.
Appendix B presents more experimental results including comparison of robust training strategies, effect analysis on different perturbations, images corruptions and inference with the discretizer Q .
Besides, we analyse the training budget of our DAT.
Appendix C discusses the implementation details of the three attackers in ablation experiments.
Finally we do some visualizations in Appendix D.
A Details of Training VQGAN
We add more details about how VQGAN can be trained to discretize a continous image.
The forward pass of VQGAN has been introduced in Sec 3.2.1.
Suppose a reconstructed image ˆ x , the training objective for VQGAN is deﬁned between ˆ x and x as:
[Equation]
where (cid:107) x − ˆ x (cid:107) precept is the perceptual reconstruction loss instead of L 2 loss, sg [ · ] denotes the stopgradient operation, and (cid:107) sg [ v q ] − Enc ( x ) (cid:107) 2 2 is the commitment loss [38].
A patch-based discriminator
D is introduced for improving the quality of generated images by adversarial learning [74].
In this work, we use the pre-trained VQGAN weights for DAT directly, which are opened on GitHub 23 .
VQGAN with f = 8 , d = 4 and K = 16384 is used for main experiments.
We acknowledge the authors of [13, 42], whose works have greatly promoted our DAT.
B More Experimental Results B.1 Strategies Comparison on ResNet
Table 6 presents the comparison results of DAT with other robust training strategies on ResNet50.
Note that Advprop, Fast Advprop and Debiased use auxiliary BN while Pyramid AT and DAT not.
Advprop achieves the best clean accuracy.
The reason may lie in the auxiliary BN reduces the impact of adversarial examples on standard performance.
On ResNet50, DAT is not always the best on the robustness benchmarks.
By augmenting with the style transferred data, Debiased achieves best on
IN-C, IN-Sketch and Stylized IN.
On the contrary, such robust training method proposed for ViTs, e.g. Pyramid AT, is not incompatible with ResNet50 and obtain unsatisfactory results.
However, our
DAT is a general method which works for both CNNs and ViTs
[Table]
and can be more practical
[Table]
of DAT
with other training strategies.
We use ResNet50 as the base model.
B.2 Comparing with Traditional Adversarial Training
We compare our DAT with open-sourced adversarially robust models 4 in Table 7.
From the results, we can see a clear quantiﬁcation of the beneﬁt of our proposed discrete AT scheme compared with traditional AT.
For clean performance, traditional AT plays a negative impact.
However DAT can reduce the negative impact and achieve higher accuracy on validation set of ImageNet.
It even surpasses the clean performance of normal training.
The results also suggest AT with a very small (cid:15) can slightly beneﬁt the generalization, e
[Table]
g., with (cid:15) =0.01 L2 AT, ImageNet-C mCE value from 76.
drops to 75.33.
But with the (cid:15) becoming larger, AT greatly damages the generalization, e
[Table]
g. with (cid:15) =5.0 L2 AT, ImageNet-C mCE value increases to 88.98.
In contrast, our DAT achieves signiﬁcant improvement on generalization compared with traditional AT.
[Table]
B.3 DAT with Different Perturbations
Table 8 counts for the percentage of the modiﬁed visual words with different magnitude α .
We show only 3.8% of the visual words are changed when we set α as 0.1.
This proportion does not have negative impact on the trained models, but even somewhat beneﬁts the clean accuracy.
With larger proportion of visual words being adversarially altered, the standerd performance and generalization are getting worse.
We also experiment DAT with random perturbations.
To introducing the randomness, we selects 3.8% visual words and replaces them with other words.
We ﬁnd this operation can slightly improvement the generalization of learned representation, but it still cannot achieve the comparable effect as our DAT.
It suggests adversarially altering the visual words is a better way.
[Table]
B.4 The Effect of DAT on Corruptions in ImageNet-C
To analyse the effect of DAT on each image corruption in ImageNet-C, we report the detailed results in Table 9.
For ResNet50, we ﬁnd DAT reduces the accuracy on images with contrast and fog corruptions.
It demonstrates that ResNet50 trained by DAT can be sensitive to the image lack of the hierarchy.
However, for ViT, DAT can improve the performance on all corruptions.
It suggests DAT works more efﬁciently on transformer-based vision models
[Table]
B.5 Training Budget for DAT
DAT is experimented on 32 2080Ti GPUs.
We compare the training cost with other robust training strategies in same setting.
The results is shown in Table 10.
DAT only needs one attack step to generate discrete adversarial examples, which makes it less expensive than standard adversarial training.
However, DAT still requires 3.5 × training budget than normal training.
We believe that reducing the cost of DAT is necessary, which will be remained as the future work
[Table]
B.6 Inference with the Discretizer Q
In this work [Table] we delete the discretizer Q at inference time.
However, there is another option that remaining the discretizer for test inputs.
To study the effect of this alternative [Table] we report some results in Table 11.
Although inference with Q brings improvement on adversarial robustness, it meanwhile harms the standard performance and generalization.
The inference cost also increases by the additional computation on Q .
Therefore, such alternative cannot yield the best trade-off on speed and performance, which is not adopted by our DAT
[Table]
B.7 Is it necessary for bounding δ ?
Traditional adversarial attacks always add constraints on perturbations.
While in this work, there are no restrictions on δ .
To explore if bounding δ is necessary in DAT, we add l ∞ bound on δ with different (cid:15) and re-run the DAT.
The result is shown in Table 12, DAT achieves best performance when δ is not bounded.
The worst result is appeared when δ is bounded with (cid:15) = 1 / 255 .
With larger (cid:15) , the results become better.
Therefore, it seems bounding δ in our DAT is not necessary [Table] and even plays negative affect on the overall performance
[Table]
B.8 DAT for Domain Generalization
In addition to training on large-scale ImageNet, we evaluate our DAT on Domain Generalization (DG) tasks.
DG task is more challenging since it needs the learned model to transfer between multiple domains, using only small amount of the data.
We adopt PACS dataset, which consists of four domains, namely Photo, Art Painting, Cartoon and Sketch.
Each domain contains seven categories.
For each trial, we train on 3 domains for generalizing to remaining unseen domain.
To keep the setting consistent with previous works, we use AlexNet as the backbone.
The results are shown in below Table 13.
We only compare with previously adversarial augmentation based DG methods.
DAT can also achieve better domain generalization performance on PACS dataset.
It has slight drop on domain of photo, but improves the transferability on other three domains.
[Table]
C Implementation of the Stronger Attacks in Ablation Experiments
For AutoAttack, we attack a subset of ImageNet provided in RobustBench 5 , which consists ofimages.
Three perturbations bounded with l ∞ -norm of 1 / 255 , 0 .
5 / 255 and 0 .
25 / 255 are adopted.
For AdvDrop, the test dataset is 1000 random sampled images on ImageNet, which is provided in the ofﬁcial implementaton 6 .
The bound of the quantization table is a key factor in AdvDrop which controls the attack strength.
We use three bounds with 10, 15 and 20 to regularize the quantization table.
For PerC-Adversarial, we use the Perceptual Color distance Alternating Loss (PerC-AL) method to generate adversarial examples.
PerC-Adversarial uses the test data of Defense Against
Adversarial Attack Challenge in NeurIPS 2017.
We change the attack strength by three different attack iterations: 20, 40 and 60.
D Visualization D.1 Comparison of the Reconstruction Quality
We compare the reconstruction quality of different discretizers Q in Figure 7.
VQGAN with k = 16384 and f = 8 can retain the most of the image details, which has the least impact on the classiﬁcation models when used for training.
With the growing of the downsampling factor f , some ﬁne-grained attributes of the objects can be changed.
For example, in third row the spotted texture on the peacock’s tail is partly lost after reconstruction.
Compared with VQGAN models, DALL-E blurs the image to a greater extent, yielding a low quality reconstructed image.
Accordingly, DAT which uses DALL-E for image discretization performs worst on clean accuracy in Table 5.
It shows that the reconstruction quality is indeed proportional to the standard performance in DAT.
D.2 Visualization of Discrete Adversarial Examples
We visualize the discrete adversarial examples which is generated by DAT for training in Figure 8.
With the growing of α , the alteration on images become larger.
However, different from traditional adversarial example which essentially adds global high-frequency noise on images, we ﬁnd DAT modiﬁes the properties of the local part of the object.
For example, in the ﬁfth row of Figure 8, the discrete adversarial example is changing the eye color of the cat.
Such modiﬁcation is large but imperceptible and semantic-preserving.
There are also some failure case where DAT changes the semantics.
In sixth row of Figure 8, after discrete reconstruction, the otter in image looks more like a dog.
We believe that such cases are only a minority, and will not affect the overall performance of
DAT.
D.3 Visualization of the Straight-Through Gradients and Backward Gradients in DAT
We visualize the estimated straight-through gradients in Eq 7 and the directly backward gradients in
Eq 6.
As shown in Figure 9, the approximated gradient by straight-through estimator does accurately
[Figure]
Original 𝛼 = 𝛼 =0.
𝛼 =0.
𝛼 =0. 𝛼 =0.
S t r a
i g h t - T
h r o u
g h
G r a d i e n t s
B a c k w a r d
G r a d i e n t s estimate the ground-truth gradients in Eq 6. It reﬂects the rationality of the proposed efﬁcient straight-through gradient in DAT.